Cannot increase open file limit beyond 999999 on 14.04+












0















I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf. But, cannot get past 999999, anything above this value reverts to default 1024 nor I can set it to unlimited



Here is my limits.conf



* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999


The above works fine and ulimit -n return 999999.



But, if I make the value one step higher or to unlimited or -1 or infinity it doesn't work. ulimit -n return 1024



I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage



Thanks










share|improve this question




















  • 1





    What is your real problem?

    – Jos
    Mar 22 '18 at 19:58











  • Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

    – Shan
    Mar 23 '18 at 13:10













  • Do you really need more than a million files open?

    – Jos
    Mar 23 '18 at 13:13











  • Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

    – Shan
    Mar 23 '18 at 13:53
















0















I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf. But, cannot get past 999999, anything above this value reverts to default 1024 nor I can set it to unlimited



Here is my limits.conf



* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999


The above works fine and ulimit -n return 999999.



But, if I make the value one step higher or to unlimited or -1 or infinity it doesn't work. ulimit -n return 1024



I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage



Thanks










share|improve this question




















  • 1





    What is your real problem?

    – Jos
    Mar 22 '18 at 19:58











  • Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

    – Shan
    Mar 23 '18 at 13:10













  • Do you really need more than a million files open?

    – Jos
    Mar 23 '18 at 13:13











  • Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

    – Shan
    Mar 23 '18 at 13:53














0












0








0








I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf. But, cannot get past 999999, anything above this value reverts to default 1024 nor I can set it to unlimited



Here is my limits.conf



* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999


The above works fine and ulimit -n return 999999.



But, if I make the value one step higher or to unlimited or -1 or infinity it doesn't work. ulimit -n return 1024



I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage



Thanks










share|improve this question
















I'm trying to increase open file limit for root user as well as non-root user via /etc/security/limits.conf. But, cannot get past 999999, anything above this value reverts to default 1024 nor I can set it to unlimited



Here is my limits.conf



* hard nofile 999999
* hard nofile 999999
root hard nofile 999999
root soft nofile 999999


The above works fine and ulimit -n return 999999.



But, if I make the value one step higher or to unlimited or -1 or infinity it doesn't work. ulimit -n return 1024



I refered here: limits.conf - configuration file for the pam_limits module | Ubuntu Manpage



Thanks







ulimit resource-limiting






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 23 '18 at 12:25









Drakonoved

7362515




7362515










asked Mar 22 '18 at 19:39









ShanShan

353136




353136








  • 1





    What is your real problem?

    – Jos
    Mar 22 '18 at 19:58











  • Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

    – Shan
    Mar 23 '18 at 13:10













  • Do you really need more than a million files open?

    – Jos
    Mar 23 '18 at 13:13











  • Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

    – Shan
    Mar 23 '18 at 13:53














  • 1





    What is your real problem?

    – Jos
    Mar 22 '18 at 19:58











  • Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

    – Shan
    Mar 23 '18 at 13:10













  • Do you really need more than a million files open?

    – Jos
    Mar 23 '18 at 13:13











  • Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

    – Shan
    Mar 23 '18 at 13:53








1




1





What is your real problem?

– Jos
Mar 22 '18 at 19:58





What is your real problem?

– Jos
Mar 22 '18 at 19:58













Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

– Shan
Mar 23 '18 at 13:10







Hi Jos, let me make it simple for you. Thought the pam_limits documentation says All items support the values -1, unlimited or infinity indicating no limit, except for priority and nice. I can't set unlimited to nofile, if I do so ulimit -n returns 1024, not sure if this is an expected behaviour

– Shan
Mar 23 '18 at 13:10















Do you really need more than a million files open?

– Jos
Mar 23 '18 at 13:13





Do you really need more than a million files open?

– Jos
Mar 23 '18 at 13:13













Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

– Shan
Mar 23 '18 at 13:53





Its a development machine with Kafka and Zookeeper, openfiles tend to grow with Topics and number of partitions.

– Shan
Mar 23 '18 at 13:53










2 Answers
2






active

oldest

votes


















1














I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



* hard nofile 999999
* soft nofile 999999
root hard nofile 999999
root soft nofile 999999


After doing this you should see:



$ ulimit -n
999999
$


Your file should have always worked for root as the limits were correctly defined.



Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



$ cat /proc/sys/fs/nr_open
1048576
$


You also have to start a new session for the limits to update.






share|improve this answer































    0














    There are three files to modify to get this to work:




    • /etc/security/limits.conf

    • /etc/pam.d/common-session

    • /etc/pam.d/common-session-noninteractive


    You have already added the required lines in the first file, then in the other two files this line is needed:



    session required pam_limits.so


    Please note:




    The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




    Hence if such a process exists then you will have to modify the config file to get it to work.



    Source:
    https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors






    share|improve this answer
























    • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

      – Shan
      Mar 23 '18 at 13:07











    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "89"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1018351%2fcannot-increase-open-file-limit-beyond-999999-on-14-04%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    1














    I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



    * hard nofile 999999
    * soft nofile 999999
    root hard nofile 999999
    root soft nofile 999999


    After doing this you should see:



    $ ulimit -n
    999999
    $


    Your file should have always worked for root as the limits were correctly defined.



    Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



    $ cat /proc/sys/fs/nr_open
    1048576
    $


    You also have to start a new session for the limits to update.






    share|improve this answer




























      1














      I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



      * hard nofile 999999
      * soft nofile 999999
      root hard nofile 999999
      root soft nofile 999999


      After doing this you should see:



      $ ulimit -n
      999999
      $


      Your file should have always worked for root as the limits were correctly defined.



      Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



      $ cat /proc/sys/fs/nr_open
      1048576
      $


      You also have to start a new session for the limits to update.






      share|improve this answer


























        1












        1








        1







        I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



        * hard nofile 999999
        * soft nofile 999999
        root hard nofile 999999
        root soft nofile 999999


        After doing this you should see:



        $ ulimit -n
        999999
        $


        Your file should have always worked for root as the limits were correctly defined.



        Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



        $ cat /proc/sys/fs/nr_open
        1048576
        $


        You also have to start a new session for the limits to update.






        share|improve this answer













        I know it's 10 months late, but you've set the hard limit twice for all users. Change the second line to be a soft ulimit.



        * hard nofile 999999
        * soft nofile 999999
        root hard nofile 999999
        root soft nofile 999999


        After doing this you should see:



        $ ulimit -n
        999999
        $


        Your file should have always worked for root as the limits were correctly defined.



        Furthermore you can't use -1, infinity, unlimited or the like, you must use a literal value. The max supported value for this is defined in the kernel and is exposed in /proc/sys/fs/nr_open. In my centos 7 & debian strech environments I get the same value:



        $ cat /proc/sys/fs/nr_open
        1048576
        $


        You also have to start a new session for the limits to update.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Jan 27 at 2:49









        HamishHamish

        112




        112

























            0














            There are three files to modify to get this to work:




            • /etc/security/limits.conf

            • /etc/pam.d/common-session

            • /etc/pam.d/common-session-noninteractive


            You have already added the required lines in the first file, then in the other two files this line is needed:



            session required pam_limits.so


            Please note:




            The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




            Hence if such a process exists then you will have to modify the config file to get it to work.



            Source:
            https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors






            share|improve this answer
























            • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

              – Shan
              Mar 23 '18 at 13:07
















            0














            There are three files to modify to get this to work:




            • /etc/security/limits.conf

            • /etc/pam.d/common-session

            • /etc/pam.d/common-session-noninteractive


            You have already added the required lines in the first file, then in the other two files this line is needed:



            session required pam_limits.so


            Please note:




            The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




            Hence if such a process exists then you will have to modify the config file to get it to work.



            Source:
            https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors






            share|improve this answer
























            • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

              – Shan
              Mar 23 '18 at 13:07














            0












            0








            0







            There are three files to modify to get this to work:




            • /etc/security/limits.conf

            • /etc/pam.d/common-session

            • /etc/pam.d/common-session-noninteractive


            You have already added the required lines in the first file, then in the other two files this line is needed:



            session required pam_limits.so


            Please note:




            The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




            Hence if such a process exists then you will have to modify the config file to get it to work.



            Source:
            https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors






            share|improve this answer













            There are three files to modify to get this to work:




            • /etc/security/limits.conf

            • /etc/pam.d/common-session

            • /etc/pam.d/common-session-noninteractive


            You have already added the required lines in the first file, then in the other two files this line is needed:



            session required pam_limits.so


            Please note:




            The thing that most resources neglect to emphasize, is that your limits can really easily be modified by anything responsible for execution of your processes. If ulimit -n (run as the correct user) is giving you the number you just set, but cat /proc/{process_id}/limits is still printing the low number, you almost certainly have a process manager, an init script, or something similar messing your limits up. You also need to keep in mind that processes inherit the limits of the parent process.




            Hence if such a process exists then you will have to modify the config file to get it to work.



            Source:
            https://underyx.me/2015/05/18/raising-the-maximum-number-of-file-descriptors







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Mar 22 '18 at 21:35









            George UdosenGeorge Udosen

            21.2k94570




            21.2k94570













            • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

              – Shan
              Mar 23 '18 at 13:07



















            • Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

              – Shan
              Mar 23 '18 at 13:07

















            Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

            – Shan
            Mar 23 '18 at 13:07





            Thanks George. I understand all the above. The issue here is setting it to unlimted is not working. I mean setting that in upstart, systemd scripts of the process as well.

            – Shan
            Mar 23 '18 at 13:07


















            draft saved

            draft discarded




















































            Thanks for contributing an answer to Ask Ubuntu!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1018351%2fcannot-increase-open-file-limit-beyond-999999-on-14-04%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Questions related to Moebius Transform of Characteristic Function of the Primes

            List of scandals in India

            Can not write log (Is /dev/pts mounted?) - openpty in Ubuntu-on-Windows?