Are there any drawbacks to creating ext4 file systems with maximum inodes?












3














Are there any drawbacks to creating ext4 file systems with 2^32-1 inodes?



I have a 1tb drive and I would like to store 800 million to 1.5 billion small files on it. It seems that the max is 4 billion, so I'm wondering whether I could just set it to the max when creating an fs, or I should find some other solution.










share|improve this question




















  • 2




    Each ext4 inode takes up 250 bytes of disk space ...
    – Soren A
    2 days ago










  • @SorenA, even if it's unused?
    – Fluffy
    2 days ago






  • 1




    If size of single inode is smaller than the blocksize of the filesystem, then in that case more inodes would be made than can ever be used.
    – Gaurav Gandhi
    2 days ago










  • @PerlDuck, please see edit
    – Fluffy
    2 days ago
















3














Are there any drawbacks to creating ext4 file systems with 2^32-1 inodes?



I have a 1tb drive and I would like to store 800 million to 1.5 billion small files on it. It seems that the max is 4 billion, so I'm wondering whether I could just set it to the max when creating an fs, or I should find some other solution.










share|improve this question




















  • 2




    Each ext4 inode takes up 250 bytes of disk space ...
    – Soren A
    2 days ago










  • @SorenA, even if it's unused?
    – Fluffy
    2 days ago






  • 1




    If size of single inode is smaller than the blocksize of the filesystem, then in that case more inodes would be made than can ever be used.
    – Gaurav Gandhi
    2 days ago










  • @PerlDuck, please see edit
    – Fluffy
    2 days ago














3












3








3


1





Are there any drawbacks to creating ext4 file systems with 2^32-1 inodes?



I have a 1tb drive and I would like to store 800 million to 1.5 billion small files on it. It seems that the max is 4 billion, so I'm wondering whether I could just set it to the max when creating an fs, or I should find some other solution.










share|improve this question















Are there any drawbacks to creating ext4 file systems with 2^32-1 inodes?



I have a 1tb drive and I would like to store 800 million to 1.5 billion small files on it. It seems that the max is 4 billion, so I'm wondering whether I could just set it to the max when creating an fs, or I should find some other solution.







filesystem ext4 inode






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 2 days ago

























asked 2 days ago









Fluffy

80231133




80231133








  • 2




    Each ext4 inode takes up 250 bytes of disk space ...
    – Soren A
    2 days ago










  • @SorenA, even if it's unused?
    – Fluffy
    2 days ago






  • 1




    If size of single inode is smaller than the blocksize of the filesystem, then in that case more inodes would be made than can ever be used.
    – Gaurav Gandhi
    2 days ago










  • @PerlDuck, please see edit
    – Fluffy
    2 days ago














  • 2




    Each ext4 inode takes up 250 bytes of disk space ...
    – Soren A
    2 days ago










  • @SorenA, even if it's unused?
    – Fluffy
    2 days ago






  • 1




    If size of single inode is smaller than the blocksize of the filesystem, then in that case more inodes would be made than can ever be used.
    – Gaurav Gandhi
    2 days ago










  • @PerlDuck, please see edit
    – Fluffy
    2 days ago








2




2




Each ext4 inode takes up 250 bytes of disk space ...
– Soren A
2 days ago




Each ext4 inode takes up 250 bytes of disk space ...
– Soren A
2 days ago












@SorenA, even if it's unused?
– Fluffy
2 days ago




@SorenA, even if it's unused?
– Fluffy
2 days ago




1




1




If size of single inode is smaller than the blocksize of the filesystem, then in that case more inodes would be made than can ever be used.
– Gaurav Gandhi
2 days ago




If size of single inode is smaller than the blocksize of the filesystem, then in that case more inodes would be made than can ever be used.
– Gaurav Gandhi
2 days ago












@PerlDuck, please see edit
– Fluffy
2 days ago




@PerlDuck, please see edit
– Fluffy
2 days ago










1 Answer
1






active

oldest

votes


















1














According to similar questions over on Stack Overflow
and Unix & Linux (see below) it is a bad idea
to max out the number of inodes on an ext4 filesystem.



Better use another filesystem or split your disk into multiple filesystems.



To summarize:





  • An inode occupies 256 bytes. It may be configured to 128, but even though:



    2³² inodes × 256 bytes each = 1 TB



  • When creating an ext4 filesystem, you can specify the usage type
    as defined in /etc/mke2fs.conf:



    mkfs.ext4 -T usage-type /dev/something


    For storing many small files the type small might be used:



    small = {
    blocksize = 1024
    inode_size = 128
    inode_ratio = 4096
    }


    This means: for every 4096 bytes of diskspace (filesystem size) one inode will be reserved
    each of which has a size of 128 bytes. The command mkfs.ext4 -T small /dev/something
    would thus create 244 million inodes on a 1TB filesystem that occupy 31 GB. These
    244 million files will at least occupy 250 GB (min. 1024 bytes each).



  • To hold 1 billion inodes with the small footprint (128 bytes) you would need 128 GB
    just for the inodes. If the smallest possible blocksize (1024 according to mk2efs's
    manpage) is used, then these 1 billion files would at least occupy 1TB (but remember
    you only have 872 GB left because of the 128 GB for the inodes).


  • The smallest possible blocksize for ext4 is 1024 bytes. Hence, you cannot store more than
    1 TB / 1024 = 1 billion files and it's pointless to have more inodes.


  • As a general rule of thumb the inode_ratio should not be smaller than the blocksize
    because you cannot (easily) store more than one file in one block.


  • It is possible to store the first 60 bytes of a file directly whithin the inode if the
    filesystem is configured that way. In that case, the file would not occupy a (regular)
    block; read about such inline data here but also consider the inode size.



  • From a comment to the U&L question:




    Too many inodes has a deep cost for sure, they take up space themselves, and believe me that fsck performance is dead serious. We're talking practically exponential slowdowns, like 30 seconds vs 2 days... and this is I/O bound. Not to mention slow downs of file listing, indexing... etc.







References:




  • How to store one billion files on ext4?

  • Drawbacks of increasing number of inodes in EXT4

  • Ext4 Disk Layout

  • Smallest file block size (ext 3, 4)






share|improve this answer





















    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "89"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1104904%2fare-there-any-drawbacks-to-creating-ext4-file-systems-with-maximum-inodes%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    1














    According to similar questions over on Stack Overflow
    and Unix & Linux (see below) it is a bad idea
    to max out the number of inodes on an ext4 filesystem.



    Better use another filesystem or split your disk into multiple filesystems.



    To summarize:





    • An inode occupies 256 bytes. It may be configured to 128, but even though:



      2³² inodes × 256 bytes each = 1 TB



    • When creating an ext4 filesystem, you can specify the usage type
      as defined in /etc/mke2fs.conf:



      mkfs.ext4 -T usage-type /dev/something


      For storing many small files the type small might be used:



      small = {
      blocksize = 1024
      inode_size = 128
      inode_ratio = 4096
      }


      This means: for every 4096 bytes of diskspace (filesystem size) one inode will be reserved
      each of which has a size of 128 bytes. The command mkfs.ext4 -T small /dev/something
      would thus create 244 million inodes on a 1TB filesystem that occupy 31 GB. These
      244 million files will at least occupy 250 GB (min. 1024 bytes each).



    • To hold 1 billion inodes with the small footprint (128 bytes) you would need 128 GB
      just for the inodes. If the smallest possible blocksize (1024 according to mk2efs's
      manpage) is used, then these 1 billion files would at least occupy 1TB (but remember
      you only have 872 GB left because of the 128 GB for the inodes).


    • The smallest possible blocksize for ext4 is 1024 bytes. Hence, you cannot store more than
      1 TB / 1024 = 1 billion files and it's pointless to have more inodes.


    • As a general rule of thumb the inode_ratio should not be smaller than the blocksize
      because you cannot (easily) store more than one file in one block.


    • It is possible to store the first 60 bytes of a file directly whithin the inode if the
      filesystem is configured that way. In that case, the file would not occupy a (regular)
      block; read about such inline data here but also consider the inode size.



    • From a comment to the U&L question:




      Too many inodes has a deep cost for sure, they take up space themselves, and believe me that fsck performance is dead serious. We're talking practically exponential slowdowns, like 30 seconds vs 2 days... and this is I/O bound. Not to mention slow downs of file listing, indexing... etc.







    References:




    • How to store one billion files on ext4?

    • Drawbacks of increasing number of inodes in EXT4

    • Ext4 Disk Layout

    • Smallest file block size (ext 3, 4)






    share|improve this answer


























      1














      According to similar questions over on Stack Overflow
      and Unix & Linux (see below) it is a bad idea
      to max out the number of inodes on an ext4 filesystem.



      Better use another filesystem or split your disk into multiple filesystems.



      To summarize:





      • An inode occupies 256 bytes. It may be configured to 128, but even though:



        2³² inodes × 256 bytes each = 1 TB



      • When creating an ext4 filesystem, you can specify the usage type
        as defined in /etc/mke2fs.conf:



        mkfs.ext4 -T usage-type /dev/something


        For storing many small files the type small might be used:



        small = {
        blocksize = 1024
        inode_size = 128
        inode_ratio = 4096
        }


        This means: for every 4096 bytes of diskspace (filesystem size) one inode will be reserved
        each of which has a size of 128 bytes. The command mkfs.ext4 -T small /dev/something
        would thus create 244 million inodes on a 1TB filesystem that occupy 31 GB. These
        244 million files will at least occupy 250 GB (min. 1024 bytes each).



      • To hold 1 billion inodes with the small footprint (128 bytes) you would need 128 GB
        just for the inodes. If the smallest possible blocksize (1024 according to mk2efs's
        manpage) is used, then these 1 billion files would at least occupy 1TB (but remember
        you only have 872 GB left because of the 128 GB for the inodes).


      • The smallest possible blocksize for ext4 is 1024 bytes. Hence, you cannot store more than
        1 TB / 1024 = 1 billion files and it's pointless to have more inodes.


      • As a general rule of thumb the inode_ratio should not be smaller than the blocksize
        because you cannot (easily) store more than one file in one block.


      • It is possible to store the first 60 bytes of a file directly whithin the inode if the
        filesystem is configured that way. In that case, the file would not occupy a (regular)
        block; read about such inline data here but also consider the inode size.



      • From a comment to the U&L question:




        Too many inodes has a deep cost for sure, they take up space themselves, and believe me that fsck performance is dead serious. We're talking practically exponential slowdowns, like 30 seconds vs 2 days... and this is I/O bound. Not to mention slow downs of file listing, indexing... etc.







      References:




      • How to store one billion files on ext4?

      • Drawbacks of increasing number of inodes in EXT4

      • Ext4 Disk Layout

      • Smallest file block size (ext 3, 4)






      share|improve this answer
























        1












        1








        1






        According to similar questions over on Stack Overflow
        and Unix & Linux (see below) it is a bad idea
        to max out the number of inodes on an ext4 filesystem.



        Better use another filesystem or split your disk into multiple filesystems.



        To summarize:





        • An inode occupies 256 bytes. It may be configured to 128, but even though:



          2³² inodes × 256 bytes each = 1 TB



        • When creating an ext4 filesystem, you can specify the usage type
          as defined in /etc/mke2fs.conf:



          mkfs.ext4 -T usage-type /dev/something


          For storing many small files the type small might be used:



          small = {
          blocksize = 1024
          inode_size = 128
          inode_ratio = 4096
          }


          This means: for every 4096 bytes of diskspace (filesystem size) one inode will be reserved
          each of which has a size of 128 bytes. The command mkfs.ext4 -T small /dev/something
          would thus create 244 million inodes on a 1TB filesystem that occupy 31 GB. These
          244 million files will at least occupy 250 GB (min. 1024 bytes each).



        • To hold 1 billion inodes with the small footprint (128 bytes) you would need 128 GB
          just for the inodes. If the smallest possible blocksize (1024 according to mk2efs's
          manpage) is used, then these 1 billion files would at least occupy 1TB (but remember
          you only have 872 GB left because of the 128 GB for the inodes).


        • The smallest possible blocksize for ext4 is 1024 bytes. Hence, you cannot store more than
          1 TB / 1024 = 1 billion files and it's pointless to have more inodes.


        • As a general rule of thumb the inode_ratio should not be smaller than the blocksize
          because you cannot (easily) store more than one file in one block.


        • It is possible to store the first 60 bytes of a file directly whithin the inode if the
          filesystem is configured that way. In that case, the file would not occupy a (regular)
          block; read about such inline data here but also consider the inode size.



        • From a comment to the U&L question:




          Too many inodes has a deep cost for sure, they take up space themselves, and believe me that fsck performance is dead serious. We're talking practically exponential slowdowns, like 30 seconds vs 2 days... and this is I/O bound. Not to mention slow downs of file listing, indexing... etc.







        References:




        • How to store one billion files on ext4?

        • Drawbacks of increasing number of inodes in EXT4

        • Ext4 Disk Layout

        • Smallest file block size (ext 3, 4)






        share|improve this answer












        According to similar questions over on Stack Overflow
        and Unix & Linux (see below) it is a bad idea
        to max out the number of inodes on an ext4 filesystem.



        Better use another filesystem or split your disk into multiple filesystems.



        To summarize:





        • An inode occupies 256 bytes. It may be configured to 128, but even though:



          2³² inodes × 256 bytes each = 1 TB



        • When creating an ext4 filesystem, you can specify the usage type
          as defined in /etc/mke2fs.conf:



          mkfs.ext4 -T usage-type /dev/something


          For storing many small files the type small might be used:



          small = {
          blocksize = 1024
          inode_size = 128
          inode_ratio = 4096
          }


          This means: for every 4096 bytes of diskspace (filesystem size) one inode will be reserved
          each of which has a size of 128 bytes. The command mkfs.ext4 -T small /dev/something
          would thus create 244 million inodes on a 1TB filesystem that occupy 31 GB. These
          244 million files will at least occupy 250 GB (min. 1024 bytes each).



        • To hold 1 billion inodes with the small footprint (128 bytes) you would need 128 GB
          just for the inodes. If the smallest possible blocksize (1024 according to mk2efs's
          manpage) is used, then these 1 billion files would at least occupy 1TB (but remember
          you only have 872 GB left because of the 128 GB for the inodes).


        • The smallest possible blocksize for ext4 is 1024 bytes. Hence, you cannot store more than
          1 TB / 1024 = 1 billion files and it's pointless to have more inodes.


        • As a general rule of thumb the inode_ratio should not be smaller than the blocksize
          because you cannot (easily) store more than one file in one block.


        • It is possible to store the first 60 bytes of a file directly whithin the inode if the
          filesystem is configured that way. In that case, the file would not occupy a (regular)
          block; read about such inline data here but also consider the inode size.



        • From a comment to the U&L question:




          Too many inodes has a deep cost for sure, they take up space themselves, and believe me that fsck performance is dead serious. We're talking practically exponential slowdowns, like 30 seconds vs 2 days... and this is I/O bound. Not to mention slow downs of file listing, indexing... etc.







        References:




        • How to store one billion files on ext4?

        • Drawbacks of increasing number of inodes in EXT4

        • Ext4 Disk Layout

        • Smallest file block size (ext 3, 4)







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered 2 days ago









        PerlDuck

        5,34411231




        5,34411231






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Ask Ubuntu!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1104904%2fare-there-any-drawbacks-to-creating-ext4-file-systems-with-maximum-inodes%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Human spaceflight

            Can not write log (Is /dev/pts mounted?) - openpty in Ubuntu-on-Windows?

            張江高科駅