Use of block size in dd and ddrescue












0















I have seen examples of using the dd and ddrescue command, and I see that instead of asking the user for the size of the block on their device, they recommend using bs=1M and I do not understand why.



What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?



Can any value be used independently of the size of the disk sector?



In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?










share|improve this question



























    0















    I have seen examples of using the dd and ddrescue command, and I see that instead of asking the user for the size of the block on their device, they recommend using bs=1M and I do not understand why.



    What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?



    Can any value be used independently of the size of the disk sector?



    In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?










    share|improve this question

























      0












      0








      0


      2






      I have seen examples of using the dd and ddrescue command, and I see that instead of asking the user for the size of the block on their device, they recommend using bs=1M and I do not understand why.



      What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?



      Can any value be used independently of the size of the disk sector?



      In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?










      share|improve this question














      I have seen examples of using the dd and ddrescue command, and I see that instead of asking the user for the size of the block on their device, they recommend using bs=1M and I do not understand why.



      What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?



      Can any value be used independently of the size of the disk sector?



      In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?







      dd ddrescue






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Jan 3 at 8:58









      MarianoMMarianoM

      6810




      6810






















          1 Answer
          1






          active

          oldest

          votes


















          1














          What is bs option and why it's used ?



          It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:




          bs=BYTES read and write up to BYTES bytes at a time (default: 512);




          Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:




          I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.




          The choice of bs value has also been discussed on another stackoverflow post:




          The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size




          As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.





          Concerning your question:




          What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?




          Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.



          Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.




          In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?




          Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.



          And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.






          share|improve this answer

























            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "89"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1106521%2fuse-of-block-size-in-dd-and-ddrescue%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1














            What is bs option and why it's used ?



            It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:




            bs=BYTES read and write up to BYTES bytes at a time (default: 512);




            Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:




            I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.




            The choice of bs value has also been discussed on another stackoverflow post:




            The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size




            As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.





            Concerning your question:




            What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?




            Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.



            Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.




            In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?




            Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.



            And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.






            share|improve this answer






























              1














              What is bs option and why it's used ?



              It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:




              bs=BYTES read and write up to BYTES bytes at a time (default: 512);




              Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:




              I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.




              The choice of bs value has also been discussed on another stackoverflow post:




              The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size




              As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.





              Concerning your question:




              What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?




              Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.



              Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.




              In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?




              Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.



              And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.






              share|improve this answer




























                1












                1








                1







                What is bs option and why it's used ?



                It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:




                bs=BYTES read and write up to BYTES bytes at a time (default: 512);




                Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:




                I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.




                The choice of bs value has also been discussed on another stackoverflow post:




                The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size




                As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.





                Concerning your question:




                What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?




                Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.



                Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.




                In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?




                Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.



                And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.






                share|improve this answer















                What is bs option and why it's used ?



                It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:




                bs=BYTES read and write up to BYTES bytes at a time (default: 512);




                Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:




                I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.




                The choice of bs value has also been discussed on another stackoverflow post:




                The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size




                As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.





                Concerning your question:




                What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?




                Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.



                Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.




                In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?




                Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.



                And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.







                share|improve this answer














                share|improve this answer



                share|improve this answer








                edited Jan 3 at 9:34

























                answered Jan 3 at 9:21









                Sergiy KolodyazhnyySergiy Kolodyazhnyy

                70.5k9146309




                70.5k9146309






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Ask Ubuntu!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1106521%2fuse-of-block-size-in-dd-and-ddrescue%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Questions related to Moebius Transform of Characteristic Function of the Primes

                    List of scandals in India

                    Can not write log (Is /dev/pts mounted?) - openpty in Ubuntu-on-Windows?