Use of block size in dd and ddrescue
I have seen examples of using the dd and ddrescue command, and I see that instead of asking the user for the size of the block on their device, they recommend using bs=1M and I do not understand why.
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Can any value be used independently of the size of the disk sector?
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
dd ddrescue
add a comment |
I have seen examples of using the dd and ddrescue command, and I see that instead of asking the user for the size of the block on their device, they recommend using bs=1M and I do not understand why.
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Can any value be used independently of the size of the disk sector?
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
dd ddrescue
add a comment |
I have seen examples of using the dd and ddrescue command, and I see that instead of asking the user for the size of the block on their device, they recommend using bs=1M and I do not understand why.
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Can any value be used independently of the size of the disk sector?
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
dd ddrescue
I have seen examples of using the dd and ddrescue command, and I see that instead of asking the user for the size of the block on their device, they recommend using bs=1M and I do not understand why.
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Can any value be used independently of the size of the disk sector?
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
dd ddrescue
dd ddrescue
asked Jan 3 at 8:58
MarianoMMarianoM
6810
6810
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
What is bs option and why it's used ?
It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:
bs=BYTES read and write up to BYTES bytes at a time (default: 512);
Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:
I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.
The choice of bs value has also been discussed on another stackoverflow post:
The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size
As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.
Concerning your question:
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.
Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.
And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1106521%2fuse-of-block-size-in-dd-and-ddrescue%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
What is bs option and why it's used ?
It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:
bs=BYTES read and write up to BYTES bytes at a time (default: 512);
Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:
I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.
The choice of bs value has also been discussed on another stackoverflow post:
The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size
As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.
Concerning your question:
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.
Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.
And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.
add a comment |
What is bs option and why it's used ?
It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:
bs=BYTES read and write up to BYTES bytes at a time (default: 512);
Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:
I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.
The choice of bs value has also been discussed on another stackoverflow post:
The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size
As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.
Concerning your question:
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.
Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.
And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.
add a comment |
What is bs option and why it's used ?
It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:
bs=BYTES read and write up to BYTES bytes at a time (default: 512);
Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:
I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.
The choice of bs value has also been discussed on another stackoverflow post:
The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size
As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.
Concerning your question:
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.
Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.
And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.
What is bs option and why it's used ?
It has been explained in dd is producing a 32 MB random file instead of 1 GB ( which I recommend reading, btw ) that bs option refers to how much single read() syscall would read and store in memory ( if size of memory and the device you're reading from allow for such size ). This is also referenced in the manual:
bs=BYTES read and write up to BYTES bytes at a time (default: 512);
Thus, it has nothing to do with number of blocks/sectors on the disk, but with dd's performance - storing more things in memory for faster processing. To quote one ServerFault answer:
I know when I'm copying hard disks I get a faster rate by specifying bs=1M than by using bs=4k or the default. I'm talking speed improvements of 30 to 300 percent. There's no need to tune it for absolute best unless it's all you do every day. but picking something better than the default can cut hours off the execution time.
The choice of bs value has also been discussed on another stackoverflow post:
The optimal block size depends on various factors, including the operating system (and its version), and the various hardware buses and disks involved. Several Unix-like systems (including Linux and at least some flavors of BSD) define the st_blksize member in the struct stat that gives what the kernel thinks is the optimal block size
As for why 1M it's potentially because people found this to be an optimal value. Remember that RAM size used to be 512M. So using 1M would be small enough to keep RAM usage low, but have decent dd speed.
Concerning your question:
What happens if the disk sector size is 512 bytes and 1MB is assigned to bs?
Nothing. dd will read data in chunks of 1MB. It doesn't change the data in any way. It may improve dd speed as compared to smaller block size, but has no effect on the data itself.
Different story would happen if you read in chunks smaller than the sector size of the drive: drives have to serve full sector size and the kernel caches that information. See Matthew Ife's answer on the same Serverfault post. But again, no effect on data itself.
In the case of discs of advanced format and with a capacity of 1TB, what do you recommend to use?
Advanced format such as what ? As far as dd itself goes, it should not care about any disk format - all it knows is to read and write from and to block devices; it doesn't matter if you write to USB stick or SCSI drive - the syntax is the same, it's the kernel that handles the minute details and has drivers for those devices to properly pass the data.
And again, capacity also doesn't matter. 1TB SATA drive is still an SATA drive.
edited Jan 3 at 9:34
answered Jan 3 at 9:21
Sergiy KolodyazhnyySergiy Kolodyazhnyy
70.5k9146309
70.5k9146309
add a comment |
add a comment |
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1106521%2fuse-of-block-size-in-dd-and-ddrescue%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown