Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It's even easier once you realize that dd is not required to read/write disks:

pv /dev/sdb | gzip -c | ssh name@host "gzip -dc > /dev/sdc"

Not only is this shorter with less cargo cult steps, it also gives a much better `pv` output now that it can determine the size and show a percentage.



Not that parent poster's example is better than yours, since it also doesn't do it, but I believe the biggest strength of dd over cat or redirection, is that you can specify block size. With right block size you will get far better performance and reduce wear of the device when writing block by click than when writing data byte by byte.

The key though is to be aware of block size, and use multiply of it.


It's not quite as simple as that when you're dealing with pipelines. For example:

    cat /dev/zero | strace dd bs=1M of=/dev/null
will show that you're not actually writing 1M blocks. You're just reading whatever `cat` outputs (128KiB blocks on my system), so all you've done is an unnecessary pipe and copy. You would have been better off just redirecting.

Similarly,

    dd if=/dev/zero bs=1M | strace gzip -1 > /dev/null
will show you that `gzip` will not start processing larger blocks. You're just adding an additional copy. Linux already has readahead, so I don't know if you'll ever see a benefit to this.

On my system, adding dd with higher block size on both sides of a gzip -1 pipeline just ends up being slower.


That's because bs means "up to" so if input comes at smaller blocks it will write in those. You should use obs, and preferably specify ibs to being multiple of obs.


I omitted block size to simplify it.

That said, what you say is correct on writes. On reads, specifying block size is mostly to reduce syscall overhead because readahead will prefetch data to accelerate things.


You don't need any of those gzip options.

pv /dev/sdb | gzip | ssh name@host "gunzip > /dev/sdc"


you dont need gzip at all, ssh offers compression too:

pv /dev/sdb | ssh -C name@host "cat > /dev/sdc"


Then I would not have been able to demonstrate how to use the gzip command with dd.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: