You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Assuming the situation downloading part of large file e.g. curl * | dd skip=* bs=1M.
$ hyperfine "time head -c 1G /dev/zero |dd skip=1GiB bs=1M" "time head -c 1G /dev/zero |target/release/dd skip=1GiB bs=1M"
Benchmark 1: time head -c 1G /dev/zero |dd skip=1GiB bs=1M
Time (mean ± σ): 407.0 ms ± 14.0 ms [User: 10.6 ms, System: 788.1 ms]
Range (min … max): 394.2 ms … 433.5 ms 10 runs
Benchmark 2: time head -c 1G /dev/zero |target/release/dd skip=1GiB bs=1M
Time (mean ± σ): 235.7 ms ± 8.2 ms [User: 3.2 ms, System: 451.5 ms]
Range (min … max): 224.2 ms … 248.5 ms 13 runs
Summary
time head -c 1G /dev/zero |target/release/dd skip=1GiB bs=1M ran
1.73 ± 0.08 times faster than time head -c 1G /dev/zero |dd skip=1GiB bs=1M
Skip an intermittent issue tests/misc/io-errors (fails in this run but passes in the 'main' branch)
Congrats! The gnu test tests/tail/tail-n0f is now passing!
skip=n
Skip n input blocks (using the specified input block size) before starting to copy. On seekable files, the implementation shall read the blocks or seek past them; on non-seekable files, the blocks shall be read and the data shall be discarded.
I don't think posix requires block should be existing on user space and read/write means read/write syscalls.
Also I think we can use sealed memfd for dd to exactly control block size. Rust std::io::copy's dev said splice to memfd is not zero-copy (thus, changing input during sinding data is safe), but still removes context switch.
Moreover, memfd is able to mmap and edit them for dd conv=. @collinfunk ANy thought.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Assuming the situation downloading part of large file e.g.
curl * | dd skip=* bs=1M.