Increase default buffer size

[Keno]

Makes creating a tarball and compressing it with zstdmt about 6x
faster (30s vs 5s). Raw tar is still about 20% faster, but we'd
probably need #33 to make
up the difference.

[codecov]

Codecov Report

Merging #34 into master will increase coverage by 0.11%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #34      +/-   ##
==========================================
+ Coverage   92.65%   92.76%   +0.11%     
==========================================
  Files           4        4              
  Lines         381      387       +6     
==========================================
+ Hits          353      359       +6     
  Misses         28       28              

Impacted Files	Coverage Δ
src/create.jl	94.24% <100.00%> (+0.25%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b8bd833...4b99196. Read the comment docs.

[StefanKarpinski]

Looks good to me. Perhaps the same treatment for extract?

[Keno]

Done, but performance is pretty impressively bad for extract. Almost 50s, vs

time tar -xf ~/2020-05-02T00-45-17-Keno.tar.zst

real	0m0.871s
user	0m0.574s
sys	0m0.663s

[Keno]

Also, this seems bad:

keno@arctic4:~$ ~/julia-comparison/julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.0-DEV.751 (2020-05-01)
 _/ |\__'_|_|_|\__'_|  |  kf/disablermprocstests/8d367c15f1* (fork: 1 commits, 1 day)
|__/                   |

julia> local_path = "2020-05-02T00-45-17-Keno.tar.zst"
"2020-05-02T00-45-17-Keno.tar.zst"

julia> dir = "/tmp/test"
"/tmp/test"

julia> using Tar, Zstd_jll
[ Info: Precompiling Tar [a4e569a6-e804-4fa4-b0f3-eef7a1d5b13e]
[ Info: Precompiling Zstd_jll [3161d3a3-bdf6-5164-811a-617609db77b4]

julia> proc = zstdmt() do zstdp
           open(`$zstdp --stdout -d $local_path`, "r+")
       end
Process(`/home/keno/.julia/artifacts/c94fb31d9b839d8eb032df37bcf19691f092f4be/bin/zstdmt --stdout -d 2020-05-02T00-45-17-Keno.tar.zst`, ProcessRunning)

julia> Tar.extract(proc, dir)
"/tmp/test"

julia>
keno@arctic4:~$ ~/julia-comparison/julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.0-DEV.751 (2020-05-01)
 _/ |\__'_|_|_|\__'_|  |  kf/disablermprocstests/8d367c15f1* (fork: 1 commits, 1 day)
|__/                   |

julia> local_path = "2020-05-02T00-45-17-Keno.tar.zst"
"2020-05-02T00-45-17-Keno.tar.zst"

julia> dir = "/tmp/test"
"/tmp/test"

julia> using Tar, Zstd_jll

julia> proc = zstdmt() do zstdp
           open(`$zstdp --stdout -d $local_path`, "r+")
       end
Process(`/home/keno/.julia/artifacts/c94fb31d9b839d8eb032df37bcf19691f092f4be/bin/zstdmt --stdout -d 2020-05-02T00-45-17-Keno.tar.zst`, ProcessRunning)

julia> Tar.extract(proc.out, dir)
ERROR: invalid octal digit: 'Ø'
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] read_header_int(::SubArray{UInt8,1,Array{UInt8,1},Tuple{UnitRange{Int64}},true}, ::Int64, ::Int64) at /home/keno/.julia/dev/Tar/src/extract.jl:261
 [3] read_standard_header(::Base.PipeEndpoint; buf::Array{UInt8,1}) at /home/keno/.julia/dev/Tar/src/extract.jl:215
 [4] read_header(::Base.PipeEndpoint; buf::Array{UInt8,1}) at /home/keno/.julia/dev/Tar/src/extract.jl:121
 [5] extract_tarball(::Tar.var"#27#28", ::Base.PipeEndpoint, ::String; buf::Array{UInt8,1}) at /home/keno/.julia/dev/Tar/src/extract.jl:40
 [6] extract_tarball(::Function, ::Base.PipeEndpoint, ::String) at /home/keno/.julia/dev/Tar/src/extract.jl:38
 [7] extract(::Function, ::Base.PipeEndpoint, ::String) at /home/keno/.julia/dev/Tar/src/Tar.jl:130
 [8] extract(::Base.PipeEndpoint, ::String) at /home/keno/.julia/dev/Tar/src/Tar.jl:145
 [9] top-level scope at REPL[5]:1

julia>

[Keno]

JuliaLang/julia#35695

[Keno]

With JuliaLang/julia#35696 and this, the performance is acceptable.

[StefanKarpinski]

Nice! What is the extract performance for your example data after this?

The macOS nightly failures are pretty concerning but don't seem related to this PR—did yesterday's PCRE upgrade perhaps break something?

[StefanKarpinski]

Glad you could make this change easily enough. Hopefully it means that this is a pretty clear, understandable code base. Making modifications to GNU tar seems significantly more challenging 😬

[Keno]

Nice! What is the extract performance for your example data after this?

Around one second for the extract benchmark from above, down from 50. I don't think my benchmark methodology is precise enough to say much more, but that makes it on par with GNU tar.

[Keno]

I'll do a version bump with this PR included.