Too many open files

[guo-yong-zhi]

Function save seems failed to close files automatically.
The code

for i in 1:20000
    pic = randpic()
    f = open("datasets/$i.png", "w")
    save(Stream{format"PNG"}(f), pic)
    close(f)
end

runs fine. However the code

for i in 1:20000
    pic = randpic()
    save("datasets2/$i.png", pic)
end

leads to

SystemError: opening file "datasets2/8075.png": Too many open files

[timholy]

Not for me. Can you provide a fully-reproducible example? Something that runs if you just copy/paste it in the REPL. I can handle the directory-creation, don't worry about that part.

[guo-yong-zhi]

Sorry for the confusion. I thought the bug was common. But it may only be triggered by a specific type, I guess.
Here is a a fully-reproducible example:

using FileIO
using Luxor

function randpic(size=32)
    d = Drawing(size, size, :image)
    origin()
    background(1,1,1)
    setcolor(1,0,0)
    circle(Point(0, 0), size÷2; action=:fill)
    mat = image_as_matrix()
    finish()
    mat
end
pic = randpic();
@show typeof(pic)

mkdir("datasets2")
for i in 1:20000
    save("datasets2/$i.png", pic)
end

typeof(pic) = Base.ReinterpretArray{ARGB32, 2, UInt32, Matrix{UInt32}, false}
All errors:
===========================================
MethodError: no method matching _get_color_type(::Matrix{ARGB32})
Closest candidates are:
_get_color_type(::IndirectArrays.IndirectArray) at ~/.julia/packages/PNGFiles/oYmtV/src/io.jl:526
_get_color_type(::AbstractArray{<:Gray{T}}) where T at ~/.julia/packages/PNGFiles/oYmtV/src/io.jl:518
_get_color_type(::AbstractArray{<:GrayA{T}}) where T at ~/.julia/packages/PNGFiles/oYmtV/src/io.jl:519
...
===========================================
WriteBlob Failed `datasets2/4040.png' @ error/png.c/MagickPNGErrorHandler/1642
===========================================
SystemError: opening file "datasets2/4040.png": Too many open files
===========================================
Errors encountered while save File{DataFormat{:PNG}, String}("datasets2/4040.png").

---

My environment:

Julia Version 1.7.0-rc2
Commit f23fc0d27a (2021-10-20 12:45 UTC)
Platform Info:
OS: Linux (x86_64-pc-linux-gnu)
CPU: Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-12.0.1 (ORCJIT, skylake)

Status ~/.julia/environments/v1.7/Project.toml
[5789e2e9] FileIO v1.11.2
[916415d5] Images v0.24.1
[ae8d54c2] Luxor v2.16.0

[guo-yong-zhi]

Is there anything can be done in FileIO.jl? After all the manual open-save-close form works fine. I don't know FileIO's mechanism, but as a user I hope the framework will force close everything it opened. Thank you.

[timholy]

https://github.com/JuliaIO/ImageMagick.jl/blob/079d62b859d5af2ef1c05b4b3e0948cd0ec1d9bd/src/libmagickwand.jl#L292-L296

So FileIO doesn't handle the opening/closing.