Invalid JSON from container when trying to open input file

[gregorypease280]

MacOS 13.3.1 / Intel i7

Looks like there are still some issues mounting the input file(s)
I get invalid JSON in both the gui and cli

IsADirectoryError: [Errno 21] Is a directory: '/tmp/input_file'

I've given docker and Dangerzone full filesystem access
which docker -> /usr/local/bin/docker
when running the gui I see containers being made and destroyed rather quickly in Docker Desktop as well.

Related Issues:
#335
#371

Converting document to safe PDF
> /usr/local/bin/docker run --network none -u dangerzone --security-opt=no-new-privileges:true --cap-drop all --rm -v /var/folders/yc/2v_c1gsd4glcscmrhv03w3m80000gn/T/tmppann7g2m/unsafe/input_file:/tmp/input_file:Z -v /var/folders/yc/2v_c1gsd4glcscmrhv03w3m80000gn/T/tmppann7g2m/pixels:/dangerzone:Z -e ENABLE_TIMEOUTS=1 dangerzone.rocks/dangerzone /usr/bin/python3 /usr/local/bin/dangerzone.py document-to-pixels
ERROR Invalid JSON returned from container:

	Traceback (most recent call last):

ERROR Invalid JSON returned from container:

	  File "/usr/local/bin/dangerzone.py", line 558, in <module>

ERROR Invalid JSON returned from container:

	    sys.exit(asyncio.run(main()))

ERROR Invalid JSON returned from container:

	  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run

ERROR Invalid JSON returned from container:

	    return loop.run_until_complete(main)

ERROR Invalid JSON returned from container:

	  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete

ERROR Invalid JSON returned from container:

	    return future.result()

ERROR Invalid JSON returned from container:

	  File "/usr/local/bin/dangerzone.py", line 547, in main

ERROR Invalid JSON returned from container:

	    await converter.document_to_pixels()

ERROR Invalid JSON returned from container:

	  File "/usr/local/bin/dangerzone.py", line 218, in document_to_pixels

ERROR Invalid JSON returned from container:

	    mime_type = mime.from_file("/tmp/input_file")

ERROR Invalid JSON returned from container:

	  File "/usr/lib/python3.10/site-packages/magic/__init__.py", line 111, in from_file

ERROR Invalid JSON returned from container:

	    with _real_open(filename):

ERROR Invalid JSON returned from container:

	IsADirectoryError: [Errno 21] Is a directory: '/tmp/input_file'

ERROR documents-to-pixels failed

Failed to convert document(s)
/Users/greg/Desktop/dev/kindling_unsafe/test.pdf

[apyrgio]

Interesting. I've seen this error when the source file (/var/folders/yc/2v_c1gsd4glcscmrhv03w3m80000gn/T/tmppann7g2m/unsafe/input_file) did not exist in the host. Docker's default behavior is to assume the user wanted to mount an empty directory to the container, so it proceeded to create a directory with that name in the host.

So, my guess is that somehow, Dangerzone does create a temporary directory, but never copies the file to that directory. Which is weird, since we would have caught it during testing.

Do you have any more info that can help us here?

1. Does this happen with other files in other directories?
2. Can you mount this file with Docker reliably (docker run -v /Users/greg/Desktop/dev/kindling_unsafe/test.pdf:/tmp/input_file debian ls -l /tmp/input_file)?
3. Does perhaps Docker interpret the path as a volume (docker volume ls)?

[gregorypease280]

So on my machine i have TMPDIR=/var/folders/yc/2v_c... so it definitely thinks it wrote something there.

1. Yes it happened with other files by testing with the command from (2)
2. docker run -v /Users/greg/Desktop/dev/kindling_unsafe/test.pdf:/tmp/input_file debian ls -l /tmp/input_file

The path /Users/greg/Desktop/dev/kindling_unsafe/test.pdf is not shared from the host and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> Resources -> File Sharing.
See https://docs.docker.com/desktop/mac for more info.
(or actually this https://docs.docker.com/desktop/settings/mac/#file-sharing)

if I change the setting to allow the dev/ directory docker can mount it. However running Dangerzone continues to use the temp dir

running the container manually with the dev/ path seems to work until

/usr/local/bin/docker run --network none -u dangerzone --security-opt=no-new-privileges:true --cap-drop all --rm -v /Users/greg/Desktop/dev/kindling_unsafe/test.pdf:/tmp/input_file:Z -v /var/folders/yc/2v_c1gsd4glcscmrhv03w3m80000gn/T/tmppeef6l2f/pixels:/dangerzone:Z -e ENABLE_TIMEOUTS=1 dangerzone.rocks/dangerzone /usr/bin/python3 /usr/local/bin/dangerzone.py document-to-pixels

{"error": false, "text": "Converting page 706/708 to pixels", "percentage": 47}
{"error": false, "text": "Converting page 707/708 to pixels", "percentage": 47}
{"error": false, "text": "Converting page 708/708 to pixels", "percentage": 47}
{"error": false, "text": "Converted document to pixels", "percentage": 47}
Traceback (most recent call last):
  File "/usr/lib/python3.10/shutil.py", line 816, in move
    os.rename(src, real_dst)
OSError: [Errno 18] Cross-device link: '/tmp/page-551.rgb' -> '/dangerzone/page-551.rgb'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/dangerzone.py", line 558, in <module>
    sys.exit(asyncio.run(main()))
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/usr/local/bin/dangerzone.py", line 547, in main
    await converter.document_to_pixels()
  File "/usr/local/bin/dangerzone.py", line 394, in document_to_pixels
    shutil.move(filename, "/dangerzone")
  File "/usr/lib/python3.10/shutil.py", line 836, in move
    copy_function(src, real_dst)
  File "/usr/lib/python3.10/shutil.py", line 434, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.10/shutil.py", line 256, in copyfile
    with open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/dangerzone/page-551.rgb'`

3. docker volume ls isn't showing it.

conclusions:

so for whatever reason it looks like I wasn't sharing any directories with docker so I changed that.

sharing the /private/var/folders in the docker settings and running
/Applications/Dangerzone.app/Contents/MacOS/dangerzone-cli ~/Desktop/dev/kindling_unsafe/test.pdf
now works.

digging into it some...
https://medium.com/effy-tech/fixing-the-var-folders-error-in-docker-for-mac-v2-2-3-2a40e776132d

however for me I only needed to share /private/var/folders/ in order to get it to work

If possible I think a good fix is checking that docker has the correct folders shared before running.

thanks for the help!

[apyrgio]

We no longer use temporary directories as of #625, so I believe users will not experience an issue similar to this with the upcoming 0.8.0 release. If that's not the case, feel free to reopen the issue.