Temporary files: Permission denied

[felixdivo]

I am trying to write some temporary files using Python's NamedTemporaryFile and alike, and get errors:

IOError: [Errno 13] Permission denied: 'c:\\projects\\python-can\\test\\__tmpdir__\\tmpj6uyhy.asc'

This occurs in the python-can repo, for example right here. The .appveyor.yml file looks like this:

environment:
  matrix:

    # For Python versions available on Appveyor, see
    # http://www.appveyor.com/docs/installed-software#python

    - PYTHON: "C:\\Python27"
    - PYTHON: "C:\\Python34"
    - PYTHON: "C:\\Python35"
    - PYTHON: "C:\\Python36"
    - PYTHON: "C:\\Python27-x64"
    - PYTHON: "C:\\Python34-x64"
    - PYTHON: "C:\\Python35-x64"
    - PYTHON: "C:\\Python36-x64"

    # Python 3.3 has reached EOL

install:
  # Prepend Python installation to PATH
  - set PATH=%PYTHON_INSTALL%;%PATH%

  # Prepend Python scripts to PATH (e.g. py.test)
  - set PATH=%PYTHON_INSTALL%\\Scripts;%PATH%

  # We need to install the python-can library itself
  - "python -m pip install .[test,neovi]"

build: off

before_test:
  # see: https://help.appveyor.com/discussions/problems/938-getting-permission-denied-errors-when-trying-to-make-a-temp-directory
  # TMPDIR is used by Python to detect the temporary directory: https://docs.python.org/3/library/tempfile.html#tempfile.tempdir
  - set TMPDIR=%APPVEYOR_BUILD_FOLDER%\test\__tmpdir__
  - mkdir %TMPDIR%
  - icacls %TMPDIR% /inheritance:r /grant Everyone:(OI)(CI)F

test_script:
  - "pytest"
  - "codecov"

after_test:
  - rmdir /S /Q %TMPDIR%

Note: The before_test section was added in hope to solve the problem, but didn't help.

[IlyaFinkelshteyn]

I have no exact solution, but let me share my investigation, maybe it will bring some ideas.

I cloned your repo and was able to reproduce the problem. Then connected to build worker with RDP and was able to reproduce it manually in command line (was needed to set TMPDIR before). Then installed Process monitor and monitored access to everything in C:\projects\python-can\test\__tmpdir__ and I see that with both 2 files from pytest error (and only with them), SHARING VIOLATION error occurred (see pic).

Can it be some concurrency issue? I have no knowledge about pytest, but maybe you can disable some tests concurrency? Also what if you add some retry logic?

And finally, does it work when being executed the same way on local Windows machine?

[felixdivo]

Thanks for your investigation!

Can it be some concurrency issue?

I doubt it, since Pytest without any plugins does not use concurrent testing AFAIK.

Also what if you add some retry logic?

I could add some retry logic, although that would only be a solution for debugging the problem.

And is it the same way on a local Windows machine?

I will check.

[felixdivo]

Okay, I solved the problem. Actually, it was not a problem about permissions at all. ☹️

It was a problem of how tempfile.NamedTemporaryFile works on Windows. This now works:

with tempfile.NamedTemporaryFile(suffix=extension, delete=False) as my_file:
    filename = my_file.name
with can.Logger(filename) as writer:
    self.assertIsInstance(writer, klass)
# delete "manually" afterwards

[IlyaFinkelshteyn]

Great, thanks for the update!