sage.misc.temporary_file: Move SAGE_TMP implementation here

[mkoeppe]

We move the code for creating the temporary directory to sage.misc.temporary_file, removing the dependency on sage.misc.misc and sage.misc.lazy_string.

This is preparation for #29941, allowing us to keep sage.misc.misc out of sagemath-objects and sagemath-repl; sage.misc.temporary_file goes into sagemath-environment.

CC: @orlitzky @kiwifb

Component: refactoring

Branch/Commit: u/mkoeppe/sage_misc_temporary_file__move_sage_tmp_implementation_here @ 8c4d015

Issue created by migration from https://trac.sagemath.org/ticket/32986

[mkoeppe]

Branch: u/mkoeppe/sage_misc_temporary_file__move_sage_tmp_implementation_here

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

81d0fff

sage.misc.temporary_file: Move SAGE_TMP implementation here

[sagetrac-git]

Commit: 81d0fff

[mkoeppe]

Author: Matthias Koeppe

[mkoeppe]

comment:4

I'll need to revise this - @functools.cache is py>=3.9

[sagetrac-git]

Changed commit from 81d0fff to 7e78f59

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

7e78f59

src/sage/misc/temporary_file.py: Remove use of functools.cache

[kwankyu]

comment:7

LGTM. Tested through #32987.

[kwankyu]

Reviewer: Kwankyu Lee

[mkoeppe]

comment:8

Thanks!

[vbraun]

comment:10

With this ticket I'm seeing a lot of flakiness with maxima-related tests, e.g.

sage -t --long --warn-long 50.7 --random-seed=36889336955867588730901666695941730921 src/sage/interfaces/maxima_abstract.py
**********************************************************************
File "src/sage/interfaces/maxima_abstract.py", line 314, in sage.interfaces.maxima_abstract.MaximaAbstract._commands
Failed example:
    sorted(maxima._commands(verbose=False))
Exception raised:
    Traceback (most recent call last):
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.interfaces.maxima_abstract.MaximaAbstract._commands[0]>", line 1, in <module>
        sorted(maxima._commands(verbose=False))
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/interfaces/maxima_abstract.py", line 327, in _commands
        [self.completions(chr(65+n), verbose=verbose)+
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/interfaces/maxima_abstract.py", line 327, in <listcomp>
        [self.completions(chr(65+n), verbose=verbose)+
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/interfaces/maxima_abstract.py", line 296, in completions
        cmd_list = self._eval_line('apropos("%s")'%s, error_check=False)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/interfaces/maxima.py", line 814, in _eval_line
        self._expect_expr(self._display_prompt)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/interfaces/maxima.py", line 731, in _expect_expr
        i = self._expect.expect(expr)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/pexpect/spawnbase.py", line 343, in expect
        return self.expect_list(compiled_pattern_list,
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/pexpect/spawnbase.py", line 372, in expect_list
        return exp.expect_loop(timeout)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/pexpect/expect.py", line 179, in expect_loop
        return self.eof(e)
      File "/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/pexpect/expect.py", line 122, in eof
        raise exc
    pexpect.exceptions.EOF: End Of File (EOF). Exception style platform.
    Maxima with PID 1795009 running /home/release/Sage/local/bin/maxima -p /home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/interfaces/sage-maxima.lisp
    command: /home/release/Sage/local/bin/maxima
    args: ['/home/release/Sage/local/bin/maxima', '-p', '/home/release/Sage/local/var/lib/sage/venv-python3.9.9/lib/python3.9/site-packages/sage/interfaces/sage-maxima.lisp']
    buffer (last 100 chars): b''
    before (last 100 chars): ''
    after: <class 'pexpect.exceptions.EOF'>
    match: None
    match_index: None
    exitstatus: None
    flag_eof: True
    pid: 1795009
    child_fd: 20
    closed: False
    timeout: None
    delimiter: <class 'pexpect.exceptions.EOF'>
    logfile: None
    logfile_read: None
    logfile_send: None
    maxread: 4194304
    ignorecase: False
    searchwindowsize: None
    delaybeforesend: None
    delayafterclose: 0.1
    delayafterterminate: 0.1
    searcher: searcher_re:
        0: re.compile(b'<sage-display>')
**********************************************************************

Always disappears when testing individually, but running multiple maxima-using tests in parallel triggers them quite reliably. E.g.

./sage -t -p 8 --long src/sage/interfaces/maxima_abstract.py src/sage/dynamics/complex_dynamics/mandel_julia.py src/sage/interfaces/maxima.py src/sage/tests/books/computational-mathematics-with-sagemath/sol/mpoly_doctest.py src/sage/coding/kasami_codes.pyx

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

fc509ec

Merge tag '9.6.beta0' into t/32986/sage_misc_temporary_file__move_sage_tmp_implementation_here

[sagetrac-git]

Changed commit from 7e78f59 to fc509ec

[mkoeppe]

comment:12

Confirmed the test failures when running in parallel. Not sure what's happening there, help welcome

[kiwifb]

comment:13

The only thing I think could happen, should have been happening before this ticket. You mostly moved code around. May be there was always a potential race condition on the closing of sage_tmp and it just got enhanced.

[orlitzky]

comment:14

The diff looks OK to me too, but over the past week or so I've realized that you can't trust anything that goes on in the expect interfaces. Using lazy_string adds a layer of unnecessary complexity too; there's no need to encode the PID into the tempfile path.

I'm in the process of removing direct uses of SAGE_TMP anyway if you just want to wait. Afterwards it will be trivial to switch tmp_filename() and tmp_dir(), and SAGE_TMP can be removed from the misc module.

[orlitzky]

comment:15

Color me unsurprised that I'm seeing random (both non-reproducible, and can't-possibly-be-related-to-anything-I-changed) pexpect failures on #33213 as well.

[sagetrac-git]

Changed commit from fc509ec to 8c4d015

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

2e81b67	sage.misc.temporary_file: Move SAGE_TMP implementation here
8c4d015	src/sage/misc/temporary_file.py: Remove use of functools.cache

[mkoeppe]

comment:18

Superseded by #33797/#33213

[mkoeppe]

Changed author from Matthias Koeppe to none

[mkoeppe]

Changed reviewer from Kwankyu Lee to none