Document and manage temporary directories

[jhpalmieri]

This is a followup to #33213. Some users prefer (or need) to control where temporary files are created, and we should document this:

• as described at https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir, users can set the environment variable TMPDIR

Possible follow-up:

• Also, according to https://manpages.ubuntu.com/manpages/bionic/man8/tmpreaper.8.html, at least on some systems it would be helpful to create a non-writable file .tmpreaper in the directory to prevent the system from doing automatic cleanup on it.

Component: misc

Author: John Palmieri

Branch/Commit: c729c7b

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/34593

[jhpalmieri]

comment:1

Of course I don't know where we should document this, since it's a Python feature which may be used in various components of Sage. I don't know the natural home for it.

[egourgoulhon]

comment:2

Regarding the ticket title, IMHO the objective should be more "manage" than "document" since 3D plots are badly broken in Sage 9.7 for (at least) Ubuntu 22.04 users with the default settings, as revealed by https://ask.sagemath.org/question/64192/temporary-html-files-location-in-sage-97/

[jhpalmieri]

comment:4

Should Sage itself set the environment variable TMPDIR? (If the user hasn't set it, I mean.)

[egourgoulhon]

comment:5

Replying to John Palmieri:

Should Sage itself set the environment variable TMPDIR? (If the user hasn't set it, I mean.)

This could a solution, for instance defining TMPDIR to be a subdirectory of DOT_SAGE. But there may be some undesirable effects, e.g. for Sage running on servers or SageMathCell. Also would it be desirable/doable to clean that directory when the Sage session ends?

[dimpase]

comment:7

Replying to Eric Gourgoulhon:

Replying to John Palmieri:

Should Sage itself set the environment variable TMPDIR? (If the user hasn't set it, I mean.)

This could a solution, for instance defining TMPDIR to be a subdirectory of DOT_SAGE. But there may be some undesirable effects, e.g. for Sage running on servers or SageMathCell. Also would it be desirable/doable to clean that directory when the Sage session ends?

I beleive that the cleanup is automatic, using Python's atexit callback.
https://docs.python.org/3/library/atexit.html

(there is of course always a chance of a segfault-like abnormal exit, which might leave
rubbish after itself:

The functions registered via this module are not called when the program is killed
by a signal not handled by Python, when a Python fatal internal error is detected,
or when os._exit() is called.

This is what we have in src/misc/temporary_file.py:

import tempfile

import atexit

# Until tmp_dir() and tmp_filename() are removed, we use this directory
# as the parent for all temporary files & directories created by them.
# This lets us clean up after those two functions when sage exits normally
# using an atexit hook
TMP_DIR_FILENAME_BASE=tempfile.TemporaryDirectory()
atexit.register(lambda: TMP_DIR_FILENAME_BASE.cleanup())

[dimpase]

comment:8

In an environment such as SageMathCell, I think one user runs many Sage instances, so one cannot clean anything on startup without risking to affect other instances.

[mkoeppe]

comment:9

Let's move the documentation of runtime environment variables, starting at "Sage uses the following environment variables when it runs:" https://doc.sagemath.org/html/en/installation/source.html to the "Launching section" and add TMPDIR there (perhaps pointer to Python runtime environment variables)

[jhpalmieri]

comment:10

It's not just Sage, it's a general Jupyter problem. See ContinuumIO/anaconda-issues#12891 and https://askubuntu.com/questions/1389798/how-to-correctly-configure-snapd-firefox-to-open-local-html-file-generated-by.

[jhpalmieri]

comment:11

And also jupyter/notebook#4500.

[jhpalmieri]

Branch: u/jhpalmieri/document-TMPDIR

[jhpalmieri]

Author: John Palmieri

[jhpalmieri]

comment:13

Here is an addition to the documentation.

---

New commits:

c729c7b

trac 34593: document the variable TMPDIR.

[jhpalmieri]

Commit: c729c7b

[mkoeppe]

Reviewer: Matthias Koeppe

[jhpalmieri]

comment:15

Thank you!

[vbraun]

Changed branch from u/jhpalmieri/document-TMPDIR to c729c7b