temp_dir does not exist

[PiotrDabrowskey]

Issue

Using a {temp_dir} in env commands is inconvenient due to the fact the directory expanded by the variable does not exist. Also, the documentation does state that it's a directory to put temporary files, but doesn't say it should be created manually. I'm not sure whether it is an expected behavior, but the directory was existing tox 3.x, so the issue appeared in tox 4.

Current workaround is to add mkdir to allowlist_externals and add mkdir --parents {temp_dir} to commands or commands_pre.

By the way: the documentation states that temp_dir defaults to {tox_root}/.temp, while my testing showed the subdirectory is named .tmp, not .temp.

Environment

• OS: Ubuntu 22.04.1 LTS (jammy)
• pip list of the host Python where tox is installed:

Package       Version
------------- ---------
autoflake     2.0.0
cachetools    5.2.0
certifi       2022.12.7
chardet       5.1.0
colorama      0.4.6
distlib       0.3.6
filelock      3.8.2
packaging     22.0
pip           22.3.1
platformdirs  2.6.0
pluggy        1.0.0
pyflakes      3.0.1
pyproject_api 1.2.1
setuptools    65.5.0
tomli         2.0.1
tox           4.0.16
virtualenv    20.17.1
wheel         0.37.1

Output of running tox

Provide the output of tox -rvv:

tox -rvv
py310: 182 W remove tox env folder /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310 [tox/tox_env/api.py:311]
py310: 197 I find interpreter for spec PythonSpec(major=3, minor=10) [virtualenv/discovery/builtin.py:56]
py310: 197 I proposed PythonInfo(spec=CPython3.10.8.final.0-64, exe=/home/pdabrowski/miniconda3/envs/tox/bin/python, platform=linux, version='3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:63]
py310: 197 D accepted PythonInfo(spec=CPython3.10.8.final.0-64, exe=/home/pdabrowski/miniconda3/envs/tox/bin/python, platform=linux, version='3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:65]
py310: 198 D filesystem is case-sensitive [virtualenv/info.py:24]
py310: 224 I create virtual environment via CPython3Posix(dest=/home/pdabrowski/PycharmProjects/tox-bug/.tox/py310, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:48]
py310: 224 D create folder /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/bin [virtualenv/util/path/_sync.py:9]
py310: 224 D create folder /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages [virtualenv/util/path/_sync.py:9]
py310: 225 D write /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
py310: 225 D 	home = /home/pdabrowski/miniconda3/envs/tox/bin [virtualenv/create/pyenv_cfg.py:34]
py310: 225 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
py310: 225 D 	version_info = 3.10.8.final.0 [virtualenv/create/pyenv_cfg.py:34]
py310: 225 D 	virtualenv = 20.17.1 [virtualenv/create/pyenv_cfg.py:34]
py310: 225 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
py310: 225 D 	base-prefix = /home/pdabrowski/miniconda3/envs/tox [virtualenv/create/pyenv_cfg.py:34]
py310: 225 D 	base-exec-prefix = /home/pdabrowski/miniconda3/envs/tox [virtualenv/create/pyenv_cfg.py:34]
py310: 225 D 	base-executable = /home/pdabrowski/miniconda3/envs/tox/bin/python [virtualenv/create/pyenv_cfg.py:34]
py310: 225 D symlink /home/pdabrowski/miniconda3/envs/tox/bin/python to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/bin/python [virtualenv/util/path/_sync.py:28]
py310: 225 D create virtualenv import hook file /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:89]
py310: 225 D create /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:92]
py310: 225 D ============================== target debug ============================== [virtualenv/run/session.py:50]
py310: 225 D debug via /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/bin/python /home/pdabrowski/miniconda3/envs/tox/lib/python3.10/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:197]
py310: 225 D {
  "sys": {
    "executable": "/home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/bin/python",
    "_base_executable": "/home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/bin/python",
    "prefix": "/home/pdabrowski/PycharmProjects/tox-bug/.tox/py310",
    "base_prefix": "/home/pdabrowski/miniconda3/envs/tox",
    "real_prefix": null,
    "exec_prefix": "/home/pdabrowski/PycharmProjects/tox-bug/.tox/py310",
    "base_exec_prefix": "/home/pdabrowski/miniconda3/envs/tox",
    "path": [
      "/home/pdabrowski/miniconda3/envs/tox/lib/python310.zip",
      "/home/pdabrowski/miniconda3/envs/tox/lib/python3.10",
      "/home/pdabrowski/miniconda3/envs/tox/lib/python3.10/lib-dynload",
      "/home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages"
    ],
    "meta_path": [
      "<class '_virtualenv._Finder'>",
      "<class '_frozen_importlib.BuiltinImporter'>",
      "<class '_frozen_importlib.FrozenImporter'>",
      "<class '_frozen_importlib_external.PathFinder'>"
    ],
    "fs_encoding": "utf-8",
    "io_encoding": "utf-8"
  },
  "version": "3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0]",
  "makefile_filename": "/home/pdabrowski/miniconda3/envs/tox/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile",
  "os": "<module 'os' from '/home/pdabrowski/miniconda3/envs/tox/lib/python3.10/os.py'>",
  "site": "<module 'site' from '/home/pdabrowski/miniconda3/envs/tox/lib/python3.10/site.py'>",
  "datetime": "<module 'datetime' from '/home/pdabrowski/miniconda3/envs/tox/lib/python3.10/datetime.py'>",
  "math": "<module 'math' from '/home/pdabrowski/miniconda3/envs/tox/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so'>",
  "json": "<module 'json' from '/home/pdabrowski/miniconda3/envs/tox/lib/python3.10/json/__init__.py'>"
} [virtualenv/run/session.py:51]
py310: 244 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/pdabrowski/.local/share/virtualenv) [virtualenv/run/session.py:55]
py310: 248 D got embed update of distribution pip from /home/pdabrowski/.local/share/virtualenv/wheel/3.10/embed/3/pip.json [virtualenv/app_data/via_disk_folder.py:129]
py310: 248 D got embed update of distribution wheel from /home/pdabrowski/.local/share/virtualenv/wheel/3.10/embed/3/wheel.json [virtualenv/app_data/via_disk_folder.py:129]
py310: 249 D got embed update of distribution setuptools from /home/pdabrowski/.local/share/virtualenv/wheel/3.10/embed/3/setuptools.json [virtualenv/app_data/via_disk_folder.py:129]
py310: 254 D install pip from wheel /home/pdabrowski/miniconda3/envs/tox/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-22.3.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py310: 254 D install wheel from wheel /home/pdabrowski/miniconda3/envs/tox/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/wheel-0.38.4-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py310: 254 D install setuptools from wheel /home/pdabrowski/miniconda3/envs/tox/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/setuptools-65.6.3-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
py310: 257 D copy /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip-22.3.1.virtualenv to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/pip-22.3.1.virtualenv [virtualenv/util/path/_sync.py:36]
py310: 257 D copy directory /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.dist-info to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/wheel-0.38.4.dist-info [virtualenv/util/path/_sync.py:36]
py310: 257 D copy directory /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/_distutils_hack to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:36]
py310: 258 D copy directory /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip-22.3.1.dist-info to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/pip-22.3.1.dist-info [virtualenv/util/path/_sync.py:36]
py310: 260 D copy /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools-65.6.3.virtualenv to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/setuptools-65.6.3.virtualenv [virtualenv/util/path/_sync.py:36]
py310: 260 D copy directory /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/setuptools [virtualenv/util/path/_sync.py:36]
py310: 262 D copy directory /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/wheel [virtualenv/util/path/_sync.py:36]
py310: 264 D copy directory /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-22.3.1-py3-none-any/pip to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/pip [virtualenv/util/path/_sync.py:36]
py310: 274 D copy /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.virtualenv to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/wheel-0.38.4.virtualenv [virtualenv/util/path/_sync.py:36]
py310: 277 D generated console scripts wheel wheel3 wheel3.10 wheel-3.10 [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py310: 315 D copy directory /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/setuptools-65.6.3.dist-info to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/setuptools-65.6.3.dist-info [virtualenv/util/path/_sync.py:36]
py310: 316 D copy directory /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/pkg_resources to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/pkg_resources [virtualenv/util/path/_sync.py:36]
py310: 323 D copy /home/pdabrowski/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-65.6.3-py3-none-any/distutils-precedence.pth to /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/lib/python3.10/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:36]
py310: 324 D generated console scripts  [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py310: 334 D generated console scripts pip pip-3.10 pip3.10 pip3 [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
py310: 334 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:61]
py310: 335 D write /home/pdabrowski/PycharmProjects/tox-bug/.tox/py310/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
py310: 335 D 	home = /home/pdabrowski/miniconda3/envs/tox/bin [virtualenv/create/pyenv_cfg.py:34]
py310: 336 D 	implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
py310: 336 D 	version_info = 3.10.8.final.0 [virtualenv/create/pyenv_cfg.py:34]
py310: 336 D 	virtualenv = 20.17.1 [virtualenv/create/pyenv_cfg.py:34]
py310: 336 D 	include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
py310: 336 D 	base-prefix = /home/pdabrowski/miniconda3/envs/tox [virtualenv/create/pyenv_cfg.py:34]
py310: 336 D 	base-exec-prefix = /home/pdabrowski/miniconda3/envs/tox [virtualenv/create/pyenv_cfg.py:34]
py310: 336 D 	base-executable = /home/pdabrowski/miniconda3/envs/tox/bin/python [virtualenv/create/pyenv_cfg.py:34]
py310: 337 W commands[0]> touch /home/pdabrowski/PycharmProjects/tox-bug/.tox/.tmp/testfile [tox/tox_env/api.py:417]
/usr/bin/touch: cannot touch '/home/pdabrowski/PycharmProjects/tox-bug/.tox/.tmp/testfile': No such file or directory
py310: 340 C exit 1 (0.00 seconds) /home/pdabrowski/PycharmProjects/tox-bug> touch /home/pdabrowski/PycharmProjects/tox-bug/.tox/.tmp/testfile pid=72591 [tox/execute/api.py:275]
  py310: FAIL code 1 (0.16=setup[0.16]+cmd[0.00] seconds)
  evaluation failed :( (0.26 seconds)

Minimal example

Simple tox.ini file allowing to recreate the problem:

[tox]
envlist = py310

[testenv]
allowlist_externals = touch
commands = touch {temp_dir}/testfile