Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support tox 4.x for testing modularized distributions #35208

Merged
merged 15 commits into from
May 22, 2023

Conversation

mkoeppe
Copy link
Contributor

@mkoeppe mkoeppe commented Feb 27, 2023

📚 Description

Previous fixes for tox 4.x were not complete, as observed in #35095 (comment)

The files pkgs/*/tox.ini now requires tox >= 4.2.
tox 3.x and 4.x auto-provision a suitable version, so it is not immediately necessary to upgrade our tox package (or tighten the version requirements in build/pkgs/tox/spkg-configure.m4.

📝 Checklist

  • I have made sure that the title is self-explanatory and the description concisely explains the PR.
  • I have linked an issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation accordingly.

⌛ Dependencies

@mkoeppe mkoeppe added this to the sage-10.0 milestone Feb 27, 2023
@mkoeppe mkoeppe requested a review from kwankyu February 27, 2023 03:08
@mkoeppe mkoeppe self-assigned this Feb 27, 2023
@codecov-commenter
Copy link

codecov-commenter commented Feb 27, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.01 ⚠️

Comparison is base (f449b14) 88.62% compared to head (eabf6fd) 88.61%.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop   #35208      +/-   ##
===========================================
- Coverage    88.62%   88.61%   -0.01%     
===========================================
  Files         2148     2148              
  Lines       398653   398653              
===========================================
- Hits        353308   353283      -25     
- Misses       45345    45370      +25     

see 25 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 16, 2023

Running

make SAGE_WHEELS=yes sagemath_categories-build-deps && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=8 tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)'

I get

Writing sagemath-categories-10.0b4/setup.cfg
Creating tar archive
removing 'sagemath-categories-10.0b4' (and everything under it)
Backend: Wrote response {'return': 'sagemath-categories-10.0b4.tar.gz'} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517_build_sdist-_eofd7b5.json
.pkg-sagepython-sagewheels-nopypi: 17005 I exit None (5.42 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=2054 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 17006 D package .tmp/package/2/sagemath-categories-10.0b4.tar.gz links to .pkg-sagepython-sagewheels-nopypi/dist/sagemath-categories-10.0b4.tar.gz (/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox) [tox/util/file_view.py:36]
sagepython-sagewheels-nopypi-norequirements: 17007 W install_package_deps> python -I -m pip install 'sagemath-objects~=10.0b4' sagemath-repl [tox/tox_env/api.py:428]
Looking in links: file:///Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/var/lib/sage/wheels
Processing /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/var/lib/sage/wheels/sagemath_objects-10.0b4-cp311-cp311-macosx_12_0_x86_64.whl
ERROR: Could not find a version that satisfies the requirement sagemath-repl (from versions: none)
ERROR: No matching distribution found for sagemath-repl
sagepython-sagewheels-nopypi-norequirements: 17779 C exit 1 (0.77 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories> python -I -m pip install 'sagemath-objects~=10.0b4' sagemath-repl pid=2056 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 17780 W _exit> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517__exit-ejch1cup.json
.pkg-sagepython-sagewheels-nopypi: 17783 I exit None (0.00 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=2054 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 17872 D delete package /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-categories/.tox/.tmp/package/2/sagemath-categories-10.0b4.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
  sagepython-sagewheels-nopypi-norequirements: FAIL code 1 (17.62 seconds)
  evaluation failed :( (17.71 seconds)

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 16, 2023

Running

 ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e sagepython-sagewheels-nopypi)'

results in

...
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/doc/de/a_tour_of_sage/index.rst  # 1 doctest failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/generators/families.py  # 3 doctests failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/graph_generators.py  # 1 doctest failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/data_structures/stream.py  # Timed out
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py  # 5 doctests failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 2173.7 seconds
    cpu time: 12668.0 seconds
    cumulative wall time: 16102.6 seconds
Features detected for doctesting: 4ti2,csdp,cvxopt,dvipng,gfan,graphviz,imagemagick,nauty,palp,pandoc,pdf2svg,pdftocairo,polytopes_db,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.misc.cython,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.symbolic,sagemath_doc_html,sphinx
pytest is not installed in the venv, skip checking tests that rely on it
sagepython-sagewheels-nopypi: 5220971 C exit 5 (2182.09 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> sage -t -p --all pid=4907 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 5220972 W _exit> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517__exit-8m9b2g26.json
.pkg-sagepython-sagewheels-nopypi: 5220975 I exit None (0.00 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=776 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 5221052 D delete package /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/.tmp/package/1/sagemath-standard-10.0b4.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
  sagepython-sagewheels-nopypi: FAIL code 5 (5221.03=setup[3034.81]+cmd[2.00,2.12,2182.09] seconds)
  evaluation failed :( (5221.11 seconds)

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 16, 2023

make SAGE_WHEELS=yes sagemath_categories-build-deps && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=8 tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)'

...

ERROR: Could not find a version that satisfies the requirement sagemath-repl (from versions: none)

Nice catch. Of course, sagemath-repl is not part of the build-deps of sagemath-categories unless SAGE_CHECK=yes is set. I'll update the instructions.

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 19, 2023

I ran

make SAGE_WHEELS=yes sagemath_categories-build-deps sagemath_repl && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=8 tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)' 

after merging 10.0.beta5, and got

...
[sagemath_objects-10.0.beta5] * Creating venv isolated environment...
[sagemath_objects-10.0.beta5] * Installing packages in isolated environment... (cysignals >=1.10.2, cython >=0.29.21, <1.0, gmpy2 >=2.1.0, sage-setup ~= 10.0b5, sagemath-environment ~= 10.0b5, setuptools >=49.6.0, wheel >=0.36.2)
[sagemath_objects-10.0.beta5] Looking in links: file:///Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/var/lib/sage/wheels
[sagemath_objects-10.0.beta5] ERROR: Could not find a version that satisfies the requirement sage-setup~=10.0b5 (from versions: 10.0b4)
[sagemath_objects-10.0.beta5] ERROR: No matching distribution found for sage-setup~=10.0b5
[sagemath_objects-10.0.beta5] 

Then I did make wheels and tried again. Now it succeeded.

Was the initial failure expected?

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 19, 2023

I ran

make SAGE_WHEELS=yes sagemath_categories-build-deps sagemath_repl && ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=8 tox -v -v -v -e sagepython-sagewheels-nopypi-norequirements)' 

after merging 10.0.beta5

These instructions from the top of tox.ini assume that no packages have been built yet. If sage_setup or other sagemath... packages were built already (in editable mode), this won't rebuild them.

make wheels (as shown in the developer's guide) takes care of first uninstalling the editable installations, which ensures that the wheels are available.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 20, 2023

sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py # 5 doctests failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py # 1 doctest failed

I think only these two may have something to do with this PR.

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 20, 2023

Now I am getting

ERROR: Could not find a version that satisfies the requirement scipy==1.10.1 (from versions: 1.9.3)
ERROR: No matching distribution found for scipy==1.10.1
sagepython-sagewheels-nopypi: 1496 C exit 1 (1.21 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> python -I -m pip install -r requirements.txt pid=79397 [tox/execute/api.py:275]
  sagepython-sagewheels-nopypi: FAIL code 1 (1.27 seconds)
  evaluation failed :( (1.35 seconds)

from the run of

./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e sagepython-sagewheels-nopypi)'

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 20, 2023

Is this after running make wheels?

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 20, 2023

Yes. It didn't make a difference.

Then I removed the old wheel scipy1.9.3, and make scipy. I am running the command again. It is now at

sagepython-sagewheels-nopypi: 98255 W install_package> python -I -m pip install --force-reinstall --no-deps /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/.tmp/package/3/sagemath-standard-10.0b5.tar.gz [tox/tox_env/api.py:428]
Looking in links: file:///Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/var/lib/sage/wheels
Processing ./.tox/.tmp/package/3/sagemath-standard-10.0b5.tar.gz
  Installing build dependencies ... \done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: sagemath-standard
  Building wheel for sagemath-standard (pyproject.toml) ... /

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 20, 2023

I see scipy-1.10.1-cp311-cp311-macosx_12_0_x86_64.whl in the wheels directory.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 20, 2023

OK, this happened because scipy is missing in build/pkgs/sagelib/dependencies. I think this was deliberate at one time, to allow the sagelib build to start before scipy has finished building.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 20, 2023

In fact, several other packages listed in src/setup.cfg.m4 under "SAGERUNTIME" and "DOC_DEPENDENCIES" should really be added

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 20, 2023

The run finished with

sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/doc/de/tutorial/latex.rst  # 1 doctest failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/generators/families.py  # 3 doctests failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/graph_generators.py  # 1 doctest failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/data_structures/stream.py  # Timed out
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/dynamics/arithmetic_dynamics/projective_ds.py  # Timed out
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/interfaces/gap_workspace.py  # 2 doctests failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py  # 5 doctests failed
sage -t --random-seed=226979055037151283592527401300349772391 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 2340.1 seconds
    cpu time: 13316.0 seconds
    cumulative wall time: 17139.9 seconds
Features detected for doctesting: 4ti2,csdp,cvxopt,dvipng,gfan,graphviz,imagemagick,nauty,palp,pandoc,pdf2svg,pdftocairo,polytopes_db,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.misc.cython,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.symbolic,sagemath_doc_html,sphinx
pytest is not installed in the venv, skip checking tests that rely on it
sagepython-sagewheels-nopypi: 5497901 C exit 5 (2351.72 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> sage -t -p --all pid=39122 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 5497903 W _exit> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517__exit-6m1dxphn.json
.pkg-sagepython-sagewheels-nopypi: 5497907 I exit None (0.00 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=35633 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 5498017 D delete package /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/.tmp/package/3/sagemath-standard-10.0b5.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
  sagepython-sagewheels-nopypi: FAIL code 5 (5497.98=setup[3141.85]+cmd[2.07,2.34,2351.72] seconds)
  evaluation failed :( (5498.07 seconds)

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 20, 2023

sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py # 5 doctests failed
sage -t --random-seed=233132637117582060430838751734210350117 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py # 1 doctest failed

I think only these two may have something to do with this PR.

The tests pass in the normal sage install.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Mar 20, 2023

Could you post the full log of the failures please?

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 20, 2023

Ah, I closed the terminal! I started running it again. I will take long to see the result.

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 20, 2023

sage -t --warn-long 80.0 --random-seed=311058290845287139560044406479667880892 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py", line 353, in sage.repl.ipython_extension.SageMagics.cython
Failed example:
    shell.run_cell('''
    %%cython
    def f():
        print('test')
    ''')
Expected nothing
Got:
    doctest:warning
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/bin/sage-runtests", line 154, in <module>
        err = DC.run()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/control.py", line 1384, in run
        self.run_doctests()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/control.py", line 1059, in run_doctests
        self.dispatcher.dispatch()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2017, in dispatch
        self.parallel_dispatch()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1912, in parallel_dispatch
        w.start()  # This might take some time
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2186, in start
        super().start()
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/process.py", line 121, in start
        self._popen = self._Popen(self)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/context.py", line 224, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/context.py", line 281, in _Popen
        return Popen(process_obj)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/popen_fork.py", line 19, in __init__
        self._launch(process_obj)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/popen_fork.py", line 71, in _launch
        code = process_obj._bootstrap(parent_sentinel=child_r)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
        self.run()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2158, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2488, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 2540, in _run
        result = runner.run(test)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 866, in run
        return self._run(test, compileflags, out)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.repl.ipython_extension.SageMagics.cython[2]>", line 1, in <module>
        shell.run_cell('''
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/repl/interpreter.py", line 390, in run_cell
        super().run_cell(*args, **kwds)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 2940, in run_cell
        result = self._run_cell(
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 2995, in _run_cell
        return runner(coro)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
        coro.send(None)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3194, in run_cell_async
        has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3373, in run_ast_nodes
        if await self.run_code(code, result, async_=asy):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3433, in run_code
        exec(code_obj, self.user_global_ns, self.user_ns)
      File "<ipython-input-1-b9ac10b76270>", line 1, in <module>
        get_ipython().run_cell_magic('cython', '', "def f():\n    print('test')\n")
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 2417, in run_cell_magic
        result = fn(*args, **kwargs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/repl/ipython_extension.py", line 362, in cython
        return cython_compile(cell)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 673, in cython_compile
        return cython_import_all(tmpfile, get_globals(), **kwds)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 563, in cython_import_all
        m = cython_import(filename, **kwds)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 534, in cython_import
        name, build_dir = cython(filename, **kwds)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 291, in cython
        standard_libs, standard_libdirs, standard_includes, aliases = _standard_libs_libdirs_incdirs_aliases()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/sage/misc/cython.py", line 60, in _standard_libs_libdirs_incdirs_aliases
        standard_incdirs = sage_include_directories() + aliases["CBLAS_INCDIR"] + aliases["NTL_INCDIR"]
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/env.py", line 307, in sage_include_directories
        import distutils.sysconfig
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1140, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 1080, in _find_spec
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/_distutils_hack/__init__.py", line 97, in find_spec
        return method()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/_distutils_hack/__init__.py", line 108, in spec_for_distutils
        mod = importlib.import_module('setuptools._distutils')
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/importlib/__init__.py", line 126, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 940, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/setuptools/__init__.py", line 16, in <module>
        import setuptools.version
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 940, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/setuptools/version.py", line 1, in <module>
        import pkg_resources
      File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
      File "<frozen importlib._bootstrap>", line 1149, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 940, in exec_module
      File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3248, in <module>
        @_call_aside
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3223, in _call_aside
        f(*args, **kwargs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3274, in _initialize_master_working_set
        tuple(dist.activate(replace=False) for dist in working_set)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3274, in <genexpr>
        tuple(dist.activate(replace=False) for dist in working_set)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 2790, in activate
        self.insert_on(path, replace=replace)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 2922, in insert_on
        self.check_version_conflict()
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 2963, in check_version_conflict
        issue_warning(
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/pkg_resources/__init__.py", line 3076, in issue_warning
        warnings.warn(stacklevel=level + 1, *args, **kw)
      File "/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.11.1/lib/python3.11/warnings.py", line 109, in _showwarnmsg
        sw(msg.message, msg.category, msg.filename, msg.lineno,
    :
    UserWarning: Module sage was already imported from None, but /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard is being added to sys.path
    ld: warning: -undefined dynamic_lookup may not work with chained fixups
**********************************************************************
1 item had failures:
   1 of   5 in sage.repl.ipython_extension.SageMagics.cython
    [93 tests, 1 failure, 14.79 s]

@kwankyu
Copy link
Collaborator

kwankyu commented Mar 20, 2023

sage -t --warn-long 80.0 --random-seed=311058290845287139560044406479667880892 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 389, in sage.misc.package.installed_packages
Failed example:
    sorted(installed_packages().keys())  # optional - sage_spkg
Expected:
    [...'gmpy2', ...'sage_conf', ...]
Got:
    ['arb',
     'brial',
     'bzip2',
     'cddlib',
     'cliquer',
     'combinatorial_designs',
     'conway_polynomials',
     'eclib',
     'ecm',
     'elliptic_curves',
     'fflas_ffpack',
     'flint',
     'flintqs',
     'gap',
     'gf2x',
     'gfortran',
     'giac',
     'givaro',
     'glpk',
     'gnulib',
     'graphs',
     'gsl',
     'iml',
     'jmol',
     'lcalc',
     'libbraiding',
     'libhomfly',
     'linbox',
     'lrcalc',
     'm4ri',
     'm4rie',
     'mathjax',
     'maxima',
     'mpc',
     'openblas',
     'palp',
     'pari',
     'pari_galdata',
     'pari_seadata_small',
     'patchelf',
     'pcre',
     'planarity',
     'polytopes_db',
     'ppl',
     'pplpy_doc',
     'rw',
     'sagemath_doc_html',
     'singular',
     'sqlite',
     'suitesparse',
     'symmetrica',
     'sympow',
     'tachyon',
     'threejs']
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 391, in sage.misc.package.installed_packages
Failed example:
    installed_packages()['gmpy2']  # optional - sage_spkg, random
Exception raised:
    Traceback (most recent call last):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.installed_packages[1]>", line 1, in <module>
        installed_packages()['gmpy2']  # optional - sage_spkg, random
        ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
    KeyError: 'gmpy2'
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 624, in sage.misc.package.package_manifest
Failed example:
    sagetex_manifest = package_manifest('sagetex')  # optional - sage_spkg
Exception raised:
    Traceback (most recent call last):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.package_manifest[1]>", line 1, in <module>
        sagetex_manifest = package_manifest('sagetex')  # optional - sage_spkg
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/misc/package.py", line 637, in package_manifest
        version = installed_packages()[package]
                  ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
    KeyError: 'sagetex'
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 625, in sage.misc.package.package_manifest
Failed example:
    sagetex_manifest['package_name'] == 'sagetex'  # optional - sage_spkg
Exception raised:
    Traceback (most recent call last):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.package_manifest[2]>", line 1, in <module>
        sagetex_manifest['package_name'] == 'sagetex'  # optional - sage_spkg
        ^^^^^^^^^^^^^^^^
    NameError: name 'sagetex_manifest' is not defined
**********************************************************************
File "/Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py", line 627, in sage.misc.package.package_manifest
Failed example:
    'files' in sagetex_manifest  # optional - sage_spkg
Exception raised:
    Traceback (most recent call last):
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/sagepython-sagewheels-nopypi/lib/python3.11/site-packages/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.misc.package.package_manifest[3]>", line 1, in <module>
        'files' in sagetex_manifest  # optional - sage_spkg
                   ^^^^^^^^^^^^^^^^
    NameError: name 'sagetex_manifest' is not defined
**********************************************************************
2 items had failures:
   2 of   3 in sage.misc.package.installed_packages
   3 of   6 in sage.misc.package.package_manifest
    [56 tests, 5 failures, 11.24 s]

@kwankyu
Copy link
Collaborator

kwankyu commented Apr 1, 2023

With the latest beta, I get from ./sage -sh -c '(cd pkgs/sagemath-standard && tox -v -v -v -e sagepython-sagewheels-nopypi)'

sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/tests/cmdline.py
    [217 tests, 76.30 s]
----------------------------------------------------------------------
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/generators/families.py  # 3 doctests failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/graph_generators.py  # 1 doctest failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py  # 5 doctests failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py  # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 1788.2 seconds
    cpu time: 10438.1 seconds
    cumulative wall time: 13431.9 seconds
Features detected for doctesting: 4ti2,csdp,cvxopt,dvipng,gfan,graphviz,imagemagick,nauty,palp,pandoc,pdf2svg,pdftocairo,polytopes_db,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.libs.pari,sage.misc.cython,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.rings.real_mpfr,sage.symbolic,sagemath_doc_html,sphinx
pytest is not installed in the venv, skip checking tests that rely on it
sagepython-sagewheels-nopypi: 4880330 C exit 1 (1799.78 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> sage -t -p --all pid=15086 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 4880331 W _exit> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /var/folders/td/fw1q9ljs311ggyph77rs53_40000gn/T/pep517__exit-408penwy.json
.pkg-sagepython-sagewheels-nopypi: 4880334 I exit None (0.00 seconds) /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard> python /usr/local/Cellar/tox/4.4.6/libexec/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=873 [tox/execute/api.py:275]
.pkg-sagepython-sagewheels-nopypi: 4880422 D delete package /Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/.tox/.tmp/package/5/sagemath-standard-10.0b6.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
  sagepython-sagewheels-nopypi: FAIL code 1 (4880.31=setup[3076.63]+cmd[1.81,2.08,1799.78] seconds)
  evaluation failed :( (4880.42 seconds)

What does evaluation failed mean?

@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 1, 2023

What does evaluation failed mean?

This message comes from tox, in response to the error exit of the doctester.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 1, 2023

sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/generators/families.py # 3 doctests failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/graphs/graph_generators.py # 1 doctest failed

These two were fixed in #35250, I think.

sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/misc/package.py # 5 doctests failed
sage -t --random-seed=160851146271410412654781755279246610499 /Users/kwankyu/GitHub/sage-dev/src/sage/repl/ipython_extension.py # 1 doctest failed

I've fixed these two

#
# pkgs/sagemath-categories/.tox/sagepython/bin/python
# pkgs/sagemath-categories/.tox/sagepython-sagewheels-nopypi-norequirements/bin/sage
#
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove initial two spaces.

the Sage distribution (indicated by feature ``sage_spkg``), we should have
the installation record for this package. (We do not test for installation
records of Python packages. Our ``SAGE_VENV`` is not necessarily the
main Sage venv; it could be a user-created venv or a venv created by tox.)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add :: at the end.

@kwankyu
Copy link
Collaborator

kwankyu commented May 2, 2023

Otherwise, all works well.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 2, 2023

Thanks! I've made these changes.

@github-actions
Copy link

github-actions bot commented May 2, 2023

Documentation preview for this PR is ready! 🎉
Built with commit: 0c8bb55

Copy link
Collaborator

@kwankyu kwankyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. All works well for me.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 3, 2023

Thank you!

@vbraun vbraun merged commit 9de8e26 into sagemath:develop May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants