Update scipy 1.7.x; add pythran package (scipy prerequisite) #32105

mkoeppe · 2021-07-03T06:27:08Z

https://docs.scipy.org/doc/scipy/release.1.7.1.html

scipy has a new build requirement: pythran https://github.com/scipy/scipy/blob/master/pyproject.toml

[build-system]
requires = [
    "wheel",
    "setuptools",
    "Cython>=0.29.18",
    "pybind11>=2.4.3",
    "pythran>=0.9.11",

Depends on #32488

CC: @kiwifb @antonio-rojas @dimpase @kliem

Component: packages: standard

Author: Matthias Koeppe, Jonathan Kliem

Branch/Commit: 6f5b6f7

Reviewer: François Bissey, Dima Pasechnik, Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/32105

The text was updated successfully, but these errors were encountered:

slel · 2021-07-19T23:45:50Z

comment:2

NumPy 1.21.1 was released.

https://github.com/numpy/numpy/releases/tag/v1.21.1

The NumPy 1.21.1 is a maintenance release that fixes bugs
discovered after the 1.21.0 release and updates OpenBLAS
to v0.3.17 to deal with problems on arm64.

The Python versions supported for this release are 3.7-3.9.
The 1.21.x series is compatible with development Python 3.10.
Python 3.10 will be officially supported after it is released.

⚠️ There are unresolved problems compiling NumPy 1.20.0
with gcc-11.1.

Optimization level -O3 results in many incorrect
warnings when running the tests.

On some hardware NumPy will hang in an infinite loop.

mkoeppe · 2021-08-12T02:28:38Z

Branch: u/mkoeppe/numpy_1_21_x__scipy_1_7_x

mkoeppe · 2021-08-12T02:29:17Z

Commit: 461d65f

mkoeppe · 2021-08-12T02:29:17Z

New commits:

`39acb9d`	`build/pkgs/numpy: Update to 1.21.1`
`8cb5e08`	`build/pkgs/pythran: New`
`461d65f`	`build/pkgs/scipy: Update to 1.7.1`

sagetrac-git · 2021-08-14T23:39:03Z

Branch pushed to git repo; I updated commit sha1. New commits:

`b5d301d`	`build/pkgs/pythran/patches/1877.patch: New`

sagetrac-git · 2021-08-14T23:39:03Z

Changed commit from 461d65f to b5d301d

mkoeppe · 2021-08-15T03:26:21Z

Author: Matthias Koeppe

sagetrac-git · 2021-08-25T18:41:14Z

Changed commit from b5d301d to e183ac0

sagetrac-git · 2021-08-25T18:41:14Z

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

`bca7fb2`	`build/pkgs/numpy: Update to 1.21.1`
`565255e`	`build/pkgs/pythran: New`
`0580568`	`build/pkgs/scipy: Update to 1.7.1`
`5880de8`	`build/pkgs/pythran/patches/1877.patch: New`
`e183ac0`	`build/pkgs/numpy: Update to 1.21.2`

kiwifb · 2021-08-25T21:54:35Z

Reviewer: François Bissey

kiwifb · 2021-08-25T21:54:35Z

comment:14

LGTM

vbraun · 2021-08-29T12:42:51Z

comment:15

[scipy-1.7.1]       FOUND:
[scipy-1.7.1]         libraries = ['openblas', 'openblas']
[scipy-1.7.1]         library_dirs = ['/usr/lib64']
[scipy-1.7.1]         language = f77
[scipy-1.7.1]         define_macros = [('NO_ATLAS_INFO', 1)]
[scipy-1.7.1]         include_dirs = ['/usr/local/include', '/home/release/Sage/local/include']
[scipy-1.7.1] 
[scipy-1.7.1]     Traceback (most recent call last):
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 280, in <module>
[scipy-1.7.1]         main()
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 263, in main
[scipy-1.7.1]         json_out['return_val'] = hook(**hook_input['kwargs'])
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 133, in prepare_metadata_for_build_wheel
[scipy-1.7.1]         return hook(metadata_directory, config_settings)
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/setuptools/build_meta.py", line 166, in prepare_metadata_for_build_wheel
[scipy-1.7.1]         self.run_setup()
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/setuptools/build_meta.py", line 258, in run_setup
[scipy-1.7.1]         super(_BuildMetaLegacyBackend,
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/setuptools/build_meta.py", line 150, in run_setup
[scipy-1.7.1]         exec(compile(code, __file__, 'exec'), locals())
[scipy-1.7.1]       File "setup.py", line 629, in <module>
[scipy-1.7.1]         setup_package()
[scipy-1.7.1]       File "setup.py", line 625, in setup_package
[scipy-1.7.1]         setup(**metadata)
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/numpy/distutils/core.py", line 135, in setup
[scipy-1.7.1]         config = configuration()
[scipy-1.7.1]       File "setup.py", line 536, in configuration
[scipy-1.7.1]         config.add_subpackage('scipy')
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/numpy/distutils/misc_util.py", line 1016, in add_subpackage
[scipy-1.7.1]         config_list = self.get_subpackage(subpackage_name, subpackage_path,
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/numpy/distutils/misc_util.py", line 982, in get_subpackage
[scipy-1.7.1]         config = self._get_configuration_from_setup_py(
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/numpy/distutils/misc_util.py", line 924, in _get_configuration_from_setup_py
[scipy-1.7.1]         config = setup_module.configuration(*args)
[scipy-1.7.1]       File "scipy/setup.py", line 12, in configuration
[scipy-1.7.1]         config.add_subpackage('interpolate')
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/numpy/distutils/misc_util.py", line 1016, in add_subpackage
[scipy-1.7.1]         config_list = self.get_subpackage(subpackage_name, subpackage_path,
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/numpy/distutils/misc_util.py", line 982, in get_subpackage
[scipy-1.7.1]         config = self._get_configuration_from_setup_py(
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/numpy/distutils/misc_util.py", line 924, in _get_configuration_from_setup_py
[scipy-1.7.1]         config = setup_module.configuration(*args)
[scipy-1.7.1]       File "scipy/interpolate/setup.py", line 58, in configuration
[scipy-1.7.1]         from pythran.dist import PythranExtension
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pythran/__init__.py", line 41, in <module>
[scipy-1.7.1]         from pythran.toolchain import (generate_cxx, compile_cxxfile, compile_cxxcode,
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pythran/toolchain.py", line 6, in <module>
[scipy-1.7.1]         from pythran.backend import Cxx, Python
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pythran/backend.py", line 7, in <module>
[scipy-1.7.1]         from pythran.analyses import LocalNodeDeclarations, GlobalDeclarations, Scope
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pythran/analyses/__init__.py", line 12, in <module>
[scipy-1.7.1]         from .aliases import Aliases, StrictAliases
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pythran/analyses/aliases.py", line 3, in <module>
[scipy-1.7.1]         from pythran.analyses.global_declarations import GlobalDeclarations
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pythran/analyses/global_declarations.py", line 3, in <module>
[scipy-1.7.1]         from pythran.passmanager import ModuleAnalysis
[scipy-1.7.1]       File "/home/release/Sage/local/lib64/python3.9/site-packages/pythran/passmanager.py", line 14, in <module>
[scipy-1.7.1]         import gast as ast
[scipy-1.7.1]     ModuleNotFoundError: No module named 'gast'
[scipy-1.7.1]     Preparing wheel metadata: finished with status 'error'
[scipy-1.7.1] WARNING: Discarding file:///home/release/Sage/local/var/tmp/sage/build/scipy-1.7.1/src. Command errored out with exit status 1: /home/release/Sage/local/bin/python3 /home/release/Sage/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpne_r946f Check the logs for full command output.
[scipy-1.7.1] ERROR: Command errored out with exit status 1: /home/release/Sage/local/bin/python3 /home/release/Sage/local/lib64/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpne_r946f Check the logs for full command output.

mkoeppe · 2021-08-29T19:29:27Z

comment:16

Indeed, from https://github.com/serge-sans-paille/pythran/blob/0.9.12.post1/requirements.txt:

ply>=3.4
gast~=0.5.0
numpy
beniget~=0.4.0

sagetrac-git · 2021-11-15T04:03:20Z

Changed commit from 9d6b0db to 21463e2

sagetrac-git · 2021-11-15T04:03:20Z

Branch pushed to git repo; I updated commit sha1. New commits:

`7072bd6`	`build/pkgs/scipy: Update to 1.7.2`
`9e167c8`	`build/pkgs/pybind11: Update to 2.8.1`
`21463e2`	`Merge tag '9.5.beta6' into t/32105/numpy_1_21_x__scipy_1_7_x`

mkoeppe · 2021-11-15T04:03:40Z

comment:33

In the meantime new versions have come out

sagetrac-git · 2021-11-15T04:45:44Z

Changed commit from 21463e2 to 581f874

sagetrac-git · 2021-11-15T04:45:44Z

Branch pushed to git repo; I updated commit sha1. New commits:

`581f874`	`build/pkgs/pybind11/patches/3270.patch: Remove`

kliem · 2021-11-15T07:25:56Z

comment:35

Replying to @vbraun:

After a make distclean (and not just an incremental build, and not just make clean) test fail with

sage -t --warn-long 59.1 --random-seed=32710823712882651180286903013949123952 src/sage/ext/memory_allocator.pxd  # 2 doctests failed
sage -t --warn-long 59.1 --random-seed=32710823712882651180286903013949123952 src/sage/ext/memory_allocator.pyx  # 1 doctest failed

I can reproduce those failures and they are trivial:

sage -t --warn-long 59.1 --random-seed=32710823712882651180286903013949123952 src/sage/ext/memory_allocator.pxd
**********************************************************************
File "src/sage/ext/memory_allocator.pxd", line 88, in sage.ext.memory_allocator.MemoryAllocator.?
Failed example:
    cython('''
    from sage.ext.memory_allocator cimport MemoryAllocator
    cdef MemoryAllocator mem = MemoryAllocator()
    cdef void* ptr
    for i in range(12):
        ptr = mem.aligned_calloc(2**i, i, 2**i)
        assert <size_t> ptr == (<size_t> ptr) & ~(2**i-1)
    ''')
Expected nothing
Got:
    doctest:warning
      File "/srv/public/kliem/sage/src/bin/sage-runtests", line 151, in <module>
        err = DC.run()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1257, in run
        self.run_doctests()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 959, in run_doctests
        self.dispatcher.dispatch()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1996, in dispatch
        self.parallel_dispatch()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1891, in parallel_dispatch
        w.start()  # This might take some time
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2163, in start
        super(DocTestWorker, self).start()
      File "/usr/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/usr/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/usr/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2135, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2465, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2517, in _run
        result = runner.run(test)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 861, in run
        return self._run(test, compileflags, out)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.ext.memory_allocator.MemoryAllocator.?[0]>", line 8, in <module>
        ''')
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 661, in cython_compile
        return cython_import_all(tmpfile, get_globals(), **kwds)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 551, in cython_import_all
        m = cython_import(filename, **kwds)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 531, in cython_import
        return builtins.__import__(name)
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 1050, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/superseded.py", line 99, in deprecation
        warning(trac_number, message, DeprecationWarning, stacklevel)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/superseded.py", line 180, in warning
        warn(message, warning_class, stacklevel)
      File "/usr/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: this class is deprecated; use the class from the python package `memory_allocator`
    See https://trac.sagemath.org/31591 for details.
**********************************************************************
File "src/sage/ext/memory_allocator.pxd", line 119, in sage.ext.memory_allocator.MemoryAllocator.?
Failed example:
    cython('''
    from sage.ext.memory_allocator cimport MemoryAllocator
    cdef MemoryAllocator mem = MemoryAllocator()
    cdef void* ptr
    for i in range(12):
        ptr = mem.aligned_allocarray(2**i, i, 2**i)
        assert <size_t> ptr == (<size_t> ptr) & ~(2**i-1)
    ''')
Expected nothing
Got:
    doctest:warning
      File "/srv/public/kliem/sage/src/bin/sage-runtests", line 151, in <module>
        err = DC.run()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1257, in run
        self.run_doctests()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 959, in run_doctests
        self.dispatcher.dispatch()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1996, in dispatch
        self.parallel_dispatch()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1891, in parallel_dispatch
        w.start()  # This might take some time
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2163, in start
        super(DocTestWorker, self).start()
      File "/usr/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/usr/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/usr/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2135, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2465, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2517, in _run
        result = runner.run(test)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 861, in run
        return self._run(test, compileflags, out)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.ext.memory_allocator.MemoryAllocator.?[0]>", line 8, in <module>
        ''')
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 661, in cython_compile
        return cython_import_all(tmpfile, get_globals(), **kwds)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 551, in cython_import_all
        m = cython_import(filename, **kwds)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 531, in cython_import
        return builtins.__import__(name)
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 1050, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/superseded.py", line 99, in deprecation
        warning(trac_number, message, DeprecationWarning, stacklevel)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/superseded.py", line 180, in warning
        warn(message, warning_class, stacklevel)
      File "/usr/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: this class is deprecated; use the class from the python package `memory_allocator`
    See https://trac.sagemath.org/31591 for details.
**********************************************************************
1 item had failures:
   2 of   6 in sage.ext.memory_allocator.MemoryAllocator.?
    [3 tests, 2 failures, 1.77 s]
----------------------------------------------------------------------
sage -t --warn-long 59.1 --random-seed=32710823712882651180286903013949123952 src/sage/ext/memory_allocator.pxd  # 2 doctests failed

sage -t --warn-long 59.1 --random-seed=32710823712882651180286903013949123952 src/sage/ext/memory_allocator.pyx
**********************************************************************
File "src/sage/ext/memory_allocator.pyx", line 34, in sage.ext.memory_allocator.MemoryAllocator.__cinit__
Failed example:
    cython(
    '''
    from sage.ext.memory_allocator cimport MemoryAllocator
    cdef MemoryAllocator mem = MemoryAllocator.__new__(MemoryAllocator)
    mem.malloc(10000)
    print(mem.n)
    print(mem.size)
    ''')
Expected:
    1
    16
Got:
    doctest:warning
      File "/srv/public/kliem/sage/src/bin/sage-runtests", line 151, in <module>
        err = DC.run()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 1257, in run
        self.run_doctests()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 959, in run_doctests
        self.dispatcher.dispatch()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1996, in dispatch
        self.parallel_dispatch()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1891, in parallel_dispatch
        w.start()  # This might take some time
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2163, in start
        super(DocTestWorker, self).start()
      File "/usr/lib/python3.7/multiprocessing/process.py", line 112, in start
        self._popen = self._Popen(self)
      File "/usr/lib/python3.7/multiprocessing/context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "/usr/lib/python3.7/multiprocessing/context.py", line 277, in _Popen
        return Popen(process_obj)
      File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 20, in __init__
        self._launch(process_obj)
      File "/usr/lib/python3.7/multiprocessing/popen_fork.py", line 74, in _launch
        code = process_obj._bootstrap()
      File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
        self.run()
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2135, in run
        task(self.options, self.outtmpfile, msgpipe, self.result_queue)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2465, in __call__
        doctests, extras = self._run(runner, options, results)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 2517, in _run
        result = runner.run(test)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 861, in run
        return self._run(test, compileflags, out)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 694, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.ext.memory_allocator.MemoryAllocator.__cinit__[0]>", line 8, in <module>
        ''')
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 661, in cython_compile
        return cython_import_all(tmpfile, get_globals(), **kwds)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 551, in cython_import_all
        m = cython_import(filename, **kwds)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/cython.py", line 531, in cython_import
        return builtins.__import__(name)
      File "<frozen importlib._bootstrap>", line 983, in _find_and_load
      File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 1050, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/superseded.py", line 99, in deprecation
        warning(trac_number, message, DeprecationWarning, stacklevel)
      File "/srv/public/kliem/sage/local/lib/python3.7/site-packages/sage/misc/superseded.py", line 180, in warning
        warn(message, warning_class, stacklevel)
      File "/usr/lib/python3.7/warnings.py", line 110, in _showwarnmsg
        msg.file, msg.line)
    :
    DeprecationWarning: this class is deprecated; use the class from the python package `memory_allocator`
    See https://trac.sagemath.org/31591 for details.
    1
    16
**********************************************************************
1 item had failures:
   1 of   2 in sage.ext.memory_allocator.MemoryAllocator.__cinit__
    [8 tests, 1 failure, 1.76 s]
----------------------------------------------------------------------
sage -t --warn-long 59.1 --random-seed=32710823712882651180286903013949123952 src/sage/ext/memory_allocator.pyx  # 1 doctest failed

Doesn't require a distclean in my case. I have no idea why this is happening now and how to determine when deprecation messages are raised and when not.

For some reason it changed now.

kliem · 2021-11-15T07:47:47Z

comment:36

This has nothing to do with the current ticket. Something change in the new develop:

On debian buster I get (not on ubuntu bionic):

sage -t --random-seed=217031171464721311270444360210868711734 src/sage/ext/memory_allocator.pxd  # 2 doctests failed
sage -t --random-seed=217031171464721311270444360210868711734 src/sage/ext/memory_allocator.pyx  # 1 doctest failed

kliem · 2021-11-15T10:51:12Z

comment:37

Sorry about the mess. It has something to do with the ticket. When running on make distclean on develop, the error goes away again. I'm not sure exactly what packages caused this.

kliem · 2021-11-15T14:02:30Z

Changed commit from 581f874 to 6f5b6f7

kliem · 2021-11-15T14:02:30Z

Changed branch from u/mkoeppe/numpy_1_21_x__scipy_1_7_x to u/gh-kliem/numpy_1_21_x__scipy_1_7_x

kliem · 2021-11-15T14:02:30Z

comment:38

Don't ask me why those deprecation warnings are somewhat random. They are given in each case, but sometimes they are given again and sometimes not and for some strange reason the current branch changed that.

I pushed a patch that works on top of develop and on this ticket.

New commits:

`6f5b6f7`	`work around unstable deprecation warnings in ext/memory_allocator`

kliem · 2021-11-15T14:02:30Z

Changed author from Matthias Koeppe to Matthias Koeppe, Jonathan Kliem

mkoeppe · 2021-11-15T17:32:39Z

comment:39

Try if using deprecation_cython instead of deprecation helps

kliem · 2021-11-15T19:09:32Z

comment:40

Not really. It only changes the stacklevel. Then there are two failures each in the files, one of which is also on develop. So it remains that we must have a workaround for the other 3 cases.

mkoeppe · 2021-11-16T18:27:39Z

comment:41

LGTM, thanks.

mkoeppe · 2021-11-16T18:27:39Z

Changed reviewer from François Bissey, Dima Pasechnik to François Bissey, Dima Pasechnik, Matthias Koeppe

kiwifb · 2021-11-16T18:56:45Z

comment:42

I missed that last commit. I guess I am quite busy elsewhere. I would have treated all those like the second block

sage: foo()  # random  # might raise deprecation warning
sage: foo()

I have noticed this very failure in sage-on-gentoo for the last couple of weeks wondering where it was coming from. I was tempted to add the deprecation warning to the output when I noticed that some earlier tests already had it.
https://github.com/sagemath/sage-prod/blob/develop/src/sage/ext/memory_allocator.pxd#L67
https://github.com/sagemath/sage-prod/blob/develop/src/sage/ext/memory_allocator.pyx#L27

So, it followed the pattern of one warning per session and somehow we are not having one session anymore.

I think the current branch will work because this is the "new stable", rather than being unstable. If I am wrong and it is unstable, we can expect failures on some systems because a warning was displayed that wasn't expected.

slel · 2021-11-24T22:01:23Z

comment:43

Bugfix release SciPy 1.7.3 was just released,
with fixes for Apple M1 processors.

https://mail.python.org/archives/list/python-announce-list@python.org/thread/GDFAHAADIZEGQS6ZK4ITROE3GUJBTABJ/

kiwifb · 2021-11-24T22:04:48Z

comment:44

Current branch is in Volker's merging branch, so it will need to be in a follow up unless Volker finds a reason to reject it. From my perspective in sage-on-gentoo the current branch is all good.

vbraun · 2021-12-05T11:15:37Z

Changed branch from u/gh-kliem/numpy_1_21_x__scipy_1_7_x to 6f5b6f7

mkoeppe added this to the sage-9.4 milestone Jul 3, 2021

mkoeppe added c: packages: standard labels Jul 3, 2021

mkoeppe modified the milestones: sage-9.4, sage-9.5 Jul 19, 2021

This comment has been minimized.

Sign in to view

mkoeppe changed the title ~~numpy 1.21.x, scipy 1.7.x~~ Update numpy 1.21.x, scipy 1.7.x; add pythran package (scipy prerequisite) Aug 12, 2021

This comment has been minimized.

Sign in to view

mkoeppe added the s: needs review label Aug 25, 2021

kiwifb added s: positive review and removed s: needs review labels Aug 25, 2021

vbraun added s: needs work and removed s: positive review labels Aug 29, 2021

kliem added s: positive review and removed s: needs work labels Nov 15, 2021

kliem added s: needs work and removed s: positive review labels Nov 15, 2021

kliem added s: needs review and removed s: needs work labels Nov 15, 2021

mkoeppe added s: positive review and removed s: needs review labels Nov 16, 2021

vbraun removed the s: positive review label Dec 5, 2021

vbraun closed this as completed in 6101ac6 Dec 5, 2021

Update scipy 1.7.x; add pythran package (scipy prerequisite) #32105

Update scipy 1.7.x; add pythran package (scipy prerequisite) #32105

Comments

mkoeppe commented Jul 3, 2021

slel commented Jul 19, 2021

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

mkoeppe commented Aug 12, 2021

mkoeppe commented Aug 12, 2021

mkoeppe commented Aug 12, 2021

This comment has been minimized.

sagetrac-git mannequin commented Aug 14, 2021

sagetrac-git mannequin commented Aug 14, 2021

mkoeppe commented Aug 15, 2021

sagetrac-git mannequin commented Aug 25, 2021

sagetrac-git mannequin commented Aug 25, 2021

This comment has been minimized.

kiwifb commented Aug 25, 2021

kiwifb commented Aug 25, 2021

vbraun commented Aug 29, 2021

mkoeppe commented Aug 29, 2021

sagetrac-git mannequin commented Nov 15, 2021

sagetrac-git mannequin commented Nov 15, 2021

mkoeppe commented Nov 15, 2021

sagetrac-git mannequin commented Nov 15, 2021

sagetrac-git mannequin commented Nov 15, 2021

kliem commented Nov 15, 2021

kliem commented Nov 15, 2021

kliem commented Nov 15, 2021

kliem commented Nov 15, 2021

kliem commented Nov 15, 2021

kliem commented Nov 15, 2021

kliem commented Nov 15, 2021

mkoeppe commented Nov 15, 2021

kliem commented Nov 15, 2021

mkoeppe commented Nov 16, 2021

mkoeppe commented Nov 16, 2021

kiwifb commented Nov 16, 2021

slel commented Nov 24, 2021

kiwifb commented Nov 24, 2021

vbraun commented Dec 5, 2021