Fix introspection with ? when doc source is not available

[saraedum]

Currently, introspection does not work in conda and Arch Linux (if the sagemath-doc package is not installed):

sage: ZZ?
…
/usr/lib/python2.7/site-packages/sage/misc/sagedoc.pyc in process_extlinks(s, embedded)
    497     oldpath = sys.path
    498     sys.path = [os.path.join(SAGE_DOC_SRC, 'common')] + oldpath
--> 499     from conf import extlinks
    500     sys.path = oldpath
    501     for key in extlinks:

ImportError: cannot import name extlinks

The problem is that src/doc/common/conf.py is not in the place that Sage expects.

I don't think that we should require the documentation configuration to be installed for Sage to work.

A docker image to play with this is being built here: https://gitlab.com/sagemath/dev/sage/pipelines/25262253

Depends on #25843

CC: @antonio-rojas @jdemeyer @slel

Component: documentation

Keywords: conda, Arch Linux, introspection

Work Issues: is the patchbot happy?

Author: Julian Rüth, Isuru Fernando

Branch/Commit: a401a17

Reviewer: Julian Rüth, Isuru Fernando, François Bissey

Issue created by migration from https://trac.sagemath.org/ticket/25786

[saraedum]

Branch: u/saraedum/25786

[jdemeyer]

comment:2

Is this needs review?

---

New commits:

13958db

Move extlinks into the main Sage code

[jdemeyer]

Commit: 13958db

[saraedum]

comment:3

I had a comment in #24655 that src/doc/en/introspect is required as well…let's see if that's still the case.

[saraedum]

comment:4

jdemeyer: No, I have not tested this yet. But feel free to give feedback if you know something about this :)

[saraedum]

comment:5

See #25787 for a followup regarding the docker images.

[jdemeyer]

comment:6

In any case, I agree with the idea behind the patch.

[jdemeyer]

Author: Julian Rüth

[jdemeyer]

Reviewer: Jeroen Demeyer

[saraedum]

Work Issues: is the patchbot happy?

[saraedum]

comment:9

So, if the patchbot and my tests are Ok, then this is a positive review?

[jdemeyer]

comment:11

Replying to @saraedum:

So, if the patchbot and my tests are Ok, then this is a positive review?

Yes

[antonio-rojas]

comment:13

This still doesn't allow introspection on Arch without having the doc sources installed, due to

    # Sphinx constructor: Sphinx(srcdir, confdir, outdir, doctreedir,
    # buildername, confoverrides, status, warning, freshenv).
    confdir = os.path.join(SAGE_DOC_SRC, 'en', 'introspect')

    open(os.path.join(srcdir, 'docutils.conf'), 'w').write(r"""
[parsers]
smart_quotes = no
""")
    doctreedir = os.path.join(srcdir, 'doctrees')
    confoverrides = {'html_context': {}, 'master_doc': 'docstring'}

    import sys
    old_sys_path = list(sys.path)  # Sphinx modifies sys.path
    sphinx_app = Sphinx(srcdir, confdir, srcdir, doctreedir, format,
                        confoverrides, None, None, True)
    sphinx_app.build(None, [rst_name])
    sys.path = old_sys_path

in misc/sphinxify.py

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

0e5b29a

Fix sphinxify for introspection

[sagetrac-git]

Changed commit from 13958db to 0e5b29a

[saraedum]

comment:15

arojas: Right, now it works for me. But probably the from common.conf import * is there for a reason and I should not simply delete it.

[jdemeyer]

comment:16

I really prefer one ticket, one issue.

[jdemeyer]

Changed work issues from is the patchbot happy? to split ticket

[saraedum]

Changed work issues from split ticket to is the patchbot happy?

[saraedum]

comment:18

Replying to @jdemeyer:

I really prefer one ticket, one issue.

I believe that this is one issue: "Fix introspection with ? in conda and ArchLinux". And even the solution can be seen as one thing: "Make Sage runtime not depend on documentation configuration."

[saraedum]

comment:19

The patchbots report weird errors that seem unrelated to the changes here…let me try this out locally.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

5c63a80

Fix mention of common/conf.py

[sagetrac-git]

Changed commit from 863d433 to 5c63a80

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

edd3ab4

Replace conf.conf with conf

[sagetrac-git]

Changed commit from 5c63a80 to edd3ab4

[saraedum]

Reviewer: Julian Rüth, Isuru Fernando

[saraedum]

comment:49

@isuruf: Could you have a look at my latest changes? (That I did not test yet, because I have not built recent Sage…)

[saraedum]

Changed author from Julian Rüth to Julian Rüth, Isuru Fernando

[saraedum]

Changed work issues from failing doctests to is the patchbot happy?

[isuruf]

comment:50

Looks good to me. Thanks

[kiwifb]

comment:51

There is strange stuff about lcalc in the branch. May be some rebasing is needed?

[sagetrac-git]

Changed commit from edd3ab4 to 2f528be

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

2f528be

Revert lcalc changes

[saraedum]

comment:53

Thanks fbissey. That should not have been here.

[isuruf]

New commits:

5eec95a

Fix some pyflakes issues

[isuruf]

Changed branch from u/saraedum/25786 to u/isuruf/25786

[isuruf]

Changed commit from 2f528be to 5eec95a

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

a401a17

More pyflakes fixes

[sagetrac-git]

Changed commit from 5eec95a to a401a17

[kiwifb]

comment:56

I think the patchbot is happy with it (the last run had a failure that should be unrelated). Anything to add or is it completely ready for review?

[isuruf]

comment:57

Anything to add or is it completely ready for review?

Nothing to add. Ready for review

[kiwifb]

Changed reviewer from Julian Rüth, Isuru Fernando to Julian Rüth, Isuru Fernando, François Bissey

[kiwifb]

comment:58

Looks good to me :)

[vbraun]

Changed branch from u/isuruf/25786 to a401a17