Meta-ticket: deprecate useless or misleading functions in the global namespace

[videlec]

Globals starting with a-f

Cryptic

• addgp
• benchmark Deprecate the import of some development-related names from the global namespace #34259

Too specific

• absolute_igusa_invariants_kohel deprecate a few invariants commands in global namespace #28064
• absolute_igusa_invariants_wamelen deprecate a few invariants commands in global namespace #28064
• all_max_clique (should be a method of a graph. Done in deprecate import of all_max_clique, max_clique and clique_number in global namespace #26200)
• backtrack_all (for sudoku) deprecate a few commands in global namespace #27066
• berlekamp_massey deprecate a few commands in global namespace #27066
• bernoulli_mod_p
• bernoulli_mod_p_single
• branching_rule (done in some import cleanup and lazification in algebras and root systems #25791)
• branching_rule_from_plethysm (done in some import cleanup and lazification in algebras and root systems #25791)
• bsgs (baby step giant step, done in Meta-ticket: deprecate useless or misleading functions in the global namespace #25383)
• build_alphabet
• buzzard_tpslopes deprecate a few commands in global namespace #27066
• clebsch_gordan
• clebsch_invariants deprecate a few invariants commands in global namespace #28064
• clique_number (done in deprecate import of all_max_clique, max_clique and clique_number in global namespace #26200)
• cm_j_invariants
• cm_j_invariants_and_orders
• cm_orders
• coercion_traceback
• coincidence_discriminant
• coincidence_index
• continuant
• convergents -> continued_fraction(x).convergents() deprecate a few commands in global namespace #27066
• cremona_curves
• cremona_optimal_curves
• cunningham_prime_factors
• cython_create_local_so Minor cleanup in sage.misc.cython #24722
• delta_lseries
• delta_qexp
• designs_from_XML deprecate a few commands in global namespace #27066
• designs_from_XML_url deprecate a few commands in global namespace #27066
• dimension_cusp_forms
• dimension_eis
• dimension_modular_forms
• dimension_new_cusp_forms
• dimension_supersingular_module
• direct_product_permgroups
• discrete_log_generic, discrete_log_lambda, discrete_log_rho
• eisenstein_series_lseries
• eisenstein_series_qexp
• enum_affine_finite_field
• enum_affine_rational_field
• enumerate_totallyreal_fields_all
• enumerate_totallyreal_fields_prim
• enumerate_totallyreal_fields_rel
• eta_poly_relations cleanup the imports of Eta products in global namespace #26196
• eulers_method, eulers_method_2x2 and eulers_method_2x2_plot are accessible via desolvers.
• exists_conway_polynomial
• explain_pickle
• expnums ?
• extend_to_primitive
• external_ray
• find_a_ternary_qf_by_level_disc
• find_all_ternary_qf_by_level_disc
• firing_graph
• four_squares
• frequency_distribution
• fundamental_discriminant
• FiniteEnumeratedSet see Deprecate FiniteEnumeratedSet in the global namespace, add method is_subset to Sets.ParentMethods #34398
• PowerSeries, PuiseuxSeries, LaurentSeries see Deprecate the import of some element classes in sage/ring/all.py #33602

Shortcut f(x) -> x.f():

• additive_order
• base_ring
• base_field
• basis
• canonical_coercion -> coercion_model.canonical_coercion
• matrices (block_matrix, block_diagonal_matrix, column_matrix, companion_matrix, diagonal_matrix, ...) can be accessed with matrix.block, matrix.block_diagonal, etc
• decomposition
• denominator, numerator
• desolve, desolve_laplace, desolve_mintides, etc -> desolvers.XXX

WTF

• absolute_import (nothing to do here)
• abstract_method (nothing to do in the global namespace)
• addition_names (a useless tuple)
• alarm (nothing to do here)
• attrcall (nothing to do here)
• banner (no reason to have it in the global namespace) Deprecate the import of some development-related names from the global namespace #34259
• cancel_alarm
• class_graph
• db, db_save (what is the Sage database?)
• degree_lowest_rational_function
• differences
• eratosthenes

Functions mainly useful for developers

• set_edit_template, runsnake, Profiler, trace Deprecate the import of some development-related names from the global namespace #34259
• install_scripts — the function was deprecated in Deprecate sage.misc.dist.install_scripts #30207, its import into the global namespace in Deprecate the import of some development-related names from the global namespace #34259
• imports from sage.misc.package Deprecate the import of some development-related names from the global namespace #34259
• logstr, benchmark (probably belong in the WTF category) Deprecate the import of some development-related names from the global namespace #34259

Globals starting with g-z

• get_display_manager (Fix global import of sage.repl.rich_output.display_manager.get_display_manager #33016)
• random_DAG (deprecated in deprecate random_DAG #30479, removed in Remove random_DAG from global namespace #33604)

See also

• Deprecate FiniteDimensionalAlgebrasWithBasis and friends from being in the global namespace #15741

CC: @sagetrac-tmonteil @tscrim @DavidAyotte

Component: misc

Author: Vincent Delecroix

Issue created by migration from https://trac.sagemath.org/ticket/25383

[jdemeyer]

comment:2

I'm not entirely sure that banner is unused. I can see that being actually used somewhere as from sage.all import banner. Although deprecating it cannot hurt.

The rest should be no problem.

[kcrisman]

comment:6

Sage did actually have some proto-database functionality but that never really went very far.

I'm not sure that desolve should move; similarly with the various types of matrices. In a lot of contexts you really want the absolutely easiest dumb-sounding command, and extra object-oriented stuff can be confusing there.

That said, most of these probably indeed are a bit specific for top-level. Seems like the kind of thing to put on sage-devel and allow people to ask for things to be separately discussed if necessary: see e.g. consent agendas.

[jdemeyer]

comment:7

What's wrong with alarm and column_matrix? (just picking two which look genuinely useful)

[videlec]

comment:8

Replying to @jdemeyer:

What's wrong with alarm and column_matrix? (just picking two which look genuinely useful)

This is not a strong opinion. I just made up a list by quickly looking at dir(). Concerning the two names you mention:

• what is alarm useful for for a Sage user that does not know how to import it?
• column_matrix is accessible via matrix.column and I believe it is easier (if properly advertised) to use the second version

Removing from the global namespace is different from removing.

[videlec]

comment:9

Replying to @kcrisman:

Thanks for your comment! I am just trying to make clear that we are at a point where the global namespace is so confusing that it is impossible to find anything relevant. What should be kept and how it should be organized is definitely open to suggestion. But I am not sure that discussing one by one the entries on this ticket is the best strategy.

Sage did actually have some proto-database functionality but that never really went very far.

I'm not sure that desolve should move; similarly with the various types of matrices. In a lot of contexts you really want the absolutely easiest dumb-sounding command, and extra object-oriented stuff can be confusing there.

That said, most of these probably indeed are a bit specific for top-level. Seems like the kind of thing to put on sage-devel and allow people to ask for things to be separately discussed if necessary: see e.g. consent agendas.

I was thinking of something like this. It is very likely that commands will be moved by small amount in separate tickets.

[nbruin]

comment:10

Replying to @videlec:

Thanks for your comment! I am just trying to make clear that we are at a point where the global namespace is so confusing that it is impossible to find anything relevant. What should be kept and how it should be organized is definitely open to suggestion. But I am not sure that discussing one by one the entries on this ticket is the best strategy.

In what way is the global namespace "confusing"? Is it hard for python to find the bindings? (no because python dictionaries should basically have access time independent of the size of the dictionary)

Is the problem that browsing the global namespace gives too long a list? It may just be the case that browsing the global namespace is not such a great strategy to learn about a computer algebra system.

I think some culling can definitely be done, but the change has a cost, so I think it should be justified. What benefit do we get from culling?

[videlec]

comment:11

Replying to @nbruin:

Replying to @videlec:

Thanks for your comment! I am just trying to make clear that we are at a point where the global namespace is so confusing that it is impossible to find anything relevant. What should be kept and how it should be organized is definitely open to suggestion. But I am not sure that discussing one by one the entries on this ticket is the best strategy.

In what way is the global namespace "confusing"? Is it hard for python to find the bindings? (no because python dictionaries should basically have access time independent of the size of the dictionary)

Indeed, nobody cares about this.

Is the problem that browsing the global namespace gives too long a list? It may just be the case that browsing the global namespace is not such a great strategy to learn about a computer algebra system.

The length is not so important. The problem is that you have access to so many specialized and/or useless functions. If the functions/classes were better organized by modules (for example graphs.<TAB> is cool) it would be simpler to find functionalities. The only reasonable strategies I know for searching for a functionality in Sage is the tab-completion and grepping the source. While the latter technique is powerful it is not ideal for a newcomer. The Sage documentation could have been an alternative but it is very bad at the moment. The only references I know where things are organized are the books written about Sage (e.g. "Calcul Mathématique avec Sage" by Zimmermann et al.)

So it is not about learning but about searching. I am focusing not on the newcomers, but the next stage (learners?). The problem is "I want to do that, how do I do"?

I think some culling can definitely be done, but the change has a cost, so I think it should be justified. What benefit do we get from culling?

Cost for the developer? Cost for the reviewer? Cost for the one person who uses an ultra specialized feature that is imported by default?

[jhpalmieri]

comment:12

We also advertise tab-completion in the tutorial (and probably other places) as a way to learn about the system, and so perhaps the length of the list may actually be an issue.

[dcoudert]

comment:13

I agree that methods like:

• random_DAG, firing_graph, parallel_firing_graph defined in sandpile/ could be moved to digraphs.<TAB>
• all_max_clique, clique_number, max_clique should be methods of Graph only.

Questions:

• how to know where a method is defined ? For instance, when typing random_DAG? I don't get the source file of the method but of LazyImport.
• Is there an easy way to know where a method is used ? I'm usually relying on grep which might not be the smartest way.

[videlec]

comment:14

This works often

sage: import_statements(random_DAG)
from sage.sandpiles.sandpile import random_DAG

and this as well

sage: import_statements("random_DAG")
from sage.sandpiles.sandpile import random_DAG

[dcoudert]

comment:15

Thanks ! I'm learning every day ;)

[nbruin]

comment:16

Replying to @videlec:

The length is not so important. The problem is that you have access to so many specialized and/or useless functions. If the functions/classes were better organized by modules (for example graphs.<TAB> is cool) it would be simpler to find functionalities.

I've tried that occasionally (not particularly on graphs) and have been underwhelmed with the results. Namespaces in python are generally too cluttered to be useful in that way (plus, the standard IPython interfaces give the completion list in a very small window, without much additional information). I have generally resorted to using "google" to get a hold of some relevant documentation. Once I have an object that is probably related to the functionality I'm looking for, I'd generally use a bit of tab completion on that (and wade through all the boiler-plate infrastructure that's cluttering those results)

If we'd have a command that would print a list such that you can hover over the results to see the associated docstring, I think it would already be a lot more useful. Especially if clicking would then redirect to the HTML documentation.

The only reasonable strategies I know for searching for a functionality in Sage is the tab-completion and grepping the source. While the latter technique is powerful it is not ideal for a newcomer. The Sage documentation could have been an alternative but it is very bad at the moment. The only references I know where things are organized are the books written about Sage (e.g. "Calcul Mathématique avec Sage" by Zimmermann et al.)

Indeed, and for "cost" about changing namespace contents, I think such books suffer particularly. Any books/ articles / documentation / scripts that depend on the contents of certain namespaces are affected by changes. Changing contents of namespaces causes breakage and therefore bears a significant cost.

So it is not about learning but about searching. I am focusing not on the newcomers, but the next stage (learners?). The problem is "I want to do that, how do I do"?

Cost for the developer? Cost for the reviewer? Cost for the one person who uses an ultra specialized feature that is imported by default?

As above: cost in the form of all existing usage/articles breaking. The benefits of a change must really outweigh such costs. As I describe above, I haven't found "tab completion" such an effective way for browsing functionality either, so the benefits don't seem so large at this point.

If we'd have a better browser for module contents, it may be an easier argument to make. The standard nowadays is really "do we provide a search interface that is considerably better than google search?". Using google is a universal strategy, so unless a specific search strategy for finding sage functionality is considerably better, people are better off sticking to using google.
I'm not so convinced that the sage documentation browsing (via tab or via HTML indices) gets considerably better results than google, so in reality I don't think changes at this point would lead to significant improvements anyway (search_src definitely has advantages if you have a sufficiently specific term, but indeed might not be so suitable for anybody below "expert" level)

Having better documentation and a better way of searching it would definitely be great. I'm not quite sure that culling the global namespace quite gets us there.

(incidentally, I've been quite happy finding explain_pickle right there in the top-level namespace. I have never been disappointed by finding that something was there in the top-level namespace. Having a bloated top-level does go against a general sense of aesthetics, so there's that, but I have trouble justifying that general feeling by practical arguments)

[mantepse]

comment:17

I almost exclusively use sage through the emacs interface (in fact, I don't even really know the ipython interface), and having a cleaner top level namespace might be quite helpful there. I have, apart from aesthetics, two reasons:

• the list of suggestions becomes shorter, so when looking for something, I have less to read or search through, and thus better chances not to miss the needle in the haystack,

• tab completion has a better chance of working, that is, I have fewer characters to type to get completion.

However, I must say that things are not terribly bad currently. So in case of doubt, I'd leave a particular item in the global name space.

[fchapoton]

comment:18

some work in #25785

[mantepse]

comment:20

class_graph might actually make sense for exploration - I wouldn't know how to discover this utility otherwise. But I don't know.

[fchapoton]

comment:21

see also #25791 for more work

[videlec]

comment:23

update milestone 8.3 -> 8.4

[dcoudert]

comment:25

Methods all_max_clique, max_clique and clique_number are imported into the global namespace in file src/sage/graphs/all.py. It contains from sage.graphs.cliquer import *. This can be removed as these methods are all already methods of Graph.

How should we proceed: open a specific ticket for that? add a deprecation warning, but with what message ?

[jhpalmieri]

comment:47

What about developer-specific commands? When I look at misc.all, I see (among others):

• benchmark, Profiler, logstr, cputime, walltime, set_edit_template, the pickling stuff, trace, fortran?, everything from sage.misc.package

Other questionable commands from misc.all:

• banner (already mentioned above), newton_method_sizes, map_threaded

I don't actually know if any of these are useful. They're not useful to me, but I don't know how to judge whether they're useful enough to anyone else to keep in the global namespace.

[jhpalmieri]

comment:50

I don't know why any of the pickling stuff is in the global namespace, but it has uses scattered throughout the Sage library, so some people view it as useful or important.

[jhpalmieri]

I would add to the list: commands that require optional packages, like zeta_zeros. If someone knows enough to install a package, then they know enough to import a function.