TRACKER: Python 3 compatibility

[dev-zero]

For Python 3 compatibility we probably want additional bugs per explicit task/issue. This bug will collect discussion/decisions around the Python 3 compatibility efforts.

Feel free to convert this to a project or a milestone at any point.

[dev-zero]

Besides code compatibility, there may be other issues:

• Dependencies. Currently only supervisor is unavailable for Python 3, but we can get rid of it as per PR Supervisor removal #790 (Supervisor is no longer used)
• Plugins. We should define guidelines for plugin writers to ensure they start writing Python 3 compatible code (if required using six or future)
• Pickle. If (native) Python Pickles have been used in user data at some point, this must be converted, preferably to a different serialization format (moved away of pickles in favor of saving checkpoints as yaml dumps in the database as a node attribute).
• Celery Messages. It seems that we are currently using Pickle as serialization format, which should be changed at some point. To do that, one has to make sure that the Celery message queue in the database (remember, we are using the SQLA backend for Celery) is empty. This is a non-issue should we remove Celery (which can again easily be done replacing the calls in tasks.py by a async-based approach (since we are not using Celery for actual async operations at the moment) (Celery has been replaced by Circus and RabbitMQ and pickles are no longer used).
• Gradual conversion. To avoid large impact, we could start by gradually converting the source code. The problem here is that people might be unaware of new style and may re-add Python-2 idiosyncrasies. For that we need to add pre-commit hooks or Travis checkers. See also issue 2to3 as a pre-commit hook #798. As soon as full Python 3 compatibility is reached, this can be dropped since we can have Python 3 Travis tester. The Porting Python Guide recommends pylint for this.
• Compat libraries. We must define which compatibility library we want to use and recommend to plugin writers. Options are: six or python-future. (we settled on using six)
• Python 2.6 compat. By using from __future__ ... we are kicking Python 2.6 support (which is already not maintained anymore, but there might be some old installations around?)

[dev-zero]

Helpful tools:

• 2to3 ... includes fixers to convert Python 2 code to Python 3 code. Does not maintain Python 2 compatibility
• python-futurize ... uses 2to3 fixers and others to make code Py-2-and-3 compatible. Uses the Python Future project.
• python-modernize

[greschd]

For plugins, we need at least a working Python 3 branch so that plugins can actually test their code.

[dev-zero]

@greschd what I meant is that we could make it a requirement for them to run pylint --py3k or python-modernize which means their plugins would be python-3 ready

[giovannipizzi]

Already now we don't support python 2.6 (we have a lot of "{}".format(xxx) in the code)

[giovannipizzi]

The only thing left here is the management of new plugins. I opened #2260 so I will close this py3 issue