Adds LICENSE, NOTICE, CHANGES, tests source, and test vectors as data_files in the distribution.

[tgregg]

Issue #, if available:
#72

Description of changes:
Results in the following output from python setup.py sdist:

running sdist
running egg_info
creating amazon.ion.egg-info
writing amazon.ion.egg-info/PKG-INFO
writing dependency_links to amazon.ion.egg-info/dependency_links.txt
writing namespace_packages to amazon.ion.egg-info/namespace_packages.txt
writing requirements to amazon.ion.egg-info/requires.txt
writing top-level names to amazon.ion.egg-info/top_level.txt
writing manifest file 'amazon.ion.egg-info/SOURCES.txt'
reading manifest file 'amazon.ion.egg-info/SOURCES.txt'
writing manifest file 'amazon.ion.egg-info/SOURCES.txt'
running check
creating amazon.ion-0.3.1
creating amazon.ion-0.3.1/amazon
creating amazon.ion-0.3.1/amazon.ion.egg-info
creating amazon.ion-0.3.1/amazon/ion
creating amazon.ion-0.3.1/tests
creating amazon.ion-0.3.1/vectors
creating amazon.ion-0.3.1/vectors/iontestdata
creating amazon.ion-0.3.1/vectors/iontestdata/bad
creating amazon.ion-0.3.1/vectors/iontestdata/bad/timestamp
creating amazon.ion-0.3.1/vectors/iontestdata/bad/timestamp/outOfRange
creating amazon.ion-0.3.1/vectors/iontestdata/bad/utf8
creating amazon.ion-0.3.1/vectors/iontestdata/good
creating amazon.ion-0.3.1/vectors/iontestdata/good/equivs
creating amazon.ion-0.3.1/vectors/iontestdata/good/equivs/utf8
creating amazon.ion-0.3.1/vectors/iontestdata/good/non-equivs
creating amazon.ion-0.3.1/vectors/iontestdata/good/timestamp
creating amazon.ion-0.3.1/vectors/iontestdata/good/timestamp/equivTimeline
copying files to amazon.ion-0.3.1...
copying CHANGES.md -> amazon.ion-0.3.1
copying LICENSE -> amazon.ion-0.3.1
copying NOTICE -> amazon.ion-0.3.1
copying README.md -> amazon.ion-0.3.1
copying setup.cfg -> amazon.ion-0.3.1
copying setup.py -> amazon.ion-0.3.1
copying amazon/__init__.py -> amazon.ion-0.3.1/amazon
copying amazon.ion.egg-info/PKG-INFO -> amazon.ion-0.3.1/amazon.ion.egg-info
copying amazon.ion.egg-info/SOURCES.txt -> amazon.ion-0.3.1/amazon.ion.egg-info
copying amazon.ion.egg-info/dependency_links.txt -> amazon.ion-0.3.1/amazon.ion.egg-info
copying amazon.ion.egg-info/namespace_packages.txt -> amazon.ion-0.3.1/amazon.ion.egg-info
copying amazon.ion.egg-info/requires.txt -> amazon.ion-0.3.1/amazon.ion.egg-info
copying amazon.ion.egg-info/top_level.txt -> amazon.ion-0.3.1/amazon.ion.egg-info
copying amazon/ion/__init__.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/core.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/equivalence.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/exceptions.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/reader.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/reader_binary.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/reader_managed.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/reader_text.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/simple_types.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/simpleion.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/symbols.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/util.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/writer.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/writer_binary.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/writer_binary_raw.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/writer_binary_raw_fields.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/writer_buffer.py -> amazon.ion-0.3.1/amazon/ion
copying amazon/ion/writer_text.py -> amazon.ion-0.3.1/amazon/ion
copying tests/__init__.py -> amazon.ion-0.3.1/tests
copying tests/event_aliases.py -> amazon.ion-0.3.1/tests
copying tests/reader_util.py -> amazon.ion-0.3.1/tests
copying tests/test_core_iontype.py -> amazon.ion-0.3.1/tests
copying tests/test_equivalence.py -> amazon.ion-0.3.1/tests
copying tests/test_package.py -> amazon.ion-0.3.1/tests
copying tests/test_reader_base.py -> amazon.ion-0.3.1/tests
copying tests/test_reader_binary.py -> amazon.ion-0.3.1/tests
copying tests/test_reader_buffer.py -> amazon.ion-0.3.1/tests
copying tests/test_reader_managed.py -> amazon.ion-0.3.1/tests
copying tests/test_reader_text.py -> amazon.ion-0.3.1/tests
copying tests/test_simple_types.py -> amazon.ion-0.3.1/tests
copying tests/test_simpleion.py -> amazon.ion-0.3.1/tests
copying tests/test_symbols_catalog.py -> amazon.ion-0.3.1/tests
copying tests/test_symbols_table.py -> amazon.ion-0.3.1/tests
copying tests/test_util_enum.py -> amazon.ion-0.3.1/tests
copying tests/test_util_record.py -> amazon.ion-0.3.1/tests
copying tests/test_util_unicode.py -> amazon.ion-0.3.1/tests
copying tests/test_vectors.py -> amazon.ion-0.3.1/tests
copying tests/test_writer_base.py -> amazon.ion-0.3.1/tests
copying tests/test_writer_binary.py -> amazon.ion-0.3.1/tests
copying tests/test_writer_binary_raw.py -> amazon.ion-0.3.1/tests
copying tests/test_writer_binary_raw_fields.py -> amazon.ion-0.3.1/tests
copying tests/test_writer_buffer.py -> amazon.ion-0.3.1/tests
copying tests/test_writer_text.py -> amazon.ion-0.3.1/tests
copying tests/trampoline_util.py -> amazon.ion-0.3.1/tests
copying tests/writer_util.py -> amazon.ion-0.3.1/tests
copying vectors/LICENSE -> amazon.ion-0.3.1/vectors
copying vectors/NOTICE -> amazon.ion-0.3.1/vectors
copying vectors/README.md -> amazon.ion-0.3.1/vectors
copying vectors/iontestdata/bad/README.md -> amazon.ion-0.3.1/vectors/iontestdata/bad
copying vectors/iontestdata/bad/annotationFalse.ion -> amazon.ion-0.3.1/vectors/iontestdata/bad
copying vectors/iontestdata/bad/annotationLengthTooLongContainer.10n -> amazon.ion-0.3.1/vectors/iontestdata/bad
copying vectors/iontestdata/bad/annotationLengthTooLongScalar.10n -> amazon.ion-0.3.1/vectors/iontestdata/bad

...[hundreds more test vectors]...

copying vectors/iontestdata/good/timestamp/equivTimeline/timestamps.ion -> amazon.ion-0.3.1/vectors/iontestdata/good/timestamp/equivTimeline
Writing amazon.ion-0.3.1/setup.cfg
creating dist
Creating tar archive
removing 'amazon.ion-0.3.1' (and everything under it)

Note: tests is still not treated as a python package; its files are included as ordinary data files that may be executed by a consumer of the source distribution if they want.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[jayvdb]

I found one negative side effect of using data for this.

they will also be included in the wheel, which makes it a bit bigger than necessary, and the LICENSE and NOTICE at least is duplicated in the wheel.

> python3.7 setup.py bdist_wheel
....
running install_data
creating build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data
creating build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data
copying LICENSE -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/
copying NOTICE -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/
copying CHANGES.md -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/
creating build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/__init__.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/event_aliases.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/reader_util.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/test_core_iontype.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/test_equivalence.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/test_package.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/test_reader_base.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/test_reader_binary.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/test_reader_buffer.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/test_reader_managed.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
copying tests/test_reader_text.py -> build/bdist.linux-x86_64/wheel/amazon.ion-0.3.1.data/data/tests
...
> unzip ../dist/*whl
Archive:  ../dist/amazon.ion-0.3.1-py3-none-any.whl
  inflating: amazon.ion-0.3.1-py3.7-nspkg.pth
  inflating: amazon/ion/__init__.py
  inflating: amazon/ion/core.py
  inflating: amazon/ion/equivalence.py
  inflating: amazon/ion/exceptions.py
  inflating: amazon/ion/reader.py
  inflating: amazon/ion/reader_binary.py
  inflating: amazon/ion/reader_managed.py
  inflating: amazon/ion/reader_text.py
  inflating: amazon/ion/simple_types.py
  inflating: amazon/ion/simpleion.py
  inflating: amazon/ion/symbols.py
  inflating: amazon/ion/util.py
  inflating: amazon/ion/writer.py
  inflating: amazon/ion/writer_binary.py
  inflating: amazon/ion/writer_binary_raw.py
  inflating: amazon/ion/writer_binary_raw_fields.py
  inflating: amazon/ion/writer_buffer.py
  inflating: amazon/ion/writer_text.py
  inflating: amazon.ion-0.3.1.data/data/CHANGES.md
  inflating: amazon.ion-0.3.1.data/data/LICENSE
  inflating: amazon.ion-0.3.1.data/data/NOTICE
  inflating: amazon.ion-0.3.1.data/data/tests/__init__.py
  inflating: amazon.ion-0.3.1.data/data/tests/event_aliases.py
  inflating: amazon.ion-0.3.1.data/data/tests/reader_util.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_core_iontype.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_equivalence.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_package.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_reader_base.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_reader_binary.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_reader_buffer.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_reader_managed.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_reader_text.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_simple_types.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_simpleion.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_symbols_catalog.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_symbols_table.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_util_enum.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_util_record.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_util_unicode.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_vectors.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_writer_base.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_writer_binary.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_writer_binary_raw.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_writer_binary_raw_fields.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_writer_buffer.py
  inflating: amazon.ion-0.3.1.data/data/tests/test_writer_text.py
  inflating: amazon.ion-0.3.1.data/data/tests/trampoline_util.py
  inflating: amazon.ion-0.3.1.data/data/tests/writer_util.py
  inflating: amazon.ion-0.3.1.dist-info/LICENSE
  inflating: amazon.ion-0.3.1.dist-info/METADATA
  inflating: amazon.ion-0.3.1.dist-info/NOTICE
  inflating: amazon.ion-0.3.1.dist-info/WHEEL
  inflating: amazon.ion-0.3.1.dist-info/namespace_packages.txt
  inflating: amazon.ion-0.3.1.dist-info/top_level.txt
  inflating: amazon.ion-0.3.1.dist-info/RECORD

Now try sdist and wheels with this ...

diff --git a/setup.py b/setup.py
index bba7ef6..fdf7075 100644
--- a/setup.py
+++ b/setup.py
@@ -56,7 +56,7 @@ setup(
     ],

     data_files=[
-        ('', ['LICENSE', 'NOTICE', 'CHANGES.md']),
+        ('../..', ['LICENSE', 'NOTICE', 'CHANGES.md']),
         ('tests', find_files('tests', '.py')),
         ('vectors', find_files('vectors'))
     ]

I am pretty sure that isnt supposed to be allowed, and could be used for ill-intents.

[jayvdb]

How we going with this? It would be really nice to get a new release out with the i586 fixes & test data , so I can tidy https://build.opensuse.org/package/show/home:jayvdb:moban/python-amazon.ion and submit it ;-)

[tgregg]

@jayvdb Sorry for the delay. I'm planning to resume work on this this week.

Can you elaborate on this to help me understand the perceived risk?

I am pretty sure that isnt supposed to be allowed, and could be used for ill-intents.

[jayvdb]

There is no risk related to that for your proejct, as it is was a comment regarding using ../../ in data_files, which incorrectly allows a package to write anywhere in site-packages, and I cant help but worry what would be possible if the wheel used ../../../ , which I assume will let them write outside of the python area.

The only problem with this PR at the moment is that it will include tests/ in the wheel if bdist_wheel is used, afaics. And tests in the wheel are absolutely useless for most people, and it includes them as data/tests which is likely not where the tests are expecting to be run from. However, this is not too different from projects which use a repo layout of foo with foo/tests , and often they leave the subdir tests in the wheel, bloating it a bit.

None of this has any impact on rpms, which have shell scripting to fiddle with whatever you provide. As long as all of the test stuff is in the next sdist, I am a very happy camper.

If you really like this approach, instead of MANIFEST.in, I'm sure we can find a way to convince the wheel to exclude the tests. Worst case a tiny script to unzip and re-zip the wheel will do the trick.

[tgregg]

@jayvdb Got it. Thanks for pointing out bdist_wheel. I'm closing this PR in favor of #80, which uses MANIFEST.in instead of data_files.