Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements for the update_reference section [docs] #1008

Merged
merged 3 commits into from
Jan 23, 2020
Merged

Improvements for the update_reference section [docs] #1008

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1a86917
Initial commit
epassaro Jan 21, 2020
32a65e9
Minor changes
epassaro Jan 21, 2020
89c9829
Last changes
epassaro Jan 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/development/git_workflow.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -476,6 +476,8 @@ If you forgot to make a backup branch::
git reset --hard cool-feature@{2}


.. _reviewing-and-helping-with-pr:

Reviewing and helping others with Pull Requests
-----------------------------------------------

Expand Down
76 changes: 44 additions & 32 deletions docs/development/update_refdata.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,71 +2,83 @@
Procedure to update the reference data
**************************************

We assume that you have added the necessary changes to TARDIS and have a
PR open.
We assume that you have added the necessary changes to TARDIS and have an open pull request.

#. Fork the tardis ref-data repository like described in :ref:`development-workflow`.
#. Fork the ``tardis-refdata`` repository like described in :ref:`development-workflow`.

#. Clone tardis-refdata. Then make a new branch named the same as your new tardis feature branch. Make sure to set the upstream of the repository to `https://github.com/tardis-sn/tardis-refdata.git`
* You can check that you have the right upstream set using:
.. code-block:: None

git checkout upstream/pr/11
#. Clone ``tardis-refdata`` to your computer. Then make a new branch named the same as your new TARDIS feature branch. Make sure to have correctly set up the ``upstream`` remote as explained in :ref:`reviewing-and-helping-with-pr`.

#. Make sure to have sure to have git-lfs installed. If not already done, the process can be seen here: :ref:`git_lfs_instructions`
#. Activate your ``tardis`` conda environment.

#. Open tardis-refdata directory. Fetch from upstream and then use git-lfs to pull ref-data files from github using:
.. code-block:: None

conda activate tardis

#. Navigate to your ``tardis-refdata`` directory and type:

.. code-block:: None

git lfs install
git lfs fetch upstream
git lfs pull

#. Open into tardis directory. Generate new reference data (in the correct branch) offline using:
#. Go to your ``tardis`` directory. Make sure you are working on the correct branch and generate new reference data using:

.. code-block:: None

python setup.py test --args="--tardis-refdata=<path to refdata repo/with the right branch> --generate-reference"
python setup.py test --args="--tardis-refdata=<path to refdata repo on the right branch> --generate-reference"

#. Rerun the tests to make sure it does not fail using:
#. Re-run the tests to make sure it does not fail using:

.. code-block:: None

python setup.py test --args="--tardis-refdata=<path to refdata repo/with the right branch>"
python setup.py test --args="--tardis-refdata=<path to refdata repo on the right branch>"

#. Open and run the refdata comparer notebook provided in TARDIS-refdata to check if there are any unexpected changes in the updated reference data and the previous reference data.
#. Go back to your ``tardis-refdata`` folder, and run a Jupyter Notebook session inside the ``notebook`` folder.

#. Switch to tardis-refdata. Commit the changed ref-data and open a PR on tardis-refdata
#. Open the ``ref_data_compare.ipynb`` file notebook and look for the cell with the following code:

#. Switch back to the TARDIS directory. Open `.travis.yml`
.. code-block:: None

#. Change the following lines
comparer = ReferenceComparer(ref2_hash='upstream/pr/24')

Replace '24' for the number of the last merged pull request showed `here <https://github.com/tardis-sn/tardis-refdata/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aclosed>`_.

#. Run all cells and check if there are any unexpected changes in the updated reference data respect the previous one.

#. Commit the changes made to the reference data and open a PR on ``tardis-refdata``.

#. Switch back to the ``tardis`` directory. Open the ``.travis.yml`` file and change the following lines:

.. code-block:: None

- if [[ $TEST_MODE == 'spectrum' ]]; then git fetch origin pull/<your TARDIS-refdata PR number; not the TARDIS PR number>/head:<some descriptive name>; fi
- if [[ $TEST_MODE == 'spectrum' ]]; then git fetch origin pull/<your tardis-refdata PR number; not the TARDIS PR number>/head:<some descriptive name>; fi
- if [[ $TEST_MODE == 'spectrum' ]]; then git checkout <some descriptive name>; fi```

#. Commit the `.travis.yml` to your Pull request
#. Make sure that your TARDIS PR now passes on TRAVIS.
#. Then merge the PR on tardis-refdata.
#. Then change `.travis.yml` to
#. Commit the ``.travis.yml`` to your pull request.

#. Make sure that your TARDIS pull request now passes all tests on Travis-CI.

#. Ask for review for your PR on ``tardis-refdata`` and wait for merge.

#. Then change the ``.travis.yml`` in ``tardis`` directory to:

.. code-block:: None

- if [[ $TEST_MODE == 'spectrum' ]]; then git fetch origin; fi
- if [[ $TEST_MODE == 'spectrum' ]]; then git checkout origin/master; fi```
#. Then make sure that your TARDIS PR passes again.
#. Then merge your PR to TARDIS master
#. Congratulations - you updated TARDIS to be better. Have a beer and steak (or Tofu if you are vegetarian/vegan)

#. Ensure TARDIS pull request passes Travis-CI again and ping someone to merge your PR to the TARDIS master branch.


Congratulations! You have updated TARDIS to be better. Have a beer and steak (or Tofu if you are vegetarian/vegan)


Troubleshooting
###############

* Unable to generate reference data
* If generating fails due to an inability to open chianti_He.h5, make sure that you have installed git-lfs and have pulled the files from github (See steps 3 and 4).

* Error in running `comparer = ReferenceComparer(ref2_hash='upstream/pr/11')` on the comparer notebook: `No such file or directory: '.../unit_test_data.h5'`
* If notebook file is unable to find the file /unit_test_data.h5, make sure you have correctly set your upstream. To check this, use:

* If generating fails due to an inability to open ``chianti_He.h5``, make sure that you have activated your `tardis` conda environment, and ``git-lfs`` is installed. Fetch and pull the files from GitHub as explained in step 4.

* If this fails, make sure that your upstream is set correctly to `https://github.com/tardis-sn/tardis-refdata.git` (See step 2).
* Error when running ``comparer = ReferenceComparer(ref2_hash='upstream/pr/XX')`` on the comparer notebook: ``No such file or directory: '.../unit_test_data.h5'``
* If notebook file is unable to find the file ``unit_test_data.h5``, make sure you have correctly set your upstream as explained in :ref:`reviewing-and-helping-with-pr`.