|
| 1 | +Developing with DSMR-reader |
| 2 | +=========================== |
| 3 | + |
| 4 | + |
| 5 | +.. contents:: |
| 6 | + :depth: 2 |
| 7 | + |
| 8 | +.. note:: |
| 9 | + |
| 10 | + In this document there are many references to:: |
| 11 | + |
| 12 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 13 | + |
| 14 | + You will only have to execute it once per terminal session, to make sure you are working in the designated virtual env for DSMR-reader. |
| 15 | + |
| 16 | + |
| 17 | +Setting up a development environment in Ubuntu 18.04 |
| 18 | +---------------------------------------------------- |
| 19 | + |
| 20 | +System packages:: |
| 21 | + |
| 22 | + sudo apt-get install -y postgresql postgresql-server-dev-all git python3 python3-pip python3-virtualenv virtualenvwrapper libmysqlclient-dev mariadb-server poedit |
| 23 | + |
| 24 | +Clone DSMR-reader repository from Github:: |
| 25 | + |
| 26 | + git clone ... (your fork) |
| 27 | + cd dsmr-reader/ |
| 28 | + |
| 29 | +Create virtualenv and install all packages:: |
| 30 | + |
| 31 | + mkdir ~/.virtualenvs |
| 32 | + virtualenv ~/.virtualenvs/dsmrreader --no-site-packages --python python3 |
| 33 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 34 | + pip3 install -r dsmrreader/provisioning/requirements/base.txt -r dsmrreader/provisioning/requirements/dev.txt -r dsmrreader/provisioning/requirements/mysql.txt -r dsmrreader/provisioning/requirements/postgresql.txt -r dsmrreader/provisioning/requirements/test.txt -r dsmrreader/provisioning/requirements/travis.txt |
| 35 | + |
| 36 | +Copy development config:: |
| 37 | + |
| 38 | + cp dsmrreader/provisioning/django/development.py dsmrreader/settings.py |
| 39 | + |
| 40 | +Try a check, output should be something like ``System check identified no issues (0 silenced).``:: |
| 41 | + |
| 42 | + ./manage.py check |
| 43 | + |
| 44 | +Run quick tests (with in-memory database):: |
| 45 | + |
| 46 | + ./tools/quick-test.sh |
| 47 | + |
| 48 | +Set up PostgreSQL test database:: |
| 49 | + |
| 50 | + sudo sudo -u postgres createuser -DSR dsmrreader |
| 51 | + sudo sudo -u postgres psql -c "alter user dsmrreader with password 'dsmrreader';" |
| 52 | + sudo sudo -u postgres psql -c "alter user dsmrreader CREATEDB;" |
| 53 | + |
| 54 | +Set up MySQL (or MariaDB) test database:: |
| 55 | + |
| 56 | + mysql_tzinfo_to_sql /usr/share/zoneinfo | sudo mysql --defaults-file=/etc/mysql/debian.cnf mysql |
| 57 | + sudo mysql --defaults-file=/etc/mysql/debian.cnf |
| 58 | + |
| 59 | + # Execute these queries: |
| 60 | + GRANT ALL PRIVILEGES ON *.* TO 'dsmrreader'@'localhost' IDENTIFIED BY 'dsmrreader'; |
| 61 | + FLUSH PRIVILEGES; |
| 62 | + |
| 63 | +Now check whether tests run well with all three database backends (this may take a while):: |
| 64 | + |
| 65 | + ./tools/test-all.sh |
| 66 | + |
| 67 | + |
| 68 | +Initial data to develop with |
| 69 | +---------------------------- |
| 70 | + |
| 71 | +To be honest, the best initial/fixture data is simply a backup of your own system in production. |
| 72 | + |
| 73 | +Just import it as you should on your RaspberryPi. Copy a database backup to ``/var/lib/postgresql/`` on your PC and import it:: |
| 74 | + |
| 75 | + # Create empty database if it does not exist yet. |
| 76 | + sudo sudo -u postgres createdb -O dsmrreader dsmrreader |
| 77 | + |
| 78 | + # For .SQL |
| 79 | + sudo sudo -u postgres psql dsmrreader -f <PATH-TO-POSTGRESQL-BACKUP.sql> |
| 80 | + |
| 81 | + # For .SQL.GZ |
| 82 | + zcat <PATH-TO-POSTGRESQL-BACKUP.sql.gz> | sudo sudo -u postgres psql dsmrreader |
| 83 | + |
| 84 | +.. warning:: |
| 85 | + |
| 86 | + Please note that you should not run any (backend) processes in your DSMR-reader development environment, until you've unlinked all external services. |
| 87 | + |
| 88 | + After importing the backup of your production system, simply run:: |
| 89 | + |
| 90 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 91 | + ./manage.py development_reset |
| 92 | + |
| 93 | + This will remove all API keys and other links to externals systems, as well as reset the admin user credentials to ``admin / admin`` (user / password). |
| 94 | + |
| 95 | + |
| 96 | +Fake datalogger |
| 97 | +--------------- |
| 98 | + |
| 99 | +There is a builtin command that can somewhat fake a datalogger:: |
| 100 | + |
| 101 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 102 | + ./manage.py dsmr_fake_datasource --ack-to-mess-up-my-data --with-gas --with-electricity-returned |
| 103 | + |
| 104 | +It will generate random data every second in a certain pattern and should be fine for basic testing. |
| 105 | + |
| 106 | +Please note that it only inserts unprocessed readings, so you'll still have to run the ``./manage.py dsmr_backend --run-once`` command to have the readings processed. |
| 107 | + |
| 108 | + |
| 109 | +Running DSMR-reader locally |
| 110 | +--------------------------- |
| 111 | + |
| 112 | +You can run the Django development server with:: |
| 113 | + |
| 114 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 115 | + ./manage.py runserver |
| 116 | + |
| 117 | +The application will be accessible on: ``http://localhost:8000/``. |
| 118 | +Any code changes you make will let the application reload automatically. |
| 119 | + |
| 120 | + |
| 121 | +Tests |
| 122 | +----- |
| 123 | + |
| 124 | +The easiest way to run tests is to use the in-memory tests:: |
| 125 | + |
| 126 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 127 | + ./tools/quick-test.sh |
| 128 | + |
| 129 | +To test a single app within DSMR-reader, just append it:: |
| 130 | + |
| 131 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 132 | + ./tools/quick-test.sh dsmr_frontend |
| 133 | + |
| 134 | +To test all database backends, run:: |
| 135 | + |
| 136 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 137 | + ./tools/test-all.sh |
| 138 | + |
| 139 | +Translations |
| 140 | +------------ |
| 141 | + |
| 142 | +You can find the translations (.PO files) for the main application in ``dsmrreader/locales/``. |
| 143 | +To regenerate them, just execute the ``./tools/quick-test.sh`` script, as one of the tests checks translations. |
| 144 | + |
| 145 | + |
| 146 | +Editing documentation |
| 147 | +--------------------- |
| 148 | + |
| 149 | +The documentation is part of the repository and can be generated (automatically) with Sphinx:: |
| 150 | + |
| 151 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 152 | + cd docs/ |
| 153 | + sphinx-autobuild . _build/html -p 10000 |
| 154 | + |
| 155 | +You can now view the documentation in your browser by accessing: ``http://127.0.0.1:10000``. |
| 156 | +Any changes you make will be reflected instantly in the browser, as Sphinx continuously checks for changed files. |
| 157 | + |
| 158 | + |
| 159 | +Translating documentation |
| 160 | +------------------------- |
| 161 | + |
| 162 | +Translations are done using gettext and .PO files. Regenerate the .PO files with:: |
| 163 | + |
| 164 | + source ~/.virtualenvs/dsmrreader/bin/activate |
| 165 | + cd docs/ |
| 166 | + make gettext && sphinx-intl update -p _build/locale -l nl |
| 167 | + |
| 168 | +The .PO files in ``docs/locale`` should be regenerated now. You can use ``poedit`` to view and translate the files. |
| 169 | + |
| 170 | +After editing the .PO files, you can check the result by building the Dutch translations locally:: |
| 171 | + |
| 172 | + make -e SPHINXOPTS="-D language='nl'" html |
| 173 | + |
| 174 | +Now view the generated HTML in your browser by opening: ``docs/_build/html/index.html`` |
| 175 | + |
| 176 | + |
| 177 | +Pull requests |
| 178 | +------------- |
| 179 | + |
| 180 | +Please make sure to always point any pull requests to the ``development`` branch of DSMR-reader, as the ``master`` branch will only be affected by release merges. |
0 commit comments