Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor with API #2

Merged
merged 99 commits into from
May 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
f3eef38
Clean up the README file and license/copyright notices
pkhalaj Apr 5, 2024
559b20f
Preliminary implementation of the API and SDK.
pkhalaj May 5, 2024
f92b25c
Commit
pkhalaj May 6, 2024
b4e9784
Commit
pkhalaj May 6, 2024
3d81e5d
Fix Sphinx
pkhalaj May 6, 2024
5622ad5
Fix Sphinx
pkhalaj May 6, 2024
5109c96
Fix Sphinx
pkhalaj May 6, 2024
621cb6b
Fix Sphinx
pkhalaj May 6, 2024
ae93435
Fix Sphinx
pkhalaj May 6, 2024
d15b0e8
Fix Sphinx
pkhalaj May 6, 2024
4baba93
Fix Sphinx
pkhalaj May 6, 2024
04b9fa1
Refactor error handling.
pkhalaj May 13, 2024
29a3b45
Better errors handling
pkhalaj May 15, 2024
8522453
Change imports to absolute
mraspaud May 15, 2024
55edaa4
Switch to pyproject.toml
mraspaud May 15, 2024
83aa075
Add github actions
mraspaud May 15, 2024
cfdaa82
Fix ruff stuff (WIP).
pkhalaj May 15, 2024
5482c29
Try fixing ci
mraspaud May 15, 2024
4591f64
Fix ci
mraspaud May 15, 2024
959ff5d
Add ci.yml
mraspaud May 15, 2024
67e8dac
Merge branch 'master' into add_ci
mraspaud May 15, 2024
1b5c61a
Fix package
mraspaud May 15, 2024
6f36e15
Fix mongodb installation
mraspaud May 15, 2024
e3af5bf
Add pytest cov to ci installation
mraspaud May 15, 2024
a0bc628
Merge branch 'add_ci' into feature/refactor-with-api
mraspaud May 15, 2024
93dd84d
Fix more ruff stuff (WIP).
pkhalaj May 15, 2024
a48f963
Just run ci on 3.12
mraspaud May 15, 2024
351926d
Add pre-commit-ci config
mraspaud May 15, 2024
9bc3f85
Add pre-commit-ci
mraspaud May 15, 2024
86d20f5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 15, 2024
9d74b51
Remove ruff from ci in favour of pre-commit
mraspaud May 15, 2024
16fefaf
Run pytest on all test files
mraspaud May 15, 2024
99b7800
Make ruff happy.
pkhalaj May 15, 2024
e209a29
Make ruff even happier.
pkhalaj May 15, 2024
2b36906
Commit.
pkhalaj May 15, 2024
3513776
Terminate mongodb instead of killing it
mraspaud May 15, 2024
7336f0d
Refactor tests into a separate directory.
pkhalaj May 15, 2024
f6c5a72
Add a cli db test
mraspaud May 15, 2024
4453f64
Sphinx
pkhalaj May 15, 2024
42a9eba
Docstrings!
pkhalaj May 16, 2024
43f9b07
Hard code some API configs.
pkhalaj May 16, 2024
ab855bf
Hard code some API configs.
pkhalaj May 16, 2024
386cf7b
Hard code some API configs.
pkhalaj May 16, 2024
abf0354
More docstrings!
pkhalaj May 16, 2024
9ef3340
More docstrings!
pkhalaj May 16, 2024
c75d270
More docstrings!
pkhalaj May 16, 2024
c5008a5
Add a command line interface function
mraspaud May 16, 2024
56a5ab6
More docstrings!
pkhalaj May 16, 2024
ddd5e09
More docstrings!
pkhalaj May 16, 2024
d50824e
More docstrings!
pkhalaj May 16, 2024
bec9f64
More docstrings!
pkhalaj May 16, 2024
7b5f6a5
More docstrings!
pkhalaj May 16, 2024
592e72a
More docstrings!
pkhalaj May 17, 2024
5221399
Convert make.sh to an actual Makefile.
pkhalaj May 17, 2024
60f8145
Add tests for pipelines.
pkhalaj May 17, 2024
9c13ac2
More docstrings!
pkhalaj May 17, 2024
ceebf5f
More docstrings!
pkhalaj May 17, 2024
d7782d5
More docstrings!
pkhalaj May 17, 2024
ac304b1
More docstrings!
pkhalaj May 17, 2024
00aa2f5
More docstrings!
pkhalaj May 17, 2024
c4a03c3
More docstrings!
pkhalaj May 17, 2024
c1508bb
More docstrings!
pkhalaj May 17, 2024
80a985c
More docstrings!
pkhalaj May 17, 2024
b1a7d7d
More docstrings!
pkhalaj May 17, 2024
a06acab
More logging!
pkhalaj May 17, 2024
540a58c
Update README.rst
pkhalaj May 17, 2024
332e977
More logging
pkhalaj May 17, 2024
e9fbb44
More docstrings.
pkhalaj May 19, 2024
2d166ab
More docstrings.
pkhalaj May 19, 2024
d1494a5
More docstrings.
pkhalaj May 19, 2024
8748ff9
Add script entrypoint
mraspaud May 20, 2024
e206d65
CI/CD.
pkhalaj May 20, 2024
f84a761
Refactor test_db tests.
pkhalaj May 20, 2024
047f232
Refactor.
pkhalaj May 20, 2024
cd5eac3
CI/CD.
pkhalaj May 20, 2024
c1b726b
Refactor.
pkhalaj May 20, 2024
b756c12
Clean up.
pkhalaj May 20, 2024
c1d141c
Add model for SubscriberConfig.
pkhalaj May 20, 2024
4c28ebd
Refactor.
pkhalaj May 21, 2024
4927ebc
Address PR comments.
pkhalaj May 21, 2024
3a0098f
Address PR comments.
pkhalaj May 21, 2024
71ad57d
Address PR comments.
pkhalaj May 21, 2024
b5fba56
Address PR comments.
pkhalaj May 21, 2024
22c38b4
Address PR comments.
pkhalaj May 21, 2024
9bae659
Address PR comments.
pkhalaj May 21, 2024
425fe9b
Address PR comments.
pkhalaj May 21, 2024
ba3c1ae
Address PR comments.
pkhalaj May 21, 2024
6e292a6
Address PR comments.
pkhalaj May 21, 2024
a1bc1be
Address PR comments.
pkhalaj May 21, 2024
f451754
Address PR comments.
pkhalaj May 21, 2024
9fc425c
Address PR comments.
pkhalaj May 21, 2024
cbcbe59
Address PR comments.
pkhalaj May 22, 2024
16610e4
CI/CD.
pkhalaj May 22, 2024
7f6081d
Mock sphinx autodoc imports.
pkhalaj May 22, 2024
b36dbd2
Mock sphinx autodoc imports.
pkhalaj May 22, 2024
5d09360
Args.
pkhalaj May 22, 2024
12f1a0b
Fix a bug with auto-generated docs.
pkhalaj May 22, 2024
a936c8e
pytest warnings.
pkhalaj May 22, 2024
167b588
pytest warnings.
pkhalaj May 22, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: Continuous Integration

on:
push:
branches: [ "master", "main" ]
pull_request:
branches: [ "master", "main" ]

jobs:
build:

runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.12"]
env:
PYTHON_VERSION: ${{ matrix.python-version }}

steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install mongodb
run: |
sudo apt install -y gnupg curl
curl -fsSL https://www.mongodb.org/static/pgp/server-7.0.asc | sudo gpg -o /usr/share/keyrings/mongodb-server-7.0.gpg --dearmor
echo "deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-7.0.gpg ] https://repo.mongodb.org/apt/ubuntu jammy/mongodb-org/7.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-7.0.list
sudo apt-get update
sudo apt install -y mongodb-org
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install ruff pytest pytest-lazy-fixtures pytest-asyncio pytest-cov
python -m pip install -e .
- name: Test with pytest
run: |
pytest --asyncio-mode=auto --cov=trolldb --cov-report=xml
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4.0.1
with:
token: ${{ secrets.CODECOV_TOKEN }}
slug: pytroll/pytroll-db
file: ./coverage.xml
env_vars: PYTHON_VERSION
27 changes: 27 additions & 0 deletions .github/workflows/deploy-sdist.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Deploy sdist

on:
release:
types:
- published

jobs:
test:
runs-on: ubuntu-latest

steps:
- name: Checkout source
uses: actions/checkout@v4

- name: Create sdist
shell: bash -l {0}
run: |
python -m pip install -q build
python -m build -s

- name: Publish package to PyPI
if: github.event.action == 'published'
uses: pypa/gh-action-pypi-publish@v1.8.14
with:
user: __token__
password: ${{ secrets.pypi_password }}
23 changes: 0 additions & 23 deletions .github/workflows/pylint.yml

This file was deleted.

17 changes: 16 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,21 @@ coverage.xml
# Django stuff:
*.log
*.pot
log

# Sphinx documentation
docs/_build/
docs/build/
*.rst
!index.rst
*.doctree
*.pickle

# the actual config file [HAS TO BE ALWAYS EXCLUDED!]
config.yaml
config.yml

# temp log and storage for the test database
__temp*

# version file
version.py
15 changes: 15 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
exclude: '^$'
fail_fast: false

repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
# Ruff version.
rev: 'v0.3.7'
hooks:
- id: ruff
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: no-commit-to-branch
23 changes: 23 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the OS, Python version and other tools you might need
build:
os: ubuntu-22.04
tools:
python: "3.12"

# Build documentation in the "docs/" directory with Sphinx
sphinx:
configuration: docs/source/conf.py

# Optional but recommended, declare the Python requirements required
# to build your documentation
# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- requirements: docs/requirements.txt
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -671,4 +671,4 @@ into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
<http://www.gnu.org/philosophy/why-not-lgpl.html>.
35 changes: 32 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,33 @@
doobie
======
The database interface of `Pytroll <https://pytroll.github.io/>`_

Database interface for pytroll

.. image:: https://results.pre-commit.ci/badge/github/pytroll/pytroll-db/master.svg
:target: https://results.pre-commit.ci/latest/github/pytroll/pytroll-db/master
:alt: pre-commit.ci status


Copyright (C)
2012, 2014, 2015, 2024

Martin Raspaud, Pouria Khalaj, Esben S. Nielsen, Adam Dybbroe, Kristian Rune Larsen


Authors
- Martin Raspaud <martin.raspaud@smhi.se>
- Pouria Khalaj <pouria.khalaj@smhi.se>
- Esben S. Nielsen <esn@dmi.dk>
- Adam Dybbroe <Adam.Dybbroe@smhi.se>
- Kristian Rune Larsen <krl@dmi.dk>


License
Consult the `LICENSE` file which is included as a part of this package.


Disclaimer
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details. You should have
received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
137 changes: 0 additions & 137 deletions bin/pytroll-mongo.py

This file was deleted.

2 changes: 0 additions & 2 deletions changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -215,5 +215,3 @@ Other
postgreSQL/postGIS database. [Adam Dybbroe]

- Initial commit. [Martin Raspaud]


11 changes: 11 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
SOURCE_DIR := source

.DEFAULT_GOAL := html
.PHONY: clean html

clean:
@cd $(SOURCE_DIR) && find . -type f ! -name 'index.rst' ! -name 'conf.py' -delete
@cd $(SOURCE_DIR) && find . -type d -delete

html:
@cd $(SOURCE_DIR) && python -m sphinx -T -b html -d _build/doctrees -D language=en . build/html
2 changes: 2 additions & 0 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
sphinx
sphinx-rtd-theme
Loading
Loading