Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add data store parameters and data ID to the assets in the STAC published by xcube server #1029

Merged

Conversation

konstntokas
Copy link
Contributor

@konstntokas konstntokas commented Jun 28, 2024

Closes #1020

Checklist:

  • Add unit tests and/or doctests in docstrings
  • Add docstrings and API docs for any new/modified user-facing classes and functions
  • New/modified features documented in docs/source/*
  • Changes documented in CHANGES.md
  • GitHub CI passes
  • AppVeyor CI passes
  • Test coverage remains or increases (target 100%)

@konstntokas konstntokas requested review from pont-us and forman June 28, 2024 14:32
@pont-us pont-us removed their request for review July 2, 2024 09:00
@konstntokas konstntokas marked this pull request as draft July 3, 2024 15:08
@konstntokas konstntokas force-pushed the konstntokas-1020-xcube_server_add_store_parameters_to_stac_asset branch from b0eed73 to 94891be Compare July 8, 2024 12:53
@konstntokas konstntokas marked this pull request as ready for review July 8, 2024 12:53
@konstntokas konstntokas force-pushed the konstntokas-1020-xcube_server_add_store_parameters_to_stac_asset branch from 55a51d4 to 12dba1d Compare July 10, 2024 06:25
@konstntokas konstntokas force-pushed the konstntokas-1020-xcube_server_add_store_parameters_to_stac_asset branch from 12dba1d to c806c6d Compare July 10, 2024 07:38
Copy link
Member

@forman forman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please adjust naming.

CHANGES.md Outdated
@@ -26,6 +26,11 @@
* The `xcube.core.store.DataDescriptor` class now supports specifying time ranges
using both `datetime.date` and `datetime.datetime` objects. Previously,
only `datetime.date` objects were supported.
* The xcube server STAC publication has been adjusted so that the data store
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* The xcube server STAC publication has been adjusted so that the data store
* The xcube server STAC API has been adjusted so that the data store

CHANGES.md Outdated
@@ -26,6 +26,11 @@
* The `xcube.core.store.DataDescriptor` class now supports specifying time ranges
using both `datetime.date` and `datetime.datetime` objects. Previously,
only `datetime.date` objects were supported.
* The xcube server STAC publication has been adjusted so that the data store
parameters and data ID, which are needed to open the data,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
parameters and data ID, which are needed to open the data,
parameters and data ID, which are needed to open the data referred to by a STAC item,

CHANGES.md Outdated
@@ -26,6 +26,11 @@
* The `xcube.core.store.DataDescriptor` class now supports specifying time ranges
using both `datetime.date` and `datetime.datetime` objects. Previously,
only `datetime.date` objects were supported.
* The xcube server STAC publication has been adjusted so that the data store
parameters and data ID, which are needed to open the data,
are now included with the asset; furthermore, a second assert called
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
are now included with the asset; furthermore, a second assert called
are now included with the item's `analytic` asset.
Furthermore, a second assert called

@@ -7,6 +7,19 @@
],
"type": "application/zarr",
"href": "http://localhost:8080/s3/datasets/demo.zarr",
"xcube:store_kwargs": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"xcube:store_kwargs": {
"xcube:data_store_params": {

to align it with keyword argument data_store_params of the data store API function new_data_store().

@@ -7,6 +7,19 @@
],
"type": "application/zarr",
"href": "http://localhost:8080/s3/datasets/demo.zarr",
"xcube:store_kwargs": {
"data_store_id": "s3",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use separate asset xcube:data_store_id to align it with the positional argument data_store_id of the data store API function new_data_store().

}
}
},
"xcube:open_kwargs": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"xcube:open_kwargs": {
"xcube:open_data_params": {

to align it with data store API method DataStore.open_data() (-> open_data) that takes all the given parameters (-> params). And align it API method DataStore.get_open_data_params_schema() .

}
}
},
"xcube:open_kwargs": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

],
"type": "application/zarr",
"href": "http://localhost:8080/s3/pyramids/demo.levels",
"xcube:store_kwargs": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.


return {
"analytic": {
"title": f"{dataset_id} data access",
"roles": ["data"],
"type": "application/zarr",
"href": f"{default_storage_url}/{dataset_id}.zarr",
"href": f"{base_url}/s3/datasets/{dataset_id}.zarr",
"xcube:store_kwargs": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comments on test/webapi/ows/stac/demo-collection.json above.

"xcube:analytic": {
v["name"]: {
"title": f"{v['name']} data access",
"roles": ["data"],
"type": "application/zarr",
"href": f"{default_storage_url}/" f"{dataset_id}.zarr/{v['name']}",
"href": f"{base_url}/s3/datasets/{dataset_id}.zarr/{v['name']}",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comments on test/webapi/ows/stac/demo-collection.json above.

@konstntokas konstntokas requested a review from forman July 10, 2024 09:51
Copy link
Member

@forman forman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeeha!

@forman forman merged commit 7fee68e into main Jul 10, 2024
3 checks passed
@konstntokas konstntokas deleted the konstntokas-1020-xcube_server_add_store_parameters_to_stac_asset branch July 10, 2024 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

xcube server STAC implementation, further information needed to access data
2 participants