New cube store framework and cube generator

[forman]

Is your feature request related to a problem? Please describe.

xcube-sh and xcube-cci are new tools and xcube plugins that can open data cube views from some data web API and allow users to generate cubes by persisting them as Zarrs to the local filesystem or to a S3 bucket. As it is planned to support more APIs and develop more such tools we need to abstract from specific data stores and allo a generic generator tool to lookup existing stores by a store id and use the cube views to generate user-defined output cubes.

Here are some use cases for the new framework to be developed:

• UC1: Find cubes and variables by constraints from a set of known cube stores
• UC2: Collect cubes to form a new cube store
• UC3: Open and generate cube from some data API passing a target cube configuration
• UC4: Generate cube into some persistent storage

Describe the solution you'd like

• Develop a cube store framework that can deal with the above use cases.
• Develop a new, flexible "xcube gen" command and generate_cube() function that uses the new cube store framework and that works as follows:

1. open cube (cube view) from some cube store (data API or files in local file system)
2. optionally transform the cube using some user-defined, xarray/dask-based Python code
3. write the cube to some cube store (S3 or local file system)