[DO NOT MERGE] feat(cogify): migrate argo-task work to basemaps-cogify BM-1127

[tawera-manaena]

Motivation

As a Basemaps user, I want to consume the NZTopo 50 & 250 Maps as a tile service.

Modifications

This work defines a new CLI command. The command's purpose is to generate STAC files for an NZTopo Map Series imagery collection. We have designed the command to process the following collections stored in the AWS S3 TopoReleaseArchive directory:

s3://topographic-upload/TopoReleaseArchive/NZTopo50_GeoTif_Gridless/
s3://topographic-upload/TopoReleaseArchive/NZTopo250_GeoTif_Gridless/

There is also a new Argo workflow in development that depends on this work. The workflow's purpose is to automate the standardisation of an NZTopo Map Series imagery collection. The workflow executes this command as a step in its process flow.

CLI Command: topo-stac-creation

Arguments

The arguments that can be passed to the command are as follows:

• Title

  The name/title of the Map Series imagery collection.

  Argument	title
  Type	string
  Required	yes
  Example	Raster Topographic Maps 50k

• Source

  The source directory URL of the Map Series imagery collection.

  Argument	source
  Type	string
  Required	yes
  Example	s3://topographic-upload/TopoReleaseArchive/NZTopo50_GeoTif_Gridless/

• Target

  The target directory URL into which to save the generated directory structure of StacItem and StacCollection files.

  Argument	target
  Type	string
  Required	yes
  Example	s3://linz-workflows-scratch/<date>/<hash>/

• Scale

  The scale of the Map Series imagery collection's map sheets.

  Argument	scale
  Type	string
  Required	yes
  Options	topo25, topo50, or topo250

• Resolution

  The resolution of the Map Series imagery collection's map sheets.

  Argument	resolution
  Type	string
  Required	yes
  Options	gridded_600dpi or gridless_600dpi

• Latest Only

  A flag used to indicate whether all of the generated files should be saved to the target location, or only that of each map sheet's latest version.

  Argument	latestOnly
  Type	boolean
  Required	no
  Options	true or false

Process

The command processes a collection as follows:

1. Loads the collection of GeoTiff images into memory
2. Extracts the map code, version, and metadata (e.g. EPSG) from each image
3. Identifies the latest version of each map sheet by code
4. Groups the images by EPSG
5. Generates a StacItem file for each image, and a StacCollection file for each EPSG grouping
6. Structures the groups of StacItem and StacCollection files into a directory tree
7. Saves the files as they are structured to a target location

Outputs

The command groups the images by EPSG and then structures the generated StacItem and StacCollection files, as illustrated:

Schema	Example

The command then saves the generated tree of folders and files into the target location directory.

Limitations

• Tile Matrices

  For each image in a collection, the command extracts the EPSG of the image and converts it to a runtime Epsg Enum value. This value is then mapped to the EPSG's corresponding Tile Matrix definition. At this time, there is no such Tile Matrix definition for the Chatham Islands EPSG code, 3793. The task will need to be updated once Basemaps supports this definition.

[tawera-manaena]

Migration Test

I've run two tests. I've run the argo-task process to generate outputs. I've also run this process and compared the outputs to those of the argo-task process. Fortunately, the outputs match.

[tawera-manaena]

Migration Strategy

I have migrated the argo-task work into a standalone directory within the cogify package. I'm not convinced if this is the best way with concern to introducing bloat or overhead for other tools that consume the cogify package as a dependency. Maybe this work should live in a seperate package all together? Or is this fine?

Example Command

node packages/cogify/build/bin.js stac --title "NZTopo250" --source s3://linz-topographic-upload/topographic/TopoReleaseArchive/NZTopo250_GeoTif_Gridless/ --target ./output-folder --scale "topo250" --resolution "gridless_600dpi" --force-output

[Wentao-Kuang]

I believe, the cover command have all stac creation logic which are very similar, wonder we can extract them to reuse together.

[tawera-manaena]

We have migrated this pull request's contents into this pull request to allow testing the Topo Raster Map STAC file and COG creation processes within a single container. Closing this pull request now.