Export verification to harp

[tnipen]

This could be implemented as an Output.

• Write a POC to get format correct
• Delete /lustre/storeB/project/IT/geout/larsfp/bris-inference-issue-7 after
• Merge POC into bris-inference as a format output test
• Write a general writer for harp using model output

[ways]

Waiting for #17 to ease testing

[ways]

Seems Harp data format is about to change: https://gitlab.met.no/yrop/yrprod/-/issues/505

[ways]

https://arrow.apache.org/docs/python/parquet.html

[ways]

Har snakket med Andrew. Han setter opp miljø for å kjøre. Venter på info om filene som skal skrives.

[ways]

When I'm writing code for this, it will be best to test with a pre-generated dataset. @havardhhaugen Can you point me to a representative one in lustre?

[ways]

Got some documentation: https://docs.google.com/document/d/1XOHNAcNLZ6aizUmi_PAFNUkNSm-f3y-athnLUi5tv1I/edit?tab=t.0

[ways]

When I'm writing code for this, it will be best to test with a pre-generated dataset. havardhhaugen, Can you point me to a representative one in lustre?

Was given /lustre/storeB/project/nwp/bris/inference_files/for_lars_testing

[ways]

If I understand correctly, this is what's required to create a usable file for harp:

• get metadata from forecast netcdf-file
• get all station IDs from frost within coordinates and times from forecast
• for timestep and SID
  • get forecast closest to station
  • write frost observation and forecast data to table

[ways]

New version of POC for Andrew, waiting for ack.