Skip to content

Commit 5549447

Browse files
guillaumevernieresguillaumevernieres
authored
Marine obs reorganization and more ... (#876)
Things done: - Assumes that the obs will be organized like something like this from now on: ```console └── gdas.20180415 └── 12 └── ocean ├── adt │   ├── rads_adt_3a_2018105.nc │   └── rads_adt_3b_2018105.nc ├── icec │   ├── AMSR2-SEAICE-NH_v2r2_GW1_s201804150835180_e201804151014170_c201804151052280.nc │   ├── AMSR2-SEAICE-NH_v2r2_GW1_s201804151014190_e201804151150170_c201804151309570.nc │   ├── AMSR2-SEAICE-SH_v2r2_GW1_s201804150835180_e201804151014170_c201804151052280.nc │   └── AMSR2-SEAICE-SH_v2r2_GW1_s201804151014190_e201804151150170_c201804151309570.nc ├── sss └── sst ├── 20180415095000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc ├── 20180415114000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc └── 20180415132000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc ``` - Use of the `*` wildcard instead of `?` to simplify the obs file selection - Addition of binning for the sst retrieval - The `soca` # needs to be more recent. - This wasn't planned as part of this work, but the bump localization was having issues, so I replaced it with `diffusion` ### Issues addressed - fixes #869 - fixes #874
1 parent 9ba6bb9 commit 5549447

16 files changed

+53
-92
lines changed

‎.gitmodules

+1
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@
7272
[submodule "sorc/crtm"]
7373
path = sorc/crtm
7474
url = https://github.com/jcsda/crtm.git
75+
branch = release/crtm_jedi_v2.4.1
7576
[submodule "sorc/fms"]
7677
path = sorc/fms
7778
url = https://github.com/jcsda/fms.git

‎parm/soca/berror/saber_blocks.yaml

+17-8
Original file line numberDiff line numberDiff line change
@@ -74,15 +74,24 @@ components:
7474
localization:
7575
localization method: SABER
7676
saber central block:
77-
saber block name: BUMP_NICAS
77+
saber block name: EXPLICIT_DIFFUSION
78+
active variables: [tocn, socn, ssh]
79+
geometry:
80+
mom6_input_nml: mom_input.nml
81+
fields metadata: ./fields_metadata.yaml
82+
group mapping:
83+
- name: ocean
84+
variables: [tocn, socn, socn, ssh]
85+
- name: ice
86+
variables: [cicen]
7887
read:
79-
io:
80-
data directory: bump
81-
drivers:
82-
multivariate strategy: duplicated
83-
read local nicas: true
84-
model:
85-
do not cross mask boundaries: false
88+
groups:
89+
- name: ocean
90+
horizontal:
91+
filename: hz_ocean.nc
92+
- name: ice
93+
horizontal:
94+
filename: hz_ice.nc
8695

8796
weight:
8897
read_from_file: 3

‎parm/soca/obs/obs_list.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
observers:
22
- !INC ${OBS_YAML_DIR}/adt_rads_all.yaml
3-
- !INC ${OBS_YAML_DIR}/sst_metopb_l3u.yaml
3+
#- !INC ${OBS_YAML_DIR}/sst_metopb_l3u.yaml
44
- !INC ${OBS_YAML_DIR}/icec_amsr2_north.yaml
55
- !INC ${OBS_YAML_DIR}/icec_amsr2_south.yaml

‎parm/soca/obsprep/obsprep_config.yaml

+15-12
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,44 @@
11
observations:
22
- obs space:
33
name: sss_smap
4-
dmpdir subdir: SSS
5-
dmpdir regex: SMAP_L2B_SSS_NRT_?????_[AD]_????????T??????.h5
4+
dmpdir subdir: ocean/sss
5+
dmpdir regex: SMAP_L2B_SSS_NRT_*.h5
66
provider: SMAP
77
output file: sss_smap.ioda.nc
88
- obs space:
99
name: sss_smos
1010
provider: SMOS
11-
dmpdir subdir: SSS
11+
dmpdir subdir: ocean/sss
1212
output file: sss_smos.ioda.nc
13-
dmpdir regex: SM_OPER_MIR_OSUDP2_????????T??????_????????T??????_700_001_1.nc
13+
dmpdir regex: SM_OPER_MIR_OSUDP2_*.nc
1414
- obs space:
1515
name: adt_rads_all
16-
dmpdir subdir: ADT
17-
dmpdir regex: rads_adt_??_???????.nc
16+
dmpdir subdir: ocean/adt
17+
dmpdir regex: rads_adt_*.nc
1818
provider: RADS
1919
output file: adt_rads_all.nc4
2020
- obs space:
2121
name: icec_amsr2_north
2222
provider: AMSR2
23-
dmpdir subdir: icec
23+
dmpdir subdir: ocean/icec
2424
output file: icec_amsr2_north.ioda.nc
25-
dmpdir regex: AMSR2-SEAICE-NH_v2r2_GW1_s???????????????_e???????????????_c???????????????.nc
25+
dmpdir regex: AMSR2-SEAICE-NH_v2r2_GW1_s*.nc
2626
- obs space:
2727
name: icec_amsr2_south
2828
provider: AMSR2
29-
dmpdir subdir: icec
29+
dmpdir subdir: ocean/icec
3030
output file: icec_amsr2_south.ioda.nc
31-
dmpdir regex: AMSR2-SEAICE-SH_v2r2_GW1_s???????????????_e???????????????_c???????????????.nc
31+
dmpdir regex: AMSR2-SEAICE-SH_v2r2_GW1_s*.nc
3232
- obs space:
3333
name: sst_metopb_l3u
3434
provider: GHRSST
35-
dmpdir subdir: 'sst'
35+
dmpdir subdir: 'ocean/sst'
3636
output file: sst_metopb_l3u.ioda.nc
37-
dmpdir regex: '??????????????-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc'
37+
dmpdir regex: '*-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V*.nc'
3838
bounds:
3939
units: C
4040
min: -3.0
4141
max: 50.0
42+
binning:
43+
stride: 5
44+
min number of obs: 10

‎scripts/exgdas_global_marine_analysis_bmat.sh

+1-33
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# - generates the DA grid
1414
# - computes diagonal of B based on the background if a std. dev. file
1515
# was not staged. TODO: Remove this option in the future
16-
# - creates the bump correlation operators
16+
# - initialize the loacalization and correlation operator
1717
#
1818
# $Id$
1919
#
@@ -110,38 +110,6 @@ if [ $err -gt 0 ]; then
110110
exit $err
111111
fi
112112

113-
################################################################################
114-
# Correlation and Localization operators
115-
shopt -s nullglob
116-
files=(./bump/*.nc)
117-
echo $files
118-
if [ ${#files[@]} -gt 0 ]; then
119-
echo "BUMP/NICAS correlation and localization already staged, skipping BUMP initialization"
120-
set +x
121-
if [ $VERBOSE = "YES" ]; then
122-
echo $(date) EXITING $0 with return code $err >&2
123-
fi
124-
exit $err # Exit early, we're done with B
125-
shopt -u nullglob
126-
fi
127-
128-
################################################################################
129-
# Set localization scales for the hybrid en. var.
130-
$APRUN_OCNANAL $JEDI_BIN/soca_setcorscales.x soca_setlocscales.yaml
131-
export err=$?; err_chk
132-
if [ $err -gt 0 ]; then
133-
exit $err
134-
fi
135-
136-
################################################################################
137-
# Compute convolution coefs for L
138-
clean_yaml soca_bump_loc.yaml
139-
$APRUN_OCNANAL $JEDI_BIN/soca_error_covariance_toolbox.x soca_bump_loc.yaml
140-
export err=$?; err_chk
141-
if [ $err -gt 0 ]; then
142-
exit $err
143-
fi
144-
145113
################################################################################
146114
set +x
147115
if [ $VERBOSE = "YES" ]; then

‎scripts/exgdas_global_marine_analysis_prep.py

-18
Original file line numberDiff line numberDiff line change
@@ -401,24 +401,6 @@ def find_clim_ens(input_date):
401401
config = Template.substitute_structure(config, TemplateConstants.DOUBLE_CURLY_BRACES, envconfig.get)
402402
config.save(diffu_vt_yaml)
403403

404-
################################################################################
405-
# generate yaml for bump/nicas (used for correlation and/or localization)
406-
407-
logger.info(f"---------------- generate BUMP/NICAS localization yamls")
408-
# localization bump yaml
409-
bumpdir = 'bump'
410-
ufsda.disk_utils.mkdir(os.path.join(anl_dir, bumpdir))
411-
bump_yaml = os.path.join(anl_dir, 'soca_bump_loc.yaml')
412-
bump_yaml_template = os.path.join(gdas_home,
413-
'parm',
414-
'soca',
415-
'berror',
416-
'soca_bump_loc.yaml')
417-
config = YAMLFile(path=bump_yaml_template)
418-
config = Template.substitute_structure(config, TemplateConstants.DOUBLE_CURLY_BRACES, envconfig.get)
419-
config = Template.substitute_structure(config, TemplateConstants.DOLLAR_PARENTHESES, envconfig.get)
420-
config.save(bump_yaml)
421-
422404
################################################################################
423405
# generate yaml for soca_var
424406

‎sorc/fv3-jedi

‎sorc/ioda

‎sorc/iodaconv

Submodule iodaconv updated 25 files

‎sorc/land-imsproc

‎sorc/oops

Submodule oops updated 33 files

‎sorc/saber

Submodule saber updated 80 files

‎sorc/soca

‎sorc/ufo

Submodule ufo updated 26 files

‎test/soca/gw/prepdata.sh

+9-9
Original file line numberDiff line numberDiff line change
@@ -19,23 +19,23 @@ project_source_dir=$1
1919
cdldir=${project_source_dir}/test/soca/testdata
2020

2121
# GHRSST
22-
cdl2nc4 sst/20180415114000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc \
22+
cdl2nc4 ocean/sst/20180415114000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc \
2323
${cdldir}/20180415114000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.cdl
24-
cdl2nc4 sst/20180415132000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc \
24+
cdl2nc4 ocean/sst/20180415132000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc \
2525
${cdldir}/20180415132000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.cdl
26-
cdl2nc4 sst/20180415095000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc \
26+
cdl2nc4 ocean/sst/20180415095000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.nc \
2727
${cdldir}/20180415095000-STAR-L3U_GHRSST-SSTsubskin-AVHRRF_MB-ACSPO_V2.80-v02.0-fv01.0.cdl
2828

2929
# Fake AMSR2 icec
30-
cdl2nc4 icec/AMSR2-SEAICE-NH_v2r2_GW1_s201804150835180_e201804151014170_c201804151052280.nc \
30+
cdl2nc4 ocean/icec/AMSR2-SEAICE-NH_v2r2_GW1_s201804150835180_e201804151014170_c201804151052280.nc \
3131
${project_source_dir}/test/soca/testdata/icec_amsr2_north_1.cdl
32-
cdl2nc4 icec/AMSR2-SEAICE-NH_v2r2_GW1_s201804151014190_e201804151150170_c201804151309570.nc \
32+
cdl2nc4 ocean/icec/AMSR2-SEAICE-NH_v2r2_GW1_s201804151014190_e201804151150170_c201804151309570.nc \
3333
${project_source_dir}/test/soca/testdata/icec_amsr2_north_2.cdl
34-
cdl2nc4 icec/AMSR2-SEAICE-SH_v2r2_GW1_s201804150835180_e201804151014170_c201804151052280.nc \
34+
cdl2nc4 ocean/icec/AMSR2-SEAICE-SH_v2r2_GW1_s201804150835180_e201804151014170_c201804151052280.nc \
3535
${project_source_dir}/test/soca/testdata/icec_amsr2_south_1.cdl
36-
cdl2nc4 icec/AMSR2-SEAICE-SH_v2r2_GW1_s201804151014190_e201804151150170_c201804151309570.nc \
36+
cdl2nc4 ocean/icec/AMSR2-SEAICE-SH_v2r2_GW1_s201804151014190_e201804151150170_c201804151309570.nc \
3737
${project_source_dir}/test/soca/testdata/icec_amsr2_south_2.cdl
3838

3939
# Fake RADS ADT
40-
cdl2nc4 ADT/rads_adt_3a_2018105.nc ${project_source_dir}/test/soca/testdata/rads_adt_3a_2018105.cdl
41-
cdl2nc4 ADT/rads_adt_3b_2018105.nc ${project_source_dir}/test/soca/testdata/rads_adt_3b_2018105.cdl
40+
cdl2nc4 ocean/adt/rads_adt_3a_2018105.nc ${project_source_dir}/test/soca/testdata/rads_adt_3a_2018105.cdl
41+
cdl2nc4 ocean/adt/rads_adt_3b_2018105.nc ${project_source_dir}/test/soca/testdata/rads_adt_3b_2018105.cdl

‎test/soca/gw/setup_obsprep.sh

+1-3
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,7 @@ mkdir -p ${test_dmpdir}
1111

1212
cd ${test_dmpdir}
1313

14-
mkdir SSS ADT icec sst
14+
mkdir -p ocean/sss ocean/adt ocean/icec ocean/sst
1515

1616
${project_source_dir}/test/soca/gw/prepdata.sh ${project_source_dir}
1717

18-
19-

0 commit comments

Comments
 (0)