hydro convert ctd zip_exchange_to_zip_netcdf

[DocOtak]

Original report by see (Bitbucket: sescher, GitHub: sescher).

---

one ct1.csv file in the exchange ct1.zip file contains the line

DEPTH = 5129.000000

the online documentation for DEPTH states that the value is %11s

the command
hydro convert ctd zip_exchange_to_zip_netcdf
aborts when it reaches this statement.

#!python

1503:19,965 W formats/netcdf.py:303 create_and_fill_data_variables      Parameter FLUOR does not have a format string acceptable for numeric data. Defaulting to '%f' to prevent ncdump segfault.
1503:20,12  C scripts.py:2829 main      Traceback (most recent call last):
  File "/usr/local/cchdo/hydro-vm/src/libcchdo/scripts.py", line 2827, in main
    hydro_parser.exit(args.main(args))
  File "/usr/local/cchdo/hydro-vm/src/libcchdo/scripts.py", line 780, in ctdzip_exchange_to_ctdzip_netcdf
    ctdzipnc.write(dfc, out_file)
  File "/usr/local/cchdo/hydro-vm/src/libcchdo/formats/ctd/zip/netcdf.py", line 46, in write
    zipnc.write(self, handle, 'ctd', ctdnc, zipnc.get_identifier_ctd)
  File "/usr/local/cchdo/hydro-vm/src/libcchdo/formats/zip_netcdf.py", line 66, in write
    Zip.write(self, handle, writer, counting_fname.get_filename, **kwargs)
  File "/usr/local/cchdo/hydro-vm/src/libcchdo/formats/zip.py", line 189, in write
    writer.write(dfile, tempfile, **kwargs)
  File "/usr/local/cchdo/hydro-vm/src/libcchdo/formats/ctd/netcdf.py", line 183, in write
    int(self.globals.get('DEPTH', FILL_VALUE)),
ValueError: invalid literal for int() with base 10: '5129.000000'

[DocOtak]

Original comment by abarna (Bitbucket: abarna, GitHub: abarna).

---

Format traceback

[DocOtak]

Original comment by see (Bitbucket: sescher, GitHub: sescher).

---

I just looked at the proposed ctd specific documentation, and it says DEPTH is I4.

-sharon

[DocOtak]

Original comment by abarna (Bitbucket: abarna, GitHub: abarna).

---

This can be worked around by removing the decimal from the offending file (so it is only 5129).

I don't know the "format" of the netcdf files other than "COARDS".

[DocOtak]

Original comment by abarna (Bitbucket: abarna, GitHub: abarna).

---

DEPTH in the current exchange doc (PDF on cchdo) says I5 as format.

Going forward should it be an integer or decimal number?

[DocOtak]

Original comment by see (Bitbucket: sescher, GitHub: sescher).

---

Yes, I will change the values to I4. I guess this is a documentation issue, which has been resolved in the proposed documentation.

[DocOtak]

Original comment by abarna (Bitbucket: abarna, GitHub: abarna).

---

The format on the proposed CTD spec should be I5 to ensure full ocean coverage (depths exceeding 10000 meters exist).

Should I close this issue or leave it open as something to develop a programatic solution for?

[DocOtak]

Original comment by abarna (Bitbucket: abarna, GitHub: abarna).

---

Correction, the format on the proposed spec should probably just be "integer" with no length specified.

[DocOtak]

Original comment by see (Bitbucket: sescher, GitHub: sescher).

---

Going Forward: Good Question!

1.   This is usually an estimate anyways,  eg  ctd depth + altimiter off bottom,   or knudson depth, or seabeam (most accurate)
2.   In the future,  DEPTH may be more accurate with some new device, thus a decimal will already be “ready"
3.   Probably safer to make it a decimal because a integer can be expressed accurately as a decimal, but not visa-versa
4.   will JOA or ODV break if this is a decimal?

[DocOtak]

Original comment by abarna (Bitbucket: abarna, GitHub: abarna).

---

JOA does not seem to break when it is a decimal, however it does not report the decimal part (but does display a .0). JOA does not round either, it seems to simply truncate and append the .0.

ODV appears to round the depth to the nearest integer value (and does not report a decimal).

I think we can safely set this to a 'decimal' type in the future exchange format spec.