Add validation rules for input rasters

[soaressgabriel]

Description

• The project utilizes a variety of raster datasets, each with specific data integrity requirements to ensure the quality and usability of the data for analysis and modeling;

• Currently, there is a need for a systematic approach to validate these rasters against predefined rules to identify and rectify any data inconsistencies;

• The specific rasters and their corresponding rules are as follows:

  • Digital Elevation Map (DEM):
    • None of the pixels in the raster must contain NO_DATA value;
    • Raster pixels cannot consist entirely of zero values;
    • Raster pixels cannot consist entirely of one values;
  • Mask of Catchment (Clone):
    • Raster pixels cannot consist entirely of zero values;
  • Normalized Difference Vegetation Index (NDVI):
    • None of the pixels in the raster must contain NO_DATA value.
  • Soil Map:
    • None of the pixels in the raster must contain NO_DATA value;
    • Raster pixels cannot consist entirely of zero values.
  • Stations Locations (Samples):
    • Raster pixels cannot consist entirely of zero values.
  • Potential Evapotranspiration (ETP):
    • None of the pixels in the raster must contain NO_DATA value.
  • Rainfall (PM):
    • None of the pixels in the raster must contain NO_DATA value.
  • Class A Pan Coefficient (KP):
    • None of the pixels in the raster must contain NO_DATA value.
  • Land Use Map-series:
    • None of the pixels in the raster must contain NO_DATA value;
    • Raster pixels cannot consist entirely of zero values.

Solution

• Implement a validation mechanism that checks each raster dataset against the following rules, as applicable:

  • FORBID_NO_DATA: Ensure no parts of the raster contain NO_DATA values;
  • FORBID_ALL_ZEROES: Ensure the raster does not consist entirely of zero values;
  • FORBID_ALL_ONES: Ensure the raster does not consist entirely of one values.

• The validation should be performed automatically as part of the data processing pipeline, flagging any datasets that violate their respective rules for further investigation or correction.

Alternative solution(s)

• N/A

Additional context

Raster	FORBID_NO_DATA	FORBID_ALL_ZEROES	FORBID_ALL_ONES
"dem"	✅	✅	✅
"clone"	✖️	✅	✖️
"ndvi"	✅	✖️	✖️
"soil"	✅	✅	✖️
"sample_locations"	✖️	✅	✅
"etp"	✅	✖️	✖️
"precipitation"	✅	✖️	✖️
"kp"	✅	✖️	✖️
"landuse"	✅	✅	✖️

Acknowledgements

• @LINAMARIAOSORIO