Skip to content

Commit 3683455

Browse files
davemfishdavemfish
committed
merged upstream main and resolved trivial conflict
2 parents edfda1a + 9973dad commit 3683455

File tree

75 files changed

+4590
-3141
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

75 files changed

+4590
-3141
lines changed

‎.github/workflows/build-and-test.yml

+77-67
Large diffs are not rendered by default.

‎.readthedocs.yml

+4
Original file line numberDiff line numberDiff line change
@@ -18,5 +18,9 @@ build:
1818
tools:
1919
python: "mambaforge-4.10"
2020
jobs:
21+
post_create_environment:
22+
- pip install --upgrade-strategy=only-if-needed -r requirements.txt
23+
- pip install --upgrade-strategy=only-if-needed -r requirements-dev.txt
24+
- pip install --upgrade-strategy=only-if-needed -r requirements-docs.txt
2125
post_install:
2226
- make install

‎.readthedocs_environment.yml

+1-5
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,5 @@ channels:
1111
- nodefaults
1212
dependencies:
1313
- python=3.11
14-
- gdal>=3.4.2,<3.6.0
14+
- gdal>=3.4.2
1515
- pip
16-
- pip:
17-
- -r requirements.txt
18-
- -r requirements-dev.txt
19-
- -r requirements-docs.txt

‎HISTORY.rst

+121-1
Original file line numberDiff line numberDiff line change
@@ -35,23 +35,143 @@
3535

3636
.. :changelog:
3737
38-
3938
Unreleased Changes
4039
------------------
40+
* General
41+
* InVEST has been updated to build against numpy 2.
42+
https://github.com/natcap/invest/issues/1641
43+
* Updating validation to handle a change in exceptions raised by GDAL in
44+
``pygeoprocessing.get_raster_info`` and
45+
``pygeoprocessing.get_vector_info``.
46+
https://github.com/natcap/invest/issues/1645
47+
* Workbench
48+
* Several small updates to the model input form UI to improve usability
49+
and visual consistency (https://github.com/natcap/invest/issues/912).
50+
* Fixed a bug that caused the application to crash when attempting to
51+
open a workspace without a valid logfile
52+
(https://github.com/natcap/invest/issues/1598).
53+
* Fixed a bug that was allowing readonly workspace directories on Windows
54+
(https://github.com/natcap/invest/issues/1599).
55+
* Fixed a bug that, in certain scenarios, caused a datastack to be saved
56+
with relative paths when the Relative Paths checkbox was left unchecked
57+
(https://github.com/natcap/invest/issues/1609).
58+
* Improved error handling when a datastack cannot be saved with relative
59+
paths across drives (https://github.com/natcap/invest/issues/1608).
60+
* Coastal Vulnerability
61+
* Fixed a regression where an AOI with multiple features could raise a
62+
TypeError after intersecting with the landmass polygon.
63+
https://github.com/natcap/invest/issues/1657
64+
* Forest Carbon Edge Effects
65+
* Updating vector reprojection to allow partial reprojection. Related to
66+
https://github.com/natcap/invest/issues/1645
67+
* Habitat Quality
68+
* Access raster is now generated from the reprojected access vector
69+
(https://github.com/natcap/invest/issues/1615).
70+
* Rarity values are now output in CSV format (as well as in raster format)
71+
(https://github.com/natcap/invest/issues/721).
72+
* Urban Flood Risk
73+
* Fields present on the input AOI vector are now retained in the output.
74+
(https://github.com/natcap/invest/issues/1600)
75+
* Urban Nature Access
76+
* The model now works as expected when the user provides an LULC raster
77+
that does not have a nodata value defined.
78+
https://github.com/natcap/invest/issues/1293
79+
80+
3.14.2 (2024-05-29)
81+
-------------------
82+
* General
83+
* Validation now covers file paths contained in CSVs. CSV validation
84+
will fail if the files listed in a CSV fail to validate.
85+
https://github.com/natcap/invest/issues/327
86+
* We have updated validation in several ways that will improve the
87+
developer experience of working with InVEST models, and we hope will also
88+
improve the user experience:
89+
90+
* Symbols in conditional requirement expressions now represent the values
91+
of parameters instead of whether the value of the parameter is
92+
sufficient. If a symbol is not present in ``args`` as a key, the
93+
symbol will have a value of ``False``. This allows for value-based
94+
comparisons, which is useful in models that have overlapping modes of
95+
operation. https://github.com/natcap/invest/issues/1509
96+
* Vector fields, CSV rows/columns and the 1st level of directory
97+
contents may now all be conditionally required based on a python
98+
expression.
99+
* Under certain circumstances, validation may return more warnings than
100+
before. This specifically applies to model inputs that have conditional
101+
requirement expressions where their expression evaluates to ``False``,
102+
and the user has provided a value for this parameter. Previous
103+
versions of InVEST would skip these parameters' type-specific
104+
validation. Now, these parameters will be validated with their
105+
type-specific validation checks.
106+
* Add support for latest GDAL versions; remove test-specific constraint on
107+
GDAL versions from invest requirements.
108+
https://github.com/natcap/invest/issues/916
109+
* Updated to Cython 3 (https://github.com/natcap/invest/issues/556)
110+
* Annual Water Yield
111+
* Added the results_suffix to a few intermediate files where it was
112+
missing. https://github.com/natcap/invest/issues/1517
113+
* Coastal Blue Carbon
114+
* Updated model validation to prevent the case where a user provides only
115+
one snapshot year and no analysis year
116+
(`#1534 <https://github.com/natcap/invest/issues/1534>`_).
117+
Also enforces that the analysis year, if provided, is greater than the
118+
latest snapshot year. An analysis year equal to the latest snapshot year
119+
is no longer allowed.
120+
* Coastal Vulnerability
121+
* Fixed a bug in handling ``nan`` as the nodata value of the bathymetry
122+
raster. ``nan`` pixels will now be propertly ignored before calculating
123+
mean depths along fetch rays.
124+
https://github.com/natcap/invest/issues/1528
125+
* HRA
126+
* Fixed a bug where habitat and stressor vectors were not being rasterized
127+
with the `ALL_TOUCHED=TRUE` setting.
128+
* Scenic Quality
129+
* Fixed an issue with viewshed calculations where some slight numerical
130+
error was introduced on M1 Macs, but not on x86-based computers. This
131+
numerical error was leading to slightly different visibility results.
132+
https://github.com/natcap/invest/issues/1562
133+
* SDR
134+
* Fixed an issue encountered in the sediment deposition function where
135+
rasters with more than 2^32 pixels would raise a cryptic error relating
136+
to negative dimensions. https://github.com/natcap/invest/issues/1431
137+
* Optimized the creation of the summary vector by minimizing the number of
138+
times the target vector needs to be rasterized.
139+
* Seasonal Water Yield
140+
* Fixed an issue with the precip directory units. Units for these input
141+
rasters are now correctly stated as mm/month.
142+
https://github.com/natcap/invest/issues/1571
143+
* Fixed an issue where the monthly quickflow values were being summed over
144+
a block area and not summed pixelwise. This caused the quickflow
145+
output ``QF.tif`` to have malformed values.
146+
https://github.com/natcap/invest/issues/1541
147+
* Wind Energy
148+
* Fixed a bug where some number inputs were not being properly cast to
149+
``float`` or ``int`` types. If the inputs happened to be passed as
150+
a ``str`` this caused unintended side effects such as a concatenation
151+
error. (https://github.com/natcap/invest/issues/1498)
41152
* Urban Nature Access
42153
* Fixed a ``NameError`` that occurred when running the model using
43154
search radii defined per population group with an exponential search
44155
kernel. https://github.com/natcap/invest/issues/1502
45156
* Fixed an issue where Urban Nature Access would crash if an administrative
46157
boundary geometry did not overlap any people in the population raster.
47158
https://github.com/natcap/invest/issues/1503
159+
* Fixed an issue where validation was failing to catch missing values in
160+
the uniform search radius args key when using uniform search radii.
161+
https://github.com/natcap/invest/issues/1509
48162
* Fixed an issue where the output administrative units vector's
49163
``Pund_adm`` and ``Povr_adm`` fields representing undersupplied and
50164
oversupplied populations, respectively, had values of 0 when running the
51165
model with search radii defined per population group. The output
52166
administrative units vector now has the correct values for these fields,
53167
consistent with the user's guide chapter.
54168
https://github.com/natcap/invest/issues/1512
169+
* Fixed an issue where certain nodata values were not being handled
170+
correctly, leading to pixel values of +/- infinity in the urban nature
171+
balance output raster. https://github.com/natcap/invest/issues/1519
172+
* Fixed an issue where an LULC raster without a nodata value would
173+
always raise in exception during reclassification.
174+
https://github.com/natcap/invest/issues/1539
55175

56176
3.14.1 (2023-12-18)
57177
-------------------

‎Makefile

+4-3
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,15 @@
22
DATA_DIR := data
33
GIT_SAMPLE_DATA_REPO := https://bitbucket.org/natcap/invest-sample-data.git
44
GIT_SAMPLE_DATA_REPO_PATH := $(DATA_DIR)/invest-sample-data
5-
GIT_SAMPLE_DATA_REPO_REV := 2e7cd618c661ec3f3b2a3bddfd2ce7d4704abc05
5+
GIT_SAMPLE_DATA_REPO_REV := ab8c74a62a93fd0019de2bca064abc0a5a07afab
66

77
GIT_TEST_DATA_REPO := https://bitbucket.org/natcap/invest-test-data.git
88
GIT_TEST_DATA_REPO_PATH := $(DATA_DIR)/invest-test-data
9-
GIT_TEST_DATA_REPO_REV := da013683e80ea094fbb2309197e2488c02794da8
9+
GIT_TEST_DATA_REPO_REV := 324abde73e1d770ad75921466ecafd1ec6297752
1010

1111
GIT_UG_REPO := https://github.com/natcap/invest.users-guide
1212
GIT_UG_REPO_PATH := doc/users-guide
13-
GIT_UG_REPO_REV := fa6b181d49136089dce56d4ff8f3dcaf12eb4ced
13+
GIT_UG_REPO_REV := f203ec069f9f03560c9a85b268e67ebb6b994953
1414

1515
ENV = "./env"
1616
ifeq ($(OS),Windows_NT)
@@ -251,6 +251,7 @@ $(INVEST_BINARIES_DIR): | $(DIST_DIR) $(BUILD_DIR)
251251
-$(RMDIR) $(INVEST_BINARIES_DIR)
252252
$(PYTHON) -m PyInstaller --workpath $(BUILD_DIR)/pyi-build --clean --distpath $(DIST_DIR) exe/invest.spec
253253
$(CONDA) list > $(INVEST_BINARIES_DIR)/package_versions.txt
254+
$(PYTHON) -m pip list >> $(INVEST_BINARIES_DIR)/package_versions.txt
254255
$(INVEST_BINARIES_DIR)/invest list
255256

256257
# Documentation.

‎constraints_tests.txt

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# This file contains package constraints needed to run the invest test suite.
2+
# It follows the pip constraints file format:
3+
# https://pip.pypa.io/en/stable/user_guide/#constraints-files
4+
5+
# A gdal bug caused our test suite to fail, but this issue is unlikely to
6+
# occur with regular use of invest. https://github.com/OSGeo/gdal/issues/8497
7+
GDAL!=3.6.*,!=3.7.*
8+
9+
# Pyinstaller 6.10 breaks our windows builds. Until we can figure out the
10+
# root cause, let's cap the versions to those that work.
11+
# https://github.com/natcap/invest/issues/1622
12+
#pyinstaller<6.10

‎doc/api-docs/conf.py

+5-7
Original file line numberDiff line numberDiff line change
@@ -78,13 +78,11 @@
7878
# -- Options for HTML output ----------------------------------------------
7979

8080
import sphinx_rtd_theme
81+
8182
# The theme to use for HTML and HTML Help pages. See the documentation for
8283
# a list of builtin themes.
8384
html_theme = 'sphinx_rtd_theme'
8485

85-
# Add any paths that contain custom themes here, relative to this directory.
86-
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
87-
8886
# The name of an image file (relative to this directory) to place at the top
8987
# of the sidebar.
9088
html_logo = "_static/invest-logo.png"
@@ -130,16 +128,16 @@
130128
# dir menu entry, description, category)
131129
texinfo_documents = [
132130
('index', 'InVEST', 'InVEST Documentation',
133-
'The Natural Capital Project', 'InVEST',
131+
'The Natural Capital Project', 'InVEST',
134132
'Integrated Valuation of Ecosystem Services and Tradeoffs',
135133
'Scientific Software'),
136134
]
137135

138136

139137
# -- Prepare for sphinx build ---------------------------------------------
140138

141-
# Use sphinx apidoc tool to generate documentation for invest. Generated rst
142-
# files go into the api/ directory. Note that some apidoc options may not work
139+
# Use sphinx apidoc tool to generate documentation for invest. Generated rst
140+
# files go into the api/ directory. Note that some apidoc options may not work
143141
# the same because we aren't using their values in the custom templates
144142
apidoc.main([
145143
'--force', # overwrite any files from previous run
@@ -164,7 +162,7 @@
164162
165163
All InVEST models share a consistent python API:
166164
167-
- Every InVEST model has a corresponding module or subpackage in the
165+
- Every InVEST model has a corresponding module or subpackage in the
168166
``natcap.invest`` package
169167
- The model modules contain a function called ``execute``
170168
- The ``execute`` function takes a single argument (``args``), a dictionary

‎docker/Dockerfile

+5-1
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,15 @@ RUN cd / && \
1313

1414
# Create the container for distribution that has runtime dependencies.
1515
FROM mambaorg/micromamba:1.5.0-bookworm-slim
16+
# Python version should match the version used in stage 1.
17+
# If we update the stage 1 debian version, also update this python version
18+
ARG PYTHON_VERSION="3.11"
1619
COPY --from=build /invest/dist/*.whl /tmp/
1720

1821
# The environment.yml file will be built during github actions.
1922
COPY --chown=$MAMBA_USER:$MAMBA_USER environment.yml /tmp/environment.yml
20-
RUN micromamba install -y -n base -c conda-forge -f /tmp/environment.yml && \
23+
RUN micromamba install -y -n base -c conda-forge python==${PYTHON_VERSION} && \
24+
micromamba install -y -n base -c conda-forge -f /tmp/environment.yml && \
2125
micromamba clean --all --yes && \
2226
/opt/conda/bin/python -m pip install /tmp/*.whl && \
2327
/opt/conda/bin/python -m pip cache purge && \

‎exe/hooks/rthook.py

+1-4
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
1-
import sys
21
import os
3-
import multiprocessing
42
import platform
5-
6-
multiprocessing.freeze_support()
3+
import sys
74

85
os.environ['PROJ_LIB'] = os.path.join(sys._MEIPASS, 'proj')
96

‎exe/invest.spec

+3
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,9 @@ kwargs = {
3030
'pkg_resources.py2_warn',
3131
'cmath',
3232
'charset_normalizer',
33+
'scipy.special._cdflib',
34+
'scipy.special._special_ufuncs',
35+
'scipy._lib.array_api_compat.numpy.fft',
3336
],
3437
'datas': [proj_datas],
3538
'cipher': block_cipher,

‎pyproject.toml

+13-4
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
name = "natcap.invest"
33
description = "InVEST Ecosystem Service models"
44
readme = "README_PYTHON.rst"
5-
requires-python = ">=3.8,<3.12"
5+
requires-python = ">=3.8"
66
license = {file = "LICENSE.txt"}
77
maintainers = [
88
{name = "Natural Capital Project Software Team"}
@@ -21,6 +21,7 @@ classifiers = [
2121
"Programming Language :: Python :: 3.9",
2222
"Programming Language :: Python :: 3.10",
2323
"Programming Language :: Python :: 3.11",
24+
"Programming Language :: Python :: 3.12",
2425
"Programming Language :: Cython",
2526
"License :: OSI Approved :: BSD License",
2627
"Topic :: Scientific/Engineering :: GIS"
@@ -44,8 +45,9 @@ invest = "natcap.invest.cli:main"
4445
# that we can provide a much easier build experience so long as GDAL is
4546
# available at runtime.
4647
requires = [
47-
'setuptools>=61', 'wheel', 'setuptools_scm>=8.0', 'cython', 'babel',
48-
'oldest-supported-numpy'
48+
'setuptools>=61', 'wheel', 'setuptools_scm>=8.0', 'cython>=3.0.0', 'babel',
49+
'oldest-supported-numpy; python_version<="3.8"',
50+
'numpy>=2; python_version>="3.9"', # numpy 2 only available for 3.9+
4951
]
5052
build-backend = "setuptools.build_meta"
5153

@@ -78,4 +80,11 @@ where = ["src"]
7880

7981
[tool.pytest.ini_options]
8082
# raise warnings to errors, except for deprecation warnings
81-
filterwarnings = ["error", "default::DeprecationWarning"]
83+
filterwarnings = [
84+
"error",
85+
"default::DeprecationWarning",
86+
"default::FutureWarning",
87+
# don't error on a specific runtime warning coming from a shapely
88+
# issue on M1: https://github.com/natcap/invest/issues/1562
89+
"default:invalid value encountered in intersection:RuntimeWarning",
90+
]

‎requirements-dev.txt

+6-3
Original file line numberDiff line numberDiff line change
@@ -9,15 +9,18 @@
99
# Any lines with "# pip-only" at the end will be processed by
1010
# scripts/convert-requirements-to-conda-yml.py as though it can only be found
1111
# on pip.
12+
# Sometimes conda-forge does not install the latest available version,
13+
# pip-only can be a workaround for that.
1214

13-
Cython<3.0.0
1415
virtualenv>=12.0.1
1516
pytest
1617
pytest-subtests
1718
wheel>=0.27.0
1819
pypiwin32; sys_platform == 'win32' # pip-only
19-
setuptools>=8.0,<60.7.0 # https://github.com/pyinstaller/pyinstaller/issues/6564
20-
PyInstaller>=4.10
20+
21+
# 60.7.0 exception because of https://github.com/pyinstaller/pyinstaller/issues/6564
22+
setuptools>=8.0,!=60.7.0
23+
PyInstaller>=4.10 # pip-only
2124
setuptools_scm>=6.4.0
2225
requests
2326
coverage

‎requirements.txt

+2-2
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@ pandas>=1.2.1
1616
numpy>=1.11.0,!=1.16.0
1717
Rtree>=0.8.2,!=0.9.1
1818
shapely>=2.0.0
19-
scipy>=1.9.0
20-
pygeoprocessing>=2.4.2 # pip-only
19+
scipy>=1.9.0,!=1.12.*
20+
pygeoprocessing>=2.4.6 # pip-only
2121
taskgraph>=0.11.0
2222
psutil>=5.6.6
2323
chardet>=3.0.4

‎scripts/invest-autovalidate.py

+6-6
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
#!python
22

3-
import os
4-
import tempfile
5-
import logging
63
import argparse
7-
import unittest
84
import glob
95
import importlib
10-
import shutil
6+
import logging
7+
import os
118
import pprint
9+
import shutil
10+
import tempfile
11+
import unittest
1212

1313
from natcap.invest import datastack
1414

@@ -81,7 +81,7 @@ def main(sampledatadir):
8181

8282
model_warnings = [] # define here in case of uncaught exception.
8383
try:
84-
LOGGER.info('validating %s ', datastack_path)
84+
LOGGER.info('validating %s ', os.path.abspath(datastack_path))
8585
model_warnings = getattr(
8686
model_module, 'validate')(paramset.args)
8787
except AttributeError as err:

0 commit comments

Comments
 (0)