Generalized implementation of simple pumped hydro with an example/tes…

…t case. This implementation requires the sequence of simple_hydro, storage in modules.txt.

My goals were modeling accuracy, backwards compatibility, avoiding redundancy, checks to aid debugging of common errors, and allow pumped hydro to participate in reserves like any other generator.

Pumped hydro is different from normal hydro because of need to the track energy balance, and that average dispatch is not solely determined by average stream flow (but also includes storage decisions). It's also different from normal storage because energy tracking needs to include both stream inflow and any spilled water.

In this implementation, if any pumped hydro generators are specified, the simple_hydro module will validate that the storage module is also included. The pre-existing rules for maintaining average stream flow are restricted to non-pumped hydro. Special rules for pumped hydro energy tracking and streamflow are implemented with an extra few lines in the storage energy tracking constraint, which capture the mathematical requirements of average daily balancing and additionally tracks state-of-charge. If the pumped hydro terms are not available, the storage module will skip that stanza without complaint.

Storage documentation updates: Describe how StateOfCharge is unbound for one timepoint in each timeseries (that is, the optimization can choose to start a day with an arbitrary storage level), because I found myself having to re-read through equations to convince myself of that behavior. Explicit describe units of various energy-based components in the storage model (to reduce confusion between power & energy). Rewrap line endings to conform to PEP-8 recommendations for reading two files side-by-side in a single screen (or a single file in large font on a small screen)

examples/pumped_hydro_simple/README.md

@@ -0,0 +1,7 @@
+SYNOPSIS:
+
+	switch solve --verbose --log-run
+
+This example illustrates modeling pumped hydro storage by using the hydro_simple module in concert with the storage module. 
+
+This adds pumped hydro to the hydro_simple example, and illustrates pumped hydro being used for arbitrage within each example day.

examples/pumped_hydro_simple/inputs/financials.csv

@@ -0,0 +1,2 @@
+base_financial_year,interest_rate,discount_rate
+2015,0.07,0.05

examples/pumped_hydro_simple/inputs/fuel_cost.csv

@@ -0,0 +1,2 @@
+load_zone,fuel,period,fuel_cost
+South,NaturalGas,2020,4

examples/pumped_hydro_simple/inputs/fuels.csv

@@ -0,0 +1,2 @@
+fuel,co2_intensity,upstream_co2_intensity
+NaturalGas,0.05306,0

examples/pumped_hydro_simple/inputs/gen_build_costs.csv

@@ -0,0 +1,10 @@
+GENERATION_PROJECT,build_year,gen_overnight_cost,gen_fixed_om,gen_storage_energy_overnight_cost
+S-NG_CC,2000.0,1143900.0,5868.3,.
+S-Central_PV-1,2000.0,2334300.0,41850.0,.
+S-Geothermal,1998.0,5524200.0,0.0,.
+Hydro,2000.0,10000000.0,100000.0,.
+Hydro_RoR,2000.0,1000000.0,100000.0,.
+Hydro_Pumped,2000.0,1000000.0,100000.0,0.0
+S-Geothermal,2020.0,5524200.0,0.0,.
+S-NG_CC,2020.0,1143900.0,5868.3,.
+S-Central_PV-1,2020.0,2334300.0,41850.0,.

examples/pumped_hydro_simple/inputs/gen_build_predetermined.csv

@@ -0,0 +1,7 @@
+GENERATION_PROJECT,build_year,gen_predetermined_cap
+S-NG_CC,2000,5.0
+S-Central_PV-1,2000,1.0
+S-Geothermal,1998,1.0
+Hydro,2000,1.0
+Hydro_RoR,2000,1.0
+Hydro_Pumped,2000,5.0

examples/pumped_hydro_simple/inputs/generation_projects_info.csv

@@ -0,0 +1,7 @@
+GENERATION_PROJECT,gen_dbid,gen_tech,gen_load_zone,gen_connect_cost_per_mw,gen_capacity_limit_mw,gen_variable_om,gen_max_age,gen_min_build_capacity,gen_scheduled_outage_rate,gen_forced_outage_rate,gen_is_variable,gen_is_baseload,gen_is_cogen,gen_energy_source,gen_full_load_heat_rate,gen_is_pumped_hydro,gen_storage_efficiency
+S-Geothermal,33.0,Geothermal,South,134222.0,10.0,28.83,30,0,0.0075,0.0241,0,1,0,Geothermal,.,.,.
+S-NG_CC,34.0,NG_CC,South,57566.6,.,3.4131,20,0,0.04,0.06,0,0,0,NaturalGas,6.705,.,.
+S-Central_PV-1,41.0,Central_PV,South,74881.9,2.0,0.0,20,0,0.0,0.02,1,0,0,Solar,.,.,.
+Hydro,.,Hydro,South,0.0,1.0,0.1,100,0,0.019,0.05,0,0,0,Water,.,.,.
+Hydro_RoR,.,Hydro_RoR,South,0.0,1.0,0.0,30,0,0.019,0.05,1,0,0,Water,.,.,.
+Hydro_Pumped,.,Hydro_Pumped,South,0.0,5.0,0.1,100,0,0.019,0.05,0,0,0,Water,.,1.0,0.75

examples/pumped_hydro_simple/inputs/hydro_timeseries.csv

@@ -0,0 +1,5 @@
+hydro_project,timeseries,hydro_min_flow_mw,hydro_avg_flow_mw
+Hydro,2020_winter,0.6,0.75
+Hydro,2020_summer,0.2,0.6
+Hydro_Pumped,2020_winter,0.6,0.75
+Hydro_Pumped,2020_summer,0.2,0.6

examples/pumped_hydro_simple/inputs/load_zones.csv

@@ -0,0 +1,2 @@
+LOAD_ZONE,cost_multipliers,ccs_distance_km,dbid
+South,1,0,3

examples/pumped_hydro_simple/inputs/loads.csv

@@ -0,0 +1,5 @@
+LOAD_ZONE,TIMEPOINT,zone_demand_mw
+South,1,9.0
+South,2,2.5
+South,3,10.0
+South,4,4.0

examples/pumped_hydro_simple/inputs/modules.txt

@@ -0,0 +1,14 @@
+# Core Modules
+switch_model
+switch_model.timescales
+switch_model.financials
+switch_model.balancing.load_zones
+switch_model.energy_sources.properties
+switch_model.generators.core.build
+switch_model.generators.core.dispatch
+switch_model.reporting
+# Custom Modules
+switch_model.generators.core.no_commit
+switch_model.energy_sources.fuel_costs.simple
+switch_model.generators.extensions.hydro_simple
+switch_model.generators.extensions.storage

examples/pumped_hydro_simple/inputs/non_fuel_energy_sources.csv

@@ -0,0 +1,4 @@
+energy_source
+Solar
+Geothermal
+Water

examples/pumped_hydro_simple/inputs/periods.csv

@@ -0,0 +1,2 @@
+INVESTMENT_PERIOD,period_start,period_end
+2020,2017,2026

examples/pumped_hydro_simple/inputs/switch_inputs_version.txt

@@ -0,0 +1 @@
+2.0.5

examples/pumped_hydro_simple/inputs/timepoints.csv

@@ -0,0 +1,5 @@
+timepoint_id,timestamp,timeseries
+1,2025011512,2020_winter
+2,2025011600,2020_winter
+3,2025071512,2020_summer
+4,2025071600,2020_summer

examples/pumped_hydro_simple/inputs/timeseries.csv

@@ -0,0 +1,3 @@
+TIMESERIES,ts_period,ts_duration_of_tp,ts_num_tps,ts_scale_to_period
+2020_winter,2020,12,2,1826
+2020_summer,2020,12,2,1826

examples/pumped_hydro_simple/inputs/variable_capacity_factors.csv

@@ -0,0 +1,9 @@
+GENERATION_PROJECT,timepoint,gen_max_capacity_factor
+S-Central_PV-1,1,0.61
+S-Central_PV-1,2,0.0
+S-Central_PV-1,3,0.81
+S-Central_PV-1,4,0.0
+Hydro_RoR,1,0.25
+Hydro_RoR,2,0.5
+Hydro_RoR,3,0.2
+Hydro_RoR,4,0.4

examples/pumped_hydro_simple/outputs/total_cost.txt

@@ -0,0 +1 @@
+30423392.46

switch_model/generators/extensions/hydro_simple.py

@@ -22,11 +22,23 @@
 but the advanced framework would take longer to read and understand. To really
 take advantage of it, you'll also need more data than we usually have
 available.
+
+This module can model pumped hydro systems if the storage module is listed
+after simple_hydro in modules.txt, and pumped hydro generation projects are
+flagged via gen_is_pumped_hydro. The current implementation of pumped_hydro
+implicitly assumes that the lower reservoir always has sufficient water in it
+for pumping uphill into storage. Existing resevoir energy capacity can be
+constrained via gen_predetermined_storage_energy_mwh as needed. If existing
+reservoir energy capacity is never a binding constraint for day-to-day
+operations of pumped hydro, leave gen_predetermined_storage_energy_mwh
+unspecified, and set gen_storage_energy_overnight_cost to 0 and the
+optimization will set the energy value to a conveniently large value.
+
 """
 # ToDo: Refactor this code to move the core components into a
 # switch_model.hydro.core module, the simplest components into
 # switch_model.hydro.simple, and the advanced components into
-# switch_model.hydro.water_network. That should set a good example
+# switch_model.hydro.water_network. That could set a good example
 # for other people who want to do other custom handling of hydro.
 
 from __future__ import division
@@ -51,9 +63,19 @@ def define_components(mod):
     GENERATION_PROJECTS, and is determined by the inputs file
     hydro_timeseries.csv.
 
+    gen_is_pumped_hydro[g in GENERATION_PROJECTS] is an optional parameter
+    that denotes whether a hydro project includes pumped storage. To use this
+    pumped hydro implementation, you must include the storage module after
+    hydro_simple in modules.txt. The storage module will look for storage
+    generators flagged as pumped hydro and will define custom constraints for
+    their dispatch & storage that takes into account stream flow.
+
     HYDRO_GEN_TS is the set of Hydro projects and timeseries for which
     minimum and average flow are specified.
 
+    HYDRO_NONPUMPED_GEN_TS is a subset of HYDRO_GEN_TS for hydro projects that
+    do not include pumped storage, and is used to establish flow constraints.
+
     HYDRO_GEN_TPS is the set of Hydro projects and available
     dispatch points. This is a filtered version of GEN_TPS that
     only includes hydro projects.
@@ -76,7 +98,11 @@ def define_components(mod):
     Enforce_Hydro_Avg_Flow[(g, ts) in HYDRO_NONPUMPED_GEN_TS] is a constraint
     that enforces average flow levels across each timeseries. It requires the
     average of dispatched and spilled hydro over the course of a timeseries
-    must equal to the corresponding hydro_avg_flow_mw parameter.
+    must equal to the corresponding hydro_avg_flow_mw parameter. The
+    corresponding constraint for pumped hydro is defined in the storage
+    module (after storage decision variables are available), via an augmented
+    version of the Track_State_Of_Charge constraint that adds average incoming
+    streamflow and subtracts any spilled power.
     """
 
     mod.HYDRO_GEN_TS_RAW = Set(
@@ -87,13 +113,23 @@ def define_components(mod):
     )
     mod.HYDRO_GENS = Set(
         initialize=lambda m: set(g for (g, ts) in m.HYDRO_GEN_TS_RAW),
-        doc="Dispatchable hydro projects")
+        doc="Dispatchable hydro projects (both pumped & non-pumped)")
+    mod.gen_is_pumped_hydro = Param(
+        mod.GENERATION_PROJECTS, 
+        within=Boolean, 
+        default=False,
+        validate=lambda m, value, g: (value == False) or (g in m.HYDRO_GENS))
+
     mod.HYDRO_GEN_TS = Set(
         dimen=2,
         initialize=lambda m: set(
             (g, m.tp_ts[tp])
                 for g in m.HYDRO_GENS
                     for tp in m.TPS_FOR_GEN[g]))
+    mod.HYDRO_NONPUMPED_GEN_TS = Set(
+        dimen=2,
+        initialize=mod.HYDRO_GEN_TS,
+        filter=lambda m, g, ts: m.gen_is_pumped_hydro[g] == False)
     mod.HYDRO_GEN_TPS = Set(
         initialize=mod.GEN_TPS,
         filter=lambda m, g, t: g in m.HYDRO_GENS)
@@ -125,8 +161,8 @@ def _warn_on_extra_HYDRO_GEN_TS(m):
             "could indicate a benign issue where the process that built "
             "the dataset used simplified logic and/or didn't know the "
             "scheduled operating dates. If you expect those datapoints to "
-            "be useful, then those plants need to either come online earlier "
-            ", have longer lifetimes, or have options to build new capacity "
+            "be useful, then those plants need to either come online earlier, "
+            "have longer lifetimes, or have options to build new capacity "
             "when the old capacity reaches the provided end-of-life date."
             "\n".format(num_impacted_generators))
         if extra_indexes:
@@ -153,14 +189,26 @@ def _warn_on_extra_HYDRO_GEN_TS(m):
         mod.HYDRO_GEN_TPS,
         within=NonNegativeReals)
     mod.Enforce_Hydro_Avg_Flow = Constraint(
-        mod.HYDRO_GEN_TS,
+        mod.HYDRO_NONPUMPED_GEN_TS,
         rule=lambda m, g, ts: (
             sum(m.DispatchGen[g, t] + m.SpillHydro[g,t]
                 for t in m.TPS_IN_TS[ts]
             ) == m.hydro_avg_flow_mw[g, ts] * m.ts_num_tps[ts]))
 
     mod.min_data_check('hydro_min_flow_mw', 'hydro_avg_flow_mw')
 
+    def storage_module_avail_for_pumped_hydro_check(m):
+        no_pumped_hydro = all(
+            value(m.gen_is_pumped_hydro[g]) == False
+            for g in m.GENERATION_PROJECTS)
+        has_storage = (
+            'switch_model.generators.extensions.storage' in m.module_list)
+        return (no_pumped_hydro or has_storage)
+    mod.storage_module_avail_for_pumped_hydro = BuildCheck(
+        rule=storage_module_avail_for_pumped_hydro_check,
+        doc="Ensure that the user has included the storage module if they are"
+            "attempting to model pumped hydro.")
+
 
 def load_inputs(mod, switch_data, inputs_dir):
     """
@@ -177,6 +225,15 @@ def load_inputs(mod, switch_data, inputs_dir):
         hydro_generation_project, timeseries, hydro_min_flow_mw,
         hydro_avg_flow_mw
 
+    To model pumped hydro projects that use both river flows and pumped
+    storage, include the storage module in modules.txt after the hydro_simple
+    module. You will also need to populate the gen_is_pumped_hydro &
+    gen_storage_efficiency columns of generation_projects_info.tab for the
+    pumped hydro projects.
+
+    generation_projects_info.csv
+        GENERATION_PROJECT, ..., gen_is_pumped_hydro
+
     """
     switch_data.load_aug(
         optional=True,
@@ -185,3 +242,8 @@ def load_inputs(mod, switch_data, inputs_dir):
         index=mod.HYDRO_GEN_TS_RAW,
         param=(mod.hydro_min_flow_mw, mod.hydro_avg_flow_mw)
     )
+    switch_data.load_aug(
+        filename=os.path.join(inputs_dir, 'generation_projects_info.csv'),
+        auto_select=True,
+        optional_params=['gen_is_pumped_hydro'],
+        param=(mod.gen_is_pumped_hydro,))