started port to pyswmm

Author: abhiramm7

.DS_Store

@@ -1,5 +1,6 @@
-*.npy
 __pycache__/
 .ipynb_checkpoints
 .ropeproject
 .DS_Store
+.pyc
+.DS_Store

.gitignore

@@ -4,7 +4,9 @@ Source code and data used in the *Deep Reinforcement Learning for the Real Time
 
 ![RLagent](./data/RL_main_fig_1.png)
 
-## Dependencies
+## Dependencies 
+Code supports python 3.7 and python 2.7.
+
 Python dependencies for the project can be installed using **requirements.txt**
 
 Storm water network is simulated using EPA-SWMM and pyswmm/matswmm. Matswmm has been deprecated and we strongly suggest using pyswmm. 
@@ -13,6 +15,13 @@ pyswmm/matswmm makes function calls to a static c library. Hence, we advice gcc-
 
 ## Agents
 
+#### Classical Q Learning 
+
+Classical Q learning algorithm implementation for controlling the water level a single tank is provided in **classical_q**
+
+This version observers the water level and sets the gate position between 0-100.
+
+#### DQN
 There are two types of deep rl agents.
 1. Centralized controller that can observe multiple states across the network and control the ponds in the network 
 2. Localised controller that can observe the state of an individual assent and control it 
@@ -21,6 +30,11 @@ To use these agents to control a storm water network, you would just need to kno
 
 Refer to the example implementation for further details.
 
+#### Training
+
+University of Michigan's FLUX was used to train the agents. They can be run on any local computer too, but we **strongly** recommend a GPU.
+
+
 ## Data presented in the paper
 
 Weights of the neural network used for plots can be found in **./data**

README.md

@@ -5,7 +5,6 @@
 from keras.optimizers import RMSprop
 import itertools
 import sys
-import swmm
 
 # Simulation parameters
 epi_start = float(sys.argv[1])

centralized_controller.py

@@ -1,5 +1,4 @@
 import numpy as np
-#import matplotlib.pyplot as plt
 import swmm
 from keras.models import Sequential
 from keras.layers import Dense, Activation, Dropout
@@ -69,7 +68,6 @@ def build_network(input_states,
     model.compile(loss='mean_squared_error', optimizer=sgd)
     return model
 
-"""
 def plot_network(Ponds_network, components_tracking,
                  components_bookkeeping, figure_num=1, show=True):
     rows = len(components_tracking) + len(components_bookkeeping)
@@ -95,7 +93,6 @@ def plot_network(Ponds_network, components_tracking,
     # else:
     #     return fig
 
-"""
 # SWMM Network finder
 def swmm_states(Network, state):
     temp = []

single_tank_classical_q/q learning control.ipynb classical_q/q learning control.ipynb

@@ -0,0 +1,15 @@
+import numpy as np
+
+# Single tank reward functions 
+def reward1():
+    return pass
+
+def reward1():
+    return pass
+
+def reward1():
+    return pass
+
+# Reward Function: System Scale Control
+def reward_sys(depth, outflow, gate_postions_rate, flood):
+    return pass

lib/__init__.pyc

@@ -2,3 +2,4 @@ numpy
 matplotlib
 tensorflow
 keras
+pyswmm