You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Added dbimporter command to ant to aid users in batch creation of input databases.
This fixes#1 and is documented in \docs\CommandLineMOVES.md.
This commit also improves the ant command to convert old input databases to be compatible with MOVES3. All of the information provided in \database\ConversionScripts\InputDatabaseConversionHelp.pdf still applies to when this feature is used on the command line.
Copy file name to clipboardexpand all lines: docs/CommandLineMOVES.md
+63-26
Original file line number
Diff line number
Diff line change
@@ -2,84 +2,82 @@
2
2
3
3
## Introduction
4
4
5
-
While the easiest way to interact with MOVES is via its graphical user interface (GUI) it is sometimes useful to process MOVES runspecs in batches via the windows command line. To this end, MOVES has included a set of commands that can be accessed via the command line tool Ant. These commands can in turn be further scripted using standard scripting languages such as Windows Batch or PowerShell. This document provides a brief introduction to the MOVES ant commands and a very simple example batch script for running multiple MOVES runspecs in series. All of the examples presented here will use the windows command shell (cmd.exe)
5
+
While the easiest way to interact with MOVES is via its graphical user interface (GUI) it is sometimes useful to process MOVES RunSpecs in batches via the windows command line. To this end, MOVES has included a set of commands that can be accessed via the command line tool Ant. These commands can in turn be further scripted using standard scripting languages such as Windows Batch or PowerShell. This document provides a brief introduction to the MOVES ant commands and a very simple example batch script for running multiple MOVES RunSpecs in series. All of the examples presented here will use the windows command shell (cmd.exe)
6
6
7
7
## Setup the command environment
8
8
9
-
The first step to running MOVES from the command line is to setup the shell environment to point to all of the resources it needs to run MOVES. To do this, open the windows command prompt and navigate to the MOVES program folder (default: C:\Users\Public\EPA\MOVES3). Enter the command `setenv`:
9
+
The first step to running MOVES from the command line is to setup the shell environment to point to all of the resources it needs to run MOVES. To do this, open the windows command prompt and navigate to the MOVES directory. Enter the command `setenv`:
10
10
11
11
```cmd
12
-
C:\Users\Public\EPA\MOVES3>setenv
12
+
C:\Users\Public\EPA\MOVES\MOVES3>setenv
13
13
```
14
14
15
15
This should execute the setenv.bat script without displaying anything to the command window. The script tells the windows command shell where to find the Java runtime environment (JRE) bundled with MOVES, and where to find the Ant utility.
16
16
17
17
## View available MOVES Ant commands
18
18
19
-
After running setenv, MOVES ant commands should be available in the windows command shell for the duration of the shell session. To test this you can type `ant` at the command prompt:
19
+
After running `setenv`, MOVES ant commands should be available in the windows command shell for the duration of the shell session. To test this you can type `ant` at the command prompt:
20
20
21
21
```cmd
22
22
C:\Users\Public\EPA\MOVES\MOVES3>ant
23
23
```
24
24
25
25
This will display a list of the ant commands that are defined in the MOVES build.xml file. Each command listed will include a brief description of what it does. There are commands included for recompiling MOVES, testing the MOVES code, and for running MOVES in a number of ways. This document will focus only on the basic commands needed for executing MOVES runs.
26
26
27
-
NOTE: several of the commands require that `ant setlogin` be run first. If you installed MOVES using the MOVES installer you will not need to run `ant setlogin`
28
-
29
27
## Example calls to the MOVES Ant commands
30
28
31
-
To execute a MOVES run you only need to use one or two commands, `ant run` and `ant manyworkers`. The first command runs MOVES using a designated runspec, the other can be used to start additional workers to help speed up a MOVES run. Below are brief examples of how these commands are entered.
29
+
To execute a MOVES run you only need to use one or two commands, `ant run` and `ant manyworkers`. The first command runs MOVES using a designated RunSpec, the other can be used to start additional workers that may help speed up a MOVES run. Below are brief examples of how these commands are entered.
32
30
33
31
### Running MOVES
34
32
35
-
The following command runs MOVES
33
+
The following command runs MOVES:
36
34
37
35
```
38
-
C:\Users\Public\EPA\MOVES\MOVES3>ant run -Drunspec="PathToRunspec\runspec.mrs"
36
+
C:\Users\Public\EPA\MOVES\MOVES3>ant run -Drunspec="PathToRunSpec\RunSpec.mrs"
39
37
```
40
38
41
-
In this example the runspec is specified using the `-Drunspec=` flag. The flag takes a relative path from the MOVES directory to the runspec to be run. By default `run` will also start one worker to process bundles for the MOVES run.
39
+
In this example the RunSpec is specified using the `-Drunspec=` flag. The flag takes a relative path from the MOVES directory to the RunSpec to be run. By default `run` will also start one worker to process bundles for the MOVES run.
42
40
43
41
### Launching MOVES workers
44
42
45
-
The following command launches additional workers to process bundles in a MOVES run.
43
+
The following command launches additional workers to process bundles in a MOVES run. It should be run from a separate command prompt (after running `setenv` to configure the new command prompt's environment), so that this can be run at the same time as the `run` command above.
Here the option `-Dmaxworkers=3` specifies the number of additional workers to start (in this case 3). The option `-Dnoshutdown=1` keeps the workers running if no bundles are received after a certain amount of time. This can be a helpful option when the bundles take a long time to be generated, or when running many back to back MOVES runs.
49
+
Here the option `-Dmaxworkers=3` specifies the number of additional workers to start (in this case 3). The option `-Dnoshutdown=1` keeps the workers running if no bundles are received after a certain amount of time. This can be a helpful option when the bundles take a long time to be generated, or when running many back to back MOVES runs. However, this is not a necessary step, as MOVES will launch a worker by default as part of the `run` command if it does not detect at least one already running.
52
50
53
51
## Example script for executing MOVES batch runs
54
52
55
-
The above Ant commands can be used in batch scripts written to run in the Windows command shell. Batch scripts are text files with a .bat file extension that contain Windows shell commands that are run sequentially from the Windows command shell. Two example batch scripts are shown below. The first is a simple example that demonstrates how to run multiple MOVES runspecs sequentially. The second shows how to start additional workers to help speed up the MOVES run. Both of these scripts assume that MOVES is installed in its default location on the windows C drive `C:\Users\Public\EPA\MOVES\MOVES3` and that the folder containing the script and runspecs is also on the `C:` drive. Finally both of these scripts assume that any required input databases specified in the runspecs have already been placed in the appropriate database folder.
53
+
The above Ant commands can be used in batch scripts written to run in the Windows command shell. Batch scripts are text files with a .bat file extension that contain Windows shell commands that are run sequentially from the Windows command shell. Two example batch scripts are shown below. The first is a simple example that demonstrates how to run multiple MOVES RunSpecs sequentially. The second shows how to start additional workers to help speed up the MOVES run. Both of these scripts assume that MOVES is installed in its default location on the windows C drive `C:\Users\Public\EPA\MOVES\MOVES3` and that the folder containing the script and RunSpecs is also on the `C:` drive. Finally both of these scripts assume that any required input databases specified in the RunSpecs have already been placed in the appropriate database folder.
56
54
57
55
### Basic MOVES batch script
58
56
59
-
Below is a simple MOVES batch script for running multiple runspecs sequentially.
57
+
Below is a simple MOVES batch script for running multiple RunSpecs sequentially.
60
58
61
59
```batch
62
60
:: Find current folder
63
-
set RunspecDir=%CD%
61
+
set RunSpecDir=%CD%
64
62
65
63
:: Set MOVES install location
66
-
set MOVESDir=C:\Users\Public\EPA\MOVES\MOVES3-Beta
64
+
set MOVESDir=C:\Users\Public\EPA\MOVES\MOVES3.0
67
65
68
66
:: setup moves environment
69
67
cd /d %MOVESDir%
70
68
call setenv.bat
71
69
72
-
:: Run MOVES for each runspec
73
-
::(add a new line for each runspec)
74
-
call ant run -Drunspec="%RunspecDir%\BatchTest1.mrs"
70
+
:: Run MOVES for each RunSpec
71
+
::(add a new line for each RunSpec)
72
+
call ant run -Drunspec="%RunSpecDir%\BatchTest1.mrs"
75
73
76
-
call ant run -Drunspec="%RunspecDir%\BatchTest2.mrs"
74
+
call ant run -Drunspec="%RunSpecDir%\BatchTest2.mrs"
77
75
78
76
:: Return to your test folder when everything is finished
79
-
cd /d %RunspecDir%
77
+
cd /d %RunSpecDir%
80
78
```
81
79
82
-
In this batch script the runspecs`BatchTest1.mrs`, and `Batchtest2.mrs` are stored in the same folder as the script. The script finds its location, sets up the MOVES run environment, and executes the runspecs sequentially. After the last run is complete the script returns the command prompt to the directory containing itself and the two runspecs.
80
+
In this batch script the RunSpecs`BatchTest1.mrs`, and `Batchtest2.mrs` are stored in the same folder as the script. The script finds its location, sets up the MOVES run environment, and executes the RunSpecs sequentially. After the last run is complete the script returns the command prompt to the directory containing itself and the two RunSpecs.
83
81
84
82
### MOVES batch script with extra workers
85
83
@@ -92,7 +90,7 @@ The script below is exactly like the previous script, except that this script al
92
90
set TestingDir=%CD%
93
91
94
92
:: Set MOVES install location
95
-
set MOVESDir=C:\Users\Public\EPA\MOVES\MOVES3-Beta
93
+
set MOVESDir=C:\Users\Public\EPA\MOVES\MOVES3.0
96
94
97
95
:: setup moves environment
98
96
cd /d %MOVESDir%
@@ -101,8 +99,8 @@ call setenv.bat
101
99
:: Launch two additional MOVES workers
102
100
start cmd /k ant manyworkers -Dmaxworkers=3 -Dnoshutdown=1
103
101
104
-
:: Run MOVES for each runspec
105
-
::(add a new line for each runspec, or get fancy and write a loop here)
102
+
:: Run MOVES for each RunSpec
103
+
::(add a new line for each RunSpec, or get fancy and write a loop here)
106
104
call ant run -Drunspec="%TestingDir%\BatchTest1.mrs"
107
105
108
106
call ant run -Drunspec="%TestingDir%\BatchTest2.mrs"
@@ -113,13 +111,52 @@ cd /d %TestingDir%
113
111
114
112
Once these moves runs finish, the workers will remain active until the user closes the command window that they are running in. Note when the `run` command is executed, MOVES checks to see if any workers are running. It if doesn't find any it will automatically start one worker to process its bundles.
115
113
114
+
## Batch mode input database creation
115
+
MOVES includes an Ant command to create input databases, which can save time if you need to create many input databases at once. The following steps assume that you already have all of your input data files QA-ed and ready to go.
116
+
117
+
1. Using the MOVES GUI, create your base RunSpec.
118
+
119
+
2. Using the Create Input Database Panel, create a corresponding database, and enter the Data Manager. Fully populate the database so that you get all green checks. *Note: do not close and re-enter the Data Manager during this step or before performing step 3. All tables must be populated in one session for step 3 to work.*
120
+
121
+
3. On the Tools tab of the Data Manager, click the Generate Importer XML File button, and give it a name. The file extension should be `.xml`.
122
+
123
+
4. Make copies of the XML file for each database that you want to build on the command line. Open the file and make the following changes:
124
+
125
+
* Within the `<filters>` tab, make any changes reflective of differences between the base RunSpec and the RunSpec that you will use with this database. For example, if the new database will be for a different calendar year, change the `<year key="XXXX">` entry.
126
+
* Name the database in the `<databaseselection>` tag. For example, `<databaseselection servername="localhost" databasename="new_db" />`
127
+
* Update the `<filename>` and `<section>` tags for each input group to point to the corresponding data file and tab (if the file is .xlsx)
128
+
129
+
5. For each input database to be created, run the following command (changing the path to the importer as appropriate):
MOVES includes an Ant command to convert input databases from previous versions of MOVES to the current version, which can save time if you need to do this for many databases. Please see the help file at [database\ConversionScripts\InputDatabaseConverstionHelp.pdf](..\database\ConversionScripts\InputDatabaseConverstionHelp.pdf) for more information on the uses and limitations of this feature. Note that more work is needed after running this command before the converted database can be used with MOVES.
137
+
138
+
The following command can be used to convert a MOVES2014 (not a MOVES2014a or MOVES2014b) input database to the MOVES3 format:
`-Dinput` is used to specify the old input database name, and `-Doutput` is used to specify the name for the new, converted database. `convert2014_3` tells Ant to use the conversion scripts for going from MOVES2014 to MOVES3.
144
+
145
+
The following command can be used to convert a MOVES2014a or MOVES2014b input database to the MOVES3 format:
`-Dinput` and `-Doutput` are the same as with the other conversion command, and `convert2014ab_3` tells Ant to use the conversion scripts for going from a MOVES2014a or MOVES2014b input database to MOVES3 format.
152
+
116
153
## Additional remarks
117
154
118
155
The above examples demonstrate a simple way of running MOVES from the windows command shell. The example scripts should be considered just EXAMPLES. Different MOVES install configurations may change some of the details needed to run the scripts. Likewise the Windows command shell offers a rich scripting environment that can be used to expand these examples to do things such as:
119
156
120
157
* archive the `moveslog.txt` generated with each run
121
158
122
-
* loop through an arbitrary set of runspecs by searching the folder for files with .mrs extensions
159
+
* loop through an arbitrary set of RunSpecs by searching the folder for files with .mrs extensions
123
160
124
161
* copy input databases stored elsewhere into the MariaDB data directory
Copy file name to clipboardexpand all lines: readme.md
+3
Original file line number
Diff line number
Diff line change
@@ -1,3 +1,6 @@
1
+
# MOVES3.0.1
2
+
This branch contains code fixes that will be released in the next patch to MOVES.
3
+
1
4
# MOVES3
2
5
3
6
EPA's MOtor Vehicle Emission Simulator (MOVES) is a state-of-the-science emission modeling system that estimates emissions for mobile sources at the national, county, and project level for criteria air pollutants, greenhouse gases, and air toxics, available under EPA's Open Source Software policy.
0 commit comments