Deploy to GitHub pages

Author: github-actions[bot]

.buildinfo

@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file records the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: f281a4555231eb4de48eb734f1a4b949
+tags: 645f666f9bcd5a90fca523b33c5a78b7

.nojekyll

@@ -0,0 +1,65 @@
+########################################
+Hamlet Architecture and Testing Workflow
+########################################
+
+Currently, Hamlet performs spatial-temporal hazard model checks and
+statistical evaluation of model consistency and performance against an observed
+earthquake catalog.
+
+Hamlet follows this work process:
+
+1. Read in :doc:`YAML configuration file <./yaml_config_file>`, that specifies:
+
+   - Which tests to be run
+
+     - What parameters for each test
+
+   - What input files:
+
+     - Seismic Source Model files
+
+     - Seismic catalog (observed earthquakes)
+
+     - GIS file of spatial bins (optional)
+
+   - What to output:
+
+     - Reports
+
+     - GIS files
+
+2. Reads and process SSM:
+
+   1. Loads sources from a single logic tree branch
+
+   2. Sorts sources based on their type, with a list for each
+
+3. Sorts the ruptures from all sources by magnitude and into spatial bins:
+
+   - Makes :class:`~openquake.hme.utils.bins.SpacemagBin` class that holds
+     ruptures, observed earthquakes, and both model and empirical
+     Magnitude-Frequency distributions for each bin.
+
+4. Runs the tests:
+
+   - Basic sanity checks (e.g., whether the observed earthquake maximum
+     magnitude exceeds the model maximum magnitude in each spatial bin)
+
+   - Statistical evaluation (i.e., model likelihoods based on the calculated
+     probabilities of observing the earthquakes in a catalog given the SSM)
+
+   - Multiple tests can be run sequentially, without reloading the SSM.
+
+5. Print/write output.
+
+  - HTML reports summarizing the results
+
+  - GIS files with the test results for each bin
+
+  - CSV files of the total model and catalog (within the source bins) MFDs
+
+
+At this pre-release stage, most of the development has focused on writing the
+test framework, rather than creating a broad suite of tests. However, the
+framework is functional at this point, and the development of a test suite is
+the next priority.

_sources/architecture.rst.txt

@@ -0,0 +1,102 @@
+# Getting started with Hamlet
+
+Hamlet is easy to use once the input data and configuration are prepared. The
+steps to do this are:
+
+## Prepare the hazard model
+
+The hazard model must be in the OpenQuake format. There must be a single XML
+file that describes the seismic source model logic tree, including the locations
+of the source XML files and other logic tree parameters.
+
+### Decide how the model should be evaluated
+
+Choose whether Hamlet should be run for the entire model as a whole, or for
+different components of the model (different logic tree branches, different
+seismic source types, etc.). This will control how the data preparation and
+testing are done.
+
+### (Optional) Organize the hazard model with a `hamlet` directory
+
+It is recommended to add a new `hamlet` directory with `data` and `output`
+sub-directories.
+
+The `data` directory can hold the seismic catalogs and, if
+necessary, GIS files specifying the model domain and grid cells.
+
+The `output` directory will hold the HTML reports and any other outputs that are
+written during the testing procedure.
+
+The YAML configuration files can be placed in the main `hamlet` directory:
+
+```
+model/
+    in/
+        ssm/
+        gmm/
+    out/
+    hamlet/
+        data/
+            crustal_catalog.csv
+            slab_catalog.csv
+            full_catalog.csv (unused in testing)
+        output/
+        test_ssm_crustal.yml
+        test_ssm_slab.yml
+```
+
+However, the user is free to organize the Hamlet files in any way; there is no
+requirement that the files are in the same directory as the source model, as
+long as the paths to the source model and earthquake catalogs are correct in the
+YAML configuration file.
+
+### (Optional) Make a new `git` branch for Hamlet
+
+This may be a good way of organizing the results and running Hamlet in a
+continuous integration system.
+
+For example, a `git` branch called `hamlet` can have a separate `hamlet`
+directory, as specified above.  This directory does not exist in the `master`
+branch or other branches, and when changes are made to those branches, they can
+be pulled into the `hamlet` branch and be evaluated.
+
+This can be configured in a continuous integration environment such that
+Hamlet is run on the CI server when the `hamlet` branch is committed, but not
+necessarily when commits to `master` or development branches are made.
+
+
+## Prepare the earthquake catalog(s)
+
+The earthquake catalog should be declustered and, ideally, classified according
+to the source types of the earthquakes (i.e., subduction thrust, in-slab,
+crustal, etc.). The catalog should also be truncated to some acceptable
+completeness date that corresponds to the `investigation_time` parameter used
+during the Hamlet evaluations (in the future, completeness tables may be able to
+be used instead of a single date, but this is not currently implemented).
+
+The catalog(s) must be CSV files, with columns describing the fields and one row
+for each earthquake.
+
+If you are interested in running Hamlet separately for different seismic source
+types, then make separate catalogs for the different earthquake categories,
+i.e. make CSV files with only crustal events, only subduction megathrust
+events, only slab events, etc.
+
+Alternately if the different branches in the model's logic tree would for some
+reason correspond to different subsets of the earthquake catalog, the catalog
+should be split into separate files for each subset.
+
+
+## Make the YAML configuration file(s)
+
+See [YAML configuration file](./yaml_config_file.html) for more information.
+
+## Run Hamlet
+
+Once the model, seismic catalog(s) and YAML configuration file(s) (and Hamlet
+has been installed), Hamlet can be run like this:
+
+```
+hamlet test_ssm_crustal.yml
+```
+

_sources/getting_started.md.txt

@@ -0,0 +1,100 @@
+==================================================
+Hamlet: Hazard Model Evaluation and Testing
+==================================================
+
+Hamlet (``openquake.hme``) is a Python package developed (OK, in development)
+for qualitative and quantitative evaluation of Probabilistic Seismic Hazard
+Analysis (PSHA) models, with the intention of providing feedback to modelers
+during the model construction process, to aid model development. Hamlet is
+developed by the `GEM Foundation`_, and uses the OpenQuake_ software
+extensively.
+
+Hamlet will incorporate several model test frameworks, including those
+developed by GEM and some of those developed outside of GEM such as the RELM_
+tests. Currently, and likely in the future, the model files will be required to
+be in the OpenQuake_ format, regardless of the format of their original
+implementation.
+
+Most of the Hamlet evaluations are spatial in nature; the model domain is
+discretized into grid cells, and comparisons between observations and model
+predictions are performed in each grid cell, to highlight where in the domain
+the model matches the observations, and where it might need some refinement.
+
+Additionally, unlike some other hazard model testing frameworks, Hamlet is
+designed to operate on separate components of a hazard model, so that each
+component can be evaluated against its corresponding data. For example, each
+branch of a source model logic tree can be tested independently, and each type
+of source (e.g., subduction megathrust, crustal, in-slab) can be tested
+independently as well, in the spatial framework described above.
+
+Quickstart
+==========
+
+Installation
+------------
+
+Hamlet requires installation Python v.3.7+, the OpenQuake_ engine, and some
+additional dependencies as well. These are specified in the ``requirements.txt``
+file.
+
+(*Note:* A few of the dependencies might be challenging to install.  These are
+Rtree_ and h3-py_. You may have to install ``libspatialindex`` or
+``libspatialindex-dev`` on Linux or MacOS first, depending on your system, for
+``Rtree``.  ``h3-py`` requires ``cc`` and ``make``, but then on Linux/MacOS can
+be installed easily. Please see the documentation for each.)
+
+First, install the OpenQuake_ engine, following directions on that website. You
+probably want to install it into a virtual environment, and you may even want to
+have a separate virtual environment for running Hamlet than the OpenQuake_
+virtual environment that you normally use (this is up to you).
+
+Then, clone the Hamlet repository, and from that directory, install the
+requirements::
+
+    pip install -r requirements.txt
+
+and then install Hamlet::
+
+    pip install -e .
+
+
+Running Hamlet
+--------------
+
+Hamlet only requires a seismic hazard model (implemented in OpenQuake_) and a
+processed seismic catalog (declustered, and ideally classified by source type)
+to run. Once installed, Hamlet can be run from the command line::
+
+    hamlet test_model.yml
+
+``test_model.yml`` is a :doc:`configuration file <yaml_config_file>` in YAML_
+format that specifies the source model, seismic catalog, tests to be run, and
+other variables and parameters.
+
+
+Documentation
+=============
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   getting_started
+   architecture
+   model_test_frameworks/model_test_frameworks
+
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
+
+
+.. _YAML: https://yaml.org
+.. _GEM Foundation: https://www.globalquakemodel.org
+.. _OpenQuake: https://github.com/GEM/oq-engine
+.. _RELM: http://cseptesting.org/documents/relm.php
+.. _h3-py: https://github.com/uber/h3-py
+.. _Rtree: https://toblerity.org/rtree/

_sources/index.rst.txt

@@ -0,0 +1,122 @@
+.. _gem-tests-evaluations:
+
+
+GEM Tests and Evaluations
+=========================
+
+
+These are developed by GEM, sometimes based on the literature, or sometimes
+based on our own ideas and implementations. See for
+more information, or :mod:`~openquake.hme.model_test_frameworks.gem` for the
+function documentation. Sanity checks (as detailed below) are also available
+from the GEM testing framework, for convenience during the workflow.
+
+Magnitude-Frequency Distribution based tests
+--------------------------------------------
+
+These tests evaluate a model based on the magnitude-frequency distribution (MFD)
+inside each grid cell, or in the model as a whole, (for the model component
+being tested).
+
+
+.. _gem-like-test:
+
+*Likelihood test*
+
+Currently, there are two implementations of the MFD likelihood tests, one
+'empirical', based on Monte Carlo sampling of the source model in each bin, and
+one based on the frequencies in the MFD itself.
+
+In both of these tests, the likelihood of observing the seismicity in the
+catalog given the model MFD is calculated through the 
+
+Parameters (all are optional, as default values are supplied):
+
+``likelihood_method``
+    This is how the computations are performed.  ``poisson`` uses the Poisson
+    likelihood, and ``empirical`` uses a Monte Carlo sampling of the MFD.  The
+    default value is ``poisson``.
+
+``investigation_time``
+    This is how long the time period is for comparing the observed seismicity to
+    the MFD. Unless you're doing something very crafty, the value should be in
+    years, and should be the length of the observed earthquake catalog.  In the
+    future, a completeness table may be used instead of this parameter.  The
+    default value is ``1.``
+
+``default_likelihood``
+    This is the likelihood that results if no earthquake sources are present in
+    the grid cell. If the cells are built using `h3` (the default option, if no
+    GIS file for the test is supplied), this parameter will have no effect.  The
+    default value is ``1.``
+
+
+``not_modeled_val``
+    This is the likelihood that results if the rate of earthquake production in
+    that magnitude bin is zero, but there are earthquakes within the magnitude
+    bin. Standard (or naive) statistical theory suggests that this value should
+    be zero, as it is in the RELM tests, but because this value is multiplied by
+    all of the other magnitude bins inside each spatial bin/grid cell, a single
+    zero value will make the whole model likelihood zero.  The default is
+    ``1e-5`` which is a bit more pragmatic.
+
+
+
+.. _gem-model-mfd-test:
+
+*Model MFD Evaluation*
+
+The Model MFD evaluation sums up the MFDs from each
+:class:`~openquake.hme.utils.bins.SpacemagBin` and makes an MFD for the whole
+model, which is then compared to the observed MFD from the earthquake catalog.
+This is to produce a figure and currently does not yield any quantitative values
+or evaluate the entire model goodness of fit.
+
+Parameters (all are optional):
+
+``investigation_time``
+    This is the duration of the comparison between the observed seismicity to
+    the MFD. See :ref:`likelihood <gem-like-test>` above for more information.
+
+``out_csv``
+    This parameter specifies a filename. If this is given, a CSV table of the
+    observed and modeled MFDs will be written.
+
+``out_plot``
+    This parameter specifies a filename. If this is given, a plot of the
+    observed and modeled MFDs will be written. The file suffix will determine
+    the plot format.  Common formats include ``png``, ``svg`` and ``pdf``.  See
+    the ``matplotlib`` docs for more info.
+
+
+.._max_mag_check:
+
+*Ensures that the model can produce the maximum observed seismcity in each cell*
+
+This test is borrowed from the Sanity checks. It simply checks to see whether
+the sources inside each cell are capable of producing earthquakes as large as
+the largest observed earthquakes. Note that there can be some issues with very
+large earthquakes (with ruptures larger than the cell size) as the hypocenter
+for an observed event may be in a different cell than the most compatible
+hypocenter from the sources.
+
+
+.. _gem-moment-over-under-eval:
+
+*Compares observed and stochastic moment release*
+
+This evaluation generates many synthetic catalogs (stochastic event sets) and
+compares the total moment release in each cell for each of the catalogs to the
+observed moment release. This evalution helps highlight areas that are more or
+less seismically productive than the observations may support.
+
+Parameters:
+
+``investigation_time``:  Duration of the catalog (and of the generated
+    stochastic event sets).
+
+``n_iters``: Number of iterations (stochastic event sets) generated. Note that
+    generating these catalogs is fairly time-intensive in the current
+    implementation. For large models, the number of iterations should be kept to
+    under 50-100 until performance is improved.
+