Home
MDPSolver (mdpsolver) is a Python package for large Markov Decision Processes (MDPs) with infinite horizons. This wiki includes details about the documentation of MDPSolver and other information, such as performance tests.
- Fast solver: Our C++-based solver is substantially faster than other MDP packages available for Python, depending on the problem size and parameters. See the Performance tests section in this wiki.
- Available on PyPI.
- Two optimality criteria: Discounted and Average reward.
- Three optimization algorithms: Value iteration, Policy iteration, and Modified policy iteration.
- Three value-update methods: Standard, Gauss–Seidel, and Successive over-relaxation.
- Uses span norm or supremum norm stopping criterion depending on the selected update method.
- Supports sparse matrices and parallel computing.
- Operating systems: Linux and Windows.
Install directly from PyPI with:
pip install mdpsolver
MDPSolver works out of the box on Ubuntu 22 and newer.
Some users will encounter the version 'GLIBC_2.32' not found error when attempting to import MDPSolver in Python. In this case, it might help to manually compile and replace the SO-file for the optimization module in the MDPSolver package. Follow these steps to compile the module, and these steps to install the module in the MDPSolver package.
Requires Visual Studio 2022 (17.9) with MSVC C++ compiler and libraries installed (see below).
After installing Visual Studio (incl. MSVC C++ compiler and libraries), install directly from PyPI with:
pip install mdpsolver
- Operating system: Windows or Linux.
- Windows users must install Visual Studio 2022 incl. the MSVC C++ compiler (see the installation instructions above).
- MDPSolver has been tested on Windows 11 and Ubuntu 22.04.5 LTS.
- Linux users must ensure that GLIBC 2.32 or newer is installed (check your version with
ldd --version).
