You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<spanclass="k">raise</span><spanclass="ne">ValueError</span><spanclass="p">(</span><spanclass="s2">"Plausible interval ranges PLB and PUB need to be finite."</span><spanclass="p">)</span>
492
+
<spanclass="k">raise</span><spanclass="ne">ValueError</span><spanclass="p">(</span><spanclass="s2">"Plausible interval ranges plb and pub need to be finite."</span><spanclass="p">)</span>
493
493
494
494
<spanclass="c1"># Check that the order of bounds is respected</span>
<spanclass="k">raise</span><spanclass="ne">ValueError</span><spanclass="p">(</span><spanclass="s2">"Interval bounds needs to respect the order LB <= PLB < PUB <= UB for all coordinates."</span><spanclass="p">)</span>
498
+
<spanclass="k">raise</span><spanclass="ne">ValueError</span><spanclass="p">(</span><spanclass="s2">"Interval bounds needs to respect the order lb <= plb < pub <= ub for all coordinates."</span><spanclass="p">)</span>
499
499
500
500
501
501
<spanclass="c1"># A variable is converted to log scale if all bounds are positive and</span>
Copy file name to clipboardexpand all lines: _sources/api/classes/bads.rst
+1-1
Original file line number
Diff line number
Diff line change
@@ -6,7 +6,7 @@ BADS
6
6
7
7
The ``BADS`` class implements the Bayesian Adaptive Direct Search (BADS) algorithm.
8
8
9
-
BADS attempts to solve a constraint optimization for rough and possible noisy functions, by finding optimistically its global minimum.
9
+
BADS attempts to solve an unbounded, bounded or nonlinearly constrained optimization (minimization) problem, and is compatible with both noiseless and noisy target functions.
10
10
11
11
To perform the optimization, first initialize a ``BADS`` object and then call ``bads.optimize()`` on the instance.
The options can be divided into two types of options:
5
-
- **Basic default options:** We expect that these options are changed by many users.
6
-
- **Advanced options:** These options are for advanced users of BADS. Do not modify them unless you *know* what you are doing.
4
+
The options can be divided into two types:
5
+
- **Basic options:** These options are of interest to most users, and cover all regular usage needs.
6
+
- **Advanced options:** These options should rarely if ever be touched and are reserved to advanced users / developers of BADS. Do not modify them unless you *know* what you are doing.
7
7
8
8
You can find the default options for both groups below.
9
9
10
10
Basic options
11
11
=====================
12
-
We expect that these options are changed by many users.
12
+
We expect these options to be routinely changed by many users.
Copy file name to clipboardexpand all lines: _sources/index.rst
+27-7
Original file line number
Diff line number
Diff line change
@@ -2,22 +2,42 @@
2
2
PyBADS
3
3
******
4
4
5
-
PyBADS is a Python implementation of the Bayesian Adaptive Direct Search (BADS) algorithm for solving difficult and moderate expensive optimization problems, originally implemented :labrepos:`in MATLAB <bads>`.
5
+
PyBADS is a Python implementation of the Bayesian Adaptive Direct Search (BADS) algorithm for solving difficult and moderately expensive optimization problems, originally implemented :labrepos:`in MATLAB <bads>`.
6
6
7
7
What is it?
8
8
###########
9
9
10
10
BADS is a fast hybrid Bayesian optimization algorithm designed to solve difficult optimization problems, in particular related to fitting computational models (e.g., `via maximum likelihood estimation <https://en.wikipedia.org/wiki/Maximum_likelihood_estimation>`__).
11
11
12
-
BADS has been intensively tested for fitting a variety of computational models, and is currently being used in many computational labs around the world (see `Google Scholar <https://scholar.google.co.uk/scholar?cites=7209174494000095753&as_sdt=2005&sciodt=0,5&hl=en>`__ for many example applications). In our benchmark with real model-fitting problems, BADS performed on par or better than many other common and state-of-the-art optimizers, such as `fminsearch`, `fmincon`, and `cmaes` as shown in the original paper presented at NeurIPS in 2017 `(Acerbi, 2017) <#references>`_.
12
+
BADS has been intensively tested for fitting a variety of computational models, and is currently being used in many computational labs around the world (see `Google Scholar <https://scholar.google.co.uk/scholar?cites=7209174494000095753&as_sdt=2005&sciodt=0,5&hl=en>`__ for many example applications).
13
13
14
-
BADS requires no specific tuning and runs off-the-shelf like other built-in MATLAB optimizers such as `fminsearch`.
14
+
In our benchmark with real model-fitting problems, BADS performed on par or better than many other common and state-of-the-art optimizers, as shown in the original BADS paper (`Acerbi and Ma, 2017 <#references>`_).
15
+
16
+
BADS requires no specific tuning and runs off-the-shelf similarly to other Python optimizers, such as those in `scipy.optimize.minimize`.
15
17
16
18
*Note*: If you are interested in estimating posterior distributions (i.e., uncertainty and error bars) over model parameters, and not just point estimates, you might also want to check out Variational Bayesian Monte Carlo for Python (:labrepos:`PyVBMC <pyvbmc>`), a package for Bayesian posterior and model inference which can be used in synergy with PyBADS.
17
19
18
-
Example run
20
+
How does it work?
19
21
-----------
20
-
TODO: Put a Gif here showing a BADS run on a simple problem (e.g on the Rosenbrock's banana function).
22
+
23
+
PyBADS/BADS follows a `mesh adaptive direct search <http://epubs.siam.org/doi/abs/10.1137/040603371>`__ (MADS) procedure for function minimization that alternates **poll** steps and **search** steps (see **Fig 1**).
24
+
25
+
- In the **poll** stage, points are evaluated on a mesh by taking steps in one direction at a time, until an improvement is found or all directions have been tried. The step size is doubled in case of success, halved otherwise.
26
+
- In the **search** stage, a `Gaussian process <https://en.wikipedia.org/wiki/Gaussian_process>`__ (GP) is fit to a (local) subset of the points evaluated so far. Then, we iteratively choose points to evaluate according to a *lower confidence bound* strategy that trades off between exploration of uncertain regions (high GP uncertainty) and exploitation of promising solutions (low GP mean).
27
+
28
+
.. image:: _static/bads-cartoon.png
29
+
:align:center
30
+
:alt:Fig 1: BADS procedure
31
+
32
+
Fig 1: BADS procedure
33
+
34
+
See `here <https://github.com/lacerbi/optimviz>`__ for a visualization of several optimizers at work, including BADS.
35
+
36
+
See our paper for more details (`Acerbi and Ma, 2017 <#references>`_).
37
+
38
+
.. Example run
39
+
-----------
40
+
TODO: Put a Gif here showing a BADS run on a simple problem (e.g on the Rosenbrock's banana function).
21
41
22
42
Should I use PyBADS?
23
43
--------------------
@@ -61,7 +81,7 @@ Contributing
61
81
References
62
82
###############
63
83
64
-
1. Acerbi, L. (2017). Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search. In *Advances in Neural Information Processing Systems 31*: 8222-8232. (`paper + supplement on arXiv <https://arxiv.org/abs/1705.04405>`__, `NeurIPS Proceedings <https://papers.nips.cc/paper/2017/hash/df0aab058ce179e4f7ab135ed4e641a9-Abstract.html>`__)
84
+
1. Acerbi, L. & Ma, W. J. (2017). Practical Bayesian Optimization for Model Fitting with Bayesian Adaptive Direct Search. In *Advances in Neural Information Processing Systems 31*: 8222-8232. (`paper + supplement on arXiv <https://arxiv.org/abs/1705.04405>`__, `NeurIPS Proceedings <https://papers.nips.cc/paper/2017/hash/df0aab058ce179e4f7ab135ed4e641a9-Abstract.html>`__)
65
85
66
86
You can cite BADS in your work with something along the lines of
67
87
@@ -90,7 +110,7 @@ You may also want to check out the original :labrepos:`MATLAB toolbox <bads>`.
90
110
91
111
Acknowledgments:
92
112
################
93
-
Work on the PyBADS package was funded by the `Finnish Center for Artificial Intelligence FCAI <https://fcai.fi/>`_.
113
+
Work on the PyBADS package was supported by the Academy of Finland Flagship programme: `Finnish Center for Artificial Intelligence FCAI <https://fcai.fi/>`_.
Copy file name to clipboardexpand all lines: _sources/quickstart.rst
+6-6
Original file line number
Diff line number
Diff line change
@@ -23,26 +23,26 @@ Running the optimizer in step 3 only involves a couple of lines of code:
23
23
24
24
from pybads importBADS
25
25
# ...
26
-
bads = BADS(target, x0, LB, UB, PLB, PUB)
26
+
bads = BADS(target, x0, lb, ub, plb, pub)
27
27
optimize_result = bads.optimize()
28
28
29
29
with input arguments:
30
30
31
31
- ``target``: the target function, it takes as input a vector and returns its function evaluation;
32
32
- ``x0``: the starting point of the optimization problem. If it is not given, the starting point is randomly drawn from the problems bounds;
33
-
- ``LB`` and ``UB``: hard lower and upper bounds for the optimization region (can be ``-inf`` and ``inf``, or bounded);
34
-
- ``PLB`` and ``PUB``: *plausible* lower and upper bounds, that represent our best guess at bounding the region where the solution might lie;
33
+
- ``lb`` and ``ub``: hard lower and upper bounds for the optimization region (can be ``-inf`` and ``inf``, or bounded);
34
+
- ``plb`` and ``pub``: *plausible* lower and upper bounds, that represent our best guess at bounding the region where the solution might lie;
35
35
- ``non_box_cons`` (optional): a callable non-bound constraints function.
36
36
37
37
The outputs are:
38
38
39
-
- ``optimize_result``: a ``OptimizeResult`` which presents the most important information about the solution and the optimization problem.
39
+
- ``optimize_result``: a ``OptimizeResult`` which presents the most important information about the solution and the optimization problem. In particular:
40
40
41
41
- ``"x"``: the minimum point found by the optimizer;
42
42
- ``"fval"``: the value of the function at the given solution.
43
43
44
-
The ``optimize_result`` object can be manipulated in various ways, see the :ref:`OptimizeResult` class documentation.
44
+
The ``optimize_result`` object contains more information about the optimization solution, see the :ref:`OptimizeResult` class documentation.
45
45
46
46
See the examples for more detailed information. The :ref:`Basic options` may also be useful.
47
47
48
-
In addition, checkout the `BADS FAQ <https://github.com/acerbilab/bads/wiki#bads-frequently-asked-questions>`__ page for practical recommendations, such as how to set `LB` and `UB`, and other practical insights. Although, the FAQ refers to the MATLAB version of BADS, most of the concepts still apply for PyBADS.
48
+
In addition, checkout the `BADS FAQ <https://github.com/acerbilab/bads/wiki#bads-frequently-asked-questions>`__ page for practical recommendations, such as how to set `lb` and `ub`, and other practical insights. Even though the FAQ refers to the MATLAB version of BADS, most of the concepts still apply to PyBADS.
Copy file name to clipboardexpand all lines: about_us.html
+1-1
Original file line number
Diff line number
Diff line change
@@ -488,7 +488,7 @@ <h2> Contents </h2>
488
488
<sectionid="about-us">
489
489
<h1>About us<aclass="headerlink" href="#about-us" title="Permalink to this headline">#</a></h1>
490
490
<p>PyBADS is an open-source Python software for resource-efficient optimization problems.</p>
491
-
<p>The development of PyBADS has been generously funded by the <aclass="reference external" href="https://fcai.fi/">Finnish Center for Artificial Intelligence FCAI</a>.</p>
491
+
<p>The development of PyBADS has been partly funded by the <aclass="reference external" href="https://fcai.fi/">Finnish Center for Artificial Intelligence FCAI</a>.</p>
492
492
<sectionid="development-team">
493
493
<h2>Development team<aclass="headerlink" href="#development-team" title="Permalink to this headline">#</a></h2>
494
494
<p>PyBADS is actively developed mainly by members of <aclass="reference external" href="https://www2.helsinki.fi/en/researchgroups/machine-and-human-intelligence">Luigi Acerbi’s research group</a> at the University of Helsinki.</p>
0 commit comments