Maybee is a restful API that implements non-contextual multi armed bandits. Its name is a play on the abbreviation MAB (multi armed bandit).
MABs are a class of algorithms in reinforcement learning that aim to quickly learn how to make good decisions when choosing between many different 'arms' (the equivalent to an 'experimental condition' in A/B/C testing) in order to optimize for some reward function.
MABs are a particularly attractive alternative to A/B/C testing when perfect certainty about which arm yields the highest avg rewards is less important than quickly making smart decisions, or when it's impractical or even impossible to keep a constant set of arms / experimental conditions.
This project is in the 0.x stage. You should expect this project to change rapidly new versions to include breaking changes.
- Python
- Poetry
- FastAPI
- SQLModel
To get started, please explore the Makefile. Since this project is docker based, you'll need an active Docker machine to execute these commands.
Build the docker image
make buildStart the application in docker compose with a postgres database
make upThis API implements the core concepts of MABs:
- environments (analogous to an experiment)
- arms (analogous to an experimental condition)
- actions (which represent the actions taken by the bandit)
- observations (which represent the observations of a reward, or the lack thereof, as a result of the actions taken)
- Create python backend
- Implement epsilon-greedy, softmax, and UCB1 bandits
- Add unit tests
- Create python client library
- Set up CRUD methods for all available entities.
- Create React frontend
See the open issues for a full list of proposed features (and known issues).
Distributed under the GPL-3.0 license.
Walter Hoekstra (https://www.linkedin.com/in/walter-hoekstra-aa26b750/)
Project Link: https://github.com/WSHoekstra/maybee_backend
Special thanks to John Myles White (https://github.com/johnmyleswhite), the author of 'Bandit Algorithms for Website Optimization' for inspiring this project.