Skip to content

Commit ee38d8b

Browse files
committed
Refactor entire project
This commit is a big refactor of the pybm project. Highlights include: * Providers were cut from the scope, with venv now being the sole virtual environment provider. * A new `PythonVenv` class was added to manage central venv operations in a class. * The source was reformatted globally with isort, a new precommit hook. * A __main__ file was added to enable calling pybm via python module syntax. * The JSON reporter was refactored to make a cleaner presentation of results. Specifically, aggregates are not recalculated by reductions if already present.
1 parent f5c807d commit ee38d8b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+1322
-1320
lines changed

.github/workflows/ci.yaml

+2-3
Original file line numberDiff line numberDiff line change
@@ -46,8 +46,7 @@ jobs:
4646

4747
- name: Install pybm and other Python dependencies
4848
run: |
49-
python -m pip install wheel
50-
python -m pip install .. google-benchmark
49+
python -m pip install .. wheel google-benchmark
5150
working-directory: ${{ matrix.example }}
5251

5352
- name: Create virtual environment with the current pybm checkout
@@ -66,6 +65,6 @@ jobs:
6665

6766
- name: Run ${{matrix.example}} end-to-end test
6867
run: |
69-
pybm run -m benchmarks master linear-time constant-time --checkout
68+
pybm run -M benchmarks master linear-time constant-time --use-checkouts
7069
pybm compare master linear-time constant-time
7170
working-directory: ${{ matrix.example }}

.pre-commit-config.yaml

+8-1
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,11 @@ repos:
1414
rev: 22.3.0
1515
hooks:
1616
- id: black
17-
language_version: python3
17+
language_version: python3
18+
19+
- repo: https://github.com/pycqa/isort
20+
rev: 5.10.1
21+
hooks:
22+
- id: isort
23+
name: isort (python)
24+
args: ["--profile", "black"]

README.md

+23-25
Original file line numberDiff line numberDiff line change
@@ -2,24 +2,24 @@
22

33
## What is pybm?
44

5-
**pybm** is a Python CLI for repeatable, declarative benchmarking of Python code inside a git repository. It uses git's
6-
version control mechanisms to track changes in performance between points of interest in git history. Out of many
5+
**pybm** is a Python CLI for reproducible benchmarking of Python code inside a git repository. It uses git's
6+
version control features to track changes in performance between points of interest in git history. Out of many
77
possible use cases, three specific ones immediately emerge:
88

9-
- Comparing performance of a feature branch to the current development branch.
10-
- Tracking changes in performance across git history (e.g. releases, refactors).
11-
- Adding a benchmarking step into a continuous integration (CI) pipeline, and rejecting changes if they reduce
9+
- Comparing performance of a feature branch (e.g. a pull request on GitHub) to the current development branch.
10+
- Tracking changes in performance across the git history (e.g. releases, refactors).
11+
- Adding a benchmarking step to a continuous integration (CI) workflow, and rejecting changes if they reduce
1212
performance to a significant degree.
1313

1414
## Notable features
1515

1616
The main concepts of pybm are:
1717

18-
- Creation and management of benchmark environments, consisting of a git reference (commit/branch/tag) and an associated
19-
Python virtual environment containing the desired dependencies.
20-
- Running benchmarks inside an environment, with optional filtering, additional context information, and user-specific
18+
- Creation and management of workspaces, consisting of a git reference (commit/branch/tag) and an associated
19+
Python virtual environment with the desired dependencies.
20+
- Running benchmarks inside a workspace, with optional filtering, additional context information, and user-specific
2121
metrics tracking.
22-
- Reporting results in direct comparison between environments to find and quantify performance changes.
22+
- Reporting results in direct comparison between workspaces to find and quantify performance changes.
2323

2424
## Installation
2525

@@ -37,19 +37,18 @@ way to install pybm is through use of pip and git in the way above.
3737
Initialize `pybm` inside your Python git repository:
3838

3939
```shell
40-
# location: /path/to/python-project
40+
# Inside your project's main directory
4141
pybm init
4242
```
4343

44-
If you need to maintain different requirements between different git references, create a new benchmark environment and
45-
install your requirements:
44+
If you need to maintain different requirements between different git references, create a new workspace and install
45+
your requirements:
4646

4747
```shell
48-
pybm env create my-ref my-env
49-
pybm env install my-env -r my-requirements.txt
48+
pybm create my-ref my-workspace
5049
```
5150

52-
Now, locate your benchmarks. To each of your benchmark files, simply add the following small module execution block:
51+
Now, locate your benchmarks. To each of your benchmark files, simply add the following small execution block:
5352

5453
```python
5554
# benchmark.py
@@ -73,15 +72,15 @@ if __name__ == "__main__":
7372

7473
This way, `pybm` can properly process the information and requirements for each benchmark environment individually.
7574

76-
Now, to run your benchmarks, there are two options:
75+
There are two options to run your benchmarks:
7776

7877
1) With custom requirements and environments,
7978

8079
```shell
8180
# folders or glob expressions also work for benchmark discovery.
8281

8382
# or use --all to run in all existing environments
84-
pybm run benchmark.py my-env1 my-env2 [...] my-envN
83+
pybm run benchmark.py my-workspace1 my-workspace2 [...] my-workspaceN
8584
```
8685

8786
or
@@ -95,18 +94,17 @@ pybm run benchmark.py my-ref1 my-ref2 [...] my-refN --checkout
9594
If you did not install your Python package locally, consider running your benchmarks in module mode:
9695

9796
```shell
98-
pybm run -m benchmark.py my-ref1 [...] my-refN --checkout
97+
pybm run benchmark.py my-ref1 [...] my-refN --as-module --checkout
9998
```
10099

101100
## Requirements
102101

103-
The most central requirement to **pybm** is `git`, a version control system, which is responsible for building benchmark
104-
environments. Currently, at least `git version 2.17.0` (April 2018) is required for pybm to work correctly. To check
105-
your git version, run `git --version`, which should result in an output showing the version number, similar to the one
106-
above.
102+
The most central requirement to **pybm** is `git`, which is responsible for building benchmark workspaces. Currently,
103+
at least `git version 2.17.0` (April 2018) is required for pybm to work correctly. To check your current git version,
104+
run `git --version`, which should result in an output showing the version number, similar to the one above.
107105

108106
On the Python side, in its most standard configuration, **pybm** works almost entirely within the Python standard
109-
library - only the `toml` package is required for configuration management. Additional functionality is available via
107+
library - only the `pyyaml` package is required for configuration management. Additional functionality is available via
110108
extras installation:
111109

112110
```
@@ -119,8 +117,8 @@ status, check the Issues tab and the upcoming milestones.
119117

120118
## Documentation and examples
121119

122-
For some additional documentation on pybm commands, check the [docs](docs)
123-
section. For introductory examples in the form of step-by-step walkthroughs, refer to the [examples](examples) section.
120+
For some additional documentation on pybm commands, check the [docs](docs) section. For introductory examples in the
121+
form of step-by-step walkthroughs, refer to the [examples](examples) section.
124122

125123
## Current status
126124

docs/commands/create.md

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# The `create` command
2+
3+
```
4+
pybm create <commit-ish> <name> <dest> [<options>]
5+
6+
positional arguments:
7+
<commit-ish> Commit, branch or tag to create a git worktree for.
8+
<name> Unique name for the created workspace. Can be used to reference workspaces from the command line.
9+
<dest> Destination directory of the new worktree. Defaults to repository-name@{commit|branch|tag}.
10+
11+
optional arguments:
12+
-h, --help Show this message and exit.
13+
-v Enable verbose mode. Makes pybm log information that might be useful for debugging.
14+
-f, --force Force worktree creation. Useful for checking out a branch multiple times with different custom requirements.
15+
-R, --resolve-commits
16+
Always resolve the given git ref to its associated commit. If the given ref is a branch name, this detaches the HEAD (see https://git-scm.com/docs/git-checkout#_detached_head).
17+
--no-checkout Skip worktree checkout after creation. Useful for sparsely checking out branches.
18+
-L <path-to-venv>, --link-existing <path-to-venv>
19+
Link an existing Python virtual environment to the created pybm workspace. Raises an error if the path does not exist or is not recognized as a valid Python virtual environment.
20+
```
21+
22+
Use `pybm create` to create a new benchmark workspace for a specified git reference:
23+
24+
```shell
25+
pybm create my-branch my-workspace
26+
```
27+
28+
This operation creates a benchmark workspace for a git branch named my-branch, and gives it the name my-workspace. This
29+
given name can be used in pybm to reference a workspace, so it is useful to give expressive names to workspaces.
30+
31+
By default, the benchmark workspace with the repository root worktree is given the name `main`; if you choose not to
32+
specify a name, pybm defaults to the `workspace_i` naming scheme, where `i` is an index starting at 1.
33+
34+
Further positional and optional arguments are:
35+
36+
The `-L` option can be used to link an existing virtual environment to a benchmark workspace. Suppose you have a
37+
ready workspace at `/path/to/venv`; then `pybm workspace create my-branch -L /path/to/venv` will link the existing virtual
38+
environment into the benchmark workspace.
39+
40+
✅ Not only branch names work as valid git references - you can also supply tag names or full/partial commit SHAs. In the
41+
latter case, the SHA fragment is directly passed to git, which can fail to resolve a unique reference if the fragment is
42+
too short. For a project with lots of commits, increasing the SHA fragment length can help avoid resolution errors.

docs/commands/delete.md

+23
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# The `delete` command
2+
3+
```
4+
pybm delete <identifier> [<options>]
5+
6+
positional arguments:
7+
<id> Information that uniquely identifies the workspace. Can be name, checked out commit/branch/tag name, or worktree root directory.
8+
9+
optional arguments:
10+
-h, --help Show this message and exit.
11+
-v Enable verbose mode. Makes pybm log information that might be useful for debugging.
12+
-f, --force Force worktree removal, including untracked files and changes.
13+
```
14+
15+
Use the `pybm delete` command to delete a benchmark workspace. The identifier can be the git reference name (
16+
partial SHAs also work), a benchmark workspace name or a directory name.
17+
18+
In the (standard) case of a virtual environment being created directly inside the git worktree, this virtual environment
19+
will be removed upon deletion of the benchmark workspace; this behavior cannot be changed as git physically removes
20+
the associated worktree. If you want to reuse a Python virtual environment, consider linking it explicitly from another
21+
location with the `-L` switch in the `pybm workspace create` command - pybm will not remove these.
22+
23+
⚠️ As the main git worktree cannot be removed, the `main` pybm workspace cannot be deleted via `pybm delete`.

docs/commands/run.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ optional arguments:
2424
--context <context> Additional global context, given as strings in the format--context='key'='value'. Keys must be unique, supplying two or more context values for the same key results in an error.
2525
```
2626
27-
The `pybm run` command is perhaps the heart of `pybm`'s functionality. It is responsible for discovering, dispatching
27+
The `pybm run` command is responsible for discovering, dispatching
2828
and running the appropriate benchmarks across the chosen workspaces. There are multiple nuances to running benchmarks
2929
in pybm, all of which will be covered now.
3030

docs/commands/workspace.md

+6-72
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,14 @@
22

33
```shell
44
➜ pybm workspace -h
5-
usage: pybm workspace create <commit-ish> <name> <dest> [<options>]
6-
or: pybm workspace delete <identifier> [<options>]
7-
or: pybm workspace install <packages> [<options>]
8-
or: pybm workspace uninstall <packages> [<options>]
5+
usage: pybm workspace install <name> <packages> [<options>]
6+
or: pybm workspace link <name> <path>
97
or: pybm workspace list
10-
or: pybm workspace update <env> <attr> <value>
8+
or: pybm workspace sync [<options>]
9+
or: pybm workspace uninstall <name> <packages> [<options>]
10+
11+
Inspect, list, and manage pybm benchmark workspaces.
1112

12-
Create and manage pybm benchmark workspaces.
13-
1413

1514
optional arguments:
1615
-h, --help Show this message and exit.
@@ -27,71 +26,6 @@ Python virtual environment. Virtual workspaces are the de facto standard solutio
2726
installations to avoid cluttering the system Python; for more information, please refer to the Python docs on virtual
2827
workspaces.
2928

30-
## pybm workspace create
31-
32-
```
33-
pybm workspace create <commit-ish> <name> <dest> [<options>]
34-
35-
positional arguments:
36-
<commit-ish> Commit, branch or tag to create a git worktree for.
37-
<name> Unique name for the created workspace. Can be used to reference workspaces from the command line.
38-
<dest> Destination directory of the new worktree. Defaults to repository-name@{commit|branch|tag}.
39-
40-
optional arguments:
41-
-h, --help Show this message and exit.
42-
-v Enable verbose mode. Makes pybm log information that might be useful for debugging.
43-
-f, --force Force worktree creation. Useful for checking out a branch multiple times with different custom requirements.
44-
-R, --resolve-commits
45-
Always resolve the given git ref to its associated commit. If the given ref is a branch name, this detaches the HEAD (see https://git-scm.com/docs/git-checkout#_detached_head).
46-
--no-checkout Skip worktree checkout after creation. Useful for sparsely checking out branches.
47-
-L <path-to-venv>, --link-existing <path-to-venv>
48-
Link an existing Python virtual environment to the created pybm workspace. Raises an error if the path does not exist or is not recognized as a valid Python virtual environment.
49-
```
50-
51-
Use `pybm workspace create` to create a new benchmark workspace for a specified git reference:
52-
53-
```shell
54-
pybm workspace create my-branch my-workspace
55-
```
56-
57-
This operation creates a benchmark workspace for a git branch named my-branch, and gives it the name my-workspace. This
58-
given name can be used in pybm to reference a workspace, so it is useful to give expressive names to workspaces.
59-
60-
By default, the benchmark workspace with the repository root worktree is given the name `main`; if you choose not to
61-
specify a name, pybm defaults to the `workspace_i` naming scheme, where `i` is an index starting at 1.
62-
63-
Further positional and optional arguments are:
64-
65-
The `-L` option can be used to link an existing virtual environment to a benchmark workspace. Suppose you have a
66-
ready workspace at `/path/to/venv`; then `pybm workspace create my-branch -L /path/to/venv` will link the existing virtual
67-
environment into the benchmark workspace.
68-
69-
✅ Not only branch names work as valid git references - you can also supply tag names or full/partial commit SHAs. In the
70-
latter case, the SHA fragment is directly passed to git, which can fail to resolve a unique reference if the fragment is
71-
too short. For a project with lots of commits, increasing the SHA fragment length can help avoid resolution errors.
72-
73-
## pybm workspace delete
74-
75-
```
76-
pybm workspace delete <identifier> [<options>]
77-
78-
positional arguments:
79-
<id> Information that uniquely identifies the workspace. Can be name, checked out commit/branch/tag name, or worktree root directory.
80-
81-
optional arguments:
82-
-h, --help Show this message and exit.
83-
-v Enable verbose mode. Makes pybm log information that might be useful for debugging.
84-
-f, --force Force worktree removal, including untracked files and changes.
85-
```
86-
87-
Use the `pybm workspace delete` command to delete a benchmark workspace. The identifier can be the git reference name (
88-
partial SHAs also work), a benchmark workspace name or a directory name.
89-
90-
In the (standard) case of a virtual environment being created directly inside the git worktree, this virtual environment
91-
will be removed upon deletion of the benchmark workspace; this behavior cannot be changed as git physically removes
92-
the associated worktree. If you want to reuse a Python virtual environment, consider linking it explicitly from another
93-
location with the `-L` switch in the `pybm workspace create` command - pybm will not remove these.
94-
9529
## pybm workspace install
9630

9731
```

0 commit comments

Comments
 (0)