Skip to content

Commit

Permalink
mention new run all script
Browse files Browse the repository at this point in the history
  • Loading branch information
adamkarvonen committed Nov 16, 2024
1 parent b91c210 commit 916df28
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ python evals/sparse_probing/main.py \

The results will be saved to the evals/sparse_probing/results directory.

We use regex patterns to select SAE Lens SAEs. For more examples of regex patterns, refer to `sae_selection.ipynb`.
We use regex patterns to select SAE Lens SAEs. For more examples of regex patterns, refer to `sae_regex_selection.ipynb`.

Every eval folder contains an `eval_config.py`, which contains all relevant hyperparamters for that evaluation. The values are currently set to the default recommended values.

Expand All @@ -68,7 +68,7 @@ There are two ways to evaluate custom SAEs:
- Simpler interface requiring only model, SAE, and config values
- Graphing will require manual formatting

We currently have a suite of SAE Bench SAEs on layers 3 and 4 of Pythia-70M and layers 5, 12, and 19 of Gemma-2-2B, each trained on 200M tokens. These SAEs can serve as baselines for any new custom SAEs. We also have baseline eval results, saved at TODO.
The script `run_all_evals_custom_saes()` will run approach 1 on all SAE Bench evals. We currently have a suite of SAE Bench SAEs on layers 3 and 4 of Pythia-70M and layers 5, 12, and 19 of Gemma-2-2B, each trained on 200M tokens. These SAEs can serve as baselines for any new custom SAEs. We also have baseline eval results, saved at TODO.

## Training Your Own SAEs

Expand Down

0 comments on commit 916df28

Please sign in to comment.