From 98d5f57a03dffaa481657b9154cb7197d2adfaeb Mon Sep 17 00:00:00 2001 From: canrager <61095597+canrager@users.noreply.github.com> Date: Fri, 27 Sep 2024 14:06:58 +0200 Subject: [PATCH] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 657f567..e9349a8 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ This repo contains the template we would like to use for the SAE Bench project. The `template.ipynb` is a good starting point explaining the input to your custom eval (SAEs hosted on SAELens) and the output (a standardized results file). -The `sparse_probing` folder contains a full example implementation of a custom eval. In `sparse_probing_eval.py`, we have a function that takes a list of SAELens SAE names (defined in `eval_config.py`) and an sae release and returns a dictionary of results in a standard format. This folder contains some helper functions, like pre-computing model activations in `activation_collection.py`, that might be useful for you, too! We try to reuse functions as much as possible across evals to reduce bugs. Let Adam and Can know if you've implemented a helper function that might be useful for other evals as well (like autointerp, feauture scoring). +The `sparse_probing` folder contains a full example implementation of a custom eval. In `sparse_probing_eval.py`, we have a function that takes a list of SAELens SAE names (defined in `eval_config.py`) and an sae release and returns a dictionary of results in a standard format. This folder contains some helper functions, like pre-computing model activations in `activation_collection.py`, that might be useful for you, too! We try to reuse functions as much as possible across evals to reduce bugs. Let Adam and Can know if you've implemented a helper function that might be useful for other evals as well (like autointerp, feature scoring). `python3 sparse_probing_eval.py` should run as is and demonstrate how to use our SAE Bench SAEs with Transformer Lens and SAE Lens. It will also generate a results file which can be graphed using `graph_sae_results.ipynb`.