Skip to content
BranchesTags

Latest commit

 

History

19 Commits

Folders and files

NameName
Last commit message
Last commit date
rm_save_files
rm_save_files
 
 
sae_results
sae_results
 
 
README.md
README.md
 
 
atp_feature_search.py
atp_feature_search.py
 
 
attr_patch_feature_search.ipynb
attr_patch_feature_search.ipynb
 
 
check_top_reward.ipynb
check_top_reward.ipynb
 
 
custom_prompts.ipynb
custom_prompts.ipynb
 
 
dictionary.py
dictionary.py
 
 
e2e.ipynb
e2e.ipynb
 
 
find_top_feature_activations.ipynb
find_top_feature_activations.ipynb
 
 
hh_feature_viewer.ipynb
hh_feature_viewer.ipynb
 
 
interp_utils.py
interp_utils.py
 
 
requirements.txt
requirements.txt
 
 
rm_dataset_filtering.ipynb
rm_dataset_filtering.ipynb
 
 
rm_feature_interp.ipynb
rm_feature_interp.ipynb
 
 
sae_metrics.ipynb
sae_metrics.ipynb
 
 
supervised_sae_training.ipynb
supervised_sae_training.ipynb
 
 
task_attri_patching_evals.ipynb
task_attri_patching_evals.ipynb
 
 
task_patching_utils.py
task_patching_utils.py
 
 

Repository files navigation

Use custom_prompts.ipynb to find important features for various SAE layers for a given prompt.

Note: we use attribution patching w/ integrated gradients. You might run out of GPU memory. Keep prompts short or integrated gradient steps small to avoid OOM. 45GB recommended.

About

Using SAE's to interpret Reward Models (RM)

Resources

Readme
Activity

Stars

7 stars

Watchers

1 watching

Forks

2 forks
Report repository

Releases

Packages

Used by

Contributors

Languages