Skip to content

Commit 76956f7

Browse files
committed
Fix(examples): fix the activation_factory settings of lorsa examples
1 parent 5c38684 commit 76956f7

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

examples/train_pythia_lorsa_topk.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -81,8 +81,8 @@
8181
],
8282
target=ActivationFactoryTarget.ACTIVATIONS_2D,
8383
hook_points=["blocks.6.ln1.hook_normalized", "blocks.6.hook_attn_out"],
84-
batch_size=4096,
85-
buffer_size=4096 * 4,
84+
batch_size=32,
85+
buffer_size=None,
8686
buffer_shuffle=BufferShuffleConfig(
8787
perm_seed=42,
8888
generator_device="cuda",

0 commit comments

Comments
 (0)