Lorenzo Basile (@lorebasile) header
Lorenzo Basile (@lorebasile) profile picture

Postdoc @UniLeiden

Trieste, Italy
Joined June 2020
🌈 Property enhancement/inhibition We can control image captioning with head-level interventions. Rescaling heads specialized on colors or sentiments we can promote or remove related words, without finetuning. [5/6]
1
3
148
🙃 Head inversion By switching the sign of only 1–3% attention heads selected by our method, we can: •reduce the toxicity of generated text •targetedly disrupt image classification performance of VLMs ⬇️ [4/6]
1
3
86
🔎 How to identify specialized heads? Using Matching Pursuit, we rank heads by their alignment with interpretable directions. This approach reveals consistent head specialization (numbers, colors...), improving on what single-sample methods like the Logit Lens can detect. [3/6]
1
3
93
We show that attention heads in LLMs & VLMs specialize in distinct semantic properties, and that editing a few of them enables targeted control over generation. [2/6]
1
3
113