---
title: "Mechanistic interpretability for enhancing RAG models"
format: 
  revealjs:
    toc: true
    toc-depth: 2
    slide-level: 3
    mouse-wheel: true
jupyter : python3
style: |
  .columns {
    display: grid;
    grid-template-columns: repeat(2, minmax(0, 1fr));
    gap: 1rem;
  }
  .small {
    font-size: 20px
  }
  .midsize{
    font-size: 25px
  }
---

## Retrieval Augmented Generation - RAG

### Retrieval Augmented Generation - Definition

![Simple RAG Architecture](images/DefRAG.drawio.png)  

![Advanced RAG Achitecture](images/DefRAGAvance_integration.drawio.png)


### Approche plus théorique de retriever et generateur

_Retriever comme plongement des documents et questions dans un espace_

_Generateur comme fonction d'une paire (question, ensemble de documents) vers texte_



### Example

![RAG Architecture used here](images/RAGHN-perso.drawio.png)

## Explainability in Artificial Intelligence

### Explainability in Artificial Intelligence

Aims:

- Trustability
- Understandability
- Model Rectification

### Definitions

**XAI** : Make model's behavior understandable for human [@bell_its_2022]  
**Understand** : Predict model's behavior [@bell_its_2022]  
**Explanation** : Any way to make decision process understandable for human

:::: {.columns}

::: {.column width="40%"}
**Interpretability**  
_How ?_
:::

::: {.column width="40%"}

**Explanability**  
_Why ?_
:::

::::

## Explanation through creation of a model

![Explanation through creation of a model](images/VieModele-TempsXAI.drawio.png)

## MechIR [@parry_mechir_2025]

### MechIR [@parry_mechir_2025]

#### Mechanistic interpretability 
Understand the internal mechanisms of neural networks by **performing causal interventions** on specific model components

#### MechIR
- Encoder-only models 
- For Information Retrieval models

- Identify components responsible for some behavior
- Activation Patching Technique 

### Activation Patching [@chen_axiomatic_2024] {.smaller}
Let $Q \times D \subset \mathcal{Q}\times\mathcal{D}$ be a set of pairs of questions and documents  
Let $Q \times \tilde{D}$ the same set of pairs but with perturbed documents  

1. Forward pass all $Q\times D$
  - record $o_{i,j}^e$ the output of each component $n_{i,j}, \forall e \in Q\times D$
  - record $p_D$ the performance of the model
2. Forward pass all $Q\times \tilde{D}$
  - record $o_{i,j}^\tilde{e}$ the output of each component $n_{i,j}, \forall \tilde{e} \in Q\times \tilde{D}$
  - record $p_\tilde{D}$ the performance of the model  
3. Rewrite $D, e, \tilde{D} \text{ and } \tilde{e}$ as
- $\hat{D}, \hat{e}, \check{D} \text{ and } \check{e}$ if $p_D > p_\tilde{D}$
- $\check{D}, \check{e}, \hat{D} \text{ and } \hat{e}$ otherwise
4. For each component $n_{i,j}$ forward pass $Q\times\check{D}$ but replace $o_{i,j}^{\check{e}}$ by $o_{i,j}^{\hat{e}}$ for each $\check{e}$. Record the performance $\bar{p}$
5. $P = \frac{\bar{p} - p_\hat{D} }{\p_\check{D} - p_\hat{D}}$ gives the impact of the perturbation on the model performance


### Animation de l'execution de Activation patching

### Perturbation

Function that applies the same modification on each document.  
Example :  
``` {python}
#| echo: true
def perturbation(doc):
  return doc.replace("microwave", "toaster")
``` 


#### Perturbation creation technique

- Identify vocabulary specific to the dataset 
- Find in the vocabulary words with several meaning $m_D$ and $m_D$
- Replace that word by a synonym of the $m_D$ meaning


### What is a good perturbation 

- Has an impact of the documents representation
- Be useful for interpretation

![Perturbation Score](images/perturbation-score.png)

### Chart Information Retrieval Model

![Chart of the impact of a perturbation on the components](images/ActivationPatchingAllHeadExempleMechIR.png)

### Perspective : Enhance a model with MechIR



---
---
# Brouillon
- Mechir
  - but et concept : cartographier les sensibilités des modèles encoder-based 
  - Activation Patching [Chen et al,. 2024]
    - Étapes
    - recul sur le résultat obtenu
  - Perturbation
    - Definition
    - approche de création par étude du vocabulaire important et utilisation des mots poly-sémantiques
    - 3 types de perturbation (append, prepend, replace) -> préférer replace
    - identifier une perturbation pertinente
  - Améliorer le modèle
    - Quelle modification effectuer?
    - 