---
title: "Mechanistic interpretability for enhancing RAG models"
author: Marine Delvallez
format: 
  revealjs:
    toc: true
    toc-depth: 2
    slide-level: 3
    mouse-wheel: true
jupyter : python3
bibliography: biblio.bib
style: |
  .columns {
    display: grid;
    grid-template-columns: repeat(2, minmax(0, 1fr));
    gap: 1rem;
  }
  .small {
    font-size: 20px
  }
  .midsize{
    font-size: 25px
  }
---

## Retrieval Augmented Generation - RAG

### Retrieval Augmented Generation - Definition

![Simple RAG Architecture [@lewis_retrieval-augmented_2020]](images/DefRAG.drawio.png)  

. . . 

![Advanced RAG Achitecture [@fan_survey_2024]](images/DefRAGAvance_integration.drawio.png)


### Example {.center}

![RAG Architecture used here (inspired from [@tran_retrieval_2024])](images/RAGHN-perso.drawio.png)

## Explainability in Artificial Intelligence

### Explainability in Artificial Intelligence

_Open black box models_

#### Aims

- Trustability
- Understandability
- Model Rectification

### Definitions

**XAI** : Make model's behavior understandable for human [@bell_its_2022]  
**Understand** : Predict model's behavior [@bell_its_2022]  
**Explanation** : Any way to make the decision process of the model understandable for human

:::: {.columns}

::: {.column width="40%"}
**Interpretability**  
_How ?_
:::

::: {.column width="40%"}

**Explanability**  
_Why ?_
:::

::::

### Explanation through creation of a model

![Explanation through creation of a model](images/VieModele-TempsXAI.drawio.png)

## Presentation of MechIR 

### MechIR

#### Mechanistic interpretability 
Understand the internal mechanisms of neural networks by **performing causal interventions** on specific model components

#### MechIR [@parry_mechir_2025]
- Encoder-only models 
- For Information Retrieval models

- Identify components responsible for some behavior
- Activation Patching Technique 

### Activation Patching [@chen_axiomatic_2024] {.smaller}
Let $Q \times D \subset \mathcal{Q}\times\mathcal{D}$ be a set of pairs of questions and documents  
Let $Q \times \tilde{D}$ the same set of pairs but with perturbed documents  

:::{.incremental}

1. Forward pass all $Q\times D$
    - record $o_{i,j}^e$ the output of each component $n_{i,j}, \forall e \in Q\times D$
    - record $p_D$ the performance of the model
2. Forward pass all $Q\times \tilde{D}$
    - record $o_{i,j}^\tilde{e}$ the output of each component $n_{i,j}, \forall \tilde{e} \in Q\times \tilde{D}$
    - record $p_\tilde{D}$ the performance of the model  
3. Rewrite $D, e, \tilde{D} \text{ and } \tilde{e}$ as
    - $\hat{D}, \hat{e}, \check{D} \text{ and } \check{e}$ if $p_D > p_\tilde{D}$
    - $\check{D}, \check{e}, \hat{D} \text{ and } \hat{e}$ otherwise

:::

### Activation Patching [@chen_axiomatic_2024] {.smaller}

:::{.incremental}

3. Rewrite $D, e, \tilde{D} \text{ and } \tilde{e}$ as
    - $\hat{D}, \hat{e}, \check{D} \text{ and } \check{e}$ if $p_D > p_\tilde{D}$
    - $\check{D}, \check{e}, \hat{D} \text{ and } \hat{e}$ otherwise
4. For each component $n_{i,j}$ forward pass $Q\times\check{D}$ but replace $o_{i,j}^{\check{e}}$ by $o_{i,j}^{\hat{e}}$ for each $\check{e}$. Record the performance $\bar{p}$
5. $P = \frac{\bar{p} - p_\hat{D} }{p_\check{D} - p_\hat{D}}$ gives the impact of the perturbation on the model performance

:::

### Animation de l'execution de Activation patching

### Perturbation

Function that applies the same modification on each document.  
Example :  
``` {python}
#| echo: true
def perturbation(doc):
  return doc.replace("microwave", "toaster")
``` 


#### Perturbation creation technique

- Identify vocabulary specific to the dataset 
- Find in the vocabulary words with several meaning $m_D$ and $m_D$
- Replace that word by a synonym of the $m_D$ meaning


### What is a good perturbation 

- Has an impact of the documents representation
- Be useful for interpretation

![Perturbation Score](images/perturbation-score.png)

### Chart Information Retrieval Model

![Chart of the impact of a perturbation on the components](images/ActivationPatchingAllHeadExempleMechIR.png)

### Perspective : Enhance a model with MechIR

### References

::: {#refs}
:::

