{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "title: \"Mechanistic interpretability for enhancing RAG models\"\n",
        "format: \n",
        "  revealjs:\n",
        "    toc: true\n",
        "    toc-depth: 2\n",
        "    code-fold: true\n",
        "    slide-level: 3\n",
        "    mouse-wheel: true\n",
        "jupyter : python3\n",
        "style: |\n",
        "  .columns {\n",
        "    display: grid;\n",
        "    grid-template-columns: repeat(2, minmax(0, 1fr));\n",
        "    gap: 1rem;\n",
        "  }\n",
        "  .small {\n",
        "    font-size: 20px\n",
        "  }\n",
        "  .midsize{\n",
        "    font-size: 25px\n",
        "  }\n",
        "---\n",
        "\n",
        "## Retrieval Augmented Generation - RAG\n",
        "\n",
        "### Retrieval Augmented Generation - Definition\n",
        "\n",
        "![Simple RAG Architecture](images/DefRAG.drawio.png)  \n",
        "\n",
        "![Advanced RAG Achitecture](images/DefRAGAvance_integration.drawio.png)\n",
        "\n",
        "\n",
        "### Approche plus théorique de retriever et generateur\n",
        "\n",
        "_Retriever comme plongement des documents et questions dans un espace_\n",
        "\n",
        "_Generateur comme fonction d'une paire (question, ensemble de documents) vers texte_\n",
        "\n",
        "\n",
        "\n",
        "### Example\n",
        "\n",
        "![RAG Architecture used here](images/RAGHN-perso.drawio.png)\n",
        "\n",
        "## Explainability in Artificial Intelligence\n",
        "\n",
        "### Explainability in Artificial Intelligence\n",
        "\n",
        "Aims:\n",
        "\n",
        "- Trustability\n",
        "- Understandability\n",
        "- Model Rectification\n",
        "\n",
        "### Definitions\n",
        "\n",
        "**XAI** : Make model's behavior understandable for human [@bell_its_2022]  \n",
        "**Understand** : Predict model's behavior [@bell_its_2022]  \n",
        "**Explanation** : Any way to make decision process understandable for human\n",
        "\n",
        ":::: {.columns}\n",
        "\n",
        "::: {.column width=\"40%\"}\n",
        "**Interpretability**  \n",
        "_How ?_\n",
        ":::\n",
        "\n",
        "::: {.column width=\"40%\"}\n",
        "\n",
        "**Explanability**  \n",
        "_Why ?_\n",
        ":::\n",
        "\n",
        "::::\n",
        "\n",
        "## Explanation through creation of a model\n",
        "\n",
        "![Explanation through creation of a model](images/VieModele-TempsXAI.drawio.png)\n",
        "\n",
        "## MechIR [@parry_mechir_2025]\n",
        "\n",
        "### MechIR [@parry_mechir_2025]\n",
        "\n",
        "#### Mechanistic interpretability \n",
        "Understand the internal mechanisms of neural networks by **performing causal interventions** on specific model components\n",
        "\n",
        "#### MechIR\n",
        "- Encoder-only models \n",
        "- For Information Retrieval models\n",
        "\n",
        "- Identify components responsible for some behavior\n",
        "- Activation Patching Technique \n",
        "\n",
        "### Activation Patching [@chen_axiomatic_2024] {.smaller}\n",
        "Let $Q \\times D \\subset \\mathcal{Q}\\times\\mathcal{D}$ be a set of pairs of questions and documents  \n",
        "Let $Q \\times \\tilde{D}$ the same set of pairs but with perturbed documents  \n",
        "\n",
        "1. Forward pass all $Q\\times D$\n",
        "  - record $o_{i,j}^e$ the output of each component $n_{i,j}, \\forall e \\in Q\\times D$\n",
        "  - record $p_D$ the performance of the model\n",
        "2. Forward pass all $Q\\times \\tilde{D}$\n",
        "  - record $o_{i,j}^\\tilde{e}$ the output of each component $n_{i,j}, \\forall \\tilde{e} \\in Q\\times \\tilde{D}$\n",
        "  - record $p_\\tilde{D}$ the performance of the model  \n",
        "3. Rewrite $D, e, \\tilde{D} \\text{ and } \\tilde{e}$ as\n",
        "- $\\hat{D}, \\hat{e}, \\check{D} \\text{ and } \\check{e}$ if $p_D > p_\\tilde{D}$\n",
        "- $\\check{D}, \\check{e}, \\hat{D} \\text{ and } \\hat{e}$ otherwise\n",
        "4. For each component $n_{i,j}$ forward pass $Q\\times\\check{D}$ but replace $o_{i,j}^{\\check{e}}$ by $o_{i,j}^{\\hat{e}}$ for each $\\check{e}$. Record the performance $\\bar{p}$\n",
        "5. $P = \\frac{\\bar{p} - p_\\hat{D} }{\\p_\\check{D} - p_\\hat{D}}$ gives the impact of the perturbation on the model performance\n",
        "\n",
        "\n",
        "### Animation de l'execution de Activation patching\n",
        "\n",
        "### Perturbation\n",
        "\n",
        "Function that applies the same modification on each document.\n",
        "Example : "
      ],
      "id": "fa2ddf7d"
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "def perturbation(doc):\n",
        "  return doc.replace(\"microwave\", \"toaster\")"
      ],
      "id": "2898ee6e",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "raw",
      "metadata": {},
      "source": [
        "#### Perturbation creation technique\n",
        "\n",
        "- Identify vocabulary specific to the dataset \n",
        "- find in the vocabulary words with several meaning $m_D$ and $m_D$\n",
        "- Replace that word by a synonym of the $m_D$ meaning\n",
        "\n",
        "\n",
        "### What is a good perturbation \n",
        "\n",
        "- Have an impact of the documents representation\n",
        "_Des images de courbes à ajouter ici_\n",
        "- Be useful for interpretation\n",
        "\n",
        "### Enhance a model with MechIR\n",
        "TODO\n",
        "\n",
        "\n",
        "---"
      ],
      "id": "069b4e0f"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "---\n",
        "# Brouillon\n",
        "- Mechir\n",
        "  - but et concept : cartographier les sensibilités des modèles encoder-based \n",
        "  - Activation Patching [Chen et al,. 2024]\n",
        "    - Étapes\n",
        "    - recul sur le résultat obtenu\n",
        "  - Perturbation\n",
        "    - Definition\n",
        "    - approche de création par étude du vocabulaire important et utilisation des mots poly-sémantiques\n",
        "    - 3 types de perturbation (append, prepend, replace) -> préférer replace\n",
        "    - identifier une perturbation pertinente\n",
        "  - Améliorer le modèle\n",
        "    - Quelle modification effectuer?\n",
        "    - "
      ],
      "id": "0253d293"
    }
  ],
  "metadata": {
    "kernelspec": {
      "name": "python3",
      "language": "python",
      "display_name": "Python 3 (ipykernel)",
      "path": "/home/marine/miniconda3/share/jupyter/kernels/python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}