3. Rewrite $D, e, \tilde{D} \text{ and } \tilde{e}$ as
- $\hat{D}, \hat{e}, \check{D} \text{ and } \check{e}$ if $p_D > p_\tilde{D}$
- $\check{D}, \check{e}, \hat{D} \text{ and } \hat{e}$ otherwise
:::{.incremental}
4. For each component $n_{i,j}$ forward pass $Q\times\check{D}$ but replace $o_{i,j}^{\check{e}}$ by $o_{i,j}^{\hat{e}}$ for each $\check{e}$. Record the performance $\bar{p}$
5. $P = \frac{\bar{p} - p_\hat{D} }{p_\check{D} - p_\hat{D}}$ gives the impact of the perturbation on the model performance
...
...
@@ -121,14 +233,22 @@ Let $Q \times \tilde{D}$ the same set of pairs but with perturbed documents
### Animation de l'execution de Activation patching
### Perturbation
### Step 1: Choose a perturbation
Function that applies the same modification on each document.
"Let $Q \\times D \\subset \\mathcal{Q}\\times\\mathcal{D}$ be a set of pairs of questions and documents \n",
"Let $Q \\times \\tilde{D}$ the same set of pairs but with perturbed documents \n",
"\n",
"1. Forward pass all $Q\\times D$\n",
" - record $o_{i,j}^e$ the output of each component $n_{i,j}, \\forall e \\in Q\\times D$\n",
" - record $p_D$ the performance of the model\n",
"2. Forward pass all $Q\\times \\tilde{D}$\n",
" - record $o_{i,j}^\\tilde{e}$ the output of each component $n_{i,j}, \\forall \\tilde{e} \\in Q\\times \\tilde{D}$\n",
" - record $p_\\tilde{D}$ the performance of the model \n",
"3. Rewrite $D, e, \\tilde{D} \\text{ and } \\tilde{e}$ as\n",
"- $\\hat{D}, \\hat{e}, \\check{D} \\text{ and } \\check{e}$ if $p_D > p_\\tilde{D}$\n",
"- $\\check{D}, \\check{e}, \\hat{D} \\text{ and } \\hat{e}$ otherwise\n",
"4. For each component $n_{i,j}$ forward pass $Q\\times\\check{D}$ but replace $o_{i,j}^{\\check{e}}$ by $o_{i,j}^{\\hat{e}}$ for each $\\check{e}$. Record the performance $\\bar{p}$\n",
"5. $P = \\frac{\\bar{p} - p_\\hat{D} }{\\p_\\check{D} - p_\\hat{D}}$ gives the impact of the perturbation on the model performance\n",
"\n",
"\n",
"### Animation de l'execution de Activation patching\n",
"\n",
"### Perturbation\n",
"\n",
"Function that applies the same modification on each document.\n",
"Example : "
],
"id": "fa2ddf7d"
},
{
"cell_type": "code",
"metadata": {},
"source": [
"def perturbation(doc):\n",
" return doc.replace(\"microwave\", \"toaster\")"
],
"id": "2898ee6e",
"execution_count": null,
"outputs": []
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"#### Perturbation creation technique\n",
"\n",
"- Identify vocabulary specific to the dataset \n",
"- find in the vocabulary words with several meaning $m_D$ and $m_D$\n",
"- Replace that word by a synonym of the $m_D$ meaning\n",
"\n",
"\n",
"### What is a good perturbation \n",
"\n",
"- Have an impact of the documents representation\n",
"_Des images de courbes à ajouter ici_\n",
"- Be useful for interpretation\n",
"\n",
"### Enhance a model with MechIR\n",
"TODO\n",
"\n",
"\n",
"---"
],
"id": "069b4e0f"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"# Brouillon\n",
"- Mechir\n",
" - but et concept : cartographier les sensibilités des modèles encoder-based \n",
" - Activation Patching [Chen et al,. 2024]\n",
" - Étapes\n",
" - recul sur le résultat obtenu\n",
" - Perturbation\n",
" - Definition\n",
" - approche de création par étude du vocabulaire important et utilisation des mots poly-sémantiques\n",
"Let $Q \\times D \\subset \\mathcal{Q}\\times\\mathcal{D}$ be a set of pairs of questions and documents \n",
"Let $Q \\times \\tilde{D}$ the same set of pairs but with perturbed documents \n",
"\n",
"1. Forward pass all $Q\\times D$\n",
" - record $o_{i,j}^e$ the output of each component $n_{i,j}, \\forall e \\in Q\\times D$\n",
" - record $p_D$ the performance of the model\n",
"2. Forward pass all $Q\\times \\tilde{D}$\n",
" - record $o_{i,j}^\\tilde{e}$ the output of each component $n_{i,j}, \\forall \\tilde{e} \\in Q\\times \\tilde{D}$\n",
" - record $p_\\tilde{D}$ the performance of the model \n",
"3. Rewrite $D, e, \\tilde{D} \\text{ and } \\tilde{e}$ as\n",
"- $\\hat{D}, \\hat{e}, \\check{D} \\text{ and } \\check{e}$ if $p_D > p_\\tilde{D}$\n",
"- $\\check{D}, \\check{e}, \\hat{D} \\text{ and } \\hat{e}$ otherwise\n",
"4. For each component $n_{i,j}$ forward pass $Q\\times\\check{D}$ but replace $o_{i,j}^{\\check{e}}$ by $o_{i,j}^{\\hat{e}}$ for each $\\check{e}$. Record the performance $\\bar{p}$\n",
"5. $P = \\frac{\\bar{p} - p_\\hat{D} }{\\p_\\check{D} - p_\\hat{D}}$ gives the impact of the perturbation on the model performance\n",
"\n",
"\n",
"### Animation de l'execution de Activation patching\n",
"\n",
"### Perturbation\n",
"\n",
"Function that applies the same modification on each document.\n",
"Example : "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "d02801b1",
"metadata": {},
"outputs": [],
"source": [
"def perturbation(doc):\n",
" return doc.replace(\"microwave\", \"toaster\")"
]
},
{
"cell_type": "raw",
"id": "a4e2efff",
"metadata": {},
"source": [
"#### Perturbation creation technique\n",
"\n",
"- Identify vocabulary specific to the dataset \n",
"- find in the vocabulary words with several meaning $m_D$ and $m_D$\n",
"- Replace that word by a synonym of the $m_D$ meaning\n",
"\n",
"\n",
"### What is a good perturbation \n",
"\n",
"- Have an impact of the documents representation\n",
"_Des images de courbes à ajouter ici_\n",
"- Be useful for interpretation\n",
"\n",
"### Enhance a model with MechIR\n",
"TODO\n",
"\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "a80fea01",
"metadata": {},
"source": [
"---\n",
"# Brouillon\n",
"- Mechir\n",
" - but et concept : cartographier les sensibilités des modèles encoder-based \n",
" - Activation Patching [Chen et al,. 2024]\n",
" - Étapes\n",
" - recul sur le résultat obtenu\n",
" - Perturbation\n",
" - Definition\n",
" - approche de création par étude du vocabulaire important et utilisation des mots poly-sémantiques\n",
"Let $Q \\times D \\subset \\mathcal{Q}\\times\\mathcal{D}$ be a set of pairs of questions and documents \n",
"Let $Q \\times \\tilde{D}$ the same set of pairs but with perturbed documents \n",
"\n",
"1. Forward pass all $Q\\times D$\n",
" - record $o_{i,j}^e$ the output of each component $n_{i,j}, \\forall e \\in Q\\times D$\n",
" - record $p_D$ the performance of the model\n",
"2. Forward pass all $Q\\times \\tilde{D}$\n",
" - record $o_{i,j}^\\tilde{e}$ the output of each component $n_{i,j}, \\forall \\tilde{e} \\in Q\\times \\tilde{D}$\n",
" - record $p_\\tilde{D}$ the performance of the model \n",
"3. Rewrite $D, e, \\tilde{D} \\text{ and } \\tilde{e}$ as\n",
"- $\\hat{D}, \\hat{e}, \\check{D} \\text{ and } \\check{e}$ if $p_D > p_\\tilde{D}$\n",
"- $\\check{D}, \\check{e}, \\hat{D} \\text{ and } \\hat{e}$ otherwise\n",
"4. For each component $n_{i,j}$ forward pass $Q\\times\\check{D}$ but replace $o_{i,j}^{\\check{e}}$ by $o_{i,j}^{\\hat{e}}$ for each $\\check{e}$. Record the performance $\\bar{p}$\n",
"5. $P = \\frac{\\bar{p} - p_\\hat{D} }{\\p_\\check{D} - p_\\hat{D}}$ gives the impact of the perturbation on the model performance\n",
"\n",
"\n",
"### Animation de l'execution de Activation patching\n",
"\n",
"### Perturbation\n",
"\n",
"Function that applies the same modification on each document. \n",
"Example : "
],
"id": "99dbc701"
},
{
"cell_type": "code",
"metadata": {},
"source": [
"#| echo: true\n",
"def perturbation(doc):\n",
" return doc.replace(\"microwave\", \"toaster\")"
],
"id": "a87c9cff",
"execution_count": null,
"outputs": []
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"#### Perturbation creation technique\n",
"\n",
"- Identify vocabulary specific to the dataset \n",
"- find in the vocabulary words with several meaning $m_D$ and $m_D$\n",
"- Replace that word by a synonym of the $m_D$ meaning\n",
"\n",
"\n",
"### What is a good perturbation \n",
"\n",
"- Have an impact of the documents representation\n",
"_Des images de courbes à ajouter ici_\n",
"- Be useful for interpretation\n",
"\n",
"### Enhance a model with MechIR\n",
"TODO\n",
"\n",
"\n",
"---"
],
"id": "dab3024c"
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"# Brouillon\n",
"- Mechir\n",
" - but et concept : cartographier les sensibilités des modèles encoder-based \n",
" - Activation Patching [Chen et al,. 2024]\n",
" - Étapes\n",
" - recul sur le résultat obtenu\n",
" - Perturbation\n",
" - Definition\n",
" - approche de création par étude du vocabulaire important et utilisation des mots poly-sémantiques\n",
"Let $Q \\times D \\subset \\mathcal{Q}\\times\\mathcal{D}$ be a set of pairs of questions and documents \n",
"Let $Q \\times \\tilde{D}$ the same set of pairs but with perturbed documents \n",
"\n",
"1. Forward pass all $Q\\times D$\n",
" - record $o_{i,j}^e$ the output of each component $n_{i,j}, \\forall e \\in Q\\times D$\n",
" - record $p_D$ the performance of the model\n",
"2. Forward pass all $Q\\times \\tilde{D}$\n",
" - record $o_{i,j}^\\tilde{e}$ the output of each component $n_{i,j}, \\forall \\tilde{e} \\in Q\\times \\tilde{D}$\n",
" - record $p_\\tilde{D}$ the performance of the model \n",
"3. Rewrite $D, e, \\tilde{D} \\text{ and } \\tilde{e}$ as\n",
"- $\\hat{D}, \\hat{e}, \\check{D} \\text{ and } \\check{e}$ if $p_D > p_\\tilde{D}$\n",
"- $\\check{D}, \\check{e}, \\hat{D} \\text{ and } \\hat{e}$ otherwise\n",
"4. For each component $n_{i,j}$ forward pass $Q\\times\\check{D}$ but replace $o_{i,j}^{\\check{e}}$ by $o_{i,j}^{\\hat{e}}$ for each $\\check{e}$. Record the performance $\\bar{p}$\n",
"5. $P = \\frac{\\bar{p} - p_\\hat{D} }{\\p_\\check{D} - p_\\hat{D}}$ gives the impact of the perturbation on the model performance\n",
"\n",
"\n",
"### Animation de l'execution de Activation patching\n",
"\n",
"### Perturbation\n",
"\n",
"Function that applies the same modification on each document. \n",
"Example : "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "70279707",
"metadata": {},
"outputs": [],
"source": [
"#| echo: true\n",
"def perturbation(doc):\n",
" return doc.replace(\"microwave\", \"toaster\")"
]
},
{
"cell_type": "raw",
"id": "d1b08c4e",
"metadata": {},
"source": [
"#### Perturbation creation technique\n",
"\n",
"- Identify vocabulary specific to the dataset \n",
"- find in the vocabulary words with several meaning $m_D$ and $m_D$\n",
"- Replace that word by a synonym of the $m_D$ meaning\n",
"\n",
"\n",
"### What is a good perturbation \n",
"\n",
"- Have an impact of the documents representation\n",
"_Des images de courbes à ajouter ici_\n",
"- Be useful for interpretation\n",
"\n",
"### Enhance a model with MechIR\n",
"TODO\n",
"\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "3f975028",
"metadata": {},
"source": [
"---\n",
"# Brouillon\n",
"- Mechir\n",
" - but et concept : cartographier les sensibilités des modèles encoder-based \n",
" - Activation Patching [Chen et al,. 2024]\n",
" - Étapes\n",
" - recul sur le résultat obtenu\n",
" - Perturbation\n",
" - Definition\n",
" - approche de création par étude du vocabulaire important et utilisation des mots poly-sémantiques\n",
