@@ -303,35 +304,38 @@ Let <span class="math inline">\(Q \times \tilde{D}\)</span> the same set of pair
...
@@ -303,35 +304,38 @@ Let <span class="math inline">\(Q \times \tilde{D}\)</span> the same set of pair
<oltype="1">
<oltype="1">
<liclass="fragment">Forward pass all <spanclass="math inline">\(Q\times D\)</span>
<liclass="fragment">Forward pass all <spanclass="math inline">\(Q\times D\)</span>
<ul>
<ul>
<liclass="fragment">record <spanclass="math inline">\(o_{i,j}^e\)</span> the output of each component <spanclass="math inline">\(n_{i,j}, \forall e \in Q\times D\)</span></li>
<liclass="fragment">record <spanclass="math inline">\(o_{n}^e\)</span> the output of each component <spanclass="math inline">\(n, \forall e \in Q\times D\)</span></li>
<liclass="fragment">record <spanclass="math inline">\(p_D\)</span> the performance of the model</li>
<liclass="fragment">record <spanclass="math inline">\(p_D\)</span> the performance of the model</li>
</ul></li>
</ul></li>
<liclass="fragment">Forward pass all <spanclass="math inline">\(Q\times \tilde{D}\)</span>
<liclass="fragment">Forward pass all <spanclass="math inline">\(Q\times \tilde{D}\)</span>
<ul>
<ul>
<liclass="fragment">record <spanclass="math inline">\(o_{i,j}^\tilde{e}\)</span> the output of each component <spanclass="math inline">\(n_{i,j}, \forall \tilde{e} \in Q\times \tilde{D}\)</span></li>
<liclass="fragment">record <spanclass="math inline">\(o_{n}^\tilde{e}\)</span> the output of each component <spanclass="math inline">\(n, \forall \tilde{e} \in Q\times \tilde{D}\)</span></li>
<liclass="fragment">record <spanclass="math inline">\(p_\tilde{D}\)</span> the performance of the model<br>
<liclass="fragment">record <spanclass="math inline">\(p_\tilde{D}\)</span> the performance of the model<br>
</li>
</li>
</ul></li>
</ul></li>
<liclass="fragment">Rewrite <spanclass="math inline">\(D, e, \tilde{D} \text{ and } \tilde{e}\)</span> as
<liclass="fragment">Rewrite <spanclass="math inline">\(D, e, \tilde{D}\)</span> and <spanclass="math inline">\(\tilde{e}\)</span> as
<ul>
<ul>
<liclass="fragment"><spanclass="math inline">\(\hat{D}, \hat{e}, \check{D} \text{ and } \check{e}\)</span> if <spanclass="math inline">\(p_D > p_\tilde{D}\)</span></li>
<liclass="fragment"><spanclass="math inline">\(\hat{D}, \hat{e}, \check{D}\)</span> and <spanclass="math inline">\(\check{e}\)</span> if <spanclass="math inline">\(p_D > p_\tilde{D}\)</span></li>
<liclass="fragment"><spanclass="math inline">\(\check{D}, \check{e}, \hat{D} \text{ and } \hat{e}\)</span> otherwise</li>
<liclass="fragment"><spanclass="math inline">\(\check{D}, \check{e}, \hat{D}\)</span> and <spanclass="math inline">\(\hat{e}\)</span> otherwise</li>
<h3>Activation Patching <spanclass="citation"data-cites="chen_axiomatic_2024">(<ahref="#/references"role="doc-biblioref"onclick="">Chen et al. 2024</a>)</span></h3>
<h3>Activation Patching <spanclass="citation"data-cites="chen_axiomatic_2024">(<ahref="#/references"role="doc-biblioref"onclick="">Chen et al. 2024</a>)</span></h3>
<olstart="3"type="1">
<olstart="3"type="1">
<li>Rewrite <spanclass="math inline">\(D, e, \tilde{D} \text{ and } \tilde{e}\)</span> as
<li>Rewrite <spanclass="math inline">\(D, e, \tilde{D}\)</span> and <spanclass="math inline">\(\tilde{e}\)</span> as
<ul>
<ul>
<li><spanclass="math inline">\(\hat{D}, \hat{e}, \check{D} \text{ and } \check{e}\)</span> if <spanclass="math inline">\(p_D > p_\tilde{D}\)</span></li>
<li><spanclass="math inline">\(\hat{D}, \hat{e}, \check{D}\)</span> and <spanclass="math inline">\(\check{e}\)</span> if <spanclass="math inline">\(p_D > p_\tilde{D}\)</span></li>
<li><spanclass="math inline">\(\check{D}, \check{e}, \hat{D} \text{ and } \hat{e}\)</span> otherwise</li>
<li><spanclass="math inline">\(\check{D}, \check{e}, \hat{D}\)</span> and <spanclass="math inline">\(\hat{e}\)</span> otherwise</li>
</ul></li>
</ul></li>
</ol>
</ol>
<divclass="fragment">
<p>For each component <spanclass="math inline">\(n\)</span></p>
<olstart="4"type="1">
<olstart="4"type="1">
<liclass="fragment">For each component <spanclass="math inline">\(n_{i,j}\)</span> forward pass <spanclass="math inline">\(Q\times\check{D}\)</span> but replace <spanclass="math inline">\(o_{i,j}^{\check{e}}\)</span> by <spanclass="math inline">\(o_{i,j}^{\hat{e}}\)</span> for each <spanclass="math inline">\(\check{e}\)</span>. Record the performance <spanclass="math inline">\(\bar{p}\)</span></li>
<liclass="fragment">Forward pass <spanclass="math inline">\(Q\times\check{D}\)</span> but replace <spanclass="math inline">\(o_{i,j}^{\check{e}}\)</span> by <spanclass="math inline">\(o_{n}^{\hat{e}}\)</span> for each <spanclass="math inline">\(\check{e}\)</span>. Record the performance <spanclass="math inline">\(\bar{p}_n\)</span></li>
<liclass="fragment"><spanclass="math inline">\(P = \frac{\bar{p} - p_\hat{D} }{p_\check{D} - p_\hat{D}}\)</span> gives the impact of the perturbation on the model performance</li>
<liclass="fragment"><spanclass="math inline">\(P_{n} = \frac{\bar{p}_n - p_\hat{D} }{p_\check{D} - p_\hat{D}}\)</span> gives the impact of the perturbation on the model performance</li>
<h3>MechIR <spanclass="citation"data-cites="parry_mechir_2025">(<ahref="#/references"role="doc-biblioref"onclick="">Parry et al. 2025</a>)</span></h3>
<h3>MechIR <spanclass="citation"data-cites="parry_mechir_2025">(<ahref="#/references"role="doc-biblioref"onclick="">Parry et al. 2025</a>)</span></h3>
<ul>
<ul>
...
@@ -369,9 +377,9 @@ Let <span class="math inline">\(Q \times \tilde{D}\)</span> the same set of pair
...
@@ -369,9 +377,9 @@ Let <span class="math inline">\(Q \times \tilde{D}\)</span> the same set of pair
<p>Considering that activation patching charts the sensibility of a model to a perturbation</p>
<p>Considering that activation patching charts the sensibility of a model to a perturbation</p>
<p>Hypothesis :</p>
<p>Hypothesis :</p>
<ul>
<ul>
<li>Models have <strong>specific components dedicated to domain specific language</strong></li>
<li>Models have <strong>specific components on which rely domain specific language</strong></li>
<li>This domain specific language dedicated part of the model can be <strong>charted with activation patching</strong></li>
<li>This part of the model implementing domain specific language can be <strong>charted with activation patching</strong></li>
<li>That map can be used to <strong>enhance model comprehension and robustness</strong> of the domain specific language dedicated part of the model</li>
<li>That map can be used to <strong>enhance model comprehension and robustness</strong> of the part of the model implementing domain specific language</li>
</ul>
</ul>
</section>
</section>
</section>
</section>
...
@@ -400,7 +408,7 @@ Let <span class="math inline">\(Q \times \tilde{D}\)</span> the same set of pair
...
@@ -400,7 +408,7 @@ Let <span class="math inline">\(Q \times \tilde{D}\)</span> the same set of pair
<li>solution -> answer
<li>solution -> answer
<ul>
<ul>
<li><em>microwave a measurements of dielectric absorption in dilute <strong>solutions</strong></em></li>
<li><em>microwave a measurements of dielectric absorption in dilute <strong>solutions</strong></em></li>
<li><em>microwave a measurements of dielectric absorption in dilute answer</em></li>
<li><em>microwave a measurements of dielectric absorption in dilute <strong>answers</strong></em></li>
</ul></li>
</ul></li>
<li>microwave -> toaster
<li>microwave -> toaster
<ul>
<ul>
...
@@ -408,7 +416,7 @@ Let <span class="math inline">\(Q \times \tilde{D}\)</span> the same set of pair
...
@@ -408,7 +416,7 @@ Let <span class="math inline">\(Q \times \tilde{D}\)</span> the same set of pair
<li><em><strong>toaster</strong> a measurements of dielectric absorption in dilute solutions</em></li>
<li><em><strong>toaster</strong> a measurements of dielectric absorption in dilute solutions</em></li>
@@ -476,7 +485,8 @@ Perturbed Document: [CLS] toaster measurements of dielectric absorption in dilut
...
@@ -476,7 +485,8 @@ Perturbed Document: [CLS] toaster measurements of dielectric absorption in dilut
<p>E5-small <spanclass="citation"data-cites="wang_text_2022">(<ahref="#/references"role="doc-biblioref"onclick="">Wang et al. 2022</a>)</span></p>
<p>E5-small <spanclass="citation"data-cites="wang_text_2022">(<ahref="#/references"role="doc-biblioref"onclick="">Wang et al. 2022</a>)</span></p>
<ul>
<ul>
<li>Constrative bi-encoder</li>
<li>Constrative bi-encoder</li>
<li>Trained on unlabeled pairs (query, passage) CCPairs</li>
<li>Pre-trained on unlabeled pairs (query, passage) CCPairs</li>
<li>Finetuned with labelled pairs</li>
<li>Based on MiniLM <spanclass="citation"data-cites="wang_minilmv2_2021">(<ahref="#/references"role="doc-biblioref"onclick="">Wang et al. 2021</a>)</span></li>
<li>Based on MiniLM <spanclass="citation"data-cites="wang_minilmv2_2021">(<ahref="#/references"role="doc-biblioref"onclick="">Wang et al. 2021</a>)</span></li>
<li>33M params</li>
<li>33M params</li>
</ul>
</ul>
...
@@ -490,7 +500,7 @@ Perturbed Document: [CLS] toaster measurements of dielectric absorption in dilut
...
@@ -490,7 +500,7 @@ Perturbed Document: [CLS] toaster measurements of dielectric absorption in dilut
3. Rewrite $D, e, \tilde{D} \text{ and } \tilde{e}$ as
3. Rewrite $D, e, \tilde{D}$ and $\tilde{e}$ as
- $\hat{D}, \hat{e}, \check{D} \text{ and } \check{e}$ if $p_D > p_\tilde{D}$
- $\hat{D}, \hat{e}, \check{D}$ and $\check{e}$ if $p_D > p_\tilde{D}$
- $\check{D}, \check{e}, \hat{D} \text{ and } \hat{e}$ otherwise
- $\check{D}, \check{e}, \hat{D}$ and $\hat{e}$ otherwise
. . .
For each component $n$
:::{.incremental}
:::{.incremental}
4. For each component $n_{i,j}$ forward pass $Q\times\check{D}$ but replace $o_{i,j}^{\check{e}}$ by $o_{i,j}^{\hat{e}}$ for each $\check{e}$. Record the performance $\bar{p}$
4. Forward pass $Q\times\check{D}$ but replace $o_{i,j}^{\check{e}}$ by $o_{n}^{\hat{e}}$ for each $\check{e}$. Record the performance $\bar{p}_n$
5. $P = \frac{\bar{p} - p_\hat{D} }{p_\check{D} - p_\hat{D}}$ gives the impact of the perturbation on the model performance
5. $P_{n} = \frac{\bar{p}_n - p_\hat{D} }{p_\check{D} - p_\hat{D}}$ gives the impact of the perturbation on the model performance
:::
:::
...
@@ -164,13 +170,17 @@ Let $Q \times \tilde{D}$ the same set of pairs but with perturbed documents
...
@@ -164,13 +170,17 @@ Let $Q \times \tilde{D}$ the same set of pairs but with perturbed documents
