Skip to content

R
RAGRights
Commit 69859c8a authored by Delvallez Delvallez's avatar Delvallez Delvallez
Browse files
Options

rangement présentations et rapports

parent 7f0fd3d8
Hide whitespace changes
Inline Side-by-side
Showing with 3237 additions and 530 deletions
.gitignore
View file @ 69859c8a
......@@ -25,4 +25,11 @@ seminaireCA/indications_oral.md
seminaireCA/test/
expes/exp5_MainV2_perturbations_multiples.html
expes/exp7_pos-neg-all.html
expes/styles_recap-expes.css
\ No newline at end of file
expes/styles_recap-expes.css
rapports_presentations/rapportS10/notes.md
rapports_presentations/rapportS9/brouillon.md
rapports_presentations/rapportS9/consignes.md
rapports_presentations/SoutenanceS9/SoutenanceXAI_S3_2026-01-07_10h.pdf
rapports_presentations/SoutenanceS9/brouillon.md
rapports_presentations/seminaireCA/indications_oral.md
rapports_presentations/seminaireCA/archives-utiles.md
\ No newline at end of file
rapportS10/notes.md deleted 100644 → 0
View file @ 7f0fd3d8
# Notes des idées et choses à intégrer pour le rapport
## À mentionner dans le rapport
- Génération de questions pour l'entraînement : biais dans les questions (temps) et dans les sujet (lecture rapide des données)
- Les constantes de rerank et chunking ont été choisies arbitrairement ou par contrainte technique. Il serai intéressant de les optimiser. Ça n'a pas été fait faut de temps et pas choix de s'intéresser à d'autres choses
- les éléments pris en compte pour l'extraction du vocabulaire caractéristique (quand commencer, quand s’arrêter, quel biais de compréhension)
- présentation de l'archi utilisée (+ présentation de RAG4HN??)
- justifier utilisation de Md comme format de texte
- dans les mesure d'effet d'une perturbation, la baseline change un peu de forme (dû à kde ?). Les petites perturbations pourraient être des artefacts liés à l'**affichage** de kde
## Éléments bibliographiques
- E5
- Cohere
- RAG
- llama3
- explicabilité rapidement
# Récapitulatif du stage
## Introduction
### Contexte du stage
- LIFO
- Challenge RAG@EvalLLM
- Sorties et Événements au cours du stage
- TransIA à Tours [Cahier Minerve]
- Journée GDR TAL-I (Jussieu) [Cahier Minerve]
- 2nd Symposium on Mental Health and AI [R16/02]
- Place du Numérique (ambassadrice)
- Journée CA (présentation) [Cahier Minerve]
### Outils utilisés
- CaSciModOT et Mirev pour l'execution des architectures et protocoles long/lourds
- HuggingFace, PyTorch, TransformerLens
- Scripts python et Notebooks Python
- git, gogit
- IA : reformulation, exploration de la littérature, traduction, rédaction de scripts simple (parfois améliorés à la main)
### Difficultés Techniques
- Contraintes d'outil de CaSciModOT (conda-forge)
- Reprendre le code de qq1
- pas de contact (code recherche)
- montée en version nécessaire pour l'exploitation des librairies (conda-forge)
- Conflit de requirements entre deux pans du projet (difficultés à installer un environnement fonctionnel)
- Modèles entrainés sur des données anglophone -> données à traiter francophones
### Introduction du sujet
TODO
## État de l'art
### RAG et RALLM
- définition, enjeux visés
- RAG vs RALLM [Fan+23]
- Architecture RAG (intuition et archi complete) [Fan+23, R9/02]
- extraction (base à préparer si bi-encodeur)
- generation
- amélioration
- intégration
- Tâches associées (QA, IR) [Git-Csv]
- Quelques approches vues [R16/02]
- Multimodal [Lui+25]
- Chain of Thought [Xu+24]
- Gestion BDD dynamique
- BDD sparse et interprétable
- Archis et modèle pour la suite [R24/04]
- E5 [Wang+24a], mE5 [Wang+24b]
- RAG4HN [Tran+25]
### Trustworthy RAG [R9/02]
- Définition et composantes de Trustworthy
- 2 temps pour le RAG (Extraction et Génération)
- Métriques
- Interprétation et Explication
- Paysage [Somvanshi+2026] + se situer
- Activation Patching [Chen+24] [R30/03]
- MechIR [Parry+25]
- Perturbation
- Interprétation des graphiques/ données
## Construction d'un dataset [R5/05, R20/05]
- Données dont on dispose
- Structure de données visée (motivations à cette structure : taille des documents/textes, format une question, 50% de +/-) [RAGRIGHTS-Specs]
- Outils d'extraction et mise en forme du texte (PyMuPDF4LLM, Langchain-text-splitter)
## Adaptation de RAG4HN [R20/05, Tran+25]
- Présentation de l'Archi (si pas déjà fait)
- Motivation des modifications (morceaux inutiles, format de données différent, reranking)
## Cartographie d'un modèle d'extraction d'information (nom à trouver)
### Méthodologie
- rappel de l'objectif
- Méthode perturbation [ragrights-protocole]
- stats
- langue de spécialité
- niveaux de perturbation
- type de perturbation
- questionner les biais à la construction (dû à la compréhension humaine inconsciente)
- Méthode application de activation patching [ragrights-protocole]
### Résultats et analyse
- comportements observés & interprétation
- Préconisations d'usage
- couverture de la perturbation
- activation des dernières couches
- définition du critère de sensibilité
- les moyens/ méthodes de visualisation des données
- utiliser la question comme document pour avoir un témoin ???
## Perspectives
- Reprendre les expériences avec les préconisations
- Généraliser le protocole pour les dataset à plusieurs questions ??
- Élargir les préconisations et le protocole pour la langue de spécialité (pas juste le vocabulaire)
- Proposer un protocole d'évaluation de l'importance d'un noeud identifié
- Transformer ces résultats en explication
- formulation
- exploitation lors de l'usage
- Intégration des explications dans l'architecture
- mise en perspective des explications dans le modèle
- explicabilité sur les autres composants du modèle
- Exploitation des explications pour l'affinement/ amélioration du modèle d'extraction
--------------------------------------------------------------------------------------
# Plan (gros grain)
Introduction
- Environnement du stage
- Outils utilisés
- **Difficultés techniques** (à placer autrement)
- Sujet du stage
État de l'art
- RAG & RALLM
- Le problème de la confiance pour les systèmes de RAG
Travaux réalisés
- Construction d'un dataset
- Adaptation du modèle RAG4HN
- Cartographie d'un modèle d'extraction d'information
- Méthodologie
- Résultats et Analyse
- Perspectives ?
- Conclusion et Perspectives
# Détail des sous parties
## Environnement du stage
- Stage à lieu au LIFO, dans l'équipe CA, encadré par ...
- Stage de fin de master d'informatique Minerve GPEx
- Challenge@EvalLLM comme cas d'usage, participation envisagée mais timing pas OK
- différents événements et sorties organisées pendant le stage
- TransIA : Tours, Présentations pluridisciplinaires sur l'intégration de l'IA (sous toutes ses formes) dans la société
- TAL-I : journée du GDR TaL-I à Paris (Jussieu) sur les travaux portant sur l'interprétabilité et l'explicabilité dans le contexte des tâches de traitement de la langue naturelle
- 2nd Symposium on Mental Health and AI : deux jours de conférence (suivi en pointillé en distanciel) présentation des travaux à la croisée entre santé mentale et intelligence artificielle (impact, outils de détection, de prise en charge)
- Place du Numérique : événement public de sensibilisation et de communication sur le Numérique organisé place du Martoi à Orléans (Des événements similaires organisés dans chaque département de la région CVL) J'ai participé à la tenue du stand de l'université, été ambassadrice de l'événement et ai joué le rôle d'un témoin dan sle tribunal des génération futures portant sur "L'IA nous contrôle-t-elle?
- Présentation aux journées d'équipe CA
## Outils utilisés au cours du stage
- Développement de l'architecture et réalisation des expérimentations en python dans 3 formats (script, implémentation d'un CLI, notebook)
- Principales librairies python utilisées : PyTorch, TransformerLens, HuggingFace, PyMuPDF4LLM,...
- Utilisation de plusieurs git pour assurer le suivi des versions des outils développé lors du stage
- Pour l'execution de tâches demandantes en charge de calcul (Génération de la base de représentation des données, execution des différentes architectures explorées et/ou implémentées) : utilisation des grappes de calcul accessibles aux chercheurs du LIFO CaSciModOT et Mirev
- utilisation d'outils à base d'IA très modérée et pour les tâches de l'ordre de reformulation (ChatGPT), traduction (DeepL), exploration de la littérature (LitMaps, Perplexity), rédaction de petites fonctions et outils python (ChatGPT)
## Introduction du sujet du stage
TODO
## Générateurs et Modèles de Langue Augmentés par l'Extraction (RAG et RALLM)
- **Besoin et définition**
- Les modèles profonds apprennent sur les données [ref à récupérer de Lewis+20]
- Le problème des modèles génératifs : hallucination et connaissance bornée dans le temps [ref à récupérer de Lewis+20]
- Il y a quelques années, on a cherché à associer un base de connaissances à ces modèles [Lewis+20] (= définition de RAG)
- Parmis ces approches, on retrouve le cas des modèles de langue : RALLM
- **L'architecture RAG** Description de l'archi de façon plus avancée
- Ces modèles peuvent être utilisés pour différents types de tâches (voir tableau csv sur RAGRights)
- Avec le temps, l'idée de [Lewis+20] a été adaptée et augmentée de composants supplémentaires
- [Fan+23] propose la structure générique ... (exploiter slides Séminaire CA)
- **RAG amélioré**??? Au delà de l'innovation de l'architecture, on retrouve des innovations sur plusieurs caratéristiques de ces modèles : [R24/04] [R16/02]
- Multimodal [Liu+25]
- Chain of Thought [Xu+24]
- Gestion BDD dynamique [ref à trouver dans R24/05]
- BDD sparse et interprétable [Prouteau]???
- **Modèle et Architecture utilisées durant le stage** Dans le cadre du stage,
- travailler sur une adaptation du modèle RAG4HN. [Tran+25]
- _description de l'architecture RAG4HN_ + parallèle avec l'architecture générique de [Fan+23]
- L'extracteur de RAG4HN est mE5 [Wang+25b] (version multilingue de E5 [Whang+25a])
## Construire des systèmes RAG de confiance
- **Trustworthy AI** La définition d'IA de confiance (Trustworthy AI) tend à varier en fonction des situation et des utilisations de cette expression. Danss le contexte du Machine Learning, on désigne l'IA de confiance comme l'ensembles des systèmes d'IA dans lesquels il est légitime (dans le sens humainement entendable) d'avoir confiance. Le NIST (National Institute of Standards and Technology (US)) propose différents axes ou composantes de l'IA de confiance : ...
- **Explicabilité dans le contexte du RAG** Dans la suite, nous nous concentrons sur l'explicabilité dans le contexte du RAG. [Ni+25] distingue deux temps pour l'explicabilité : Extraction et génération ... (prévoir la question de l'association/intégration des deux)
- Métrique ??
- **Interprétabilité Mécaniste** [Somvanshi+25]
- Interprétabilité vs Explicabilité
- Définition d'Interprétabilité Mécaniste et approches (Manual Circuit Tracing, *Intervention-based technique*, Representation analysis, Toy Model and synthetic tasks)
- **Activation patching**
- Définition de activation patching [Chen+24] [R30/04] [Séminaire CA]
- MechIR [Parry+25]
- Perturbation
- Données produites, visualisation et interprétation des graphiques
---
# Reprise locale du plan
- Construction préalable des outils /!\ à reformuler
- Construction d'un dataset
- Architecture visée
- Objectif de l'expérience
- Localiser le vocabulaire/ la langue de spécialité dans le modèle
- j
- Contribution méthodologique
- Construction de perturbations
# Remarques et éléments à prendre en compte
## 2026-07-17
- Gestion de la traduction des concepts : ???
- et ce papier de l'autre jour, qu'en faire
-
- Rester théorie et résultats
- Prévoir annexe pour les résultats
- Prévoir annexe pour difficultés techniques (librairie, prise en main CaSciModOT)
- Placer "on ne dispose pas de données gold"
- Prouteau : dans Explicabilité RAG
- [x] 2.2 Système de RAG de confiance -> Interprétabilité pour les systèmes de RAG
- Développer le Challenge dans dataset :
- Le challenge la tache
- Les données fournies
- ce qui nous manque
- --
- Le format de dataset visé
- tous les dérouler V0, V1 et format V2 (à chaque fois: format, ce qui manque, les modifications proposées)
- [à rédiger] 1.3 la progression du stage en bref et le glissement du sujet
- 1.4 placer problématique résultats du cheminement
- garder l'évolution du sujet dans le bilan [personnel] ou intro (pas dans le mémoire)
- 3, 4 et 5 => reformulation
- 3 : prérequis à l'éxpé
- datset
- reprise d'un modèle
- 4 : ce qui était le 5.2 (Construction des pert, Estimation pertinence des pert, interprétation de activation Patching) _contribution méthodologique_
- 5 : résultat expérimentaux (ancinement 5.3)
- liste des perturbations choisies
- Ajouter touts les graphiques avec un texte succin => tri plus tart
---
# Typos à pister
- mécanistique -> mécaniste
- toin, iton, taion....
- staion
- pertuabtion
- resulatas, resulatats
- présentaion -> présentation
- intelliegnce -> intelligence
- extrcation -> extraction
- accord du mot noeud + \oe
- scores de perturbation -> socre de pertinence
\ No newline at end of file
rapports_presentations/SoutenanceS9/images/Modele-Systeme-Input-Output.drawio.svg 0 → 100644
View file @ 69859c8a
<?xml version="1.0" encoding="UTF-8"?>
<!-- Do not edit this file with editors other than draw.io -->
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<svg xmlns="http://www.w3.org/2000/svg" style="background: #ffffff; background-color: light-dark(#ffffff, var(--ge-dark-color, #121212)); color-scheme: light dark;" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" width="550px" height="171px" viewBox="0 0 550 171" content="&lt;mxfile host=&quot;app.diagrams.net&quot; agent=&quot;Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:146.0) Gecko/20100101 Firefox/146.0&quot; version=&quot;29.2.9&quot; scale=&quot;1&quot; border=&quot;5&quot;&gt;&#xA; &lt;diagram name=&quot;Page-1&quot; id=&quot;qM8VPNshVXkk-n-AxLaP&quot;&gt;&#xA; &lt;mxGraphModel dx=&quot;989&quot; dy=&quot;550&quot; grid=&quot;1&quot; gridSize=&quot;10&quot; guides=&quot;1&quot; tooltips=&quot;1&quot; connect=&quot;1&quot; arrows=&quot;1&quot; fold=&quot;1&quot; page=&quot;1&quot; pageScale=&quot;1&quot; pageWidth=&quot;1169&quot; pageHeight=&quot;827&quot; math=&quot;0&quot; shadow=&quot;0&quot;&gt;&#xA; &lt;root&gt;&#xA; &lt;mxCell id=&quot;0&quot; /&gt;&#xA; &lt;mxCell id=&quot;1&quot; parent=&quot;0&quot; /&gt;&#xA; &lt;mxCell id=&quot;2&quot; parent=&quot;1&quot; style=&quot;rounded=0;whiteSpace=wrap;html=1;&quot; value=&quot;&amp;lt;div&amp;gt;&amp;lt;b&amp;gt;Système&amp;lt;/b&amp;gt; : Fonction associée à la tâche tierce&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;&amp;lt;br&amp;gt;&amp;lt;/div&amp;gt;&quot; vertex=&quot;1&quot;&gt;&#xA; &lt;mxGeometry height=&quot;160&quot; width=&quot;360&quot; x=&quot;290&quot; y=&quot;90&quot; as=&quot;geometry&quot; /&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;3&quot; edge=&quot;1&quot; parent=&quot;1&quot; style=&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;&quot; value=&quot;&quot;&gt;&#xA; &lt;mxGeometry height=&quot;50&quot; relative=&quot;1&quot; width=&quot;50&quot; as=&quot;geometry&quot;&gt;&#xA; &lt;Array as=&quot;points&quot;&gt;&#xA; &lt;mxPoint x=&quot;230&quot; y=&quot;170&quot; /&gt;&#xA; &lt;/Array&gt;&#xA; &lt;mxPoint x=&quot;200&quot; y=&quot;170&quot; as=&quot;sourcePoint&quot; /&gt;&#xA; &lt;mxPoint x=&quot;270&quot; y=&quot;170&quot; as=&quot;targetPoint&quot; /&gt;&#xA; &lt;/mxGeometry&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;4&quot; edge=&quot;1&quot; parent=&quot;1&quot; style=&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;&quot; value=&quot;&quot;&gt;&#xA; &lt;mxGeometry height=&quot;50&quot; relative=&quot;1&quot; width=&quot;50&quot; as=&quot;geometry&quot;&gt;&#xA; &lt;Array as=&quot;points&quot;&gt;&#xA; &lt;mxPoint x=&quot;230&quot; y=&quot;120&quot; /&gt;&#xA; &lt;/Array&gt;&#xA; &lt;mxPoint x=&quot;200&quot; y=&quot;120&quot; as=&quot;sourcePoint&quot; /&gt;&#xA; &lt;mxPoint x=&quot;270&quot; y=&quot;120&quot; as=&quot;targetPoint&quot; /&gt;&#xA; &lt;/mxGeometry&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;5&quot; edge=&quot;1&quot; parent=&quot;1&quot; style=&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;&quot; value=&quot;&quot;&gt;&#xA; &lt;mxGeometry height=&quot;50&quot; relative=&quot;1&quot; width=&quot;50&quot; as=&quot;geometry&quot;&gt;&#xA; &lt;Array as=&quot;points&quot;&gt;&#xA; &lt;mxPoint x=&quot;230&quot; y=&quot;170&quot; /&gt;&#xA; &lt;/Array&gt;&#xA; &lt;mxPoint x=&quot;200&quot; y=&quot;170&quot; as=&quot;sourcePoint&quot; /&gt;&#xA; &lt;mxPoint x=&quot;270&quot; y=&quot;170&quot; as=&quot;targetPoint&quot; /&gt;&#xA; &lt;/mxGeometry&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;6&quot; edge=&quot;1&quot; parent=&quot;1&quot; style=&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;&quot; value=&quot;&quot;&gt;&#xA; &lt;mxGeometry height=&quot;50&quot; relative=&quot;1&quot; width=&quot;50&quot; as=&quot;geometry&quot;&gt;&#xA; &lt;Array as=&quot;points&quot;&gt;&#xA; &lt;mxPoint x=&quot;230&quot; y=&quot;220&quot; /&gt;&#xA; &lt;/Array&gt;&#xA; &lt;mxPoint x=&quot;200&quot; y=&quot;220&quot; as=&quot;sourcePoint&quot; /&gt;&#xA; &lt;mxPoint x=&quot;270&quot; y=&quot;220&quot; as=&quot;targetPoint&quot; /&gt;&#xA; &lt;/mxGeometry&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;7&quot; parent=&quot;1&quot; style=&quot;rounded=1;whiteSpace=wrap;html=1;&quot; value=&quot;&amp;lt;div&amp;gt;&amp;lt;b&amp;gt;Modèle&amp;amp;nbsp;&amp;lt;/b&amp;gt;:&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;Représentation de l’environnement&amp;lt;/div&amp;gt;&quot; vertex=&quot;1&quot;&gt;&#xA; &lt;mxGeometry height=&quot;120&quot; width=&quot;190&quot; x=&quot;300&quot; y=&quot;120&quot; as=&quot;geometry&quot; /&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;8&quot; parent=&quot;1&quot; style=&quot;rounded=1;whiteSpace=wrap;html=1;&quot; value=&quot;&amp;lt;div&amp;gt;Partie entraînée pour&amp;amp;nbsp;&amp;lt;/div&amp;gt;&amp;lt;div&amp;gt;la tâche tierce&amp;lt;/div&amp;gt;&quot; vertex=&quot;1&quot;&gt;&#xA; &lt;mxGeometry height=&quot;120&quot; width=&quot;130&quot; x=&quot;510&quot; y=&quot;120&quot; as=&quot;geometry&quot; /&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;9&quot; edge=&quot;1&quot; parent=&quot;1&quot; style=&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;&quot; value=&quot;&quot;&gt;&#xA; &lt;mxGeometry height=&quot;50&quot; relative=&quot;1&quot; width=&quot;50&quot; as=&quot;geometry&quot;&gt;&#xA; &lt;Array as=&quot;points&quot;&gt;&#xA; &lt;mxPoint x=&quot;700&quot; y=&quot;169&quot; /&gt;&#xA; &lt;/Array&gt;&#xA; &lt;mxPoint x=&quot;670&quot; y=&quot;169&quot; as=&quot;sourcePoint&quot; /&gt;&#xA; &lt;mxPoint x=&quot;740&quot; y=&quot;169&quot; as=&quot;targetPoint&quot; /&gt;&#xA; &lt;/mxGeometry&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;10&quot; edge=&quot;1&quot; parent=&quot;1&quot; style=&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;&quot; value=&quot;&quot;&gt;&#xA; &lt;mxGeometry height=&quot;50&quot; relative=&quot;1&quot; width=&quot;50&quot; as=&quot;geometry&quot;&gt;&#xA; &lt;Array as=&quot;points&quot;&gt;&#xA; &lt;mxPoint x=&quot;700&quot; y=&quot;119&quot; /&gt;&#xA; &lt;/Array&gt;&#xA; &lt;mxPoint x=&quot;670&quot; y=&quot;119&quot; as=&quot;sourcePoint&quot; /&gt;&#xA; &lt;mxPoint x=&quot;740&quot; y=&quot;119&quot; as=&quot;targetPoint&quot; /&gt;&#xA; &lt;/mxGeometry&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;11&quot; edge=&quot;1&quot; parent=&quot;1&quot; style=&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;&quot; value=&quot;&quot;&gt;&#xA; &lt;mxGeometry height=&quot;50&quot; relative=&quot;1&quot; width=&quot;50&quot; as=&quot;geometry&quot;&gt;&#xA; &lt;Array as=&quot;points&quot;&gt;&#xA; &lt;mxPoint x=&quot;700&quot; y=&quot;169&quot; /&gt;&#xA; &lt;/Array&gt;&#xA; &lt;mxPoint x=&quot;670&quot; y=&quot;169&quot; as=&quot;sourcePoint&quot; /&gt;&#xA; &lt;mxPoint x=&quot;740&quot; y=&quot;169&quot; as=&quot;targetPoint&quot; /&gt;&#xA; &lt;/mxGeometry&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;mxCell id=&quot;12&quot; edge=&quot;1&quot; parent=&quot;1&quot; style=&quot;shape=flexArrow;endArrow=classic;html=1;rounded=0;&quot; value=&quot;&quot;&gt;&#xA; &lt;mxGeometry height=&quot;50&quot; relative=&quot;1&quot; width=&quot;50&quot; as=&quot;geometry&quot;&gt;&#xA; &lt;Array as=&quot;points&quot;&gt;&#xA; &lt;mxPoint x=&quot;700&quot; y=&quot;219&quot; /&gt;&#xA; &lt;/Array&gt;&#xA; &lt;mxPoint x=&quot;670&quot; y=&quot;219&quot; as=&quot;sourcePoint&quot; /&gt;&#xA; &lt;mxPoint x=&quot;740&quot; y=&quot;219&quot; as=&quot;targetPoint&quot; /&gt;&#xA; &lt;/mxGeometry&gt;&#xA; &lt;/mxCell&gt;&#xA; &lt;/root&gt;&#xA; &lt;/mxGraphModel&gt;&#xA; &lt;/diagram&gt;&#xA;&lt;/mxfile&gt;&#xA;"><defs/><rect fill="#ffffff" style="fill: light-dark(#ffffff, var(--ge-dark-color, #121212));" width="100%" height="100%" x="0" y="0"/><g><g data-cell-id="0"><g data-cell-id="1"><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-1"><g transform="translate(0.5,0.5)"><rect x="95" y="5" width="360" height="160" fill="#ffffff" style="fill: light-dark(#ffffff, var(--ge-dark-color, #121212)); stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke="#000000" pointer-events="all"/></g><g><g><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 358px; height: 1px; padding-top: 85px; margin-left: 96px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; color: #000000; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: light-dark(#000000, #ffffff); line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; "><div><b>Système</b> : Fonction associée à la tâche tierce</div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div><div><br /></div></div></div></div></foreignObject><image x="96" y="13.5" width="358" height="147" xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABZgAAAJMCAYAAAB3pcNzAAAgAElEQVR4XuzdC/w122PPd0pFilKqqqhD6lohibqEag8iQequSiV6XEtciqKu7T9uEUSTiFJCD21oRKO0JaTilEpR1J1ocaql1RZF9aLVdn3yOvP6TyYz+7J+v72f/TzPe71e83p+z++3Z/bMe6+ZPfNda9Z8yS+hECBAgAABAgQIECBAgAABAgQIECBAgACBCYEvOTGPWQgQIECAAAECBAgQIECAAAECBAgQIECAwJcQMKsEBAgQIECAAAECBAgQIECAAAECBAgQIDAlIGCeYjMTAQIECBAgQIAAAQIECBAgQIAAAQIECAiY1QECBAgQIECAAAECBAgQIECAAAECBAgQmBIQME+xmYkAAQIECBAgQIAAAQIECBAgQIAAAQIEBMzqAAECBAgQIECAAAECBAgQIECAAAECBAhMCQiYp9jMRIAAAQIECBAgQIAAAQIECBAgQIAAAQICZnWAAAECBAgQIECAAAECBAgQIECAAAECBKYEBMxTbGYiQIAAAQIECBAgQIAAAQIECBAgQIAAAQGzOkCAAIE5ga86ZvsvxvS1N7P/T+P/X3dM/9fcYs1FgAABAgQIECBAgAABAgQIEHh5BATML89nZU0JLALfcPzwHcb0Lcb0zcb0Vcb0Fcf05cb0t8f0N8f018b0x8b0h8b0n43pT+O7icA3GUv9/DF9hc3Sf8T4/6+6yTtaKAECBAgQeP0Eumb5fWP68M2m/9Tx/098xTi+9NiezxrT99xs1yeP//+EV2xbX4XN+fpjI/74mD5gtTF/ffzc+fr//CpsoG0gQIAAAQKXCAiYL1HyGgKPIfD9x2r8mDF924nV+f1jnl8xpl83pv9vYn6zHAt8l/Gn/2hMX2r1kj83fv5GY/p/wb12Av/K2OJCgFuVLzsWrHf8rXRPL/eXjz//yNVLfvz4+VNezKp4VwIPLfBnxtoVri3lQ8YPNXo/pbTvtQ+uy78x/vOzn7LQB573y4x1++wxffRqHf/e+PlbjekPv4D1zvp9q/f9BePnn/YC1uMR3/J3jZWq48e2fMb4xcc84Ar/u5v16vriMx9wPR91lW5xfHvUbbVeBAgQuEpAwHwVlxcTeCEC//h4108f03d6hnev988PHdMXPMOyLOL9Aj92/PipG5DvPf7/myG9dgIC5lf3Ixcwv7qfrS17XoHnDmD+0bF6f3ZM3a21lJ8/fvgZz7vaD7e0esTWgL0+/ytcLmQubL5nETDva3/s+HWdN5byP44fqq9L+cjxw+fe84O64L0EzBcgnXjJcx/fnrY25iZAgMADCQiYH+jDsCoEdgS+zvjd7xnT13xGnb8xlvXtx/TU3kTPuEp3XdT3He/21Xbe8f8ev/uVT1iTXzbm/bjV/PUa397K+4TFm/UlERAwvyQf1MRqCpgn0MzyWgo8dwBT78rvt5L8pPHzT3pNZD9wbOdvG9Obq+1tmIxb3imzRytg/uIqX3n8qoaPnslRaUi67zWm/3xM3/i93/358e8Hj+n/eKD6KmB+2ofx3Me3p62NuQkQIPBAAgLmB/owrAqBjcCXH/9vTLdC5ucujQn3zcf03z33gl+C5fVgvm+9s57/2/jddizlazanITL+kzF91Gqmbzd+7kJDeX0E9gLmxkV/rqFp6hlliIznqU9/aiymgGAp32D88LdOLFrA/DzulvLqCzxnANMwVAWsS/m08UN3Db1OpWds/I4xLUOkdb5SgHnPc7hXLWC+9vi/V9961sYPe+8P//34t/PqHvT8T4zpvxzTV3rvb58w/v3pD1RhBczv/zD63n9n9dlULz7izGf1nMe3B6oWVoUAAQJPFxAwP93QEgjcSqAT0h5ec678nfGC/3ZM/8+YGk5jOaE9N99vGC/4F8+96BX8+60C5qj+wTG1/KXnSre2fvdX0NAmHQvsBczGTX7MGlND2z+8WrWOnTUGHBUB82N+jtbq8QSeK4D5B8amFfi88d4mdpdRYzE/V4Pd48kdr1HnFw218C3fe8lvHf9+jztuwKsWMF97/N9SF/b/3jF1LV2jbx0KCpWX8p3HD3U6+PvG1B1yHzamP3nHz+vUWwmY36/zT24+lz86/v+hZz6n5zq+PUh1sBoECBB4PgEB8/NZWhKB5xRonMG/Oqb1E6nXy6/3Sj0nfumY/uLmjeuF+x3HVDjdOH2nSidRnUy9TuWWAXOOBVYf9B5oF8F/8HXCta1fQsD88lSCawOGjsdferV5f3f8XHCgECDwRQUKhtfXGA0PMPPQ21845vvJ7y36145/f/CYXsdwedGtEawHyhVWVu75rAcB8/vr+N8/fvwjY/om7/2qXsy/eucg8FPG7z7xvd9//vj3n36Q+itgfv+HNRMwP9fxzfcGAQIEXjkBAfMr95HaoFdE4AeM7fj1B9tSj+V/dkyXPEV876nr68X+3PGfn/WKmF26GbcOmC9dD697NQUEzC/P53ptwPzybJk1JfDyC3SN0rjLNerUkPObxjQTUr/8El90C77K+G+9Yyt1RPhP77SBAub3Q3/t8eM/995//9r4998/8Rn8S+NvDXlX+Q/G1EMAX3QRML//E5gJmF/05+f9CRAg8LACAuaH/Wis2GsuUE+IH3Jg8OPG7+u5fGn5JeOFPRBmr/xX45dLT5hLl/eyv07A/LJ/go+9/gLmx/581msnYH55PitrSoDAixUQML9Y/+d8dwHz+zUFzM9ZsyyLAIHXXkDA/NpXAQAPKtDDXD7yYN0a37fxvy4t3VJZD4vGgduWekMvPSsuXd49XvePjTep51IP4/tmY6rHzj80pm6z/V/H1MNU/tCYGv+ucQiveejZcwfM/8h4/+82pnqz9GCXHsLWECd/Y0w9gOf3j6neLbd62F+f3w8aUw9CagzA//299+022rfH9Od2PrCO/W+OqZ7yGX/1MTW+418f038zpt89pho5Gtt7plTXPnxMjQ/ZLaEtP5d6oVUX/9iYetr6rxnTqTFvZ977Rc8jYH7Rn8Dl7y9gvtzKKwkQeL0FBMyvzucvYH7/ZylgfnXqtS0hQOABBATMD/AhWAUCOwL1LP6QA5kCzYKRa0rDaRz1VC6gLGiu/Mtj+hUHCy4I/KpjumTM0ULFwt+98u+NX37swd++xvh9Pa6/z5i+1IUbWJD7b43p542pAHpbPnD84uNWv6wHeA9D3JbGU10/5bug9becWIcC2X99TC2v8fjOlTx+/pg+59wLx9+7/fLdg9f94vH7xqTs+P0Tx/Qzx3T0YMfGqszzp41p+dy+5vi5YPc7nVmPHhr5i8bUECp/74J1Xl7yHcYPnzqmTtrPlerULxvTp4ypJ6+/CuVRAuYaDgr43xxTAf9XG9P/OaZuz23c9t8+ph5C+e4zolcnP3pM33dMjU35Dd/7XGvk+K/H1BiqNQzNlpb5z4+pJ7x3rKjRojr0V8b0J8b02WOqcW7vOLD3nvcOmGuA6jOpMahjUOtfY0y3ubcNNQr9x0802tvOPv8fPqbG5P/gMdUA1mfS1EOp/u0rzGY/u/V8PSegRrmGeuqBZX2v1IhY+dtj+h/G1OdZQ9dvHlMNXzOlxsmlofLrj59rpOxY3XdGjZSNj98+UJ2ZGXrh1stvm3Op8bKHxX6DMVVn+s6u7lZvajCtznzemC75bj7l+Nz718xntszTd1p15HuNqc+uOvxlxvQXxtR3cw9L+/Qx1Yj71PJPjQV0zvHmmGrcXs6x/vz4uem3jek/HNNMHXnqunXe9lFjal/5umOqPnTuUUNyx74/O6bOF9tP/sCYbjk+9VMC5o5znXN0ftD5Yc5tS+dnf2tMNTz3MMe24bPec3+q3bn5n+P4f8/P5zn3z1MBc5/Vm2PqIeAdnzvmdMzuvKHv8Tq4dN72lH3vnvv3uXrwIgLmF7H97X//6pg6B+jz/Dlj6hz/VOm427GxY/EbY+q8q++Zzlc6FneN1HTtNen6PR/5+Huu7vg7AQI7AgJm1YLAYwoUwnzzg1Vr7L0uiK8pnQR/rYMZWlbBU+Urj6kA6igw7eKgi/5zpROXgs+90sVyT9beli6ifuOYuniaKZ3sFG51wbUuXcT8LxML7IKyi9u90glaQXEXSdeWzxgzNCbfqdD2XMD88WP+xvI76uW+XacukHMvoCjo7kGEl5bfMF7Yhca50vfJJ4+pwP3aUq/0Qrd6e9+ydGK8fkhb79X/rwnQz63fiw6YO24U2PeE+3OlwKSQpkaESwL+H/re65flrvfl6mJD9xQGHZXCj3rz9wDSv3Ru5VZ/L4yt0aJw9lxpO37SmLqA3pYulBrH9dJSI1+97ZfSxVjLXsqPHz9kfa50kVbjUvv9JeddHWMb1mj93qfeowv9Go4qfXd8i/d+7kFErV/ve6oRrLC1OvD2uQ154t+/7Ji/z75jRGHvJaXvpk8aU98pl96p0l0+NToWyl1SvmC8qLr9+y558XjNrZffamRVA2KNnl/ugvUqCK1u9r11bXmu/Wv9vjVcFBwsZbsvHa1j9bT9qjDz3HZ3PO940r61/d6/xKDv8YKy7vw5Vwq1q4e/ckz3CJprIKzRvHOuS0v7/o8ZUyHtLcpswPwxY2U6vpz6btiu72eOXxSG/eVn3JCnHv/Xq3LPz+cW++dRwNx7/boxnTt2dlzufK862kPHLy333L9PrVPnKu0rl5TOT7ffV494fGt/qQPKUt4cP3SnYOe4b4/pB242tmNndzHulY69daDpPORcB5o653RudNTJ58j4kY+/l9QLryFA4EDgkgsdeAQI3F/gd463POphWq+zbzOmenzcojTkRK3Ve+XfHL/sovdc2Z58La+v51i9ITohWZfCsB5U00X1U0rheMsqbF7KcwfMhfWfO6bC+NlSyFVoe9Tj7FTA3AXxNxrTNReerecvHNP3H9Ne7+1z23H0hPRlvr5LGlLjB59b0Im/d5FSA8HvecIyzs36qgfMP2IA/PIx7Q2Hc8qm3ic18NQT7lQ5CpgLDOspf+ldB4VhHzqmequeK99xvKAeMufCpu1y6s38L4ypnvhLeWrAMBMw1wOs3uLXHi9q9Ggc/i72z5W9gLnAuWN5zpeW7vSo/tyidNwv/KwX9Uypd3cNGktj6NEy+u6qB+QHXPkmedc7vp6gp8qtl99716O7ht9rPrtlnQtAf9SYLg1Bn3P/WrvNBDA1fNZwe9S4fvS5dByp8emSxu9lGR0r+y7dNjieqzY17BbUXNrYcW55e3/vboPW7Vywszdv69UdJPVqf+5ybcDc+tfQ0/nDTKkRsvOcvi+eozz1+L+swz0/n1vtn3sBc72TO84e3RG39xl0rO0ukUvKPffvc+vzIgLmW2//XsDc0HwFyR0TtuUoYK4zUvXg651D3Py9htpvP6buQDpXHvn4e27d/Z0AgTMCAmZVhMBjCnziWK2fcmLVGhv357534nBN74FLtrYQsl6re6VePOdOOro47tbdveNLIeT2YqOLkD8+pm6nf47SrcKdlC/lOQPmbk3u1u03nmFFT/UeOBUwF5LOXHg+ZZWrb92eexRa1BOkE/anlm77rTfZH3nqgg7mf5UD5sLBQom90gV6F+s14DRMQ/votnS7dRcHp0LmvYC54LBe0Ev50+OHetDVmFSdKSSrPm/LJT3j248bwqDbqNelMLCLmS5kClI/aEx7QVF3RBQyL+WpAcO1AXNhao1R3Y66LTWG9blUJ/P5Ojuvqcd3x8uGtDlVtgFzn2MB33JMbb/qgrGGt3o11wO33qXbELbwtp55l/acPrNaX+TPNSCuj8vLHxueqR7U9VTsc+4W3Ka9xopPGL9fD2O0ff96yVb3GkphXTputfxlTPk33nuP7XdU21/j3bsHG3br5fe2NUTUk3rv+7AgtQbm9q16xfeaPacaJeq1fq489/61fr9rA+aOSQXEe0MrdVdC4VcN0zWQtr9vS8FqjbY1LJ0rP3q8oO+r7effcaXjV+cvhWz57j2jonOMAu3nPvdqvY+GF+tY0PGuetyQEh13O/Y13M22VD+qx23Hc5ZrA+buYuvOg23pmNd2tN9n3p1gbcv2ON98n/+eyXMM/fHU4/+9P59b7p/bgPlnjI2robihtCp9LjX2d2dAjbSd99d5Y++7rMB9fQ6wV+fuuX9fUufvHTDfY/v3AuYaaOqJvFf2rkHaD/vc985H3h2/b+ou0+6G3Gv0b/iUjmGnhrV65OPvJXXHawgQOCMgYFZFCDymQCcFBQLnShfEDXnQbVANL1Aw18XFU0oBRBd0Rz0GO7HYe3Dc8p717mmc5b3SbfQFLutyKtDuJLeeOPXmasyvLvq6wGyeQpKjUu/vAo3KcwbMBXjr8ZzX799FX7fQNq5nvcvr6dx6vnliPbsNca/H7qmAeb24gpNC/96vi+FLbztvGc3bRXv1pXnP9VqpV9le8NuJaKH7UX3ps+t21xoRupAswGoYiaPXFzJc2zv7BPEX+dOrGjA3NmPjrG+DtS766rne57yUejdX72rEWoZSWP7WuKZ9zts7DJa/bwPmLkoLEQoGurBoDPeOR+vS+/3IMf2CMW0vTo/qVPMXtBX2LBe8/a462z7WrbkdD5ZSg0vvXaPbNnBZDynU2KrdlrmUwqj1kDzfc/x/HRoVVq7/f03A3HG0oLZAf13yKSQtRFyHJh3PGiu928nXpXD4m47pVC++dcBcPSh8L2xbvH72ZjtafutV78LtnTIF+o33+5yloYa2wV/rWPjUsWHdy7z37TMvWM8px6X0WXShftSLOdPu7llK219dacpoXfoe6/3rtbwub4//HN2Jcevltx6fMabtkEQ1DtXgXA/wdc/ZnOoJ1n64bSxoGUcNxb3PLfavteO1AXPbth0Cp3Cxbes7ct242WfX7z9289k19np/237W65fVsNK50jqYr5Gj4+GvGNN6LNEa5Or5V8PGMgTNsqxPGz/82M37P/W/HSsb5mLdc71jRGOkd+v79hjQNVz7b8eNNzdv3pAUHQ+fs1wTMOdVOLn+nm+ogcW5n9el75AaA1vvGibXpUbn53hI8lOP//f8fG69f24D5s61l32iXskNjbAdnqR16vu+v61L1ws1eJ26a+Je+/el9b3jxLJPdw77q1Yztp91PrGUvp+6xlqXRzy+bQPmjgsdA9bHus7TOsb0mXWtUiP8uuzdPds5QZ975/BL6djTXW8NwbUd+qZzvd57rzzy8fdglf2aAIFrBQTM14p5PYH7CZwah/nUWvQgjoLAeiL2bydC145BXEC8Ha9rec/GeSzkOSr1ntpe+PXaLtw6Ce1Edl0KGI5usSuY2BtTsmNXrfLvO1iJvZ7Sy0t7KFIh57YUXuz1zlhe18nou2Pa6zFWz8DvOqZCk3XpgqRxyRp3dK90Ab03Vu4lAXO9Dwq7lwaF1qseKH02547t9YjsIqFQvFLvz/7fxd/RvEe3z3fxW8ixVzq57XPa9j6ql1q3728vJJdl7DVEHLzFVb9+VQPmbZ3uQq/9tzpyVKov7Sfbno4FN41pule2AfPSm77jTL2ttqHBehmFB9v16SKki5G98u+MX761+kMXeQVQjSd+VLpQ7Li3DpkLJgoo9krHpPV45DWy1JP7qFwTMLdd/9pmQTVQFUqd6o1Xb7CGOViXGuVOjbe+DpiXz6T3yO/UEBsF8wXS69C98LZQ9zl6DC7bUKher6al1HOxxo16cZ8qHVM7TqyPudWzepBuSxe52+NvPaUK0Y9Kx7oaMHufpXQ8LYjaht63Xn7v3wV7Q0SsS2Fov1+O1XvbUkNNJuvGkr7z6+l7NN+t969rApi9Rub2lb7PToVWHbs6hq3rR+cue+cfuVXfO1bVILeU6mC+6/Bka1yw1ueybpCrfrTfzIz9vPcZ9rvq9tIwvrzm1PF4eU2NC63fm6sFt8/9M0dvNPn7awLmGm8KpZaSV2H4O2feu3Of6nw9tJfSso56YU5uyhfOdu3x/56fz633z23AvDheMgxeYez2TsQeeHq0D91r/56tC52P1ri+lD86fjg3PNEjHt+2AXONkUvDY8N01WDbth2Vjqdvb/7YuWDHoKPS8jvWrI+N3WnTfrzt7PTox9/Z+mM+AgQ2AudCCGAECLw4gU56Co5OhZ6XrF1BQU/nrmW6sOGS25+74N57EF/v986YugV7r3RM6YKtC/RtaWzPvd6/rdteb+TCgnPDZnSy1InttnSiu/f7XjcbML9vzNsF1l6px1+t/EelIPnDD/6499DDcwHzqeE1cq7H6FEpED76+96Fw7Kctv/jNwutbtaTdO824oKr9UOetutTYFbv5r2yflDZiU15yD/tPeSvHuZPCezqwdftvXulxpLq9Lp0YV/DxrlSw0I9c9a9Pvs8u0tg2xDUsrYBc79ru+qtf+rCZVmPtmMd9hb+FQJuS0MkNJzBOjjq4qlxns+VHlrz6zcvKhxc9+Je/nxtwHBpwFxvvOWOi+W9Ov7Wm/qSelCv85+82YaOkfU+2ivrgHn5e2MJf+9zWOPve3fLdPxe9+S8YDGHL6mRrV7Y6x6253rXrhe2bWj9QeOPhSPb8tb4RaHMUvoeqkHzXOm43PF5XbodvDtD1uXWy++9CgkLO5fSEAd97qduN15eu/ed3VjM9crdlnvsX9cEMNtzgO4c6HO5ZF/Z3urePI17Xr3Zlhqya9Bel48e/znVaLW8tgbm7txaD+NQ48TR8yp23v7srwpy1r2OLzkHWhbaecT6HKRG7xoYnrNcEzDXKPYRqzev0WzdK/TUetVbu+P9UmoQ77vnucu1x/97fT732D/3AuaOeZ1z7333r+3r2V9D7PqOqVP70b3279n6ceuA+V7bvw2YF4/OsxrG7tzxtPOL9TVXjbuXPFi5u6E6/1vfrVCnk3Wv8Nbl0Y+/s/XHfAQIbAQEzKoEgccWKPjpgXCXXChfuiXd5tuD+ro96qgUPDXe37p33/LaeqJ0m/Jeb8WCpoLFvdJt+XvDQdTTqmEstqWL63q4Fk4clXrcFjhsSye/R+H0bMD87lhmwe+2nAtSe32B/F6vu/62d+F1KmDu5L/xFfcCs5a31wtuWecsMz0am7GLhKOHA3XRtw293hq/W4c6a5suvM89aKhw86iXVRdZ62EQdugf8ld7AfNTV7RebduhDJZlvj1+WPdC7vbOQqmjYS6261IjQGHQunRR0cXFtuwFzJcGmS1rO7Z86743HME2yGibqvP1zj1XOq/p2LUeWuOt8f9fuzPjtQHDpQFz77feL9pn+0xODS20Xr0abNq/1734TvUs2wbM14T+XRTW42h9PvjG+P8yXvE573N/77ui23GX0roVYF96V03Hx3p1L6XGsRrJtqXe4uve8Od6fS/zF5YUMK63v4vxQoF1ufXy631fPV+vR/vbufG31+vY8bZj+FLqzd938rbcY/+6NGDuvOCd1QrWY7mA+Og8YrstnaMUjK3vWih0rvfztvzu8Ys3V7/s4aENi3NpKfxd9+ZrXTvOXFqXz73PdgiuTx0z9H1ySWn4qfWQBpc2sFyy7OU11wTM21Ctxq6+Ky4pNUA1VMxSTjWoX7K8o9dce/y/1+dzj/1zL2Cu5//R8HZbw4ZFW98J0FAyPQBzW+65f8/WhVsGzPfc/r2AueC34+m5c6ftenad1znL0XXG1np7J+rb4wXbc7tHP/7O1h/zESCwERAwqxIEHl+gi/HGpatFeDvG6uzad5Hf7VJHt6e33FPjDddLcO/2+5a513OyoPBrjWnvdtfGWS082iv9rVCnYTJO3X5/jcNMwFzY2W3de6WLzj6fU6Vjbbcq7/X07US9MVbX5VTA3IXb+sR++74F60c9HesZtjc8yLKMltv67JW9gPmox3Mnp/VuPhordVl+gXW9NfdKPXYveWDTGfq7//neAXNBwjpMLQg7Mj3CqF50EbKUo57FewFz820D6qP36YJjHZi9Pf6/FzB3B8J62IZTQ2nsvVdh8rrhqWPS+nbtZZ5rA4ZLA+aOjeuHC3Z76noYhksq5TaM71jYhfBe2QbMBabroPHc+xUm12t9KW+MH54rYO622PUYjYXt1wwrcGnAXCNL9Wkpfee0HecurM/ZLH+/9fILzrv7ZCl939XAsB5z+dy6fpfxgm1v3I4N64C/Zdxj/7o0YK6H9bpn60yYuA3kql/bc4oa6bcNln1XbodVOWXcd1oh7vqusm7/3/aKPvc5Hf2984z1sxBqKLuk93rLe7SAuX2+fX8pNQJcWpcfNWC+1+dzj/1zGzA3znuNXKeGpFnX28Lk9R0yRwHzvfbv2X2u+W4ZMN9z+/cC5oZK297RtWdVY9Z6TPnuLuyOq0vL9tyujkzrYbFehuPvpdvqdQQInBEQMKsiBF4egS4Uu8WohxJ9qzE9R9h8Krj5tuM9jh6sUu+S7cOokjzqkdpDlrYPBlnk69VyrhdRgWXjfHULaL20Lm1V3/t0ZwLmTqT3eme0/KNxorfvvR2HdPl7J/Q9nG89VuapgLmhS7od9qj08JKjAKcTzU44j8r2RHv9ur2AeXshtLy+YVg+5MT7LH/aG9Nw+dve+12wyBf+knsGzHv1ZO/2/nMo26C/cGgdOC/zbwPmwsJ6gG7Hqz16v+reuofU2+P/24C58KZwraEVlnJtEFSj3Pquj4Yo2WsgulXA/BfH+72xWv964H76uQ9h8/fGNFzfZdJxovCp3sbbsg2YG8rmfVe8X8fT9cMIW/fnCpivWI0v9tLqQD1Z18eSox7MfSc2duu61NDQ6881dF2yjrdefj3e31qtyNF37Kl17c6jetOue/Nub12/1/51acC8/Q45Gtbj1HZvP5uOSzXmrj/37Xd4d/Gs7xC4pA70mhqL1sHLUePVpct7rtdtt+9F92B+ynZtw66ZRodL3v/a4/8lyzx6zaWfz732z23AfOlwCMv29QDRGleWchQw32v/fspnc8uA+Z7bvw2YOy9rXP4efnqudK6xHkf56Hv2aDk9t+ToL+4AACAASURBVKFzz6XUmLS+DnnVj7/nfP2dwGslIGB+rT5uG/sKCRTqFAB1YdUDfpoab+/afboLsebdG5e5ZRWU7A0LUa+aQpz1WG2dyBTkdJG7LY2nuL34X16z7bl1ycfUreZL2FwIfmnA1bJnAub3jfnqJbVX6tV4NOTE+vX1aFyfwK3/1jAF61uCTwXM9Ro+eqheyzwVMH/a+Pu6l8J2e64JmBsft2EY1kHgsrzq0944qdv3K1g/eoDIc49veUm9eo7X7AXM7a+X9t66Zh22J+3XBr7Le705fuj2xaX0uXbBsB2LcRswd3w4elDj3nZcEjBvh5MpUC0w666L5y7XBgyX9GDugWAdB9elbXrnypWvAbEHj657ATacTA1V27INmOtte+rhftv5HzFgbgz9jg01qK7L0YVv9TWH/Nel39XA2XAIDUExW269/G0QcW0jwbJd1bNud15KD0f7Oav/32v/uiRg3gvTOh/pQXzXlJazbqBt3u2Y5dtxzTt/6NkJ15ZPGjM0xNhSfuP4YX23wrXLe+rrC9LreNAdD+shzV7GgLle2O3f7ffr84qXOWC+9vO51/65DZgvebjfuq5eEjDfc/9+yn50q4D53tu/DZgv7ejR8xE6z1qfa2yvSZ7i27yv6vH3qS7mJ/BKClwbRr2SCDaKwCsiUAjTk48Lnet93MOb1g9dONrMThS7JXGvfML45U89+Ns27Kgn796QBvWE69a7o4CoQLphGY7GTD738dTbsduCC0+3Dzvbm3cmYN72qDm3Ttf+vfF110+PPxUwd8tdvbyOyr0C5r0g7drtPvX67S12z7nsWy7rngFzt5avH+LVba57DULntrfGom0jSWHFNijdBsynxobee89LAuaGtqjn6VKOelOf26ZL/n6LgHlv/yuEL4y/tmyHDzrqJbYNmLs1tf3n0vKiAua+n/Kq93QNpH1PdGt94fJ6qID1dpzqWVVP8YbUOCptZ88BqFGyoP7awPmWy284hHWP2h8y/l+v5mtLw9usvx+2jYr32r8uCZgbBqdgfSn10q938KXjxy/zdS3TsahGz6UUsq+f+bC9pb+A7Vdfizte3zi16wfO1TDXudYtS9vXEGPtK8t+0r8FY+0ve428jxowF2K17q33si3t/9WFHqS4Vx49YH7Oz+de++c2YG7c+YbFu7RcEjDfc/++dL33XnergPne278NmBtasOuyc6XhsbZ3LPXdfOr5N+eWuf37y3z8vXZbvZ7Aay8gYH7tqwCAV1igXpPdft7Yo/UMOSp/Z/yhi/m9XsDbE6T1MmqRbrzXpfTgpb2etdvX7a1H4yV2AnI0FvOlH1PhcWH5uydmmAmY6w3YheWtSj2g6gm1lJchYC40uzagucavcaTrhfaylXsGzNsHjxWaHT008ZRjjTzbsWq76N9+vtuA+drb+C8JmLd+144FeE19uUXAvL1dv9CsY/HMWMAFoTUWLuWHjR/2QrFtwHztMCn3DJhrTKvHZfW09bz2PPTcrbs/fiyz75x1b6yjOtGwKQ150NBBDe90SS/5Wy2/25irJ0tp+KCjB8OequPbB9E1JM36u+te+9clAXN1IPdblO3QVVnWO/S5y6W9BK9934ZEaxiyAveOKetxny9Z1iMFzAV41cHqdI1Hl+yb6218xID5Vp/PvfbPbcB89FyVo7p2ScB8z/37kn3i6DW3Cpjvvf3bgPntscHbIcj2DNonexjgUrrbbv1d9BTbZd6X7fj7HNtsGQReW4FrT+xfWygbTuAlFuhEoYvoN09sQ0+a74nze2X7tOjlNdsAsBbw9YOiltedWvb6/Wox7wSpXpmnAvFzH0U9wQoxehjeXpkJmC8ZJ/rcep36+8eNP64f8PQyBMzbk9KnbP/evD2Yav3wuude/q2Wd8+AeRsm/aaxUdshBS7dzsYr7VbJpezdqr4NmLfh1bn3uiRg7pb+hgdYSg1P9dy9RblFwPwRY0U/d7WyjYn7VSdX/nPGfB+1mrfj4y/ZWdY2YN5rHDi1CvcImPse+JQxrR/8c46lY3jB/CVjMK+XVUNlQxo1hMylYVbDMnQxvu5Re7R+z738vQaeHrh69B12yq07jrrzaCnboSDutX9dEjB/j7GS9bK7Rdk22nZ+c8lzAa5dl+duCO0Y3BAcDRNxyR1orW89DWuIW/dWfISAuR75P39Mb43p0uvN1rt6Xxi9lEcKmG/9+dxr/9wGzNc+rPKSgPme+/e1++369bcKmO+9/duA+dzdjovBNgjvzrX1cDtPsV3mfVmOv8+xrZZB4LUXuPQL/7WHAkDgJRco4Ohp6Y15u1d6aFwPj9sr2wvW9WuWIKPhLbrQ2paCi26FvKZ0u2e3nBaU9fC/bt+/tpwaXmEmYN6ejF+7Pude/6PHC7q1eSkvQ8Dc7a09Gf5WpfBv5rO/1fpcutx7BszbfbNbwtfjr166zu1z3cGwPifYjmHasu4RMG97Zd/yFvRbBMzfejith+qpB3OhxDXjxC+fW7071z3Stw1Ry+sePWAuJO8hUkcPpm2s6b4/Op70PVWjZgFl2/VLx9Tt20s514N5Xee7SK5xol6g9V7tOQGnSmH2TxpTQyJdUp5z+dsGnr4D1+OiX7I+vabGmUKqpWzHCL7X/vWiA+Z6zK4fKHqrgKO7PNYPyLz0c9p7XftHzx6ogfyoNNROz6Do3KohdOpB/YfGVA/nGrOW8qID5jobdAdGQ3vslY6HPQSsfb7t+ZNjqhdl/9YwVGPpUh4lYL7H53Ov/fNlD5i3+/fxHnP+Ly9jwLy3/bMB8/auq/bNOiZtn8FxXvL4FS/D8fcp22deAgRWAgJm1YHA4wl0cdE4bHulwOHTJ1d5+/Tz9WLqWdhtwnvljfHLvzCmveNFYVoX49023ENCtqWHC60vdq9d9YKvbhH/PmOqd85eD+mjZRYovLPzx5mAuZ53P+7gjZ57rLLe5mUImGus6GGPe6WHof2Uaz/sV+T19wyYt2MwdxfBGxOOe5/lVx/LKaRYl3sEzA2z03A7S+nYU2PGLcotAua9MZgbW/jdiQ0oPPqmq/mObmN+5IC5BsYuLntI3roUItfD6vPH1M9HF7NPCZjX79f4vPVgLbBvqiFmr8G19fg2Y/qDV35eT13+dgzmelO/feU69PLtd1X7UqH8Uu61f10SMH+7sVLrcZKvfWjoNTy/a7x4PVbytQ/CvOa9Zl/bA3wbBmddul29sbjrANCDktch8vp1X2XztxcZMB89V6O7khrn950xFYofjfHa+d4jBsz3+HzutX/eI2C+5/49u881360C5ntv/2zA3Hd0Dbvr0rVW5xXPVV6G4+9zbavlEHjtBQTMr30VAPCAAqduq+r23YYmmCnbh/+sl/Fdx38KoI/KdizQ5XXLQ762t3Ivf5+9zfdoPXqycb1bugA51zP6Z43XNITAtswEzN2uureslv3cJ2It82UImAv/Gzd0r1firxm/Xz8Iaaa+vqzz3DNg3l6IF47VW/banicfPuYp6FtKY9F+4JgKN9blHgFz+3fDYiylXqX1pqkn8HOXWwTM26CndX5zTNeONVtg2ZPd+xyW8pHjh/XwG8vvHzlg3o7NX92sHjWu/SVjHj9XwLytOw2dUeDYXQB9Puvyh8d/+q55Srl2+duhqN433nw9VMyl69JQCdWTpTREQd9fS7nX/nVJwLwNdnpg79FdVpdu/9HrPmv8YT3UTo3ihfGPUurpW2Na4exSCt97psRfvmAlHylgriFs22Gh/bh97ZIHhz1iwHyvz+de++c9AuZ77t8X7CKHL7lVwHzv7Z8NmPce2H3tg4LP+T/68ffc+vs7AQJXCAiYr8DyUgJ3EtielKzftgvyxrWrJ8i15ShYbTkfOqb1Qx62y+7W7L0nTBf+9OTvek5uHwrROHoFzLcqnQC1Tusefuv3qtfPD9l585mAuQv2Ltz3ypvjl9eGR+dMXoaAuW0olCyc3JZ3xi9u8UClc26P8Pd7Bsx7D1q8dvzdzOqdvw5bjvbdewTM9fYtaFmXLu57INulpUa4T169+N3x897Dbm4RMPe2f2lM61vDjx7Od2p7Oq7Vg3ldGjKmdd6WRw2YO8ds+It17+WfPv6/HiP43Gf6aeMFDSG0lGuGyDi37OXvfY/0HbeUGjO+/JhqQHuOcsnytw+SvXZ889azRr/ODQobl9JQU+veoPfavy4JmGsM+5tjWo/9flTHn/o5NCb3+1YLqcF9Xa+euvynzr8dGuGvjAV2HDvqsbx9v4ZrWR8bXmQP5u3dcp07fZcxXdKg1HbVEFAgtZRHGCLjXp/PvfbPewTM99y/n7L/3Spgvvf2zwbM2W2foXPtECQ1qHaXxfqZBw1x2PG98ujH36fUH/MSILAREDCrEgQeT6AeLF1cHD0Yqofi1Mvh0pP1trAgqvHu9h54VC/FLk4KAo5K69I6rXvXLK9tyI7tbZ397agH8fo96t3Wum1LY/JdcvG3d2vXsqzPHD/04JJtmQmYG7vzb4ypC/ht6aFbndidK92SfRS4/+rxt8bgXMrLEjA3FEZjlm7L3x2/+BpjOndxXPC0F/y1vHr0rW+fPuf7KH+/Z8DcNhcorY8VfR6fdCXG7x2vXz98rX36h+8s4x4Bc29bOLJ+wGNDgfzKK7bpfeO1XdAsZfugs+X3twqYG/d2/bDFxlX9blesfy9teKGfuZqnsUq/3sEyHjVg7mGt296Xb4zfdTF7afns8cL1w8v2AuaOz5+4WWBDM+2F8Xvv+5XGLzu+r8u60fXWy+99t424DT/UMDUdSy8tPRitu4rWpQbg7Wdwj/3rkoC59ex5CQ1JspQfNX5o6JRrSgHRx6xmqH5tl9HQY79z9ZoarDr3qJH8mtLxvUb+pfTd/RzPIuj4tj7mvj3+f/TduLe+2wapFxkwd/62Pla9Nf7/a69A3jZ4PkLAfM/P5x775z0C5j7ye+3fV1SvL/bSWwXM997+pwTM7WM9GHUpPcSxuycuLW+OF66fGdDdV+tnHjz68ffS7fQ6AgQuEBAwX4DkJQRegMCp4SxanXrnNcbtJRdHXeQ3xEU9I/bKUfiyfe1vG7+oF8q21NtrL3gt/O1BNKdKwctH77ygi+oujM8FBF3EdDGzVwrZ9sLPo4A5y6OHULX8Tp7e3HmjQtTWo9t7j0q3oGXRv9vyR8Yvvvnmly9LwFxo/s7BRheO/bwTJv3pfWNaB4Hrl3/38Z/q5stW7h0wby8U2x9qyLjk2JBt49I2Ru66fL/xn3UPsuVv9wqY3x5v2BipS2m8zm85pksb1Ro/91us5q8ersPa5U+3Cpi3Tj00pweh9kCwS0oNL32ONdIs5VSPy0cNmPsM1mMZVyfr1XXp59j3Sr3B1w57AXMNpzWUrs9pzw37tP4c9oYjaez/ZdiYWy+/denOgz7z9Ta8Nf5/TTD3m8fr6zW2lB4CV3iyLW+PX9x6/7o0YG4YkPVzGtpHGsf8mmF++p7oQcVL6XkQP3Gz0fVIL7j7cqvfX+vbsClt11I696lxfts4sUN+9lf1/qvOLuWSBvr1Qrfj8b/IgPnvjBVb37XQWLSdg15aCrfWnQMeIWC+5+fz9tj+W++f9wqY77V/X1q39l53y4D5ntv/lIC5Y2F3fS6loWxqgPurF8L+wvG6n7x67fZB649+/L1wM72MAIFLBATMlyh5DYH7C3zYeMsuZPaC22VtChV+8ZjqLddJwPqivX27gLeL8XrBfMUTm7C9hfbopfUQ6qT0krIXmu7N163Sjcu3VxprtLH4agnfK13AdCFSELlXjrbr1FAhp27PrRddven2yueNX9arfLkdbP2ablfu4qQnNe+VvdvnX5aAue3ZPohs2cZCtXpEHJm9Nf7WQ3P2esX3sKfqb8t42cq9A+Z6HtcDeV3ap7Y9OvccC9Ya3qUwbSkFE40rvhdQ3ytg3j7VvHVruJv1BdBRvdgbw76gs5B6W24VMBdidcfHugdPt43XmHZJuPoLxuu6JXtd6qFYr/698qgB894DDz9obMB2CJSjz7JGgXpyr8vREBnbW3yPeuHvvdc2CO8127H1b7383rPex/VCXkoP/vvGYzrVeLm89juPH7bPUfix43cNMbIt99i/Lg2Ya0jueL/+HrhmGJUe2LgdoqqGz727X375+P36gYc1XtRwdUmI0rlYQz18xAqzRuf1gwP36talv9uGqp1rHT3sebvMerrXkJPlUl5kwLx9YGXfGz2X4ZLy5nhR+0HfTUt5hID5np/PPfbPewXM99y/L6lfe6+5ZcB8z+1/SsDcsx6602U9Bn7DNq0bOo58Gw7sT45pfb5T2Nz16bo88vF3tu6YjwCBHQEBs2pB4HEFtl/Gp9a03ltdLP21MRWSdlJzqjfusqx3xg+XjpVbC3QXYuueKUfrVO/qhk84Vxpj8NTYz39r/L2xKAtWumgpnGn7un258Hl9G/36vQql+9veGJq/Zfz+KJR+e/yt0LvW+y54310ttAue1uMbHWxUt9x2Md/FboFAn0G29Sxaj4m5nr0gvnBj+wCzlylg7ja6zzgw6deN/9lnWA/uTmILnQoL10HKdvZ64fU5PXdpGJL1hWvLb+zwa3rLnVunewfMrU/h6boXfNtTuL9+WN52vXMohHtr84dTPc/vFTC3Sttba9uXC9FODZvSresdT9bHhaNenL3HNmA+N/7rdkiYUw8K6+JqO3ROPSv73amQOeMaXtbnZwUu3WJ6VB41YG7faqiH9YMKO74WIJ4qhXn1at27u6Ghk+rNvS3V5eyW0j7w5pjO9Zzse60n3Bc0LuXPjB8Kdtfl1svvveqFu71ro/Wv3tcr9KjUIF0j57ohOffuWuo7dK/cev+6NGBu3fp++IGrlWz/6Huz/eBU6e6LQt6GOFlK77v+LNfz993dg5LXYXav73ygQPaotC82FMZ2yIpresmf2ZQvNkZpd0Z1flRD1anSNtVzve/VdenY1vHsOct2HNUawn7azhts7/aqAbRzoXPfszUOFnx+hc0yu5umu2qeu1xz/N9u+60/n1vvn/cKmPvM7rV/z9aPbcB86pxheY9HPL49JWBuu7rTa/vdfO5arrt7urt13fDWHajdkdN5ybo88vF3tu6YjwCBHQEBs2pB4HEFumjqFt2jQPOpa94Jcj0Xj4aY2Fv+thfH0Tq8Mf5w6Tib29tbn7pdzV/vv27Z2ivbW7mO3q+xrtdjf/a6Qrx6QO+NZX3tetdboIuuvWFEXqaAue0uyKwH93OUPp9t783nWG7LqFfutsd0/z934XvN+7+IgLkgoguedb0spKmRqlD03dUG9L1fr+d+v+1VX/hSg8fRuK/3DJg77tUAs354aJ9fPbNryFn3OiyQLKBqPPRuWV+XU0OtbAOGeiOuxxHcfu7XBMz1Yq5Bajs0UUFgocx66Ijepx77P2NM216LBYt9vqeG13jUgLnt2jboVS8buqgH39Uwui1vjl/83DGte9WvX3PUM7nb8LeNDy2/+lI4WAPsurSv1EjZZ7F9UOxez99bL39Zt54dsA3T6vGdWUNKre8sqK4XxNYotH3IbkMMtKyjcuv965oApiC0fWUbiPYd3Of3+zcbUS+7nzCmjrXrXnM9R6LP6VSj9TYobNE1zNeg0fnNdriLjgnve2+569WYeQjjiY/jCxvNO96tS4Ydc/fuXMigsYrrKbge9mOZv/2sXvjXPBz11Pr1t0sD5j6bjsXrUm/I1nXvAdUNMVYdPuqx/QXjbzX4bBviz63vub9fc/y/9+dz6/3zngHzPffvc5/53t+3AXPHkYYWPLp7smU84vHtqQFzDcENl7ZtrKqTSPt+wfu69IDvnqPzwZvfn3ouzaMef2fqjXkIEDgQEDCrGgQeW6CeeAUezx0yFy534XR0y/WRSg+q+q1nyApg1w/tOSfciVw9MLvN8zlKPb7atqMxaLvw7kLyXNkLmJunW2zrQfeU42eBUOHyUWj0sgXMjS1duNOJ+lPKJb0bn7L8VzVgzmQv2O73BQ094LM6VwjVxfzevlaPx24t71bHo3LPgLl16KFXXcBs97UaBP7UmLqroWNk4xtvA7bmb0iNessflS6Y1sfWwo+GHOoOhBr4Gsf93dXM1wTMzdZxsAeL7QVAfR7t/9XJ9vcC5m0pUHlrTIUBp8ojB8w1WPyBMW0/w3pm9l2x+PY90HAH6/GWu+Pgc8a0Hlc4k3q29p1RSF+jyFK2YxCvzQoLakwtTOzuksz3Ppe+P9oP9sKsWy+/9S00rmF574GOhZ/tyw3FVG/9xlrfG2Ionx+x3viDn2+5f10TwLR6HzWmGiM+YGddqysNo9FdDDXY9Nltt7tjQj1gG47qVGm+6tTeHTQ1SFRHer++0+qFt75lfFlu33XdUXDNAxgv+Di+sPf6eizp5un4XQ/gGqI7PrU+fc9+6zGtGxTfGf/vOLh+AGHBbA87rN5cM5b30bpeGjB3V0Dru16XllljWXd49VkW4vVw2o4P24Cq87MfsFmJhidp/3t3TP38HOXa4/+9P59b7p/3DJj7rO61f8/Ui+rhtuGja6PqWceEetR3frUuj3h8e2rA3PbViaZG8HXD3bLdPcy0jkMF0TX4rO8cWV7T93EdGI7C+Uc+/s7UHfMQILAj8JSABCgBAvcR6LbXenV2svkc+2wXcfXQ2t6+dMnWdEHTraR7D6tb5j912/jRe7wx/tCF4fa25EvWaf2ad8Z/6rF4qudBvR0LEWp9P1WOAubmqbdk4dVMT+YuvDrZ7iLrqLxsAXPb0YVvF2BHPQ9PWRfm/Owxffy1H/iVr3+VA+YoPm5M9e699jhRUPuRYzoVLrf8ewfMvWdjlBcynxqPfq8a1Ku+kOKooal56lX3sSfqULfgN8b4Uq4NmJuv40y3kO5djJ2qvq13Pfoaf/RceeSAuXWv5+LRHSVH21aP4+4gabiihqzYKz9m/LKe0EvporiL4/VwMefs1n8vyCvgOxpW4tbLX9alRpMaJrY9qy/Zlk8dL+o7+NQwLOvl3Gr/ujaAaZ1qGO57dztEwrntLlzunKY7Ni4phdi/fkwzd910jKwBoga55y597jUu9HCta0qfeeFSvaobGmlb6lBQiPbUcmnA3PvUk7w6vNfwd7QeNSjVMFL4uR3HeZmn3pQ9X+M5yrXH/xfx+dxq/7x3wNznda/9e6Zu9H3TWMJ7pQbnbUPTIx7fniNgbvt7mGmNcEdD+x351ujfg+DPXVs+6vF3pt6YhwCBHYFrL0IhEiDw4gTqsdItkV10X3PS3hr3sLRCji7AOnF4SinsOeodVVDYSdq5cQP33r8xoxtfszDi2t7MXYx0W3W9dS65jbJbwAqgTvW4PRUwt/71Fur23aPxnLfb2O2YnzymekXu3Rq+fv3LGDC3/vVOqG50IXrJ+I+FII1/2jhv3Zp36/KqB8z5Nf5o9excA0qvLZhpf+7zKoQ4V15EwNw6FRj+0gu3qe34OWMqdDlXOlZV744upJ4jYG4d6pVb7/welHrJeVfBzE8cUxdsl5RHD5jbhsar7dbZ7e232+0rZKr+5rUEvT0odDtcUfNtA+Z+V++qxpLsu+SS5xA0T3Wmeapj54bLufXyF4/ep+/Chgu65LkH9bwtYGgYjWvLLfavmQCm9e67r8++O40u2Vfqyd5nvR1y5pxBy65xqc99/XC8o/nqcVv9qBH03Pf3ufc+9feORb1Hd0p9qTML+sPvfeb1Cq4cDeH1IgLm1qcOA407X+PludI4yz2cdnkAaOe6n7Iz03MGzNce/1udF/H53GL/fBEBc3732r/P1bft3z96/KIOEnvHnOcKmG+9/c8VMLeeXYN1fXPJOUt3cjR0Vefxp54VsDZ/1OPvtfXG6wkQ2BG45OQNHAECjyVQS3q3Z9bK3IlnJ8n1cm5qn+5kqFtoC3kb068LsEK8S56UfsmW1nvnnYMXdqHz5iULOfGaeip2a3kXJd0C3C3s9ZjutstavrvQa/u6vbzta9sKZM4FA9u37OKt2+cLiOsx9MaYFr9uQ6430CVPPm/9Gjqk9W3Mw3q5LA+3apzlbg9vHbvwf+5barfb9Cj/LxDJo96A1dFMuvW73oj1/Or2w26P76Ly1Niyj7I9L+N6tA9Vt98cUxcLBf7Vv+5A6CK+BqfuZtiOTfuo29q+2TbV67ChFNqmeuVVp2q86TjXw/Dq8dvvLi0NzVCIV++qN8ZUKNkt5Rn1AMt3L13QBa9r2INu46+XTxfafSY1ynRs7ljR+jcE0XYs1gsW/VK8pDs+aqTouNC40oU1NQhWJ7uzowv8wuTtA9fq0drDh77PmPq+K+CrV3PjtnZs3Ssdb+rB3h0VvVd1pR7INbb2HdnYtMt3Y+977bH51stftqn36fulqUbNjqV9FxYa5tRxtDpTz+22bbbcav+aXZ+GjGlf6W6f9pW2u32z74/u/ukhaNWV7fjM175f39V9V+XbcA29T/WyoKTjSvtiw5R1XOm8416lxu/uSOl28wLw6m7Du9SY3hAdHbtbr21P9c7PGk+6YSc6J+z174zprXut+M771LjUsbROEh3zOo/rmFfDWMMQ1Ni/bUyrPnasqPHgg8bU+dq7Y2r4lxqgnqvMHv/v/fk82v75VP977d/XrGfnqjVy1LDc90zXFB1nu4vp0o4kl77fI27/3rq3np1z9aDZrm8a9iaXjkUNMdP3TneDzHQq6v0e9fh76efodQQI7AgImFULAgSuFSgAbizEvd5hP2r8vl7ECgECBAgQIECAAAECBAgQIECAwGsgIGB+DT5km0jgmQXqObjXWl2rdn+r149CgAABAgQIECBAgAABAgQIECDwGggImF+DD9kmEnhmgR48tfdE8oap6HZWhQABAgQIECBAgAABAgQIECBA4DUREDC/Jh+0zSTwTAKNTdh4ZD20alsaL++SMYufaVUshgABAgQIECBAgAABAgQIECBAQMImlAAAIABJREFU4EULCJhf9Cfg/Qk8tkAPsHt3TD1oqIfN/KIx7YXLjcncw1J6eJJCgAABAgQIECBAgAABAgQIECDwmggImF+TD9pmEpgU6Gntb1wwb0NmvHXB67yEAAECBAgQIECAAAECBAgQIEDgFRIQML9CH6ZNIXADgUsC5h7u96Fj+hM3eH+LJECAAAECBAgQIECAAAECBAgQeGABAfMDfzhWjcADCFwSMH/iWM+f+gDrahUIECBAgAABAgQIECBAgAABAgTuLCBgvjO4tyPwkgmcCpgbd7lw+eNfsm2yugQIECBAgAABAgQIECBAgAABAs8kIGB+JkiLIfCKCvz2sV0fNqYPGNNfH9NfHdMXjOnzxvQ73vv/K7rpNosAAQIECBAgQIAAAQIECBAgQOCcgID5nJC/EyBAgAABAgQIECBAgAABAgQIECBAgMCugIBZxSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIEBoUmGZAAAapklEQVSAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAAAECBAgQIECAAAECBAgQIECAgIBZHSBAgAABAgQIECBAgAABAgQIECBAgACBKQEB8xSbmQgQIECAAAECBAgQIECAAAECBAgQIEBAwKwOECBAgAABAgQIECBAgAABAgQIECBAgMCUgIB5is1MBAgQIECAAAECBAgQIECAAAECBAgQICBgVgcIECBAgAABAgQIECBAgAABAgQIECBAYEpAwDzFZiYCBAgQIECAAAECBAgQIECAAAECBAgQEDCrAwQIECBAgAABAgQIECBAgAABAgQIECAwJSBgnmIzEwECBAgQIECAAAECBAgQIECAAAECBAgImNUBAgQIECBAgAABAgQIECBAgAABAgQIEJgSEDBPsZmJAAECBAgQIECAAAECBAgQIECAAAECBATM6gABAgQIECBAgAABAgQIECBAgAABAgQITAkImKfYzESAAAECBAgQIECAAAECBAgQIECAAAECAmZ1gAABAgQIECBAgAABAgQIECBAgAABAgSmBATMU2xmIkCAAAECBAgQIECAAAECBAgQIECAAAEBszpAgAABAgQIECBAgAABAgQIECBAgAABAlMCAuYpNjMRIECAwP/fjh3SAAAAMAzz73oehivgpDkaAQIECBAgQIAAAQIECBAgQICAwOwDBAgQIECAAAECBAgQIECAAAECBAgQILAEBObFZkSAAAECBAgQIECAAAECBAgQIECAAAECArMPECBAgAABAgQIECBAgAABAgQIECBAgMASEJgXmxEBAgQIECBAgAABAgQIECBAgAABAgQICMw+QIAAAQIECBAgQIAAAQIECBAgQIAAAQJLQGBebEYECBAgQIAAAQIECBAgQIAAAQIECBAgIDD7AAECBAgQIECAAAECBAgQIECAAAECBAgsAYF5sRkRIECAAAECBAgQIECAAAECBAgQIECAgMDsAwQIECBAgAABAgQIECBAgAABAgQIECCwBATmxWZEgAABAgQIECBAgAABAgQIECBAgAABAgKzDxAgQIAAAQIECBAgQIAAAQIECBAgQIDAEhCYF5sRAQIECBAgQIAAAQIECBAgQIAAAQIECAjMPkCAAAECBAgQIECAAAECBAgQIECAAAECS0BgXmxGBAgQIECAAAECBAgQIECAAAECBAgQICAw+wABAgQIECBAgAABAgQIECBAgAABAgQILAGBebEZESBAgAABAgQIECBAgAABAgQIECBAgIDA7AMECBAgQIAAAQIECBAgQIAAAQIECBAgsAQE5sVmRIAAAQIECBAgQIAAAQIECBAgQIAAAQICsw8QIECAAAECBAgQIECAAAECBAgQIECAwBIQmBebEQECBAgQIECAAAECBAgQIECAAAECBAgIzD5AgAABAgQIECBAgAABAgQIECBAgAABAktAYF5sRgQIECBAgAABAgQIECBAgAABAgQIECAgMPsAAQIECBAgQIAAAQIECBAgQIAAAQIECCwBgXmxGREgQIAAAQIECBAgQIAAAQIECBAgQICAwOwDBAgQIECAAAECBAgQIECAAAECBAgQILAEBObFZkSAAAECBAgQIECAAAECBAgQIECAAAECArMPECBAgAABAgQIECBAgAABAgQIECBAgMASEJgXmxEBAgQIECBAgAABAgQIECBAgAABAgQICMw+QIAAAQIECBAgQIAAAQIECBAgQIAAAQJLQGBebEYECBAgQIAAAQIECBAgQIAAAQIECBAgIDD7AAECBAgQIECAAAECBAgQIECAAAECBAgsAYF5sRkRIECAAAECBAgQIECAAAECBAgQIECAgMDsAwQIECBAgAABAgQIECBAgAABAgQIECCwBATmxWZEgAABAgQIECBAgAABAgQIECBAgAABAgKzDxAgQIAAAQIECBAgQIAAAQIECBAgQIDAEhCYF5sRAQIECBAgQIAAAQIECBAgQIAAAQIECAjMPkCAAAECBAgQIECAAAECBAgQIECAAAECS0BgXmxGBAgQIECAAAECBAgQIECAAAECBAgQICAw+wABAgQIECBAgAABAgQIECBAgAABAgQILAGBebEZESBAgAABAgQIECBAgAABAgQIECBAgIDA7AMECBAgQIAAAQIECBAgQIAAAQIECBAgsAQE5sVmRIAAAQIECBAgQIAAAQIECBAgQIAAAQICsw8QIECAAAECBAgQIECAAAECBAgQIECAwBIQmBebEQECBAgQIECAAAECBAgQIECAAAECBAgIzD5AgAABAgQIECBAgAABAgQIECBAgAABAktAYF5sRgQIECBAgAABAgQIECBAgAABAgQIECAgMPsAAQIECBAgQIAAAQIECBAgQIAAAQIECCwBgXmxGREgQIAAAQIECBAgQIAAAQIECBAgQICAwOwDBAgQIECAAAECBAgQIECAAAECBAgQILAEBObFZkSAAAECBAgQIECAAAECBAgQIECAAAECArMPECBAgAABAgQIECBAgAABAgQIECBAgMASEJgXmxEBAgQIECBAgAABAgQIECBAgAABAgQICMw+QIAAAQIECBAgQIAAAQIECBAgQIAAAQJLQGBebEYECBAgQIAAAQIECBAgQIAAAQIECBAgIDD7AAECBAgQIECAAAECBAgQIECAAAECBAgsAYF5sRkRIECAAAECBAgQIECAAAECBAgQIECAgMDsAwQIECBAgAABAgQIECBAgAABAgQIECCwBATmxWZEgAABAgQIECBAgAABAgQIECBAgAABAgKzDxAgQIAAAQIECBAgQIAAAQIECBAgQIDAEhCYF5sRAQIECBAgQIAAAQIECBAgQIAAAQIECAjMPkCAAAECBAgQIECAAAECBAgQIECAAAECS0BgXmxGBAgQIECAAAECBAgQIECAAAECBAgQICAw+wABAgQIECBAgAABAgQIECBAgAABAgQILAGBebEZESBAgAABAgQIECBAgAABAgQIECBAgIDA7AMECBAgQIAAAQIECBAgQIAAAQIECBAgsAQE5sVmRIAAAQIECBAgQIAAAQIECBAgQIAAAQICsw8QIECAAAECBAgQIECAAAECBAgQIECAwBIQmBebEQECBAgQIECAAAECBAgQIECAAAECBAgIzD5AgAABAgQIECBAgAABAgQIECBAgAABAktAYF5sRgQIECBAgAABAgQIECBAgAABAgQIECAgMPsAAQIECBAgQIAAAQIECBAgQIAAAQIECCwBgXmxGREgQIAAAQIECBAgQIAAAQIECBAgQICAwOwDBAgQIECAAAECBAgQIECAAAECBAgQILAEBObFZkSAAAECBAgQIECAAAECBAgQIECAAAECArMPECBAgAABAgQIECBAgAABAgQIECBAgMASEJgXmxEBAgQIECBAgAABAgQIECBAgAABAgQICMw+QIAAAQIECBAgQIAAAQIECBAgQIAAAQJLQGBebEYECBAgQIAAAQIECBAgQIAAAQIECBAgIDD7AAECBAgQIECAAAECBAgQIECAAAECBAgsAYF5sRkRIECAAAECBAgQIECAAAECBAgQIECAgMDsAwQIECBAgAABAgQIECBAgAABAgQIECCwBATmxWZEgAABAgQIECBAgAABAgQIECBAgAABAgKzDxAgQIAAAQIECBAgQIAAAQIECBAgQIDAEhCYF5sRAQIECBAgQIAAAQIECBAgQIAAAQIECAjMPkCAAAECBAgQIECAAAECBAgQIECAAAECS0BgXmxGBAgQIECAAAECBAgQIECAAAECBAgQICAw+wABAgQIECBAgAABAgQIECBAgAABAgQILAGBebEZESBAgAABAgQIECBAgAABAgQIECBAgIDA7AMECBAgQIAAAQIECBAgQIAAAQIECBAgsAQE5sVmRIAAAQIECBAgQIAAAQIECBAgQIAAAQICsw8QIECAAAECBAgQIECAAAECBAgQIECAwBIQmBebEQECBAgQIECAAAECBAgQIECAAAECBAgIzD5AgAABAgQIECBAgAABAgQIECBAgAABAktAYF5sRgQIECBAgAABAgQIECBAgAABAgQIECAgMPsAAQIECBAgQIAAAQIECBAgQIAAAQIECCwBgXmxGREgQIAAAQIECBAgQIAAAQIECBAgQICAwOwDBAgQIECAAAECBAgQIECAAAECBAgQILAEBObFZkSAAAECBAgQIECAAAECBAgQIECAAAECArMPECBAgAABAgQIECBAgAABAgQIECBAgMASEJgXmxEBAgQIECBAgAABAgQIECBAgAABAgQICMw+QIAAAQIECBAgQIAAAQIECBAgQIAAAQJLQGBebEYECBAgQIAAAQIECBAgQIAAAQIECBAgIDD7AAECBAgQIECAAAECBAgQIECAAAECBAgsAYF5sRkRIECAAAECBAgQIECAAAECBAgQIECAgMDsAwQIECBAgAABAgQIECBAgAABAgQIECCwBATmxWZEgAABAgQIECBAgAABAgQIECBAgAABAgKzDxAgQIAAAQIECBAgQIAAAQIECBAgQIDAEhCYF5sRAQIECBAgQIAAAQIECBAgQIAAAQIECAjMPkCAAAECBAgQIECAAAECBAgQIECAAAECS0BgXmxGBAgQIECAAAECBAgQIECAAAECBAgQICAw+wABAgQIECBAgAABAgQIECBAgAABAgQILAGBebEZESBAgAABAgQIECBAgAABAgQIECBAgIDA7AMECBAgQIAAAQIECBAgQIAAAQIECBAgsAQE5sVmRIAAAQIECBAgQIAAAQIECBAgQIAAAQICsw8QIECAAAECBAgQIECAAAECBAgQIECAwBIQmBebEQECBAgQIECAAAECBAgQIECAAAECBAgIzD5AgAABAgQIECBAgAABAgQIECBAgAABAktAYF5sRgQIECBAgAABAgQIECBAgAABAgQIECAgMPsAAQIECBAgQIAAAQIECBAgQIAAAQIECCwBgXmxGREgQIAAAQIECBAgQIAAAQIECBAgQICAwOwDBAgQIECAAAECBAgQIECAAAECBAgQILAEBObFZkSAAAECBAgQIECAAAECBAgQIECAAAECArMPECBAgAABAgQIECBAgAABAgQIECBAgMASEJgXmxEBAgQIECBAgAABAgQIECBAgAABAgQICMw+QIAAAQIECBAgQIAAAQIECBAgQIAAAQJLQGBebEYECBAgQIAAAQIECBAgQIAAAQIECBAgIDD7AAECBAgQIECAAAECBAgQIECAAAECBAgsAYF5sRkRIECAAAECBAgQIECAAAECBAgQIECAgMDsAwQIECBAgAABAgQIECBAgAABAgQIECCwBATmxWZEgAABAgQIECBAgAABAgQIECBAgAABAgKzDxAgQIAAAQIECBAgQIAAAQIECBAgQIDAEhCYF5sRAQIECBAgQIAAAQIECBAgQIAAAQIECAjMPkCAAAECBAgQIECAAAECBAgQIECAAAECS0BgXmxGBAgQIECAAAECBAgQIECAAAECBAgQICAw+wABAgQIECBAgAABAgQIECBAgAABAgQILAGBebEZESBAgAABAgQIECBAgAABAgQIECBAgIDA7AMECBAgQIAAAQIECBAgQIAAAQIECBAgsAQE5sVmRIAAAQIECBAgQIAAAQIECBAgQIAAAQICsw8QIECAAAECBAgQIECAAAECBAgQIECAwBIQmBebEQECBAgQIECAAAECBAgQIECAAAECBAgIzD5AgAABAgQIECBAgAABAgQIECBAgAABAktAYF5sRgQIECBAgAABAgQIECBAgAABAgQIECAgMPsAAQIECBAgQIAAAQIECBAgQIAAAQIECCwBgXmxGREgQIAAAQIECBAgQIAAAQIECBAgQICAwOwDBAgQIECAAAECBAgQIECAAAECBAgQILAEAhlJzy6JMeurAAAAAElFTkSuQmCC"/></switch></g></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-2"><g transform="translate(0.5,0.5)"><path d="M 5.5 90 L 5.5 80 L 35 80 L 55.5 80 L 55.5 69.5 L 74.5 85 L 55.5 100.5 L 55.5 90 L 35 90 Z" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="1.42" pointer-events="all"/><path d="M 55.5 80 L 55.5 69.5 L 74.5 85 L 55.5 100.5 L 55.5 90" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="4" pointer-events="all"/></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-3"><g transform="translate(0.5,0.5)"><path d="M 5.5 40 L 5.5 30 L 35 30 L 55.5 30 L 55.5 19.5 L 74.5 35 L 55.5 50.5 L 55.5 40 L 35 40 Z" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="1.42" pointer-events="all"/><path d="M 55.5 30 L 55.5 19.5 L 74.5 35 L 55.5 50.5 L 55.5 40" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="4" pointer-events="all"/></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-4"><g transform="translate(0.5,0.5)"><path d="M 5.5 90 L 5.5 80 L 35 80 L 55.5 80 L 55.5 69.5 L 74.5 85 L 55.5 100.5 L 55.5 90 L 35 90 Z" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="1.42" pointer-events="all"/><path d="M 55.5 80 L 55.5 69.5 L 74.5 85 L 55.5 100.5 L 55.5 90" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="4" pointer-events="all"/></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-5"><g transform="translate(0.5,0.5)"><path d="M 5.5 140 L 5.5 130 L 35 130 L 55.5 130 L 55.5 119.5 L 74.5 135 L 55.5 150.5 L 55.5 140 L 35 140 Z" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="1.42" pointer-events="all"/><path d="M 55.5 130 L 55.5 119.5 L 74.5 135 L 55.5 150.5 L 55.5 140" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="4" pointer-events="all"/></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-6"><g transform="translate(0.5,0.5)"><rect x="105" y="35" width="190" height="120" rx="18" ry="18" fill="#ffffff" style="fill: light-dark(#ffffff, var(--ge-dark-color, #121212)); stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke="#000000" pointer-events="all"/></g><g><g><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 188px; height: 1px; padding-top: 95px; margin-left: 106px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; color: #000000; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: light-dark(#000000, #ffffff); line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; "><div><b>Modèle </b>:</div><div>Représentation de l’environnement</div></div></div></div></foreignObject><image x="106" y="81" width="188" height="32" xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAvAAAACACAYAAABtG12SAAAgAElEQVR4Xu2dCdx3zVjH35KQZCuSt3rqRVmya5HyyJI9UirrIxG9VEKicPNSpEKWstWDqLyFlFDoCYVCylK2PCUpWyJbtq6v/lPXe70z58yc/znzX+7f9fnM577/58yZ5Tdz5vzmmmuu+YJTJEJACAgBISAEhIAQEAJCQAjsDAJfsDMlVUGFgBAQAkJACAgBISAEhIAQOEUEXp1ACAgBISAEhIAQEAJCQAjsEAIi8DvUWCqqEBACQkAICAEhIASEgBAQgVcfEAJCQAgIASEgBISAEBACO4SACPwONZaKKgSEgBAQAkJACAgBISAERODVB4SAEBACQkAICAEhIASEwA4hIAK/Q42logoBISAEhIAQEAJCQAgIARF49QEhIASEgBAQAkJACAgBIbBDCIjA71BjqahCQAgIASEgBISAEBACQkAEXn1ACAgBIXC4EbinVf9hGQjuZdcefbihUe2FgBAQAtuJgAj8draLSiUEhIAQ6InAb1hmdwgZ/rP9Ps3Cp3sWRHkJASEgBITAOAIi8OMYKYYQEALzIPD3lsw3ViT1KYtzfgsfr4iborzQ/vnuyvhHLd6fV8bd9mjfYwV8bqGQd7HrT6iswDkt3p9auGaIfxv7/YzKNHLRHmEX0eTn5HJ28U1rpK1HhYAQEAKHFgER+EPb9Kq4EOiOQC2Bp2AQyZdVlpBx7IMWLlAZ/6jFE4E/O1gXskuvtnAJd+vv7P8rVOIqAr8GUHpUCAgBIdCCgAh8C1qKKwSEwDoItBD4+1pGObvsXP6XtotvbiiYCHwZrEvZrVdZuKCLcn37/0UN+Pqo0sBPBE6PCQEhIASGEBCBV/8QAkKgFwItBP6PrFA3qSwYttvYcNeKCPwwUt9ltzFJwqwGeYmF69SCG+KJwE8ETo8JASEgBETg1QeEgBDYBgRaCDwmMV9u4XMVBX+ixblTRbwURQR+HKw7WpQnu2hXtf9fO/7Y2WKIwE8ATY8IASEgBMYQkAZ+DCHdFwJCYC4EWgg8ebLh9S0VmWOn/U0V8UTgG0CyqFe0cK7VI++yv//a9vjnY4vATwBNjwgBISAExhAQgR9DSPeFgBCYC4FWAo8WeMw05nwW50MWvrChkEctrjaxNgC2RlQR+DXA06NCQAgIgRICIvDqG0JACPRCYIjAYyoTx6On2LUfGSncte3+izNxcumlaCLwvVpcGvh+SCsnISAEDhUCIvCHqrlVWSGwUQSGCPwbrWT4BfdC/MuMlPhn7f5DQpx32u+vsPClhWdbCfw3WDp4Yrmuha+1cFELuKzETv+9Fk5awIf6n1ioMfkZawTcOd7Kws0tfL2Fi1n4gIV3W2Bz6dMsvH2VyFx+4FnB+DYLpHeNVZ5faX/xyU/ef2uBVQtWRP5zrALu/lwa+KXK11AVRRUCQkAIbA8CIvDb0xYqiRDYdwSGCDwHDv1oAAAtOhtZIcolwVvNjcJNDh6CiK5L4K9safy8hdoDoigGBJtJxesmNCYk9SctPNTCuQee/4zdO1iVDU896x7khNeZX7Vw2YoyQ94fZ+HRFpi8jMkcBH7J8o2VX/eFgBAQAluJgAj8VjaLCiUE9hKBIQKPqQzEMG2aTADc2P55/gAa77d7Fw73T7ffD7cwlcAzLp5h4X4WpoyRTDwg/ve3UONFh+J/iYU/tABZrZWnWkTI+3MKD4ydxErdHmXhx2szdPHYd3ADC/iMH5J1CHyP8k2o+lkeYYXii0Ii/GaSJRECQkAILIbAlI/TYoVRwkJACOw1AkME/jZW8x+zcPWAAEQYjXZOLmkX35q5gfeUV1iYQuDRgmOicusZWuKZlsZtLXx2JC0mLawkTPG1/lJ7rkT6hwg8Yz97DPChP1X+yx5k9WPoxNypBL5X+abWPT0nAr8ugnpeCAiBSQiIwE+CTQ8JASEwAYExAn8FS/PeId0T9vtahbwgx5BtL5BK7NPREE8h8A+w5x40oW6lR9DCRxv9GPdX7MI9ZswzJTVE4O9mkR4zQ54fszS+w0LJZGgqge9VvnUhEIFfF0E9LwSEwCQEROAnwaaHhIAQmIDAGIGHfEd7bgji+S18OpMfJjdo7b3gkYbNph+x0Erg2cT5cgvnGKjb2+zeH1t4jwU2l2Lic9pAfMr97Rb+qhCHA5IwQxnKk7z+2gLmOFexcGol9iUC/3X2/BssnLeQzovs+u9awL/+V1n4Vgs/MRD/z+xeaRVgCoHvWb5KKIvRRODXRVDPCwEhMAkBEfhJsOkhISAEJiAwRuAhju/LpHs1u/aazHW0vlcK1x9svx9oYQqBx4sM5D8nn1yRWDbbemEMhShjS/7FhWch/HGjbYqK95qS6Qx5/pQFTppNExhMfG5n4bEWSgQ8pV0i8NThzoWyslrAKkS03WeD6/Ms4BUnJ9ezi9QlyhQC37N8hepUXxaBr4ZKEYWAEJgTARH4OdFUWkJACAwhMEbg8R6DG8ZLhUTQ/uIlxQubPvGIEjcQ4jEGIt5K4DnJFY1zSW5pN84cuP9Ddg+b95xAhjlVNtrrY8NPfXPjMM/cwkJpgyoTDSYGsf4+/xyB5+ArTlTNrU681q6zIlASSDqTrJwwwWKiFaWVwPcu30B1dUsICAEhsL0IiMBvb9uoZEJg3xCoIfD4GY8bKzHn+MEAxjXt94lwjc2i+FCH2LcS+F+0Z6L9fUqejaIcGDUmbObEHjwneLVBs+2FDbr3LcT/A7t+s5EMf8vuD222zRH4Y/bMbxbSxSUlm2mHBF/w31mIcHG7zuTASyuB712+sTbVfSEgBITAViIgAr+VzaJCCYG9RKCGwN/Rav7kUPt/sd9fHa7dx34/LFzDrvvyq2utBP6V9hy23jnhEKeS5tnHHzpUKWcn/pf2MHb3OcGmnGeG5Jvt5qsHIuQI/JMsfu50W0x00H5/YiRPJjlMdnLCisGzw41WAt+7fCPV1W0hIASEwHYiIAK/ne2iUgmBfUSghsBz6uk/ZCr/NXbtXe46m10hzF6wnYa0Ii0E/jwWH639OTP5vsOuXaKyMbBP/ycLuU2mcTMuriM/bCFnN49ZDSY3NZLbB5CeyxF4zIQwF4rCSau43xwTViLYKJyTX7KLcRWjlcD3Lt9YfXVfCAgBIbCVCIjAb2WzqFBCYC8RqCHwVJyNrJzA6gUTGkxpkvyb/XPREIfNnU9fXWsh8NjcQ5pzwuFKN21ojaGNsEcsHQg+cjkLrBjk5Fl28Qcq88Qc5lghbiTweLr5bwtMNKJA4BN2Q1lf0G6W/PJjfoMZjpcWAr+J8lXCrGhCQAgIge1CQAR+u9pDpREC+4xALYHH/juSZnyWpxNDj9j/78wAhaYcjTnSQuAxncGEJic+35q2wWPMnQoR8Zjz+tW9nA1/eiynyS7ljc/6aFuf4kYCz/6AD9RUYmKcv7DnrhGebSHwmyjfxKrqMSEgBITAZhEQgd8s/spdCBwmBGoJ/E8bKA8PwHgPKTmPL++1+F4j30Lgb2jPPr/QEPey67/c0Ehop0sHN2F+woZY5Hst/H4h3bvbddxE1khuz0B6LhJ4XECmCU5N2q1xaN/LhIdaCPwmytdaR8UXAkJACGwFAiLwW9EMKoQQOBQI1BJ4Dj56RUCETZacsPpRC4+2kLTxKRo28Td3z7QQeHy0l7yv3NPucVJqrQxt8mSi8IJVQt9nf0tuKTmc6tcqM8QLDd5ochIJPCfdphWAyuSbosVJFA+3EPhNlK+pgoosBISAENgWBETgt6UlVA4hsP8I1BJ4NniyqZS/XpJnFk41jT7HIc6YniRpIfBzmtA83gpw10JTfotdTyeyUpeXFOKx+vAzld3hfhbvoYW4kcBzYuzbK9OdEo29CxcJD7YQ+E2Ub0o99YwQEAJCYOMIiMBvvAlUACFwaBCoJfAAgj311QMyP2e/MWfBe0v0GIPWHreMSVoI/JybWF9oBeAwqZx4G/0hbfPv2MOYCdXIkM19JPBsQP1gIVGINqZLc0sLgd9E+eaur9ITAkJACHRBQAS+C8zKRAgIAUOghcDnDlbi5FEOP4rmNXhW+TILn3QotxB4TnX9kIV13Ugynp60gMvLKJgAUcaPr26c2/5SxtxJqm+06zlXj7lOBBZMXnISCTzeZ8g/57qSA7Swp59bWgj8Jso3d32VnhAQAkKgCwIi8F1gViZCQAgYAi0EHi80eKPx8h/24xcsxIOE8CATtfUtBJ48XmUBE5ecoFHHPeSY4ELxeYVIpB8PbRry4c6JrnGiEpOG5OM3vSQ5P/Clw6NOWCLXGqvghPstBJ7ke5dvQpX0iBAQAkJg8wiIwG++DVQCIXBYEGgh8PiBx6Y6ypvtQvR0knO72ErgSYMNqznBVv06FY2UM/tJj+VMVHKbcVN8PNSw0XVIhnzA81yOwJcINasYF7fw/pE8Wa24QyEOfu1fFu61Evje5atoVkURAkJACGwfAiLw29cmKpEQ2FcEWgg8GHAiKyezjgkuGZ8TIrUSeE4h/ZuBjG5h9549cP/2du/4wP0rZ9JH449mPieftYusQpTcW6Itf5GFnNlPSi9H4If8z7PHoLQhNqV5YP88sFBmysvBV15aCXzv8o31Ld0XAkJACGwlAiLwW9ksKpQQ2EsEWgn8UwyFH65A4mIWh5NZvbQSeJ5F0453mJx8wi6ebgFbcS+Mobh9fKSFEpl+sd27biFdToBlE21OsFfHXSZ5QugR8ruNhcdZOF/huXQ5R+C5x6mrl888i50+J8CWJirH7N6TLOTs9jlYi3qQhpdWAt+7fCMQ6rYQEAJCYDsREIHfznZRqYTAPiLQSuAh75D4IflHu4n7wShTCDx25ycssJmyJJjwoBX/dwtMHNA6X3Ig/mfsHlplzGtyclu7+LSROr7L7r96Feeq9vfISPx0u0Tgb2URnjGQxu/ZPXzLv83CeSywCkJbcBBVSW5mN+KeBeJOIfA9y1cJZTEaE7tzhLtsUKbdJUJACAiBxRAQgV8MWiUsBIRAQKCVwEMcMaMZkqfbzdtlIkwh8CTzIAsPmLHlSOuMgfQYg19jARObuaVE4MkHG3tMj+YQNhXfp5DQFALfs3zr1v9TlkBckeC3CPy6yOp5ISAEBhEQgVcHEQJCoBcCrQSecrGRlQ2tJeHQpF/P3JxK4NGmPtPCLWcABS03k4tk/lJKkhUENPQXnZAnNvQcRJWTIQJ/IXuADaeXnZCnfwSvQBwmVZKpBL5X+das/iki8OsiqOeFgBCYhIAI/CTY9JAQEAITEJhC4DHLwEylJByIlHOlOJXAkw/jIsS0pFUeq/rnLMKDLRyMRXT3qQemOXiCqRW06EwSSjbrQwSePDg4iU2nJT/yQ+VgUkIdWbEYkqkEvlf5arEuxROBXxdBPS8EhMAkBETgJ8Gmh4SAEJiAwBQCf2/LJ/p9T1lD0i9gIafhXofAp/SvZv9wcFSNC8n0DCQcby6vn4DP+e0Z3Flibz5kh099Icb3t3BDC88t5DVG4HkMc487W8CzzEUqyszkhM2+nNo65LUnJbUOge9RvooqD0YRgV8XQT0vBITAJARE4CfBpoeEgBA4RAhc2up6fQsQ+SMrogvZ5vRWTHzwwMJBTy+08NYZcDnV0ri1hRus8mOzLIdYvXuVDxtM3zRDPj4J/Ltfz8KNLVzFAuY8mC59zMIHLeDjnQOzzrTwjpnzrklu28tXUwfFEQJCQAjMhoAI/GxQKiEhIASEgBAQAkJACAgBIbA8AiLwy2OsHISAEBACQkAICAEhIASEwGwIiMDPBqUSEgJCQAgIASEgBISAEBACyyMgAr88xspBCAgBISAEhIAQEAJCQAjMhoAI/GxQKiEhIASEgBAQAkJACAgBIbA8AiLwy2OsHISAEBACQkAICAEhIASEwGwIiMDPBqUSEgJCQAgIASEgBISAEBACyyMgAr88xspBCAgBISAEhIAQEAJCQAjMhoAI/GxQKiEhIASEgBAQAkJACAgBIbA8AiLwy2OsHISAEBACQkAICAEhIASEwGwIiMDPBqUSEgJCQAgIASEgBISAEBACyyMgAr88xspBCAgBISAEhIAQEAJCQAjMhoAI/GxQKiEhIASEgBAQAkJACAgBIbA8AiLwy2OsHISAEBACQkAICAEhIASEwGwIiMDPBqUSEgJCQAgIASEgBISAEBACyyMgAr88xspBCAgBISAEhIAQEAJCQAjMhoAI/GxQKiEhIASEgBAQAkJACAgBIbA8AiLwy2OsHISAEBACQkAICAEhIASEwGwIiMDPBqUSEgJCQAgIASEgBISAEBACyyMgAr88xspBCAgBISAEhIAQEAJCQAjMhoAI/GxQbn1CD7MS3ieU8g/t9y0sfGrrS68CCgEhIASEgBAQAkJACHwegRoC/wiLd6818PqcPfthC/9h4Z0WXm3hBRZetkaaerQNgW+y6K+z8EXusRfa/zez8Mm2pBT7kCPwa1b/uzgMftL+f/Qhx6RH9X/WMnmIy+jA/n9Qj4xnzuOBlh5lT4Ji4b6FPN5l10919x5r/9995vIc5uT+3ir/jQ6AK9r/f3uYAVHdhcAuIdCDwJfweLPduKcFiKRkOQRo47+w8G0ui5fa/zey8InlslXKe4qACPxmGlYE/pRTRODn7Xsi8PPiqdSEQFcENkngU0XR8P9011ofrszuatV9vKvyy+3/61v42OGCQbWdCQER+JmAbExGBF4EvrHLjEYXgR+FSBGEwPYiMIXAYxLzn5VVIv3zWPjikfjYZv9iZZqKVo/AxSwqg/T5V4+80v5ez8J/1SehmGsi8CZ7/kIujW+w/zEp2xZpLZ8I/GZaTgReBH7unicCPzeiSk8ITEMAXnDCPcp3+TpjSU0h8O+3RL9iLOFw/7z2+3IW7mbhVha+MNzHDpsK/FNjuoo+jMCz7Pb3r6K8ZtUhaidfwnYeBN5nyXy5S+oC9v82tUFr+UTg5+kXramIwIvAt/aZsfgi8GMI6b4Q6IPAZS2bN7qsXm//X2ks614E3peDQrGB9UtD4WTfONZabfdvaNGfv3qEjUnXssBGYklfBFoJct/SnXJKa/nOZQX0m6H/237Li9HyrXYYCXxC9Un2z49YYNPuwfJQH5ocvsRq6jnAx+33Zw9N7VVRIbA9COwMgQeyH7bwlIAdHmq+fnvw3PmSQNi/alWLF9lfVk4k/RFoJci9S7jt5euNx7bmd5gJ/JnWKN+3IvHxu7Gt7aVyCQEhIARqEdgpAk+l8EJz6VC7r7Tf/15bY8UTAjuAwLYT5G0v3w40cZciHlYCj4YYd5IXt3B1C+zjkQgBISAE9gmBnSPwv23o/2Bogcvb7zfsU6uoLocegW0nyNtevkPfgVYAHFYC/xNW/0dZqLIJVWcRAkJACOwgAjtH4H/OQD4jAI2v8lc1gM+GwJtYuLmFS1nA6woeb/7RwtstsCngyRbQ4MwhaIOua+EHLHDoBYdgfMjCv1rAPSObRlvKP1Qm7Iyx+7y9BVYqPmrhdhZeMlKRc9r9a1tgyZn9BpjRgNM/W3iHhbdZ+C0Lf70GINhOsjn2uyxc1cJFVnngmvKDFv7OAr7nn27hPWvk07t9KSptjI988GPjNW38XgtvXWH3VPvLhuBamYMgX9ky+24L17SAmdmFLXyZBfBmQ+w/WPgbC8+xwEFpeIqqlTnKV5sX8S5h4Xss3MDC11hg1Y1N7ay88R7Rv//IQgvGNfkzNtzJwrdY4GAzPDPRpgTehSdYwAa4l/Be0sduauGIBX7Tnv9iAbL6zBUWn1kVaF0Cz3vKqctHV3nxztL2jAmEP7bwXAtL20C3HOREX2CM4bwKcPrTxsbZh/GjscobiQ7OHMbFGHVJC3x/8aAxtpr+rRaHsYA+yft50VVb/5v9xaSWAx85LfzkjLWae3wfKhrfydta4HvMmMN4R10Yc7BAeJwFxrypson+zUoYvOQaFvg+spfx3RZQvPJtpM3SmOXrxTfshyzAn2jnC1qAk7zFAjyNSfpYfxnCaRPj25ztu3ME/v7WGg8OLfK1q0Yd69CQVE6A5GOAh5shYYPd71j4eQsQnSGJIPpTAnkJH2Mhmv3E9E7YhR+1wEs6JgxON3aR0oYiSA1266xIeOEETIhGSfjIcSrmkbGM7f6fWwCTP6mIm6Kcw/65hwXaDgI5Jp+2CGxOvp+FFnK0VPtS3jtaYFKXBPzTZl9cbNLGTAZLAjmmP/2MBQagKJCk3xsDxt0fOv2QD9xDLUBiagXii7cniHxO1i1fPJm59iRWPtD0NyakNZvn/8zi0ddqT4b0p3aCwdVWlWeQ5Z0gX/pVSSDO9OvjtUBPjIdbXZQX9x4pD8nzYYMAQOinEniIA0ThOyrKi9Ljly080cJSRL6FwEPiaBfeNRQCtbLL48etrZIoWLwwHqF4aRH6zdPcA4xbkGsma0mYuEJ8kgyNRR+xSMnxxAn7nz1WCETuDyx4V7lcR/EBOcvJVewi7+S3F+77y/RDxmveTRQpY7L0+O7zR0F1G3cBfFC0IYxdvOcRF/88E/ZfsQDPSM+N1Y/7m+jfOC+grHe24J0YxPK+2C6g4ESRh5zPAmMKypMhQRF1YAEi3yJLjm9Lty9cg291jaAoZtJzFqn5kMYP9hQ3krkCRnB4UekkkL4hwSUfp7cyCLQIAxCzfYhBSUoEHu0CqwUQ2BpBY/S9FpiNDkmOwDM7PWHBH3Gd0igReMqFp4Y71BTOxWFQ5xCtX6p4DuJBeZnItAofX2beNSR+yfal3KUB/sftHgNUbRvzIWSFg37lZV2CnNJiwIN4DZHOUjvglhXtLprsKOuWbwqB/2YrBO/C0McsVxc0OWx490SkVOccgT/VIj9v1U61ffbHLCKuMpcQtN584Pjo1AoKCD6Il7HwEPfQgf2PV5Yh4WNLHxr64Oae/327CJGkH80tLQSeMadmzPBl3PXxA4UUmkivmGLyxuS3RVhRYZUryQn7J5HudG1dAg8B51scPcqRfonA0yd5v6Ir6bG6sVqEhp+VxiFZenz3eecIHvyFCYcn9mN1Y5P2Lccire5von+/1vLmW1LLuVA4sNIJeaffoamvlWMWEU1+jSw9vi3dvjtN4GnkK7hWwgTlO0daDf/zEHCIdhRm52gpcGvHUtVpmTh8kPBD/+xCPjkCz3JenBV+wK6dtIC5ztdZyA1gkHhcOQ5NGCKBZ1Bj2TgOtKm4OQLPMyy385GPwqweX7/M3pgYQAJyBBVSNnYarvcp7/OBwOK/H5xYOThSwIOVA8o/JEu3L3nnBnjK7bXyLG+iwcbtJiYrEHVWh6Kwj4P+5GVdgkxaaLV4H6Iw4cIkiSVLDoOi3JDUdFCXj0/ZWS2Ky5Lrlq+VwDOQY/rAYB6FPsNECJIKvrxLuTqzZPsbmXv+UiTwvEMQlDQRRtvFu8XyPlp53gW0jygNvPDesvJRq/kfKdb/3UZ78opVvvEZJiq8Q4xhvKdg4QkOZQdDFBBJDuyfIQJ/ut3nAxGVNORF/6ZfsAQPPrnx66Wr/OY+9K2FwNdim+Ltw/hBXZ5hwY8rrfb/4IBphp+4oZGPmv11CDzfG8wmmJTmJEfgmRwzocwJ4wArmue2gJld7qwZtLS810Mkfunx3Zc9EjzGOCYnibyjlER5xaE8jHFgAgnOKTJYtWfla0g20b8xlb2XBcZxBM061gF8h45YYKXzqzOFBgcUN570026YGPMdg9STZhx/4VZ8t5iwDUmP8W3p9t1ZAh+X92goiEWJWKeGxEbTf8S4/pcW0FDgW94v+3IwFNfJywsaHe7l7OIjgeeDy8c8DYS8jCmvZGcMicdmGg1J1Jqz/EtnLH0EI4FnWR2ClISXnk7PAM7gxUw92q5jwvDIUEfqhtkKmjSvwWIAgESzooB2ywsaW+LnBHvGaH/K4A2BwO7a447GGHMFTD/84A5xYCkYAlWSpduXfOMAT3uy1AkeTHYYSCN5hkyBG0udkYgyQL3OVYg6ew0rfdqbG93Mfvv+wETB/yYvTED8IQ70NSZArJT4JXCyTfsyaNOjAViWnb3Wltvrlq+FwEOUIcJ8kL2AL/2T98nb60OqqUfUXkFgMSeLdfdpegKPxojle0gQfZMlXMz14ntIuR5vgdUhL7yXmKPNKWiV2MPihXeT9+QpFpjMJEEBcXcLLMOXVoQO7F6JwDMx4UPpn2Uy/3ALv27BfxwhTLz7v2CByaCXJc7mWJLA78P4Af58T+LqGeYvmDjVSCTKfDswYYurGVMJPOaXKG68+ScKhb+ygJkq7xnvnHddDGHjvYynsqM44RR2byLEGMh+H/prMoVL9casjDEXRV1Olh7ffZ6R4P2U3WQVF8EsE/OIk6GQfAtYzef99pNrFC6MzUMWCJvo35ixJcUKbUW5UXIk4XvPd4ZvaMmiA1LO2MeqkBfGeyaV8cAicCtN9Hi+1/i2dPvCQ9OYC8ZYUSThWwcXSUK/4L07i2zChIZlNz6Q3p6HDzoz69zmh1RgPNag8fRCI2P6MGSvCZnkA+k/ZnSaSOxJNxJ4nxcfPmZ9pbz4EFI+CJoXXuh7RuBXvyOBZ4WAGSmkBhtBBrahTaBohyHSEKUkTGgYWIcObeKFYRbttRwQayYbuYERDT8bUJJAjnDpluzcctVDi8zqg9cCgUMa4OIzPdqXPOMAzySJQQgSjnkQqxUlQeuE/bsXCCfEviSQJZY9k6D1HDqJlTJgZuGlZgmdfsNy9lH3YM2qVmv5Wgg8uNwn1IV3lo+AJ+4RO8yHojaKCSR7FEriCXxqU/I4ZmHIBIe2h1j4SRcfKN6poTIOFOVst3JtCmFnbwOTxpIwcYbIRS0V8Q8s5Ah86st+2Zq8MD9gc3lJ0ArSfzxh4qMBLmN7h1qwWIrA79P4wZjJuO/HDSa8TLJqhPee8TcJ2lBIfZSpBD59p0gP4k7ZWCEbMnd6pd1HGZaE7yhmWnE89WXkm823m2+4l6HxsOf4HgleGnfAgtLQb9cAABsySURBVFXDofGDNokr0kNOPDbVvxPu8Cw0xiVhjM1xKsZlOF9OYUpaaO/hE16hyDeiZBvec3zr2b6Re1atui1N4EkfkkgjMcPAntXb5dGAaJgh70OkhngsQzFjS4Lmkg5f85GNSxU8w/JO9HRRIvAQKj6AYxu7+NCitfcbg9BGoP3IaeEjgadulA2y/LuurqV/0R7e1d1kMGVW57V5pWdzphQslaE5iXLSLngTkpItfnyOSYInXSwNl2wDe7Qv5YsDfMKcXfK8NGPCSo/fEEgbMLErSStBjpsVGdxy+yFy+TFxo08lYVJ22kiFWstXS+AZkFnGZ8KShA3T17dQ884yeWVFygvvf4nwegKfnmF1iL0oYwKRxrzGS/LSMvZszX20cZjTJeFDz/hT098gL8czmRzYtRyBx5Y2jh1odKP2K1dutEFsvvcfUyYQN6mpZGWcpQj8vo0fcWxHwVBjg8w4jdbUf9v5HjFJjTKVwKd0UOAwURiahBIX4g6B94LGltWnMWEyg+YRhVESxhVWqXIKv57jeyR4lI+JF4o1r6XO1ZGVXL7TXvnGu8sqe0422b9rFEF8Z3IrRDWWFSgrmSAkwXwPpUdOeo5vPdu3G4Efe+Fa7jNrY5l4SJNLeiynnXAJQ6T5AOYGpVz+aDLoXN5WGFLvOw3P5Qg8gwTarFotFBosJhd+AGVQydnw5gj80OzT140BAFtob9IBli27uKOmBm1+9HxDnmhWWGFIwowaTf+YsMnEa+6ZMEVSRhq92pe8cgN8LdHjeZZ1/X6B4/b7DgNAtBJk2t9ry37VfuMLu0ZwRUifSMIHgsnjkLSWr5bAH7NMf9NlzHsEAa/xzsRj2GWzrI6dfxJMxVimzkkk8EwSaidl57W4mAT4d/aI/cYufV2BUDGR8jbt9CE8q9RK1F7y3IGFHIFn1euoSxgPIXFVcChfTK6YRCZhrMUu35tD1JY7F28JAr+P4wdjLMogL5h8DZmREZcVL78iyKqL32vm01uXwGMfXeNx67jF81p06sBYUDKDif2GCUg0HcWUlg3qUXqO7zmCx1jNmF0j8AR4TJJj9g+mdlE23b9LE8BYTsZQv58GE0m/ElTChHr7bwVmUqWN/j3Ht17tCy47ReCxx+SjEk0FSg2M+Yq3B2LZzRPDmpclfjgg5NElZI7Ao7lCg9UizCBZVUiC7drNMwlEAg/JYfaec08YH49Latjbs8oxZEMX04gDA/chTHHjIxqdI+5hNM5oiOaSXu1LeXMDPINo/ECU6gZZ95Ox4/Z7TgJ/cUvPa63R6IxNcFNZt4nA8476jdUvsN9eC13Td+Jkic2XvKM5iQS+9b2FrKPVS0J/n4PAM4n1rl95P1mRrFklS2XBlp/VKy8H9iMSeCZr0a90bjPhEPY5xQBjTc2KYE2bLkHg93H8YDLJxI9+mIRJH+/EkLDnxCthhkwf1iHwTMTpWzWrafR1JoFJmGSwwtYikeyWVj57ju+R4GHrzSrWmPY91Ruy7vfFHLPfOQK/yf4NKa9xGU2dmCx64s2+QK8MKLU3zku8fXeJwPce33q1L7jsFIGnwAxOmGyg/RyT2DEwG6FTtwg7ntnYlQSyzGzRv2w5Ao/Zj58d1uSJ/ZvfkIBWNG4QI51I4NFqo3mpEWb52BIn4QPLh7ZFWJ7EVt7PmtnQFydWkC9MH5Jg7oRJEYPqHNKrfSlrHODpB6wu1E58sNv03hyO2+85Cfw6eGIu4jcib1IDHyd92LWzCapFWM1iY1wStMFMbvioRIkEHnJ70JAZ2n6/2faI/Z6DwDN2HHPlwKzMv0s1RWSZnb0Z3qUodYsEPrY/E3G/glGTF3Hi+46pAyYPc8gSBH5fxw8IEHtskrDi7M0zY3ug1cbUIgm26kzqSwqAdQh8LTljBepkKGjLhtz0KCu3nvRTdq+9TvF6ju+R4EFCjza8JHAEuEKSY/ZPjsBvsn+z+ufNl4aqF/tT7cQ/crMSge89vvVqXzDtRuCZcY/Zq/tGZte5t/OKHWDM7AONEB8vvwQdPX/UvDOkg424l2hTmyPwteYiPl00+2gLvTCQxg2pkcBjC4ZXmRqB2PgNZ0MbRIfS46OAqUESzDfYYOMFzU/cQEU/YNKAGRTLWrVah1iWnu1L3nGAh2iy6lEr20jgmYBhG4h2zm982xSBZ0Mk2igvrEidqAV5FY+xg70jnriircH0K0ok8CzZ1/iPT+ksReCju9wHWIZnNOJAdMwOfD89sN+RwMd9A1O96bAPxpsqPct++9WUCcX/v0fmJvD7PH7kvkf0AcasnETzpzGlzjoEnhVlVpbHJJKuVoVJSv+o/cN3JgnmN/CKaAffc3yPBI/JOgq/Wqkh8Jvu3zjmiK6SS/WL/Ym9YtEMLPdsLYHvPb71aN+ERzcCP+UgJ5bPmHWjqYwdHK0aJip4QMgJSzLeewLx0WDV2s+lNFmSRLN8DpcJJiRsSiyByHVMGuKy9NgLymQD13fee0Ru0hEJPOQdEl8jECTvTxYCX7snwKfPx9pvjuIDi8s9LxBE7NfZIJsT6oq9Gy8r5ArNPNdqpGf7Up44wNMnohvBoXJvisDTfzG9oA3YMIS5FH958XHP6Se4qfybIvCUMe4ZGSIeQ3gzEfambiV3p5HAY3tJn6yVpQg8q29M3pNgOohdeqtEs7wDSyASeFZf/KZdPkB48WgVvEnwniSBOLHRdw6Zm8Dv+/gRTWLYf8M+lJzESR4OBPDeVJJ1CPwVLdGasxIwffWr5ZiHopVvFTaVR9NOlBVRUdBzfI8Er3Zjbqp7DYHfdP8ueezLtV/sT7VjcC2B7z2+9WjfhONWE3jf2JAlyLonHJDD6O81PRPto1pf/KH4UYsQQWSSgEeGMe8zuTwg/X4D4bXtNx9hL5HAH7ObuSW0mD5kDk8WfjIyFy6lVQDsg/FpXuMJgbJB5qkfL8HQBrie7QtGcYAf8oyTw7QngWfii9ceJqwMcn7Dck17b4rAxwGZ9wczJfpFq9CPvFkZS845UhoJfOsy/VIEnoms9+pS+1GLOB23C34j4IH9jgQ+kvxWrEvxIWoQtjlkbgK/7+MHhN3bvZdMR6KnF8y/mDQPfbvWIfBHLO0aE7O4qRYFD23WKph7xvEjt6m35/geCV6rmWANgd90/94mAt97fOvRvuk92BkCT4HZBBjthnE3mSN67DavWaprHRCIz7Iwy8MlENkYitZziqCB9BrrnDulSOBrbcbwpjPkr3xKedMzJZ/B3MeUgQGSjwoa4BphEsTSLiY4OTvznu1LeeMA3zJA8XwPAs/KDSYMbADCQ0qNQBSxr/abpTdF4PFf7jV/U1btUp2Z7LPfIknJVCwS+BqPHR7XpQg8BIoJd5JazWVsc8xuvB36gf2OBB6XvHMRbZ8/bgK9C9+a/liKMzeB3/fxA8XJydCHGHu55iW64sv1j9gm6xD4nLODXJtHsx681uC9Zopgpjm2qt1zfI8Er6RcKNW1hsDvUv9eWgPfe3zr0b6pb+wUgWf3evQdm9NQU7klOzBLxX5DYk4Dj+awZqd9fElZKvTkHzOh6It5Gwk82k2/sSY3+EBI0CRiDsAyrT80pjRY8fLhpzyaI/VsX8rWc4Anv/dZaDnICbtvfG8PmfVgA4sXCEgnJiZoSFnFQkPvJ8GbIvBRGwiJ5cNbu1HY9yE2hnmNXW6PBvG3lcAz0fbua6fsqaF+0SPPgV3rReAxzfAbfEvveM31XSLw8fuwifGDPOM7wIZOTmVOwkosplrJ0wvvG9r3MQ15DwIf909hsorpaquwYs/44SfDuXMheo7vPQjeLn0fd5XAl8a3Hu2b3oOdIvAshzGb9iYguFMCsChshPB26q2bDlsGitymodzm05o02Xzntae5U9amEvicCQ12hTXuJ2vK3hqH1RPaCaLF4FzSAuY21fVsX+rVc4Anv1YCH7UypIE3CTZIcSAQnpRKJkkXDvc2ReBzNvA5rWFNP4s2wLiPxUVllG0l8CetoN7mt+S/egyL2C8O7IFI4DmMytuqt27kHSvDHPfnJvD7Pn6AebQjx4EBZmpJmOxzSFoS/verVqV260HgY9mZVByZ0JE4uT1608FENbpj7Tm+9yB4u9S/lybwvce3Hu2bXoWdIvAUmhfZ+13GJeJjMy92rBgaLV7mJSRH4HGhFE+RG8sbsoKbTC9sNERj6mUqgSeNSAynLs2P1WXKfdoVUxNcoEW77ZvYNTTMSXq2L3n2HOBz7YQbxJIXJ1Zs6DdMcJMwecULgD+gqdQm20LgYzko71ELaBJbhAk+LiO9DXlpY962Evh4CFPrRreEFxtyvTu3A/sdCfyZdo1NvklaNsW3tMs6cecm8Ps+foA1zgogqt4bk1fYMLk/5holmoaW2qsHgcd01B/2hNcYVuOi95ixPoUCzB8eyKo44wLKDS89x/ceBG+X+vfSBL73+NajfVPf3TkCz9HQV3JvXml3PS87hMfbvs15zLl/+XMEHlv942OjS7gf7aQhIRC3uKFoHQKPZ4ijLt+hY5gbiz9bdJbd8YzjD4LAFv5+Loee7Uu2PQd48mvRwMcNX5gbcYpi7SmYmOqQX5JNaeDJP5qQtdqHkgYH0kRPF6V3f1sJPEqJ012bTDnQCrMqzmvw7ngP7Hck8JEclw67me0Fn5DQ3AR+38ePBDGnjqL8SJL2glB/vLMkMy201KwaR2Kba6oeBB5Tnnh6bOv+FMrOgVTeQxv+7nOmmz3H9x4Eb5f699IEvvf41qN903u5cwQ+Lo3f32rChpecRO3TlIOcAAiPHklYAYiHQeUI/BRfyti638DlxdIPG/uirEPgOVzFE+Exn785XCEEaASTXSFaDfxUe1tltL8s4yVh+RaNT6080iJ63/a5o917tS9l7jnAk18LgX+ixceTQZLj9k/c7D2EeyS8myTwbA73m9VYdfEEpKb/xI2bb7eH8C6Tk20l8Jix0I5JeLdwTfveGgBWcaIvbS4fWIgEPppSsAkfAtXq/Yfj4P0BUOyLAfs5ZG4CT5n2efxImKNV96ZjmNKhlY59gwP+aL8a6UHgKQd9HTPLJBzgiPviFnm5RWbfVRIOhfNjZbrec3zvRfB2pX8vTeB7j2+92pe+u3MEntO2KHSSqJn1LzcfKohlEmb02Nm2LMNFsgyx9IeV5EDkGh9cysmmwRrBbysbNr19f2l1YR0Cfy3Lw7ulBAswidqOoTKjxfGboSi3P9SJZyOJivaXY5hgRsOpfUnwToIZhJde7UuePQd48msh8Ni439ABMzSpzeEe7U03SeAjzrxHbF6v7Z9MLnnnILtJhjTK20rg2VgIkfZmUUNjXa5dT9jFuPHvwK5FAs95DbS533tzzH4/NZdo4RonffIhTsKqISs7rADMIUsQ+H0ePxLmmItAhNOp2ShbMKPhO4aZShJW7Py5KUNt1ovARyLEe432vHZiiXko3yYvpRXnnuN7L4K3K/17aQLfe3zr1b70650j8PF4YLThaNZzcqpdZPOq/wiifeZDWCM5X6p8EP3m2ByIKW3c86FRH/NGg40idr5oRpLgRpHye9OGdG8dAk8a8ZAbNP83tVAzseGjzDIkJglJ+LjGQ5zQAOP2Mwlpn2ZhzMNBih9JKWkxyHrp1b7k2XOAJ78WAv/bFh9XokkYQNjcXSNs6GJyBZZJNkngIZGYAHnzKcxHbmRh7D2i/A+zgEmRF1YY3lAAY1sJPMV9jgU8NiVhTODcixqixQoYZxVEObALkcATBzewd3GRMWXiyPl4CE4ORjx9MNb51UJM9eY6xIk8lyDw+zx++HbiZGE84yShD+DlBU9pyNB5Krn27kXg0ZyjQfdCub1/+1z5uIYijG+qPwuCcY19VrkJQM/xvRfB25X+vTSBpz/0HN96tS/12jkCH3cUj52IibtHbMuTQALQOOKdYUiYvfMRwgY9CR2Nj1qUnAlNivMY+4elyRL5wFaNBo8+bnOENaW5LoHHrjjWn+VubG6HbCDBAky8txj8iKPRifbWDB6QIy+YQ7B0O6ZBgYxCPvyhXaUNVj3alzr0HODJLxL4of0bkdzQFmjUxk4C5qRSSKI/dyDl7Sdosb+3lo/4j7DAEniSoY2SrO6wyuMFjSHXhkg8bUS/9i7jxsaHbSbwKAtOBBxoU4jxW3KNsrrGc0x6/CbeFP3A/skRePoCEwOv7GC8Y2IfPXb4rMGasSOabLEiRBnmkiUIPGXb1/HD445nGX9iORNB9kckaTUt7UXgKR+TC38IIIogvgWcrlkSyDumMsdChKGN4D3H954Ebxf6dw8C33N869m+kXuinPUWKtl3xH8gSy9R/GCvcyiLzwN7Pl7gJCwPYndZ+rBDRNC+RULCIU/M5LEJ9IKnmntYgHR7LSDuHbHpfn2mwkMEnugnLKD5915p+FByUiYawwg45I0OF497TlmvS+BpP4hN1JDxAedDifbbk2zKikaXARBPOV6GTpGLky2eA+8DC2hHcAnqBQ09m47w2e2JBCQLkvnxEJ+fPdqXfHoO8OQXCTxtxeQpJ2zqZnO3FwZFypzTPNPHwRm/0N5sIj3Pu4SmChOOkrSUjzRaCDxlotyxr2H6hWkVKwZe8NTE4VVx1eGjdo2JzJD5zTYTeOrIvpFjob5Mms+wAHH2K3SsptzNAqZ36f3BhMV73zqw3zkCTxaRJHONMQgzRFZ5ojkMfZL0/F4XnoE0eI1vKP6kn0sR+H0dPzzIEFomfrlJOWMq/abk4SrXWD0JPO8v+XlPOoxPaFQZU066AvJdQ2vPde8ukyh821i9YvKSk57je0+Ctwv9uweBp817jW892zdyT3gqm9FxgFKUTRL4eDAJhUR77V1OxYKjgWATpPdIk+IwsGFmw0AGYUCb7Mkj8Zj144cZYpuTCCLLzkxYIjFHk3XSAtoPyKo/qCWlC3FGS+1dJsY81yXwpIeGnAmFN51I+TCY47qSOnCfsua0eUw+IFQlwS6WPCKexGcQZpmefCBsEEY6Xuxb2NLeyILXIMX8lm5f8us5wJNfNHNiosoGT9yhshLCZq6TDojYJxLGLEGDMc9D5OiTHJjkP4gn7Dd25n4DIhpezNMgbTlb6NbytRB4yo7rQ/xS5yYYkG5IOe8K7ysEPgr95pgFBtMh2XYCT5u9wkLuRFP2B4ADbcuHGi8dkLUkrKaxAdj7+j6w3yUCz3vKe3btDGCkhQ0y4yXuCckr55YX80I2jZWI0khzFG8vReDJcB/Hjwgkm1RxuRyF96PW3C4925PAkyfKtEdlys43hJPLeYcxB2KjOpORKHjYYVWK/XMl6Tm+9yR4u9C/exH4XuNbz/Zlk3d0bIDyC5NGxmzccZ9tc/omCXzcUUwHRUvEkm3UzPmXFW0RWvfoX3zgnf78Lcg7Ax8z/pJEAo/nBTQBJyxAjGoFQsLqAmYNQzIHgSd9/IfzcW8pYyoXJi5o2cbskolz3II3h6nFAxJwewu5A3hiGku2L3n1HODJL9qtxvpG//1sesTfMd5DWoQPO6YpaE39ylZKo7Ry1lq+VgJP/uwJYX+GN2OrqRvvEaSkpt9sO4GnvrQtqw85El/Cg3cHDFAEoJVJcmD/lAg8cVByPNMCSoRWgSBBlOLBOa3p5OIvSeDJb9/Gj4hhPOU43T9q/7Aa2iK9CTxlY1UW16o13MPX5T32A+cHQ+Sd+D3H954EL2Gxzf27F4HvNb71bt/oetn3/+z5RzUv0VImNHxg+EB438YUeGgza6oQ2jo2sGJjXVMH7O9OtzA0MSDtHIFHG4AJDisG2NyP5YfLJzaRjQ005DcXgSctdmiz5A6Ji5j6jpD+52AgtO5jmk3/LBuJcHXYQkCYWDBzRMNSK0u1L/n3HODJj8kVHhQ43CgnuQO4iAs5ox95TWzuefzs0+bp442dKaslXjPPcyUC31q+KQSe/PEmwzuLK9exd4j49Bu8RLHRukZ2gcBTD95NTFmoW2yjWE+08kzGaGNWMFoIPGmBMxNvXM7mVuhifqTPXh/6Xo0f8Zp2iXGWJvDkt0/jRw5jFEuspiahn/CdGlPCxLQ2QeApA/vP0MR7Zw+lvoTi7QkW6Dc152H0HN97E7yE0bb2754Evsf41rt9sVCAE+a+j5MJ/JRButczLLljEsPSKZ0aDRdmLUwMMKeBTD/bQrSPL5WvROBTfIgrkwZs3iEkyb0ay9Esj5+5+ts6kM6JF0sxN14F7O/BhNUKOgBaDLBgeZ2O4v29t5QB7Ry4QxaTCRGmOR9eYc/khckSvulr3Qbm8p+7fVvqOGdcTIrwqIL25IiFdDAPp67iZeRkITP6IxorVoEgYEwkWaWiHTFxwJwMe/rY32gfSCIuTTHvIv4JC8cK+Uwt3xSMIBr0Hbw68c5iMsKSKOZqTCrZ08GhNXEvwJS8tvkZxg/c/2Eag8kfJgO8j4wlmDWxkoLWfWyjeE0dMUtAe0le2A8zJjBJZG8BtvdgTT9ipaPFhrom703G2ZfxY5MYLpk35nVsrj666v+MBaw4YaLK2MiqHWMcmknJ2RFQ//5fTPZpfINT4aEJxR7KNSawTFw5N4l35SxSowk7TC/OGIE/TFiorkJACAgBISAEhIAQEAJbiIAI/FkbRQR+CzupiiQEhIAQEAJCQAgIASHw/wiIwIvA630QAkJACAgBISAEhIAQ2CEEROBF4Heou6qoQkAICAEhIASEgBAQAiLwIvB6C4SAEBACQkAICAEhIAR2CAEReBH4HequKqoQEAJCQAgIASEgBISACLwIvN4CISAEhIAQEAJCQAgIgR1CQAReBH6HuquKKgSEgBAQAkJACAgBISACLwKvt0AICAEhIASEgBAQAkJghxAQgT9rY13AfnJKZJKP2D+c5CoRAkJACAgBISAEhIAQEAJbgYAI/FY0gwohBISAEBACQkAICAEhIATqEBCBr8NJsYSAEBACQkAICAEhIASEwFYgIAK/Fc2gQggBISAEhIAQEAJCQAgIgToERODrcFIsISAEhIAQEAJCQAgIASGwFQiIwG9FM6gQQkAICAEhIASEgBAQAkKgDgER+DqcFEsICAEhIASEgBAQAkJACGwFAiLwW9EMKoQQEAJCQAgIASEgBISAEKhDQAS+DifFEgJCQAgIASEgBISAEBACW4GACPxWNIMKIQSEgBAQAkJACAgBISAE6hAQga/DSbGEgBAQAkJACAgBISAEhMBWICACvxXNoEIIASEgBISAEBACQkAICIE6BETg63BSLCEgBISAEBACQkAICAEhsBUIiMBvRTOoEEJACAgBISAEhIAQEAJCoA6B/wG2hwFxz8KkvQAAAABJRU5ErkJggg=="/></switch></g></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-7"><g transform="translate(0.5,0.5)"><rect x="315" y="35" width="130" height="120" rx="18" ry="18" fill="#ffffff" style="fill: light-dark(#ffffff, var(--ge-dark-color, #121212)); stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke="#000000" pointer-events="all"/></g><g><g><switch><foreignObject style="overflow: visible; text-align: left;" pointer-events="none" width="100%" height="100%" requiredFeatures="http://www.w3.org/TR/SVG11/feature#Extensibility"><div xmlns="http://www.w3.org/1999/xhtml" style="display: flex; align-items: unsafe center; justify-content: unsafe center; width: 128px; height: 1px; padding-top: 95px; margin-left: 316px;"><div style="box-sizing: border-box; font-size: 0; text-align: center; color: #000000; "><div style="display: inline-block; font-size: 12px; font-family: Helvetica; color: light-dark(#000000, #ffffff); line-height: 1.2; pointer-events: all; white-space: normal; word-wrap: normal; "><div>Partie entraînée pour </div><div>la tâche tierce</div></div></div></div></foreignObject><image x="316" y="81" width="128" height="32" xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAgAAAACACAYAAAB9V9ELAAAgAElEQVR4Xu2dBbQsR/HGgxPcgssN7i7BHxDc3eHhkuDugaB/LLjDC+4QggS/wd1dH+7u/q8fZxuKSs9M9+7M7OzuV+f0uXt3e2a6v+7p/rqquvpIe0iEgBAQAkJACAiBjUPgSBtXY1VYCAgBISAEhIAQ2EMEQJ1ACAgBISAEhMAGIiACsIGNrioLASEgBISAEBABUB8QAkJACAgBIbCBCIgAbGCjq8pCQAgIASEgBEQA1AeEgBAQAkJACGwgAiIAG9joqrIQEAJCQAgIAREA9QEhIASEgBAQAhuIgAjABja6qiwEhIAQEAJCQARAfUAICAEhIASEwAYiIAKwgY2uKguBgRA4ud33q5aO5+7/V/t87tn3Az12kNs+1u56v3DnQ+3/61r62yBP1E2FwMgILIMAvMTqeLM56/k7u+6Xlr5r6YOW3mHpvXPeax0v+7JV6qyuYue1z59dx4qqTpNE4GVWqptkSvZu+27fSZY4X6hz2defsnRU9/Nh9vlalv6yQvVQUYVAKwKrRgBylfm8fflAS29WW+8hAqBOsCwELmMPfo97+K/s8wnd/zeyz69aVuEqnsuYyOLiou4a6nVVS3+uuI+yCoHJI7AOBCCB/ET7cO/JIz5sAUUAhsVXd88jcDT7Gk3T2WY/Ywa4tCVWzWihkB9aQjuFFm/Kcicr3DNdAd9vn69k6Y9TLrTKJgTmQWAKBOCfVvDfFhT+2JaHgaZNHmY/PqLgXquQ5SxWyG1X0C/a5y41qgjA9FuWdjyRKybtXNL/p1wzbOXYzJE/WLqIJeq5t6VPuPo+2T7fc8IVOYWVjXfo+LMyftj+XsHS7ydcZhVNCMyNwBQIwBes9NjcuuQolmHL0pksYWfEjyCW/1/23YVng07X/ab++zmsgGCT5DP24XwdhRYBmHqr7rHHz6yIJ3HFPIF9/s30i91YwtPYL/Q7CDpyU0svd7kvb5/fZon39++WLmDpcxOt76utXNeflQ3iAuFe5baZKMwq1lQQWCUCEDFjIHmlpTOGH/AFuPpUAF6gHPMQgGPZ83yb/sn+R8MimQ4C60YAXm/QXnsG79Pt710yUN/Xvnvc7Hvs65e0BFmfklzFCvOWWYEwZ+DTgB+DRAisLQKrTABolAtZ+pAl763LhMcKa9Vf3nkIwNp21DWq2DoRALb9YXZDUP3jjMu2v5w8xL485ewHTAFfm1ibMuGn8r3dPv98YuVTcYRA7wisOgEAEAaTuwdkrmP/v6F3tMa9oQjAuHiP9bR1IgBjYabnCAEhMAAC60AAmOxfF7B5gP2fnJIGgG2UW4oAjALz6A8RARgdcj1QCAiBHALrQADOYBX7Rqhc6ZbAI9t1OCld1tIlLKECPLGlPS3hmf0LS3gzf9TSayx9s6duhOMXJOWKlnBq/JYlHI5+4u6/DAJAufCfwKZ7Zkt4RR99Vj4wxinx+Za+1xMOtbdZRvl4R9gDfj1L57TEVrafWkKF/XVLB1vCYaxU+iYA9N97WcKR9riWDrT0+I7CnN9+p++xVe/0lujzRO9jqxtOb1+x9GlLaNHo+8u01/eNf4SGnUWXm7UvTraMAfQzgo3xvtPGL7X08dIGbsm3jP47T7FPahfdxhJj4nkssSviB5YYA+gTJAKyzSv0t6tZuoYldsFgSjqOJd4NxkB2X+DLRfyFVYy6WDK+z4tdr9etKwF4hqG0fwdS7CLALslEVyoEMmGw5WXoEvY706mRbUvYGBFeqkMs+a1gfM/EQpm7yp2e+2v74AOt8D2D1AVdwUojATIIYkbBnpu8uZvqxwuJ8+WjLTFRjCFDlo+BDlKThIEpOYOxBexpltr6CJMjeNzfEpNGFELHvrYCpNhm9LcnuOt32OfDLeH3sssSXvdeKMuNG563j33/KEsQ3lKB3NAnIQIlQh86wGV88OyZTdcOjX9bmZmAnmJpq6BiYE6fJ/porQzZf2vLQn4WMxDaJMe0D0Q4xImYvnZrS8douTHbImlX3o0aJ2OewzZQfEW6xhkeDwEjtssbCyvJwuTULi/Ewi+qum4D2fNO5Vv2/3caLppnfCc+xqRkHQhAzgTwcEP5gAakeRkJ9HHbOVuCQZ4BtEsbkOsgF7frCI6SiIEvwrIIAA6TlIldFTVC/a5paehQzEOXr2kCuqvV7UmW2L5WIvQHVpAx0M0QBOAD9hwmej+IpzI2EYDbWQaIcVcsjVxdmRx4Vkm0zb4IQF/45+pDmz7P0q1KGtblgeyxo8ETsq5bDN1/u56f+z1HANjOyY6Oki3Z6Z44SxIeuSRC4l6Wj/xdW5lz5X2ufUmApi6ysWwC0DW+iwBYI8azAErjADR1dAbpe4QfGez8qs7/DHNFTRqFlS0r++9b+oclVIGwScwBUdh5wEq+TTUaCcANLT9hi1Gv5WQZBICXkgkcc0MU1NwwYry6T2sJU0sUJgZiMjBwDCFjlC9HAFg5+P7zJfufFTA7S1CZM4idLlPhV8zw8D8NQQAgoA9tADxHAOirRLSLQv/9kSX6PSYv6k2fT4FwfH7qTqS/rhVVHwSgT/xjnTH7EaeA9zEKOxmIaYB27WSWzm4pRwAxsUAEumSM/ttVhtzvkQAw1kEq6dtJiNnwbUuo+jEFMgbkhBgPmA0ZM5sEbSfbP/05JSkv4yTmNPoXzyFPDvMX2/e37KjsMglAyfguAmAN2CcBwJb5EUtxVUMEst2ZzsLghtraq5942dmj/GxLfPbC5E/DYirwLwd52MvMS9MkngCgPuR/1MtJGHA/NisPKjX8FrCNJRUWdWCVkoQV5h3c/7yg3NdLrQkA1RqreC+QmwdZep8lz7ix1fH9zUN+Yg3w2xB+AWOULxIA6ghJpO2ZDMA8Tp5MIne0hKMpdncvaFI4SCYJhM+vqiBL/rQ8VlA+0hxEw/8fTQD4jjzSkh8kKSeqekgbfYqANkkoK7/5lRcT/3MssZKNmiy0gvgV8JwdoW68Bzy7TRYlAH3jH8uKqYudQ17ou6ilcSamPydhAqedwSIuBNCIROfj+Kwx+m9Hc2R/jgRg27U1q3miqUKAscknYfwjxgOaGfqUFzDFlNIkuUOi0KRCosDIH7AE8bq9JfpBNEOw2IBkN8myCEDp+D65raWrbALAOYWBLtpnIQT+IA/fWVj5M7gnYRJlsNtu6VT8xGqP+7IyScK9mlZh5PEEgA6eOjMTP4PNCy35wSYWYR4nwBoCwOEs8WVCRcwL3qZqg4W/wJKfgHCSisSgA9LOn8cqXyQAaIIglEziOIdFUugLDjlkxe2lawdKrRNgJAC+L7H6oi8RJbJJqMO7wo8Mrtiz24T+imloh8sEEbpUx3WLEoAh8WcSQwuHrTsJhBdi3hY3BPKE+hpCkATHXTQiTXEPxuq/Hc2R/TkSgJQJp2d8lcCoSdA+YQryhAjCyiKA8x6i4GxKP/LCWMr3bSGwIdI4AXqyzASKJrLpumURgHnG93narfdrVokAUFZWxykUMPa7yEQBiAEqp+7kt3da8vH0sS35VXUbwKyWGIyTMIEzeTSJJwApD+o01LGs2LpkaALA7gZUnElYeUKcSjy+cf7xzopDhGAeq3yRAIAH9UG71DaxJtzQlKANSoJ/yX4tjbsoAfDPoQ262ovJ3q/aUUPmVLG5IjMxHup+YNLLmYL8tYsSgCHxp22wJSdhImHi+nFLe6WfcqYcHNTQ3OVkrP5bUPQjZMkRAPoRk3/UKubun8Oi6RwWJn8m+ySYkBh3SnYR+OiM6XraD21tTpZFAFJZasb3edqt92umQACwHZVE7cNxDi/SNulalceXsiZgEOonVFlJ2rytyZMjAMQZL/UIH5IAsP1r29WFFT9nKHyysIfh2MSWIG8rhhSgPehDxixfjgCwzYm+USKYj7w9eJf93+Zc1gcBgJjQXiVbpNDq3NlV5Kn2+W4lFbM82Ib9jhcmSuy0bdIHARgCf0w11MWbbPAdOqgQC7KxsIDAJ2GlfO7M9WP234ri/ydrjgAwtrEzqlQght6kibc8GhavPcSMiYnJzzO8byyeSgVtA9twk7A9FXKek2UTgJrxvbT+g+abAgHoq4IlJ41hLvD+Akxi3v7UVpZFCQCOLqy8ulZsqQxDEgAYtNd8dJGZHC5xoMe3Ih0Hu2ibjlm+HAFgci3d981k7we0Xfb/0AQgHrjThvep7Ef2JSfB6a9k9UX+ZRGAIfCPKnmcfZmgMAOWSpzYuS631WzM/ltadp8vRwBqMOdeaFqjtiBuYcV/4lnuwZjTwKt0zOXSK1t6a6gkfgL4u0RZJgGoHd/nabfer1kHAsCLDJMvXVnPCyIrJ3/QSdekGTUA2FxRx5bKkASA09i8Y1qbWq2pvBz5ii0vCZoctDQlW4K6MBizfJEAUA80TaUTA5MxPhBJdtmHIQkA5cIu2uY/0oVv6e9xi+0YGoCh8I/vLzE9IAU1QuwFtJV+Gy8+RNHHYsz+W1P+lDcSABzycrta2u7N3MGqn+2DSeLuqxfZDzvd77VaBi4Fc2z/XtuIRiCSAvIukwDUju/ztFvv16wiAWBiRY3KCo1OwETc5IjTB2CsgmCyTN7e56CWABBdrzSgBeUeigCgAoWJ+7pEz/US3LhPdMbBtlfi39B2/7HLFwkAW5/ijo+28o5NADipjpXWkMIEdwNLmDf80cVjEICh8Gd3BIeHJcGfhy3EtYKZzKugMa/4Ve7Y/be2/OSPBADfKIJe1Qokin6SJPpURSLUFp+l7dnb9iPalyQ4X2PujbJMAlA7vtdiPUj+KRCAReMA9AEMZgGcmzAR8BfVIBGhWCX7yFL+WbUEoDQqX3rGUASAOvnz2LHZXclSLYmi77Dy8bsBeElxiltExi5fJADUiVVdqYxNACCR6fjd0jLm8tF+rN5wgkt9nr/0O96DnIPtGARgKPzxcPfRNyEApT4vHj+c/iDMSaLz29j9d54+EAkA243Zelcr7EAhsmQS/AKIrpgEc5PfOUWEQbQCtRKdN5uOnV4mAagd32sxGCT/JhMABjq2rrFNii2FtRHSagnAlj0DlVmpDEUAcra70jJ15euDBY9dvkgAatWUYxOAXdYIbSaGtjbCdoqjF4QPE06MYdDVvmMQgCHwZ5zDYdKT1a66lv7O/nf2wScZu/+WltPniwSArdF+Ii+9J6SBeBJJ4jZRzFTecZuxlq19tcIuFm8+bdp2vEwCsGVlrBnfazEYJP8mEgAYKfaanZZK68/Axw4COnCSWgJQG5d6KAJA4J8aU0RNx2NfvA9CU3Ntyjt2+SIBqI1pMDYBwMHMb2UrwZg9/cRgZxA9dskFloeDgdj77rUNYxCAIfDHftwWz6EQkmw21P9+l8XY/XeeskcCQFh0YnvUSnSs9LsisN3HXSrnnI2jtc+5v13wGHdR1DSkn5ZJAGrH91oMBslfOgH2+fCX2M38dpMxTQCEsyR6n3dc8XXDwQpvdnYH4NVJ2dhyxV8coryjoQjAEXsFGhXvEDdPvxlyAM2Vb90JAKc5spWqzayB3T2dbkjYY/wMiB6IhsBHLxMBOGKPZuL054qM3X/necciASgJDJV7Dip9Txzwy2I3QRIcgn00P4IIvXeOAuM74IOuscjIhXIWAagEd5MIAIwUhhqDoLCdhL3S27NBj5VPTmLwi1UlAASt8Xb6Wqeryi5WnX3s8q07AcC+Gw++YhsWtlhOPWQnR1OIUkJTrwMByJkA8HrH+71vGbv/zlP+SABqAqL55xEI6fHuCzRGmJeSRB8ATFe75igwZhYfYwSzA47ZURYlABBczGRJtuxDk1o/7vKSBqCwYZelAeCIVA4B8ULwGtRLTZO+z7suBCCaFlCNxmOFC5tykGxjl2+dCQCaLqL3QX6TQP6IaeED/DQ15LoQAOoXAzAN5bQ1dv+d5yWMBIAjjn20vtJ7HmwZb+EyM75iEkvCggu1f5ID7AOr+VqBWPhdCk1b7hYhALldTVv2XBGA2tbqyL8sAkDcdM9O6VQEmSgNzMPhH7w4SVZVA4BK7jeWvGqOA2sYIKcgY5dvnQnA/axBH+salVjtOLyWHkrCFkDfL1bVBAAEqJ53OCzYvubf5776/tj9d55yRwLAJLc1x404LZTdUknuYx/8Ucmc4OfPCKn17+C+7EZBSwsZTdIUcW8RAsB5D/4QL54FJiIAc3SMtkuWRQCwcXKOQJKd9gEGWyqooPyJV6tKAKgvR3NezFV8nkBArHS8LwcvSlOM7lKMU74xy7fOBADVLsFZkuyyDzU7CAhziz9AklUmAHi5s20tyTyBgDhECI/5ZDpl8YBtOgaNGrP/1r5b5M9FAqyNBYKtn/NDvMQTUnGOxLyaJB0tXLPlGMfrGGiJrdk5DVYkAE35cpihyYjzwZZ9JwIwTw9ruWZZBICzvv0pYF3H+cYqcHKejxy2ygQgOtUQr5v94G1nekc8YizwklDMpV1pzPKtMwHAxs+BKklKjvP1bUS4aE/qVpkAXMbq4reg0dfp8/E45LY+SuwAv8Jtiks/Zv8tfad8vhwBYLXOSZ+lgiMeK/EkTO4ETfNhftEOsPDyvmY77f+ahRfnQnBkdhKcVFl85CQSgB2W6fCCClE+nL3juQ5b9p0IQAGANVmWRQCiQwoDf+mhFHSkGPRmlQkAzBjnP28bZnXkt9q0tWlur3MfQYDSM8cs3zoTgEhaefe8zbatjTn0h+h5PhDWKhMA6srk4c+rIJIogWtKiC/mELYCYy5L0nQC3pj9t2bsTXlzBABtBositBddEk+JJD9RFf1pqekejJt++zTjMBFDS7ZlYrLFdOuFcOwEAsoJu7fO4H7gkC7vpNhUr5x/GHm3LIkAdPWGyt+XRQCiDZCgFawKul5+tvVQ5hg0hZfIh8GMMCzqJRqdidqYb3o223Au6ArS5uiEPc477DAAsOLDa7xNuCdY+gNm4vafyi6RzT5W+ZZNALr8L+KqsyYOQDywCds/PgC5c9t9IzBJsvJihewFfwA/AeYabtHTAGvtxDVxGNgNEfs329j2s9R2QA19nT7vQzDjOMxOgiZ/irH67zzvWo4AcB/aFyLfFs4bksCk7ONJoNJnvGICjpIjC2zFZnJHK9skhFtGY+PPAEDLsLelGII83YPVPouTJJzbQLkgHU1C2G8cYzk0K8qWfSEC0ALePD8tiwBwYFCM/Y3aC8eV3MlS+Atg72taMXGuOkzWH3/p8eibAPzebo6Kjfs2SQ0BYCDHSzcO6AQJIgY828O8sFMADDlKlsNoklAuBgVUaH3KWOVbNgHo2hu9CAHIOTbRR6gzbR+FNsbXhXfCD/ApHySRWBocwNUkUyYAqHpZkYK5F0JjU25MJj54DRoy3n/GASYeL/Hgm4jHWP23pSkaf2oiAFzApAwWaEf9Me2Qnf0tMQYcJdy5KTZ/yhbPDOB7dqewjZAYFR5zNC0sRMDcRxHkGkyw3KtJuObA8CNty/3ieEZfwLGbQ6JSuGL8CjwR2LL/RQBaAJ/np2URAA44wWvVx6am/HR4mCMqcSazvSxxaIg/LY98qFNRFXlhJwErpd2W+OxlUQJAOSIxYdDmOaxW0EjEc91rCABlZevPIZb8joBUB1aJYEI4TwY/BgBvMiAf2hM0JAycQ8gY5RubAEQ1NG2MPRWVKCtNYs3vdmAuQgC4TfTV4DsmcjRgvA88n4mfldI+lnxI7G37n7gZ/p2B+KKFYHLI2XKnTACoO+r5D8/+xj7L7hgwYVVPPtTJe8ZM9j87Kx6Q+T5+NUb/LSjGEbJEAsBKO5IiVvWs6Fl10/5g4W356aasnve1FKP++YcyqX/IknfCTr/Tjwi+BvY8h22DcZwhb8l5BfRjFiKQVC/0d8xZvHv0d55DfX1AuNfb/wS/Yothki37IAIQwFz032URAMrNSpU9r5FZttWJaFbEvKbc0Y8gXUeEQO8Qw/eLEgDuQaCSpqiFuf37tQSAZ/AisOqvjQvP5I89zp+EtmjfyF0/dPnGJgBxa1SsczTbLEoACGzC4FtzwiFlYmXEs1Fl56KuMUlCUqNMnQBQXt4pxoEYFKyk/3JWAVvbSrcPD91/S8oc80QCwG4eJumDKm/GwumqltpU+emW9EMwj452JY+kL3LeQgnmkFhISc3ZLpgkIGuMZ37b7Jb9LwJQ0kIVeZZJACgmans81kuOv+RFIVAQ6iokbgVM1R6KAPBysYLLMe++CAB1YHWPAyAqttyzYvPClLGbwqrHkCHLNzYBYPLBe9zva/YY9k0AuDfPwjOd6GlRfRvbjxPymPiT9zTbw1gxxwF1lQkAdUYjiJMYdfW7g5r6M+phVv2MX7UyZP+tLQv5cwQAYkO4aLaObnXcFP8H+hMm1bgFsu1StCmYl4hPUYI5OwhoH8wENbLDMrNbg77bJhAK8uEATT1i3AxwEAGoQX6F8rI9iohoMEbsdajBf2KJrSSo2V9nCa9fL0yOTBhMfqjEGEx3W0I9VcueS6GiE0NCmBiYPFh5M/iyP9sfvVl6v7Z8HAOLSh82zKAFayeWPGpAzAF4CKMqi/a0Pp5dco+pl6+kDuTBl4PBhtXh1gxjVKEQTfrk7tIbVeZDzc/e7EtYQsWNLwfH5KLZYtWEOQhnt7jSwjEMOy9mMZyyyL9taWfl86eYHS0GjmoknB/p82jDINjUk75+mCWIeM1kl6vrVPpvEwGgzEzSjCtoNGlv1OWo5DEJYhKgjzA24jA4r6BtuPosoYUBcwgZ4xo7TSCcb7KEaWJezBmrcdJmLMP0QMLExljGOA+pQLuFSWtjpWS1t7HgqOJCQAgIgTVEoI0ArGF1VaUmBEQA1DeEgBAQApuFgAjAZrV3Y21FANQRhIAQEAKbhYAIwGa1twiA2lsICAEhIAT+jYAIgDrCvxGQBkAdQQgIASGwWQiIAGxWe0sDoPYWAkJACAgBaQDUB/6LgDQA6g1CQAgIgc1CQBqAzWpvaQDU3kJACAgBISANgPqANADqA0JACAiBTUVAGoBNbflQb5kA1BGEgBAQApuFwA6rLpE+k3AwVAp3vllIbHhtRQA2vAOo+kJACAgBIbCZCIgAbGa7q9ZCQAgIASGw4QiIAGx4B1D1hYAQEAJCYDMREAHYzHZXrYWAEBACQmDDERAB2PAOoOoLASEgBITAZiIgArCZ7a5aCwEhIASEwIYjIAKw4R1A1RcCQkAICIHNREAEYDPbXbUWAkJACAiBDUdABGDDO4CqLwSEgBAQApuJgAjAZra7ai0EhIAQEAIbjoAIwIZ3gDWt/ousXjtD3R5p/z9kTeuragkBISAEqhEQAaiGbDIX/CuU5Jj2/18mU7rlFeTS9ujtzOP/at+dz9KXlle07JNvY98+3/3ybPt8p4mVccrFeZYV7o6ugHe3z0+ZcoFVNiEwFQREAKbSEvXlEAE4ImZHt68+a+mss59+bX/3tHSM2f8fsL+XshSxq0e/vytEABbDUgRgMfx09QYjIAKwuo0vAnDEtnuwfXXg7Ot/2t+rWDqVpRe4rLe3z8+bULOLACzWGCIAi+GnqzcYARGA1W38dSIATMhXc01xK/t8WGXTnMHyf8ESphDkAZYeO/v8NPu7/+zzr+zv2Sz9pPL+Q2UXAfhfZL9o/57IfXUW+/zbFvBFAIbqmbrv2iMgArC6TbxOBOA11gzXc01xLft8SGXTvN3yX2F2zevt73Xd9Ue1z++yhH8A8gpLN6m8/1DZRQD+F9mf2b8ncV+dwD7/RgRgqO6n+24yAiIAq9v6IgD/bbsb28eXz/79sv29sKXfh6bdy/7/hKXTzr6/kv2FNCxbRAAWIwD4d0DwkuDs+bdlN6qeLwRWAQERgFVopXwZRQD+i8v17eNJZ/++zf5+q6FZz27fX2b22/fs75sm0PwiAIsRgAk0oYogBFYTARGA1Ww3Si0CsLpt50suAiACsB49WbVYOQREAFauyf5TYBGA1W07EYDmtqv1AViPXqBaCIElICACsATQe3rkIgTg/FaGK1rCKe70lk5s6XiW/mgJh6uvWPq0pTdY+mhG29BTFf5zmz6cAI9rd7v6rE74AGDzp17I7yz9yNLnLb13Vq9fLlCJU9i1OBnyvC1LbDXE7vxDS5gfcGAkMZl1SZcG4Bx2AxwWcXDkuTjIUZevz9JL7O9Huh7S8vvR7LfLWcIJk0BJp7SE4913LX1z9oyX2t+PL/CMmkvHJgDLqD/4skuFd/BMsz6zr/1t25lyFPudGBaYu+jftBO7JX466w8ftL+vs/QhS/PGuaBc9OlrWzqzJfobsTXo09+wxC4bglZhPpOsAQIiAKvbiPMQgH2suo+ydNmKauM4xxY6iEBfwsT884qbtUV3Y9vf/S3d1dIJC+/5Z8v3REvEDKiJnnhsy/9QS/ewxMTRJjijPX6G959aMjYRAAIYcf1+BXV6reW5r6VvF+T1Wa5h/xA1b6vgusMtz6MtvaMgb00WiBTlL5XzWkaCPSUBo3u7/2siAQ5Zf0jncWbl2ra/yffkEvYZcui3OpKN4FVfbQCBa9jueM4CkD5jeYgkWUMK6cvg9jBL9PE2gei+0hJ9gYWCZIUREAFY3carJQC3s6o+w1LXxJVDhEmSFeKbe4KrLwJwcivPGy1dZM5yvduuI/4AhKBLTmMZyM+KrUYY1Bn8WbXnJEcAWB1uWzpPxYNYpTE5MvF0CatJYi8Qb6FG6HMQjSfUXNSRdxkEYIz65wjAxQ0L4lskYuChaSIA9AVIe81YTTvdxxIkt0vQKFGmC3RlDL9Tv2taQqMmWVEEajrVilZxbYtdQwBYQbw/gwT3YGL6gSWCrTChntrS8TN5+wyg0xcBYG8/6usof7Avvj+rFytpVPQkBv4oj7EvHtjRS8DkfZb2zuTbbd+RMKEQtCa3gmJrIm2QMztEAnCw5cMsc0n3rM/ZZ1TwmGe2LKECpkxR2Ap50466HNl+J98NM/nAjbISQvlkltg1kcOMVTdEoA8Zm1QX0PoAABFdSURBVACMVf9IAMAbE1TarRKxyxEA+iWTfxTaCWJJfzjjrC/kxvI7229oDpoEMxkTOGamKJgWMDOhyWLrLIG2orAwwDxF3A3JCiIgArCCjTYrcikBYMBDjY99NwnXPscSKznsvF7oE5e3xMpjR/iN0/Q4VW9RQQuBPTMJqkc/4fE/Ns0kX7MP0e6InTIOPAyKqPVfZenvoZBMaLe1xKB6LPcb8QIYCNu0AKi9wcTLofYPoYeZnJOAHXZd1OrYUL0QlRBMo0QC8A/LkCZd1Ll3sPSxcBH47Zw9B4LjBTUx0fSaBFXvk8OPYAsu2JC9uQJcOGiHcsfnoBEi/6LChHgudxPaFDKVhKBQPqYDpij/f60JYKz6ewKA+YT/fbRLCDftihqd+rBa92YxCCOk04/RELP7WXqZJUhAEswJD7eE6t8TNvKcztIvGhoJ7RmreC/4EDxo9mzCaSeB3PL9zUN++gu/yS+gAeQpfy0CMOXWaS9bKQFghcxK2QsvMja8NiHACqrBHS4TWgQ/cfeF3jxOgJSFQTIJK/4LWfpxR6E4H4D9/36gBKP3NFx3S/t+V/gN7MCwScCO8lGeJEwADMZoUrxEApB+4+AighX5gT4+j9U+Dnpe7mb/PLWhYGgWWIV6AsSAz8QUy+VvAXkkaBKEIAkmB0Iqs0LsU2qdAGsIwJj19wSAlXI6kIqJH7L1QktNviE43tFOnkTiYIoJYXcL2DlS3HQM9o3sPkTE9IKJEF8aP/HHx/E+cLaGf3/og5EY9NkndK+BEBABGAjYEW5bSgCYqPyqnVVyOi2vq5hMDKx0kzDo51SBXffp+r2WAKDVYMdCGlS5P6rIOKA1PReNiLd53sL+x5s+J6jEPV6Qh7hqyl2HapYVvDcJ5A4iyhEAtBc8M2pncs9hlei1J23q+WdaXn/UMJMRq7cu0sRzc6p6nO9K7Mxd7e9/H5IAjFl/TwBS/TABQVrpU22CuQBHuyRMyDjwluzEQLNHP0sCccD8FQUtESaeJGhWLmqpZAeBP1uD67kGsxTvlWSFEBABWKHGCkUtJQCwemyBSVgdskosEbYa4R+QhImCrUF9Sy0BYCWKjTIJWKBKLt1Z8FzLi1NkEtTcDJxR2Ca57b5kYmbQxDZaIpgibuAy7rLP0fEuRwBYYWGuKJE4GD/dLrpL5kK2SdKW/E3CboaDSh4yyxO1LqxSz11xfUnWoQjA2PXPEQC28JXseMAuv8OB9WL7zMq7RNDKfClk3LL/v+O+i/0agsEE/smSB1geHAfZFuh9heiHaA8kK4SACMAKNVYoaikBgP2zvzcJTn+le+CnSgCwgXv1KHbzmi1JpQQAsuQnU9TgqOVLhckeVW8S/Bq82YLvcwQA7UHJ6p/rKZ9X+TcRgKjyxWSCU2P0lWirW5w4yIvjaJ8nKw5FAMaufyQA+LGg1elaYeccZFmZ12zrY7JPZ17QRviveDPgs+1/fEuSoG3gPI0awU/nAHcB7x/kQ7JCCIgArFBjhaKWEoBFangdu9g7ek1FA7BInTAfsNJhy1ySJg0ATlrejt+Ur6k82Nr9tkFswZGoRAKAWj63C6PpGQzkDOhJmghAJDNoJ5gUa4RDd/AV8NvY4uRSc79c3qEIwNj1jwSgy28kYYGPylscMBB2SHwXcfBY4uvgNT0QAhwIk+C46h0vMQv5PlTShmy99aQEEk6/KNlSW3J/5RkBARGAEUAe6BFDEgBeZFTXj7Pkj2ZddQLAvnp8IlDFeslN7PgXMIj7uAkXtP9L1aSlzR4JABEYidRYKqUEIJKZe9kDnlT6EJeP+vvydW01q33EUARg7PpHAoCDHl73XYI3P8GmkhB7g+h8fQnEADIAEU6CP8ynKh/AfSCrXjCPdfk3VD5G2YdEQARgSHSHvfeiBIC2J7gNTmA49qEO5i97glGv+wEi1WQVCABOd9QJNXqqF/Vh8vemEN86OQKACtXbTcnPvXE+7FMiAcBGHAlK2/NKCQBbwXz0OQjAPGQGpz/vQIkq+BE9AjIUARi7/pEAxAiGTZBhMvJ+In3GXOCZrPz91lXs/5i1andzMH5gVvC7ATAR4ZQqWREERABWpKEyxZyHALAX/mazFx4VnlcTliAxVQKAGhqNBVsUUbnX9uscAYAw4MWfBPU9YYf7lkgAcEakPKVSQgDAgxCuuaA+pc9pykfMA/bW9yVDEIBl1D8SgC0DKBLKHGacv0HsgyTEYCCGRF/CO0JcgiGkVMsxxLN1zzkQqB0o53iELhkIgRoCgDr7npZQf3fF+k7FZaWL0xsvdZKpEQBU0UxA0bGuDXK2PzEZdvkAxIGSFaQ3h/TVrJEAYIv1W/W6nlNCAPAp8DbgrnvW/E6kOb/LpObaXN4hCMAy6h8JQKmzJPEoLuOAwckTv46+hC2sJaaIeZ7H9sVXz3OhrlkOAiIAy8G9j6eWEgCCimBHjJHsfBm+bf/gpcz2NrYQcdgKe3rREPitdVMiAETcY08+9csJ0dWwR7JdidgHbFljHzURy+LWuZwGIDo54S2PBgBnpz5l1QlAzZbFEtxWjQA01X9eAvA2A8nvNOnbxDIkASAYUAxMVdLmyrMkBEQAlgR8D48tJQAc+hL3lKPOfpElvI3x5G3aPx+3JE2FAGDTx1nOR7QDUiZ5VtBEt+Nz02RdQgB4RjydDb+AvkOejkEAcipwohJy5O/UZAgCsIz6z0sAOKfBb8nDJ4A+0pcQNMrb6SH/7BqQbCACIgCr2+glBAAnP6L3sX0rCS8/UfN8gJ8mFKZKAGK0MyZ6BkkCppRslyohADjMxRjqmBr8GQV99J4xCADljBNrqVNaH3WsuccQBGAZ9Z+XAMTtijjatWnvarAlL06+X3AXYRoqPUa79lnKP3EERAAm3kAtxSshABwc4h2ICAuKc1tpxDxs3gzISaagAaDPot73q39iq3OqX6lgU93PZW7a3x8DqtSqONlCiJbFbyXEwYtT3JKMRQBidDmcJonAODUZigCMXf95CUAMHkVAKHa01AhRA3e6C7D54yuD4A9E//NhtImi6d/zmmcp7wojIAKwuo1XQgBixLtdVl2/xair9oR5xR8gyRQIQIxOSNm2LJV4WKd6cOKcd25sIgBESPPH5nLWANqTUtlhGZl4kjAp+JPu+H4sAsCxsv7Y43kCAUG6OAExjRv0Qfas10QT7MJuKAIwdv3nJQBEC4x76dnWio9OqWxbRrbkJYkxH9BiXcz9Pk8gIDQJ7ChKwvtXG0yotD7KNxACIgADATvCbUsIAKtPIoslqT3ON3qYT4EAEJnPH4+LRz+rmRLVPzgQ3wDbtz8gpYkA7LR8+EokYWcE9tLS0Lf/Z3nv464vCQU8xC4AioBnuT/xELMJE0tpyGHuwUTCEdJJaoMWuUsbPw5FAMau/7wEgDEZs92WQ4jYCxy8VCLseEDD581+MVrjw+13H2yIPkBfqHFwPdTy++ONOWKanUaSFUJABGCFGisUtYQAsGL14V458Y6T70qEQ3+YaE/tMk+BADBQxXC6BPxh0CwRVrAHhoxNBGBPy4evhLeRlh7Mgv8Ftla/4ocM+AmUYoylAeBZ7PDw8drfav9fw1LJwI85iC2UqIuT9O2hzn2HIgBj139eAkA5o+mOID1o46JTaq6/MxH7uAzY+GkziHIS3mmc/zxJqDGj5WIJKAhQrjUm/p0IwMQbqKV4JQSAAfoAdw9WBvgA4AvQJkwSBCRhsvXC4OwngL7Qi6cBttmn2YrHYUZMzkmw/3v1dq5crPxZ9YBJFPwBOCo2J1F1TJ77WiJCW5Ng82dy3ddlYBDHlht3EYxJANgNwq4QL2xjo/7sDGkSIihiyvCxE9CGsJOg1J+k5fb/81MkAF32adrBr46Z/JK9Oz5zzPovQgByp+2xSseDn7MBmuSy9sM7Lfkonk1xGtiud1N3I8YTNH6xf8Rn0QfoCz6qJttrOU1QsmIIiACsWIOFF9aXnokxDuLns+9ijG9eViYdtslFYaXLkZ6sVHMBgxgk2ArHSXJ9SiQAhJbNTdTpmYfYB1auSSgXkwBHH+cmsh32/SMtXbyh0M+37/3xwD4bRANVdyRDhOyljPHoVU5uY5eCP2yF+xF3HxV6lDEJAO87XuVMFF4IDUtdMBn5lSIrRDRGaE32DteAF7j1LZEAUFbvRxGfV0MAxqz/IgSAOkZnQL4DG8gn/ht/ckDwrkKAeW+9w+kf7H8iY+ZIA8SKMSASehwGOQPEH/TDoxgbOD6ao8S9VguHXIiJj5rZd5/Q/QZCQARgIGBHuG2JBoBiRFsd33EtZ7sT+Oens5cbp559wgCybf/jlEQUsySoIbFTcyrcwT3Vk9W3j37HahkHPGz1TMBMWoe5Z+EH8FFLsf+i2fiwpd2zvDgMoq709n5OK+NePtwq8dBZ+RD8CLOHj5XOrYh9j/08OvDxG4GGcICinByGkjtvgPuxhZBJIcqYBIBno/4FI2/aSWXCO5w+waqe3zGteE1LysfOEkLUDiHRTEH/JLocqmywxR6+2z24hgCMWf9FCQBljav0VG0mXXDiHcQnhZQL88xuAExWTUIwLci03xGQ8vIuYSaAaED+0PZ4kwH5MB0RWMifXtjyOP00NQREAKbWIuXlKSUAxP8nME5tsA/2I7NiZRDynvCphEwSe5UXtzXnre1XVNFNklPrstrBya5GIBR4/6O+bjq1bH/7DU1CFE4ChDgQG6FGsJtf2VJTAKGxCQBlxz/hHZYgd7XyMruA7ZClTpe192fC4v5NEuMX1BKAserfBwFAlY+WpWbnDvWD0OJHEP1NcpiiYWHVX3suCJM/YYoxMUhWFAERgBVtuMwAnDMBpNoxaeH5i7Nb14EwnBDHxH/47GJWv6wYvWqRn/okAKwsti01qeib7LrscEC1HtXzsVVZ9R9kCV+BdIRp3AqYrmkiAPyOYyTqUbY/db07aDEYvFHZooptkmUQAMrCkc+UjbaOERVzZcUZklU/jqRDCuQEk0sT0eqDAIxR/z4IQMIZAs5kntPaxLZAM4CaHq1ZqbC6593AYbirX3NPNGX4jfjdOKXPUr4JIVDS2BMqroqyIAKo+Tm4BXU0gwkqbaLdYSMkQiDqQOytcXWHhy8OdNi12WZE/m1LOxcsj7+cSQiCQhjULUuoexlEUa9jnybuf04gJkyibEnCwZGJgxUQOxZQZ2MCYbLnfy+seLCbXtcSkw6+A2gFsHe/u+FZ6WvCBF/HEjHb8YnARMKKCCwZgDEXENK1y9my4zGj/IwWB+xIOH+iMUpnxtPO2ILRfIBjn/v92yqH6YYVLKvTLUuc94C6m50exGHY3SMyU6x/rnoQfPobGix8e+hzvCOYRminD1jC8ZQ0r3aGfo1KH9MApIC+APY43WIOYBsr71L0D+ixOXSrMREQARgTbT1LCAgBISAEhMBEEBABmEhDqBhCQAgIASEgBMZEQARgTLT1LCEgBISAEBACE0FABGAiDaFiCAEhIASEgBAYEwERgDHR1rOEgBAQAkJACEwEARGAiTSEiiEEhIAQEAJCYEwERADGRFvPEgJCQAgIASEwEQREACbSECqGEBACQkAICIExERABGBNtPUsICAEhIASEwEQQEAGYSEOoGEJACAgBISAExkRABGBMtPUsISAEhIAQEAITQUAEYCINoWIIASEgBISAEBgTARGAMdHWs4SAEBACQkAITAQBEYCJNISKIQSEgBAQAkJgTAREAMZEW88SAkJACAgBITARBEQAJtIQKoYQEAJCQAgIgTEREAEYE209SwgIASEgBITARBAQAZhIQ6gYQkAICAEhIATGREAEYEy09SwhIASEgBAQAhNBQARgIg2hYggBISAEhIAQGBMBEYAx0dazhIAQEAJCQAhMBAERgIk0hIohBISAEBACQmBMBEQAxkRbzxICQkAICAEhMBEE/h9D6cw1SPafoQAAAABJRU5ErkJggg=="/></switch></g></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-8"><g transform="translate(0.5,0.5)"><path d="M 475.5 89 L 475.5 79 L 505 79 L 525.5 79 L 525.5 68.5 L 544.5 84 L 525.5 99.5 L 525.5 89 L 505 89 Z" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="1.42" pointer-events="all"/><path d="M 525.5 79 L 525.5 68.5 L 544.5 84 L 525.5 99.5 L 525.5 89" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="4" pointer-events="all"/></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-9"><g transform="translate(0.5,0.5)"><path d="M 475.5 39 L 475.5 29 L 505 29 L 525.5 29 L 525.5 18.5 L 544.5 34 L 525.5 49.5 L 525.5 39 L 505 39 Z" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="1.42" pointer-events="all"/><path d="M 525.5 29 L 525.5 18.5 L 544.5 34 L 525.5 49.5 L 525.5 39" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="4" pointer-events="all"/></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-10"><g transform="translate(0.5,0.5)"><path d="M 475.5 89 L 475.5 79 L 505 79 L 525.5 79 L 525.5 68.5 L 544.5 84 L 525.5 99.5 L 525.5 89 L 505 89 Z" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="1.42" pointer-events="all"/><path d="M 525.5 79 L 525.5 68.5 L 544.5 84 L 525.5 99.5 L 525.5 89" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="4" pointer-events="all"/></g></g><g data-cell-id="Doz-i6v8k7lZ-tEtjD9M-11"><g transform="translate(0.5,0.5)"><path d="M 475.5 139 L 475.5 129 L 505 129 L 525.5 129 L 525.5 118.5 L 544.5 134 L 525.5 149.5 L 525.5 139 L 505 139 Z" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="1.42" pointer-events="all"/><path d="M 525.5 129 L 525.5 118.5 L 544.5 134 L 525.5 149.5 L 525.5 139" fill="none" stroke="#000000" style="stroke: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));" stroke-miterlimit="4" pointer-events="all"/></g></g></g></g></g></svg>
\ No newline at end of file
rapports_presentations/SoutenanceS9/main.tex 0 → 100644
View file @ 69859c8a
\documentclass[french, 12pt, aspectratio=2013]{beamer}
% valeur possibles : article, proc, book, report, letter, slides
% options possibles : twosides (recto-verso), 12pt, 11pt, ... (taille de police), draft (brouillon)
% Language setting
% Replace `english' with e.g. `spanish' to change the document language
\usepackage[french]{babel}
\usepackage{amssymb}
\usepackage{shadowtext}
% Useful packages
\usepackage{amsmath}
\usepackage{graphicx}
%tableau pleine largeur
\usepackage{tabularx}
%faire des dessins
\usepackage{tikz}
\usepackage{algorithm}
\usepackage{color}
\newcommand{\com}[1]{\textcolor{olive}{#1}}
\usepackage[dvipsnames]{xcolor}
\definecolor{FFN}{HTML}{00AEFF}
\definecolor{MHA}{HTML}{FF9100}
\definecolor{Emb}{HTML}{F94C52}
\definecolor{Norm}{HTML}{AEC500}
\usepackage{listings}
\usepackage{appendixnumberbeamer}
% Theme choice:
\usetheme{Darmstadt}
\AtBeginSection[]{
\begin{frame}{Sommaire}
\tableofcontents[currentsection]
\end{frame}
}
\title{Exploration de l'explicabilité de l'intelligence artificielle et application au traitement automatique de la langue naturelle}
\author{Marine DELVALLEZ}
\setbeamertemplate{footline}[frame number]
\setbeamertemplate{navigation symbols}{}
\begin{document}
\addtocounter{framenumber}{-1}
\begin{frame}[plain]
\includegraphics[scale=0.3]{images/Logo Minerve_RVB.jpg} \hfill
\includegraphics[scale=0.2]{images/France_2030_Logo_rouge_bleu_transparent.png} \hfill
\includegraphics[scale=0.3]{images/LIFO.png}
\centering
\Large Exploration de l'explicabilité de l'intelligence artificielle et application au traitement automatique de la langue naturelle\\
\large Soutenance du Projet Immersion\\
\vspace{2mm}
\normalsize Marine DELVALLEZ \hfill
\textit{Master 2 Informatique GPEx Minerve}\\
\vspace{2mm}
\centering \footnotesize 7 Janvier 2026 \\
\end{frame}
\begin{frame}{Sommaire}
\tableofcontents[]
\end{frame}
\section{Intelligence Artificielle et Modèles de Langue}
\subsection{Définitions}
\begin{frame}{Intelligence Artificielle}
\begin{block}{Intelligence Artificielle}
\centering
\begin{tabular}{|c|c|}
\hline
Comportement humain & Comportement rationnel \\
\hline
Raisonnement humain & Raisonnement rationnel\\
\hline
\end{tabular}
\end{block}
\pause
\emph{Données} : Images, Sons, Vidéo, Texte, Tables Attribut-Valeur, Entiers, Classes,...
\pause
\begin{block}{Tâche}
Nature de donnée d'entrée + Nature de sortie + spécification
\end{block}
\emph{Classification} : Texte $\to$ Classe, Condition\\
\emph{Sous-titrage d'images} : Image $\to$ Texte, Décrire\\
\emph{Traduction} : Texte $\to$ Texte, Changer la langue\\
...
\end{frame}
\begin{frame}{Grands Modèles de Langue}
{\centering
\includegraphics[width = \textwidth]{images/Modele-Systeme-Input-Output.drawio.pdf}}
\emph{Exemple} : BERT pour la classification de texte
\end{frame}
\subsection{Le modèle Transformer}
\begin{frame}{Architecture du Transformer \cite{vaswani_attention_2017}}
\begin{columns}
\begin{column}{0.4\textwidth}
\includegraphics[height = 0.8\textheight]{images/Transformer_Vaswani-et-al2027.png}
\end{column}
\begin{column}{0.6\textwidth}
Composants Techniques:
\begin{itemize}
\item \textcolor{MHA}{Mécanisme d'attention multi-tête}
\item \textcolor{FFN}{Réseau de neurones}
\item Connexion résiduelle \& %\shadowtext{
\textcolor{Norm}{Normalisation}%}
\item \textcolor{Emb}{Embedding} et Encoding positionnel
\end{itemize}
\pause
Modèles dérivés du Transformer:
\begin{itemize}
\item BERT \cite{devlin_bert_2019}
\item GPT \cite{radford_improving_2018}
\item T5 \cite{raffel_exploring_2020}
\item $\dots$
\end{itemize}
\end{column}
\end{columns}
\end{frame}
\begin{frame}{Mécanisme d'attention}
\begin{columns}
\begin{column}{0.5\textwidth}
\begin{figure}
\centering
\includegraphics[height=0.65\textheight]{images/scaledDotProdAttention.drawio.pdf}
\caption{Scaled Dot Product Attention \cite{vaswani_attention_2017}}
\label{fig:scaledDotProductAttention}
\end{figure}
\end{column}
\begin{column}{0.5\textwidth}
\begin{figure}
\centering
\includegraphics[width=0.6\textwidth]{images/MultiHeadAttention_Vaswani-et-al2017.png}
\caption{Mécanisme d'attention multi-tête \cite{vaswani_attention_2017}}
\label{fig:multi-headAttention}
\end{figure}
\end{column}
\end{columns}
\end{frame}
\subsection{Les modèles boite-noire}
\begin{frame}{Modèle Boite-noire}
\begin{block}{Modèle Boite noire}
Modèles pour lesquels les processus de décision ne sont pas transparents pour l'humain
\end{block}
\emph{Problème} : Perte de confiance pour certains usages
\end{frame}
\section{L'eXplicabilité de l'Intelligence Artificielle}
\subsection{Définitions et enjeux}
\begin{frame}{Définitions}
%\com{
%explicabilité de l'IA\\
%explication}
\begin{block}{Explicabilité de l'IA}
Rendre \textbf<2>{humainement compréhensible} les raisons pour lesquelles un \textbf<3>{modèle effectue} une certaine prédiction \cite{garouani_investigating_2024, bell_its_2022}
\end{block}
\pause[4]
\begin{block}{Comprendre}
Être capable d'anticiper et/ou justifier le comportement du modèle \cite{bell_its_2022}
\end{block}
\pause[5]
\begin{block}{Explication}
Tout support rendant le processus de décision d'un modèle humainement intelligible.
\end{block}
\end{frame}
\begin{frame}{Formes d'explication}
\begin{columns}
\begin{column}{0.5\textwidth}
\centering
\includegraphics[width=0.7\textwidth]{images/37_grad_LRP.png}
\onslide<3->\includegraphics[width=\textwidth]{images/SHAP-Ill.png}
\end{column}
\begin{column}{0.5\textwidth}
\onslide<2->\includegraphics[width=\textwidth]{images/Their-CF.png}
\onslide<4->
\begin{align*}
C_1 &\iff A>B \wedge C \in \mathcal{S}\\
C_2 &\iff A<B \vee C \not\in \mathcal{S}
\end{align*}
\end{column}
\end{columns}
\end{frame}
\begin{frame}{Temps de l'explicabilité}
\includegraphics[width=\textwidth]{images/VieModele-TempsXAI.drawio.pdf}
\vspace{5mm}
\pause
\begin{columns}
\begin{column}{0.5\textwidth}
Interprétabilité\\
\textit{Comment?}
\end{column}
\pause
\begin{column}{0.5\textwidth}
Explicabilité\\
\textit{Pourquoi?}
\end{column}
\end{columns}
\end{frame}
\begin{frame}{Fidélité vs Accessibilité}
\begin{block}{Des attendus contradictoires avec des conséquences}
\begin{itemize}
\item sur-simplification $\implies$ confiance indue \cite{gilpin_explaining_2019}
\item sous-simplification $\implies$ inutilisable \cite{mersha_evaluating_2025}
\end{itemize}
\end{block}
\pause
\begin{block}{Et un rapport avec le grand public compliqué}
\begin{itemize}
\item Les explications créent une surcharge d'informations \cite{bell_its_2022}
\item Créé une crainte \cite{kastner_relation_2021}
\end{itemize}
\end{block}
\end{frame}
\subsection{Taxonomie des méthodes d'explication}
\begin{frame}{Caractéristiques}
\includegraphics[width=\textwidth]{images/CaracteristiquesMethodeXAI.drawio.pdf}
\end{frame}
\begin{frame}{Stratégies}
\includegraphics[width=\textwidth]{images/StrategiesMethodeXAI.drawio.pdf}
\end{frame}
\iffalse
\begin{frame}{Méthode d'explication à base d'attention}
\begin{block}{Matrice d'attention}
Mesure l'importance de la relation entre chaque mot de l'entrée
\end{block}
\begin{columns}
\begin{column}{0.5\textwidth}
\emph{Approches}:\\
\begin{itemize}
\item Visualisation
\item Exploitation directe d'une seule matrice
\item Combinaison des matrices pour obtenir des scores
\end{itemize}
\end{column}
\begin{column}{0.5\textwidth}
\pause
\vspace{2mm}
\emph{Remarques}: \\
\begin{itemize}
\item Exploitation d'une forme de représentation de l'entrée
\item mais prise en compte partielle du modèle
\item \cite{lopardo_attention_2024} remet en question cette approche d'un point de vue théorique
\end{itemize}
\end{column}
\end{columns}
\end{frame}
\begin{frame}{Méthode d'explication par propagation}
\begin{block}{Intuition}
Propager de la sortie vers l'entrée pour retrouver l'importance de chaque partie de l'entrée sur une sortie
\end{block}
\begin{figure}
\centering
\includegraphics[width=\textwidth]{images/IllLRP.png}
\caption{Intuition de Layer-wise Relevance Propagation \cite{bento_improving_2021}}
\label{fig:IntuitionLRP}
\end{figure}
$
R^{(n)}_j = \sum_i \frac{x_j^+ w_{ji}^+}{\sum_{j'} x_{j'}^+ w_{j'i}^+} R_i^{(n+1)}
$
où $v^+ = max(0,v)$
\cite{montavon_layer-wise_2019}
\end{frame}
\begin{frame}{Méthode d'explication par perturbation}
\begin{block}{Intuition}
Observer le comportement du modèle au voisinage d'une entrée et l'impact des variations sur la prédiction.
\end{block}
\emph{Local Interpretable Model-agnostic Explanation (LIME)} : Soient $M$ un modèle et $x$ une entrée
\begin{enumerate}
\item Construire un ensemble $\mathcal{X}$ de versions perturbées de $x$
\item Entraîner un modèle interprétable $M'$ sur $\mathcal{X}\times M(\mathcal{X})$
\item L'interprétation de $M'$ est une approximation de $M$ au voisinage de $x$
\end{enumerate}
\end{frame}
\fi
\section{Évaluer les méthodes d'explications}
\subsection{Les attendus}
\begin{frame}{Les attendus}
\begin{block}{Attendus d'une explication \cite{gilpin_explaining_2019}}
\begin{itemize}
\item Fidèle au raisonnement du modèle
\item Intelligible pour le public visé
\end{itemize}
\end{block}
\pause
\begin{block}{Attendus d'une méthode d'explication \cite{mersha_evaluating_2025}}
\begin{itemize}
\item Robustesse %: Des entrées similaires ont des explications similaires.
\item Consistance %: Les explications d'une même entrée pour deux modèles différents sont aussi différentes que les modèles.
\item Contrastivité %: Les explications distinguent les différentes prédictions possibles.
\end{itemize}
\end{block}
\end{frame}
\subsection{Les approches}
\begin{frame}{Les approches}
\begin{itemize}
\item Comparer à des explications canoniques
\item Comparer les méthodes d'explication entre elles
\item Évaluer individuellement
\item Analyser théoriquement
\end{itemize}
\end{frame}
\begin{frame}{Comparer les explications à des explications canoniques}
\begin{block}{Idée}
\begin{enumerate}
\item Définir une explication attendue
\item Calculer la performance de la méthode d'explication
\end{enumerate}
%Définir une explication attendue pour chaque modèle et calculer la performance de la méthode d'explication
\end{block}
\begin{itemize}
\item La \emph{Ground Truth humaine} n'est pas adaptée
\item Construire une Ground Truth à partir du modèle utilisé ?
\item Explication \textit{idéale} ? \cite{neely_song_2022}
\end{itemize}
\end{frame}
\begin{frame}{Comparer les méthodes d'explications entre elles}
\begin{block}{Idée}
\begin{enumerate}
\item Prendre une méthode d'explication comme référence
\item Comparer ses explications avec celles de la méthode d'explication évaluée
\end{enumerate}
\end{block}
\begin{itemize}
\item \emph{Approches différentes} d'une méthode à l'autre
\item Suppose la méthode de référence \emph{idéale}
\item Quel sens? \cite{neely_song_2022}
\end{itemize}
\end{frame}
\begin{frame}{Évaluer individuellement}
\begin{block}{Idée}
Utiliser des métriques indépendantes qui évaluent les attendus
\end{block}
\begin{itemize}
\item Définir les attendus (exhausivité)
\item Trouver des métriques qui correspondent
\item Besoin de métriques \emph{universelles} ou \emph{comparables}
\end{itemize}
\end{frame}
\begin{frame}{Aparté : Cas du jugement humain}
%\com{
%qualitatif\\
%biasié vers accessibilité\\
%cout de réalistaion\\
%personens différentes d'une évaluatoin à l'autre : comparabilité?}
\begin{block}{Idée }
Demander à des personnes d'évaluer la méthode à l'aide de critères définis
\end{block}
Mais
\begin{itemize}
\item Évaluation qualitative
\item Risque de biais \cite{gilpin_explaining_2019}
\item Difficile à réaliser
\item Problèmes de comparabilité
\end{itemize}
\end{frame}
\begin{frame}{Analyse théorique des méthodes d'explication}
\begin{block}{Idée}
\begin{enumerate}
\item Reprendre les définitions mathématiques du modèle et de la méthode d'explication
\item Juger les attendus d'un point de vue théorique
\end{enumerate}
\end{block}
\begin{itemize}
\item Modèles boite-noire
\item Définir mathématiquement les attendus
\item \textit{Ex : \cite{lopardo_attention_2024}}
\end{itemize}
\end{frame}
\subsection{Retour sur les enjeux de l'évaluation des méthodes d'explication}
\begin{frame}{Existence des idéaux}
\begin{itemize}
\item Explication \emph{idéale}
\item Méthode d'explication idéale
\begin{itemize}
\item Pas de consensus \cite{garouani_investigating_2024}
\item Une méthode \emph{universelle} est-elle possible? (ex: \cite{mersha_evaluating_2025})
\end{itemize}
\end{itemize}
\end{frame}
\begin{frame}{Comparaison}
\begin{itemize}
\item Les explications sont-elles comparables? \item Les méthodes d'explication sont-elles comparables?
\item Les métriques donnent-elles des résultats comparables?
\item Quel sens?
\end{itemize}
\end{frame}
\begin{frame}{Universalité}
\begin{itemize}
\item Les méthodes d'explication sont-elles adaptées ...
\begin{itemize}
\item ... à tous les modèles?
\item ... à toutes les tâches?
\end{itemize}
\item Les métriques sont elles adaptées à tous les méthodes d'explication?
\end{itemize}
\end{frame}
\section{Conclusion}
%\begin{frame}{Autres sujets abordés dans ce stage}
% \com{peut-être à placer ailleurs/ prévoir une annexe}
% \begin{itemize}
% \item Autres méthodes d'explication
% \item Explicabilité by-design
% \begin{itemize}
% \item Modèles neuro-symboliques
% \item Modèles à base de concepts
% \end{itemize}
% \end{itemize}
%\end{frame}
\begin{frame}{Perspectives}
\begin{itemize}
\item Faire un bilan des attendus et des métriques existantes
\item Comparer les méthodes d'explications en fonction de la tâche traitée par le système
\begin{itemize}
\item Déterminer des besoins spécifiques aux tâches impliquant du texte
\item Évaluer la pertinence d'une métrique \emph{universelle}
\end{itemize}
\end{itemize}
\end{frame}
\bibliographystyle{apalike} %alpha
\bibliography{sample.bib}
\appendix
\section{Annexes}
\subsection{Architecture de BERT}
\begin{frame}{Architecture de BERT}
\begin{figure}
\centering
\begin{minipage}[c]{0.33\textwidth}
\includegraphics[height=0.85\textheight]{images/BERTArchi.png}
\end{minipage}\hfill
\begin{minipage}[c]{0.67\textwidth}
\caption{Architecture du modèle BERT}
\label{fig:BERT}
\end{minipage}
\end{figure}
\end{frame}
\subsection{Architecture de GPT}
\begin{frame}{Architecture de GPT}
\begin{figure}
\centering
\begin{minipage}[c]{0.33\textwidth}
\includegraphics[height=0.85\textheight]{images/GPT_Radford-et-al2018.png}
\end{minipage}\hfill
\begin{minipage}[c]{0.67\textwidth}
\caption{Architecture du modèle GPT \cite{radford_improving_2018}}
\label{fig:GPT}
\end{minipage}
\end{figure}
\end{frame}
\subsection{Architecture de T5}
\begin{frame}{Architecture de T5}
\begin{columns}
\begin{column}{0.4\textwidth}
\begin{figure}
\centering
\includegraphics[width=0.75\textwidth]{images/Transformer_Vaswani-et-al2027.png}
\caption{Architecture du Transformer \parbox{5em}{\cite{vaswani_attention_2017}}}
\label{fig:Transformer}
\end{figure}
\end{column}
\begin{column}{0.6\textwidth}
Différences:
\begin{itemize}
\item Normalisation sans biais
\item Connexion résiduelle après la normalisation
\item Encodding positionnel relatif
\end{itemize}
\end{column}
\end{columns}
\end{frame}
\subsection{Méthode d'explication à base d'attention}
\begin{frame}{Méthode d'explication à base d'attention}
\begin{block}{Matrice d'attention}
Mesure l'importance de la relation entre chaque mot de l'entrée
\end{block}
\begin{columns}
\begin{column}{0.5\textwidth}
\emph{Approches}:\\
\begin{itemize}
\item Visualisation
\item Exploitation directe d'une seule matrice
\item Combinaison des matrices pour obtenir des scores
\end{itemize}
\end{column}
\begin{column}{0.5\textwidth}
\pause
\vspace{2mm}
\emph{Remarques}: \\
\begin{itemize}
\item Exploitation d'une forme de représentation de l'entrée
\item mais prise en compte partielle du modèle
\item \cite{lopardo_attention_2024} remet en question cette approche d'un point de vue théorique
\end{itemize}
\end{column}
\end{columns}
\end{frame}
\subsection{Visualisation de l'attention}
\begin{frame}{Visualisation de l'attention}
\begin{figure}
\centering
\includegraphics[width=\textwidth]{images/BertViz_allscales.png}
\caption{BertViz \cite{vig_multiscale_2019} permet de visualiser le mécanisme d'attention à différentes échelles}
\label{fig:bertviz}
\end{figure}
\end{frame}
\subsection{Méthode d'explication par propagation}
\begin{frame}{Méthode d'explication par propagation}
\begin{block}{Intuition}
Propager de la sortie vers l'entrée pour retrouver l'importance de chaque partie de l'entrée sur une sortie
\end{block}
\begin{figure}
\centering
\includegraphics[width=0.9\textwidth]{images/IllLRP.png}
\caption{Intuition de Layer-wise Relevance Propagation \parbox{5em}{\cite{bento_improving_2021}}}
\label{fig:IntuitionLRP}
\end{figure}
$
R^{(n)}_j = \sum_i \frac{x_j^+ w_{ji}^+}{\sum_{j'} x_{j'}^+ w_{j'i}^+} R_i^{(n+1)}
$
où $v^+ = max(0,v)$
\parbox{5em}{\cite{montavon_layer-wise_2019}}
\end{frame}
\subsection{Méthode d'explication par perturbation}
\begin{frame}{Méthode d'explication par perturbation}
\begin{block}{Intuition}
Observer le comportement du modèle au voisinage d'une entrée et l'impact des variations sur la prédiction.
\end{block}
\emph{Local Interpretable Model-agnostic Explanation (LIME)} : Soient $M$ un modèle et $x$ une entrée
\begin{enumerate}
\item Construire un ensemble $\mathcal{X}$ de versions perturbées de $x$
\item Entraîner un modèle interprétable $M'$ sur $\mathcal{X}\times M(\mathcal{X})$
\item L'interprétation de $M'$ est une approximation de $M$ au voisinage de $x$
\end{enumerate}
\end{frame}
\subsection{Méthodes d'explication par l'exemple}
\begin{frame}{Méthodes d'explication par l'exemple}
\begin{center}
\includegraphics[width=0.6\textwidth]{images/AdversarialPerturbation_Panda.png}
\includegraphics[width=0.4\textwidth]{images/ConterfactualExample.png}
\end{center}
\end{frame}
\subsection{Modèles à base de concepts}
\begin{frame}{Modèles à base de concepts}
\begin{figure}
\centering
\includegraphics[width=0.9\textwidth]{images/C-XAI_defs_parrot.png}
\caption{Types de concepts et d'explications à base de concepts \cite{poeta_concept-based_2023}}
\label{fig:CXIA}
\end{figure}
\end{frame}
\end{document}
\ No newline at end of file
rapports_presentations/SoutenanceS9/sample.bib 0 → 100644
View file @ 69859c8a
@article{bento_improving_2021,
title = {Improving deep learning performance by using {Explainable} {Artificial} {Intelligence} ({XAI}) approaches},
volume = {1},
issn = {2731-0809},
url = {https://doi.org/10.1007/s44163-021-00008-y},
doi = {10.1007/s44163-021-00008-y},
abstract = {In this work we propose a workflow to deal with overlaid images—images with superimposed text and company logos—, which is very common in underwater monitoring videos and surveillance camera footage. It is demonstrated that it is possible to use Explaining Artificial Intelligence to improve deep learning models performance for image classification tasks in general. A deep learning model trained to classify metal surface defect, which previously had a low performance, is then evaluated with Layer-wise relevance propagation—an Explaining Artificial Intelligence technique—to identify problems in a dataset that hinder the training of deep learning models in a wide range of applications. Thereafter, it is possible to remove this unwanted information from the dataset—using different approaches: from cutting part of the images to training a Generative Inpainting neural network model—and retrain the model with the new preprocessed images. This proposed methodology improved F1 score in 20\% when compared to the original trained dataset, validating the proposed workflow.},
language = {en},
number = {1},
urldate = {2025-12-23},
journal = {Discover Artificial Intelligence},
author = {Bento, Vitor and Kohler, Manoela and Diaz, Pedro and Mendoza, Leonardo and Pacheco, Marco Aurelio},
month = oct,
year = {2021},
keywords = {Deep learning, Explain Artificial Intelligence, Layer-wise relevance propagation, XAI},
pages = {9},
}
@article{raffel_exploring_2020,
title = {Exploring the limits of transfer learning with a unified text-to-text transformer},
volume = {21},
issn = {1532-4435},
abstract = {Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts all text-based language problems into a text-to-text format. Our systematic study compares pretraining objectives, architectures, unlabeled data sets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new "Colossal Clean Crawled Corpus", we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our data set, pre-trained models, and code.},
number = {1},
journal = {J. Mach. Learn. Res.},
author = {Raffel, Colin and Shazeer, Noam and Roberts, Adam and Lee, Katherine and Narang, Sharan and Matena, Michael and Zhou, Yanqi and Li, Wei and Liu, Peter J.},
month = jan,
year = {2020},
keywords = {attention based models, deep learning, multi-task learning, natural language processing, transfer learning},
}
@misc{touvron_llama_2023,
title = {{LLaMA}: {Open} and {Efficient} {Foundation} {Language} {Models}},
shorttitle = {{LLaMA}},
url = {http://arxiv.org/abs/2302.13971},
doi = {10.48550/arXiv.2302.13971},
abstract = {We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.},
urldate = {2025-12-18},
publisher = {arXiv},
author = {Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timothée and Rozière, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
month = feb,
year = {2023},
note = {arXiv:2302.13971},
keywords = {Computer Science - Computation and Language},
}
@article{johansson_trade-off_2011,
title = {Trade-{Off} {Between} {Accuracy} and {Interpretability} for {Predictive} {In} {Silico} {Modeling}},
volume = {3},
issn = {1756-8919, 1756-8927},
url = {https://www.tandfonline.com/doi/full/10.4155/fmc.11.23},
doi = {10.4155/fmc.11.23},
language = {en},
number = {6},
urldate = {2025-12-15},
journal = {Future Medicinal Chemistry},
author = {Johansson, Ulf and Sönströd, Cecilia and Norinder, Ulf and Boström, Henrik},
month = apr,
year = {2011},
pages = {647--663},
}
@inproceedings{marzouk_tractability_2024,
series = {{ICML}'24},
title = {On the tractability of {SHAP} explanations under {Markovian} distributions},
abstract = {Thanks to its solid theoretical foundation, the SHAP framework is arguably one the most widely utilized frameworks for local explainability of ML models. Despite its popularity, its exact computation is known to be very challenging, proven to be NP-Hard in various configurations. Recent works have unveiled positive complexity results regarding the computation of the SHAP score for specific model families, encompassing decision trees, random forests, and some classes of boolean circuits. Yet, all these positive results hinge on the assumption of feature independence, often simplistic in real-world scenarios. In this article, we investigate the computational complexity of the SHAP score by relaxing this assumption and introducing a Markovian perspective. We show that, under the Markovian assumption, computing the SHAP score for the class of Weighted automata, Disjoint DNFs and Decision Trees can be performed in polynomial time, offering a first positive complexity result for the problem of SHAP score computation that transcends the limitations of the feature independence assumption},
booktitle = {Proceedings of the 41st {International} {Conference} on {Machine} {Learning}},
publisher = {JMLR.org},
author = {Marzouk, Reda and De La Higuera, Colin},
year = {2024},
note = {event-place: Vienna, Austria},
}
@inproceedings{kim_help_2023,
address = {Hamburg Germany},
title = {"{Help} {Me} {Help} the {AI}": {Understanding} {How} {Explainability} {Can} {Support} {Human}-{AI} {Interaction}},
isbn = {9781450394215},
shorttitle = {"{Help} {Me} {Help} the {AI}"},
url = {https://dl.acm.org/doi/10.1145/3544548.3581001},
doi = {10.1145/3544548.3581001},
language = {en},
urldate = {2025-12-11},
booktitle = {Proceedings of the 2023 {CHI} {Conference} on {Human} {Factors} in {Computing} {Systems}},
publisher = {ACM},
author = {Kim, Sunnie S. Y. and Watkins, Elizabeth Anne and Russakovsky, Olga and Fong, Ruth and Monroy-Hernández, Andrés},
month = apr,
year = {2023},
pages = {1--17},
}
@inproceedings{kim_interpretability_2018,
title = {Interpretability {Beyond} {Feature} {Attribution}: {Quantitative} {Testing} with {Concept} {Activation} {Vectors} ({TCAV})},
shorttitle = {Interpretability {Beyond} {Feature} {Attribution}},
url = {https://proceedings.mlr.press/v80/kim18d.html},
abstract = {The interpretation of deep learning models is a challenge due to their size, complexity, and often opaque internal state. In addition, many systems, such as image classifiers, operate on low-level features rather than high-level concepts. To address these challenges, we introduce Concept Activation Vectors (CAVs), which provide an interpretation of a neural net’s internal state in terms of human-friendly concepts. The key idea is to view the high-dimensional internal state of a neural net as an aid, not an obstacle. We show how to use CAVs as part of a technique, Testing with CAVs (TCAV), that uses directional derivatives to quantify the degree to which a user-defined concept is important to a classification result–for example, how sensitive a prediction of “zebra” is to the presence of stripes. Using the domain of image classification as a testing ground, we describe how CAVs may be used to explore hypotheses and generate insights for a standard image classification network as well as a medical application.},
language = {en},
urldate = {2025-12-11},
booktitle = {Proceedings of the 35th {International} {Conference} on {Machine} {Learning}},
publisher = {PMLR},
author = {Kim, Been and Wattenberg, Martin and Gilmer, Justin and Cai, Carrie and Wexler, James and Viegas, Fernanda and Sayres, Rory},
month = jul,
year = {2018},
pages = {2668--2677},
}
@inproceedings{poursabzi-sangdeh_manipulating_2021,
address = {Yokohama Japan},
title = {Manipulating and {Measuring} {Model} {Interpretability}},
isbn = {9781450380966},
url = {https://dl.acm.org/doi/10.1145/3411764.3445315},
doi = {10.1145/3411764.3445315},
language = {en},
urldate = {2025-12-11},
booktitle = {Proceedings of the 2021 {CHI} {Conference} on {Human} {Factors} in {Computing} {Systems}},
publisher = {ACM},
author = {Poursabzi-Sangdeh, Forough and Goldstein, Daniel G and Hofman, Jake M and Wortman Vaughan, Jennifer Wortman and Wallach, Hanna},
month = may,
year = {2021},
pages = {1--52},
}
@article{ciatto_symbolic_2024,
title = {Symbolic {Knowledge} {Extraction} and {Injection} with {Sub}-symbolic {Predictors}: {A} {Systematic} {Literature} {Review}},
volume = {56},
issn = {0360-0300, 1557-7341},
shorttitle = {Symbolic {Knowledge} {Extraction} and {Injection} with {Sub}-symbolic {Predictors}},
url = {https://dl.acm.org/doi/10.1145/3645103},
doi = {10.1145/3645103},
abstract = {In this article, we focus on the opacity issue of sub-symbolic machine learning predictors by promoting two complementary activities—
symbolic knowledge extraction
(SKE) and
symbolic knowledge injection
(SKI)—from and into sub-symbolic predictors. We consider as symbolic any language being intelligible and interpretable for both humans and computers. Accordingly, we propose general meta-models for both SKE and SKI, along with two taxonomies for the classification of SKE and SKI methods. By adopting an explainable artificial intelligence (XAI) perspective, we highlight how such methods can be exploited to mitigate the aforementioned opacity issue. Our taxonomies are attained by surveying and classifying existing methods from the literature, following a systematic approach, and by generalising the results of previous surveys targeting specific sub-topics of either SKE or SKI alone. More precisely, we analyse 132 methods for SKE and 117 methods for SKI, and we categorise them according to their purpose, operation, expected input/output data and predictor types. For each method, we also indicate the presence/lack of runnable software implementations. Our work may be of interest for data scientists aiming at selecting the most adequate SKE/SKI method for their needs, and may also work as suggestions for researchers interested in filling the gaps of the current state-of-the-art as well as for developers willing to implement SKE/SKI-based technologies.},
language = {en},
number = {6},
urldate = {2025-12-11},
journal = {ACM Computing Surveys},
author = {Ciatto, Giovanni and Sabbatini, Federico and Agiollo, Andrea and Magnini, Matteo and Omicini, Andrea},
month = jun,
year = {2024},
pages = {1--35},
}
@article{bhuyan_neuro-symbolic_2024,
title = {Neuro-symbolic artificial intelligence: a survey},
volume = {36},
issn = {1433-3058},
shorttitle = {Neuro-symbolic artificial intelligence},
url = {https://doi.org/10.1007/s00521-024-09960-z},
doi = {10.1007/s00521-024-09960-z},
abstract = {The goal of the growing discipline of neuro-symbolic artificial intelligence (AI) is to develop AI systems with more human-like reasoning capabilities by combining symbolic reasoning with connectionist learning. We survey the literature on neuro-symbolic AI during the last two decades, including books, monographs, review papers, contribution pieces, opinion articles, foundational workshops/talks, and related PhD theses. Four main features of neuro-symbolic AI are discussed, including representation, learning, reasoning, and decision-making. Finally, we discuss the many applications of neuro-symbolic AI, including question answering, robotics, computer vision, healthcare, and more. Scalability, explainability, and ethical considerations are also covered, as well as other difficulties and limits of neuro-symbolic AI. This study summarizes the current state of the art in neuro-symbolic artificial intelligence.},
language = {en},
number = {21},
urldate = {2025-12-10},
journal = {Neural Computing and Applications},
author = {Bhuyan, Bikram Pratim and Ramdane-Cherif, Amar and Tomar, Ravi and Singh, T. P.},
month = jul,
year = {2024},
keywords = {Artificial intelligence, Knowledge representation and reasoning, Machine learning, Neural networks, Neuro-symbolic artificial intelligence, Spatial-temporal data},
pages = {12809--12844},
}
@misc{ferrer_comment_2024,
title = {Comment fonctionnent les transformateurs : {Exploration} détaillée de l'architecture des transformateurs},
url = {https://www.datacamp.com/fr/tutorial/how-transformers-work},
urldate = {2024-10-04},
author = {Ferrer, Josep},
month = oct,
year = {2024},
}
@incollection{neely_song_2022,
title = {A {Song} of ({Dis})agreement: {Evaluating} the {Evaluation} of {Explainable} {Artificial} {Intelligence} in {Natural} {Language} {Processing}},
shorttitle = {A {Song} of ({Dis})agreement},
url = {https://ebooks.iospress.nl/doi/10.3233/FAIA220190},
language = {en},
urldate = {2025-12-10},
booktitle = {{HHAI2022}: {Augmenting} {Human} {Intellect}},
publisher = {IOS Press},
author = {Neely, Michael and Schouten, Stefan F. and Bleeker, Maurits and Lucic, Ana},
year = {2022},
doi = {10.3233/FAIA220190},
pages = {60--78},
}
@book{russell_intelligence_2021,
address = {Paris},
edition = {4e éd},
title = {Intelligence artificielle: une approche moderne},
isbn = {9782326002210},
shorttitle = {Intelligence artificielle},
language = {fre},
publisher = {Pearson},
author = {Russell, Stuart Jonathan and Norvig, Peter and Popineau, Fabrice and Miclet, Laurent and Cadet, Claire},
year = {2021},
}
@inproceedings{kastner_relation_2021,
title = {On the {Relation} of {Trust} and {Explainability}: {Why} to {Engineer} for {Trustworthiness}},
shorttitle = {On the {Relation} of {Trust} and {Explainability}},
url = {https://ieeexplore.ieee.org/document/9582305},
doi = {10.1109/REW53955.2021.00031},
abstract = {Recently, requirements for the explainability of software systems have gained prominence. One of the primary motivators for such requirements is that explainability is expected to facilitate stakeholders’ trust in a system. Although this seems intuitively appealing, recent psychological studies indicate that explanations do not necessarily facilitate trust. Thus, explainability requirements might not be suitable for promoting trust.One way to accommodate this finding is, we suggest, to focus on trustworthiness instead of trust. While these two may come apart, we ideally want both: a trustworthy system and the stakeholder’s trust. In this paper, we argue that even though trustworthiness does not automatically lead to trust, there are several reasons to engineer primarily for trustworthiness – and that a system’s explainability can crucially contribute to its trustworthiness.},
urldate = {2025-12-08},
booktitle = {2021 {IEEE} 29th {International} {Requirements} {Engineering} {Conference} {Workshops} ({REW})},
author = {Kästner, Lena and Langer, Markus and Lazar, Veronika and Schomäcker, Astrid and Speith, Timo and Sterz, Sarah},
month = sep,
year = {2021},
keywords = {Conferences, Explainability, NFR, Psychology, Requirements, Requirements engineering, Software systems, Stakeholders, Trust, Trustworthiness, XAI},
pages = {169--175},
}
@inproceedings{deters_x_2024,
address = {Uppsala Sweden},
title = {The {X} {Factor}: {On} the {Relationship} between {User} {eXperience} and {eXplainability}},
isbn = {9798400709661},
shorttitle = {The {X} {Factor}},
url = {https://dl.acm.org/doi/10.1145/3679318.3685352},
doi = {10.1145/3679318.3685352},
language = {en},
urldate = {2025-12-08},
booktitle = {Nordic {Conference} on {Human}-{Computer} {Interaction}},
publisher = {ACM},
author = {Deters, Hannah and Droste, Jakob and Hess, Anne and Klös, Verena and Schneider, Kurt and Speith, Timo and Vogelsang, Andreas},
month = oct,
year = {2024},
pages = {1--12},
}
@article{lin_survey_2022,
title = {A survey of transformers},
volume = {3},
issn = {2666-6510},
url = {https://www.sciencedirect.com/science/article/pii/S2666651022000146},
doi = {10.1016/j.aiopen.2022.10.001},
abstract = {Transformers have achieved great success in many artificial intelligence fields, such as natural language processing, computer vision, and audio processing. Therefore, it is natural to attract lots of interest from academic and industry researchers. Up to the present, a great variety of Transformer variants (a.k.a. X-formers) have been proposed, however, a systematic and comprehensive literature review on these Transformer variants is still missing. In this survey, we provide a comprehensive review of various X-formers. We first briefly introduce the vanilla Transformer and then propose a new taxonomy of X-formers. Next, we introduce the various X-formers from three perspectives: architectural modification, pre-training, and applications. Finally, we outline some potential directions for future research.},
urldate = {2025-12-08},
journal = {AI Open},
author = {Lin, Tianyang and Wang, Yuxin and Liu, Xiangyang and Qiu, Xipeng},
month = jan,
year = {2022},
keywords = {Deep learning, Pre-trained models, Self-attention, Transformer},
pages = {111--132},
}
@inproceedings{delobelle_robbert_2020,
address = {Online},
title = {{RobBERT}: a {Dutch} {RoBERTa}-based {Language} {Model}},
shorttitle = {{RobBERT}},
url = {https://www.aclweb.org/anthology/2020.findings-emnlp.292},
doi = {10.18653/v1/2020.findings-emnlp.292},
language = {en},
urldate = {2025-12-08},
booktitle = {Findings of the {Association} for {Computational} {Linguistics}: {EMNLP} 2020},
publisher = {Association for Computational Linguistics},
author = {Delobelle, Pieter and Winters, Thomas and Berendt, Bettina},
year = {2020},
pages = {3255--3265},
}
@inproceedings{aftan_survey_2023,
title = {A {Survey} on {BERT} and {Its} {Applications}},
url = {https://ieeexplore.ieee.org/document/10092289/references#references},
doi = {10.1109/LT58159.2023.10092289},
abstract = {A recently developed language representation model named Bidirectional Encoder Representation from Transformers (BERT) is based on an advanced trained deep learning approach that has achieved excellent results in many complex tasks, the same as classification, Natural Language Processing (NLP), prediction, etc. This survey paper mainly adopts the summary of BERT, its multiple types, and its latest developments and applications in various computer science and engineering fields. Furthermore, it puts forward BERT's problems and attractive future research trends in a different area with multiple datasets. From the findings, overall, the BERT and their recent types have achieved more accurate, fast, and optimal results in solving most complex problems than typical Machine and Deep Learning methods.},
urldate = {2025-12-08},
booktitle = {2023 20th {Learning} and {Technology} {Conference} ({L}\&{T})},
author = {Aftan, Sulaiman and Shah, Habib},
month = jan,
year = {2023},
keywords = {BERT, Bit error rate, Computer science, Deep learning, Machine Learning, Natural Language Processing model, Predictive models, Text analysis, Text mining, Transformers, bidirectional encoder},
pages = {161--166},
}
@misc{wahab_dibert_2021,
title = {{DIBERT}: {Dependency} {Injected} {Bidirectional} {Encoder} {Representations} from {Transformers}},
copyright = {https://creativecommons.org/licenses/by/4.0/},
shorttitle = {{DIBERT}},
url = {https://www.techrxiv.org/doi/full/10.36227/techrxiv.16444611.v2},
doi = {10.36227/techrxiv.16444611.v2},
abstract = {{\textless}div{\textgreater}
{\textless}div{\textgreater}
{\textless}div{\textgreater}
{\textless}p{\textgreater}
{\textless}/p{\textgreater}{\textless}div{\textgreater}
{\textless}div{\textgreater}
{\textless}div{\textgreater}
{\textless}p{\textgreater}In this paper, we propose a new model named DIBERT
which stands for Dependency Injected Bidirectional Encoder
Representations from Transformers. DIBERT is a variation of
the BERT and has an additional third objective called Parent
Prediction (PP) apart from Masked Language Modeling (MLM)
and Next Sentence Prediction (NSP). PP injects the syntactic
structure of a dependency tree while pre-training the DIBERT
which generates syntax-aware generic representations. We use
the WikiText-103 benchmark dataset to pre-train both BERT-
Base and DIBERT. After fine-tuning, we observe that DIBERT
performs better than BERT-Base on various downstream tasks
including Semantic Similarity, Natural Language Inference and
Sentiment Analysis. {\textless}/p{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}},
urldate = {2025-12-08},
author = {Wahab, Abdul and Sifa, Rafet},
month = oct,
year = {2021},
}
@article{lee_biobert_2020,
title = {{BioBERT}: a pre-trained biomedical language representation model for biomedical text mining},
volume = {36},
copyright = {http://creativecommons.org/licenses/by/4.0/},
issn = {1367-4803, 1367-4811},
shorttitle = {{BioBERT}},
url = {https://academic.oup.com/bioinformatics/article/36/4/1234/5566506},
doi = {10.1093/bioinformatics/btz682},
abstract = {Abstract
Motivation
Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora.
Results
We introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62\% F1 score improvement), biomedical relation extraction (2.80\% F1 score improvement) and biomedical question answering (12.24\% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts.
Availability and implementation
We make the pre-trained weights of BioBERT freely available at https://github.com/naver/biobert-pretrained, and the source code for fine-tuning BioBERT available at https://github.com/dmis-lab/biobert.},
language = {en},
number = {4},
urldate = {2025-12-08},
journal = {Bioinformatics},
author = {Lee, Jinhyuk and Yoon, Wonjin and Kim, Sungdong and Kim, Donghyeon and Kim, Sunkyu and So, Chan Ho and Kang, Jaewoo},
editor = {Wren, Jonathan},
month = feb,
year = {2020},
pages = {1234--1240},
}
@misc{sanh_distilbert_2020,
title = {{DistilBERT}, a distilled version of {BERT}: smaller, faster, cheaper and lighter},
shorttitle = {{DistilBERT}, a distilled version of {BERT}},
url = {http://arxiv.org/abs/1910.01108},
doi = {10.48550/arXiv.1910.01108},
abstract = {As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging. In this work, we propose a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can then be fine-tuned with good performances on a wide range of tasks like its larger counterparts. While most prior work investigated the use of distillation for building task-specific models, we leverage knowledge distillation during the pre-training phase and show that it is possible to reduce the size of a BERT model by 40\%, while retaining 97\% of its language understanding capabilities and being 60\% faster. To leverage the inductive biases learned by larger models during pre-training, we introduce a triple loss combining language modeling, distillation and cosine-distance losses. Our smaller, faster and lighter model is cheaper to pre-train and we demonstrate its capabilities for on-device computations in a proof-of-concept experiment and a comparative on-device study.},
urldate = {2025-12-08},
publisher = {arXiv},
author = {Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},
month = mar,
year = {2020},
note = {arXiv:1910.01108},
keywords = {Computer Science - Computation and Language},
}
@inproceedings{martin_camembert_2020,
address = {Online},
title = {{CamemBERT}: a {Tasty} {French} {Language} {Model}},
shorttitle = {{CamemBERT}},
url = {https://www.aclweb.org/anthology/2020.acl-main.645},
doi = {10.18653/v1/2020.acl-main.645},
language = {en},
urldate = {2025-12-08},
booktitle = {Proceedings of the 58th {Annual} {Meeting} of the {Association} for {Computational} {Linguistics}},
publisher = {Association for Computational Linguistics},
author = {Martin, Louis and Muller, Benjamin and Ortiz Suárez, Pedro Javier and Dupont, Yoann and Romary, Laurent and De La Clergerie, Éric and Seddah, Djamé and Sagot, Benoît},
year = {2020},
pages = {7203--7219},
}
@inproceedings{douka_juribert_2021,
address = {Punta Cana, Dominican Republic},
title = {{JuriBERT}: {A} {Masked}-{Language} {Model} {Adaptation} for {French} {Legal} {Text}},
shorttitle = {{JuriBERT}},
url = {https://aclanthology.org/2021.nllp-1.9/},
doi = {10.18653/v1/2021.nllp-1.9},
abstract = {Language models have proven to be very useful when adapted to specific domains. Nonetheless, little research has been done on the adaptation of domain-specific BERT models in the French language. In this paper, we focus on creating a language model adapted to French legal text with the goal of helping law professionals. We conclude that some specific tasks do not benefit from generic language models pre-trained on large amounts of data. We explore the use of smaller architectures in domain-specific sub-languages and their benefits for French legal text. We prove that domain-specific pre-trained models can perform better than their equivalent generalised ones in the legal domain. Finally, we release JuriBERT, a new set of BERT models adapted to the French legal domain.},
urldate = {2025-12-08},
booktitle = {Proceedings of the {Natural} {Legal} {Language} {Processing} {Workshop} 2021},
publisher = {Association for Computational Linguistics},
author = {Douka, Stella and Abdine, Hadi and Vazirgiannis, Michalis and El Hamdani, Rajaa and Restrepo Amariles, David},
editor = {Aletras, Nikolaos and Androutsopoulos, Ion and Barrett, Leslie and Goanta, Catalina and Preotiuc-Pietro, Daniel},
month = nov,
year = {2021},
pages = {95--101},
}
@inproceedings{segonne_jargon_2024,
title = {Jargon: {A} {Suite} of {Language} {Models} and {Evaluation} {Tasks} for {French} {Specialized} {Domains}},
shorttitle = {Jargon},
url = {https://hal.science/hal-04535557},
abstract = {Pretrained Language Models (PLMs) are the de facto backbone of most state-of-the-art NLP systems. In this paper, we introduce a family of domain-specific pretrained PLMs for French, focusing on three important domains: transcribed speech, medicine, and law. We use a transformer architecture based on efficient methods (LinFormer) to maximise their utility, since these domains often involve processing long documents. We evaluate and compare our models to state-of-the-art models on a diverse set of tasks and datasets, some of which are introduced in this paper. We gather the datasets into a new French-language evaluation benchmark for these three domains. We also compare various training configurations: continued pretraining, pretraining from scratch, as well as single- and multi-domain pretraining. Extensive domain-specific experiments show that it is possible to attain competitive downstream performance even when pre-training with the approximative LinFormer attention mechanism. For full reproducibility, we release the models and pretraining data, as well as contributed datasets.},
language = {en},
urldate = {2025-12-08},
author = {Segonne, Vincent and Mannion, Aidan and Canul, Laura Cristina Alonzo and Audibert, Alexandre and Liu, Xingyu and Macaire, Cécile and Pupier, Adrien and Zhou, Yongxin and Aguiar, Mathilde and Herron, Felix and Norré, Magali and Amini, Massih-Reza and Bouillon, Pierrette and Eshkol-Taravella, Iris and Esperança-Rodier, Emmanuelle and François, Thomas and Goeuriot, Lorraine and Goulian, Jérôme and Lafourcade, Mathieu and Lecouteux, Benjamin and Portet, François and Ringeval, Fabien and Vandeghinste, Vincent and Coavoux, Maximin and Dinarelli, Marco and Schwab, Didier},
month = may,
year = {2024},
pages = {9463},
}
@misc{antoun_arabert_2021,
title = {{AraBERT}: {Transformer}-based {Model} for {Arabic} {Language} {Understanding}},
shorttitle = {{AraBERT}},
url = {http://arxiv.org/abs/2003.00104},
doi = {10.48550/arXiv.2003.00104},
abstract = {The Arabic language is a morphologically rich language with relatively few resources and a less explored syntax compared to English. Given these limitations, Arabic Natural Language Processing (NLP) tasks like Sentiment Analysis (SA), Named Entity Recognition (NER), and Question Answering (QA), have proven to be very challenging to tackle. Recently, with the surge of transformers based models, language-specific BERT based models have proven to be very efficient at language understanding, provided they are pre-trained on a very large corpus. Such models were able to set new standards and achieve state-of-the-art results for most NLP tasks. In this paper, we pre-trained BERT specifically for the Arabic language in the pursuit of achieving the same success that BERT did for the English language. The performance of AraBERT is compared to multilingual BERT from Google and other state-of-the-art approaches. The results showed that the newly developed AraBERT achieved state-of-the-art performance on most tested Arabic NLP tasks. The pretrained araBERT models are publicly available on https://github.com/aub-mind/arabert hoping to encourage research and applications for Arabic NLP.},
urldate = {2025-12-08},
publisher = {arXiv},
author = {Antoun, Wissam and Baly, Fady and Hajj, Hazem},
month = mar,
year = {2021},
note = {arXiv:2003.00104},
keywords = {Computer Science - Computation and Language},
}
@inproceedings{vig_multiscale_2019,
address = {Florence, Italy},
title = {A {Multiscale} {Visualization} of {Attention} in the {Transformer} {Model}},
url = {https://aclanthology.org/P19-3007/},
doi = {10.18653/v1/P19-3007},
abstract = {The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention-based approach. Besides improving performance, an advantage of using attention is that it can also help to interpret a model by showing how the model assigns weight to different input elements. However, the multi-layer, multi-head attention mechanism in the Transformer model can be difficult to decipher. To make the model more accessible, we introduce an open-source tool that visualizes attention at multiple scales, each of which provides a unique perspective on the attention mechanism. We demonstrate the tool on BERT and OpenAI GPT-2 and present three example use cases: detecting model bias, locating relevant attention heads, and linking neurons to model behavior.},
urldate = {2025-12-04},
booktitle = {Proceedings of the 57th {Annual} {Meeting} of the {Association} for {Computational} {Linguistics}: {System} {Demonstrations}},
publisher = {Association for Computational Linguistics},
author = {Vig, Jesse},
editor = {Costa-jussà, Marta R. and Alfonseca, Enrique},
month = jul,
year = {2019},
pages = {37--42},
}
@article{kotipalli_role_2024,
title = {The {Role} of {Attention} {Mechanisms} in {Enhancing} {Transparency} and {Interpretability} of {Neural} {Network} {Models} in {Explainable} {AI}},
url = {https://digitalcommons.harrisburgu.edu/cgi/viewcontent.cgi?params=/context/dandt/article/1000/&path_info=Bhargav_Report_Final.pdf},
abstract = {In the rapidly evolving field of artificial intelligence (AI), deep learning models' interpretability and reliability are severely hindered by their complexity and opacity. Enhancing the transparency and interpretability of AI systems for humans is the primary objective of the emerging field of explainable AI (XAI). The attention mechanisms at the heart of XAI's work are based on human cognitive processes. Neural networks can now dynamically focus on relevant parts of the input data thanks to these mechanisms, which enhances interpretability and performance. This report covers in-depth talks of attention mechanisms in neural networks within XAI, as well as an analysis of the theoretical foundations, architectural applications, and empirical evidence showing how well they work to improve model transparency. The report provides a comprehensive analysis of the role of attention mechanisms in AI models to address ethical concerns, comply with regulatory requirements, and foster a deeper understanding and trust in AI systems. The report contributes to the discussion about bringing AI closer to human values and cognitive processes so that its advancements are impactful and responsible by conducting a thorough analysis.},
language = {en},
author = {Kotipalli, Bhargav},
month = apr,
year = {2024},
}
@inproceedings{shrikumar_not_2017,
address = {Sydney, Australia},
title = {Not {Just} a {Black} {Box}: {Learning} {Important} {Features} {Through} {Propagating} {Activation} {Differences}},
volume = {70},
shorttitle = {Not {Just} a {Black} {Box}},
url = {http://arxiv.org/abs/1605.01713},
doi = {10.48550/arXiv.1605.01713},
abstract = {Note: This paper describes an older version of DeepLIFT. See https://arxiv.org/abs/1704.02685 for the newer version. Original abstract follows: The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Learning Important FeaTures), an efficient and effective method for computing importance scores in a neural network. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. We apply DeepLIFT to models trained on natural images and genomic data, and show significant advantages over gradient-based methods.},
urldate = {2025-12-04},
publisher = {Proceedings of the 34th
International Conference on Machine Learning},
author = {Shrikumar, Avanti and Greenside, Peyton and Shcherbina, Anna and Kundaje, Anshul},
month = apr,
year = {2017},
note = {arXiv:1605.01713},
keywords = {Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Neural and Evolutionary Computing},
pages = {3145 -- 3153},
}
@inproceedings{selvaraju_grad-cam_2017,
title = {Grad-{CAM}: {Visual} {Explanations} from {Deep} {Networks} via {Gradient}-{Based} {Localization}},
shorttitle = {Grad-{CAM}},
url = {https://ieeexplore.ieee.org/document/8237336},
doi = {10.1109/ICCV.2017.74},
abstract = {We propose a technique for producing `visual explanations' for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent. Our approach - Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say logits for `dog' or even a caption), flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad- CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g. VGG), (2) CNNs used for structured outputs (e.g. captioning), (3) CNNs used in tasks with multi-modal inputs (e.g. visual question answering) or reinforcement learning, without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are more faithful to the underlying model, and (d) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show even non-attention based models can localize inputs. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a `stronger' deep network from a `weaker' one even when both make identical predictions. Our code is available at https: //github.com/ramprs/grad-cam/ along with a demo on CloudCV [2] and video at youtu.be/COjUB9Izk6E.},
urldate = {2025-12-04},
booktitle = {2017 {IEEE} {International} {Conference} on {Computer} {Vision} ({ICCV})},
author = {Selvaraju, Ramprasaath R. and Cogswell, Michael and Das, Abhishek and Vedantam, Ramakrishna and Parikh, Devi and Batra, Dhruv},
month = oct,
year = {2017},
note = {ISSN: 2380-7504},
keywords = {Cats, Computer architecture, Dogs, Knowledge discovery, Visualization},
pages = {618--626},
}
@inproceedings{radford_improving_2018,
title = {Improving {Language} {Understanding} by {Generative} {Pre}-{Training}},
url = {https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035},
abstract = {Natural language understanding comprises a wide range of diverse tasks such as textual entailment, question answering, semantic similarity assessment, and document classification. Although large unlabeled text corpora are abundant, labeled data for learning these specific tasks is scarce, making it challenging for discriminatively trained models to perform adequately. We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task. In contrast to previous approaches, we make use of task-aware input transformations during fine-tuning to achieve effective transfer while requiring minimal changes to the model architecture. We demonstrate the effectiveness of our approach on a wide range of benchmarks for natural language understanding. Our general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, significantly improving upon the state of the art in 9 out of the 12 tasks studied. For instance, we achieve absolute improvements of 8.9\% on commonsense reasoning (Stories Cloze Test), 5.7\% on question answering (RACE), and 1.5\% on textual entailment (MultiNLI).},
urldate = {2025-12-04},
author = {Radford, Alec and Narasimhan, Karthik and Salimans, Tim and Sutskever, Ilya},
year = {2018},
}
@inproceedings{kalouli_gkr_2018,
address = {New Orleans, Louisiana},
title = {{GKR}: the {Graphical} {Knowledge} {Representation} for semantic parsing},
shorttitle = {{GKR}},
url = {https://aclanthology.org/W18-1304/},
doi = {10.18653/v1/W18-1304},
abstract = {This paper describes the first version of an open-source semantic parser that creates graphical representations of sentences to be used for further semantic processing, e.g. for natural language inference, reasoning and semantic similarity. The Graphical Knowledge Representation which is output by the parser is inspired by the Abstract Knowledge Representation, which separates out conceptual and contextual levels of representation that deal respectively with the subject matter of a sentence and its existential commitments. Our representation is a layered graph with each sub-graph holding different kinds of information, including one sub-graph for concepts and one for contexts. Our first evaluation of the system shows an F-score of 85\% in accurately representing sentences as semantic graphs.},
urldate = {2025-11-26},
booktitle = {Proceedings of the {Workshop} on {Computational} {Semantics} beyond {Events} and {Roles}},
publisher = {Association for Computational Linguistics},
author = {Kalouli, Aikaterini-Lida and Crouch, Richard},
editor = {Blanco, Eduardo and Morante, Roser},
month = jun,
year = {2018},
pages = {27--37},
}
@misc{vaswani_attention_2017,
title = {Attention {Is} {All} {You} {Need}},
url = {http://arxiv.org/abs/1706.03762},
doi = {10.48550/arXiv.1706.03762},
abstract = {The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N. and Kaiser, Lukasz and Polosukhin, Illia},
month = dec,
year = {2017},
note = {arXiv:1706.03762},
keywords = {Computer Science - Computation and Language, Computer Science - Machine Learning},
}
@book{netter_regards_2019,
title = {Regards sur le nouveau droit des données personnelles},
url = {https://hal.science/hal-02357967},
abstract = {Just over 40 years after the entry into force in France of the "informatique et libertés" law, the law on personal data seems to have progressed dramatically. In 2018, the new European regulation (known as the " GDPR ") came into force. A law and then an ordinance adapted French domestic law accordingly. The first part of 2019 was marked by the rise of the sanctions imposed by the CNIL.
Some of the contributions collected in this book address cross-cutting issues related to the new regulation, such as its territorial scope, competition from the American model, the scope of the notion of data controller, the existence of post-mortem rights over the data, or the limits of the principle of transparency in the face of the opacity of predictive algorithms. Others focus on a particular sector, whether it is banking, health, insurance or data on public officials.},
urldate = {2025-11-24},
author = {Netter, Emmanuel and Ndior, Valère and Puyraimond, Jean-Ferdinand and Vergnolle, Suzanne},
editor = {d'Amiens, Centre de droit privé et de sciences criminelles},
month = nov,
year = {2019},
keywords = {algorithmes prédictifs, données de santé, données français des données personnelles, droit américain des données, droit au déréférencement, droit bancaire, droit des assurances, droit européen des données personnelles, extraterritorialité du droit, principe de transparence, souveraineté numérique, transparence de la vie publique},
}
@misc{lundberg_unified_2017,
title = {A {Unified} {Approach} to {Interpreting} {Model} {Predictions}},
url = {http://arxiv.org/abs/1705.07874},
doi = {10.48550/arXiv.1705.07874},
abstract = {Understanding why a model makes a certain prediction can be as crucial as the prediction's accuracy in many applications. However, the highest accuracy for large modern datasets is often achieved by complex models that even experts struggle to interpret, such as ensemble or deep learning models, creating a tension between accuracy and interpretability. In response, various methods have recently been proposed to help users interpret the predictions of complex models, but it is often unclear how these methods are related and when one method is preferable over another. To address this problem, we present a unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations). SHAP assigns each feature an importance value for a particular prediction. Its novel components include: (1) the identification of a new class of additive feature importance measures, and (2) theoretical results showing there is a unique solution in this class with a set of desirable properties. The new class unifies six existing methods, notable because several recent methods in the class lack the proposed desirable properties. Based on insights from this unification, we present new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.},
urldate = {2025-11-19},
publisher = {arXiv},
author = {Lundberg, Scott and Lee, Su-In},
month = nov,
year = {2017},
note = {arXiv:1705.07874},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning},
}
@misc{ribeiro_model-agnostic_2016,
title = {Model-{Agnostic} {Interpretability} of {Machine} {Learning}},
url = {http://arxiv.org/abs/1606.05386},
doi = {10.48550/arXiv.1606.05386},
abstract = {Understanding why machine learning models behave the way they do empowers both system designers and end-users in many ways: in model selection, feature engineering, in order to trust and act upon the predictions, and in more intuitive user interfaces. Thus, interpretability has become a vital concern in machine learning, and work in the area of interpretable models has found renewed interest. In some applications, such models are as accurate as non-interpretable ones, and thus are preferred for their transparency. Even when they are not accurate, they may still be preferred when interpretability is of paramount importance. However, restricting machine learning to interpretable models is often a severe limitation. In this paper we argue for explaining machine learning predictions using model-agnostic approaches. By treating the machine learning models as black-box functions, these approaches provide crucial flexibility in the choice of models, explanations, and representations, improving debugging, comparison, and interfaces for a variety of users and models. We also outline the main challenges for such methods, and review a recently-introduced model-agnostic explanation approach (LIME) that addresses these challenges.},
urldate = {2025-11-19},
publisher = {arXiv},
author = {Ribeiro, Marco Tulio and Singh, Sameer and Guestrin, Carlos},
month = jun,
year = {2016},
note = {arXiv:1606.05386},
keywords = {Computer Science - Machine Learning, Statistics - Machine Learning},
}
@article{rudin_stop_2019,
title = {Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead},
volume = {1},
copyright = {2019 Springer Nature Limited},
issn = {2522-5839},
url = {https://www.nature.com/articles/s42256-019-0048-x},
doi = {10.1038/s42256-019-0048-x},
abstract = {Black box machine learning models are currently being used for high-stakes decision making throughout society, causing problems in healthcare, criminal justice and other domains. Some people hope that creating methods for explaining these black box models will alleviate some of the problems, but trying to explain black box models, rather than creating models that are interpretable in the first place, is likely to perpetuate bad practice and can potentially cause great harm to society. The way forward is to design models that are inherently interpretable. This Perspective clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications where interpretable models could potentially replace black box models in criminal justice, healthcare and computer vision.},
language = {en},
number = {5},
urldate = {2025-11-14},
journal = {Nature Machine Intelligence},
author = {Rudin, Cynthia},
month = may,
year = {2019},
keywords = {Computer science, Criminology, Science, Statistics, technology and society},
pages = {206--215},
}
@article{montavon_explaining_2017,
title = {Explaining nonlinear classification decisions with deep {Taylor} decomposition},
volume = {65},
issn = {0031-3203},
url = {https://www.sciencedirect.com/science/article/pii/S0031320316303582},
doi = {10.1016/j.patcog.2016.11.008},
abstract = {Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems such as image recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transparency, limiting the interpretability of the solution and thus the scope of application in practice. Especially DNNs act as black boxes due to their multilayer nonlinear structure. In this paper we introduce a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements. Although our focus is on image classification, the method is applicable to a broad set of input data, learning tasks and network architectures. Our method called deep Taylor decomposition efficiently utilizes the structure of the network by backpropagating the explanations from the output to the input layer. We evaluate the proposed method empirically on the MNIST and ILSVRC data sets.},
urldate = {2025-11-07},
journal = {Pattern Recognition},
author = {Montavon, Grégoire and Lapuschkin, Sebastian and Binder, Alexander and Samek, Wojciech and Müller, Klaus-Robert},
month = may,
year = {2017},
keywords = {Deep neural networks, Heatmapping, Image recognition, Relevance propagation, Taylor decomposition},
pages = {211--222},
}
@book{molnar_interpretable_2025,
title = {Interpretable {Machine} {Learning}},
url = {https://christophm.github.io/interpretable-ml-book/},
urldate = {2025-09-11},
author = {Molnar, Christoph},
month = mar,
year = {2025},
}
@misc{abnar_quantifying_2020,
title = {Quantifying {Attention} {Flow} in {Transformers}},
url = {http://arxiv.org/abs/2005.00928},
doi = {10.48550/arXiv.2005.00928},
abstract = {In the Transformer model, "self-attention" combines information from attended embeddings into the representation of the focal embedding in the next layer. Thus, across layers of the Transformer, information originating from different tokens gets increasingly mixed. This makes attention weights unreliable as explanations probes. In this paper, we consider the problem of quantifying this flow of information through self-attention. We propose two methods for approximating the attention to input tokens given attention weights, attention rollout and attention flow, as post hoc methods when we use attention weights as the relative relevance of the input tokens. We show that these methods give complementary views on the flow of information, and compared to raw attention, both yield higher correlations with importance scores of input tokens obtained using an ablation method and input gradients.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Abnar, Samira and Zuidema, Willem},
month = may,
year = {2020},
note = {arXiv:2005.00928},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning},
}
@phdthesis{elguendouze_explainable_2024,
type = {thesis},
title = {Explainable {Artificial} {Intelligence} approaches for {Image} {Captioning}},
url = {https://theses.fr/2024ORLE1003},
abstract = {L'évolution rapide des modèles de sous-titrage d'images, impulsée par l'intégration de techniques d'apprentissage profond combinant les modalités image et texte, a conduit à des systèmes de plus en plus complexes. Cependant, ces modèles fonctionnent souvent comme des boîtes noires, incapables de fournir des explications transparentes de leurs décisions. Cette thèse aborde l'explicabilité des systèmes de sous-titrage d'images basés sur des architectures Encodeur-Attention-Décodeur, et ce à travers quatre aspects. Premièrement, elle explore le concept d'espace latent, s'éloignant ainsi des approches traditionnelles basées sur l'espace de représentation originel. Deuxièmement, elle présente la notion de caractère décisif, conduisant à la formulation d'une nouvelle définition pour le concept d'influence/décisivité des composants dans le contexte de sous-titrage d'images explicable, ainsi qu'une approche par perturbation pour la capture du caractère décisif. Le troisième aspect vise à élucider les facteurs influençant la qualité des explications, en mettant l'accent sur la portée des méthodes d'explication. En conséquence, des variantes basées sur l'espace latent de méthodes d'explication bien établies telles que LRP et LIME ont été développées, ainsi que la proposition d'une approche d'évaluation centrée sur l'espace latent, connue sous le nom d'Ablation Latente. Le quatrième aspect de ce travail consiste à examiner ce que nous appelons la saillance et la représentation de certains concepts visuels, tels que la quantité d'objets, à différents niveaux de l'architecture de sous-titrage.},
language = {fr},
urldate = {2025-10-24},
school = {Orléans},
author = {Elguendouze, Sofiane},
month = nov,
year = {2024},
}
@misc{lopardo_attention_2024,
title = {Attention {Meets} {Post}-hoc {Interpretability}: {A} {Mathematical} {Perspective}},
shorttitle = {Attention {Meets} {Post}-hoc {Interpretability}},
url = {http://arxiv.org/abs/2402.03485},
doi = {10.48550/arXiv.2402.03485},
abstract = {Attention-based architectures, in particular transformers, are at the heart of a technological revolution. Interestingly, in addition to helping obtain state-of-the-art results on a wide range of applications, the attention mechanism intrinsically provides meaningful insights on the internal behavior of the model. Can these insights be used as explanations? Debate rages on. In this paper, we mathematically study a simple attention-based architecture and pinpoint the differences between post-hoc and attention-based explanations. We show that they provide quite different results, and that, despite their limitations, post-hoc methods are capable of capturing more useful insights than merely examining the attention weights.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Lopardo, Gianluigi and Precioso, Frederic and Garreau, Damien},
month = jun,
year = {2024},
note = {arXiv:2402.03485},
keywords = {Computer Science - Computation and Language, Computer Science - Machine Learning, Statistics - Machine Learning},
}
@inproceedings{sippy_data_2020,
title = {Data {Staining}: {A} {Method} for {Comparing} {Faithfulness} of {Explainers}},
url = {https://aiweb.cs.washington.edu/ai/pubs/sippy-icml20.pdf},
author = {Sippy, Jacob D and Bansal, Gagan and Weld, Daniel},
year = {2020},
}
@misc{poeta_concept-based_2023,
title = {Concept-based {Explainable} {Artificial} {Intelligence}: {A} {Survey}},
shorttitle = {Concept-based {Explainable} {Artificial} {Intelligence}},
url = {http://arxiv.org/abs/2312.12936},
doi = {10.48550/arXiv.2312.12936},
abstract = {The field of explainable artificial intelligence emerged in response to the growing need for more transparent and reliable models. However, using raw features to provide explanations has been disputed in several works lately, advocating for more user-understandable explanations. To address this issue, a wide range of papers proposing Concept-based eXplainable Artificial Intelligence (C-XAI) methods have arisen in recent years. Nevertheless, a unified categorization and precise field definition are still missing. This paper fills the gap by offering a thorough review of C-XAI approaches. We define and identify different concepts and explanation types. We provide a taxonomy identifying nine categories and propose guidelines for selecting a suitable category based on the development context. Additionally, we report common evaluation strategies including metrics, human evaluations and dataset employed, aiming to assist the development of future methods. We believe this survey will serve researchers, practitioners, and domain experts in comprehending and advancing this innovative field.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Poeta, Eleonora and Ciravegna, Gabriele and Pastor, Eliana and Cerquitelli, Tania and Baralis, Elena},
month = dec,
year = {2023},
note = {arXiv:2312.12936},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Human-Computer Interaction},
}
@misc{dosovitskiy_image_2021,
title = {An {Image} is {Worth} 16x16 {Words}: {Transformers} for {Image} {Recognition} at {Scale}},
shorttitle = {An {Image} is {Worth} 16x16 {Words}},
url = {http://arxiv.org/abs/2010.11929},
doi = {10.48550/arXiv.2010.11929},
abstract = {While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
month = jun,
year = {2021},
note = {arXiv:2010.11929},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning},
}
@misc{deyoung_eraser_2020,
title = {{ERASER}: {A} {Benchmark} to {Evaluate} {Rationalized} {NLP} {Models}},
shorttitle = {{ERASER}},
url = {http://arxiv.org/abs/1911.03429},
doi = {10.48550/arXiv.1911.03429},
abstract = {State-of-the-art models in NLP are now predominantly based on deep neural networks that are opaque in terms of how they come to make predictions. This limitation has increased interest in designing more interpretable deep models for NLP that reveal the `reasoning' behind model outputs. But work in this direction has been conducted on different datasets and tasks with correspondingly unique aims and metrics; this makes it difficult to track progress. We propose the Evaluating Rationales And Simple English Reasoning (ERASER) benchmark to advance research on interpretable models in NLP. This benchmark comprises multiple datasets and tasks for which human annotations of "rationales" (supporting evidence) have been collected. We propose several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are (i.e., the degree to which provided rationales influenced the corresponding predictions). Our hope is that releasing this benchmark facilitates progress on designing more interpretable NLP systems. The benchmark, code, and documentation are available at https://www.eraserbenchmark.com/},
urldate = {2025-10-24},
publisher = {arXiv},
author = {DeYoung, Jay and Jain, Sarthak and Rajani, Nazneen Fatema and Lehman, Eric and Xiong, Caiming and Socher, Richard and Wallace, Byron C.},
month = apr,
year = {2020},
note = {arXiv:1911.03429},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning},
}
@misc{russakovsky_imagenet_2015,
title = {{ImageNet} {Large} {Scale} {Visual} {Recognition} {Challenge}},
url = {http://arxiv.org/abs/1409.0575},
doi = {10.48550/arXiv.1409.0575},
abstract = {The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the challenges of collecting large-scale ground truth annotation, highlight key breakthroughs in categorical object recognition, provide a detailed analysis of the current state of the field of large-scale image classification and object detection, and compare the state-of-the-art computer vision accuracy with human accuracy. We conclude with lessons learned in the five years of the challenge, and propose future directions and improvements.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Russakovsky, Olga and Deng, Jia and Su, Hao and Krause, Jonathan and Satheesh, Sanjeev and Ma, Sean and Huang, Zhiheng and Karpathy, Andrej and Khosla, Aditya and Bernstein, Michael and Berg, Alexander C. and Fei-Fei, Li},
month = jan,
year = {2015},
note = {arXiv:1409.0575},
keywords = {Computer Science - Computer Vision and Pattern Recognition},
}
@misc{devlin_bert_2019,
title = {{BERT}: {Pre}-training of {Deep} {Bidirectional} {Transformers} for {Language} {Understanding}},
shorttitle = {{BERT}},
url = {http://arxiv.org/abs/1810.04805},
doi = {10.48550/arXiv.1810.04805},
abstract = {We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5\% (7.7\% point absolute improvement), MultiNLI accuracy to 86.7\% (4.6\% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
month = may,
year = {2019},
note = {arXiv:1810.04805},
keywords = {Computer Science - Computation and Language},
}
@misc{chefer_transformer_2021,
title = {Transformer {Interpretability} {Beyond} {Attention} {Visualization}},
url = {http://arxiv.org/abs/2012.09838},
doi = {10.48550/arXiv.2012.09838},
abstract = {Self-attention techniques, and specifically Transformers, are dominating the field of text processing and are becoming increasingly popular in computer vision classification tasks. In order to visualize the parts of the image that led to a certain classification, existing methods either rely on the obtained attention maps or employ heuristic propagation along the attention graph. In this work, we propose a novel way to compute relevancy for Transformer networks. The method assigns local relevance based on the Deep Taylor Decomposition principle and then propagates these relevancy scores through the layers. This propagation involves attention layers and skip connections, which challenge existing methods. Our solution is based on a specific formulation that is shown to maintain the total relevancy across layers. We benchmark our method on very recent visual Transformer networks, as well as on a text classification problem, and demonstrate a clear advantage over the existing explainability methods.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Chefer, Hila and Gur, Shir and Wolf, Lior},
month = apr,
year = {2021},
note = {arXiv:2012.09838},
keywords = {Computer Science - Computer Vision and Pattern Recognition},
}
@inproceedings{kalouli_xplainli_2020,
address = {Barcelona, Spain (Online)},
title = {{XplaiNLI}: {Explainable} {Natural} {Language} {Inference} through {Visual} {Analytics}},
shorttitle = {{XplaiNLI}},
url = {https://www.aclweb.org/anthology/2020.coling-demos.9},
doi = {10.18653/v1/2020.coling-demos.9},
language = {en},
urldate = {2025-10-24},
booktitle = {Proceedings of the 28th {International} {Conference} on {Computational} {Linguistics}: {System} {Demonstrations}},
publisher = {International Committee on Computational Linguistics (ICCL)},
author = {Kalouli, Aikaterini-Lida and Sevastjanova, Rita and De Paiva, Valeria and Crouch, Richard and El-Assady, Mennatallah},
year = {2020},
pages = {48--52},
}
@inproceedings{kalouli_hy-nli_2020,
address = {Barcelona, Spain (Online)},
title = {Hy-{NLI}: a {Hybrid} system for {Natural} {Language} {Inference}},
shorttitle = {Hy-{NLI}},
url = {https://www.aclweb.org/anthology/2020.coling-main.459},
doi = {10.18653/v1/2020.coling-main.459},
language = {en},
urldate = {2025-10-24},
booktitle = {Proceedings of the 28th {International} {Conference} on {Computational} {Linguistics}},
publisher = {International Committee on Computational Linguistics},
author = {Kalouli, Aikaterini-Lida and Crouch, Richard and De Paiva, Valeria},
year = {2020},
pages = {5235--5249},
}
@book{samek_explainable_2019,
address = {Cham},
series = {Lecture {Notes} in {Computer} {Science}},
title = {Explainable {AI}: {Interpreting}, {Explaining} and {Visualizing} {Deep} {Learning}},
volume = {11700},
copyright = {http://www.springer.com/tdm},
isbn = {9783030289539 9783030289546},
shorttitle = {Explainable {AI}},
url = {http://link.springer.com/10.1007/978-3-030-28954-6},
language = {en},
urldate = {2025-09-17},
publisher = {Springer International Publishing},
editor = {Samek, Wojciech and Montavon, Grégoire and Vedaldi, Andrea and Hansen, Lars Kai and Müller, Klaus-Robert},
year = {2019},
doi = {10.1007/978-3-030-28954-6},
}
@incollection{ancona_gradient-based_2019,
address = {Cham},
title = {Gradient-{Based} {Attribution} {Methods}},
isbn = {9783030289546},
url = {https://doi.org/10.1007/978-3-030-28954-6_9},
abstract = {The problem of explaining complex machine learning models, including Deep Neural Networks, has gained increasing attention over the last few years. While several methods have been proposed to explain network predictions, the definition itself of explanation is still debated. Moreover, only a few attempts to compare explanation methods from a theoretical perspective has been done. In this chapter, we discuss the theoretical properties of several attribution methods and show how they share the same idea of using the gradient information as a descriptive factor for the functioning of a model. Finally, we discuss the strengths and limitations of these methods and compare them with available alternatives.},
language = {en},
urldate = {2025-09-17},
booktitle = {Explainable {AI}: {Interpreting}, {Explaining} and {Visualizing} {Deep} {Learning}},
publisher = {Springer International Publishing},
author = {Ancona, Marco and Ceolini, Enea and Öztireli, Cengiz and Gross, Markus},
editor = {Samek, Wojciech and Montavon, Grégoire and Vedaldi, Andrea and Hansen, Lars Kai and Müller, Klaus-Robert},
year = {2019},
doi = {10.1007/978-3-030-28954-6_9},
keywords = {Attribution methods, Deep Neural Networks, Explainable artificial intelligence},
pages = {169--191},
}
@incollection{montavon_layer-wise_2019,
address = {Cham},
title = {Layer-{Wise} {Relevance} {Propagation}: {An} {Overview}},
isbn = {9783030289546},
shorttitle = {Layer-{Wise} {Relevance} {Propagation}},
url = {https://doi.org/10.1007/978-3-030-28954-6_10},
abstract = {For a machine learning model to generalize well, one needs to ensure that its decisions are supported by meaningful patterns in the input data. A prerequisite is however for the model to be able to explain itself, e.g. by highlighting which input features it uses to support its prediction. Layer-wise Relevance Propagation (LRP) is a technique that brings such explainability and scales to potentially highly complex deep neural networks. It operates by propagating the prediction backward in the neural network, using a set of purposely designed propagation rules. In this chapter, we give a concise introduction to LRP with a discussion of (1) how to implement propagation rules easily and efficiently, (2) how the propagation procedure can be theoretically justified as a ‘deep Taylor decomposition’, (3) how to choose the propagation rules at each layer to deliver high explanation quality, and (4) how LRP can be extended to handle a variety of machine learning scenarios beyond deep neural networks.},
language = {en},
urldate = {2025-09-17},
booktitle = {Explainable {AI}: {Interpreting}, {Explaining} and {Visualizing} {Deep} {Learning}},
publisher = {Springer International Publishing},
author = {Montavon, Grégoire and Binder, Alexander and Lapuschkin, Sebastian and Samek, Wojciech and Müller, Klaus-Robert},
editor = {Samek, Wojciech and Montavon, Grégoire and Vedaldi, Andrea and Hansen, Lars Kai and Müller, Klaus-Robert},
year = {2019},
doi = {10.1007/978-3-030-28954-6_10},
keywords = {Deep Neural Networks, Deep Taylor Decomposition, Explanations, Layer-wise Relevance Propagation},
pages = {193--209},
}
@misc{huang_annotated_2022,
title = {The {Annotated} {Transformer}},
url = {https://nlp.seas.harvard.edu/annotated-transformer/},
urldate = {2025-09-12},
author = {Huang, Austin and Subramanian, Suraj and Sum, Jonathan and Almubarak, Khalid and Biderman, Stella},
year = {2022},
}
@inproceedings{bell_its_2022,
address = {Seoul Republic of Korea},
title = {It’s {Just} {Not} {That} {Simple}: {An} {Empirical} {Study} of the {Accuracy}-{Explainability} {Trade}-off in {Machine} {Learning} for {Public} {Policy}},
isbn = {9781450393522},
shorttitle = {It’s {Just} {Not} {That} {Simple}},
url = {https://dl.acm.org/doi/10.1145/3531146.3533090},
doi = {10.1145/3531146.3533090},
language = {en},
urldate = {2025-09-11},
booktitle = {2022 {ACM} {Conference} on {Fairness} {Accountability} and {Transparency}},
publisher = {ACM},
author = {Bell, Andrew and Solano-Kamaiko, Ian and Nov, Oded and Stoyanovich, Julia},
month = jun,
year = {2022},
pages = {248--266},
}
@misc{gilpin_explaining_2019,
title = {Explaining {Explanations}: {An} {Overview} of {Interpretability} of {Machine} {Learning}},
shorttitle = {Explaining {Explanations}},
url = {http://arxiv.org/abs/1806.00069},
doi = {10.48550/arXiv.1806.00069},
abstract = {There has recently been a surge of work in explanatory artificial intelligence (XAI). This research area tackles the important problem that complex machines and algorithms often cannot provide insights into their behavior and thought processes. XAI allows users and parts of the internal system to be more transparent, providing explanations of their decisions in some level of detail. These explanations are important to ensure algorithmic fairness, identify potential bias/problems in the training data, and to ensure that the algorithms perform as expected. However, explanations produced by these systems is neither standardized nor systematically assessed. In an effort to create best practices and identify open challenges, we provide our definition of explainability and show how it can be used to classify existing literature. We discuss why current approaches to explanatory methods especially for deep neural networks are insufficient. Finally, based on our survey, we conclude with suggested future research directions for explanatory artificial intelligence.},
urldate = {2025-09-10},
publisher = {arXiv},
author = {Gilpin, Leilani H. and Bau, David and Yuan, Ben Z. and Bajwa, Ayesha and Specter, Michael and Kagal, Lalana},
month = feb,
year = {2019},
note = {arXiv:1806.00069 [cs]},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning},
}
@misc{morgan_explainable_2023,
title = {Explainable {AI}: {Visualizing} {Attention} in {Transformers}},
shorttitle = {Explainable {AI}},
url = {https://www.comet.com/site/blog/explainable-ai-for-transformers/},
abstract = {Learn how to visualize the attention of transformers and log your results to Comet, as we work towards explainability in AI.},
language = {en-US},
urldate = {2025-08-21},
journal = {Comet},
author = {Morgan, Abby},
month = jul,
year = {2023},
keywords = {outil BERT},
}
@inproceedings{garouani_investigating_2024,
title = {Investigating the {Duality} of {Interpretability} and {Explainability} in {Machine} {Learning}},
url = {http://arxiv.org/abs/2503.21356},
doi = {10.1109/ICTAI62512.2024.00125},
abstract = {The rapid evolution of machine learning (ML) has led to the widespread adoption of complex "black box" models, such as deep neural networks and ensemble methods. These models exhibit exceptional predictive performance, making them invaluable for critical decision-making across diverse domains within society. However, their inherently opaque nature raises concerns about transparency and interpretability, making them untrustworthy decision support systems. To alleviate such a barrier to high-stakes adoption, research community focus has been on developing methods to explain black box models as a means to address the challenges they pose. Efforts are focused on explaining these models instead of developing ones that are inherently interpretable. Designing inherently interpretable models from the outset, however, can pave the path towards responsible and beneficial applications in the field of ML. In this position paper, we clarify the chasm between explaining black boxes and adopting inherently interpretable models. We emphasize the imperative need for model interpretability and, following the purpose of attaining better (i.e., more effective or efficient w.r.t. predictive performance) and trustworthy predictors, provide an experimental evaluation of latest hybrid learning methods that integrates symbolic knowledge into neural network predictors. We demonstrate how interpretable hybrid models could potentially supplant black box ones in different domains.},
urldate = {2025-08-21},
booktitle = {2024 {IEEE} 36th {International} {Conference} on {Tools} with {Artificial} {Intelligence} ({ICTAI})},
author = {Garouani, Moncef and Mothe, Josiane and Barhrhouj, Ayah and Aligon, Julien},
month = oct,
year = {2024},
note = {arXiv:2503.21356 [cs]},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning},
pages = {861--867},
}
@article{huang_explainable_2024,
title = {From explainable to interpretable deep learning for natural language processing in healthcare: {How} far from reality?},
volume = {24},
issn = {2001-0370},
shorttitle = {From explainable to interpretable deep learning for natural language processing in healthcare},
url = {https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11126530/},
doi = {10.1016/j.csbj.2024.05.004},
abstract = {Deep learning (DL) has substantially enhanced natural language processing (NLP) in healthcare research. However, the increasing complexity of DL-based NLP necessitates transparent model interpretability, or at least explainability, for reliable decision-making. This work presents a thorough scoping review of explainable and interpretable DL in healthcare NLP. The term “eXplainable and Interpretable Artificial Intelligence” (XIAI) is introduced to distinguish XAI from IAI. Different models are further categorized based on their functionality (model-, input-, output-based) and scope (local, global). Our analysis shows that attention mechanisms are the most prevalent emerging IAI technique. The use of IAI is growing, distinguishing it from XAI. The major challenges identified are that most XIAI does not explore “global” modelling processes, the lack of best practices, and the lack of systematic evaluation and benchmarks. One important opportunity is to use attention mechanisms to enhance multi-modal XIAI for personalized medicine. Additionally, combining DL with causal logic holds promise. Our discussion encourages the integration of XIAI in Large Language Models (LLMs) and domain-specific smaller models. In conclusion, XIAI adoption in healthcare requires dedicated in-house expertise. Collaboration with domain experts, end-users, and policymakers can lead to ready-to-use XIAI methods across NLP and medical tasks. While challenges exist, XIAI techniques offer a valuable foundation for interpretable NLP algorithms in healthcare.},
urldate = {2025-08-21},
journal = {Computational and Structural Biotechnology Journal},
author = {Huang, Guangming and Li, Yingya and Jameel, Shoaib and Long, Yunfei and Papanastasiou, Giorgos},
month = may,
year = {2024},
pmid = {38800693},
pmcid = {PMC11126530},
pages = {362--373},
}
@misc{mohammadi_explainability_2025,
title = {Explainability in {Practice}: {A} {Survey} of {Explainable} {NLP} {Across} {Various} {Domains}},
shorttitle = {Explainability in {Practice}},
url = {http://arxiv.org/abs/2502.00837},
doi = {10.48550/arXiv.2502.00837},
abstract = {Natural Language Processing (NLP) has become a cornerstone in many critical sectors, including healthcare, finance, and customer relationship management. This is especially true with the development and use of advanced models such as GPT-based architectures and BERT, which are widely used in decision-making processes. However, the black-box nature of these advanced NLP models has created an urgent need for transparency and explainability. This review explores explainable NLP (XNLP) with a focus on its practical deployment and real-world applications, examining its implementation and the challenges faced in domain-specific contexts. The paper underscores the importance of explainability in NLP and provides a comprehensive perspective on how XNLP can be designed to meet the unique demands of various sectors, from healthcare's need for clear insights to finance's emphasis on fraud detection and risk assessment. Additionally, this review aims to bridge the knowledge gap in XNLP literature by offering a domain-specific exploration and discussing underrepresented areas such as real-world applicability, metric evaluation, and the role of human interaction in model assessment. The paper concludes by suggesting future research directions that could enhance the understanding and broader application of XNLP.},
urldate = {2025-08-21},
publisher = {arXiv},
author = {Mohammadi, Hadi and Bagheri, Ayoub and Giachanou, Anastasia and Oberski, Daniel L.},
month = feb,
year = {2025},
note = {arXiv:2502.00837 [cs]
version: 1},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language},
}
@article{mersha_evaluating_2025,
title = {Evaluating the {Effectiveness} of {XAI} {Techniques} for {Encoder}-{Based} {Language} {Models}},
volume = {310},
issn = {09507051},
url = {http://arxiv.org/abs/2501.15374},
doi = {10.1016/j.knosys.2025.113042},
abstract = {The black-box nature of large language models (LLMs) necessitates the development of eXplainable AI (XAI) techniques for transparency and trustworthiness. However, evaluating these techniques remains a challenge. This study presents a general evaluation framework using four key metrics: Human-reasoning Agreement (HA), Robustness, Consistency, and Contrastivity. We assess the effectiveness of six explainability techniques from five different XAI categories model simplification (LIME), perturbation-based methods (SHAP), gradient-based approaches (InputXGradient, Grad-CAM), Layer-wise Relevance Propagation (LRP), and attention mechanisms-based explainability methods (Attention Mechanism Visualization, AMV) across five encoder-based language models: TinyBERT, BERTbase, BERTlarge, XLM-R large, and DeBERTa-xlarge, using the IMDB Movie Reviews and Tweet Sentiment Extraction (TSE) datasets. Our findings show that the model simplification-based XAI method (LIME) consistently outperforms across multiple metrics and models, significantly excelling in HA with a score of 0.9685 on DeBERTa-xlarge, robustness, and consistency as the complexity of large language models increases. AMV demonstrates the best Robustness, with scores as low as 0.0020. It also excels in Consistency, achieving near-perfect scores of 0.9999 across all models. Regarding Contrastivity, LRP performs the best, particularly on more complex models, with scores up to 0.9371.},
urldate = {2025-08-21},
journal = {Knowledge-Based Systems},
author = {Mersha, Melkamu Abay and Yigezu, Mesay Gemeda and Kalita, Jugal},
month = feb,
year = {2025},
note = {arXiv:2501.15374 [cs]},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Computers and Society, Computer Science - Machine Learning},
pages = {113042},
}
rapports_presentations/rapportS9/.gitignore 0 → 100644
View file @ 69859c8a
# tex
*.acn
*.acr
*.alg
*.aux
*.bak
*.bbl
*.bcf
*.blg
*.brf
*.bst
*.dvi
*.fdb_latexmk
*.fls
*.glg
*.glo
*.gls
*.idx
*.ilg
*.ind
*.ist
*.lof
*.log
*.lol
*.lot
*.maf
*.mtc
*.mtc1
*.nav
*.nlo
*.nls
*.out
*.pdf
*.pyg
*.run.xml
*.snm
*.synctex.gz
*.tex.backup
*.tex~
*.thm
*.toc
*.vrb
*.xdy
*.xml
*blx.bib
.bak
.mtc
build/
# vscode
.vscode/
\ No newline at end of file
rapports_presentations/rapportS9/rapport/IAAct.bib 0 → 100644
View file @ 69859c8a
@misc{iaact,
title = {Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance)},
shorttitle = {IAAct},
number = {1689},
year = {2024},
month = {Jul},
url = {http://data.europa.eu/eli/reg/2024/1689/oj}
}
\ No newline at end of file
rapports_presentations/rapportS9/rapport/rapport.tex 0 → 100644
View file @ 69859c8a
\documentclass{report}
% Language setting
% Replace `english' with e.g. `spanish' to change the document language
\usepackage[french]{babel}
% Set page size and margins
% Replace `letterpaper' with`a4paper' for UK/EU standard size
\usepackage[a4paper,top=2cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}
% Useful packages
\usepackage{amsmath}
\usepackage{bbold}
\usepackage{graphicx}
\usepackage[colorlinks=true, allcolors=blue]{hyperref}
\usepackage{breakcites}
\usepackage{color}
\newcommand{\com}[1]{\textcolor{olive}{#1}}
\newenvironment{comb}
{\begin{color}{olive}
}
{
\end{color}
}
\usepackage{tcolorbox}
\newcommand{\alh}[1]{\textcolor{teal}{#1}}
\title{Exploration de l'explicabilité de l'intelligence artificielle et de son application au traitement automatique de la langue naturelle} % une tentative à une heure tardive
\author{Marine Delvallez}
\begin{document}
\thispagestyle{empty}
\setcounter{page}{0}
\begin{center}
\includegraphics[scale=0.4]{images/Logo Minerve_RVB.jpg}
\hspace*{\stretch{1}}
\includegraphics[scale=0.2]{images/France_2030_Logo_rouge_bleu_transparent.png}
\hspace*{\stretch{1}}
\includegraphics[scale=0.5]{images/LIFO.png}
\end{center}
\vfill
\begin{center}
\rule{0.5\textwidth}{1pt}\\
\vspace{0.5 \baselineskip}
\huge \textbf{Projet individuel recherche Semestre 9 \newline M2 GPEx Minerve} \\
\Large \textbf{Exploration de l'explicabilité de l'intelligence artificielle \newline et de son application au traitement automatique de la langue naturelle}\\
\normalsize
\rule{0.5\textwidth}{1pt}\\
par \large Marine DELVALLEZ \normalsize \\
\vspace{2cm}
\begin{tabular}{llll}
\emph{Jury} : & Thi-Bich-Hanh DAO &\emph{Tutrice de stage} : & Anaïs HALFTERMEYER\\
& Anaïs HALFTERMEYER & & \\
& Marie HENAULT
\end{tabular}
\vfill
2 Septembre - 7 Janvier\\
Année universitaire 2025-2026
\end{center}
\tableofcontents
\chapter{Introduction}
Le domaine de l'intelligence artificielle est en fort développement. De nouvelles méthodes d'apprentissage automatique sont régulièrement publiées et la performance de ces nouveaux modèles ne cesse de croître. L'intelligence artificielle cherche à faire adopter à une machine un comportement, une façon de répondre à un ensemble d'informations, qui soit rationnel et/ou au plus proche du comportement humain. Nous nous intéressons ici à l'apprentissage automatique (machine learning) et à l'apprentissage automatique profond (deep learning) qui sont des sous domaines de l'intelligence artificielle. \cite[Chap~19]{russell_intelligence_2021} défini l'apprentissage automatique comme le domaine de l'intelligence artificielle qui "construit un modèles à partir des données" en les observant. Les modèles sont à la fois des "hypothèses sur le monde" mais aussi des fonctions ou "procédures logicielles capables de résoudre certains problèmes".
Parmi ces problèmes, on retrouve les problèmes de classification comme la classification de spams ou la détection d'objets dans une image et les problèmes de régression comme la prédiction de prix d'un bien immobilier.
%\alh{donner ici une transition un peu bateau donnant les exemples de tâche ou au moins les grandes classes de problèmes : classification, regression, partitionnement. Ou plus pragmatique : classification de spams, détection d'objet dans des images, ...}
%L'augmentation de la performance des modèles en machine learning s'accompagne pour une partie d'entre eux d'une opacification de leur fonctionnement.
Une façon intuitive d'automatiser la classification des spams est de définir des règles basées sur le contenu du mail comme la présence des mots "gratuit" et/ou "cliquez" ou l'adresse de l'expéditeur. Ces fonctionnements sont transparents pour l'humain : les règles sont intelligibles. Cependant, ces méthodes transparentes sont limitées et certains modèles et méthodes tels que les \emph{Random Forest} ou les réseaux de neurones sont plus avancées. Cette évolution qui apporte une augmentation des performances de ces systèmes engendre aussi une opacification de ces derniers qui perdent en transparence :
%\alh{c'est un peu violent : un truc du genre: la première intuition pour automatiser la classification de spam par exemple pourrait consister en l'édition de règles de repérage de mots tels que "gratuit", ou encore "cliquez", ce qui rend le système totalement transparent sur son fonctionnement puisque la règle appliquée est compréhensible pour un humain. Un tel système est transparent mais offre des capacités limitées, l'avènement de méthodes avancées en ML...}
il devient de plus en plus dur de déterminer ce qui, lors du traitement d'une entrée, justifie le fait qu'un modèle aboutisse à la sortie obtenue. Ces modèles opaques sont souvent appelés \textbf{modèles boite-noire}. Cette perte de transparence des modèles pose problème pour certains usages. Dans les domaines de la santé ou de la défense, suivre la sortie obtenue par le modèle peut directement impacter la vie d'individus. Il n'est donc pas envisageable de faire confiance à un modèle dont on ignore le fonctionnement des processus de décision. De façon plus générale, le besoin de transparence a été pointé d'un point de vue juridique par la RGPD puis l'IAAct \cite{netter_regards_2019, iaact}. Tout personne subissant une décision (refus de dossier, sélection de profil,...) a droit à une justification de cette décision. Ainsi, toute personne décisionnaire utilisant des outils à base d'intelligence artificielle dans son processus de décision doit être en capacité de justifier les éléments fournis par l'outil dans ce cadre. %\alh{ce qui a permis de faire émerger un champs d'étude complet se préoccupant de la production de méthodes d'explicabilité dont l'objectif principal est de ...}
Cela a permis de faire émerger un champ d'étude complet se préoccupant de la production de méthodes d'explicabilité dont l'objectif principal est d'exhiber les raisons faisant qu'un système obtient une certaine sortie.
Dans ce domaine, on appelle \textbf{explication} toute formulation intelligible et fidèle du fonctionnement d'un modèle d'intelligence artificielle.
Lorsqu'on s'intéresse aux modèles de langue et notamment aux modèles à base de Transformer \cite{vaswani_attention_2017, devlin_bert_2019, radford_improving_2018}, on retrouve les mêmes phénomènes.
%\alh{plutôt : notamment concernant les modèles à base transformer...}
Ces nouveaux modèles sont de plus en plus puissants mais aussi de plus en plus opaques. La question de la transparence des processus de décision mis en \oe uvre au sein de ces modèles et systèmes se pose aussi. Les méthodes d'explicabilité standard
%\alh{"standard", "connues", ou encore "désormais classiques"}
s'appliquent bien dans ce contexte mais certaines sont en difficulté du fait des spécificités des technologies utilisées. Ainsi un besoin de méthodes spécifiques au modèles de langue a aussi émergé.
\chapter{Généralités}
Dans ce chapitre, on introduit le vocabulaire utilisé ainsi que les différents concepts centraux de l'intelligence artificielle, ses outils et son explicabilité.
\section{Intelligence artificielle et modèles}
Lorsqu'on construit un modèle d'intelligence artificielle, on cherche à assimiler les notions qui sous-tendent un problème. L'objectif est de construire un système capable d'effectuer une certaine tâche le plus rationnellement et/ou humainement possible.
Un modèle peut être vu de différentes manières. Lors de premières approches, on considère souvent un modèle comme une machine qui fourni une sortie ou qui a un comportement qui dépend des informations qui lui sont données. On peut se représenter cela comme une fonction qui à une entrée (\textbf{input}) associe une sortie (\textbf{output}). On parle alors de \textbf{système}. Dans ce cas, on s'intéresse aux domaines d'entrée (\textbf{environnement}) et de sortie associés au modèle ainsi que le comportement du système attendu (\textbf{spécification}). De façon plus générale, ces trois informations définissent la \textbf{tâche} attribuée au système. Par exemple, le dataset ImageNet \cite{russakovsky_imagenet_2015} propose des images étiquetées par l'animal qu'elles contiennent. Dans ce cas, l'ensemble des images d'animaux forme l'environnement, le domaine des sorties possibles est l'ensemble des étiquettes. C'est une tâche de classification.
Une autre façon de considérer les modèles est de les voir comme des \textbf{représentations} d'un environnement. Dans ce cas, on ne s'intéresse plus aux entrées et sorties mais aux \textbf{paramètres} contenus dans le module et à ce qu'ils représentent. Ainsi, l'entraînement du modèle, qui n'était jusqu'ici qu'un protocole de construction d'une fonction, devient une procédure de capture d'un environnement. Avec cette approche du modèle en IA, la prédiction d'une sortie en fonction d'une entrée est considérée comme une \textbf{tâche tierce}. Le rôle principal d'un modèle vu ainsi est de \emph{représenter son environnement}. À partir de cette représentation, on peut extraire des informations qui permettent d'effectuer la tâche tierce. Pour cela, on ajoute parfois un composant supplémentaire à la sortie du modèle (souvent un petit réseau de neurone) pour obtenir un système capable d'exploiter la représentation de l'entrée proposée par le modèle. La Figure~\ref{fig:modele_systeme} illustre cette nuance.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{./images/PB_ModeleTacheTiers.drawio.jpg}
\caption{Illustration de la distinction entre modèle et système}
\label{fig:modele_systeme}
\end{figure}
Après son entraînement, lorsqu'on utilise un système ou un modèle, il \textbf{infère} la sortie à l'aide des données et de la représentation qu'il a apprise. Sa \textbf{prédiction} ne reflète pas nécessairement la réalité.
\section{Explicabilité}
\subsection{Définition d'explication}
\label{ssec:def_expl}
Lorsqu'on s'intéresse à l'explicabilité de l'intelligence artificielle, l'une des premières problémati\-ques rencontrée est la définition d'explication. Selon son utilisation, les attendus seront différents. Il en va de même pour le public auquel elle s'adresse (technicien, spécialiste ou grand public): sa forme dépend de la personne à qui elle est adressée.
En explicabilité pour l'intelligence artificielle, on cherche à \emph{rendre humainement compréhensible les raisons pour lesquelles un modèle effectue une certaine prédiction} \cite{garouani_investigating_2024, bell_its_2022}. On estime la compréhension pour un humain par sa capacité à anticiper ou justifier la prédiction \cite{bell_its_2022}. Ainsi, une explication correspond à \emph{tout support ou expression rendant compte le plus fidèlement possible des processus de décision d'un modèle dans une forme la plus humainement intelligible possible}.
Il ne faut pas confondre explicabilité et interprétabilité de l'intelligence artificielle. Un modèle est \textbf{interprétable} si il est possible pour un humain de comprendre les mécanismes de décision mis en \oe uvre par le modèle lui-même \cite{garouani_investigating_2024, gilpin_explaining_2019}. Du fait de cette contrainte de transparence du raisonnement du modèle, certains auteurs n'incluent dans cette catégorie que les arbres de décision, les modèles linéaires et ceux à base de listes de règles \cite{bell_its_2022}. Ainsi l'interprétabilité peut être vue comme le \emph{secteur du l'apprentissage automatique qui construit des modèles intrinsèquement humainement interprétables} \cite{rudin_stop_2019}. Là où l'explicabilité cherche \emph{pourquoi} le modèle produit une sortie ou une autre, l'interprétabilité va chercher \emph{comment} cette sortie est obtenue et construire des modèles pour lesquels il est possible de répondre à cette question. Par exemple, une interprétation pour un classifieur de spams fonctionnant avec des règles est de dire qu'on sait que les mails contenant les mots "gratuit" et "cliquez" seront classés comme spams. On fourni une verbalisation de la règle contenue dans le modèle. À contrario, une explication pour cet exemple peut être de dire que le mail est classé spam car la majorité des mails contenant à la fois les mots "gratuit" et "cliquez" ont aussi étés classés de cette façon par le système. Par conséquent, un modèle interprétable sera considéré comme explicable par la plupart des auteurs là où les modèles non interprétables peuvent aussi être expliqués.
Au delà de la nuance entre les causes d'un phénomène et les mécanismes qui l'animent, les exigences qu'il est possible d'avoir envers les explications peuvent être très variables. Dans le cas de la classification d'images, \cite{chefer_transformer_2021} propose de considérer la capacité d'une explication à pointer l'élément correspondant à la classe dans l'image (une "carte de pertinence" met en évidence le chien sur l'image étiquetée par "chien"). Ce format d'explication est illustré dans la Figure~\ref{fig:carteinfluence} Si le fait de reconnaître et être capable de désigner l'objet important dans une image être un moyen pertinent dans le processus de décision, il peut aussi être intéressant de prendre en compte le contexte dans lequel se trouve l'objet. Par exemple, \cite{elguendouze_explainable_2024} met en évidence se type de comportement dans le contexte du sous-titrage d'images. Par exemple, si les images d'éléphant de la base de données sont toujours des images d'éléphant dans un milieux aride, le modèle peut considérer le milieu aride comme un indicateur en faveur de la présence d'un éléphant. \\
Un autre format courant d'explication est d'attribuer une indication de l'importance de chaque composant de l'entrée. Elle prend souvent la forme d'un coefficient réel.
\begin{figure}
\parbox{7cm}{%
\includegraphics[width=0.3\textwidth]{images/37_orig.png}
}
\qquad
\begin{minipage}{7cm}%
\includegraphics[width=0.64\textwidth]{images/37_grad_LRP.png}
\end{minipage}%
\caption{Image et explication de la classification "Chien" sous forme de "carte d'influence"}
\label{fig:carteinfluence}
\end{figure}
\subsection{Les temps de l'explicabilité}
Il existe différents temps au cours de la vie d'un modèle d'apprentissage automatique durant lesquels ont peut chercher à expliquer son fonctionnement. Au cours de la construction d'un modèle et de son entraînement, on peut chercher à le rendre plus transparent en travaillant sur les données, l'architecture du modèle ou la fonction d'apprentissage par exemple. On parle d'\textbf{explicabilité by-design}. Une fois un modèle prêt à être utilisé, avant ou au cours de son déploiement, on peut chercher à expliquer son fonctionnement en observant son comportement sur différentes entrées ou en essayant d'analyser les paramètres du modèle. On appelle cela l'\textbf{explicabilité post-hoc}. La Figure \ref{fig:VieModeleTempsXAI} illustre ces différents temps dans la vie d'un modèle.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{./images/VieModele-TempsXAI.png}
\caption{Temps pour l'explicabilité dans la vie d'un modèle}
\label{fig:VieModeleTempsXAI}
\end{figure}
\chapter{Des modèles}
Le secteur de l'apprentissage profond possède une grande diversité de modèles allant des simples réseaux de neurones profonds (DNN) jusqu'à des architectures plus complexes telles que les Long-Short-Term Memory (LSTM) en passant par des modèles plus spécialisés (CNN). Dans ce chapitre, on se concentre sur une autre famille d'architectures : le modèle Transformer et ceux qui s'en inspirent.
\section{Le modèle Transformer}
Le modèle Transformer est introduit pour la première fois par \cite{vaswani_attention_2017}. Comme on peut le constater dans la Figure~\ref{fig:transformer}, cette architecture n'utilise pas les outils utilisés à cette période tels que les RNN et LSTM mais repose exclusivement sur le \emph{mécanisme d'attention} (voir sous-section \ref{ssec:mecattention}) qui est l'élément central de cette architecture conséquente. \\
Par la suite, plusieurs modèles se sont basés sur cette architecture faisant du mécanisme d'attention et du Transformer des éléments centraux du secteur de l'apprentissage profond \cite{lin_survey_2022}.
\begin{figure}
\centering
\includegraphics[width=0.5\textwidth]{images/Transformer_Vaswani-et-al2027.png}
\caption{Illustration du modèle Transformer proposé par \cite{vaswani_attention_2017}}
\label{fig:transformer}
\end{figure}
\subsection{Architecture du modèle Transformer}
Le modèle Transformer suit la structure \emph{encodeur-décodeur}. La partie encodeur capte les informations importantes de l'entrée et la couche décodeur déduit et construit la sortie. L'encodeur et le décodeur ont chacun une couche propre qui est répétée $N$ fois.
Une couche d'encodeur commence par un \emph{mécanisme d'auto-attention multi-tête} (voir sous-section~\ref{ssec:mecattention}). C'est le module "Multi-Head Attention" sur la Figure~\ref{fig:transformer}. Il est suivi d'une \emph{connexion résiduelle} dont le résultat est \emph{normalisé}. Une \textbf{connexion résiduelle} correspond à l'ajout de l'entrée d'un module (ici le mécanisme d'attention multi-tête) à sa sortie. La connexion résiduelle et la normalisation sont matérialisées par le module "Add\&Norm" dans la Figure~\ref{fig:transformer}. Le résultat obtenu passe alors dans un \emph{réseau de neurones} composé de deux couches ("Feed Forward" sur la Figure~\ref{fig:transformer}) puis on applique une \emph{connexion résiduelle} avant une nouvelle \emph{normalisation}. Ce processus est répété $N$ fois.
Une couche de décodeur se compose d'un \emph{mécanisme d'attention multi-tête masqué}, d'une \emph{connexion résiduelle} puis d'une \emph{normalisation}. Un \emph{mécanisme d'auto-attention multi-tête} prend en suite en entrée ce résultat ainsi que la sortie des $N$ couches de l'encodeur. On applique une \emph{connexion résiduelle} sur la sortie puis le tout est \emph{normalisé} avant de passer dans un \emph{réseau de neurones} de deux couches, de subir une nouvelle \emph{connexion résiduelle} puis d'être \emph{normalisé} une fois de plus. Ce processus aussi est répété $N$ fois.
Avant d'entrer dans l'encodeur, l'entrée est encodée. On parle d'\emph{embedding}. On lui ajoute un \emph{encodage positionnel} pour identifier la position de chaque composant dans l'entrée. Ces opérations sont aussi appliquées à la sortie obtenue par le modèle lorsqu'elle est fournie en entrée du décodeur.
\subsection{Mécanisme d'attention}
\label{ssec:mecattention}
L'outil central de l'architecture Transformer est le \emph{mécanisme d'auto-attention multi-tête}. C'est une variante du mécanisme d'attention.
Le \textbf{mécanisme d'attention} est une opération composante à composante qui souligne le lien des composants d'une entrée entre eux \cite{morgan_explainable_2023}. Mathématiquement, cette opération correspond à un produit de matrices. Plus précisément, le mécanisme d'attention décrit par \cite{vaswani_attention_2017}, appelé \textbf{scaled dot-product attention} et illustré dans la Figure~\ref{fig:scaledDotProduct}, correspond à un produit de matrices associées à un équilibrage (scaling) et une pondération:
\begin{align*}
\text{Attention}(Q,K) &= \text{softmax}(\frac{QK^T}{\sqrt{d_h}})\\
\text{Output}(Q,K,V) &= \text{Attention}(Q,K)V\\
\end{align*}
$Q$, $K$ et $V$ sont des matrices de dimension $d_h\times n$ appelées \emph{Query}, \emph{Key} et \emph{Value}. Elles correspondent respectivement aux informations recherchées, aux informations dont on dispose et aux informations que l'on peut communiquer \cite{morgan_explainable_2023}. La matrice $\text{Attention}(Q,K)$ est appelée \textbf{matrice d'attention}. Elle représente la façon dont chaque mot est lié aux autres $n$ mots de l'entrée et l'importance de ce lien.
Parfois, les matrices $Q$, $K$ et $V$ sont issues de la même donnée. Dans ce cas, le mécanisme d'attention est appelé \textbf{auto-attention}. C'est une forme très utilisée du mécanisme d'attention.
Dans leur modèle, \cite{vaswani_attention_2017} projettent les données en entrée du mécanisme d'attention dans $h$ sous-espaces de dimension $d_h$, appliquent le mécanisme d'attention sur chaque projection puis concatènent les sorties obtenues avant de reprojeter dans un nouvel espace. Ces projections et espaces sont appris lors de l'entraînement. On appelle cette variante \textbf{le mécanisme d'auto-attention multi-tête}. Si l'attention et l'auto-attention (illustré dans la Figure~\ref{fig:scaledDotProduct}) peuvent être vus comme une unité neuronale, le mécanisme d'attention multi-tête (voir Figure~\ref{fig:multi-head-attention}) est comparable à une couche de neurones.
Dans certains usages, les composants d'une entrée ne doivent porter attention qu'aux composants qui les précèdent. Par exemple, lorsqu'on cherche à construire une phrase, on ignore les mots qui se situent après celui qu'on est en train de construire. On ne peut donc pas prendre en compte ces mots dans la construction. Les \textbf{mécanisme d'attention masqués} répondent à ce besoin. Les coefficients de la matrice d'attention correspondant à de l'attention portée par un mot vers un successeur sont remplacés par $-\infty$. Dans le cas du \emph{scaled dot-product attention}, ce processus de masquage est effectué après le scaling (division par $\sqrt{d_h}$) mais avant l'application de softmax.
% \alh{ici j'aurais inversé le 3.1.1 et 3.1.2. pour aller du plus gros grain au plus fin. , voire mettre le mécanisme d'attention comme un focus dans le 3.1.2. }
\begin{figure}
\parbox{7cm}{%
\includegraphics[width=0.4\textwidth]{images/Illustration_Rapport_S9-scaledDotProdAttention.png}
\caption{Illustration du \emph{scaled dot-product} issu du modèle Transformer \cite{vaswani_attention_2017}}%
\label{fig:scaledDotProduct}}%
\qquad
\begin{minipage}{7cm}%
\includegraphics[width=0.8\textwidth]{images/MultiHeadAttention_Vaswani-et-al2017.png}
\caption{Illustration du \emph{mécanisme d'auto-attention multi-tête} proposé par \cite{vaswani_attention_2017}}%
\label{fig:multi-head-attention}%
\end{minipage}%
\end{figure}
\subsection{Des modèles à base de Transformer pour l'analyse d'image}
Dans le domaine de l'analyse et du traitement automatique de l'image, les modèles les plus répandus sont basés sur les réseaux de neurones convolutionnels. Le mécanisme d'attention est parfois intégré à ces modèles pour permettre une meilleure performance. \cite{dosovitskiy_image_2021} propose un modèle pour la classification d'images qui n'exploite pas de CNN mais se base exclusivement sur l'architecture Transformer et le mécanisme d'attention.
Le \textbf{Vision Transformer}, illustré par la Figure~\ref{fig:VisionTransformer}, traite les images en les découpant en morceaux appelés \textbf{patches}. Ces parties sont encodées puis complétées par un embedding positionnel. L'ensemble des tokens obtenus est complété d'un token de classification \texttt{[CLS]} puis passé à un encodeur de Transformer. Le token \texttt{[CLS]} ne dépend pas de l'entrée mais à la sortie de l'encodeur, sa représentation reflète l'intégralité de l'image. La représentation du token \texttt{[CLS]} est en suite passée dans un petit réseau de neurones qui classe l'image.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{images/ViT_Dosovitskiy-et-al2021.pdf}
\caption{Illustration Vision Transformer proposé par \cite{dosovitskiy_image_2021}}
\label{fig:VisionTransformer}
\end{figure}
\subsection{Des modèles à base de Transformer pour le traitement de la langue naturelle}
Le modèle Transformer de \cite{vaswani_attention_2017} a inspiré plusieurs modèles de langue basés sur cette architecture.
\cite{radford_improving_2018} propose un modèle basé sur la partie \emph{décodeur} du modèle Transformer. L'architecture du modèle GPT (\emph{Generative Pre-Training}), illustré par la Figure~\ref{fig:gpt}, est composée de $N=12$ itérations du bloc décodeur. Les blocs décodeur sont composés d'un \emph{mécanisme d'auto-attention multi-tête masqué} associé à une \emph{connexion résiduelle} suivie d'une \emph{normalisation} puis d'un \emph{réseau de neurone} avec une \emph{connexion résiduelle} encore une fois suivie d'une \emph{normalisation}. L'enchaînement de blocs décodeur est précédé d'un \emph{encodage} des composants de l'entrée associés à un \emph{encodage positionnel}.\\
Au delà de l'architecture utilisée par \cite{radford_improving_2018}, la méthode d'entraînement joue un rôle important dans la performance du modèle GPT. Dans un premier temps, le modèle est \emph{pré-entraîné} d'une façon présentée comme \emph{non-supervisée} sur un très grand corpus de texte pour apprendre une représentation universelle de son environnement. Après cela, le modèle est affiné à l'aide de données étiquetées qui correspondent à la tâche tierce attribuée au modèle: on parle de \textbf{fine-tunning}.
\begin{figure}
\centering
\includegraphics[width=0.3\textwidth]{images/GPT_Radford-et-al2018.png}
\caption{Illustration du modèle GPT proposée par \cite{radford_improving_2018}}
\label{fig:gpt}
\end{figure}
Un autre modèle basé sur le modèle Transformer est le \emph{Bi-directional Encoder Representations from Transformer} (BERT) proposé par \cite{devlin_bert_2019}. Contrairement au modèles GPT, il se base sur un \emph{encodeur} de Transformer et non pas un décodeur. Ainsi, il utilise des \emph{mécanismes d'attention} qui ne sont pas masqués. Comme pour le modèle GPT, le modèle BERT et entraîné en deux temps. Une représentation globale de l'environnement est d'abord apprise de façon \emph{semi-supervisée} à l'aide de deux tâches: prédire les tokens masqués dans une entrée et prédire la phrase suivant celle donnée en entrée. Le modèle est en suite \emph{fine-tunné} avec des données étiquetées issues d'un dataset spécialisé pour la tâche visée.\\
Depuis la publication de \cite{devlin_bert_2019}, le modèle BERT a été repris et adapté pour de multiples usages. En effet, au delà de ses bonnes performances et de sa forte polyvalence, sa méthode d'entraînement rend sa spécialisation facile. Plusieurs travaux ont été effectués sur la spécialisation du modèle \cite{aftan_survey_2023}. On retrouve un certain nombre de modèles basés sur BERT spécialisés dans une langue cible comme le français \cite{segonne_jargon_2024, martin_camembert_2020}, le néerlandais \cite{delobelle_robbert_2020} ou l'arabe \cite{antoun_arabert_2021}. D'autres modèles sont conçus pour les \emph{langues de spécialité} : dans des contextes techniques tels que la justice \cite{douka_juribert_2021} ou la médecine \cite{lee_biobert_2020}, on attend d'un modèle qu'il soit capable de s'exprimer avec un vocabulaire ou des codes précis et spécifiques à ce contexte. Au delà des travaux d'application, on retrouve aussi des versions améliorées du modèle initial comme \cite{sanh_distilbert_2020} qui propose un modèle aussi efficace mais avec moins de paramètres ou tel que \cite{wahab_dibert_2021} qui intègre d'autres compétences linguistiques telles que des notions de syntaxe.
Nous nous limitons ici aux modèles de la forme encodeur ou décodeur comme BERT et GPT. D'autres formes de modèles inspirés du modèle Transformer existent \cite{touvron_llama_2023, raffel_exploring_2020}. Certains modèles prennent la forme d'un décodeur ou d'un encodeur, d'autres exploitent les deux parties.
\section{Modèles boite-noire}
Comme mentionné précédemment, les modèles et systèmes cherchent à extraire une représenta\-tion d'un environnement afin de répondre à une tâche. Selon la technologie sur laquelle se base un modèle, la représentation sous-jacente de son environnement est plus ou moins accessible. Cela peut être dû à la complexité technique de l'algorithme d'apprentissage ou à la complexité du modèle en lui-même (calculs lourds, grand nombre de paramètres) \cite{garouani_investigating_2024}. \\
À l'inférence, le processus décisionnel suivi n'est pas transparent pour tous les modèles. La procédure est connue mais il n'est pas toujours possible de lui donner du sens d'un point de vue humain. Les modèles pour lesquels les mécanismes internes de décision sont obscures et/ou difficiles à comprendre sont appelées des \textbf{modèles boite-noire} \cite{bell_its_2022, garouani_investigating_2024}.\\
Parmi ces modèles, on retrouve les modèles de deep-learning tels que les réseaux de neurones (DNN, CNN, RNN,...), les modèles Transformer ou encore les méthodes telles que \emph{Random Forest} \cite{garouani_investigating_2024}.
\chapter{Expliquer les modèles boite-noire}
Dans le chapitre précédent, nous avons constaté que la plupart des modèles couramment utilisés sont des modèles boite-noire. Pour autant, ces modèles peu explicables pourraient être exploités dans certains secteurs nécessitant une confiance et une interprétation de ses outils. Afin de permettre cette utilisation, des méthodes ont été mises en place pour essayer d'extraire leurs enjeux internes. On présente dans ce chapitre leurs différentes familles accompagnées d'exemples et on évoque les enjeux autour du développement de ces techniques. On se situe ici dans le domaine de l'\emph{explicabilité post-hoc}.
Dans cette partie, on fera la confusion entre \emph{modèle} et \emph{système}. Dans le cas des modèles, la sortie correspond à la représentation de la donnée produite par le modèle.
\section{Taxonomie des méthodes d'explication}
Pour mettre en évidence des liens entre les entrées et les sorties des modèles boite-noire, de nouvelles méthodes sont régulièrement mises en place. On parle de \emph{méthode d'explication} ou \emph{d'explicabilité}.
Dans la littérature, on retrouve plusieurs critères qui distinguent les méthodes d'explicabilité. Certains modèles cherchent à fournir une explication du fonctionnement du modèle pour n'importe quelle entrée. On parle d'explication \textbf{globale}. Par opposition, on trouve des méthodes qui cherchent à lier les éléments d'une entrée avec la sortie obtenue par le modèle en se concentrant sur une entrée. L'explication n'est pas vraie en général mais seulement pour l'entrée étudiée. C'est une méthode d'explication \textbf{locale}.\\
Les méthodes d'explication se distinguent aussi par leur manière de procéder. Certaines méthodes, dites \textbf{model-agnostic}, ne s'intéressent pas aux paramètres du modèle mais uniquement à son comportement sur les entrées. À contrario, les méthodes \textbf{model-specific} cherchent à exploiter les paramètres du modèle pour aider à donner une explication. Cela a pour conséquence qu'elles ne sont pas exploitables pour tous les modèles.\\
Dans le cas de la classification, certaines méthodes proposent une explication différente pour chaque classe de la sortie. Ces méthodes sont dites \textbf{class-specific}. Quand l'explication est commune à toute les classes, on parle de méthode \textbf{class-agnostic}.
Selon la stratégie utilisée par les méthodes, ont peu les séparer en plusieurs familles parmi lesquelles ont retrouve :
\begin{itemize}
\item les méthodes basées sur la \emph{perturbation} des entrées
\item les méthodes basées sur la \emph{rétro-propagation} d'information dans le modèles
\item les méthodes basées sur le \emph{gradient}
\item les méthodes basées sur les \emph{matrices d'attention}
\item les méthodes basées sur l'étude d'\emph{exemples} bien choisis
\end{itemize}
Ces familles ainsi que des exemples de méthode sont présentés dans la partie suivante.
Les différentes caractéristiques des méthodes d'explication sont récapitulées dans les Figures~\ref{fig:taxoXAIMeth} et \ref{fig:recapPropXAIMeth}.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{images/Illustration_Rapport_S9-TaxonomieMethodeXAI.drawio.png}
\caption{Taxonomie des méthodes d'explication de l'intelligence artificielle incluant les méthodes présentées dans la Section~\ref{sec:XAIMeth}}
\label{fig:taxoXAIMeth}
\end{figure}
\begin{figure}
\centering
\begin{tabular}{|l||c|c||c|c||c|c|}
\hline
Famille de méthodes & locale & globale & model-specific & model-agnostic & class-specific & class-agnostic \\
\hline
par perturbation & $\times$ & & & $\times$ & & $\times$ \\
par rétro-propagation & $\times$ & & $\times$ & & $\times$ & \\
basée sur gradient & $\times$ & & $\times$ & & $\times$ & \\
basée sur l'attention & $\times$& & $\times$ & & & $\times$ \\
par l'exemple & $\times$ & & & $\times$ & & $\times$\\
\hline
\end{tabular}
\caption{Tableau récapitulatif des propriétés des familles de méthodes d'explicabilité présentées dans ce chapitre}
\label{fig:recapPropXAIMeth}
\end{figure}
\section{Les méthodes d'explication répandues}
\label{sec:XAIMeth}
\subsection{Méthodes basées sur la perturbation}
Les \textbf{méthodes d'explication par perturbation} cherchent à déduire une explication de la prédiction $M(x)$ pour une entrée $x$ et un modèle $M$ en observant les prédictions $M(x')$ associées à des entrées $x'$ proche de $x$. L'idée est, à l'aide des prédictions $M(x')$ pour des versions perturbées de $x$, de mettre en évidence les caractéristiques de $x$ qui contribuent à la prédiction $M(x)$.
La méthode LIME (Local Interpretable Model-agnostic Explanations) \cite{molnar_interpretable_2025, ribeiro_model-agnostic_2016} cherche à simplifier le modèle boite-noire $M$ par un modèle explicable $M'$ qui approxime $M$ au voisinage de l'entrée $x$. Pour cela, LIME:
\begin{enumerate}
\item Créé un ensemble $X$ d'entrées $x'$ qui sont des versions de $x$ perturbées
\item Récupère la prédiction $M(x')$ du modèle qu'on cherche à expliquer pour chaque entrée $x'$
\item Entraîne un modèle interprétable $M'$ (voir Section~\ref{mod_interpretables}) à l'aide de ces prédictions en les pondérant par la proximité de $x'$ à $x$
\item Considère l'interprétation de $M'$ comme une explication de la contribution des éléments de $x$ dans sa prédiction $M(x)$
\end{enumerate}
Cette méthode est \emph{locale}, \emph{model-agnostic} et \emph{class-agnostic}.
La méthode SHAP (SHapley Additive exPlanations) \cite{molnar_interpretable_2025, lundberg_unified_2017} cherche à attribuer à chaque élément de $x$ une contribution à la différence entre $M(x)$ et la moyenne des toutes les prédictions. À l'aide d'une base de donnée similaire à celle produite avec la méthode LIME, on calcule la contribution de chaque attribut de $x$ à sa prédiction. Une contribution peut être positive ou négative. La somme des contributions des attributs correspond à la différence entre $M(x)$ et la moyenne des prédictions. SHAP est aussi une méthode \emph{locale}, \emph{model-agnostic} et \emph{class-agnostic}.\\
\subsection{Méthodes basées sur la rétro-propagation}
Les \textbf{méthodes d'explication basées sur la rétropropagation} vise à propager une grandeur couche par couche à travers un réseau de neurones de la sortie vers l'entrée (c'est à dire en sens inverse). Pour une couche donnée, on obtient la contribution de donne issue de ce noeud dans la prédiction. En particulier, lorsqu'on atteint l'entrée du modèle, on obtient une contribution pour chaque composante pour une entrée donnée. Ces méthodes se basent sur la Deep Taylor Decomposition \cite{montavon_explaining_2017} :
$$\left\{
\begin{array}{rl}
R^{(n)}_j &= \mathcal{G}(X,W,R^{(n+1)}) = \sum_{i}X_j\frac{\partial L_i^{(n)}(X,W)}{\partial X_j}\frac{R_i^{(n+1)}}{L_i^{(n)}(X,W)} \quad \forall 0< n <N\\
R_j^{(N)} &= \mathbb{1}_{t}(j) \text{ $t$ est la classe étudiée}
\end{array}\right.
$$
$n$ représente la couche du réseau, $j$ représente le neurone dans la couche, $X$ est la sortie de la $n^e$ couche et $W$ est la matrices des poids entre les couches $n$ et $n+1$. La valeur $R_j^n$ correspond à la quantité d'information, appelée \emph{importante}, dans le $j^e$ noeud de la $n^e$ couche du réseau. $\mathbb{1}_{t}$ est l'indicatrice de $t$ : $\mathbb{1}_{t}(t) = 1$ et $\forall x\not= t\; \mathbb{1}_{t}(x) = 0$
Dans le cadre des réseaux de neurones profonds avec la fonction d'activation ReLU effectuant une classification, \cite{montavon_layer-wise_2019} propose la méthode Layer-wise Relevance Propagation (LRP).\\
Pour une entrée $x$, chaque attribut $j$ reçois un coefficient $R_j^0$ indiquant l'importance de l'attribut dans le calcul de la probabilité de la classe $t$. On l'obtient en remontant couche par couche depuis la sortie. $R^N$ est le vecteur avec que des 0 sauf à la $t^e$ classe qui vaut 1. La formule pour remonter le réseau est :
$$
R^{(n)}_j = \mathcal{G}(X^+, W^+, R^{(n+1)}) = \sum_i \frac{x_j^+ w_{ji}^+}{\sum_{j'} x_{j'}^+ w_{j'i}^+} R_i^{(n+1)}
$$
où $v^+ = max(0,v)$
Ces méthodes sont \emph{locales}, \emph{model-specific} et \emph{class-specific}.
\subsection{Méthodes basées sur le gradient}
Les \textbf{méthodes d'explication à base de gradient} \cite{ancona_gradient-based_2019} suivent une intuition similaire à celle des méthodes basées sur le rétro-propagation : raisonner depuis la sortie du modèle vers son entrée. Ce qui distingue ces deux familles de méthode est le sens associé à la propagation et la grandeur propagée. Les méthodes à base de gradient construisent une fonction au fur et à mesure de cette propagation. En effet, ces méthodes se basent sur le calcul \textbf{gradient des sorties du modèle en fonction des entrées}. Cela donne une approximation linéaire de l'impact de chaque partie de l'entrée sur chaque valeur en sortie.\\
Formellement, si on défini un modèle réseau de neurones comme une fonction $M$ qui associe à une entrée $(x^0_1,\dots,x^0_{N_0})$ une sortie $(y_1,\dots,y_{N_C})$ et qu'on note $(x^i_1,\dots,x^i_{N_i})$ les $N_i$ coefficients obtenus au cours du passage dans la $i^\text{ème}$ couche du modèle, le gradient de la sortie $y_j$ est
$$\nabla_{(x^0_i)_i}y_j = \left( \frac{\partial y_j}{\partial x^0_1}, \dots, \frac{\partial y_j}{\partial x^0_{N_0}} \right)$$
et est obtenu à l'aide de
$$\forall i \in \{1,\dots, N_l\} \forall j \in \{1,\dots, N_C\}, \, \frac{\partial y_j}{\partial x^i_{N_l}} = \sum_{k=0}^{N_{i+1}} \frac{\partial y_j}{\partial x^{i+1}_{k}} \frac{\partial x^{i+1}_{k}}{\partial x^i_{N_l}}$$
Ces méthodes dépendent de l'entrée étudiée, explore les paramètres du modèle qui est un réseau de neurones (éventuellement convolutif) : elles sont \emph{locale}, \emph{model-specific} et \emph{class-specific}.
Certaines méthodes associent le gradient de la sortie en fonction de l'entrée avec d'autres outils pour fournir une explication. La méthode GradCAM \cite{selvaraju_grad-cam_2017} associe le calcul du gradient avec l'approche Class Activation Mapping (CAM) pour obtenir une "carte d'influence" dans le cadre des modèles ayant pour environnement des images. La méthode Input$\times$Gradient \cite{shrikumar_not_2017} multiplie le gradient obtenu par l'entrée.
\begin{tcolorbox}[colback=blue!5!white,
colframe=blue!75!black,
title=Gradient à l'entraînement et lors de l'explication
]
Il faut distinguer le gradient utilisé lors de l'entraînement du modèle et celui utilisé pour fournir une explication. Lors de l'entraînement d'un modèle, on exprime le gradient des sorties \textbf{en fonction des paramètres}. Dans ce cas, les calculs intermédiaires effectués dans les noeud d'un réseau de neurones sont des coefficients et les paramètres sont les variables de la fonction. Il est généralement noté $\nabla_{(W_i,b_i)_i}y_j$ où les $W_i$ et $b_i$ sont les matrices de paramètres et les biais du modèle.\\
Lors de l'explication d'un modèle, on observe le gradient des sorties \textbf{en fonction des entrées}. Les paramètres du modèle sont des simples coefficients et les composantes de l'entrée sont des variables. On le note $\nabla_{(x_i)_i}y_j$ où les $x_i$ sont les composantes de l'entrée.
\end{tcolorbox}
\subsection{Méthodes basées sur l'attention}
Comme expliqué précédemment, le mécanisme d'attention produit et exploite une matrice d'attention qui exprime le lien des composantes de la donnée en entrée entre elles. Les \textbf{méthodes d'explication basées sur l'attention} exploitent ces informations pour déduire une explication sur le processus de décision suivi par le modèle sur l'entrée appliquée.
Une première façon d'exploiter les matrices d'attention obtenues lors de l'inférence sur une entrée est la visualisation. Des outils tels que BertViz \cite{vig_multiscale_2019} proposent une interface de visualisation des coefficients des matrices d'attention. Observer ces matrices permet notamment de trouver de potentiels biais ou d'identifier les mécanismes d'auto-attention et les têtes de mécanisme d'auto-attention multi-tête les plus importants.
Les matrices d'attention peuvent aussi être plus ou moins traitées pour en extraire des "cartes d'influence". Une façon très simple est de visualiser la carte obtenue à partir d'une seule matrice d'attention. La méthode raw attention \cite{chefer_transformer_2021} visualise la "carte d'influence" extraite de la matrice d'attention issue du mécanisme d'attention le plus proche de la sortie du modèle.
\cite{abnar_quantifying_2020} propose deux méthodes d'extraction de scores à partir des matrices d'attention issues d'un modèle. Ces méthodes cherchent à exploiter toutes les matrices d'attention obtenues lors d'une inférence. Elles se basent sur l'interprétation des matrices d'attention comme des représentations de graphes. Ces deux méthodes se différencient par la façon dont les différentes matrices d'attention (et par extension les arêtes des graphes) interagissent.\\
La première méthode appelée \emph{rollout attention} voit le passage de l'information comme le parcours d'un chemin. Si on note $A_i$ la matrice représentant les matrices d'attention du bloc $i$, on cherche $\tilde{A}_N$ la matrice de scores recherchée où $N$ est le nombre de blocs dans le modèle et on l'obtient à l'aide de l'expression :
$$\tilde{A}_i = \left\{\begin{array}{ll}
A_i & \text{si } i = 0\\
A_i\tilde{A}_{i-1} & \text{sinon}
\end{array}\right.
$$
La méthode \emph{attention flow} fait le parallèle entre le passage de l'information et le problème de flot dans les graphes. La matrice de score associée à une entrée sur un modèle correspond au flot maximal du graphe.
Ces méthodes sont \emph{locales}, \emph{model-specific} et \emph{class-agnostic}
\subsection{Méthodes basées sur les exemples}
Les \textbf{méthodes d'explication basées sur les exemples} cherchent à mettre en évidence des informations sur le modèle étudié à l'aide de données exemples construites ou issues de la base de données disponible.
Les \textbf{exemples adverses} visent à mettre le modèle en difficulté. Il sont construits à partir d'entrées bien traitées (correctement classées ou avec une bonne prédiction) qui sont perturbées de telle manière que la différence soit peu voir indiscernable pour l'humain mais que le modèle ne soit plus capable de bien la traiter \cite[Chap~30]{molnar_interpretable_2025}. La Figure~\ref{fig:adversarialExample} montre un exemple de construction d'exemple adverse.\\
Les exemples adverses sont utilisés dans le cadre de l'explicabilité pour le perfectionnement des modèles. Ils mettent en évidence les faiblesses du modèle et sont parfois utilisés en complément de la base de donnée originale pour augmenter la résistance du modèle à des attaques similaires.
\begin{figure}
\centering
\includegraphics[width=0.5\textwidth]{images/AdversarialPerturbation_Panda.png}
\caption{Donnée perturbée pour construire un exemple adverse}
\label{fig:adversarialExample}
\end{figure}
Un autre type d'exemple utilisé pour expliquer les modèles sont les \textbf{exemples contre-factuels} \cite[Chap~15]{molnar_interpretable_2025}. Ces exemples cherchent à illustrer un lien de causalité entre la perturbation d'une entrée et la sortie obtenue. Cette causalité n'est pas nécessairement cohérente avec le contexte mais souligne le changement de sortie fournie par le modèle entre les deux versions de l'entrée. De tels exemples sont obtenus en cherchant à atteindre une certaine prédiction à partir d'une entrée fixée qui sera perturbée. Une perturbation pertinente vérifie plusieurs critères : peu de composantes de l'entrée sont modifiées, les modifications sont aussi faibles que possible, la prédiction erronée recherchée est atteinte et l'entrée perturbée est plausible. On note que plusieurs perturbations peuvent vérifier toutes ces conditions et qu'elle peuvent même se contredire. Dans ce cas, il est possible de toute les présenter ou d'en sélectionner qu'une partie à l'aide des critères définis plus tôt.
Ces méthodes sont \emph{locales}, \emph{model-agnostic} et \emph{class-agnostic}.
\section{Évaluer les méthodes d'explication}
Que ce soit pour juger la qualité d'une méthode d'explication ou pour faire un choix parmi plusieurs méthodes, il est nécessaire de pouvoir comparer les méthodes d'explication. Évaluer et comparer les méthodes d'explication n'est pas un travail simple et de nouveaux frameworks et de nouvelles métriques sont encore proposés. On s'intéresse ici aux enjeux de l'évaluation des méthodes d'explicabilité, aux métriques et aux méthodes de comparaison.
\subsection{Ce qu'on attend d'une explication}
\label{ssec:attendus}
% \begin{comb}
% De façon générale on attends (rappel)\\
% - que la méthode reflète le raisonnement du modèle (réponde à pourquoi)\\
% - quelle soit humainement accessible\\
% Concrètement / plus précisément:\cite{mersha_evaluating_2025} \\
% - robuste : ne pas être trop sensible aux petits change et dans une entrée\\
% - conscistente : entre deux modèles similaires, les explication soient aussi similaire pour une même entrée\\
% - contrastante : pointe des élémnts qui font que ça change / des explications similaires ont des sortie simialires / des explications très opposées ont des sorties différentes\\
% Des éléemnts qui porent à débat:\\
% - jugement humain de l'explication : qualitatif donc pb de comparaison
% Des exigences qui dépendent de l'usage \cite{bell_its_2022}:
% - dépends du public visé\\
% - dépends de l'usage A SOURCER\\
% \end{comb}
Par la définition d'explication, on attend d'une méthode d'explication qu'elle fournisse des explications qui soient humainement accessibles tout en reflétant le plus possible le mécanisme de décision du modèle. Concernant les méthodes d'explication elles-mêmes, \cite{mersha_evaluating_2025} présente plusieurs critères attendus d'une méthode d'explication:
\begin{description}
\item[Robustesse] : Les explications fournies pour deux entrées très similaires doivent elles aussi être similaires.
\item[Consistance] : Une méthode appliquée à deux modèles similaires et entraînés sur les mêmes données doit proposer des explications similaires sur une même entrée (voir Figure~\ref{fig:consistanceMersha-et-al2025}).
\item[Contraste] : Les explications fournies permettent de marquer le contraste entre deux entrées ayant des prédictions différentes.
%\item[Adéquation au raisonnement humain] : Les explications obtenues sont similaires aux explications humaines fournies sur une entrée dont la sortie est connue.
\end{description}
Pour chacun de ces critères, \cite{mersha_evaluating_2025} propose une métrique. Une analyse de la correspondance entre ces critères et les métriques proposées serai une piste de poursuite intéressante à ce sujet.
\begin{figure}
\centering
\includegraphics[width=0.7\textwidth]{images/ConsistanceMersha-et-al2025.drawio.png}
\caption{Une méthode d'explication est consistante si, d'un modèle à l'autre, entraînés sur les mêmes données, elle fourni des explications pour une même entrée dont la différence est comparable à la différence des modèles.}
\label{fig:my_label}
\end{figure}
D'autres critères proposés dans la littérature sont plus problématiques. \cite{chefer_transformer_2021} et \cite{mersha_evaluating_2025} mesurent dans leur expérience l'adéquation des explications fournies au raisonnement humain. Cet attendu n'en est pas un concernant les méthodes d'explicabilité car ces dernières doivent être fidèles au modèle et non raisonnement humain. Par contre, l'adéquation au raisonnement humain peut être un attendu concernant les modèles dans l'optique de leur donner un comportement et/ou un raisonnement plus humain. C'est une piste étudiée dans la Section~\ref{sec:modelesconcepts}.\\
Par ailleurs, \cite{neely_song_2022} interroge la notion d'explication idéale. Ainsi, il devient difficile de définir un protocole de comparaison des méthodes d'explication (entre elles ou avec une explication de référence) qui puisse faire consensus.
D'autre part, un récapitulatif des attendus proposés dans la littérature permettrait de dégager un éventuel manque dans la liste précédente.
\subsection{Les métriques}
% \begin{comb}
% \cite{chefer_transformer_2021} propose mesure d'adéquation au modèles avec le "Pointing Game"\\
% Les problématiques rencontrées avec les métriques:\\
% - qualitatif vs quantitatif\\
% - manque de métriques reconnues\\
% - biais humain : la machine ne pense pas tjrs comme humin ~> on ne peut pas comparer explication et GoundTruth\\
% - comparabilité des explications issues de méthodes différentes (format)
% \end{comb}
Au delà de l'absence de consensus sur les critères définissant une bonne méthode d'explication, \cite{garouani_investigating_2024} souligne la rareté des métriques quantitatives qui couvrent l'ensemble des critères et qui soit adaptées à plusieurs modèles. \cite{mersha_evaluating_2025} insiste aussi sur l'absence de consensus concernant les métriques.
Par ailleurs, certaines métriques sont remises en question. \cite{gilpin_explaining_2019} dénonce l'évalua\-tion des explications à l'aide du jugement humain. On fourni une entrée, sa sortie pour un modèle et l'explication obtenue par la méthode d'explication étudiée à un évaluateur humain chargé de juger plusieurs critères. Au-delà de la difficulté d'organiser une telle évaluation, cette dernière est sujette aux biais en faveur de l'intelligibilité et en défaveur de la fidélité au modèle. De plus, elle n'est pas quantitative non plus, ce qui pose problème pour comparer les méthodes d'explication entre elles.
Une métrique adaptée pour mesurer la fidélité des explications fournies par une méthode d'explication est celle utilisée par \cite{chefer_transformer_2021}. Cette dernière mesure la proximité de l'explication au raisonnement du modèle à l'aide d'une méthode de perturbation. Les composants de l'entrée jugés importants par l'explication sont masqués. On observe la performance du modèle sur ces nouvelles entrées. Si elle baisse drastiquement, cela signifie que les composants en question sont effectivement importants dans le processus de décision du modèle.
%\alh{on s'attend un peu à voir votre position sur tout ceci}
% \begin{comb}
% Pour comparer on a des exigences en plus sur les métriques:\\
% - quantitatif\\
% - compatible avec plusieurs méthodes et nature de données\\
% - equitable : cpte bien les compétences de toutes les métriques\\
% \cite{mersha_evaluating_2025} énumère des exigences et propose un framework:\\
% - Complexité du modèle\\
% - Nature des entrées\\
% - Données elles-même\\
% - Nature de la tâche demandée\\
% - Méthodes d'explicabilité\\
% - Métriques pour l'explicabilité
% \end{comb}
Lorsqu'il s'agit d'utiliser des métriques pour comparer des méthodes d'explication, certains problèmes sont accentués. En plus de devoir permettre une comparaison, les métriques doivent être adaptées à plusieurs méthodes et contextes. \\
\cite{mersha_evaluating_2025} énumère plusieurs exigences pour concevoir un protocole d'évaluation et de comparaison de méthodes d'explication: Une grande diversité est nécessaire pour obtenir un avis global sur les méthodes. Ce besoin de diversité porte sur la complexité du modèle, la nature des entrées, la variété des tâches, les thématiques des entrées, les méthodes d'explication comparées, les attendus et les métriques utilisées,...
%\alh{c'est une mine, c'est vraiment interessant !}
\chapter{Des modèles explicables}
Jusqu'ici, nous nous sommes intéressés à l'\emph{explicabilité post-hoc}, ses outils, les besoins auxquels elle répond et les difficultés rencontrées. Un autre axe de l'explicabilité est l'\emph{explicabilité by-design}. L'objectif n'est plus de retrouver le raisonnement une fois le modèle ou le système construit mais de concevoir des modèles et systèmes qui soient plus interprétables.
\section{Modèles Interprétables simples}
\label{mod_interpretables}
Il existe des modèles et méthodes de construction de modèles qui sont moins complexes que les réseaux de neurones profonds ou les méthodes ensemblistes. Parmi ces méthodes, on retrouve les arbres de décision, les modèles basés sur la régression linéaire ou encore les méthodes à base de listes de règles \cite[Chapitres~6 à 11]{molnar_interpretable_2025}. Ces formes de modèles permettent d'exprimer de façon logique et/ou mathématique les éléments pris en compte pour obtenir une sortie ou une autre.\\
Ces modèles sont réputés pour la facilité avec laquelle on peut accéder au processus de décision appliqué lors d'une prédiction. Les modèles transparents sont souvent qualifiés de \textbf{modèles interprétables}.
\section{Des compromis inhérents aux attendus d'une explication}
\label{sec:tradesoff}
Construire des solutions pour favoriser l'explicabilité en intelligence artificielle engendre certaines confrontations. À première vue, concernant l'explicabilité by-design, il faudrait choisir entre la performance des modèles et leur interprétabilité. De la même manière, du coté de l'explicabilité post-hoc, le sentiment de devoir choisir entre fidélité au modèle et accessibilité au grand public est présent.
\subsection{Compromis entre performance des modèles et interprétabilité}
% \begin{color}{gray}
% Une autre caractéristique qui les oppose aux modèles boite-noire est la performance. En effet, si les modèles boite-noire sont réputés pour leurs bonnes performances, on reproche souvent aux arbres de décision, aux modèles linéaires et aux modèles à base de règle leur faible performances par rapport au modèles de deep learning. On notera tout de même que si cette intuition semble évidente, l'idée que l'interprétabilité d'un modèle soit inversement proportionnelle à sa performance ne semble pas documenté dans la littérature.En ça, Les modèles interprétables sont facilement opposés aux modèles boite-noire présentés précédemment.
% \end{color}
% \begin{comb}
% On constate les boites-noires performantes d'un coté et les modèles interprétables inadaptés à certaines tâches (manipulation du texte par exemple SOURCER)\\
% Il n'y a pas de publications en ce sens pourtant \\
% \cite{rudin_stop_2019} dénonce même cette idée en la qualifiant d'illusion (RETROUVER LE PASSAGE)
% \end{comb}
Les modèles boite-noires tels que les réseaux de neurones et les Transformers sont réputés pour leurs bonnes performances sur une très grande diversité de tâches. Face à ces modèles boite-noire, les modèles interprétables réputés pour leur transparence peinent à faire valoir leurs performances pourtant similaires sur certaines tâches. La simplicité des représentations assimilées par ces modèles est facilement décriée. De plus, ces performances tendent à réduire lorsque la tâche appliquée et l'environnement associé se complexifient. De ce constat naît l'idée que les notions d'interprétabilité et de performance sont inversement proportionnelles. Cependant, il est important de noter qu'il n'y a pas de consensus allant dans ce sens. \cite{johansson_trade-off_2011} montre que cette relation existe entre les modèles à base de décision (arbres et listes de décision) et les modèles ensemblistes (\emph{Random Forest}) pour les tâches de classification dans le contexte bio-pharmaceutique alors que \cite{rudin_stop_2019} dénonce cette idée en la qualifiant de mythe. Ce débat encourage les travaux cherchant à maintenir l'interprétabilité des modèles tout en augmentant leurs performances.
\subsection{Compromis entre fidélité des explications et accessibilité pour le grand public}
% \begin{color}{gray}
% Cependant, certains auteurs tels que \cite{bell_its_2022} mettent en garde sur le sens de cette interprétabilité. Si ces modèles s'expriment bien à l'aide le la logique et des mathématiques, il est toujours possible de remettre en question la légitimité de ces expressions \cite{molnar_interpretable_2025}: Représentent-elles bien l'environnement?, Peut-on parler d'interprétabilité pour un arbre de décision de profondeur 15? \\
% \end{color}
% \begin{comb}
% \cite{gilpin_explaining_2019} expose le besoin de choix entre fidélité au modèle et intelligibilité du modèle\\
% \cite{molnar_interpretable_2025} remet en question interprétabilité des modèles dits interprétables (DT de hauteur 15)\\
% cela fait écho au problème du public visé :\\
% - \cite{bell_its_2022} : le grand public ne sait pas se servir des explications\\
% - \cite{kastner_relation_2021} : les explications rendent digne de confiance MAIS ne font pas la confiance du grand public (qui se rendent compte des faiblesses)
% \end{comb}
En explicabilité post-hoc, on construit des méthodes qui fournissent des explications concernant les processus de décision des modèles boite-noire. On attend d'une explication qu'elle reflète le raisonnement du modèle tout en étant humainement intelligible. \cite{gilpin_explaining_2019} souligne l'importance de faire un bon compromis entre la fidélité au modèle d'une explication et son intelligibilité. Simplifier de façon excessive peut créer des explications persuasives induisant une confiance indue de l'utilisateur pour le modèle. \\
\cite{molnar_interpretable_2025} remet aussi en question la réelle interprétabilité des modèles dits interprétables. Est-il légitime de dire d'un arbre de décision de profondeur 20 qu'il est interprétable? La formule le représentant est-elle toujours humainement intelligible?
La sous-section~\ref{ssec:def_expl} pointe la question du public visé par l'explication. \cite{bell_its_2022} et \cite{kastner_relation_2021} se sont intéresses au cas du grand public. \cite{bell_its_2022} pointe le faible impact de l'explication sur la compréhension des décisions des modèles par le grand public. \cite{kastner_relation_2021} met aussi en évidence la nuance entre donner confiance (trust) et mériter la confiance (fiabilité) (trustability dans l'article). Les explications cherchent à mettre en évidence la fiabilité des modèles. Sachant qu'il est rationnel d'avoir confiance dans les modèles fiables, les explications devraient permettre d'avoir confiance dans les modèles qui méritent cette confiance. Cependant, cette idée n'est pas vérifiée pour le cas du grand public. \cite{kastner_relation_2021} montre que dans ce cas, utiliser les explications n'impacte pas la confiance voire provoque une baisse de celle-ci : cela s'expliquerait par la prise de conscience que les modèles et systèmes d'intelligence artificielle sont faillibles ou par la difficulté du grand public à comprendre ces explications.
Si les méthodes d'explication actuelles ne semblent pas répondre au besoin d'explicabilité pour le grand public, d'autres pistes du coté de l'explicabilité by-design sont explorées à travers les modèles neuro-symboliques (Section~\ref{sec:neuro-symbolique}) et les modèles à base de concepts (Section~\ref{sec:modelesconcepts}).
\section{Les modèles neuro-symboliques}
\label{sec:neuro-symbolique}
Bien que l'exploitation des méthodes explication soient remise en question par \cite{bell_its_2022} et \cite{kastner_relation_2021} en ce qui concerne le grand public (voir la Section~\ref{sec:tradesoff}), les spécialistes et les concepteurs en IA restent très intéressés par ce qui se passe dans les modèles boites-noires. La possibilité de mettre en évidence des biais et la miniaturisation des modèles motivent aussi les travaux sur l'explicabilité de l'intelligence artificielle.
%\alh{ne serait-ce que pour des besoins de maintenabilité}.
Ainsi, de nombreux travaux portent sur le design de nouveaux modèles qui malgré une allure de modèle boite-noire, seraient plus interprétables. Ces modèles cherchent à tirer parti des méthodes symboliques et/ou interprétables en les associant à des structures plus complexes basées sur les réseaux de neurones. On les appelle \textbf{modèles neuro-symboliques} \cite{bhuyan_neuro-symbolic_2024}. \\
Au-delà de leur interprétabilité et leur transparence, les modèles symboliques sont appréciés pour leur application stricte de la logique sur laquelle ils se basent. Lorsque les entrées sont correctement identifiées, ces modèles ont de bons résultats. Les réseaux de neurones sont reconnus pour leur capacité à apprendre des comportements dont les motivations sont plus difficiles à cerner et reproduire un comportement moins rationnel plus difficile à formuler à l'aide des modèles symboliques. Sur la base de ces constats, l'idée de les associer permettrait de profiter des points forts de chacun : exploiter la précision des raisonnements logiques des modèles symboliques tout en profitant de l'intuition des réseaux de neurones. Pour cela, on retrouve différentes façons des les associer. \cite{bhuyan_neuro-symbolic_2024} distingue plusieurs familles, nous les listons ici sans détailler plus leurs spécificités
% \alh{, nous les listons ici sans détailler plus avant leurs spécificités(Mme Dao sera friande de ce paragraphe : ) donc prémunissons nous contre des attentes non satisfaites qui ne sont pas le sujet de cet état de l'art)}
:\\
La première forme, désignée par \emph{symbolic-neuro-symbolic}, regroupe les modèles composés d'un enchaînement d'applications de méthodes symboliques et connexionnistes (à base de réseaux de neurones). \\
Les modèles de la forme \emph{symbolic[neuro]} sont utilisés de la même façon que les méthodes symboliques. Ce qui les distingue est que les prédicats et fonctions sont des petits réseaux de neurones.\\
La famille de modèles \emph{neuro $\mid$ symbolic} correspond aux modèles qui font collaborer les deux approches. On retrouves des systèmes tels que celui proposé par \cite{kalouli_hy-nli_2020}. Un modèle se base sur la logique naturelle, le second est un BERT. Un troisième modèle est entraîné pour déterminer lequel des deux premiers est le plus susceptible de fournir la bonne sortie en fonction de caractéristiques choisies et détectées par ces deux modèles. \\
La quatrième famille proposée est la famille \emph{neuro-symbolic $\rightarrow$ neuro}. Ces modèles, qui sont des réseaux de neurones, sont conçus et/ou entraînés à l'aide de concepts et techniques symboliques. Dans cette famille, on retrouve les modèles qui exploitent l'injection de connaissances \cite{ciatto_symbolic_2024}.\\
Les modèles de la famille \emph{neuro$_{\text{symbolic}}$ } sont des modèles connexionnistes entraînés à l'aide de fonctions de perte particulières. Elles forcent l'assimilation de connaissances sous forme symbolique en incluant des contraintes logiques.\\
Enfin, les modèles de la forme \emph{neuro[symbolic]} intègrent des opérations logiques à l'échelle des couches du réseau de neurones qui les constituent. Au lieu d'effectuer une produit matriciel pour passer d'une couche à l'autre, c'est une opération logique qui est effectuée.
Si ces modèles contiennent bien des notions et des outils symboliques, il reste pertinent de questionner la nature explicable de ces derniers. En effet, lors de l'assimilation par le modèles des outils symboliques, il n'est pas impossible que ces derniers perdent en transparence. De plus, les formules symboliques restent des concepts techniques et ne sont pas nécessairement accessibles à tous les public. Ainsi, l'approche neuro-symbolique semble être une piste intéressante aussi bien du point de vue de la performance que de l'explicabilité. Cependant, ces buts doivent être gardés à l'esprit lors de la conception afin de ne pas complexifier des modèles sans que cela n'apporte de gain.
% \alh{L'investigation de ces approches semblent être une piste de poursuite interessante... ou un truc du genre ? le pavé n'est pas indigeste en tout cas !}
\section{Modèles à base de concepts}
\label{sec:modelesconcepts}
La plupart des méthodes d'explication post-hoc fournissent des explications sous la forme de contributions de chaque composant de l'entrée à travers des réels ou des "cartes d'influence". C'est à l'utilisateur de comprendre le sens associé à ces éléments. Ce travail n'est pas toujours évident ni accessible à l'utilisateur \cite{poursabzi-sangdeh_manipulating_2021}.\\
Les modèles interprétables disposent d'une méthode définie pour exprimer leur structure sous forme de formule mathématique ou logique. Encore une fois, si la formule est facile à obtenir, il n'est pas nécessairement simple de la lire ni de lui donner du sens. Dans le cas des méthodes neuro-symboliques, la présence de structures symboliques dans le modèle accorde une plus grande transparence au modèle mais son intelligibilité peut rester problématique pour les mêmes raisons.
Les \textbf{modèles à base de concepts} s'attaquent à la problématique de l'intelligibilité du raisonnement sous-jacent dans les modèles. Au lieu d'injecter dans le modèle des règles parfois abstraites, on cherche à lui apprendre des concepts humainement intelligibles. Un \textbf{concept} est défini par \cite[Chap.~29]{molnar_interpretable_2025} comme "n'importe quelle abstraction, telle qu'une couleur, un objet ou même une idée". \cite{poeta_concept-based_2023} distingue plusieurs types de concepts : Les \emph{concepts symboliques} correspondent aux concepts humains tels que "bleu", "oiseau", "manger",... Les \emph{concepts non-supervisés} sont appris par la machine de façon non supervisée. Certains éléments capturés sont intelligibles mais ce n'est pas toujours le cas de tous. Les \emph{prototypes} sont des concepts représentés par des exemples. Les \emph{concepts textuels} sont des descriptions de concepts sous forme de texte.
Les modèles à base de concepts peuvent prendre plusieurs formes \cite{poeta_concept-based_2023}:
\begin{itemize}
\item Les \emph{modèles à base de concepts supervisés} sont entraînés à reconnaître des concepts choisis. Les données d'entraînement sont annotées avec les concepts qui lui sont associés. Une couche interne du réseau est entraînée pour les reconnaître en parallèle de l'entraînement du système pour la tâche tierce. Si on ne dispose pas de données d'entraînement annotées pour les concepts, un autre lot de données peut être utilisé après l'entraînement à la tâche pour superviser l'apprentissage des concepts.
\item Les \emph{modèles à base de concepts non supervisés} sont entraînés sur des données qui ne sont pas annotés pour les concepts. L'apprentissage des concepts est forcé à l'aide d'une fonction d'apprentissage adaptée. Ainsi, les n\oe uds désignés par la fonction d'apprentissage assimileront des concepts non-supervisés de façon autonome.
\item Les \emph{modèles hybrides} sont à mi chemin entre les deux types de modèles précédents : une partie des neurones est entraînée à l'aide des données annotés et une autre partie apprend des concepts non-supervisés.
\item Les \emph{modèles génératifs} fournissent eux-même une représentation des concepts nécessaires. Ceux-ci sont alignés avec la représentation obtenue de l'entrée, ce qui permet de conclure sur la sortie du modèle. Par exemple, un modèle de langue interne au modèles génère une description des classes possibles en sorties. Elles sont alors encodées afin d'être comparées avec la représentation extraite de l'entrée (par exemple une image). Le modèle génératif s'appuie sur le comparaison de ses deux informations pour classer l'entrée.
\end{itemize}
La particularité des modèles à base de concepts nous permet d'espérer une plus grande transparence et accessibilité du raisonnement impliqué lors de l'utilisation du modèle \cite{poeta_concept-based_2023}. Au delà de l'objectif d'explicabilité, \cite{kim_interpretability_2018, kim_help_2023} espèrent que les raisonnements appris et adoptés par ces modèles soient plus proche du raisonnement humain dans le sens où ils ne seraient pas seulement corrélés mais une reproduction de celui-ci. Cependant, il faut noter que forcer les modèles actuels à raisonner comme des humains revient à contraindre leur apprentissage. Cela alors même que ces modèles ne sont pas des reproductions exactes du cerveau humain mais des outils inspirés de l'idée de neurone. Par conséquent, on peut craindre une difficulté pour ces modèles à atteindre les performances attendues.
\chapter{Explicabilité pour le traitement de la langue}
Comme expliqué précédemment, les grands modèles de langue (LLM) tels que BERT et GPT sont des modèles d'intelligence artificielle boite noire. Au-delà de la complexité de l'association de leurs composants, ces modèles contiennent un très grand nombre de paramètres ce qui rend leur comportement opaque. Cette opacité des LLM est critique dans certains contextes (santé, légal,...), ce qui explique le besoin d'utiliser des méthodes d'explication pour ces modèles. Dans ce chapitre, on s'intéresse aux modèles de langue qui sont utilisés pour les problèmes impliquant des entrées et/ou sortie sous forme de texte en langue naturelle ainsi qu'aux méthodes d'explicabilité appliquées dans ces cas.
\section{Enjeux autour des méthodes d'explicabilité pour le NLP}
\subsection{Généralités}
Les méthodes d'explication présentées dans les chapitres précédents peuvent être appliquées aux LLMs comme à n'importe quel autre modèle ou système. Cependant, \cite{mersha_evaluating_2025} pointe le fait qu'une grande partie d'entre-elles n'ont pas été conçues spécifiquement pour les LLMs et que les explications obtenues ne sont pas toujours aussi fidèles ou adéquates qu'attendu.
% \alh{c'est à dire ? comme nous le verrons peut suffir} Les méthodes en question nécessitent une adaptation.
La conception de nouvelles méthodes pensées spécialement pour les LLM est même présentée comme urgente dans cet article.
Au-delà du caractère adapté ou non des méthodes d'explicabilité génériques, leur scalabilité est aussi remise en question par \cite{garouani_investigating_2024} qui pointe notamment la complexité en temps très importante de l'application des méthodes basées sur SHAP \cite{marzouk_tractability_2024}.
D'un point de vue empirique, certains résultats de travaux sur les méthodes d'explication laissent penser que les performances de ces méthodes dépendent de la tâche et/ou du modèle auquel on l'applique. \cite{chefer_transformer_2021} présente les résultats de sa méthode appliquée à des données visuelles et textuelles. On obtient des "cartes d'influence" qui semblent moins satisfaisantes pour les données textuelles que pour les images (voir la Figure~\ref{fig:textchefer}. Au delà de ce cas, il ne semble pas exister d'étude comparative des performances d'une méthode d'explication sur différentes tâches avec des entrées textuelles et non textuelles.
%\alh{voilà un sujet qui trouverait sa place en conclusion et comme proposition de semestre 2 par exemple}
\subsection{Cas des méthodes à base d'attention}
Parmi les méthodes évoquées précédemment, on retrouve les méthodes basées sur l'attention. Elles sont très répandues dans ce domaine car la majeure parties des modèles utilisent le mécanisme d'attention comme technologie principale. De plus, au delà de l'extraction des matrices d'attention, l'intuition associée à ce mécanisme laisse présager des explications fidèles et intelligibles.
Cependant, les évaluations de ces métriques ne sont pas toujours aussi encourageantes. \cite{mersha_evaluating_2025} obtient une très faible adéquation entre explications humaines et celles obtenues par visualisation de l'attention. Ces méthodes sont même remises en question (voir sous-section~\ref{ssec:attendus}. \\
D'un point de vue technique, \cite{chefer_transformer_2021} souligne le fait que n'exploiter que les matrices d'attention revient à ignorer le reste du modèle c'est-à-dire se priver de composants tout aussi importants que le mécanisme d'attention lors de la construction de l'explication recherchée. Cette publication propose de comparer les prédictions de plusieurs méthodes avec celle issue d'une méthode en combinant plusieurs (voir Figure~\ref{fig:textchefer}). D'un point de vue théorique, \cite{lopardo_attention_2024} incite même à mettre de coté l'idée d'exploiter les matrices d'attention comme outil d'explicabilité de l'IA. Son analyse mathématique d'un petit modèle à base d'attention montre que ces matrices capturent moins d'éléments que des méthodes telles que SHAP \cite{lundberg_unified_2017}.
\begin{figure}
\centering
\includegraphics[width=\textwidth]{images/0_neg_1.pdf}
\caption{Exemple de comparaison proposée par \cite{chefer_transformer_2021}. Cette entrée est un commentaire négatif bien classé par le modèle utilisé. Les explications proposées sont (dans l'ordre) la sélection humaine des mots importants (Ground Truth), l'explication pour la méthode proposée pour la classe négatif (N), l'explication pour la méthode proposée pour la classe positif (P), l'explication de Partial LRP pour la classe N, l'explication de Partial LRP pour la classe P, l'explication de GradCAM pour la classe N, l'explication de GradCAM pour la classe P, l'explication de LRP pour la classe N, l'explication de LRP pour la classe P, l'explication de raw attention et l'explication de rollout.}
\label{fig:textchefer}
\end{figure}
\section{Enjeux autour des métriques d'évaluation de ces explications}
Face aux critiques portées sur les méthodes d'explicabilité appliquées au NLP, on retrouve des critiques concernant les méthodes et les métriques d'évaluation de ces dernières.\\
\cite{neely_song_2022} montre que juger une explication en observant sa corrélation avec celles fournies par des méthodes à attribution (famille d'explication qui regroupe les méthodes à base de gradient, de rétro-propagation de pertinence et à base de perturbation) n'est pas pertinent car les méthodes à attribution elles-mêmes ne sont pas corrélées entre elles.\\
\cite{mersha_evaluating_2025} remet en question la capacité des métriques à base de perturbation à estimer l'efficacité des méthodes d'explications lorsqu'elles sont appliquées à de très grands modèles.
\chapter{Conclusion}
Dans ce rapport et au cours de mon immersion, nous avons vu comment fonctionnent les modèles actuels tels que les modèles basés sur le Transformer et constaté que leur complexité et leur grand nombre de paramètres engendrent une grande opacité. Cette nature de modèle boite-noire est problématique dans certains domaines où il est incontournable de pouvoir accéder aux raisons motivant l'obtention d'une certaine sortie ou d'un certain comportement. \\
Ce besoin de transparence peut être traité de deux façons. La première stratégie est de construire des méthodes qui, en observant le modèle ou son comportement, peuvent donner des indications humainement intelligibles sur les raisons de son comportement ou sa sortie. Ces méthodes se basent sur différentes stratégies et il reste encore difficile de les comparer et de déterminer les tâches et contextes pour lesquels elles sont le plus adaptées. La seconde stratégie vise à construire de nouveaux modèles qui soient transparents tout en restant capables de traiter les mêmes tâches avec la même performance. Il n'est pas simple de construire de tels modèles car même si des outils interprétables sont utilisés lors de leur construction, ce n'est pas pour autant qu'ils restent interprétables lors de l'usage.\\
Dans le contexte du traitement automatique de la langue, on retrouve les mêmes problématiques. Celles-ci sont d'autant plus présentes que les méthodes d'explication ne sont pas toujours aussi efficaces dans ce contexte que dans le cas général et que les métriques d'évaluation ne sont pas toujours adaptées.
Les difficultés rencontrées en explicabilité de l'intelligence artificielle pour le traitement de la langue naturelle mettent en évidence que de nombreuses études et perspectives sont encore possibles. \\
Dans un premier temps, une étude comparative des méthodes d'explication appliquées aux données textuelles et non textuelles (images, tables attribut-valeur, données séquentielles,...) permettrait d'identifier les méthodes d'explication les plus efficaces selon la tâche et le modèle étudié. Cela aidera peut-être à déterminer les besoins de l'explicabilité pour les LLMs qui ne sont pas partagés avec les autres modèles. Ces éléments sont de potentielles pistes pour construire des méthodes d'explication spécifiques aux besoins de l'explicabilité de l'IA pour le NLP. \\
Dans le domaine de l’évaluation des méthodes d’explication, un récapitulatif des attendus et des métriques existantes pourrait mettre en lumière de potentiels éléments manquants dans ce contexte et de les combler. Par ailleurs, il serait intéressant de questionner la comparabilité des modèles appliqués à des textes et des modèles appliqués à des données non-textuelles. Cela orienterait sur un besoin ou non de méthodes d'explication et de métriques d'évaluation qui soient universelles au sens des tâches et modèles.
\chapter{Lexique et acronymes}
\section{Acronymes}
\begin{description}
\item[CNN (Convolutionnal Neural Network)] : Réseau de Neurones Convolutionnel
\item[DNN (Deep Neural Network)] : Réseau de Neurones Profond
\item[IA/ AI (Intelligence Artificielle/ Artificial Intelligence)] : domaine visant à faire adopter à une machine un comportement et/ou un raisonnement rationnel et/ou humain.
\item[LLM (Large Language Model)] : Grand Modèle de Langue
\item[ML (Machine Learning)] : Apprentissage Automatique
\item[NLP/TALN (Natural Language Processing)] : Traitement Automatique de la Langue Naturelle
\item[RGPD (Règlement Générale de Protection des Données)] : réglementation établie par l'union européenne sur la gestion des données personnelles
\item[RNN (Recurrent Neural Network)] : Réseau de Neurones Récurrent
\item[XAI (eXplainable Artificial Intelligence)] : eXplicabilité pour l'Intelligence Artificielle
\end{description}
\section{Lexique}
\begin{description}
\item[Boite-noire] : Modèle ou système dont le raisonnement lors d'une prédiction n'est pas transparent ou humainement compréhensible
\item[Carte d'influence (relevance map)] : Fonction qui à chaque composante (pixel, mot,...) de l'entrée d'un modèle associe un réel reflétant l'importance de ce composant dans le processus de décision du modèle
\item[Entrée] : Donnée fournie au système lors du processus d'inférence
\item[Environnement] : Ensemble de données dont est issue l'entrée fournie à un modèle ou un système
\item[Explicabilité by-design] : méthodes d'explicabilité portant sur la conception de modèles plus transparents
\item[Explicabilité post-hoc] : méthodes d'explicabilité portant sur la conception de méthodes d'extraction de relations entre les entrées et les sorties des modèles/systèmes
\item[Fonction d'apprentissage / fonction de perte] : fonction utilisée pour entraîner le modèle
\item[Inférence] : Application du modèle/système sur une nouvelle donnée pour obtenir une sortie
\item[Intelligence artificielle connexionniste] : Partie du domaine de l'intelligence artificielle exploitant les réseaux de neurones.
\item[Intelligence artificielle symbolique] : Partie du domaine de l'intelligence artificielle portant sur les systèmes experts basés sur des ensembles de règles prédéfinies et des mécanismes de raisonnement logique.
\item[Modèle] : Représentation d'un environnement
\item[Paramètre] : Valeur dans le modèle déterminée lors de l'entraînement et utilisée lors du processus de décision. Les grands modèles de langue en contiennent plusieurs centaines de milliers.
\item[Sortie] : Donnée obtenue par le système à l'issue du processus d'inférence
\item[Système] : Fonction qui à une entrée associe une sortie en respectant une spécification
\end{description}
\bibliographystyle{apalike}
\bibliography{sample.bib,IAAct.bib}
\end{document}\documentclass{report}
rapports_presentations/rapportS9/rapport/sample.bib 0 → 100644
View file @ 69859c8a
@article{raffel_exploring_2020,
title = {Exploring the limits of transfer learning with a unified text-to-text transformer},
volume = {21},
issn = {1532-4435},
abstract = {Transfer learning, where a model is first pre-trained on a data-rich task before being fine-tuned on a downstream task, has emerged as a powerful technique in natural language processing (NLP). The effectiveness of transfer learning has given rise to a diversity of approaches, methodology, and practice. In this paper, we explore the landscape of transfer learning techniques for NLP by introducing a unified framework that converts all text-based language problems into a text-to-text format. Our systematic study compares pretraining objectives, architectures, unlabeled data sets, transfer approaches, and other factors on dozens of language understanding tasks. By combining the insights from our exploration with scale and our new "Colossal Clean Crawled Corpus", we achieve state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more. To facilitate future work on transfer learning for NLP, we release our data set, pre-trained models, and code.},
number = {1},
journal = {J. Mach. Learn. Res.},
author = {Raffel, Colin and Shazeer, Noam and Roberts, Adam and Lee, Katherine and Narang, Sharan and Matena, Michael and Zhou, Yanqi and Li, Wei and Liu, Peter J.},
month = jan,
year = {2020},
keywords = {attention based models, deep learning, multi-task learning, natural language processing, transfer learning},
}
@misc{touvron_llama_2023,
title = {{LLaMA}: {Open} and {Efficient} {Foundation} {Language} {Models}},
shorttitle = {{LLaMA}},
url = {http://arxiv.org/abs/2302.13971},
doi = {10.48550/arXiv.2302.13971},
abstract = {We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible datasets. In particular, LLaMA-13B outperforms GPT-3 (175B) on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. We release all our models to the research community.},
urldate = {2025-12-18},
publisher = {arXiv},
author = {Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timothée and Rozière, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
month = feb,
year = {2023},
note = {arXiv:2302.13971},
keywords = {Computer Science - Computation and Language},
}
@article{johansson_trade-off_2011,
title = {Trade-{Off} {Between} {Accuracy} and {Interpretability} for {Predictive} {In} {Silico} {Modeling}},
volume = {3},
issn = {1756-8919, 1756-8927},
url = {https://www.tandfonline.com/doi/full/10.4155/fmc.11.23},
doi = {10.4155/fmc.11.23},
language = {en},
number = {6},
urldate = {2025-12-15},
journal = {Future Medicinal Chemistry},
author = {Johansson, Ulf and Sönströd, Cecilia and Norinder, Ulf and Boström, Henrik},
month = apr,
year = {2011},
pages = {647--663},
}
@inproceedings{marzouk_tractability_2024,
series = {{ICML}'24},
title = {On the tractability of {SHAP} explanations under {Markovian} distributions},
abstract = {Thanks to its solid theoretical foundation, the SHAP framework is arguably one the most widely utilized frameworks for local explainability of ML models. Despite its popularity, its exact computation is known to be very challenging, proven to be NP-Hard in various configurations. Recent works have unveiled positive complexity results regarding the computation of the SHAP score for specific model families, encompassing decision trees, random forests, and some classes of boolean circuits. Yet, all these positive results hinge on the assumption of feature independence, often simplistic in real-world scenarios. In this article, we investigate the computational complexity of the SHAP score by relaxing this assumption and introducing a Markovian perspective. We show that, under the Markovian assumption, computing the SHAP score for the class of Weighted automata, Disjoint DNFs and Decision Trees can be performed in polynomial time, offering a first positive complexity result for the problem of SHAP score computation that transcends the limitations of the feature independence assumption},
booktitle = {Proceedings of the 41st {International} {Conference} on {Machine} {Learning}},
publisher = {JMLR.org},
author = {Marzouk, Reda and De La Higuera, Colin},
year = {2024},
note = {event-place: Vienna, Austria},
}
@inproceedings{kim_help_2023,
address = {Hamburg Germany},
title = {"{Help} {Me} {Help} the {AI}": {Understanding} {How} {Explainability} {Can} {Support} {Human}-{AI} {Interaction}},
isbn = {9781450394215},
shorttitle = {"{Help} {Me} {Help} the {AI}"},
url = {https://dl.acm.org/doi/10.1145/3544548.3581001},
doi = {10.1145/3544548.3581001},
language = {en},
urldate = {2025-12-11},
booktitle = {Proceedings of the 2023 {CHI} {Conference} on {Human} {Factors} in {Computing} {Systems}},
publisher = {ACM},
author = {Kim, Sunnie S. Y. and Watkins, Elizabeth Anne and Russakovsky, Olga and Fong, Ruth and Monroy-Hernández, Andrés},
month = apr,
year = {2023},
pages = {1--17},
}
@inproceedings{kim_interpretability_2018,
title = {Interpretability {Beyond} {Feature} {Attribution}: {Quantitative} {Testing} with {Concept} {Activation} {Vectors} ({TCAV})},
shorttitle = {Interpretability {Beyond} {Feature} {Attribution}},
url = {https://proceedings.mlr.press/v80/kim18d.html},
abstract = {The interpretation of deep learning models is a challenge due to their size, complexity, and often opaque internal state. In addition, many systems, such as image classifiers, operate on low-level features rather than high-level concepts. To address these challenges, we introduce Concept Activation Vectors (CAVs), which provide an interpretation of a neural net’s internal state in terms of human-friendly concepts. The key idea is to view the high-dimensional internal state of a neural net as an aid, not an obstacle. We show how to use CAVs as part of a technique, Testing with CAVs (TCAV), that uses directional derivatives to quantify the degree to which a user-defined concept is important to a classification result–for example, how sensitive a prediction of “zebra” is to the presence of stripes. Using the domain of image classification as a testing ground, we describe how CAVs may be used to explore hypotheses and generate insights for a standard image classification network as well as a medical application.},
language = {en},
urldate = {2025-12-11},
booktitle = {Proceedings of the 35th {International} {Conference} on {Machine} {Learning}},
publisher = {PMLR},
author = {Kim, Been and Wattenberg, Martin and Gilmer, Justin and Cai, Carrie and Wexler, James and Viegas, Fernanda and Sayres, Rory},
month = jul,
year = {2018},
pages = {2668--2677},
}
@inproceedings{poursabzi-sangdeh_manipulating_2021,
address = {Yokohama Japan},
title = {Manipulating and {Measuring} {Model} {Interpretability}},
isbn = {9781450380966},
url = {https://dl.acm.org/doi/10.1145/3411764.3445315},
doi = {10.1145/3411764.3445315},
language = {en},
urldate = {2025-12-11},
booktitle = {Proceedings of the 2021 {CHI} {Conference} on {Human} {Factors} in {Computing} {Systems}},
publisher = {ACM},
author = {Poursabzi-Sangdeh, Forough and Goldstein, Daniel G and Hofman, Jake M and Wortman Vaughan, Jennifer Wortman and Wallach, Hanna},
month = may,
year = {2021},
pages = {1--52},
}
@article{ciatto_symbolic_2024,
title = {Symbolic {Knowledge} {Extraction} and {Injection} with {Sub}-symbolic {Predictors}: {A} {Systematic} {Literature} {Review}},
volume = {56},
issn = {0360-0300, 1557-7341},
shorttitle = {Symbolic {Knowledge} {Extraction} and {Injection} with {Sub}-symbolic {Predictors}},
url = {https://dl.acm.org/doi/10.1145/3645103},
doi = {10.1145/3645103},
abstract = {In this article, we focus on the opacity issue of sub-symbolic machine learning predictors by promoting two complementary activities—
symbolic knowledge extraction
(SKE) and
symbolic knowledge injection
(SKI)—from and into sub-symbolic predictors. We consider as symbolic any language being intelligible and interpretable for both humans and computers. Accordingly, we propose general meta-models for both SKE and SKI, along with two taxonomies for the classification of SKE and SKI methods. By adopting an explainable artificial intelligence (XAI) perspective, we highlight how such methods can be exploited to mitigate the aforementioned opacity issue. Our taxonomies are attained by surveying and classifying existing methods from the literature, following a systematic approach, and by generalising the results of previous surveys targeting specific sub-topics of either SKE or SKI alone. More precisely, we analyse 132 methods for SKE and 117 methods for SKI, and we categorise them according to their purpose, operation, expected input/output data and predictor types. For each method, we also indicate the presence/lack of runnable software implementations. Our work may be of interest for data scientists aiming at selecting the most adequate SKE/SKI method for their needs, and may also work as suggestions for researchers interested in filling the gaps of the current state-of-the-art as well as for developers willing to implement SKE/SKI-based technologies.},
language = {en},
number = {6},
urldate = {2025-12-11},
journal = {ACM Computing Surveys},
author = {Ciatto, Giovanni and Sabbatini, Federico and Agiollo, Andrea and Magnini, Matteo and Omicini, Andrea},
month = jun,
year = {2024},
pages = {1--35},
}
@article{bhuyan_neuro-symbolic_2024,
title = {Neuro-symbolic artificial intelligence: a survey},
volume = {36},
issn = {1433-3058},
shorttitle = {Neuro-symbolic artificial intelligence},
url = {https://doi.org/10.1007/s00521-024-09960-z},
doi = {10.1007/s00521-024-09960-z},
abstract = {The goal of the growing discipline of neuro-symbolic artificial intelligence (AI) is to develop AI systems with more human-like reasoning capabilities by combining symbolic reasoning with connectionist learning. We survey the literature on neuro-symbolic AI during the last two decades, including books, monographs, review papers, contribution pieces, opinion articles, foundational workshops/talks, and related PhD theses. Four main features of neuro-symbolic AI are discussed, including representation, learning, reasoning, and decision-making. Finally, we discuss the many applications of neuro-symbolic AI, including question answering, robotics, computer vision, healthcare, and more. Scalability, explainability, and ethical considerations are also covered, as well as other difficulties and limits of neuro-symbolic AI. This study summarizes the current state of the art in neuro-symbolic artificial intelligence.},
language = {en},
number = {21},
urldate = {2025-12-10},
journal = {Neural Computing and Applications},
author = {Bhuyan, Bikram Pratim and Ramdane-Cherif, Amar and Tomar, Ravi and Singh, T. P.},
month = jul,
year = {2024},
keywords = {Artificial intelligence, Knowledge representation and reasoning, Machine learning, Neural networks, Neuro-symbolic artificial intelligence, Spatial-temporal data},
pages = {12809--12844},
}
@misc{ferrer_comment_2024,
title = {Comment fonctionnent les transformateurs : {Exploration} détaillée de l'architecture des transformateurs},
url = {https://www.datacamp.com/fr/tutorial/how-transformers-work},
urldate = {2024-10-04},
author = {Ferrer, Josep},
month = oct,
year = {2024},
}
@incollection{neely_song_2022,
title = {A {Song} of ({Dis})agreement: {Evaluating} the {Evaluation} of {Explainable} {Artificial} {Intelligence} in {Natural} {Language} {Processing}},
shorttitle = {A {Song} of ({Dis})agreement},
url = {https://ebooks.iospress.nl/doi/10.3233/FAIA220190},
language = {en},
urldate = {2025-12-10},
booktitle = {{HHAI2022}: {Augmenting} {Human} {Intellect}},
publisher = {IOS Press},
author = {Neely, Michael and Schouten, Stefan F. and Bleeker, Maurits and Lucic, Ana},
year = {2022},
doi = {10.3233/FAIA220190},
pages = {60--78},
}
@book{russell_intelligence_2021,
address = {Paris},
edition = {4e éd},
title = {Intelligence artificielle: une approche moderne},
isbn = {9782326002210},
shorttitle = {Intelligence artificielle},
language = {fre},
publisher = {Pearson},
author = {Russell, Stuart Jonathan and Norvig, Peter and Popineau, Fabrice and Miclet, Laurent and Cadet, Claire},
year = {2021},
}
@inproceedings{kastner_relation_2021,
title = {On the {Relation} of {Trust} and {Explainability}: {Why} to {Engineer} for {Trustworthiness}},
shorttitle = {On the {Relation} of {Trust} and {Explainability}},
url = {https://ieeexplore.ieee.org/document/9582305},
doi = {10.1109/REW53955.2021.00031},
abstract = {Recently, requirements for the explainability of software systems have gained prominence. One of the primary motivators for such requirements is that explainability is expected to facilitate stakeholders’ trust in a system. Although this seems intuitively appealing, recent psychological studies indicate that explanations do not necessarily facilitate trust. Thus, explainability requirements might not be suitable for promoting trust.One way to accommodate this finding is, we suggest, to focus on trustworthiness instead of trust. While these two may come apart, we ideally want both: a trustworthy system and the stakeholder’s trust. In this paper, we argue that even though trustworthiness does not automatically lead to trust, there are several reasons to engineer primarily for trustworthiness – and that a system’s explainability can crucially contribute to its trustworthiness.},
urldate = {2025-12-08},
booktitle = {2021 {IEEE} 29th {International} {Requirements} {Engineering} {Conference} {Workshops} ({REW})},
author = {Kästner, Lena and Langer, Markus and Lazar, Veronika and Schomäcker, Astrid and Speith, Timo and Sterz, Sarah},
month = sep,
year = {2021},
keywords = {Conferences, Explainability, NFR, Psychology, Requirements, Requirements engineering, Software systems, Stakeholders, Trust, Trustworthiness, XAI},
pages = {169--175},
}
@inproceedings{deters_x_2024,
address = {Uppsala Sweden},
title = {The {X} {Factor}: {On} the {Relationship} between {User} {eXperience} and {eXplainability}},
isbn = {9798400709661},
shorttitle = {The {X} {Factor}},
url = {https://dl.acm.org/doi/10.1145/3679318.3685352},
doi = {10.1145/3679318.3685352},
language = {en},
urldate = {2025-12-08},
booktitle = {Nordic {Conference} on {Human}-{Computer} {Interaction}},
publisher = {ACM},
author = {Deters, Hannah and Droste, Jakob and Hess, Anne and Klös, Verena and Schneider, Kurt and Speith, Timo and Vogelsang, Andreas},
month = oct,
year = {2024},
pages = {1--12},
}
@article{lin_survey_2022,
title = {A survey of transformers},
volume = {3},
issn = {2666-6510},
url = {https://www.sciencedirect.com/science/article/pii/S2666651022000146},
doi = {10.1016/j.aiopen.2022.10.001},
abstract = {Transformers have achieved great success in many artificial intelligence fields, such as natural language processing, computer vision, and audio processing. Therefore, it is natural to attract lots of interest from academic and industry researchers. Up to the present, a great variety of Transformer variants (a.k.a. X-formers) have been proposed, however, a systematic and comprehensive literature review on these Transformer variants is still missing. In this survey, we provide a comprehensive review of various X-formers. We first briefly introduce the vanilla Transformer and then propose a new taxonomy of X-formers. Next, we introduce the various X-formers from three perspectives: architectural modification, pre-training, and applications. Finally, we outline some potential directions for future research.},
urldate = {2025-12-08},
journal = {AI Open},
author = {Lin, Tianyang and Wang, Yuxin and Liu, Xiangyang and Qiu, Xipeng},
month = jan,
year = {2022},
keywords = {Deep learning, Pre-trained models, Self-attention, Transformer},
pages = {111--132},
}
@inproceedings{delobelle_robbert_2020,
address = {Online},
title = {{RobBERT}: a {Dutch} {RoBERTa}-based {Language} {Model}},
shorttitle = {{RobBERT}},
url = {https://www.aclweb.org/anthology/2020.findings-emnlp.292},
doi = {10.18653/v1/2020.findings-emnlp.292},
language = {en},
urldate = {2025-12-08},
booktitle = {Findings of the {Association} for {Computational} {Linguistics}: {EMNLP} 2020},
publisher = {Association for Computational Linguistics},
author = {Delobelle, Pieter and Winters, Thomas and Berendt, Bettina},
year = {2020},
pages = {3255--3265},
}
@inproceedings{aftan_survey_2023,
title = {A {Survey} on {BERT} and {Its} {Applications}},
url = {https://ieeexplore.ieee.org/document/10092289/references#references},
doi = {10.1109/LT58159.2023.10092289},
abstract = {A recently developed language representation model named Bidirectional Encoder Representation from Transformers (BERT) is based on an advanced trained deep learning approach that has achieved excellent results in many complex tasks, the same as classification, Natural Language Processing (NLP), prediction, etc. This survey paper mainly adopts the summary of BERT, its multiple types, and its latest developments and applications in various computer science and engineering fields. Furthermore, it puts forward BERT's problems and attractive future research trends in a different area with multiple datasets. From the findings, overall, the BERT and their recent types have achieved more accurate, fast, and optimal results in solving most complex problems than typical Machine and Deep Learning methods.},
urldate = {2025-12-08},
booktitle = {2023 20th {Learning} and {Technology} {Conference} ({L}\&{T})},
author = {Aftan, Sulaiman and Shah, Habib},
month = jan,
year = {2023},
keywords = {BERT, Bit error rate, Computer science, Deep learning, Machine Learning, Natural Language Processing model, Predictive models, Text analysis, Text mining, Transformers, bidirectional encoder},
pages = {161--166},
}
@misc{wahab_dibert_2021,
title = {{DIBERT}: {Dependency} {Injected} {Bidirectional} {Encoder} {Representations} from {Transformers}},
copyright = {https://creativecommons.org/licenses/by/4.0/},
shorttitle = {{DIBERT}},
url = {https://www.techrxiv.org/doi/full/10.36227/techrxiv.16444611.v2},
doi = {10.36227/techrxiv.16444611.v2},
abstract = {{\textless}div{\textgreater}
{\textless}div{\textgreater}
{\textless}div{\textgreater}
{\textless}p{\textgreater}
{\textless}/p{\textgreater}{\textless}div{\textgreater}
{\textless}div{\textgreater}
{\textless}div{\textgreater}
{\textless}p{\textgreater}In this paper, we propose a new model named DIBERT
which stands for Dependency Injected Bidirectional Encoder
Representations from Transformers. DIBERT is a variation of
the BERT and has an additional third objective called Parent
Prediction (PP) apart from Masked Language Modeling (MLM)
and Next Sentence Prediction (NSP). PP injects the syntactic
structure of a dependency tree while pre-training the DIBERT
which generates syntax-aware generic representations. We use
the WikiText-103 benchmark dataset to pre-train both BERT-
Base and DIBERT. After fine-tuning, we observe that DIBERT
performs better than BERT-Base on various downstream tasks
including Semantic Similarity, Natural Language Inference and
Sentiment Analysis. {\textless}/p{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}
{\textless}/div{\textgreater}},
urldate = {2025-12-08},
author = {Wahab, Abdul and Sifa, Rafet},
month = oct,
year = {2021},
}
@article{lee_biobert_2020,
title = {{BioBERT}: a pre-trained biomedical language representation model for biomedical text mining},
volume = {36},
copyright = {http://creativecommons.org/licenses/by/4.0/},
issn = {1367-4803, 1367-4811},
shorttitle = {{BioBERT}},
url = {https://academic.oup.com/bioinformatics/article/36/4/1234/5566506},
doi = {10.1093/bioinformatics/btz682},
abstract = {Abstract
Motivation
Biomedical text mining is becoming increasingly important as the number of biomedical documents rapidly grows. With the progress in natural language processing (NLP), extracting valuable information from biomedical literature has gained popularity among researchers, and deep learning has boosted the development of effective biomedical text mining models. However, directly applying the advancements in NLP to biomedical text mining often yields unsatisfactory results due to a word distribution shift from general domain corpora to biomedical corpora. In this article, we investigate how the recently introduced pre-trained language model BERT can be adapted for biomedical corpora.
Results
We introduce BioBERT (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining), which is a domain-specific language representation model pre-trained on large-scale biomedical corpora. With almost the same architecture across tasks, BioBERT largely outperforms BERT and previous state-of-the-art models in a variety of biomedical text mining tasks when pre-trained on biomedical corpora. While BERT obtains performance comparable to that of previous state-of-the-art models, BioBERT significantly outperforms them on the following three representative biomedical text mining tasks: biomedical named entity recognition (0.62\% F1 score improvement), biomedical relation extraction (2.80\% F1 score improvement) and biomedical question answering (12.24\% MRR improvement). Our analysis results show that pre-training BERT on biomedical corpora helps it to understand complex biomedical texts.
Availability and implementation
We make the pre-trained weights of BioBERT freely available at https://github.com/naver/biobert-pretrained, and the source code for fine-tuning BioBERT available at https://github.com/dmis-lab/biobert.},
language = {en},
number = {4},
urldate = {2025-12-08},
journal = {Bioinformatics},
author = {Lee, Jinhyuk and Yoon, Wonjin and Kim, Sungdong and Kim, Donghyeon and Kim, Sunkyu and So, Chan Ho and Kang, Jaewoo},
editor = {Wren, Jonathan},
month = feb,
year = {2020},
pages = {1234--1240},
}
@misc{sanh_distilbert_2020,
title = {{DistilBERT}, a distilled version of {BERT}: smaller, faster, cheaper and lighter},
shorttitle = {{DistilBERT}, a distilled version of {BERT}},
url = {http://arxiv.org/abs/1910.01108},
doi = {10.48550/arXiv.1910.01108},
abstract = {As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging. In this work, we propose a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can then be fine-tuned with good performances on a wide range of tasks like its larger counterparts. While most prior work investigated the use of distillation for building task-specific models, we leverage knowledge distillation during the pre-training phase and show that it is possible to reduce the size of a BERT model by 40\%, while retaining 97\% of its language understanding capabilities and being 60\% faster. To leverage the inductive biases learned by larger models during pre-training, we introduce a triple loss combining language modeling, distillation and cosine-distance losses. Our smaller, faster and lighter model is cheaper to pre-train and we demonstrate its capabilities for on-device computations in a proof-of-concept experiment and a comparative on-device study.},
urldate = {2025-12-08},
publisher = {arXiv},
author = {Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},
month = mar,
year = {2020},
note = {arXiv:1910.01108},
keywords = {Computer Science - Computation and Language},
}
@inproceedings{martin_camembert_2020,
address = {Online},
title = {{CamemBERT}: a {Tasty} {French} {Language} {Model}},
shorttitle = {{CamemBERT}},
url = {https://www.aclweb.org/anthology/2020.acl-main.645},
doi = {10.18653/v1/2020.acl-main.645},
language = {en},
urldate = {2025-12-08},
booktitle = {Proceedings of the 58th {Annual} {Meeting} of the {Association} for {Computational} {Linguistics}},
publisher = {Association for Computational Linguistics},
author = {Martin, Louis and Muller, Benjamin and Ortiz Suárez, Pedro Javier and Dupont, Yoann and Romary, Laurent and De La Clergerie, Eric and Seddah, Djamé and Sagot, Benoît},
year = {2020},
pages = {7203--7219},
}
@inproceedings{douka_juribert_2021,
address = {Punta Cana, Dominican Republic},
title = {{JuriBERT}: {A} {Masked}-{Language} {Model} {Adaptation} for {French} {Legal} {Text}},
shorttitle = {{JuriBERT}},
url = {https://aclanthology.org/2021.nllp-1.9/},
doi = {10.18653/v1/2021.nllp-1.9},
abstract = {Language models have proven to be very useful when adapted to specific domains. Nonetheless, little research has been done on the adaptation of domain-specific BERT models in the French language. In this paper, we focus on creating a language model adapted to French legal text with the goal of helping law professionals. We conclude that some specific tasks do not benefit from generic language models pre-trained on large amounts of data. We explore the use of smaller architectures in domain-specific sub-languages and their benefits for French legal text. We prove that domain-specific pre-trained models can perform better than their equivalent generalised ones in the legal domain. Finally, we release JuriBERT, a new set of BERT models adapted to the French legal domain.},
urldate = {2025-12-08},
booktitle = {Proceedings of the {Natural} {Legal} {Language} {Processing} {Workshop} 2021},
publisher = {Association for Computational Linguistics},
author = {Douka, Stella and Abdine, Hadi and Vazirgiannis, Michalis and El Hamdani, Rajaa and Restrepo Amariles, David},
editor = {Aletras, Nikolaos and Androutsopoulos, Ion and Barrett, Leslie and Goanta, Catalina and Preotiuc-Pietro, Daniel},
month = nov,
year = {2021},
pages = {95--101},
}
@inproceedings{segonne_jargon_2024,
title = {Jargon: {A} {Suite} of {Language} {Models} and {Evaluation} {Tasks} for {French} {Specialized} {Domains}},
shorttitle = {Jargon},
url = {https://hal.science/hal-04535557},
abstract = {Pretrained Language Models (PLMs) are the de facto backbone of most state-of-the-art NLP systems. In this paper, we introduce a family of domain-specific pretrained PLMs for French, focusing on three important domains: transcribed speech, medicine, and law. We use a transformer architecture based on efficient methods (LinFormer) to maximise their utility, since these domains often involve processing long documents. We evaluate and compare our models to state-of-the-art models on a diverse set of tasks and datasets, some of which are introduced in this paper. We gather the datasets into a new French-language evaluation benchmark for these three domains. We also compare various training configurations: continued pretraining, pretraining from scratch, as well as single- and multi-domain pretraining. Extensive domain-specific experiments show that it is possible to attain competitive downstream performance even when pre-training with the approximative LinFormer attention mechanism. For full reproducibility, we release the models and pretraining data, as well as contributed datasets.},
language = {en},
urldate = {2025-12-08},
author = {Segonne, Vincent and Mannion, Aidan and Canul, Laura Cristina Alonzo and Audibert, Alexandre and Liu, Xingyu and Macaire, Cécile and Pupier, Adrien and Zhou, Yongxin and Aguiar, Mathilde and Herron, Felix and Norré, Magali and Amini, Massih-Reza and Bouillon, Pierrette and Eshkol-Taravella, Iris and Esperança-Rodier, Emmanuelle and François, Thomas and Goeuriot, Lorraine and Goulian, Jérôme and Lafourcade, Mathieu and Lecouteux, Benjamin and Portet, François and Ringeval, Fabien and Vandeghinste, Vincent and Coavoux, Maximin and Dinarelli, Marco and Schwab, Didier},
month = may,
year = {2024},
pages = {9463},
}
@misc{antoun_arabert_2021,
title = {{AraBERT}: {Transformer}-based {Model} for {Arabic} {Language} {Understanding}},
shorttitle = {{AraBERT}},
url = {http://arxiv.org/abs/2003.00104},
doi = {10.48550/arXiv.2003.00104},
abstract = {The Arabic language is a morphologically rich language with relatively few resources and a less explored syntax compared to English. Given these limitations, Arabic Natural Language Processing (NLP) tasks like Sentiment Analysis (SA), Named Entity Recognition (NER), and Question Answering (QA), have proven to be very challenging to tackle. Recently, with the surge of transformers based models, language-specific BERT based models have proven to be very efficient at language understanding, provided they are pre-trained on a very large corpus. Such models were able to set new standards and achieve state-of-the-art results for most NLP tasks. In this paper, we pre-trained BERT specifically for the Arabic language in the pursuit of achieving the same success that BERT did for the English language. The performance of AraBERT is compared to multilingual BERT from Google and other state-of-the-art approaches. The results showed that the newly developed AraBERT achieved state-of-the-art performance on most tested Arabic NLP tasks. The pretrained araBERT models are publicly available on https://github.com/aub-mind/arabert hoping to encourage research and applications for Arabic NLP.},
urldate = {2025-12-08},
publisher = {arXiv},
author = {Antoun, Wissam and Baly, Fady and Hajj, Hazem},
month = mar,
year = {2021},
note = {arXiv:2003.00104},
keywords = {Computer Science - Computation and Language},
}
@inproceedings{vig_multiscale_2019,
address = {Florence, Italy},
title = {A {Multiscale} {Visualization} of {Attention} in the {Transformer} {Model}},
url = {https://aclanthology.org/P19-3007/},
doi = {10.18653/v1/P19-3007},
abstract = {The Transformer is a sequence model that forgoes traditional recurrent architectures in favor of a fully attention-based approach. Besides improving performance, an advantage of using attention is that it can also help to interpret a model by showing how the model assigns weight to different input elements. However, the multi-layer, multi-head attention mechanism in the Transformer model can be difficult to decipher. To make the model more accessible, we introduce an open-source tool that visualizes attention at multiple scales, each of which provides a unique perspective on the attention mechanism. We demonstrate the tool on BERT and OpenAI GPT-2 and present three example use cases: detecting model bias, locating relevant attention heads, and linking neurons to model behavior.},
urldate = {2025-12-04},
booktitle = {Proceedings of the 57th {Annual} {Meeting} of the {Association} for {Computational} {Linguistics}: {System} {Demonstrations}},
publisher = {Association for Computational Linguistics},
author = {Vig, Jesse},
editor = {Costa-jussà, Marta R. and Alfonseca, Enrique},
month = jul,
year = {2019},
pages = {37--42},
}
@article{kotipalli_role_2024,
title = {The {Role} of {Attention} {Mechanisms} in {Enhancing} {Transparency} and {Interpretability} of {Neural} {Network} {Models} in {Explainable} {AI}},
url = {https://digitalcommons.harrisburgu.edu/cgi/viewcontent.cgi?params=/context/dandt/article/1000/&path_info=Bhargav_Report_Final.pdf},
abstract = {In the rapidly evolving field of artificial intelligence (AI), deep learning models' interpretability and reliability are severely hindered by their complexity and opacity. Enhancing the transparency and interpretability of AI systems for humans is the primary objective of the emerging field of explainable AI (XAI). The attention mechanisms at the heart of XAI's work are based on human cognitive processes. Neural networks can now dynamically focus on relevant parts of the input data thanks to these mechanisms, which enhances interpretability and performance. This report covers in-depth talks of attention mechanisms in neural networks within XAI, as well as an analysis of the theoretical foundations, architectural applications, and empirical evidence showing how well they work to improve model transparency. The report provides a comprehensive analysis of the role of attention mechanisms in AI models to address ethical concerns, comply with regulatory requirements, and foster a deeper understanding and trust in AI systems. The report contributes to the discussion about bringing AI closer to human values and cognitive processes so that its advancements are impactful and responsible by conducting a thorough analysis.},
language = {en},
author = {Kotipalli, Bhargav},
month = apr,
year = {2024},
}
@inproceedings{shrikumar_not_2017,
address = {Sydney, Australia},
title = {Not {Just} a {Black} {Box}: {Learning} {Important} {Features} {Through} {Propagating} {Activation} {Differences}},
volume = {70},
shorttitle = {Not {Just} a {Black} {Box}},
url = {http://arxiv.org/abs/1605.01713},
doi = {10.48550/arXiv.1605.01713},
abstract = {Note: This paper describes an older version of DeepLIFT. See https://arxiv.org/abs/1704.02685 for the newer version. Original abstract follows: The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Learning Important FeaTures), an efficient and effective method for computing importance scores in a neural network. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. We apply DeepLIFT to models trained on natural images and genomic data, and show significant advantages over gradient-based methods.},
urldate = {2025-12-04},
publisher = {Proceedings of the 34th
International Conference on Machine Learning},
author = {Shrikumar, Avanti and Greenside, Peyton and Shcherbina, Anna and Kundaje, Anshul},
month = apr,
year = {2017},
note = {arXiv:1605.01713},
keywords = {Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Neural and Evolutionary Computing},
pages = {3145 -- 3153},
}
@inproceedings{selvaraju_grad-cam_2017,
title = {Grad-{CAM}: {Visual} {Explanations} from {Deep} {Networks} via {Gradient}-{Based} {Localization}},
shorttitle = {Grad-{CAM}},
url = {https://ieeexplore.ieee.org/document/8237336},
doi = {10.1109/ICCV.2017.74},
abstract = {We propose a technique for producing `visual explanations' for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent. Our approach - Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say logits for `dog' or even a caption), flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad- CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g. VGG), (2) CNNs used for structured outputs (e.g. captioning), (3) CNNs used in tasks with multi-modal inputs (e.g. visual question answering) or reinforcement learning, without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are more faithful to the underlying model, and (d) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show even non-attention based models can localize inputs. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a `stronger' deep network from a `weaker' one even when both make identical predictions. Our code is available at https: //github.com/ramprs/grad-cam/ along with a demo on CloudCV [2] and video at youtu.be/COjUB9Izk6E.},
urldate = {2025-12-04},
booktitle = {2017 {IEEE} {International} {Conference} on {Computer} {Vision} ({ICCV})},
author = {Selvaraju, Ramprasaath R. and Cogswell, Michael and Das, Abhishek and Vedantam, Ramakrishna and Parikh, Devi and Batra, Dhruv},
month = oct,
year = {2017},
note = {ISSN: 2380-7504},
keywords = {Cats, Computer architecture, Dogs, Knowledge discovery, Visualization},
pages = {618--626},
}
@inproceedings{radford_improving_2018,
title = {Improving {Language} {Understanding} by {Generative} {Pre}-{Training}},
url = {https://www.semanticscholar.org/paper/Improving-Language-Understanding-by-Generative-Radford-Narasimhan/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035},
abstract = {Natural language understanding comprises a wide range of diverse tasks such as textual entailment, question answering, semantic similarity assessment, and document classification. Although large unlabeled text corpora are abundant, labeled data for learning these specific tasks is scarce, making it challenging for discriminatively trained models to perform adequately. We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task. In contrast to previous approaches, we make use of task-aware input transformations during fine-tuning to achieve effective transfer while requiring minimal changes to the model architecture. We demonstrate the effectiveness of our approach on a wide range of benchmarks for natural language understanding. Our general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, significantly improving upon the state of the art in 9 out of the 12 tasks studied. For instance, we achieve absolute improvements of 8.9\% on commonsense reasoning (Stories Cloze Test), 5.7\% on question answering (RACE), and 1.5\% on textual entailment (MultiNLI).},
urldate = {2025-12-04},
author = {Radford, Alec and Narasimhan, Karthik and Salimans, Tim and Sutskever, Ilya},
year = {2018},
}
@inproceedings{kalouli_gkr_2018,
address = {New Orleans, Louisiana},
title = {{GKR}: the {Graphical} {Knowledge} {Representation} for semantic parsing},
shorttitle = {{GKR}},
url = {https://aclanthology.org/W18-1304/},
doi = {10.18653/v1/W18-1304},
abstract = {This paper describes the first version of an open-source semantic parser that creates graphical representations of sentences to be used for further semantic processing, e.g. for natural language inference, reasoning and semantic similarity. The Graphical Knowledge Representation which is output by the parser is inspired by the Abstract Knowledge Representation, which separates out conceptual and contextual levels of representation that deal respectively with the subject matter of a sentence and its existential commitments. Our representation is a layered graph with each sub-graph holding different kinds of information, including one sub-graph for concepts and one for contexts. Our first evaluation of the system shows an F-score of 85\% in accurately representing sentences as semantic graphs.},
urldate = {2025-11-26},
booktitle = {Proceedings of the {Workshop} on {Computational} {Semantics} beyond {Events} and {Roles}},
publisher = {Association for Computational Linguistics},
author = {Kalouli, Aikaterini-Lida and Crouch, Richard},
editor = {Blanco, Eduardo and Morante, Roser},
month = jun,
year = {2018},
pages = {27--37},
}
@misc{vaswani_attention_2017,
title = {Attention {Is} {All} {You} {Need}},
url = {http://arxiv.org/abs/1706.03762},
doi = {10.48550/arXiv.1706.03762},
abstract = {The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N. and Kaiser, Lukasz and Polosukhin, Illia},
month = dec,
year = {2017},
note = {arXiv:1706.03762},
keywords = {Computer Science - Computation and Language, Computer Science - Machine Learning},
}
@book{netter_regards_2019,
title = {Regards sur le nouveau droit des données personnelles},
url = {https://hal.science/hal-02357967},
abstract = {Just over 40 years after the entry into force in France of the "informatique et libertés" law, the law on personal data seems to have progressed dramatically. In 2018, the new European regulation (known as the " GDPR ") came into force. A law and then an ordinance adapted French domestic law accordingly. The first part of 2019 was marked by the rise of the sanctions imposed by the CNIL.
Some of the contributions collected in this book address cross-cutting issues related to the new regulation, such as its territorial scope, competition from the American model, the scope of the notion of data controller, the existence of post-mortem rights over the data, or the limits of the principle of transparency in the face of the opacity of predictive algorithms. Others focus on a particular sector, whether it is banking, health, insurance or data on public officials.},
urldate = {2025-11-24},
author = {Netter, Emmanuel and Ndior, Valère and Puyraimond, Jean-Ferdinand and Vergnolle, Suzanne},
editor = {d'Amiens, Centre de droit privé et de sciences criminelles},
month = nov,
year = {2019},
keywords = {algorithmes prédictifs, données de santé, données français des données personnelles, droit américain des données, droit au déréférencement, droit bancaire, droit des assurances, droit européen des données personnelles, extraterritorialité du droit, principe de transparence, souveraineté numérique, transparence de la vie publique},
}
@misc{lundberg_unified_2017,
title = {A {Unified} {Approach} to {Interpreting} {Model} {Predictions}},
url = {http://arxiv.org/abs/1705.07874},
doi = {10.48550/arXiv.1705.07874},
abstract = {Understanding why a model makes a certain prediction can be as crucial as the prediction's accuracy in many applications. However, the highest accuracy for large modern datasets is often achieved by complex models that even experts struggle to interpret, such as ensemble or deep learning models, creating a tension between accuracy and interpretability. In response, various methods have recently been proposed to help users interpret the predictions of complex models, but it is often unclear how these methods are related and when one method is preferable over another. To address this problem, we present a unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations). SHAP assigns each feature an importance value for a particular prediction. Its novel components include: (1) the identification of a new class of additive feature importance measures, and (2) theoretical results showing there is a unique solution in this class with a set of desirable properties. The new class unifies six existing methods, notable because several recent methods in the class lack the proposed desirable properties. Based on insights from this unification, we present new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.},
urldate = {2025-11-19},
publisher = {arXiv},
author = {Lundberg, Scott and Lee, Su-In},
month = nov,
year = {2017},
note = {arXiv:1705.07874},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning},
}
@misc{ribeiro_model-agnostic_2016,
title = {Model-{Agnostic} {Interpretability} of {Machine} {Learning}},
url = {http://arxiv.org/abs/1606.05386},
doi = {10.48550/arXiv.1606.05386},
abstract = {Understanding why machine learning models behave the way they do empowers both system designers and end-users in many ways: in model selection, feature engineering, in order to trust and act upon the predictions, and in more intuitive user interfaces. Thus, interpretability has become a vital concern in machine learning, and work in the area of interpretable models has found renewed interest. In some applications, such models are as accurate as non-interpretable ones, and thus are preferred for their transparency. Even when they are not accurate, they may still be preferred when interpretability is of paramount importance. However, restricting machine learning to interpretable models is often a severe limitation. In this paper we argue for explaining machine learning predictions using model-agnostic approaches. By treating the machine learning models as black-box functions, these approaches provide crucial flexibility in the choice of models, explanations, and representations, improving debugging, comparison, and interfaces for a variety of users and models. We also outline the main challenges for such methods, and review a recently-introduced model-agnostic explanation approach (LIME) that addresses these challenges.},
urldate = {2025-11-19},
publisher = {arXiv},
author = {Ribeiro, Marco Tulio and Singh, Sameer and Guestrin, Carlos},
month = jun,
year = {2016},
note = {arXiv:1606.05386},
keywords = {Computer Science - Machine Learning, Statistics - Machine Learning},
}
@article{rudin_stop_2019,
title = {Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead},
volume = {1},
copyright = {2019 Springer Nature Limited},
issn = {2522-5839},
url = {https://www.nature.com/articles/s42256-019-0048-x},
doi = {10.1038/s42256-019-0048-x},
abstract = {Black box machine learning models are currently being used for high-stakes decision making throughout society, causing problems in healthcare, criminal justice and other domains. Some people hope that creating methods for explaining these black box models will alleviate some of the problems, but trying to explain black box models, rather than creating models that are interpretable in the first place, is likely to perpetuate bad practice and can potentially cause great harm to society. The way forward is to design models that are inherently interpretable. This Perspective clarifies the chasm between explaining black boxes and using inherently interpretable models, outlines several key reasons why explainable black boxes should be avoided in high-stakes decisions, identifies challenges to interpretable machine learning, and provides several example applications where interpretable models could potentially replace black box models in criminal justice, healthcare and computer vision.},
language = {en},
number = {5},
urldate = {2025-11-14},
journal = {Nature Machine Intelligence},
author = {Rudin, Cynthia},
month = may,
year = {2019},
keywords = {Computer science, Criminology, Science, Statistics, technology and society},
pages = {206--215},
}
@article{montavon_explaining_2017,
title = {Explaining nonlinear classification decisions with deep {Taylor} decomposition},
volume = {65},
issn = {0031-3203},
url = {https://www.sciencedirect.com/science/article/pii/S0031320316303582},
doi = {10.1016/j.patcog.2016.11.008},
abstract = {Nonlinear methods such as Deep Neural Networks (DNNs) are the gold standard for various challenging machine learning problems such as image recognition. Although these methods perform impressively well, they have a significant disadvantage, the lack of transparency, limiting the interpretability of the solution and thus the scope of application in practice. Especially DNNs act as black boxes due to their multilayer nonlinear structure. In this paper we introduce a novel methodology for interpreting generic multilayer neural networks by decomposing the network classification decision into contributions of its input elements. Although our focus is on image classification, the method is applicable to a broad set of input data, learning tasks and network architectures. Our method called deep Taylor decomposition efficiently utilizes the structure of the network by backpropagating the explanations from the output to the input layer. We evaluate the proposed method empirically on the MNIST and ILSVRC data sets.},
urldate = {2025-11-07},
journal = {Pattern Recognition},
author = {Montavon, Grégoire and Lapuschkin, Sebastian and Binder, Alexander and Samek, Wojciech and Müller, Klaus-Robert},
month = may,
year = {2017},
keywords = {Deep neural networks, Heatmapping, Image recognition, Relevance propagation, Taylor decomposition},
pages = {211--222},
}
@book{molnar_interpretable_2025,
title = {Interpretable {Machine} {Learning}},
url = {https://christophm.github.io/interpretable-ml-book/},
urldate = {2025-09-11},
author = {Molnar, Christoph},
month = mar,
year = {2025},
}
@misc{abnar_quantifying_2020,
title = {Quantifying {Attention} {Flow} in {Transformers}},
url = {http://arxiv.org/abs/2005.00928},
doi = {10.48550/arXiv.2005.00928},
abstract = {In the Transformer model, "self-attention" combines information from attended embeddings into the representation of the focal embedding in the next layer. Thus, across layers of the Transformer, information originating from different tokens gets increasingly mixed. This makes attention weights unreliable as explanations probes. In this paper, we consider the problem of quantifying this flow of information through self-attention. We propose two methods for approximating the attention to input tokens given attention weights, attention rollout and attention flow, as post hoc methods when we use attention weights as the relative relevance of the input tokens. We show that these methods give complementary views on the flow of information, and compared to raw attention, both yield higher correlations with importance scores of input tokens obtained using an ablation method and input gradients.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Abnar, Samira and Zuidema, Willem},
month = may,
year = {2020},
note = {arXiv:2005.00928},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning},
}
@phdthesis{elguendouze_explainable_2024,
type = {thesis},
title = {Explainable {Artificial} {Intelligence} approaches for {Image} {Captioning}},
url = {https://theses.fr/2024ORLE1003},
abstract = {L'évolution rapide des modèles de sous-titrage d'images, impulsée par l'intégration de techniques d'apprentissage profond combinant les modalités image et texte, a conduit à des systèmes de plus en plus complexes. Cependant, ces modèles fonctionnent souvent comme des boîtes noires, incapables de fournir des explications transparentes de leurs décisions. Cette thèse aborde l'explicabilité des systèmes de sous-titrage d'images basés sur des architectures Encodeur-Attention-Décodeur, et ce à travers quatre aspects. Premièrement, elle explore le concept d'espace latent, s'éloignant ainsi des approches traditionnelles basées sur l'espace de représentation originel. Deuxièmement, elle présente la notion de caractère décisif, conduisant à la formulation d'une nouvelle définition pour le concept d'influence/décisivité des composants dans le contexte de sous-titrage d'images explicable, ainsi qu'une approche par perturbation pour la capture du caractère décisif. Le troisième aspect vise à élucider les facteurs influençant la qualité des explications, en mettant l'accent sur la portée des méthodes d'explication. En conséquence, des variantes basées sur l'espace latent de méthodes d'explication bien établies telles que LRP et LIME ont été développées, ainsi que la proposition d'une approche d'évaluation centrée sur l'espace latent, connue sous le nom d'Ablation Latente. Le quatrième aspect de ce travail consiste à examiner ce que nous appelons la saillance et la représentation de certains concepts visuels, tels que la quantité d'objets, à différents niveaux de l'architecture de sous-titrage.},
language = {fr},
urldate = {2025-10-24},
school = {Orléans},
author = {Elguendouze, Sofiane},
month = nov,
year = {2024},
}
@misc{lopardo_attention_2024,
title = {Attention {Meets} {Post}-hoc {Interpretability}: {A} {Mathematical} {Perspective}},
shorttitle = {Attention {Meets} {Post}-hoc {Interpretability}},
url = {http://arxiv.org/abs/2402.03485},
doi = {10.48550/arXiv.2402.03485},
abstract = {Attention-based architectures, in particular transformers, are at the heart of a technological revolution. Interestingly, in addition to helping obtain state-of-the-art results on a wide range of applications, the attention mechanism intrinsically provides meaningful insights on the internal behavior of the model. Can these insights be used as explanations? Debate rages on. In this paper, we mathematically study a simple attention-based architecture and pinpoint the differences between post-hoc and attention-based explanations. We show that they provide quite different results, and that, despite their limitations, post-hoc methods are capable of capturing more useful insights than merely examining the attention weights.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Lopardo, Gianluigi and Precioso, Frederic and Garreau, Damien},
month = jun,
year = {2024},
note = {arXiv:2402.03485},
keywords = {Computer Science - Computation and Language, Computer Science - Machine Learning, Statistics - Machine Learning},
}
@inproceedings{sippy_data_2020,
title = {Data {Staining}: {A} {Method} for {Comparing} {Faithfulness} of {Explainers}},
url = {https://aiweb.cs.washington.edu/ai/pubs/sippy-icml20.pdf},
author = {Sippy, Jacob D and Bansal, Gagan and Weld, Daniel},
year = {2020},
}
@misc{poeta_concept-based_2023,
title = {Concept-based {Explainable} {Artificial} {Intelligence}: {A} {Survey}},
shorttitle = {Concept-based {Explainable} {Artificial} {Intelligence}},
url = {http://arxiv.org/abs/2312.12936},
doi = {10.48550/arXiv.2312.12936},
abstract = {The field of explainable artificial intelligence emerged in response to the growing need for more transparent and reliable models. However, using raw features to provide explanations has been disputed in several works lately, advocating for more user-understandable explanations. To address this issue, a wide range of papers proposing Concept-based eXplainable Artificial Intelligence (C-XAI) methods have arisen in recent years. Nevertheless, a unified categorization and precise field definition are still missing. This paper fills the gap by offering a thorough review of C-XAI approaches. We define and identify different concepts and explanation types. We provide a taxonomy identifying nine categories and propose guidelines for selecting a suitable category based on the development context. Additionally, we report common evaluation strategies including metrics, human evaluations and dataset employed, aiming to assist the development of future methods. We believe this survey will serve researchers, practitioners, and domain experts in comprehending and advancing this innovative field.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Poeta, Eleonora and Ciravegna, Gabriele and Pastor, Eliana and Cerquitelli, Tania and Baralis, Elena},
month = dec,
year = {2023},
note = {arXiv:2312.12936},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Human-Computer Interaction},
}
@misc{dosovitskiy_image_2021,
title = {An {Image} is {Worth} 16x16 {Words}: {Transformers} for {Image} {Recognition} at {Scale}},
shorttitle = {An {Image} is {Worth} 16x16 {Words}},
url = {http://arxiv.org/abs/2010.11929},
doi = {10.48550/arXiv.2010.11929},
abstract = {While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Dosovitskiy, Alexey and Beyer, Lucas and Kolesnikov, Alexander and Weissenborn, Dirk and Zhai, Xiaohua and Unterthiner, Thomas and Dehghani, Mostafa and Minderer, Matthias and Heigold, Georg and Gelly, Sylvain and Uszkoreit, Jakob and Houlsby, Neil},
month = jun,
year = {2021},
note = {arXiv:2010.11929},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning},
}
@misc{deyoung_eraser_2020,
title = {{ERASER}: {A} {Benchmark} to {Evaluate} {Rationalized} {NLP} {Models}},
shorttitle = {{ERASER}},
url = {http://arxiv.org/abs/1911.03429},
doi = {10.48550/arXiv.1911.03429},
abstract = {State-of-the-art models in NLP are now predominantly based on deep neural networks that are opaque in terms of how they come to make predictions. This limitation has increased interest in designing more interpretable deep models for NLP that reveal the `reasoning' behind model outputs. But work in this direction has been conducted on different datasets and tasks with correspondingly unique aims and metrics; this makes it difficult to track progress. We propose the Evaluating Rationales And Simple English Reasoning (ERASER) benchmark to advance research on interpretable models in NLP. This benchmark comprises multiple datasets and tasks for which human annotations of "rationales" (supporting evidence) have been collected. We propose several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are (i.e., the degree to which provided rationales influenced the corresponding predictions). Our hope is that releasing this benchmark facilitates progress on designing more interpretable NLP systems. The benchmark, code, and documentation are available at https://www.eraserbenchmark.com/},
urldate = {2025-10-24},
publisher = {arXiv},
author = {DeYoung, Jay and Jain, Sarthak and Rajani, Nazneen Fatema and Lehman, Eric and Xiong, Caiming and Socher, Richard and Wallace, Byron C.},
month = apr,
year = {2020},
note = {arXiv:1911.03429},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning},
}
@misc{russakovsky_imagenet_2015,
title = {{ImageNet} {Large} {Scale} {Visual} {Recognition} {Challenge}},
url = {http://arxiv.org/abs/1409.0575},
doi = {10.48550/arXiv.1409.0575},
abstract = {The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images. The challenge has been run annually from 2010 to present, attracting participation from more than fifty institutions. This paper describes the creation of this benchmark dataset and the advances in object recognition that have been possible as a result. We discuss the challenges of collecting large-scale ground truth annotation, highlight key breakthroughs in categorical object recognition, provide a detailed analysis of the current state of the field of large-scale image classification and object detection, and compare the state-of-the-art computer vision accuracy with human accuracy. We conclude with lessons learned in the five years of the challenge, and propose future directions and improvements.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Russakovsky, Olga and Deng, Jia and Su, Hao and Krause, Jonathan and Satheesh, Sanjeev and Ma, Sean and Huang, Zhiheng and Karpathy, Andrej and Khosla, Aditya and Bernstein, Michael and Berg, Alexander C. and Fei-Fei, Li},
month = jan,
year = {2015},
note = {arXiv:1409.0575},
keywords = {Computer Science - Computer Vision and Pattern Recognition},
}
@misc{devlin_bert_2019,
title = {{BERT}: {Pre}-training of {Deep} {Bidirectional} {Transformers} for {Language} {Understanding}},
shorttitle = {{BERT}},
url = {http://arxiv.org/abs/1810.04805},
doi = {10.48550/arXiv.1810.04805},
abstract = {We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. As a result, the pre-trained BERT model can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks, such as question answering and language inference, without substantial task-specific architecture modifications. BERT is conceptually simple and empirically powerful. It obtains new state-of-the-art results on eleven natural language processing tasks, including pushing the GLUE score to 80.5\% (7.7\% point absolute improvement), MultiNLI accuracy to 86.7\% (4.6\% absolute improvement), SQuAD v1.1 question answering Test F1 to 93.2 (1.5 point absolute improvement) and SQuAD v2.0 Test F1 to 83.1 (5.1 point absolute improvement).},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
month = may,
year = {2019},
note = {arXiv:1810.04805},
keywords = {Computer Science - Computation and Language},
}
@misc{chefer_transformer_2021,
title = {Transformer {Interpretability} {Beyond} {Attention} {Visualization}},
url = {http://arxiv.org/abs/2012.09838},
doi = {10.48550/arXiv.2012.09838},
abstract = {Self-attention techniques, and specifically Transformers, are dominating the field of text processing and are becoming increasingly popular in computer vision classification tasks. In order to visualize the parts of the image that led to a certain classification, existing methods either rely on the obtained attention maps or employ heuristic propagation along the attention graph. In this work, we propose a novel way to compute relevancy for Transformer networks. The method assigns local relevance based on the Deep Taylor Decomposition principle and then propagates these relevancy scores through the layers. This propagation involves attention layers and skip connections, which challenge existing methods. Our solution is based on a specific formulation that is shown to maintain the total relevancy across layers. We benchmark our method on very recent visual Transformer networks, as well as on a text classification problem, and demonstrate a clear advantage over the existing explainability methods.},
urldate = {2025-10-24},
publisher = {arXiv},
author = {Chefer, Hila and Gur, Shir and Wolf, Lior},
month = apr,
year = {2021},
note = {arXiv:2012.09838},
keywords = {Computer Science - Computer Vision and Pattern Recognition},
}
@inproceedings{kalouli_xplainli_2020,
address = {Barcelona, Spain (Online)},
title = {{XplaiNLI}: {Explainable} {Natural} {Language} {Inference} through {Visual} {Analytics}},
shorttitle = {{XplaiNLI}},
url = {https://www.aclweb.org/anthology/2020.coling-demos.9},
doi = {10.18653/v1/2020.coling-demos.9},
language = {en},
urldate = {2025-10-24},
booktitle = {Proceedings of the 28th {International} {Conference} on {Computational} {Linguistics}: {System} {Demonstrations}},
publisher = {International Committee on Computational Linguistics (ICCL)},
author = {Kalouli, Aikaterini-Lida and Sevastjanova, Rita and De Paiva, Valeria and Crouch, Richard and El-Assady, Mennatallah},
year = {2020},
pages = {48--52},
}
@inproceedings{kalouli_hy-nli_2020,
address = {Barcelona, Spain (Online)},
title = {Hy-{NLI}: a {Hybrid} system for {Natural} {Language} {Inference}},
shorttitle = {Hy-{NLI}},
url = {https://www.aclweb.org/anthology/2020.coling-main.459},
doi = {10.18653/v1/2020.coling-main.459},
language = {en},
urldate = {2025-10-24},
booktitle = {Proceedings of the 28th {International} {Conference} on {Computational} {Linguistics}},
publisher = {International Committee on Computational Linguistics},
author = {Kalouli, Aikaterini-Lida and Crouch, Richard and De Paiva, Valeria},
year = {2020},
pages = {5235--5249},
}
@book{samek_explainable_2019,
address = {Cham},
series = {Lecture {Notes} in {Computer} {Science}},
title = {Explainable {AI}: {Interpreting}, {Explaining} and {Visualizing} {Deep} {Learning}},
volume = {11700},
copyright = {http://www.springer.com/tdm},
isbn = {9783030289539 9783030289546},
shorttitle = {Explainable {AI}},
url = {http://link.springer.com/10.1007/978-3-030-28954-6},
language = {en},
urldate = {2025-09-17},
publisher = {Springer International Publishing},
editor = {Samek, Wojciech and Montavon, Grégoire and Vedaldi, Andrea and Hansen, Lars Kai and Müller, Klaus-Robert},
year = {2019},
doi = {10.1007/978-3-030-28954-6},
}
@incollection{ancona_gradient-based_2019,
address = {Cham},
title = {Gradient-{Based} {Attribution} {Methods}},
isbn = {9783030289546},
url = {https://doi.org/10.1007/978-3-030-28954-6_9},
abstract = {The problem of explaining complex machine learning models, including Deep Neural Networks, has gained increasing attention over the last few years. While several methods have been proposed to explain network predictions, the definition itself of explanation is still debated. Moreover, only a few attempts to compare explanation methods from a theoretical perspective has been done. In this chapter, we discuss the theoretical properties of several attribution methods and show how they share the same idea of using the gradient information as a descriptive factor for the functioning of a model. Finally, we discuss the strengths and limitations of these methods and compare them with available alternatives.},
language = {en},
urldate = {2025-09-17},
booktitle = {Explainable {AI}: {Interpreting}, {Explaining} and {Visualizing} {Deep} {Learning}},
publisher = {Springer International Publishing},
author = {Ancona, Marco and Ceolini, Enea and Öztireli, Cengiz and Gross, Markus},
editor = {Samek, Wojciech and Montavon, Grégoire and Vedaldi, Andrea and Hansen, Lars Kai and Müller, Klaus-Robert},
year = {2019},
doi = {10.1007/978-3-030-28954-6_9},
keywords = {Attribution methods, Deep Neural Networks, Explainable artificial intelligence},
pages = {169--191},
}
@incollection{montavon_layer-wise_2019,
address = {Cham},
title = {Layer-{Wise} {Relevance} {Propagation}: {An} {Overview}},
isbn = {9783030289546},
shorttitle = {Layer-{Wise} {Relevance} {Propagation}},
url = {https://doi.org/10.1007/978-3-030-28954-6_10},
abstract = {For a machine learning model to generalize well, one needs to ensure that its decisions are supported by meaningful patterns in the input data. A prerequisite is however for the model to be able to explain itself, e.g. by highlighting which input features it uses to support its prediction. Layer-wise Relevance Propagation (LRP) is a technique that brings such explainability and scales to potentially highly complex deep neural networks. It operates by propagating the prediction backward in the neural network, using a set of purposely designed propagation rules. In this chapter, we give a concise introduction to LRP with a discussion of (1) how to implement propagation rules easily and efficiently, (2) how the propagation procedure can be theoretically justified as a ‘deep Taylor decomposition’, (3) how to choose the propagation rules at each layer to deliver high explanation quality, and (4) how LRP can be extended to handle a variety of machine learning scenarios beyond deep neural networks.},
language = {en},
urldate = {2025-09-17},
booktitle = {Explainable {AI}: {Interpreting}, {Explaining} and {Visualizing} {Deep} {Learning}},
publisher = {Springer International Publishing},
author = {Montavon, Grégoire and Binder, Alexander and Lapuschkin, Sebastian and Samek, Wojciech and Müller, Klaus-Robert},
editor = {Samek, Wojciech and Montavon, Grégoire and Vedaldi, Andrea and Hansen, Lars Kai and Müller, Klaus-Robert},
year = {2019},
doi = {10.1007/978-3-030-28954-6_10},
keywords = {Deep Neural Networks, Deep Taylor Decomposition, Explanations, Layer-wise Relevance Propagation},
pages = {193--209},
}
@misc{huang_annotated_2022,
title = {The {Annotated} {Transformer}},
url = {https://nlp.seas.harvard.edu/annotated-transformer/},
urldate = {2025-09-12},
author = {Huang, Austin and Subramanian, Suraj and Sum, Jonathan and Almubarak, Khalid and Biderman, Stella},
year = {2022},
}
@inproceedings{bell_its_2022,
address = {Seoul Republic of Korea},
title = {It’s {Just} {Not} {That} {Simple}: {An} {Empirical} {Study} of the {Accuracy}-{Explainability} {Trade}-off in {Machine} {Learning} for {Public} {Policy}},
isbn = {9781450393522},
shorttitle = {It’s {Just} {Not} {That} {Simple}},
url = {https://dl.acm.org/doi/10.1145/3531146.3533090},
doi = {10.1145/3531146.3533090},
language = {en},
urldate = {2025-09-11},
booktitle = {2022 {ACM} {Conference} on {Fairness} {Accountability} and {Transparency}},
publisher = {ACM},
author = {Bell, Andrew and Solano-Kamaiko, Ian and Nov, Oded and Stoyanovich, Julia},
month = jun,
year = {2022},
pages = {248--266},
}
@misc{gilpin_explaining_2019,
title = {Explaining {Explanations}: {An} {Overview} of {Interpretability} of {Machine} {Learning}},
shorttitle = {Explaining {Explanations}},
url = {http://arxiv.org/abs/1806.00069},
doi = {10.48550/arXiv.1806.00069},
abstract = {There has recently been a surge of work in explanatory artificial intelligence (XAI). This research area tackles the important problem that complex machines and algorithms often cannot provide insights into their behavior and thought processes. XAI allows users and parts of the internal system to be more transparent, providing explanations of their decisions in some level of detail. These explanations are important to ensure algorithmic fairness, identify potential bias/problems in the training data, and to ensure that the algorithms perform as expected. However, explanations produced by these systems is neither standardized nor systematically assessed. In an effort to create best practices and identify open challenges, we provide our definition of explainability and show how it can be used to classify existing literature. We discuss why current approaches to explanatory methods especially for deep neural networks are insufficient. Finally, based on our survey, we conclude with suggested future research directions for explanatory artificial intelligence.},
urldate = {2025-09-10},
publisher = {arXiv},
author = {Gilpin, Leilani H. and Bau, David and Yuan, Ben Z. and Bajwa, Ayesha and Specter, Michael and Kagal, Lalana},
month = feb,
year = {2019},
note = {arXiv:1806.00069 [cs]},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning, Statistics - Machine Learning},
}
@misc{morgan_explainable_2023,
title = {Explainable {AI}: {Visualizing} {Attention} in {Transformers}},
shorttitle = {Explainable {AI}},
url = {https://www.comet.com/site/blog/explainable-ai-for-transformers/},
abstract = {Learn how to visualize the attention of transformers and log your results to Comet, as we work towards explainability in AI.},
language = {en-US},
urldate = {2025-08-21},
journal = {Comet},
author = {Morgan, Abby},
month = jul,
year = {2023},
keywords = {outil BERT},
}
@inproceedings{garouani_investigating_2024,
title = {Investigating the {Duality} of {Interpretability} and {Explainability} in {Machine} {Learning}},
url = {http://arxiv.org/abs/2503.21356},
doi = {10.1109/ICTAI62512.2024.00125},
abstract = {The rapid evolution of machine learning (ML) has led to the widespread adoption of complex "black box" models, such as deep neural networks and ensemble methods. These models exhibit exceptional predictive performance, making them invaluable for critical decision-making across diverse domains within society. However, their inherently opaque nature raises concerns about transparency and interpretability, making them untrustworthy decision support systems. To alleviate such a barrier to high-stakes adoption, research community focus has been on developing methods to explain black box models as a means to address the challenges they pose. Efforts are focused on explaining these models instead of developing ones that are inherently interpretable. Designing inherently interpretable models from the outset, however, can pave the path towards responsible and beneficial applications in the field of ML. In this position paper, we clarify the chasm between explaining black boxes and adopting inherently interpretable models. We emphasize the imperative need for model interpretability and, following the purpose of attaining better (i.e., more effective or efficient w.r.t. predictive performance) and trustworthy predictors, provide an experimental evaluation of latest hybrid learning methods that integrates symbolic knowledge into neural network predictors. We demonstrate how interpretable hybrid models could potentially supplant black box ones in different domains.},
urldate = {2025-08-21},
booktitle = {2024 {IEEE} 36th {International} {Conference} on {Tools} with {Artificial} {Intelligence} ({ICTAI})},
author = {Garouani, Moncef and Mothe, Josiane and Barhrhouj, Ayah and Aligon, Julien},
month = oct,
year = {2024},
note = {arXiv:2503.21356 [cs]},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Machine Learning},
pages = {861--867},
}
@article{huang_explainable_2024,
title = {From explainable to interpretable deep learning for natural language processing in healthcare: {How} far from reality?},
volume = {24},
issn = {2001-0370},
shorttitle = {From explainable to interpretable deep learning for natural language processing in healthcare},
url = {https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11126530/},
doi = {10.1016/j.csbj.2024.05.004},
abstract = {Deep learning (DL) has substantially enhanced natural language processing (NLP) in healthcare research. However, the increasing complexity of DL-based NLP necessitates transparent model interpretability, or at least explainability, for reliable decision-making. This work presents a thorough scoping review of explainable and interpretable DL in healthcare NLP. The term “eXplainable and Interpretable Artificial Intelligence” (XIAI) is introduced to distinguish XAI from IAI. Different models are further categorized based on their functionality (model-, input-, output-based) and scope (local, global). Our analysis shows that attention mechanisms are the most prevalent emerging IAI technique. The use of IAI is growing, distinguishing it from XAI. The major challenges identified are that most XIAI does not explore “global” modelling processes, the lack of best practices, and the lack of systematic evaluation and benchmarks. One important opportunity is to use attention mechanisms to enhance multi-modal XIAI for personalized medicine. Additionally, combining DL with causal logic holds promise. Our discussion encourages the integration of XIAI in Large Language Models (LLMs) and domain-specific smaller models. In conclusion, XIAI adoption in healthcare requires dedicated in-house expertise. Collaboration with domain experts, end-users, and policymakers can lead to ready-to-use XIAI methods across NLP and medical tasks. While challenges exist, XIAI techniques offer a valuable foundation for interpretable NLP algorithms in healthcare.},
urldate = {2025-08-21},
journal = {Computational and Structural Biotechnology Journal},
author = {Huang, Guangming and Li, Yingya and Jameel, Shoaib and Long, Yunfei and Papanastasiou, Giorgos},
month = may,
year = {2024},
pmid = {38800693},
pmcid = {PMC11126530},
pages = {362--373},
}
@misc{mohammadi_explainability_2025,
title = {Explainability in {Practice}: {A} {Survey} of {Explainable} {NLP} {Across} {Various} {Domains}},
shorttitle = {Explainability in {Practice}},
url = {http://arxiv.org/abs/2502.00837},
doi = {10.48550/arXiv.2502.00837},
abstract = {Natural Language Processing (NLP) has become a cornerstone in many critical sectors, including healthcare, finance, and customer relationship management. This is especially true with the development and use of advanced models such as GPT-based architectures and BERT, which are widely used in decision-making processes. However, the black-box nature of these advanced NLP models has created an urgent need for transparency and explainability. This review explores explainable NLP (XNLP) with a focus on its practical deployment and real-world applications, examining its implementation and the challenges faced in domain-specific contexts. The paper underscores the importance of explainability in NLP and provides a comprehensive perspective on how XNLP can be designed to meet the unique demands of various sectors, from healthcare's need for clear insights to finance's emphasis on fraud detection and risk assessment. Additionally, this review aims to bridge the knowledge gap in XNLP literature by offering a domain-specific exploration and discussing underrepresented areas such as real-world applicability, metric evaluation, and the role of human interaction in model assessment. The paper concludes by suggesting future research directions that could enhance the understanding and broader application of XNLP.},
urldate = {2025-08-21},
publisher = {arXiv},
author = {Mohammadi, Hadi and Bagheri, Ayoub and Giachanou, Anastasia and Oberski, Daniel L.},
month = feb,
year = {2025},
note = {arXiv:2502.00837 [cs]
version: 1},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language},
}
@article{mersha_evaluating_2025,
title = {Evaluating the {Effectiveness} of {XAI} {Techniques} for {Encoder}-{Based} {Language} {Models}},
volume = {310},
issn = {09507051},
url = {http://arxiv.org/abs/2501.15374},
doi = {10.1016/j.knosys.2025.113042},
abstract = {The black-box nature of large language models (LLMs) necessitates the development of eXplainable AI (XAI) techniques for transparency and trustworthiness. However, evaluating these techniques remains a challenge. This study presents a general evaluation framework using four key metrics: Human-reasoning Agreement (HA), Robustness, Consistency, and Contrastivity. We assess the effectiveness of six explainability techniques from five different XAI categories model simplification (LIME), perturbation-based methods (SHAP), gradient-based approaches (InputXGradient, Grad-CAM), Layer-wise Relevance Propagation (LRP), and attention mechanisms-based explainability methods (Attention Mechanism Visualization, AMV) across five encoder-based language models: TinyBERT, BERTbase, BERTlarge, XLM-R large, and DeBERTa-xlarge, using the IMDB Movie Reviews and Tweet Sentiment Extraction (TSE) datasets. Our findings show that the model simplification-based XAI method (LIME) consistently outperforms across multiple metrics and models, significantly excelling in HA with a score of 0.9685 on DeBERTa-xlarge, robustness, and consistency as the complexity of large language models increases. AMV demonstrates the best Robustness, with scores as low as 0.0020. It also excels in Consistency, achieving near-perfect scores of 0.9999 across all models. Regarding Contrastivity, LRP performs the best, particularly on more complex models, with scores up to 0.9371.},
urldate = {2025-08-21},
journal = {Knowledge-Based Systems},
author = {Mersha, Melkamu Abay and Yigezu, Mesay Gemeda and Kalita, Jugal},
month = feb,
year = {2025},
note = {arXiv:2501.15374 [cs]},
keywords = {Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Computers and Society, Computer Science - Machine Learning},
pages = {113042},
}
seminaireCA/archives-utiles.md deleted 100644 → 0
View file @ 7f0fd3d8
---
marp: true
style: |
.columns {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: 1rem;
}
.small {
font-size: 20px
}
.midsize{
font-size: 25px
}
math: mathjax
paginate: true
---
# Séminaires CA - Explicabilité mécanistique pour l'amélioration des modèles profonds de RAG
---
# RAG
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Extraction - Types
- **Types d'Extracteur**
- **Sparse retrieval** : au niveau des mots (TF-IDF, BM25)
- pas entraîné
- grosse dépendance à la qualité de la BDD et requête
- **Dense retrieval** : plonger dans un espace
- 1ere couche de générateur pour projeter
- encodeur spécifique ([Contriever](https://github.com/facebookresearch/contriever) - [[Izacard et al. 2022]](https://arxiv.org/pdf/2112.09118) )
- bi-encodeur (Dense Passage Retrieval - [[Karpukhin et al. 2020]](https://aclanthology.org/2020.emnlp-main.550/))
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Extraction - Granularité
- **Granularité de d'Extraction** : taille de découpe des extraits
- **chunk retieval** - le plus répandu
- **token retrieval** - plus fin mais plus lourd en calcul
- **entity retrieval** - perspective du langage (Entities as Experts - [[Févry et al. 2020]](https://aclanthology.org/2020.emnlp-main.400/))
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Extraction - Amélioration pré-retrieval
- **Pre-Retrieval Enhancement** : réduire la charge du retriever en *complétant la requête*
- **query expansion** - générer des docs fictifs, compléter la query avec les infos pertinentes ~> désambiguïser (Query2Doc - [[Wang et al. 2023]](https://aclanthology.org/2023.emnlp-main.585/))
- **query rewrite** - reformuler l'entrée à l'aide données supplémentaires (Rewrite-Retrieve-Read - [[Ma et al. 2023]](https://arxiv.org/abs/2305.14283))
- **query augmentation** - compléter l'entrée avec une réponse sans données extraites (ReFeed - [[Yu et al. 2023]](https://arxiv.org/pdf/2305.14002))
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Extraction - Amélioration post-retrieval
- **Post-Retrieval Enhancement** : réduire la charge sur l'extracteur en *affinant les informations extraites*
- **ajustement du classement** des extraits (PRCA - [[Yang et al. 2023]](https://aclanthology.org/2023.emnlp-main.326/))
- **réduction du bruit** dans les extraits
- assembler **plusieurs extracteurs** (R<sup>2</sup>G : Retrieve Rerank Generate - [[Glass et al., 2022]](https://aclanthology.org/2022.naacl-main.194/))
- **ajouter la query** aux données
- utiliser des **reasoning paths** ([Search-in-the-Chain](https://github.com/xsc1234/Search-in-the-Chain) - [[Xu et al. 2024]](https://arxiv.org/abs/2304.14732))
- **compression/synthèse** des (gros) extraits
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Extraction - Base de données
Nature et source des données différentes
- **closed source** - locale
- **domain specific**
- **open-source** - utilisation de moteurs de recherche public
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Générateurs - Familles
Différents modèles de langues ~> Dépends de la tâche tierce
- **Parameter accessible** - peut être affiné
- **Parameter inaccessible** - via API sans accès aux paramètres
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Intégration - Stratégies
- **Input Layer Integration** - Combiner Requête et Extraits en entrée du Générateur
- tous les documents **en même temps** ([In-Context RALM](https://github.com/AI21Labs/in-context-ralm/tree/main) - [[Ram et al., 2023]](https://aclanthology.org/2023.tacl-1.75.pdf))
- un document **à la fois** ([ATLAS](https://github.com/facebookresearch/atlas) - [[Izacard et al. 2023]](https://dl.acm.org/doi/epdf/10.5555/3648699.3648950), REPLUG [reproduction](https://github.com/SashaBoguraev/REPLUG) - [[Shi et al., 2024]](https://aclanthology.org/2024.naacl-long.463/))
- **Output Layer Integration** - Fournir les données et le texte généré séparément (ou petit traitement pour unir) (ReFeed - [[Yu et al. 2023]](https://arxiv.org/pdf/2305.14002))
- **Intermediate Layer Integration** - intégrer les documents extraits au milieu de la génération (du modèle)
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Intégration - Fréquence et Nécessité
Extraction non pertinente $\implies$ hallucination / bruit
=> Définir des critères de recours ou non au Retrieval
<div class="columns"><div>
- **Coté RA-LLM**
- token spécifique
- prompt itéré
- modèles annexes entraînés pour déclencher
- définition de logit déclencheurs
</div><div>
- **Coté RAG "traditionnels"** : fréquence fixée
- one-time
- every-n-token
- every-token
</div></div>
---
<div class="small">
*Survey on RAG Meeting LLM: Towards Retrieval-Augmented Large Language Models [Fan et al., 2023]*
</div>
## Entraînement - Stratégies
Affiner les composants sur la tâche tierce
- **Training free**
- **prompt engineering based method** : injection des données dans le prompt
- **retrieval guided token generation method** : calibrer la génération à l'aide des données
- **Independent Training** : hors pipeline
- **Sequential Training** : le 2<sup>nd</sup> au sein du pipeline (2 ordres possibles)
- **Join Training** : les deux en même temps dans le pipeline
---
# XAI
---
## Modèle boite noire
**Modèles pour lesquels les processus de décision ne sont pas transparents pour l’humain**
Problème : Perte de confiance pour certains usages
---
## Définitions centrales de l'XAI
- **Explicabilité de l’IA** : Rendre humainement compréhensible les raisons pour lesquelles un
modèle effectue une certaine prédiction. [Garouani et al., 2024, Bell et al., 2022]
- **Comprendre** : Être capable d’anticiper et/ou justifier le comportement du modèle. [Bell et al., 2022]
- **Explication** : Tout support rendant le processus de décision d’un modèle humainement intelligible.
---
## Formes d'explication
- attribution
- saliency map (image ou texte)
- règles
---
## Temps de l'explicabilité
![](../images/VieModele-TempsXAI.drawio.pdf)
<div class="columns"><div>
Interprétabilité
_Comment ?_
</div><div>
Explicabilité
_Pourquoi ?_
</div></div>
---
# MechIR
---
## [[Parry et al., 2025](https://arxiv.org/abs/2501.10165)] MechIR: A Mechanistic Interpretability Framework for Information Retrieval
**Activation Patching** : Observer la réaction d'un composant à une caractéristique d'une entrée
- Relevé des valeurs intermédiaires du modèle sur l'entrée $i$
- Relevé des valeurs intermédiaires du modèle sur l'entrée perturbée $i'$
- Fixer les valeurs du/des noeuds observés avec celles observées (meilleure perf)
- Observer les output sur l'entrée fournie (moins bonne perf)
---
## [[Parry et al., 2025](https://arxiv.org/abs/2501.10165)] MechIR: A Mechanistic Interpretability Framework for Information Retrieval
<div class="columns"><div>
MechIR : Propose une implem pour les modèles encoder-based
Difficultés :
- Lourd en calcul
- Problèmes d'installation
</div><div>
![exemple de heatmap obtenue](../images/ActivationPatchingAllHeadExempleMechIR.png)
</div></div>
---
---
## [[Chen et al., 2024](https://dl.acm.org/doi/10.1145/3626772.3657841)] Axiomatic Causal Interventions for Reverse Engineering Relevance Computation in Neural Retrieval Models
Étends la méthode proposée par [[Geiger et al., 2021](https://dl.acm.org/doi/10.5555/3540261.3540994)]
Méthode implémentée par MechIR [[Parry et al., 2025](https://arxiv.org/abs/2501.10165)]
*Pourquoi prendre comme référence les données avec meilleur pref et pas données baseline? (Contrairement à la méthode originale)*
> "This modification accounts for axiomatic perturbations that may either add or remove crucial relevance concepts from a document"
*Quelle métrique utilisée?*
> "To assess the patch’s effect on model performance, we evaluate using the normalized difference in ranking scores"
---
## [[Chen et al., 2024](https://dl.acm.org/doi/10.1145/3626772.3657841)] Axiomatic Causal Interventions for Reverse Engineering Relevance Computation in Neural Retrieval Models
![](../images/Ill-activationPatching_Chen-et-al2024.png)
---
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment