Commit 657bdfc9 authored by Delvallez Delvallez's avatar Delvallez Delvallez

Completion Tableau Taches avec les données des Benchs

parent da88f685
...@@ -10,7 +10,7 @@ Bright,https://arxiv.org/abs/2407.12883v4,"https://github.com/xlang-ai/BRIGHT ...@@ -10,7 +10,7 @@ Bright,https://arxiv.org/abs/2407.12883v4,"https://github.com/xlang-ai/BRIGHT
https://huggingface.co/datasets/xlangai/BRIGHT","- 7 BDD StackExchange fait main https://huggingface.co/datasets/xlangai/BRIGHT","- 7 BDD StackExchange fait main
- TheoremQA + AoPS (Pb de maths + solutions récoltés) - TheoremQA + AoPS (Pb de maths + solutions récoltés)
- 2 BDD codage",,Tâche reasoning intensive -> pas ce qu'on veut - 2 BDD codage : pony, LeetCode",,Tâche reasoning intensive -> pas ce qu'on veut
BEIR,https://arxiv.org/abs/2104.08663v4,"https://github.com/beir-cellar/beir BEIR,https://arxiv.org/abs/2104.08663v4,"https://github.com/beir-cellar/beir
https://huggingface.co/BeIR https://huggingface.co/BeIR
...@@ -27,7 +27,11 @@ https://huggingface.co/BeIR ...@@ -27,7 +27,11 @@ https://huggingface.co/BeIR
- -
",,Zero-shot learning en IR ",,Zero-shot learning en IR
MS-MARCO,https://arxiv.org/abs/1611.09268,https://huggingface.co/datasets/microsoft/ms_marco,,"1M questions MS-MARCO,https://arxiv.org/abs/1611.09268,https://huggingface.co/datasets/microsoft/ms_marco,"Dataset construit à partir de :
- log de recherche BING- de documents sélectionné automatiquement par BING
- d'extraits de ces documents obtenu par annotateur
- de réponse rédigée par annotateur
- classification de la question en fonction de la nature de la réponse","1M questions
8,841,823 passages issus de 3,563,535 pages web 8,841,823 passages issus de 3,563,535 pages web
182,669 human generated answers 182,669 human generated answers
......
Intitulé,Benchs,Datasets,Archis/ Framework,Publis,Descriptions Intitulé,Benchs,Datasets,Archis/ Framework,Publis,Descriptions
Open Domain Question Answering,,,DPR,"[Lewis et al., 2020]" Open Domain Question Answering,,,DPR,"[Lewis et al., 2020]",Trouver l'extrait contennant la réponse dans une base de connaissance (pas qu'un seul document) [def de BEIR]
Question Answering,KILT,"Natural Question, TriviaQA, HotpotQA","PaperQA, SearChain",,Trouver les extraits attendus contenant la réponse Question Answering,"KILT, BEIR","Natural Question, TriviaQA, HotpotQA, FiQA-2018","PaperQA, SearChain",,Trouver les extraits attendus contenant la réponse
"Multi-hop question answering",,"HotpotQA, Musique, WikiMultiHopQA, StartegyQA",SearChain,"terme issu de [Xu et al., 2024] - SearChain", Multi-hop question answering,,"HotpotQA, Musique, WikiMultiHopQA, StartegyQA",SearChain,"terme issu de [Xu et al., 2024] - SearChain"," "
Long-form QA,,ELI5,SearChain
Summarization,,,PaperQA Summarization,,,PaperQA
Détection de contradiction,,,PaperQA Détection de contradiction,,,PaperQA
Jeopardy Question Generation,,,,"[Lewis et al.,2020]" Jeopardy Question Generation,,,,"[Lewis et al.,2020]"
Fact Checking,KILT,FEVER,"ATLAS, SearChain" Fact Checking,"KILT, BEIR","FEVER, Climate-FEVER, SciFact","ATLAS, SearChain",,"""verifies a claim against a big collection of evidence"" (BEIR)"
Slot Filling,KILT,"Zero shot RE (zsRE), T-REx","ATLAS, SearChain" Slot Filling,KILT,"Zero shot RE (zsRE), T-REx","ATLAS, SearChain"
Long-form QA,,ELI5,SearChain Entity Linking,KILT,"AIDA CoNLL-YAGO, WNED-WIKI, WNED-CWEB",ATLAS
Entity Linking,KILT,AIDA CoNLL-YAGO,ATLAS Entity Retrieval,BEIR,DBPedia,,,"""retrieving unique Wikipedia pages to entities mentioned in the query"" (BEIR) /!\ similaire"
Dialogue,KILT,Wizards of wikipedia,ATLAS Dialogue,KILT,Wizards of wikipedia,ATLAS
\ No newline at end of file Citation Prediction,BEIR,SCIDOCS,,,"""retrieve cited papers (output) for a given paper title as input"" (BEIR)"
Information Retrieval,"BIRCO, MS-MARCO","DORIS-MAE (Reseach), WhatsThatBook (Book description), Clinical-Trial (clinical trial description), RECLIC (litterature citation filling) ",,,Fournir les docuements conrrespondant à la query
Duplicate Question Retrieval,BEIR,"Quora, CQADupStack",,,"""identifying duplicate questions asked in community question answering (cQA) forums"" (BEIR)
input : question
ouput : duplicate question in the forum​
"
Argument Retrieval,"BEIR, BIRCO","Touche-2020, ArgAna",,,"""ranking argumentative texts in a collection of focused arguments
(output) in order of their relevance to a textual query (input) on different topics"" (BEIR)"
News Retrieval,BEIR,"TREC-NEWS, Robust04",,,"""Given a news headline, we retrieve relevant news articles that provide important context or background information."" (BEIR)"
Tweet Retrieval,BEIR,Signal-1M,,,"""retrieves relevant tweets for a given news article title"" (BEIR)"
Bio-medical Information Retrieval,BEIR,"TREC-COVID, BioASQ, NFCorpus",,,"""searching relevant scientific documents such as research papers or blogs for a given scientific query in the biomedical domain"" (BEIR)"
reasoning-intensive retrieval,Bright,"StackExchange, Pony (code), LeetCode, TheoremQA, AoPS",,,"Tache proche du OD-QA mais ""positive documents only provide useful information to help users derive an answer"" (Bright)"
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment