Summary
Workshop to be held in Portland, Oregon, on June 24, 2011
in conjunction with BioNLP, an ACL-HLT 2011 workshop.
Test set online evaluation
Each participant can use test set online evaluation once in a day for each task.
Main tasks
Main tasks are finished. Thank you very much for your participation. Final evaluation results are available
Test sets available [testset downloads]
Training/development sets for all tasks available [downloads]
Development set evaluation
Visualizations for main task datasets [updated!]
Supporting tasks
Training/development sets for all tasks available [downloads]
Development set evaluation:
Coreference [CO] supporting task: online evaluation, evaluation software
Entity relations [REL] supporting task: online evaluation
Bacteria Gene Renaming [REN] supporting task: online evaluation, evaluation software
The BioNLP Shared Task series represents a community-wide move in bio-textmining toward fine-grained information extraction (IE). The first event, the BioNLP 2009 shared task (Dec. 2008-March 2009), attracted wide attention, with 24 teams submitting final results. The task setup and data have since served as the basis of numerous studies and published event extraction systems and datasets. The BioNLP Shared Task 2011 (BioNLP-ST'11) is the follow-up event to the BioNLP 2009 shared task. While following the general outline and goals of the previous task in defining biologically relevant extraction targets and a linguistically motivated approach to event representation, the upcoming task will generalize and extend on the previous in three key aspects: text type, domain, and targeted event types. Manually annotated data where all annotations are bound to specific expressions in text will be provided for training, development and evaluation of extraction methods, and tools for detailed evaluation of system outputs will be made available. Participation to the task is open to all interested parties.
While the BioNLP 2009 shared task relied on the GENIA corpus which contained PubMed abstracts on transcription factors in human blood cells, the theme of BioNLP-ST 2011 is generalization, which will be pursued to three directions: text types, event types, and subject domains. To achieve that, various tasks and annotations were created in collaboration between several groups, and event extraction tasks are arranged in four tracks.
Bacteria Track:
BioNLP-ST 2011 also includes three supporting tasks.
Although they are not event extraction tasks themselves, according to the analysis on the results of BioNLP 2009 shared task, co-reference resolution and entity relation detection are expected to play an important role in making a breakthrough in improving event extraction performance. Gene renaming has similar feature with co-references.
Jun’ichi Tsujii (Chair), University of Tokyo, and National Centre for Text Mining (NaCTeM)
Sophia Ananiadou, National Centre for Text Mining (NaCTeM)
Kevin Cohen, University of Colorado, and MITRE
Claire Nedellec, French National Institute for Agricultural Research (INRA)
Andrey Rzhetsky, University of Chicago
Bruno Sobral, Virginia Bioinformatics Institute
Tapio Salakoski, University of Turku
Toshihisa Takagi, Database Center for Life Science (DBCLS)
Jin-Dong Kim (Chair), Database Center for Life Science (DBCLS)
Sampo Pyysalo (Chair), University of Tokyo
Tomoko Ohta, University of Tokyo
Robert Bossy, French National Institute for Agricultural Research (INRA)
Chunhong Mao, Virginia Bioinformatics Institute
Dan Sullivan, Virginia Bioinformatics Institute
Rafal Rak, National Centre for Text Mining (NaCTeM)
Nguyen Luu Thuy Ngan, University of Tokyo
Timothy Baldwin, University of Melbourne
Sabine Bergler, Concordia University
Olivier Bodenreider, National Library of Medicine (NLM)
Wendy Chapman, University of California, San Diego (UCSD)
Kevin Cohen, University of Colorado, and MITRE
Nigel Collier, National Institute of Informatics (NII)
Filip Ginter, University of Turku
Jörg Hakenberg, Arizona State University
Minlie Huang, Tsinghua University
Su Jian, Institute for Infocomm Research
Min-Yen Kan, National University of Singapore
Jung-Jae Kim, Nanyang Technological University
Martin Krallinger, National Biotechnology Center (CNB)
Zhiyong Lu, National Library of Medicine (NLM)
David McClosky, Stanford University
Roser Morante, University of Antwerp
Claire Nedellec, French National Institute for Agricultural Research (INRA)
Serguei Pakhomov, University of Minnesota
Thierry Poibeau, French Institute for Fundamental Research (CNRS)
Hoifung Poon, University of Washington
Sebastian Riedel, University of Massachusetts
Fabio Rinaldi, University of Zurich
Thomas Rindflesch, National Library of Medicine (NLM)
Yvan Saeys, Ghent University
Tapio Salakoski, University of Turku
Hagit Shatkay, University of Delaware
Rune Sætre, Norwegian University of Science and Technology (NTNU)
Yuka Tateisi, Kogakuin University
Yoshimasa Tsuruoka, Japan Advanced Institute of Science and Technology (JAIST)
Karin Verspoor, University of Colorado
Xinglong Wang, National Centre for Text Mining
Hong Yu, University of Wisconsin-Milwaukee
Pierre Zweigenbaum, French National Center for Scientific Research (CNRS)
Web: https://sites.google.com/site/bionlpst
Mail: bionlp-st@googlegroups.com