Home

CRAFT Shared Task 2019 Evaluation

This repository hosts code and scripts used for evaluation of the CRAFT Shared Tasks 2019.

There are three ways one might use the code/scripts in this repository:

Option #1: Evaluation via Docker (Recommended method)

By far, the easiest way is to use the pre-built Docker container. The Docker container contains all dependencies for the evaluation scripts, the evaluation scripts themselves, and the CRAFT corpus. It is available on DockerHub, and instructions for how to use it are available on the Evaluation via Docker wiki page.

Option #2: Build the Docker container locally

The next easiest way to evaluate your files against CRAFT is to build the Docker container yourself. The Building the CRAFT evaluation Docker container wiki page contains instructions for how to build the evaluation Docker container locally. Once it is built, please follow the instructions for Evaluation via Docker to evaluate your results.

Option #3: Install and run in your local environment (Not recommended)

Finally, the third option is to download the latest stable release of this repository, install any missing dependencies, and to run the evaluations in your local environment. NOTE, this is not the recommended approach, but instructions for a local installation are available on the Local installation wiki page.

Contact Information

For questions about the evaluation scripts and/or to report issues with running them, please submit an issue to this GitHub repository.