Guide · Open source
How to use NLP-progress: find NLP benchmarks and state-of-the-art results
Updated · written with AI from the project's own documentation and checked against our catalogue data
NLP-progress is an open-source GitHub repository that tracks progress in natural language processing. It has one Markdown page per task, listing benchmark datasets and the current state-of-the-art results. This guide shows researchers, students and practitioners how to find a task, read its results and add a new result or dataset.
NLP-progresssebastianruder/NLP-progress · ★ 23k · MIT Install options & downloads Quick start
- 1Open the repository or homepage. Go to github.com/sebastianruder/NLP-progress or nlpprogress.com in your browser. Use the homepage if you only want to read.
- 2Clone it to read locally (optional). Run the git clone command from the README to get the Markdown files on your computer.
- 3Pick a language and task. Use the table of contents in the README to find your language section and the task link.
- 4Read the task description and datasets. Check the description, the datasets listed and the evaluation metric before reading any scores.
- 5Read the results table. Look at the top rows for the best reported result, then check the Paper / Source and Code columns.
- 6Follow any leaderboard link. If the page points to a public leaderboard, use it for the most current results, since the README notes that community-maintained numbers may lag.
- 7Propose a change if you have one. Edit the file on GitHub, add your row or dataset, preview it, and submit a pull request.
What NLP-progress is and what it is not
NLP-progress is a reference of results, not a software package. The README describes it as a repository for tracking progress in Natural Language Processing, including the datasets and the current state-of-the-art for the most common NLP tasks.
Because it is a collection of pages rather than a library, there is nothing to install or run to use it. If you need code to process text, this project will not give you that.
- Good for: looking up benchmark datasets for a task, seeing which published methods score highest, and finding a starting point for public leaderboards.
- Not for: running NLP code or using an installable tool.
- The README says results rely on community contributions, so check the listed dates and linked sources before relying on a number.
Get a local copy of the repository
You can read the pages online at the homepage, nlpprogress.com. To read the files on your own machine, the README gives one command that clones the repository.
The task pages are Markdown files in folders named by language, such as english/ for English tasks.
git clone https://github.com/sebastianruder/NLP-progressFind the task you need in the table of contents
The README's table of contents groups tasks by language. The English section is the largest and includes tasks such as Automatic speech recognition, Machine translation, Named entity recognition, Question answering, Sentiment analysis and Summarization.
Each entry links to a Markdown file, for example english/named_entity_recognition.md. Other languages, including Vietnamese, Hindi, Chinese, French, Russian, Spanish and German, have their own sections, some of which link to anchors within a single file.
- Open the README and scroll to the table of contents.
- Choose your language section, then click the task name.
- If a task has a public leaderboard, the page points you there, as the README says the reader will be pointed to any regularly maintained results.
Read a task page: datasets, metrics and results
Each task page describes the task, the datasets used, the evaluation setting and metric, and tables of results. Contributors are asked to describe the evaluation setting and metric, and to show an annotated example of the dataset.
Results tables are sorted with the best result on top. Check the Paper / Source and Code columns to see where a number came from and whether an official implementation is linked. The guidelines say results from published papers are preferred, with an exception for influential preprints.
- Model: the system being reported.
- Score: the metric for that dataset. Some datasets have several metrics.
- Paper / Source: the reference for the result.
- Code: an Official or Link label where an implementation is available, or empty if none exists.
Add a result or dataset through a pull request
You can propose a change by clicking the small edit button in the top-right corner of the task file on GitHub. Add a row to the table in the same format, keep the table sorted with the best result on top, and use the Preview changes tab to check the layout.
To add a new dataset or task, the README lists steps: create a new file if the task is new and link it in the table of contents, describe the dataset and evaluation, show an annotated example, add a download link if available, and fill in at least two results including the state-of-the-art. Submit the change as a pull request.
- Datasets should have been used for evaluation in at least one published paper besides the one that introduced them.
- Use the Markdown table copied from the README template with columns Model, Score, Paper / Source and Code.
- Add new datasets and tasks in alphabetical order within their section.
| Model | Score | Paper / Source | Code |
| ------------- | :-----:| --- | --- |
| | | | |What people use it for
- A student comparing how named entity recognition is evaluated across datasets before writing a report.
- A researcher looking for a benchmark dataset for a specific task such as machine translation.
- A practitioner checking which published methods currently score highest on sentiment analysis.
- A contributor adding a new result from a published paper to an existing results table.
Frequently asked questions
Is NLP-progress free to use?
NLP-progress is released under the MIT licence, as stated in the project information. You can read the pages online or clone the repository for free.
Do I need to install anything to use NLP-progress?
No. The install method is read-online and no prerequisites are listed. You can browse the pages on the homepage, or clone the repository with git if you want local copies of the files.
Is NLP-progress a Python library?
No. The project is written in Python according to its metadata, but the README describes a reference of results rather than an installable package. It does not provide NLP code you can import.
Are the results on NLP-progress up to date?
The README relies on community contributions to add results, so some entries may be out of date. Where a task has a regularly maintained public leaderboard, the page points there, and you should check the source of any number before relying on it.
How is NLP-progress different from Papers with Code or the ACL Anthology?
NLP-progress is a curated set of task pages with datasets and state-of-the-art tables. Papers with Code, the NLP-SOTA tracker and the ACL Anthology are comparable projects that you can also use to find leaderboards and research papers; the ACL Anthology is an archive of papers rather than a results table.
How do I fix a wrong result on NLP-progress?
Edit the task's Markdown file on GitHub with the correct row, check the preview, and open a pull request. The README asks contributors to cite a published source for each result.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Use NLP-progress if you want a quick, free overview of NLP tasks, their benchmark datasets and the highest published scores. Skip it if you need software to run NLP code, or if you must have guaranteed current numbers, in which case check the linked leaderboards or a comparable tracker such as Papers with Code.
Install options & downloads