Guide · Open source
How to use newspaper: install and extract articles in Python
Updated · written with AI from the project's own documentation and checked against our catalogue data
newspaper, published as newspaper3k, is an open-source Python 3 library for pulling the main text and metadata, such as the title and authors, out of a news article web page. It is aimed at Python developers, data researchers and hobbyists. This guide shows you what it is good for, how to install it with pip, and what to expect from its results.
newspaper3kcodelucas/newspaper · ★ 15.2k · MIT Install options & downloads Quick start
- 1Confirm Python 3 is installed. newspaper requires Python 3. Make sure you are running Python 3 before you install the library.
- 2Install the package. Run pip install newspaper3k in your terminal.
- 3Write a small script. Create a Python file that imports the library and passes it an article URL. The source describes the goal of returning the main text and metadata such as title and authors.
- 4Test on several sites. Run your script on articles from the sites you plan to use, because extraction quality varies by page layout.
- 5Check the output. Compare the extracted text and title against the original page to see whether the results are good enough for your project.
- 6Try alternatives if needed. If results are poor, test trafilatura, goose3 or readability-lxml on the same URLs.
What newspaper does
Given an article URL, newspaper aims to return the main text of the article along with metadata such as the title and authors. The point is that you do not have to write your own parser for each news site.
People use it to collect news articles for analysis or to build news aggregators.
Check the prerequisites
newspaper needs Python 3. You also need pip, the Python package installer, to install it. You will be writing Python code to use the library, so this tool is not suited to people who do not program.
Install newspaper3k with pip
The project's README gives a single install command. Run it in your terminal or command prompt:
pip install newspaper3kUnderstand what to expect from the results
Extraction quality varies by page layout, so newspaper is not guaranteed to be accurate on every site. Check the output on the sites you care about before you rely on it.
The README provided no details on current maintenance status beyond the latest release, so check the project's activity yourself before building something long-lived on it.
- Works best when you want article text and basic metadata from many news sites.
- Accuracy depends on how each site is laid out.
- Requires writing Python code; there is no point-and-click interface described in the source.
Compare it with similar projects
If newspaper does not extract well from your target sites, the source lists comparable projects you can test: trafilatura, goose3 and readability-lxml. Try the same set of URLs with each and compare the text you get.
What people use it for
- Collecting news article text for text analysis or research.
- Building a news aggregator without writing a separate parser for each site.
- Experimenting with web scraping of news sites as a hobby project.
- Gathering article titles and authors alongside their text.
Frequently asked questions
Is newspaper free to use?
newspaper is published under the MIT licence, according to the project's information. That licence permits free use, and the source does not mention any paid tier.
What do I need to run newspaper?
The source lists Python 3 as the prerequisite. Install the library with pip install newspaper3k.
How do I install newspaper?
Run pip install newspaper3k in your terminal. The README gives this as the install method.
Why is the extracted text wrong for some sites?
Extraction quality varies by page layout, so the library does not give guaranteed accuracy on every site. Test it on your target sites and compare the results with the original pages.
What are alternatives to newspaper?
The source lists trafilatura, goose3 and readability-lxml as comparable projects. You can run each on the same URLs to see which gives the cleanest text for your sites.
Is newspaper still maintained?
The README provided no details on current maintenance status beyond the latest release. Check the project's repository activity before depending on it for a long-term project.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
newspaper suits Python developers, data researchers and hobbyists who want article text and basic metadata from news sites without writing their own parsers. Pick something else if you are not comfortable writing Python, if you need guaranteed accuracy on every site, or if you need a tool with confirmed active maintenance.
Install options & downloads