Guide · Open source
How to use learn_python3_spider: a guide to its Python scraping lessons
Updated · written with AI from the project's own documentation and checked against our catalogue data
learn_python3_spider is a free, MIT-licensed collection of Chinese-language tutorials on writing web scrapers in Python, organized as a course that starts from zero. It is aimed at beginners who read Chinese and want a structured path into scraping. By working through the lessons, you can learn the main scraping tools, how to capture browser and phone traffic, and how to store scraped data in MySQL or MongoDB.
learn_python3_spiderwistbean/learn_python3_spider · ★ 22.3k · MIT Install options & downloads Quick start
- 1Clone the repository. Run git clone https://github.com/wistbean/learn_python3_spider.git in a terminal to download the README and links.
- 2Confirm Python 3 is installed. The repository lists Python 3 as the prerequisite, so check your version before you start the lessons.
- 3Open the README table of contents. Read the sections in the README to see which lesson covers each topic.
- 4Start with the basics. Begin with the 爬虫负基础 section and the first lesson on what a crawler is.
- 5Learn packet capture. Work through the lessons on Chrome capture and Fiddler or mitmproxy to see how sites send data.
- 6Practise a library. Follow the Requests, BeautifulSoup or Selenium lessons and try the example scrape they describe.
- 7Save your data. Read the CSV, MySQL and MongoDB lessons to store what you scrape.
What the repository contains
The README is a table of contents for a course titled 'python爬虫教程从0到1' (Python scraper tutorial from zero to one). Each entry links to an article, and most lessons are hosted outside the repository, so you read them online rather than run code from the repo.
- Beginner material: what a crawler is and how to get started.
- Packet capture: Chrome browser capture, Fiddler for phone traffic, and mitmproxy.
- Libraries: urllib, Requests, regular expressions, BeautifulSoup, Selenium, Appium and Scrapy.
- Advanced topics: IP proxies, captcha recognition, login simulation, multithreading and multiprocessing, CSS and JS anti-scraping, and distributed crawlers.
- Storage: CSV, MySQL and MongoDB.
Get a local copy of the repository
The source README gives a single command to clone the repository. Run it in a terminal where you want the files to go. The only stated prerequisite is Python 3.
- Install Python 3 if you do not already have it.
- Clone the repository with the command below.
git clone https://github.com/wistbean/learn_python3_spider.gitFollow the course in order
The lessons are numbered and build on each other. The README places them in sections such as 爬虫负基础 (basics), python爬虫前，抓包 (packet capture before scraping), python爬虫库的使用 (using libraries), and python爬虫进阶 (advanced anti-scraping). Starting from the first section is the simplest route.
Packet capture comes early because it shows you how a site sends its data. Lessons on Requests and BeautifulSoup come next, followed by Selenium and then the storage lessons.
Pick the lessons that match your goal
You do not have to read every lesson. The table of contents lets you jump to a topic. The README also includes a series of real example projects, such as scraping Dangdang's top-rated books and Douban's top films, plus a Scrapy lesson on Qiushibaike jokes.
- Learn a library: go to the library sections for Requests, BeautifulSoup, Selenium or Scrapy.
- Store data: read the CSV, MySQL and MongoDB lessons.
- Handle blocking: read the header, IP proxy and captcha lessons.
- Study anti-scraping: read the CSS encryption and JS reverse-engineering lessons.
Watch for the Chinese-language and link limits
The lessons are written in Chinese, and most link out to external pages, some of which are WeChat articles. The README does not describe a tested codebase for the lessons, so treat any code in the articles as examples to adapt and check yourself. Some links in the README point to a commercial proxy provider; they are part of the source list, not a recommendation from this guide.
What people use it for
- Learning Python scraping from scratch in Chinese.
- Seeing how Chrome and Fiddler capture traffic from a site or phone app.
- Following a worked example that scrapes a site and saves the results to MySQL or MongoDB.
- Studying how CSS font encryption and JS obfuscation are handled in scraping.
- Reading examples of distributed crawlers and Scrapy projects.
Frequently asked questions
Is learn_python3_spider free to use?
Yes. The repository is listed under the MIT licence, so you can read and reuse it under those terms. The tutorials themselves are linked articles, and the README does not describe any paid tier.
What do I need to install to use learn_python3_spider?
The source lists Python 3 as the prerequisite and gives one command: git clone https://github.com/wistbean/learn_python3_spider.git. There is no software library to install, because the repo is a set of tutorials.
Is the course available in English?
No. The README and most lessons are in Chinese. English readers would need a translation tool or a different course.
Does the repository contain runnable code for each lesson?
Not as far as the README shows. Most lessons are linked articles, so the code is in those articles rather than in the repository. The README does not describe a maintained, tested codebase.
What alternatives are there to learn_python3_spider?
The source lists Scrapy, Beautiful Soup, Requests-HTML and Crawlee as comparable projects. Scrapy and Beautiful Soup are libraries you install and use, while this repo teaches scraping through articles.
Is this a good first course for Python scraping?
It can be, if you read Chinese. The course starts with basics and covers packet capture, Requests, BeautifulSoup and Selenium, which is a logical order for beginners.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
Choose learn_python3_spider if you read Chinese and want a structured, article-based path through Python scraping, from packet capture to anti-scraping and databases. Pick an English-language course or a maintained library with tests, such as Scrapy or Requests-HTML, if you need English material or working code you can rely on.
Install options & downloads