learn_python3_spider
A Chinese-language tutorial series that teaches Python web scraping from beginner to advanced.
- Stars
- 22.3k
- Forks
- 3.9k
- Licence
- MIT
- Language
- Python
- Last update
- 25 days ago
- Type
- Learning resource
What it is
This repository is a collection of Chinese-language tutorials on writing web scrapers in Python, organized as a step-by-step course. The README lists lessons covering browser and phone packet capture, common libraries such as requests, BeautifulSoup, Selenium, Appium and Scrapy, proxies, captcha recognition, MySQL and MongoDB storage, and reverse-engineering anti-scraping defenses. Most lessons are linked articles rather than code in the repository.
- Follow the beginner lessons to learn requests and BeautifulSoup for pulling data from a web page
- Study the Selenium and Appium lessons to automate a browser or a phone app
- Learn to store scraped data in CSV, MySQL or MongoDB using the linked lessons
- Read the anti-scraping articles to understand how sites hide data and how such protections are examined
Great for
- Chinese-speaking beginners who want a structured path into Python scraping
- People who prefer learning from written tutorials in Chinese
- Developers who want examples of handling anti-scraping techniques like CSS obfuscation and JS reverse engineering
- Students looking for a worked example of scraping a site and saving the data to a database
Not ideal for
- Readers who need English-language material, since the README and most lessons are in Chinese
- Anyone looking for a software library or tool to install, because this is a tutorial collection
- People wanting a maintained, tested codebase, since the README does not describe one beyond the tutorial links
How to get it
- 1
Read it online
You'll need: Python 3
Clone the repository
git clone https://github.com/wistbean/learn_python3_spider.git - 2
Run it
- Open the README and start with the first lesson under the beginner section.
- Follow the linked article and set up the libraries it names.
- Work through the lessons in order, beginning with browser packet capture and requests.
- Use the course source code section to find complete example projects where listed.
- 3
Get the most out of it
- Read the lessons in order, since later topics build on earlier ones
- Pair the requests and BeautifulSoup lessons with a real site you are allowed to scrape
- Check each site's terms and robots rules before applying the techniques
- Use the database lessons to save your results as you go
Downloads
Strengths
- Covers a broad range of scraping topics in one ordered series
- Very popular, with about 22,000 stars on GitHub
- Released under the MIT license, so the material can be reused with few restrictions
Trade-offs
- Most content is in Chinese and lives in external linked articles, which may change or disappear
- The README does not describe runnable code or setup for the course itself
- Some topics, such as reverse engineering and bypassing protections, need care about the terms of the sites involved
How it compares
| Project | Stars | Type | Level | Updated | Best for |
|---|---|---|---|---|---|
| learn_python3_spider (this) | 22.3k | Learning resource | beginner | 25 days ago | Chinese-speaking beginners who want a structured path into Python scraping |
| requests-html | 13.8k | Library | beginner | 2.6 years ago | Python developers who already use requests and want to parse HTML in the same style |
| learn-python | 18.5k | Learning resource | beginner | 7 months ago | Beginners who want to learn Python syntax by reading and running examples |
| chinese-xinhua | 11.7k | Learning resource | beginner | 2.9 years ago | Developers building Chinese word games such as idiom chains |
| guiadevbrasil | 15.8k | Curated list | beginner | 2 months ago | Brazilian Portuguese speakers starting out in programming |
Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.