Skip to content
Get packs

Guide · Open source

How to use jieba: install and segment Chinese text in Python

Updated · written with AI from the project's own documentation and checked against our catalogue data

Jieba is a Python library that splits Chinese text into words. Chinese is written without spaces between words, so a tool has to decide where each word begins and ends before most text processing can work. This guide shows you how to install jieba, choose a segmentation mode, and use custom dictionaries, keyword extraction and part-of-speech tagging.

jiebafxsjy/jieba · ★ 35.2k · MIT

Quick start

  1. 1Install Python and jieba. Make sure Python is installed, then run pip install jieba.
  2. 2Import the library. Add import jieba at the top of your script.
  3. 3Choose a mode. Use jieba.cut for precise segmentation, cut_all=True for full mode, or jieba.cut_for_search for search indexing.
  4. 4Add domain words. Create a UTF-8 text file with one word per line and load it with jieba.load_userdict.
  5. 5Check the output. Print the result with a delimiter such as '/' and adjust the dictionary if words are split wrongly.

Install jieba with pip

Jieba is a Python package, so you need Python first. The README lists pip as the simplest installation route and says the code works with both Python 2 and Python 3.

After installing, import it in your script with import jieba.

pip install jieba

Segment text with jieba.cut and the four modes

jieba.cut returns a generator you can loop over, and jieba.lcut returns a list directly. The default is precise mode, which tries to cut the sentence as accurately as possible and suits text analysis. Full mode scans out every possible word but cannot resolve ambiguity. Search engine mode, via jieba.cut_for_search, re-splits long words on top of precise mode to improve recall, which suits building search indexes. Paddle mode uses a PaddlePaddle sequence labelling model and also supports part-of-speech tagging.

  • Precise mode: jieba.cut(text) or cut_all=False
  • Full mode: jieba.cut(text, cut_all=True)
  • Search engine mode: jieba.cut_for_search(text)
  • Paddle mode: requires the paddlepaddle-tiny package, see the next section
import jieba seg_list = jieba.cut("我来到北京清华大学", cut_all=False) print("Default Mode: " + "/ ".join(seg_list))

Use paddle mode only if you need it

Paddle mode needs the paddlepaddle-tiny package, which the README pins to a specific version. Install it separately, then call jieba.enable_paddle() before using use_paddle=True. The README notes that paddle mode is supported from jieba v0.40 onward and that older versions should be upgraded.

If you only need standard segmentation, you can skip this step.

pip install paddlepaddle-tiny==1.6.1

Add a custom dictionary for your own words

Jieba has a built-in dictionary and can guess new words, but adding your own words gives more reliable results. A custom dictionary file has one word per line. Each line has three parts separated by spaces, in this order: the word, its frequency (optional) and its part-of-speech tag (optional). The file must be UTF-8 if you pass a path or open it in binary mode.

To change words while a program runs, use add_word, del_word and suggest_freq. The README warns that automatically calculated frequencies may not take effect when HMM new-word discovery is in use.

  • Load a file: jieba.load_userdict(file_name)
  • Add a word at runtime: jieba.add_word(word, freq=None, tag=None)
  • Remove a word: jieba.del_word(word)
创新办 3 i 云计算 5 凱特琳 nz 台中

Extract keywords and tag parts of speech

Keyword extraction uses jieba.analyse. extract_tags ranks words by TF-IDF weight and returns the top 20 by default. TextRank is available through jieba.analyse.textrank and filters to nouns and verbs by default. For part-of-speech tagging, use jieba.posseg, which uses a tag set compatible with ICTCLAS.

The README also lists parallel segmentation with enable_parallel, tokenize for start and end positions, a command-line mode, and a ChineseAnalyzer for Whoosh. Parallel mode does not yet support Windows.

  • jieba.analyse.extract_tags(sentence, topK=20, withWeight=False, allowPOS=())
  • jieba.analyse.textrank(sentence, topK=20, withWeight=False, allowPOS=('ns', 'n', 'vn', 'v'))
  • jieba.posseg.cut(text) yields word and flag pairs
import jieba.analyse tags = jieba.analyse.extract_tags("sentence text here", topK=5)

Run jieba from the command line

The README documents a module entry point that segments a file and lets you redirect the output. Options control the delimiter between words and whether part-of-speech tags are shown.

python -m jieba news.txt > cut_result.txt

What people use it for

  • Splitting Chinese articles into words before counting term frequency
  • Building a Chinese search index that needs finer-grained tokens
  • Extracting keywords from Chinese documents with TF-IDF or TextRank
  • Tagging parts of speech in Chinese text for linguistic research

Frequently asked questions

Is jieba free to use?

Yes. The project is released under the MIT licence, as stated in the README. You can check the licence file in the repository for the full terms.

What do I need to run jieba?

The README lists Python as the prerequisite and says the code is compatible with Python 2 and 3. Install it with pip install jieba. Paddle mode needs the separate paddlepaddle-tiny package.

Why does jieba split a word incorrectly?

Jieba uses a frequency dictionary and an HMM model for unknown words, so it can make mistakes on specialised terms. Add those words with a custom dictionary or suggest_freq. The README also notes that automatically calculated frequencies may not apply when HMM discovery is used.

What is the difference between cut and cut_for_search?

jieba.cut is for general text analysis and uses precise mode by default. jieba.cut_for_search re-splits longer words to give more fine-grained tokens, which the README recommends for building search indexes.

Can I use jieba outside Python?

The README covers only the Python package, so this guide does not describe integrations with other languages. If you need another language, check the project's other resources or consider a comparable tool.

Does jieba work on Windows?

Segmentation works in general, but the README says parallel segmentation does not yet support Windows, because it relies on Python's multiprocessing module in a way that is not supported there.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Jieba suits Python developers, search teams and students who need a straightforward way to split Chinese text into words, with options for keywords and part-of-speech tags. If you need non-Python integration, state-of-the-art accuracy on specialised domains without custom dictionaries, or parallel processing on Windows, compare it with alternatives such as pkuseg, THULAC, LTP or HanLP.

Install options & downloads