Skip to content
Get packs

jieba

A Python library that splits Chinese text into words.

GitHub Download v0.42.1
Stars
35.2k
Forks
6.7k
Licence
MIT
Language
Python
Last update
2.2 years ago
Type
Library

No commits in over a year — check for forks or alternatives before adopting.

What it is

Jieba is a Python library for word segmentation of Chinese text. Chinese is written without spaces between words, so software needs a tool to decide where one word ends and the next begins. Jieba offers several modes, custom dictionaries, keyword extraction, and part-of-speech tagging, and is commonly used for text analysis and search indexing.

  • Split a Chinese sentence into words with jieba.cut or jieba.lcut
  • Add domain-specific terms with a custom dictionary via jieba.load_userdict
  • Extract top keywords from an article using jieba.analyse.extract_tags
  • Tag each word with its part of speech using jieba.posseg
  • Get word start and end positions in the original text with jieba.tokenize

Great for

  • Python developers who need to split Chinese text into words
  • Teams building Chinese search indexes or text analysis pipelines
  • Researchers doing keyword extraction or part-of-speech tagging on Chinese text
  • Students learning how Chinese word segmentation works

Not ideal for

  • People who need non-Python integration, since the README covers only the Python package
  • Projects needing state-of-the-art accuracy on specialized domains without custom dictionaries, since the README relies on a frequency dictionary plus an HMM model for unknown words
  • Windows users who want parallel segmentation, since the README says parallel mode does not yet support Windows
Step-by-step guide: how to use jiebaInstall, first run, examples, tips and FAQs

How to get it

  1. 1

    Install with a package manager

    You'll need: Python

    Install with pip

    pip install jieba
  2. 2

    Run it

    • Install the package with pip install jieba
    • Open a Python shell and run import jieba
    • Call jieba.lcut('我来到北京清华大学') to get a list of words
    • Try jieba.cut with cut_all=True to compare full mode
  3. 3

    Get the most out of it

    • Use jieba.load_userdict with a UTF-8 file to add words jieba does not know
    • Adjust individual word frequencies with jieba.suggest_freq when segmentation is wrong
    • Use jieba.analyse.extract_tags with topK and allowPOS to filter keyword results
    • Use jieba.posseg for part-of-speech tags in the ictclas-compatible format
    • Run python -m jieba on a file from the command line to segment it without writing code

Downloads

Source code (.zip)master branch · straight from GitHubRelease v0.42.1Release notes and source

Strengths

  • Offers several segmentation modes, including full, precise, and search-engine modes
  • Supports custom dictionaries and runtime word adjustments
  • Includes keyword extraction, part-of-speech tagging, and a command-line interface

Trade-offs

  • The README's older install notes reference Python 2/3 compatibility and paddle mode needs an extra package pinned to an old version
  • Full mode is fast but does not resolve ambiguity, per the README
  • Parallel mode is not supported on Windows

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.