Skip to content
Get packs

Guide · Open source

How to use katana: crawl a site and list its URLs

Updated · written with AI from the project's own documentation and checked against our catalogue data

Katana is an open-source command-line crawler from ProjectDiscovery. It visits a website, follows its links and lists the URLs and endpoints it finds. This guide shows security testers and developers how to install it, run a basic crawl, switch on headless mode for JavaScript-heavy pages and save structured output. Only crawl sites you are authorized to test.

katanaprojectdiscovery/katana · ★ 17.8k · MIT

Quick start

  1. 1Check the prerequisites. Confirm you have Go 1.26.10 or newer. Install Google Chrome only if you plan to use headless mode with a local browser.
  2. 2Install katana. Run the Go install command from the README, or download a pre-compiled binary from the releases page.
  3. 3Confirm the install. Run katana -h to display the help and list of switches.
  4. 4Run a basic crawl. Point katana at a site you are authorized to test with -u and set a depth with -d.
  5. 5Add headless mode if needed. For JavaScript-heavy pages, add -headless and optionally -system-chrome.
  6. 6Narrow the scope. Use -e to exclude hosts and -iqp or -fsu to remove duplicate-looking URLs.
  7. 7Export structured output. Add -form-extraction or -xhr-extraction and save the results for your next tool.

What katana does and who it is for

Katana is a command-line tool that automatically visits a website, follows its links and lists the URLs and endpoints it finds. It can run without a browser for speed, or in headless mode to handle pages that rely on JavaScript.

People typically use it to map a site's pages and inputs as a first step in reviewing it, or to feed the results into other tools. It is a terminal program with many flags, so it suits people comfortable with the command line.

  • Security testers mapping pages and endpoints of a site they are authorized to test
  • Developers who need a scriptable crawler for automation pipelines
  • People crawling JavaScript-heavy pages with headless Chrome
  • Anyone who wants structured JSON output of discovered URLs and forms

Install katana with Go

katana requires Go 1.26.10 or newer. If you hit installation problems, the project recommends trying the latest available version of Go. You can also download a pre-compiled binary from the project's release page instead of building it yourself.

Install with the command below. Headless mode with a local browser also needs Google Chrome installed.

  • Prerequisite: Go 1.26.10+
  • Prerequisite: Google Chrome, only if you use headless mode with a local browser
  • Alternative: download a pre-compiled binary from the releases page
CGO_ENABLED=1 go install github.com/projectdiscovery/katana/cmd/katana@latest

Run your first crawl

Check that the tool is installed and see the full list of switches with the help flag. The help output is long, so you can also read it in your terminal as a reference.

Pass a target with -u for a single URL. The README's Docker example uses -u with a target URL, which shows the same pattern. Use -list (or -u with a list) to crawl several targets, and -d to set how deep the crawler goes (default 3).

  • -u: target URL or list to crawl
  • -d: maximum crawl depth (default 3)
  • -ct: maximum crawl duration, with units s, m, h or d
  • -timeout: seconds to wait per request (default 10)
  • -retry: number of retries per request (default 1)
katana -h

Use headless mode for JavaScript pages

Standard mode fetches pages without a browser, which is faster. For pages that build their links with JavaScript, enable headless mode with -headless (or -hl). The README marks headless hybrid crawling as experimental.

To use Chrome already installed on your machine rather than the browser katana provides, add -system-chrome. Use -show-browser to watch the browser while it runs, and -no-sandbox only if you understand what that option changes in your environment. The Docker example in the README uses the system-chrome and headless flags together.

  • -headless: enable headless crawling (experimental)
  • -system-chrome: use a locally installed Chrome
  • -show-browser: display the browser window
  • -page-load-strategy: heuristic (default), load, domcontentloaded, networkidle or none
katana -u https://example.com -headless -system-chrome

Control scope and filter the output

Scope control keeps the crawl inside the area you care about. Use -exclude to skip hosts matching a filter such as 'cdn', 'private-ips', a CIDR, an IP or a regex. Use -ignore-query-params to stop repeats of a path that differ only in query values, and -filter-similar to skip URLs that look alike, such as /users/123 and /users/456.

Use -max-domain-pages to cap how many pages are crawled per domain. The default is unlimited.

  • -e / -exclude: exclude hosts by filter, CIDR, IP or regex
  • -iqp / -ignore-query-params: skip same path with different query values
  • -fsu / -filter-similar: filter look-alike URLs
  • -mdp / -max-domain-pages: limit pages per domain

Save results as JSON or to a file

Katana can read targets from STDIN, a URL or a list, and write output to STDOUT, a file or JSON, according to the project's feature list. For structured output, the README describes JSON lines output, and -form-extraction adds form, input, textarea and select elements to that output. Use -xhr-extraction to add XHR request URLs and methods.

The README's flag list does not name a single flag for file output, so check the full -h output for the exact option on your version before scripting it.

  • Input: STDIN, URL or list
  • Output: STDOUT, file or JSON
  • -form-extraction: include forms in JSON lines output
  • -xhr-extraction: include XHR URL and method in JSON lines output
katana -u https://example.com -form-extraction -xhr-extraction

Tune depth, strategy and request behaviour

The visit strategy controls the order in which pages are crawled. The options are depth-first (the default), breadth-first and priority. Proxy and custom headers are available for requests that need them, and -tech-detect enables technology detection.

Some options are marked experimental in the README, including automatic form filling, TLS impersonation and the AI crawl decisions. Treat those as unproven until you test them on your target.

  • -s / -strategy: depth-first, breadth-first or priority
  • -proxy: http or socks5 proxy
  • -H / -headers: custom header or cookie in header:value format
  • -td / -tech-detect: enable technology detection
  • -aff / -automatic-form-fill: experimental

What people use it for

  • Mapping the pages and endpoints of a site during an authorized security review
  • Feeding discovered URLs into another scanner or analysis script
  • Crawling JavaScript-heavy pages with headless Chrome to find links that plain fetching misses
  • Collecting form and input details as JSON for later review
  • Running scheduled crawls inside an automation pipeline

Frequently asked questions

Is katana free to use?

Yes. The project is licensed under the MIT licence, as stated in the project's analysis. Check the repository's licence file for the full terms before you redistribute it.

How do I install katana?

Install it with Go using CGO_ENABLED=1 go install github.com/projectdiscovery/katana/cmd/katana@latest, which requires Go 1.26.10 or newer. You can also download a pre-compiled binary from the releases page.

Do I need Chrome to use katana?

No. Standard mode runs without a browser. Google Chrome is only needed if you use headless mode with a local browser, for example with -system-chrome.

How do I crawl JavaScript-heavy pages with katana?

Add the -headless flag so katana uses a headless browser. The README marks this mode as experimental, so check the results against the site manually.

What are the alternatives to katana?

The project lists Hakrawler, Colly, Gospider and Scrapy as comparable projects. Hakrawler and Gospider are also command-line tools, while Colly and Scrapy are frameworks you write code against. This guide does not compare their performance.

Why does katana say my target is blocked or my crawl stops early?

The README does not describe specific troubleshooting steps. Check your Go version, raise -timeout or -ct if requests are slow, and confirm your scope filters with -e are not excluding the hosts you want.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

Katana suits security testers who are authorized to map a site and developers who want a scriptable, MIT-licensed crawler for pipelines, especially for JavaScript pages in headless mode. It is not a point-and-click tool, and some options are marked experimental, so people who need a polished graphical interface or features the README does not document should look at alternatives such as Scrapy or Colly.

Install options & downloads