Skip to content
Get packs

Guide · Open source

How to use FauxPilot: setup, requirements and FAQs

Updated · written with AI from the project's own documentation and checked against our catalogue data

FauxPilot is an open-source, locally hosted alternative to GitHub Copilot. It runs Salesforce CodeGen models on NVIDIA's Triton Inference Server with the FasterTransformer backend, so code suggestions are generated on your own hardware instead of a cloud service. This guide walks through the prerequisites, the install steps given in the project README, and where to go next, so you can decide whether it fits your setup.

FauxPilotfauxpilot/fauxpilot · ★ 14.7k · MIT

Quick start

  1. 1Check your hardware. Confirm you have an NVIDIA GPU with Compute Capability 6.0 or higher and enough VRAM for the model you plan to run.
  2. 2Install the prerequisites. Install Docker, docker compose 1.28 or later, nvidia-docker, curl and zstd.
  3. 3Clone the repository. Run git clone https://github.com/fauxpilot/fauxpilot to get the code.
  4. 4Run the setup script. Choose a model with the setup script, which downloads it in GPT-J format and converts it for FasterTransformer.
  5. 5Follow the server guide. Use documentation/server.md in the repository to finish setting up the FauxPilot server.
  6. 6Configure a client. Connect your editor or tool using the OpenAI API, Copilot plugin or REST API method in documentation/client.md.

What FauxPilot is and who it suits

FauxPilot is an attempt to build a locally hosted alternative to GitHub Copilot. It uses Salesforce CodeGen models inside NVIDIA's Triton Inference Server with the FasterTransformer backend. Code suggestions are generated on your own machines rather than by a cloud service.

People use it when they want Copilot-style completions without sending their code to a third party. It is a better fit for developers with an NVIDIA GPU who are comfortable with Docker and server setup.

  • Developers with an NVIDIA GPU who want to keep code on their own machines
  • Teams experimenting with self-hosted code completion
  • Users who already run Docker and are comfortable with server setup
  • People who want to connect editors or tools through an OpenAI-style or REST API

Check the prerequisites before you start

The README lists what you need. Confirm each item before you clone the repository.

The GPU requirement is the main constraint. The README requires an NVIDIA GPU with Compute Capability 6.0 or higher and enough VRAM for the model you want. If you have multiple GPUs, the model can be split across them. The README gives an example: two NVIDIA RTX 3080 GPUs should be able to run the 6B model with half on each card.

  • Docker
  • docker compose version 1.28 or higher
  • An NVIDIA GPU with Compute Capability 6.0 or higher and enough VRAM for your chosen model
  • nvidia-docker
  • curl and zstd, used to download and unpack the models

Clone the repository and run the setup script

Start by cloning the project from GitHub. The README's setup section tells you to run the setup script to choose a model. The script downloads the model from Hugging Face (under the Moyix account) in GPT-J format and converts it for use with FasterTransformer.

The README points to separate guides for the server and the clients. Follow the server guide in the repository's documentation folder to finish installing the FauxPilot server.

  • Run the setup script and choose a model. The VRAM requirements it lists are totals.
  • Read documentation/server.md in the repository for full server setup.
git clone https://github.com/fauxpilot/fauxpilot

Connect a client to the server

The README says FauxPilot can be reached through an OpenAI-style API, a Copilot plugin, or a REST API. Client setup is covered in a separate guide in the repository.

Once the server is running, point your tool at it using the method described in documentation/client.md. The README does not list specific editor plugins beyond these connection options, so check that file for the exact steps.

  • OpenAI API compatible connection
  • Copilot plugin connection
  • REST API connection
  • Client instructions: documentation/client.md

Know the limits before you rely on it

FauxPilot is not a polished, supported product. The README states there is no formal support or warranty. The project has a wiki and a discussion forum where you can ask questions, but no formal support is offered.

The README's terminology section explains terms such as CC (Compute Capability), FT (FasterTransformer), and GPT-J, which helps when reading the setup docs. The project's last push shown in our analysis is April 2024, so do not assume the model lineup is current.

  • No formal support or warranty, according to the README
  • Community help via the wiki and discussion forum linked in the README
  • Not for people without an NVIDIA GPU
  • Check model currency yourself before depending on it

What people use it for

  • Getting Copilot-style code completions on a machine with an NVIDIA GPU without sending code to a cloud service
  • Running a shared self-hosted completion server for a small team experimenting with local code completion
  • Connecting tools that accept an OpenAI-style or REST API to a local model
  • Splitting a larger model across multiple NVIDIA GPUs on one machine

Frequently asked questions

Is FauxPilot free to use?

FauxPilot is released under the MIT licence, according to the project listing. You can use, modify and share it under that licence's terms. Check the licence file in the repository for the full text.

Does FauxPilot need an NVIDIA GPU?

Yes. The README lists an NVIDIA GPU with Compute Capability 6.0 or higher as a prerequisite, along with enough VRAM for the model you choose. Users without an NVIDIA GPU are not a good fit for this project.

What do I need to install FauxPilot?

You need Docker, docker compose 1.28 or later, an NVIDIA GPU, nvidia-docker, curl and zstd. The install method is to clone the repository and build it, following the setup guides in the repo.

Does FauxPilot send my code to the cloud?

No. FauxPilot is designed to generate suggestions on your own hardware instead of a cloud service. That is its main purpose, as the README describes it as a locally hosted alternative to GitHub Copilot.

What models does FauxPilot use?

It uses Salesforce CodeGen models. The setup script downloads a model you choose from Hugging Face in GPT-J format and converts it for FasterTransformer. The README's example mentions the 6B model. Our analysis notes the last push shown is April 2024, so check which models are currently available before relying on them.

What are the alternatives to FauxPilot?

Comparable projects include Tabby, Continue and Ollama. Each takes a different approach to running code assistance locally, so compare them against your GPU, Docker and support needs before choosing.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

FauxPilot suits developers and teams with an NVIDIA GPU who already run Docker and want code completions that stay on their own machines. It is a poor choice if you lack an NVIDIA GPU, want a supported product, or need a guaranteed-current model lineup, since the README states there is no formal support or warranty.

Install options & downloads