Use a local AI gateway with Claude Desktop

Quick start

How I would use it

1. 01

   Install Claude Desktop

   Install the official Windows or macOS app and open it once. Use the official download linked below.

2. 02

   Enable Developer Mode

   In versions that support this setup, open Help → Troubleshooting → Enable Developer Mode and allow the app to restart. On Windows, look for the application menu at the top left of the sign-in screen.

3. 03

   Find gateway settings

   Open Developer → Configure Third-Party Inference. Choose Gateway as the inference provider, Static API key as the credential kind and Bearer as the authentication scheme. If these controls are unavailable, check your app version and current Claude documentation before continuing.

4. 04

   Choose one setup below

   Install either OmniRoute or FreeLLMAPI. Use its local gateway key in Claude Desktop, not your upstream provider key. Enter the gateway server root without adding /v1.

Option 01 · Connect OmniRoute

OmniRoute Desktop runs a local gateway and dashboard. Keep it open while Claude Desktop uses the connection.

1. 01

   Install the desktop app

   Open OmniRoute Releases, choose the installer for your operating system, install it and launch the app.

2. 02

   Connect a provider

   In the dashboard, open Providers. Connect a supported provider account or add its API key. Confirm that the provider and at least one model are available.

3. 03

   Create a gateway key

   Open Endpoints and create an OmniRoute API key. Copy this local key; it is separate from your provider’s API key.

4. 04

   Configure Claude Desktop

   In third-party inference settings, select Gateway. Set the base URL to http://localhost:20128, use your OmniRoute key, select Static API key and Bearer, then test the connection and Apply Changes. Use the address shown by your gateway if it differs.

5. 05

   Test the Code tab

   After Claude Desktop relaunches, open Code and select a local project. Choose an available gateway model if offered. Ask ‘Explain this project’ and check the OmniRoute dashboard for the request.

6. 06

   If the gateway is unreachable

   Confirm OmniRoute is still running and its dashboard opens at http://localhost:20128. Check the current address and key. Use the server root without /v1.

Option 02 · Connect FreeLLMAPI

FreeLLMAPI Desktop provides a local gateway and dashboard. Its repository describes the service as intended for personal experimentation.

1. 01

   Install and open the dashboard

   Download a Windows or macOS installer from FreeLLMAPI Releases. Launch the app, then choose Open Dashboard from its tray or menu-bar icon. Keep the app running.

2. 02

   Add your provider key

   Open Keys and add an API key from a supported free-tier provider. Confirm the key appears healthy and a model is available.

3. 03

   Copy the unified key and address

   Copy the freellmapi-… unified key from the Keys header or tray popover. Copy the current local gateway address; the guide’s default is http://127.0.0.1:31415.

4. 04

   Configure Claude Desktop

   In third-party inference settings, select Gateway and paste the copied base URL and unified key. Choose Static API key and Bearer. Test the connection, then Apply Changes and allow Claude Desktop to relaunch.

5. 05

   Send a small Code request

   Open Code, choose a local project and an available model if offered, then send a simple prompt. Check the FreeLLMAPI dashboard to see which provider served the request.

6. 06

   If the connection fails

   Reopen the dashboard from the tray and copy its current address again. Confirm the app is running and the provider key is healthy.