ProxyAsLocalModel

ProxyAsLocalModel serves as a bridge between remote Language Learning Model (LLM) APIs and popular JetBrains IDEs by allowing the use of these APIs as local models. This tool is particularly valuable for developers who utilize JetBrains AI Assistant and want to bypass limitations imposed by free plans of LLM services.

Background

In the current landscape, JetBrains AI Assistant offers limited usage quotas on its free plan, which may not suffice for users requiring more robust capabilities. Many have invested in other LLM API tokens, such as Gemini and Qwen, but encounter compatibility issues since only local models from platforms like LM Studio and Ollama are supported. As a solution, ProxyAsLocalModel has been developed to proxy these third-party LLM APIs as if they were local models from LM Studio or Ollama.

Technical Overview

ProxyAsLocalModel employs Ktor and kotlinx.serialization to create a simple proxy server that streamlines the process of integrating remote models with JetBrains IDEs. The application is packaged as a fat runnable JAR file and can also be compiled into a GraalVM native image, ensuring cross-platform support and rapid startup times. This setup leverages the functional programming paradigms of Kotlin, which inherently reduces memory usage and improves performance when deployed as a native image.

Features and Functionality

Currently, ProxyAsLocalModel supports proxying from various LLM providers, including:

• OpenAI
• Claude
• DashScope (Alibaba Qwen)
• Gemini
• Deepseek
• Mistral
• SiliconFlow

The application focuses on the streaming chat completion API, making it easy to receive and send messages seamlessly across different LLMs.

Configuration

Setting up ProxyAsLocalModel is straightforward. Upon initialization, a default configuration file is generated, which facilitates easy customization. Modifications to this configuration file are automatically detected and applied, ensuring the server runs with the most current settings without needing a restart.

Here’s an example of the configuration file:

lmStudio:
  port: 1234
  enabled: true
  host: 0.0.0.0
  path: /your/path
ollama:
  port: 11434
  enabled: true
  host: 0.0.0.0
  path: /your/path
client:
  socketTimeout: 1919810
  connectionTimeout: 1919810
  requestTimeout: 1919810
  retry: 3
  delayBeforeRetry: 1000
apiProviders:
  OpenAI:
    type: OpenAi
    baseUrl: https://api.openai.com/v1
    apiKey: <your_api_key>
    modelList:
      - gpt-4o
  Claude:
    type: Claude
    apiKey: <your_api_key>
    modelList:
      - claude-3-7-sonnet
  Qwen:
    type: DashScope
    apiKey: <your_api_key>
    modelList:
      - qwen-max
      - qwen-plus
      - qwen-turbo
      - qwen-long
  DeepSeek:
    type: DeepSeek
    apiKey: <your_api_key>
    modelList:
      - deepseek-chat
      - deepseek-reasoner
  Mistral:
    type: Mistral
    apiKey: <your_api_key>
    modelList:
      - codestral-latest
      - mistral-large
  SiliconFlow:
    type: SiliconFlow
    apiKey: <your_api_key>
    modelList:
      - Qwen/Qwen3-235B-A22B
      - Pro/deepseek-ai/DeepSeek-V3
      - THUDM/GLM-4-32B-0414
  Gemini:
    type: Gemini
    apiKey: <your_api_key>
    modelList:
      - gemini-2.5-flash-preview-04-17

In summary, ProxyAsLocalModel not only enhances the capabilities of JetBrains AI Assistant but also allows developers to leverage the power of various LLMs effectively.