Skip to main content

LM Studio

ℹ️Note

캐럿(Caret) 기준 문서입니다. Caret v3.38.1 머지본을 따르며, 캐럿 전용 정책(지원 로컬 런타임, 인증/라우팅, 모델 제한)이 있을 경우 본문에서 <Note>로 표시합니다.

Setting Up LM Studio with Caret​

Run AI models locally using LM Studio with Caret.

Prerequisites​

  • Windows, macOS, or Linux computer with AVX2 support
  • Caret installed in VS Code

Setup Steps​

1. Install LM Studio​

  • Visit lmstudio.ai
  • Download and install for your operating system
LM Studio download page

2. Launch LM Studio​

  • Open the installed application
  • You'll see four tabs on the left: Chat, Developer (where you will start the server), My Models (where your downloaded models are stored), Discover (add new models)
LM Studio interface overview

3. Download a Model​

  • Browse the "Discover" page
  • Select and download your preferred model
  • Wait for download to complete
Downloading a model in LM Studio

4. Start the Server​

  • Navigate to the "Developer" tab
  • Toggle the server switch to "Running"
  • Note: The server will run at http://localhost:1234
Starting the LM Studio server

5. Configure Caret​

  1. Open VS Code
  2. Click Caret settings icon
  3. Select "LM Studio" as API provider
  4. Select your model from the available options
Configuring Caret with LM Studio

For the best experience with Caret, use Qwen3 Coder 30B A3B Instruct. This model delivers strong coding performance and reliable tool use.

Critical Settings​

After loading your model in the Developer tab, configure these settings:

  1. Context Length: Set to 262,144 (the model's maximum)
  2. KV Cache Quantization: Leave unchecked (critical for consistent performance)
  3. Flash Attention: Enable if available (improves performance)

Quantization Guide​

Choose quantization based on your RAM:

  • 32GB RAM: Use 4-bit quantization (~17GB download)
  • 64GB RAM: Use 8-bit quantization (~32GB download) for better quality
  • 128GB+ RAM: Consider full precision or larger models

Model Format​

  • Mac (Apple Silicon): Use MLX format for optimized performance
  • Windows/Linux: Use GGUF format

Enable Compact Prompts​

For optimal performance with local models, enable compact prompts in Caret settings. This reduces the prompt size by 90% while maintaining core functionality.

Navigate to Caret Settings → Features → Use Compact Prompt and toggle it on.

Important Notes​

  • Start LM Studio before using with Caret
  • Keep LM Studio running in background
  • First model download may take several minutes depending on size
  • Models are stored locally after download

Troubleshooting​

  1. If Caret can't connect to LM Studio:
  2. Verify LM Studio server is running (check Developer tab)
  3. Ensure a model is loaded
  4. Check your system meets hardware requirements