GrepAI Embeddings with LM Studio Skill

GrepAI Embeddings with LM Studio

This skill covers using LM Studio as the embedding provider for GrepAI, offering a user-friendly GUI for managing local models.

When to Use This Skill

Want local embeddings with a graphical interface
Already using LM Studio for other AI tasks
Prefer visual model management over CLI
Need to easily switch between models

What is LM Studio?

LM Studio is a desktop application for running local LLMs with:

🖥️ Graphical user interface
📦 Easy model downloading
🔌 OpenAI-compatible API
🔒 100% private, local processing

Prerequisites

Download LM Studio from lmstudio.ai
Install and launch the application
Download an embedding model

Installation

Step 1: Download LM Studio

Visit lmstudio.ai and download for your platform:

macOS (Intel or Apple Silicon)
Windows
Linux

Step 2: Launch and Download a Model

Open LM Studio
Go to the Search tab
Search for an embedding model:
- nomic-embed-text-v1.5
- bge-small-en-v1.5
- bge-large-en-v1.5
Click Download

Step 3: Start the Local Server

Go to the Local Server tab
Select your embedding model
Click Start Server
Note the endpoint (default: http://localhost:1234)

Configuration

Basic Configuration

# .grepai/config.yaml
embedder:
  provider: lmstudio
  model: nomic-embed-text-v1.5
  endpoint: http://localhost:1234

With Custom Port

embedder:
  provider: lmstudio
  model: nomic-embed-text-v1.5
  endpoint: http://localhost:8080

With Explicit Dimensions

embedder:
  provider: lmstudio
  model: nomic-embed-text-v1.5
  endpoint: http://localhost:1234
  dimensions: 768

Available Models

nomic-embed-text-v1.5 (Recommended)

| Property | Value | |----------|-------| | Dimensions | 768 | | Size | ~260 MB | | Quality | Excellent | | Speed | Fast |

embedder:
  provider: lmstudio
  model: nomic-embed-text-v1.5

bge-small-en-v1.5

| Property | Value | |----------|-------| | Dimensions | 384 | | Size | ~130 MB | | Quality | Good | | Speed | Very fast |

Best for: Smaller codebases, faster indexing.

embedder:
  provider: lmstudio
  model: bge-small-en-v1.5
  dimensions: 384

bge-large-en-v1.5

| Property | Value | |----------|-------| | Dimensions | 1024 | | Size | ~1.3 GB | | Quality | Very high | | Speed | Slower |

Best for: Maximum accuracy.

embedder:
  provider: lmstudio
  model: bge-large-en-v1.5
  dimensions: 1024

Model Comparison

| Model | Dims | Size | Speed | Quality | |-------|------|------|-------|---------| | bge-small-en-v1.5 | 384 | 130MB | ⚡⚡⚡ | ⭐⭐⭐ | | nomic-embed-text-v1.5 | 768 | 260MB | ⚡⚡ | ⭐⭐⭐⭐ | | bge-large-en-v1.5 | 1024 | 1.3GB | ⚡ | ⭐⭐⭐⭐⭐ |

LM Studio Server Setup

Starting the Server

Open LM Studio
Navigate to Local Server tab (left sidebar)
Select an embedding model from the dropdown
Configure settings:
- Port: 1234 (default)
- Enable Embedding Endpoint
Click Start Server

Server Status

Look for the green indicator showing the server is running.

Verifying the Server

# Check server is responding
curl http://localhost:1234/v1/models

# Test embedding
curl http://localhost:1234/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nomic-embed-text-v1.5",
    "input": "function authenticate(user)"
  }'

LM Studio Settings

Recommended Settings

In LM Studio's Local Server tab:

| Setting | Recommended Value | |---------|-------------------| | Port | 1234 | | Enable CORS | Yes | | Context Length | Auto | | GPU Layers | Max (for speed) |

GPU Acceleration

LM Studio automatically uses:

macOS: Metal (Apple Silicon)
Windows/Linux: CUDA (NVIDIA)

Adjust GPU layers in settings for memory/speed balance.

Running LM Studio Headless

For server environments, LM Studio supports CLI mode:

# Start server without GUI (check LM Studio docs for exact syntax)
lmstudio server start --model nomic-embed-text-v1.5 --port 1234

Common Issues

❌ Problem: Connection refused ✅ Solution: Ensure LM Studio server is running:

Open LM Studio
Go to Local Server tab
Click Start Server

❌ Problem: Model not found ✅ Solution:

Download the model in LM Studio's Search tab
Select it in the Local Server dropdown

❌ Problem: Slow embedding generation ✅ Solutions:

Enable GPU acceleration in LM Studio settings
Use a smaller model (bge-small-en-v1.5)
Close other GPU-intensive applications

❌ Problem: Port already in use ✅ Solution: Change port in LM Studio settings:

embedder:
  endpoint: http://localhost:8080  # Different port

❌ Problem: LM Studio closes and server stops ✅ Solution: Keep LM Studio running in the background, or consider using Ollama which runs as a system service

LM Studio vs Ollama

| Feature | LM Studio | Ollama | |---------|-----------|--------| | GUI | ✅ Yes | ❌ CLI only | | System service | ❌ App must run | ✅ Background service | | Model management | ✅ Visual | ✅ CLI | | Ease of use | ⭐⭐⭐⭐⭐ | ⭐⭐⭐⭐ | | Server reliability | ⭐⭐⭐ | ⭐⭐⭐⭐⭐ |

Recommendation: Use LM Studio if you prefer a GUI, Ollama for always-on background service.

Migrating from LM Studio to Ollama

If you need a more reliable background service:

Install Ollama:

brew install ollama
ollama serve &
ollama pull nomic-embed-text

Update config:

embedder:
  provider: ollama
  model: nomic-embed-text
  endpoint: http://localhost:11434

Re-index:

rm .grepai/index.gob
grepai watch

Best Practices

Keep LM Studio running: Server stops when app closes
Use recommended model: nomic-embed-text-v1.5 for best balance
Enable GPU: Faster embeddings with hardware acceleration
Check server before indexing: Ensure green status indicator
Consider Ollama for production: More reliable as background service

Output Format

Successful LM Studio configuration:

✅ LM Studio Embedding Provider Configured

   Provider: LM Studio
   Model: nomic-embed-text-v1.5
   Endpoint: http://localhost:1234
   Dimensions: 768 (auto-detected)
   Status: Connected

   Note: Keep LM Studio running for embeddings to work.

Agent Skills: GrepAI Embeddings with LM Studio

Install this agent skill to your local

Skill Files