GrepAI Embeddings with OpenAI Skill

GrepAI Embeddings with OpenAI

This skill covers using OpenAI's embedding API with GrepAI for high-quality, cloud-based embeddings.

When to Use This Skill

Need highest quality embeddings
Team environment with shared infrastructure
Don't want to manage local embedding server
Willing to trade privacy for quality/convenience

Considerations

| Aspect | Details | |--------|---------| | ✅ Quality | State-of-the-art embeddings | | ✅ Speed | Fast, no local compute needed | | ✅ Scalability | Handles any codebase size | | ⚠️ Privacy | Code sent to OpenAI servers | | ⚠️ Cost | Pay per token | | ⚠️ Internet | Requires connection |

Prerequisites

OpenAI API key
Billing enabled on OpenAI account

Get your API key at: https://platform.openai.com/api-keys

Configuration

Basic Configuration

# .grepai/config.yaml
embedder:
  provider: openai
  model: text-embedding-3-small
  api_key: ${OPENAI_API_KEY}

Set the environment variable:

export OPENAI_API_KEY="sk-..."

With Parallel Processing

embedder:
  provider: openai
  model: text-embedding-3-small
  api_key: ${OPENAI_API_KEY}
  parallelism: 8  # Concurrent requests for speed

Direct API Key (Not Recommended)

embedder:
  provider: openai
  model: text-embedding-3-small
  api_key: sk-your-api-key-here  # Avoid committing secrets!

Warning: Never commit API keys to version control.

Available Models

text-embedding-3-small (Recommended)

| Property | Value | |----------|-------| | Dimensions | 1536 | | Price | $0.00002 / 1K tokens | | Quality | Very high | | Speed | Fast |

Best for: Most use cases, good balance of cost/quality.

embedder:
  provider: openai
  model: text-embedding-3-small

text-embedding-3-large

| Property | Value | |----------|-------| | Dimensions | 3072 | | Price | $0.00013 / 1K tokens | | Quality | Highest | | Speed | Fast |

Best for: Maximum accuracy, cost not a concern.

embedder:
  provider: openai
  model: text-embedding-3-large
  dimensions: 3072

Dimension Reduction

You can reduce dimensions to save storage:

embedder:
  provider: openai
  model: text-embedding-3-large
  dimensions: 1024  # Reduced from 3072

Model Comparison

| Model | Dimensions | Cost/1K tokens | Quality | |-------|------------|----------------|---------| | text-embedding-3-small | 1536 | $0.00002 | ⭐⭐⭐⭐ | | text-embedding-3-large | 3072 | $0.00013 | ⭐⭐⭐⭐⭐ |

Cost Estimation

Approximate costs per 1000 source files:

| Codebase Size | Chunks | Small Model | Large Model | |---------------|--------|-------------|-------------| | Small (100 files) | ~500 | $0.01 | $0.06 | | Medium (1000 files) | ~5,000 | $0.10 | $0.65 | | Large (10000 files) | ~50,000 | $1.00 | $6.50 |

Note: Costs are one-time for initial indexing. Updates only re-embed changed files.

Optimizing for Speed

Parallel Requests

GrepAI v0.24.0+ supports adaptive rate limiting and parallel requests:

embedder:
  provider: openai
  model: text-embedding-3-small
  api_key: ${OPENAI_API_KEY}
  parallelism: 8  # Adjust based on your rate limit tier

Parallelism recommendations:

Tier 1 (Free): 1-2
Tier 2: 4-8
Tier 3+: 8-16

Batching

GrepAI automatically batches chunks for efficient API usage.

Rate Limits

OpenAI has rate limits based on your account tier:

| Tier | RPM | TPM | |------|-----|-----| | Free | 3 | 150,000 | | Tier 1 | 500 | 1,000,000 | | Tier 2 | 5,000 | 5,000,000 |

GrepAI handles rate limiting automatically with adaptive backoff.

Environment Variables

Setting the API Key

macOS/Linux:

# In ~/.bashrc, ~/.zshrc, or ~/.profile
export OPENAI_API_KEY="sk-..."

Windows (PowerShell):

$env:OPENAI_API_KEY = "sk-..."
# Or permanently
[System.Environment]::SetEnvironmentVariable('OPENAI_API_KEY', 'sk-...', 'User')

Using .env Files

Create .env in your project root:

OPENAI_API_KEY=sk-...

Add to .gitignore:

.env

Azure OpenAI

For Azure-hosted OpenAI:

embedder:
  provider: openai
  model: your-deployment-name
  api_key: ${AZURE_OPENAI_API_KEY}
  endpoint: https://your-resource.openai.azure.com

Security Best Practices

Use environment variables: Never hardcode API keys
Add to .gitignore: Exclude .env files
Rotate keys: Regularly rotate API keys
Monitor usage: Check OpenAI dashboard for unexpected usage
Review code: Ensure sensitive code isn't being indexed

Common Issues

❌ Problem: 401 Unauthorized ✅ Solution: Check API key is correct and environment variable is set:

echo $OPENAI_API_KEY

❌ Problem: 429 Rate limit exceeded ✅ Solution: Reduce parallelism or upgrade OpenAI tier:

embedder:
  parallelism: 2  # Lower value

❌ Problem: High costs ✅ Solutions:

Use text-embedding-3-small instead of large
Reduce dimension size
Add more ignore patterns to reduce indexed files

❌ Problem: Slow indexing ✅ Solution: Increase parallelism:

embedder:
  parallelism: 8

❌ Problem: Privacy concerns ✅ Solution: Use Ollama for local embeddings instead

Migrating from Ollama to OpenAI

Update configuration:

embedder:
  provider: openai
  model: text-embedding-3-small
  api_key: ${OPENAI_API_KEY}

Delete existing index:

rm .grepai/index.gob

Re-index:

grepai watch

Important: You cannot mix embeddings from different models/providers.

Output Format

Successful OpenAI configuration:

✅ OpenAI Embedding Provider Configured

   Provider: OpenAI
   Model: text-embedding-3-small
   Dimensions: 1536
   Parallelism: 4
   API Key: sk-...xxxx (from environment)

   Estimated cost for this codebase:
   - Files: 245
   - Chunks: ~1,200
   - Cost: ~$0.02

   Note: Code will be sent to OpenAI servers.

Agent Skills: GrepAI Embeddings with OpenAI

Install this agent skill to your local

Skill Files