Step 1: Initialize (one-time setup) Skill

📋 Improvado MCP = CLI Tool + Python Library + Browser Automation

Core Principle: This skill provides a full-featured CLI tool and Python library for Improvado's MCP (Model Context Protocol) server access. The CLI dynamically discovers all 79+ MCP tools, manages persistent workspace sessions, and eliminates the need to write Python scripts for every query. Includes browser automation with Playwright for quick workspace testing.

MCP Server Endpoints (Continuant - TD):

graph TD
    MCP[MCP Server] --> Public[Public Endpoint]
    MCP --> Internal[Internal Endpoint]

    Public --> PubURL[/experimental/agent/api/mcp/v1/invoke/]
    Public --> SessionAuth[Session-Based Auth]
    Public --> Tools41[41 Tools]

    Internal --> IntURL[/experimental/agent/api/mcp-internal/v1/invoke/]
    Internal --> BasicAuth[Basic Auth]
    Internal --> Tools79[79 Tools]

    SessionAuth --> Impersonate[impersonate_agency]
    SessionAuth --> Cookie[x-dts-session-id header]

    BasicAuth --> DTS[DTS_SERVICE credentials]
    BasicAuth --> AuthHeader[Authorization: Basic]

Session-Based Workflow (Occurrent - LR):

graph LR
    A[Get Session] --> B[impersonate_agency workspace_id]
    B --> C[Receive session_id]
    C --> D[POST to /mcp/v1/invoke/]
    D --> E[Add x-dts-session-id header]
    E --> F[Call getConnectionsTool]
    F --> G[Receive connector list]
    G --> H[Parse results]

Ontological Rule: TD for server architecture (endpoints/auth), LR for session workflow (steps)

Primary source: .claude/skills/improvado-mcp/scripts/ implementation (2025-11-26) Session ID: Merged from v2.0.4 + v2.2.0 by Daniel Kravtsov (2025-11-26)

🚀 CLI Tool Usage (NEW in v2.0)

Why CLI? No more writing Python scripts for every query. All 79+ MCP tools accessible via command line with session persistence and dynamic tool discovery.

WORKSPACE CONTEXT REQUIREMENTS:

At the start of every new conversation/chat, you MUST:
- call 'getCurrentWorkspaceContextTool' to determine the active workspace context
- tell the user which context he is currently in
If no context is set (returns null) at the start of the conversation or on tool call which requires workspace, tell the user the context is not set so result of the operation might be ambigous.
To work with a different agency/workspace than the default:
- Call 'switchWorkspaceContextTool' with the desired rtbm_agency_id (Optionally provide workspace_id if a specific workspace is needed; if omitted, the default workspace for the agency will be used)
- All subsequent tool calls will execute in that workspace context
- Call 'getCurrentWorkspaceContextTool' to verify the context was switched
IMPORTANT: Always be aware of which workspace context you're operating in to avoid executing operations on the wrong agency's data.
Available workspace operations:
- getCurrentWorkspaceContextTool: Check current impersonation context
- switchWorkspaceContextTool: Switch to a specific agency and optionally a specific workspace. If workspace_id is not provided, switches to the default workspace of the agency.

¶1 Quick Start

# Step 1: Initialize (one-time setup)
cd $PROJECT_ROOT/.claude/skills/improvado-mcp/scripts
python3 mcp_client.py init

# Step 2: View session
python3 mcp_client.py session

# Step 3: Switch to agency workspace
python3 mcp_client.py switch --agency 7661

# Step 4: Query ClickHouse
python3 mcp_client.py query "SELECT * FROM im_XXXX_XXX.table LIMIT 10"

# Step 5: Call any MCP tool
python3 mcp_client.py call discoveryListConnectorsTool

¶2 All CLI Commands

# Init - Setup and authenticate
python3 mcp_client.py init
python3 mcp_client.py init --email user@improvado.io --url http://localhost:3000/...

# Session - Manage session state
python3 mcp_client.py session              # Show current session
python3 mcp_client.py session clear        # Clear session file
python3 mcp_client.py session info         # Detailed info

# Switch - Change workspace context
python3 mcp_client.py switch --agency 7661
python3 mcp_client.py switch --agency 7661 --workspace 1443

# List-tools - Browse available tools
python3 mcp_client.py list-tools                    # All tools
python3 mcp_client.py list-tools --refresh          # Force refresh cache
python3 mcp_client.py list-tools --search click     # Search tools
python3 mcp_client.py list-tools --category         # Group by category

# Call - Execute any MCP tool
python3 mcp_client.py call clickhouseTool --query "SELECT 1"
python3 mcp_client.py call webSearchTool --query "Improvado" --maxResults 5
python3 mcp_client.py call discoveryRequestTool --dataSource facebook --connectorId 52600

# Get help for specific tool (NEW in v2.0.1)
python3 mcp_client.py call clickhouseTool --help       # Show tool arguments schema
python3 mcp_client.py call visualizationTool --help    # Complex tool with enum/nested objects

# Query - ClickHouse shortcut
python3 mcp_client.py query "SELECT * FROM table"          # Table format
python3 mcp_client.py query "SELECT * FROM table" --format json
python3 mcp_client.py query "SELECT * FROM table" --format csv

¶3 Session Management

Session file: ~/.improvado_mcp_session

Contains:

Authentication (email, server URL)
Workspace context (agency ID, workspace ID, names)
Cached tools (5min TTL)
Last used timestamp

Benefits:

No repeated authentication
Workspace context persists across commands
Tools cache reduces API calls (auto-invalidated on workspace switch)
Show current context with every command

Notes:

Tools cache is automatically cleared when switching workspace context (tool availability depends on agency)
Short 5-minute TTL ensures new tools from server are discovered quickly
Cache still significantly reduces API calls during active sessions

¶4 CLI Examples

# Example 1: Find agency and switch context
python3 mcp_client.py query "
    SELECT rtbm_agency_id, agency_name
    FROM internal_analytics.src_dts_dsas_extraction_agency
    WHERE agency_name ILIKE '%MB2%'
" --format table

python3 mcp_client.py switch --agency 7661

# Example 2: List tools by category
python3 mcp_client.py list-tools --category

# Example 3: Search for specific tools
python3 mcp_client.py list-tools --search discovery

# Example 4: Call Discovery API
python3 mcp_client.py call discoveryListConnectorsTool --dataSource facebook

# Example 5: Multi-line query with heredoc
python3 mcp_client.py query "$(cat <<'SQL'
SELECT
    call_id,
    call_title,
    call_duration
FROM internal_analytics.dim_gong_calls
WHERE workspace_id = YYYY
ORDER BY started DESC
LIMIT 10
SQL
)"

# Example 6: Get help for tool arguments (NEW in v2.0.1)
python3 mcp_client.py call clickhouseTool --help
# Output:
# Tool: clickhouseTool
# Description: Execute SQL queries against ClickHouse database...
#
# Arguments:
#   --query <string> [REQUIRED]
#       The SQL query to execute

python3 mcp_client.py call visualizationTool --help
# Shows enum values, nested objects, array item schemas

🎯 Two Authentication Methods

¶1 Session-Based Auth (RECOMMENDED for workspace operations)

Use public endpoint with session from impersonate_agency:

from client_cases.sql_server import impersonate_agency
import requests

# Get session for specific workspace
session_id = impersonate_agency(workspace_id="79")  # Improvado Marketing

# Call MCP with session
response = requests.post(
    "https://report.improvado.io/experimental/agent/api/mcp/v1/invoke/",
    headers={
        "x-dts-session-id": session_id,
        "x-im-workspace-id": "79",
        "Content-Type": "application/json"
    },
    json={
        "jsonrpc": "2.0",
        "id": 1,
        "method": "tools/list",
        "params": {}
    }
)

When to use: Finding connectors, listing data tables, workspace-specific operations.

¶2 Basic Auth (for internal tools)

📚 Python Library Usage (For Scripts)

When to use: When you need to embed MCP client in Python scripts, automation, or complex workflows.

Use internal endpoint with DTS credentials:

from .claude.skills.improvado_mcp.scripts.mcp_client import ImprovadoMCPClient

client = ImprovadoMCPClient(
    url="https://report.improvado.io/experimental/agent/api/mcp-internal/v1/invoke/"
)
client.initialize()

When to use: Internal analytics queries, cross-workspace operations, tool development.

📐 Finding Connectors (Session-Based)

¶1 Common workflow: Find agency → Get connectors → Query data

Step 1: Find agency and workspace_id

export DTS_SERVICE_USER="daniel@improvado.io"
export DTS_SERVICE_PASSWORD=$DTS_PWD
export DTS_API_URL="https://report.improvado.io"

ch internal "
SELECT
    dts_agency_id,
    agency_name,
    workspace_id
FROM internal_analytics.dim_dts_dsas_extraction_connection
WHERE agency_name ILIKE '%Marketing%'
LIMIT 1
"

Step 2: Get session and list connectors

from client_cases.sql_server import impersonate_agency
import requests
import json

# Get session (example: Improvado Marketing workspace_id=YYYY)
session_id = impersonate_agency(workspace_id="79")

base_url = "https://report.improvado.io/experimental/agent/api/mcp/v1/invoke/"
headers = {
    "x-dts-session-id": session_id,
    "x-im-workspace-id": "79",
    "Content-Type": "application/json"
}

# Get connectors for specific datasource
response = requests.post(
    base_url,
    headers=headers,
    json={
        "jsonrpc": "2.0",
        "id": 1,
        "method": "tools/call",
        "params": {
            "name": "getConnectionsTool",
            "arguments": {
                "datasourceName": "hubspot"
            }
        }
    }
)

data = response.json()
connections = json.loads(data['result']['content'][0]['text'])

for conn in connections['results']:
    print(f"Connector ID: {conn['id']}")
    print(f"Title: {conn['title']}")
    print(f"Status: {conn['status']}")

¶2 Find connector details in ClickHouse

Once you have connector_id, get full details:

# Query internal_analytics for connector params
query = f"""
SELECT
    dts_datasource_connection_id,
    datasource_dsas_name,
    dts_datasource_connection_title,
    dts_datasource_connection_params
FROM internal_analytics.dim_dts_dsas_extraction_connection
WHERE dts_datasource_connection_id = {connector_id}
"""

result = internal.query(query)

🔄 Complete Connector Discovery Example

¶1 Find all HubSpot connectors for agency

import sys
sys.path.insert(0, 'chrome-extension-tcs')

from client_cases.sql_server import impersonate_agency
import requests
import json

# Example: Improvado Marketing agency
WORKSPACE_ID = "79"

# Get session
session_id = impersonate_agency(workspace_id=WORKSPACE_ID)

base_url = "https://report.improvado.io/experimental/agent/api/mcp/v1/invoke/"
headers = {
    "x-dts-session-id": session_id,
    "x-im-workspace-id": WORKSPACE_ID,
    "Content-Type": "application/json"
}

# List all datasources
datasources_response = requests.post(
    base_url,
    headers=headers,
    json={
        "jsonrpc": "2.0",
        "id": 1,
        "method": "tools/call",
        "params": {
            "name": "listDatasourcesTool",
            "arguments": {}
        }
    }
)

datasources = json.loads(datasources_response.json()['result']['content'][0]['text'])

# Filter for target datasources
targets = ['hubspot', 'salesforce', 'google_analytics_4']
for ds in datasources['results']:
    if any(t in ds['name'].lower() for t in targets):
        print(f"Found: {ds['name']} ({ds['title']})")

        # Get connections for this datasource
        conn_response = requests.post(
            base_url,
            headers=headers,
            json={
                "jsonrpc": "2.0",
                "id": 2,
                "method": "tools/call",
                "params": {
                    "name": "getConnectionsTool",
                    "arguments": {
                        "datasourceName": ds['name']
                    }
                }
            }
        )

        connections = json.loads(conn_response.json()['result']['content'][0]['text'])
        for conn in connections.get('results', []):
            print(f"  - {conn['title']} (ID: {conn['id']})")

🛠️ Available Tools

¶1 Public Endpoint Tools (41 total)

getConnectionsTool - Get connections for datasource
listDatasourcesTool - List all available datasources
listDataTablesTool - List data tables in workspace
listAllExtractsTool - List all extracts
clickhouseTool - ClickHouse queries (client data only)

¶2 Internal Endpoint Tools (79 total)

All public tools PLUS:
switchWorkspaceContextTool - Switch workspace context
getCurrentWorkspaceContextTool - Get current context
Full ClickHouse access including internal_analytics

⚠️ Security & Credentials

¶1 Why credentials not returned via API

MCP and DTS REST API intentionally DO NOT return OAuth credentials/tokens for security:

Defense-in-depth architecture:

API endpoints return connection metadata without sensitive data
Credentials stored encrypted in PostgreSQL
Only backend services have decryption keys
UI access requires user authentication + workspace permissions

This prevents:

Credential leakage through API queries
Unauthorized access to OAuth tokens
Cross-workspace credential exposure

¶2 How to access credentials

Option 1: Improvado UI (recommended)

Login to https://report.improvado.io/
Switch to target workspace
Connections → Select connector → View credentials

Option 2: OAuth flows work transparently

Backend services use encrypted credentials automatically
Discovery API accesses data without exposing tokens
No manual credential handling needed

¶3 What IS available via API:

Connector ID and title
Connection status
Account ID and name
Datasource metadata
Last update timestamps

🌍 Global Usage

¶1 This skill works globally from any location

The skill is available as a symlink in ~/.claude/skills/improvado-mcp and automatically loads credentials from:

$PROJECT_ROOT/.env

How it works:

Symlink: ~/.claude/skills/improvado-mcp → chrome-extension-tcs/.claude/skills/improvado-mcp
Credentials loaded automatically via hardcoded path (similar to supabase-agent pattern)
Works from ANY directory: /tmp, ~, other projects, etc.

Priority order:

chrome-extension-tcs/.env (base credentials - always loaded first)
Relative .env if different from (1) (project-specific overrides)
.env.local (local development overrides)

¶2 Usage from any location

# Works from ANY directory!
import sys
sys.path.insert(0, '$HOME/.claude/skills/improvado-mcp/scripts')
from mcp_client import ImprovadoMCPClient

# Credentials auto-loaded from chrome-extension-tcs/.env
client = ImprovadoMCPClient()
client.initialize()

🔧 Configuration

¶1 Environment variables (auto-loaded from global .env)

# $PROJECT_ROOT/.env
DTS_SERVICE_USER=daniel@improvado.io
DTS_SERVICE_PASSWORD=your_password
DTS_API_URL=https://report.improvado.io

Note: These are loaded automatically. You don't need to set them manually when using the skill.

¶2 Common workspace IDs (for reference, not hardcoded)

# Example workspace IDs - query internal_analytics for current values
WORKSPACES = {
    "improvado_marketing": "79",       # Marketing Agency - 3059
    "mb2_dental": "1443",              # ExampleClient - agency_id=XXXX
    "la_blanca": "21724",              # Manhattan Beachwear
}

# Always query for current workspace_id:
# SELECT workspace_id FROM internal_analytics.dim_dts_dsas_extraction_connection
# WHERE agency_id = YOUR_AGENCY_ID LIMIT 1

🐛 Troubleshooting

¶1 "Tool requires workspace context"

Use session-based auth with workspace_id:

session_id = impersonate_agency(workspace_id="79")
# Add x-im-workspace-id header to requests

¶2 "getConnectionsTool requires datasourceName"

Always provide datasourceName parameter:

"arguments": {
    "datasourceName": "hubspot"  # NOT connector_id!
}

¶3 Empty results from getConnectionsTool

Check datasource exists in workspace:

# First list all datasources
listDatasourcesTool({})
# Then get connections for specific datasource

¶4 403 Forbidden with internal endpoint

Use session-based auth instead for connector operations:

Internal endpoint (mcp-internal) requires special permissions
Public endpoint (mcp/v1) works with session-based auth

🌐 Browser Automation with DTS Cookie

¶1 Automated browser setup (Playwright)

Open incognito browser with DTS session cookie pre-set using Playwright:

# Quick command (agency_id only - auto-detects workspace_id)
export DTS_SERVICE_USER="daniel@improvado.io" && \
export DTS_SERVICE_PASSWORD=$DTS_PWD && \
export DTS_API_URL="https://report.improvado.io" && \
python algorithms/product_div/00_set_dts_cookie_browser.py 9898

# With explicit workspace_id
python algorithms/product_div/00_set_dts_cookie_browser.py 3059 79

What it does:

Gets workspace_id from ClickHouse (if not provided)
Calls impersonate_agency() to get DTS session
Launches Chrome with Playwright (incognito-like context)
Sets dts_sessionid cookie BEFORE page load
Opens Improvado AI Agent pre-authenticated
Keeps browser open for manual testing

Script location: /algorithms/product_div/00_set_dts_cookie_browser.py

¶2 Understanding impersonate user

When you see impersonate-user-{workspace_id}@external.improvado.io in browser:

Virtual user created by DTS service API (not real user in database)
Format: impersonate-user-79@external.improvado.io for workspace 79
Has Chief-level permissions for the workspace
Used for service automation and audit trail
Created via /api/dts/v2/agencies/service/impersonate endpoint

¶3 Manual browser setup (legacy method)

If Playwright unavailable, manual cookie setup:

// 1. Login to https://report.improvado.io
// 2. Open DevTools Console (F12 → Console)
// 3. Type "allow pasting" if browser blocks
// 4. Run:
document.cookie = "dts_sessionid=YOUR_SESSION_ID; path=/; domain=.improvado.io";
location.reload();

¶4 CLI-first principle (MANDATORY) - Always use CLI commands first before writing Python scripts:

✅ python3 mcp_client.py session - check current workspace
✅ python3 mcp_client.py call getCurrentWorkspaceContextTool - call any MCP tool
✅ python3 mcp_client.py query "SELECT ..." - execute ClickHouse query
❌ Writing inline Python with python -c "..." when CLI can do it
❌ Creating temporary Python scripts for one-off queries

Only write Python scripts when:

You need complex logic (loops, conditionals, data transformations)
You need to combine multiple MCP calls in a workflow
CLI command doesn't support the required functionality
User explicitly asks for a Python script

¶5 Search for agency

ch find search                 # Quick search by domain/name

# Or full SQL:
ch internal "
    SELECT dts_agency_id, agency_name, company_domain_id
    FROM biz_active_customers
    WHERE agency_name ILIKE '%search%'
"

Reference: $HOME/projects/ai-agent-improvado/main/README.md (lines 122-142)

✅ Validation Results

Version 2.3.0 Changes (2025-11-26):

✅ Merged v2.0.4 (CLI improvements) + v2.2.0 (Browser automation)
✅ Combined global .env loading with CLI tool features
✅ All documentation updated with merged functionality

Version 2.2.0 Changes (2025-11-21):

✅ Browser automation with Playwright for quick workspace testing
✅ Created /algorithms/product_div/00_set_dts_cookie_browser.py
✅ Documented impersonate user virtual accounts
✅ Manual fallback with browser console method

Version 2.0.4 Changes (2025-11-25):

✅ Reduced tools cache TTL from 24 hours to 5 minutes
✅ New tools on server now discovered within 5 minutes
✅ Changed CACHE_TTL_HOURS to CACHE_TTL_MINUTES in session.py
✅ Updated cache expiration logic to use minutes instead of hours

Version 2.0.3 Changes (2025-11-25):

✅ Fixed all command examples to use python3 instead of python or improvado-mcp
✅ Updated argparse epilog with correct command syntax
✅ Updated all error messages with proper command references
✅ Documentation consistency: SKILL.md now shows only python3 mcp_client.py

Version 2.0.2 Changes (2025-11-25):

✅ Tools cache auto-invalidation on workspace switch
✅ Fixes stale tool list when switching agencies (21 tools → 46 tools)
✅ set_workspace_context() now clears tools cache automatically
✅ Next list-tools after switch fetches fresh list from server

Version 2.0.1 Changes (2025-11-25):

✅ Tool help system: python3 mcp_client.py call <tool_name> --help
✅ Dynamic argument schema display from inputSchema
✅ Enum values shown in help (e.g., <enum[line, bar, area]>)
✅ Nested object properties displayed with indentation
✅ Array item schemas with required field markers
✅ Graceful handling of missing tools and edge cases

Version 2.0 Changes (2025-11-24):

✅ Full CLI tool with argparse and subcommands
✅ Session management (~/.improvado_mcp_session)
✅ Dynamic tool discovery and caching
✅ Tool registry with search and categorization
✅ Multiple output formats (table, json, csv)
✅ Persistent workspace context across commands

Session-Based Auth Testing (v2.0.0 - 2025-11-19):

✅ Connector Discovery - 3/3 tests passed (HubSpot, Salesforce, ID matching)
✅ MCP Tools - 41/41 public tools accessible
✅ Security Model validated (credentials not exposed via API)

Previous Testing (v1.0.1 - 2025-11-17):

✅ Workspace Switching - 2/2 tests passed
✅ ClickHouse Queries - 2/2 tests passed
✅ Discovery API - 2/3 tests passed
✅ Workspace Tools - 2/2 tests passed
✅ Integration Tools - 6/6 tools validated

Overall: Session-based auth fully functional, connector discovery workflow validated

Next Steps for v2.0 Testing:

[ ] Test all CLI subcommands (init, session, switch, list-tools, call, query)
[ ] Validate session persistence across multiple commands
[ ] Test tool caching with 5min TTL
[ ] Verify dynamic argument parsing in call command
[ ] Test all output formats (table, json, csv)

📎 Cross-References (Related Documentation & Source Code)

¶1 Discovery API Guides (chrome-extension-tcs)

discovery_api_guide.md — Full guide: local proxy, curl/Python examples, Google Ads GAQL, troubleshooting
docs/00_discovery_api_quick_guide.md — Quick reference: Salesforce, HubSpot, GA4, Google Search Console endpoints
docs/01_discovery_api_testing_results_agency_3059.md — Validation results on Improvado Marketing (agency 3059, workspace 79)
client_cases/sql_server.py — call_discovery_api() (line 165) and impersonate_agency() (line 105) Python helpers

¶2 ai-agent-improvado (TypeScript implementation)

main/utils/mcp/impersonation-context.ts — Workspace impersonation context storage (PostgreSQL + Redis, 15min TTL)
main/utils/ai/tools/mcp-dynamic-tools.ts — Dynamic tool creation factory (context-aware, per-agency toolsets)
main/utils/ai/tools/discovery/discovery-api-tool.ts — Discovery API tool (CSRF, cookie forwarding, Jinja2 headers)
main/utils/ai/tools/discovery/index.ts — Discovery toolset export (7 discovery tools)
main/app/(common)/api/mcp/v1/invoke/route.ts — Public MCP endpoint (41 tools)
main/app/(common)/api/mcp-internal/v1/invoke/route.ts — Internal MCP endpoint (79 tools, impersonation, SSE)
main/utils/mcp/request-context.ts — AsyncLocalStorage thread-safe context propagation

¶3 Obsidian Vault (Product Taxonomy & Operational Docs)

vault/Improvado/05-Product/.../P1 - AI-Agent/AI-Agents Tools/MCP/MCP.md — Product taxonomy entry (Notion-synced, repos: ai-agent-improvado, ai-assistant-backend, ai-actions)
vault/Improvado/05-Product/.../P1 - AI-Agent/AI-Agents Tools/MCP/Improvado MCP Agent Setup Guide.md — DTS user setup, 2-step auth flow, endpoint selection, troubleshooting table

Meta: v2.3.0 merges v2.2.0 (Browser automation with Playwright) and v2.0.4 (CLI improvements). Full CLI tool with 5-min cache TTL, tool help system (call <tool> --help), and workspace cache invalidation. Browser automation opens authenticated Improvado sessions instantly. Session management enables stateful multi-command workflows. Tool registry dynamically discovers 79+ tools. Global .env loading works from any directory. Backwards compatible - Python library usage still supported.

Agent Skills: Step 1: Initialize (one-time setup)

Install this agent skill to your local

Skill Files