Met Museum Paintings Semantic Search Skill

Met Museum Paintings Semantic Search

Search ~2,300 paintings from the Met Museum Open Access collection using AI-powered semantic search.

Keywords: art search, museum, paintings, Met Museum, semantic search, artwork discovery, visual similarity, art history

When to Use This Skill

Use this skill when users:

Ask to find artworks by description ("find paintings of people on bridges")
Want to explore museum collections
Ask about specific artworks or artists at the Met
Want to find visually similar artworks
Upload an image to find matching paintings

API Base URL

https://museum-semantic-search.vercel.app/api/mcp

API Endpoints

POST /search - Search Artworks

Search the collection using text queries with optional filters.

curl -X POST https://museum-semantic-search.vercel.app/api/mcp/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "person with a dog",
    "mode": "semantic",
    "limit": 10
  }'

Request body:

{
  "query": "string (required)",
  "mode": "hybrid | semantic | keyword",
  "detail": "minimal | standard | full",
  "filters": {
    "artistName": "string (fuzzy matched)",
    "yearStart": 1800,
    "yearEnd": 1900,
    "department": "string (exact match)",
    "culture": "string (exact match)",
    "classification": "string (exact match)",
    "tags": ["string (exact match)"],
    "medium": "string (fuzzy matched)",
    "country": "string (exact match)",
    "onView": true,
    "isPublicDomain": true
  },
  "limit": 10,
  "offset": 0
}

Mode selection:

"hybrid" (default, recommended): Combines keyword + semantic search. Best for most queries.
"semantic": For purely conceptual/descriptive queries like "lonely figure in nature", "woman looking in mirror", "stormy seascape"
"keyword": For exact matches like artist names or specific titles

Detail levels:

"minimal": Just id, score, title, artist, date (fastest)
"standard" (default): + medium, classification, department, tags, culture, image_url, source_url, alt_text (concise AI description)
"full": + long_description (detailed AI description), similar_artworks, dimensions, etc. Use when you need to verify semantic content.

Response:

{
  "meta": {
    "query": "person with a dog",
    "mode": "hybrid",
    "total": 150,
    "returned": 10,
    "offset": 0,
    "limit": 10,
    "has_more": true,
    "filters_applied": {},
    "processing_time_ms": 234
  },
  "results": [
    {
      "id": "met_436524",
      "score": 0.85,
      "title": "A Boy with a Flying Squirrel",
      "artist": "John Singleton Copley",
      "date": "1765",
      "medium": "Oil on canvas",
      "classification": "Paintings",
      "department": "American Wing",
      "image_url": "https://images.metmuseum.org/...",
      "thumbnail_url": "https://images.metmuseum.org/.../small/...",
      "tags": ["Boys", "Portraits", "Squirrels"],
      "culture": "American",
      "source_url": "https://www.metmuseum.org/art/collection/search/436524",
      "alt_text": "Portrait of a young boy in blue satin holding a gold chain attached to a pet flying squirrel on a polished table"
    }
  ]
}

GET /artwork/{id} - Get Artwork Details

Get complete details for a specific artwork.

curl https://museum-semantic-search.vercel.app/api/mcp/artwork/met_436524

Response includes:

Basic info: title, titles (alternate), artist, artists (with roles, bio, dates)
Dates: date, date_start, date_end
Physical: medium, dimensions
Classification: classification, department, object_name
Context: culture, period, dynasty, country, region
Collection: collection, collection_id, credit_line, accession_year
Status: is_public_domain, is_highlight, on_view, gallery_number
Image: url, thumbnail_url, width, height
AI description: alt_text, long_description, emoji_summary
Tags: array of keywords
Similar artworks: id, similarity_type, confidence, explanation
Links: source_url (Met Museum page)

GET /similar/{id} - Find Similar Artworks

Find artworks similar to a given artwork using different methods.

# Precomputed (LLM-curated, recommended - includes explanations)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=precomputed&limit=10"

# Embedding-based visual similarity (CLIP)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=embedding&model=jina_clip&limit=10"

# Embedding-based conceptual similarity (text)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=embedding&model=jina_text&limit=10"

# Metadata-based (same artist, period, culture, etc.)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=metadata&limit=10"

# Combined (fuses metadata + text + image embeddings)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=combined&limit=10"

Query parameters:

method: "precomputed" (default) or "embedding"
model: "jina_clip" (visual) or "jina_text" (conceptual) - only for embedding method
limit: 1-50 (default 10)

Response:

{
  "meta": {
    "source_artwork": {
      "id": "met_436524",
      "title": "...",
      "artist": "..."
    },
    "method": "precomputed",
    "model": null,
    "total": 10,
    "returned": 10,
    "processing_time_ms": 45
  },
  "results": [
    {
      "id": "met_437234",
      "score": 0.92,
      "title": "...",
      "artist": "...",
      "similarity_type": "compositional",
      "similarity_explanation": "Both feature a central figure with a dog..."
    }
  ]
}

POST /image-search - Search by Image

Find visually similar artworks by uploading an image.

curl -X POST https://museum-semantic-search.vercel.app/api/mcp/image-search \
  -H "Content-Type: application/json" \
  -d '{
    "image": "data:image/jpeg;base64,/9j/4AAQ...",
    "limit": 10
  }'

Request body:

{
  "image": "base64-encoded image (with or without data URL prefix)",
  "mimeType": "image/jpeg",
  "filters": { },
  "limit": 10,
  "detail": "standard"
}

Response: Same format as /search but includes embedding_time_ms in meta.

GET /filters - Get Filter Options

Get available filter values for the collection. Supports search mode to find specific filter values.

# Get all top filter values
curl https://museum-semantic-search.vercel.app/api/mcp/filters

# Search for specific filter values (case-insensitive)
curl "https://museum-semantic-search.vercel.app/api/mcp/filters?search=japan"
curl "https://museum-semantic-search.vercel.app/api/mcp/filters?search=oil&field=mediums"

Query parameters (for search mode):

search: Case-insensitive search string (e.g., "japan", "portrait", "oil")
field: Limit to specific field: departments, classifications, cultures, mediums, tags, countries, periods
limit: Max results per field (1-100, default 20)

Response (default mode):

{
  "departments": [{ "value": "European Paintings", "count": 1500 }, ...],
  "classifications": [{ "value": "Paintings", "count": 2300 }, ...],
  "cultures": [{ "value": "French", "count": 450 }, ...],
  "mediums": [{ "value": "Oil on canvas", "count": 1800 }, ...],
  "tags": [{ "value": "Portraits", "count": 800 }, ...],
  "date_range": { "min": 1250, "max": 1950 },
  "schema": {
    "artistName": { "type": "string", "description": "...", "examples": [...] },
    "medium": { "type": "string", "description": "...", "examples": [...] },
    "yearStart": { "type": "number", "description": "...", "min": 1250, "max": 1950 },
    ...
  }
}

Response (search mode):

{
  "search_query": "japan",
  "field_filter": "all",
  "results": {
    "cultures": [
      { "value": "Japan", "count": 45 },
      { "value": "probably Japan", "count": 12 }
    ]
  }
}

Workflow

Step 1: Discover the Collection

Call GET /filters first to see available departments, cultures, mediums, tags, and date ranges. Use search mode to find specific filter values.

Step 2: Search

Use POST /search - hybrid mode (default) works well for most queries.

Step 3: Get Details

Call GET /artwork/{id} on interesting results, or use detail: "full" in search to get complete details inline.

Step 4: Explore

Use GET /similar/{id} to discover related works.

Example Queries

| Query | Recommended Mode | Why | |-------|------------------|-----| | "person with a dog" | hybrid (default) | Works well for most queries | | "woman looking in mirror" | semantic | Purely conceptual description | | "stormy seascape" | hybrid | Mood-based but may match titles | | "Madonna and child" | hybrid | Common in artwork titles | | "Rembrandt self portrait" | hybrid | Artist name + description | | "paintings from 1880s" | hybrid + filters | Use yearStart/yearEnd filters |

Tips

The collection is primarily European and American paintings from the Met Museum
AI-generated descriptions enable finding artworks by what's depicted, not just metadata
Use source_url to link users to the official Met Museum page
Pagination: when has_more is true, increase offset to get more results
For similar artworks: "precomputed" gives the best quality with explanations; "embedding" is faster
No authentication required - the API is publicly accessible

Agent Skills: Met Museum Paintings Semantic Search

Install this agent skill to your local

Skill Files