specter-acset Skill | Agent Skills

specter-acset

Inline-cached bidirectional navigation for Julia data structures

bmorphism Contributions

"all is bidirectional" — @bmorphism, Plurigrid Play/Coplay gist

"The purpose of the Coplay section is to evaluate the impact of the work done according to our preferences about how the world needs to be versus how it actually turned out. Focus on a bidirectional view of feedback." — all is bidirectional

Version: 1.0.0 Trit: 0 (Ergodic - coordinates navigation)

From Clojure Specter to Julia

Nathan Marz's Specter library for Clojure provides bidirectional data navigation where the same path expression works for both selection AND transformation. This skill ports those patterns to Julia with extensions for S-expressions and ACSets.

Key Insights from Specter Talks

"Rama on Clojure's Terms" (2024)

"comp-navs is fast because it's just object allocation + field sets"

Specter's performance comes from:

Inline caching: Paths compiled once, reused at callsite
Continuation-passing style: Chains of next_fn calls
Navigator protocol: Uniform interface for all data types

"Specter: Powerful and Simple Data Structure Manipulation"

"Without Specter, you need different code for selection vs transformation"

The bidirectionality principle: A path is a lens that focuses on parts of a structure.

Navigator Protocol

abstract type Navigator end

# Core operations - bidirectional by design
function nav_select(nav::Navigator, structure, next_fn)
    # Traverse and collect
end

function nav_transform(nav::Navigator, structure, next_fn)
    # Traverse and modify
end

Primitive Navigators

| Navigator | Select Behavior | Transform Behavior | |-----------|-----------------|-------------------| | ALL | Each element | Map over all | | FIRST | First element | Update first only | | LAST | Last element | Update last only | | keypath(k) | Value at key | Update value at key | | pred(f) | Stay if f(x) true | Transform if f(x) true |

Composition: comp_navs (The Key to Performance)

Nathan Marz's critical insight: composition is just allocation + field sets.

Why This Matters

Traditional approaches compile/interpret paths at composition time. Specter does zero work at composition - it just creates an object:

# Specter's key to performance: ONLY allocation + field sets
struct ComposedNav <: Navigator
    navs::Vector{Navigator}  # Just a field - no processing
end

# comp_navs does ONE thing: allocate and set field
comp_navs(navs::Navigator...) = ComposedNav(collect(navs))
# That's it. No compilation. No interpretation. No optimization.
# Just: new ComposedNav() + set navs field

The Magic: Work Happens at Traversal

All the actual work happens when you call select or transform:

# Chain of continuations - CPS (continuation-passing style)
function nav_select(cn::ComposedNav, structure, next_fn)
    function chain_select(navs, struct_val)
        if isempty(navs)
            next_fn(struct_val)  # Base case: call continuation
        else
            # Recursive case: process first nav, chain the rest
            nav_select(first(navs), struct_val, 
                      s -> chain_select(navs[2:end], s))
        end
    end
    chain_select(cn.navs, structure)
end

Why CPS + Lazy Composition = Fast

Traditional:
  compose(a, b, c) → [compile a+b+c] → CompiledPath
  
Specter:
  comp_navs(a, b, c) → ComposedNav{[a, b, c]}  # Just store refs
  select(path, data) → [chain continuations] → results

Benefits:

O(1) composition - just allocate, no work
Inline caching - same ComposedNav reused at callsite
Late binding - dynamic navs resolved at traversal time
No intermediate allocations - CPS avoids building result lists

Inline Caching Pattern

# At each callsite, the path is compiled ONCE and cached:
@compiled_select([ALL, pred(iseven)], data)

# Expands to something like:
let cached_nav = nothing
    if cached_nav === nothing
        cached_nav = comp_navs(ALL, pred(iseven))  # First call only
    end
    nav_select(cached_nav, data, identity)  # Reuse forever
end

This is why Specter achieves near-hand-written performance despite the abstraction.

S-expression Navigators

Unique to Julia - navigate typed AST nodes:

# Type definitions
abstract type Sexp end
struct Atom <: Sexp
    value::String
end
struct SList <: Sexp
    children::Vector{Sexp}
end

# Navigators
SEXP_HEAD      # → first(children)
SEXP_TAIL      # → children[2:end]
SEXP_CHILDREN  # → children vector
SEXP_WALK      # Recursive prewalk
sexp_nth(n)    # → children[n]
ATOM_VALUE     # → atom.value

Example: AST Transformation

sexp = parse_sexp("(define (square x) (* x x))")

# Rename function
renamed = transform(
    [sexp_nth(2), sexp_nth(1), ATOM_VALUE],
    _ -> "cube",
    sexp
)
# → (define (cube x) (* x x))

ACSet Navigators

Navigate category-theoretic databases:

# Navigate morphism values
acset_field(:E, :src)

# Filter parts by predicate
acset_where(:E, :src, ==(1))

# All parts of an object
acset_parts(:V)

Example: Graph Transformation

g = @acset Graph begin V=4; E=3; src=[1,2,3]; tgt=[2,3,4] end

# Select: get all source vertices
select([acset_field(:E, :src)], g)  # → [1, 2, 3]

# Transform: shift targets
g2 = transform([acset_field(:E, :tgt)], t -> mod1(t+1, 4), g)

Dynamic Navigators

selected(subpath)

Stay at current position if subpath matches:

# Select values > 5
select([ALL, selected(pred(x -> x > 5))], [1,2,3,4,5,6,7,8,9,10])
# → [6, 7, 8, 9, 10]

if_path(cond, then, else)

Conditional navigation:

if_path(pred(iseven),
        keypath(:even_branch),
        keypath(:odd_branch))

Coercion (Like Specter's coerce-nav)

coerce_nav(x::Navigator) = x
coerce_nav(s::Symbol) = keypath(s)
coerce_nav(f::Function) = pred(f)
coerce_nav(v::Vector) = comp_navs(coerce_nav.(v)...)

API

# High-level interface
select(path, data)                    # Collect matches
select_one(path, data)                # Single match or nothing
transform(path, fn, data)             # Transform matches
setval(path, value, data)             # Set matches to value

Comparison: Clojure vs Julia

| Clojure (Specter) | Julia (SpecterACSet) | Notes | |-------------------|---------------------|-------| | (select [ALL even?] data) | select([ALL, pred(iseven)], data) | Same pattern | | (transform [ALL even?] f data) | transform([ALL, pred(iseven)], f, data) | Bidirectional | | Keywords implicit | keypath(:k) explicit | Type safety | | No ACSet support | acset_field, acset_where | Category theory | | No typed sexp | Atom/SList discrimination | AST navigation |

GF(3) Triads

three-match (-1) ⊗ specter-acset (0) ⊗ gay-mcp (+1) = 0 ✓
lispsyntax-acset (-1) ⊗ specter-acset (0) ⊗ cider-clojure (+1) = 0 ✓

Files

Implementation: lib/specter_acset.jl
Babashka comparison: lib/specter_comparison.bb

Julia Scientific Package Integration

From julia-scientific skill - related Julia packages:

| Package | Category | Specter Integration | |---------|----------|---------------------| | Catlab.jl | ACSets | Primary navigation target | | DataFrames.jl | Data | Tabular navigation | | Graphs.jl | Networks | Graph traversal | | BioSequences.jl | Bioinformatics | Sequence navigation | | MolecularGraph.jl | Chemistry | Molecular graph traversal | | StructuredDecompositions.jl | Sheaves | Decomposition navigation | | AlgebraicRewriting.jl | Rewriting | Rule application paths |

Cross-Domain Navigation Patterns

# Navigate DataFrame (polars → DataFrames.jl)
using DataFrames
df = DataFrame(a=[1,2,3], b=[4,5,6])
select([keypath(:a), ALL], df)  # All values in column :a

# Navigate molecular graph (rdkit → MolecularGraph.jl)
using MolecularGraph
mol = smilestomol("CCO")
select([atoms, pred(a -> a.symbol == :O)], mol)

# Navigate protein structure (biopython → BioStructures.jl)
using BioStructures
pdb = read("1CRN.pdb", PDB)
select([chains, residues, pred(is_hydrophobic)], pdb)

# Navigate genomic features (pysam → XAM.jl)
using XAM
bam = BAM.Reader("aligned.bam")
select([records, pred(r -> r.mapq > 30)], bam)

References

Specter GitHub
Nathan Marz: "Rama on Clojure's Terms" (2024)
Nathan Marz: "Specter: Powerful and Simple Data Structure Manipulation"
Lens laws (Haskell perspective)

Scientific Skill Interleaving

This skill connects to the K-Dense-AI/claude-scientific-skills ecosystem:

Annotated Data

anndata [○] via bicomodule
- Hub for annotated matrices

Bibliography References

general: 734 citations in bib.duckdb

Cat# Integration

This skill maps to Cat# = Comod(P) as a bicomodule in the equipment structure:

Trit: 0 (ERGODIC)
Home: Prof
Poly Op: ⊗
Kan Role: Adj
Color: #26D826

GF(3) Naturality

The skill participates in triads satisfying:

(-1) + (0) + (+1) ≡ 0 (mod 3)

This ensures compositional coherence in the Cat# equipment structure.

Agent Skills: specter-acset

Install this agent skill to your local

Skill Files