Agent Skills: RealityKit API Reference

RealityKit API Reference

Complete API reference for RealityKit organized by category.

When to Use This Reference

Use this reference when:

• Looking up specific RealityKit API signatures or properties
• Checking which component types are available
• Finding the right anchor type for an AR experience
• Browsing material properties and options
• Setting up physics body parameters
• Looking up animation or audio API details
• Checking platform availability for specific APIs

Part 1: Entity API

Entity

// Creation
let entity = Entity()
let entity = Entity(components: [TransformComponent(), ModelComponent(...)])

// Async loading
let entity = try await Entity(named: "scene", in: .main)
let entity = try await Entity(contentsOf: url)

// Clone
let clone = entity.clone(recursive: true)

Entity Properties

| Property | Type | Description | |----------|------|-------------| | name | String | Identifier for lookup | | id | ObjectIdentifier | Unique identity | | isEnabled | Bool | Local enabled state | | isEnabledInHierarchy | Bool | Effective enabled (considers parents) | | isActive | Bool | Entity is in an active scene | | isAnchored | Bool | Has anchoring or anchored ancestor | | scene | RealityKit.Scene? | Owning scene | | parent | Entity? | Parent entity | | children | Entity.ChildCollection | Child entities | | components | Entity.ComponentSet | All attached components | | anchor | HasAnchoring? | Nearest anchoring ancestor |

Entity Hierarchy Methods

entity.addChild(child)
entity.addChild(child, preservingWorldTransform: true)
entity.removeChild(child)
entity.removeFromParent()
entity.findEntity(named: "name")  // Recursive search

Entity Subclasses

| Class | Purpose | Key Component | |-------|---------|---------------| | Entity | Base container | Transform only | | ModelEntity | Renderable object | ModelComponent | | AnchorEntity | AR anchor point | AnchoringComponent | | PerspectiveCamera | Virtual camera | PerspectiveCameraComponent | | DirectionalLight | Sun/directional | DirectionalLightComponent | | PointLight | Point light | PointLightComponent | | SpotLight | Spot light | SpotLightComponent | | TriggerVolume | Invisible collision zone | CollisionComponent | | ViewAttachmentEntity | SwiftUI view in 3D | visionOS | | BodyTrackedEntity | Body-tracked entity | BodyTrackingComponent |

Part 2: Component Catalog

Transform

// Properties
entity.position                    // SIMD3<Float>, local
entity.orientation                 // simd_quatf
entity.scale                      // SIMD3<Float>
entity.transform                  // Transform struct

// World-space
entity.position(relativeTo: nil)
entity.orientation(relativeTo: nil)
entity.setPosition(pos, relativeTo: nil)

// Utilities
entity.look(at: target, from: position, relativeTo: nil)

ModelComponent

let component = ModelComponent(
    mesh: MeshResource.generateBox(size: 0.1),
    materials: [SimpleMaterial(color: .red, isMetallic: true)]
)
entity.components[ModelComponent.self] = component

MeshResource Built-in Generators

| Method | Parameters | |--------|-----------| | .generateBox(size:) | SIMD3<Float> or single Float | | .generateBox(size:cornerRadius:) | Rounded box | | .generateSphere(radius:) | Float | | .generatePlane(width:depth:) | Float, Float | | .generatePlane(width:height:) | Vertical plane | | .generateCylinder(height:radius:) | Float, Float | | .generateCone(height:radius:) | Float, Float | | .generateText(_:) | String, with options |

CollisionComponent

let component = CollisionComponent(
    shapes: [
        .generateBox(size: SIMD3(0.1, 0.2, 0.1)),
        .generateSphere(radius: 0.05),
        .generateCapsule(height: 0.3, radius: 0.05),
        .generateConvex(from: meshResource)
    ],
    mode: .default,                    // .default or .trigger
    filter: CollisionFilter(
        group: CollisionGroup(rawValue: 1),
        mask: .all
    )
)

ShapeResource Types

| Method | Description | Performance | |--------|-------------|-------------| | .generateBox(size:) | Axis-aligned box | Fastest | | .generateSphere(radius:) | Sphere | Fast | | .generateCapsule(height:radius:) | Capsule | Fast | | .generateConvex(from:) | Convex hull from mesh | Moderate | | .generateStaticMesh(from:) | Exact mesh | Slowest (static only) |

PhysicsBodyComponent

let component = PhysicsBodyComponent(
    massProperties: .init(
        mass: 1.0,
        inertia: SIMD3(repeating: 0.1),
        centerOfMass: .zero
    ),
    material: .generate(
        staticFriction: 0.5,
        dynamicFriction: 0.3,
        restitution: 0.4
    ),
    mode: .dynamic                     // .dynamic, .static, .kinematic
)

| Mode | Behavior | |------|----------| | .dynamic | Physics simulation controls position | | .static | Immovable, participates in collisions | | .kinematic | Code-controlled, affects dynamic bodies |

PhysicsMotionComponent

var motion = PhysicsMotionComponent()
motion.linearVelocity = SIMD3(0, 5, 0)
motion.angularVelocity = SIMD3(0, .pi, 0)
entity.components[PhysicsMotionComponent.self] = motion

CharacterControllerComponent

entity.components[CharacterControllerComponent.self] = CharacterControllerComponent(
    radius: 0.3,
    height: 1.8,
    slopeLimit: .pi / 4,
    stepLimit: 0.3
)

// Move character with gravity
entity.moveCharacter(
    by: SIMD3(0.1, -0.01, 0),
    deltaTime: Float(context.deltaTime),
    relativeTo: nil
)

AnchoringComponent

// Plane detection
AnchoringComponent(.plane(.horizontal, classification: .table,
                           minimumBounds: SIMD2(0.2, 0.2)))
AnchoringComponent(.plane(.vertical, classification: .wall,
                           minimumBounds: SIMD2(0.5, 0.5)))

// World position
AnchoringComponent(.world(transform: float4x4(...)))

// Image anchor
AnchoringComponent(.image(group: "AR Resources", name: "poster"))

// Face tracking
AnchoringComponent(.face)

// Body tracking
AnchoringComponent(.body)

Plane Classification

| Classification | Description | |----------------|-------------| | .table | Horizontal table surface | | .floor | Floor surface | | .ceiling | Ceiling surface | | .wall | Vertical wall | | .door | Door | | .window | Window | | .seat | Chair/couch |

Light Components

// Directional
let light = DirectionalLightComponent(
    color: .white,
    intensity: 1000,
    isRealWorldProxy: false
)
light.shadow = DirectionalLightComponent.Shadow(
    maximumDistance: 10,
    depthBias: 0.01
)

// Point
PointLightComponent(
    color: .white,
    intensity: 1000,
    attenuationRadius: 5
)

// Spot
SpotLightComponent(
    color: .white,
    intensity: 1000,
    innerAngleInDegrees: 30,
    outerAngleInDegrees: 60,
    attenuationRadius: 10
)

Accessibility

var accessibility = AccessibilityComponent()
accessibility.label = "Red cube"
accessibility.value = "Interactive 3D object"
accessibility.traits = .button
accessibility.isAccessibilityElement = true
entity.components[AccessibilityComponent.self] = accessibility

Additional Components

| Component | Purpose | Platform | |-----------|---------|----------| | OpacityComponent | Fade entity in/out | All | | GroundingShadowComponent | Contact shadow beneath entity | All | | InputTargetComponent | Enable gesture input | visionOS | | HoverEffectComponent | Highlight on gaze/hover | visionOS | | SynchronizationComponent | Multiplayer entity sync | All | | ImageBasedLightComponent | Custom environment lighting | All | | ImageBasedLightReceiverComponent | Receive IBL from source | All |

Part 3: System API

System Protocol

protocol System {
    init(scene: RealityKit.Scene)
    func update(context: SceneUpdateContext)
}

SceneUpdateContext

| Property | Type | Description | |----------|------|-------------| | deltaTime | TimeInterval | Time since last update | | scene | RealityKit.Scene | The scene |

// Query entities
context.entities(matching: query, updatingSystemWhen: .rendering)

EntityQuery

// Has specific component
EntityQuery(where: .has(HealthComponent.self))

// Has multiple components
EntityQuery(where: .has(HealthComponent.self) && .has(ModelComponent.self))

// Does not have component
EntityQuery(where: .has(EnemyComponent.self) && !.has(DeadComponent.self))

Scene Events

| Event | Trigger | |-------|---------| | SceneEvents.Update | Every frame | | SceneEvents.DidAddEntity | Entity added to scene | | SceneEvents.DidRemoveEntity | Entity removed from scene | | SceneEvents.AnchoredStateChanged | Anchor tracking changes | | CollisionEvents.Began | Two entities start colliding | | CollisionEvents.Updated | Collision continues | | CollisionEvents.Ended | Collision ends | | AnimationEvents.PlaybackCompleted | Animation finishes |

scene.subscribe(to: CollisionEvents.Began.self, on: entity) { event in
    // event.entityA, event.entityB, event.impulse
}

Part 4: RealityView API

Initializers

// Basic (iOS 18+, visionOS 1.0+)
RealityView { content in
    // make: Add entities to content
}

// With update
RealityView { content in
    // make
} update: { content in
    // update: Called when SwiftUI state changes
}

// With placeholder
RealityView { content in
    // make (async loading)
} placeholder: {
    ProgressView()
}

// With attachments (visionOS)
RealityView { content, attachments in
    // make
} update: { content, attachments in
    // update
} attachments: {
    Attachment(id: "label") { Text("Hello") }
}

RealityViewContent

content.add(entity)
content.remove(entity)
content.entities          // EntityCollection

// iOS/macOS — camera content
content.camera            // RealityViewCameraContent (non-visionOS)

Gestures on RealityView

RealityView { content in ... }
    .gesture(TapGesture().targetedToAnyEntity().onEnded { value in
        let entity = value.entity
    })
    .gesture(DragGesture().targetedToAnyEntity().onChanged { value in
        value.entity.position = value.convert(value.location3D,
            from: .local, to: .scene)
    })
    .gesture(RotateGesture().targetedToAnyEntity().onChanged { value in
        // Handle rotation
    })
    .gesture(MagnifyGesture().targetedToAnyEntity().onChanged { value in
        // Handle scale
    })

Part 5: Model3D API

// Simple display
Model3D(named: "robot")

// With phases
Model3D(named: "robot") { phase in
    switch phase {
    case .empty:
        ProgressView()
    case .success(let model):
        model.resizable().scaledToFit()
    case .failure(let error):
        Text("Failed: \(error.localizedDescription)")
    @unknown default:
        EmptyView()
    }
}

// From URL
Model3D(url: modelURL)

Part 6: Material System

SimpleMaterial

var material = SimpleMaterial()
material.color = .init(tint: .blue)
material.metallic = .init(floatLiteral: 1.0)
material.roughness = .init(floatLiteral: 0.3)

PhysicallyBasedMaterial

var material = PhysicallyBasedMaterial()
material.baseColor = .init(tint: .white,
    texture: .init(try .load(named: "albedo")))
material.metallic = .init(floatLiteral: 0.0)
material.roughness = .init(floatLiteral: 0.5)
material.normal = .init(texture: .init(try .load(named: "normal")))
material.ambientOcclusion = .init(texture: .init(try .load(named: "ao")))
material.emissiveColor = .init(color: .blue)
material.emissiveIntensity = 2.0
material.clearcoat = .init(floatLiteral: 0.8)
material.clearcoatRoughness = .init(floatLiteral: 0.1)
material.specular = .init(floatLiteral: 0.5)
material.sheen = .init(color: .white)
material.anisotropyLevel = .init(floatLiteral: 0.5)
material.blending = .transparent(opacity: .init(floatLiteral: 0.5))
material.faceCulling = .back            // .none, .front, .back

UnlitMaterial

var material = UnlitMaterial()
material.color = .init(tint: .red,
    texture: .init(try .load(named: "texture")))
material.blending = .transparent(opacity: .init(floatLiteral: 0.8))

Special Materials

// Occlusion — invisible but hides content behind it
let occlusionMaterial = OcclusionMaterial()

// Video
let videoMaterial = VideoMaterial(avPlayer: avPlayer)

TextureResource Loading

// From bundle
let texture = try await TextureResource(named: "texture")

// From URL
let texture = try await TextureResource(contentsOf: url)

// With options
let texture = try await TextureResource(named: "texture",
    options: .init(semantic: .color))  // .color, .raw, .normal, .hdrColor

Part 7: Animation

Transform Animation

entity.move(
    to: Transform(
        scale: .one,
        rotation: targetRotation,
        translation: targetPosition
    ),
    relativeTo: entity.parent,
    duration: 1.5,
    timingFunction: .easeInOut
)

Timing Functions

| Function | Curve | |----------|-------| | .default | System default | | .linear | Constant speed | | .easeIn | Slow start | | .easeOut | Slow end | | .easeInOut | Slow start and end |

Playing Loaded Animations

// All animations from USD
for animation in entity.availableAnimations {
    let controller = entity.playAnimation(animation)
}

// With options
let controller = entity.playAnimation(
    animation.repeat(count: 3),
    transitionDuration: 0.3,
    startsPaused: false
)

AnimationPlaybackController

let controller = entity.playAnimation(animation)
controller.pause()
controller.resume()
controller.stop()
controller.speed = 0.5            // Half speed
controller.blendFactor = 1.0      // Full blend
controller.isComplete             // Check completion

Part 8: Audio

AudioFileResource

// Load
let resource = try AudioFileResource.load(
    named: "sound.wav",
    configuration: .init(
        shouldLoop: true,
        shouldRandomizeStartTime: false,
        mixGroupName: "effects"
    )
)

Audio Components

// Spatial (3D positional)
entity.components[SpatialAudioComponent.self] = SpatialAudioComponent(
    directivity: .beam(focus: 0.5),
    distanceAttenuation: .rolloff(factor: 1.0),
    gain: 0                          // dB
)

// Ambient (non-positional, uniform)
entity.components[AmbientAudioComponent.self] = AmbientAudioComponent(
    gain: -6
)

// Channel (multi-channel output)
entity.components[ChannelAudioComponent.self] = ChannelAudioComponent(
    gain: 0
)

Playback

let controller = entity.playAudio(resource)
controller.pause()
controller.stop()
controller.gain = -3               // Adjust volume (dB)
controller.speed = 1.5             // Pitch shift

entity.stopAllAudio()

Part 9: RealityRenderer (Metal Integration)

// Low-level Metal rendering of RealityKit content
let renderer = try RealityRenderer()
renderer.entities.append(entity)

// Render to Metal texture
let descriptor = RealityRenderer.CameraOutput.Descriptor(
    colorFormat: .bgra8Unorm,
    depthFormat: .depth32Float
)
try renderer.render(
    viewMatrix: viewMatrix,
    projectionMatrix: projectionMatrix,
    size: size,
    colorTexture: colorTexture,
    depthTexture: depthTexture
)

Resources

WWDC: 2019-603, 2019-605, 2021-10074, 2022-10074, 2023-10080, 2024-10103, 2024-10153

Docs: /realitykit, /realitykit/entity, /realitykit/component, /realitykit/system, /realitykit/realityview, /realitykit/model3d, /realitykit/modelentity, /realitykit/anchorentity, /realitykit/physicallybasedmaterial

Skills: axiom-realitykit, axiom-realitykit-diag, axiom-scenekit-ref