RealityKit Diagnostics
Systematic diagnosis for common RealityKit issues with time-cost annotations.
When to Use This Diagnostic Skill
Use this skill when:
- Entity added but not visible in the scene
- AR anchor not tracking or content floating
- Tap/drag gestures not responding on 3D entities
- Frame rate dropping or stuttering
- Material looks wrong (too dark, too bright, incorrect colors)
- Multiplayer entities not syncing across devices
- Physics bodies not colliding or passing through each other
For RealityKit architecture patterns and best practices, see axiom-realitykit. For API reference, see axiom-realitykit-ref.
Mandatory First Step: Enable Debug Visualization
Time cost: 10 seconds vs hours of blind debugging
// In your RealityView or ARView setup
#if DEBUG
// Xcode: Debug → Attach to Process → Show RealityKit Statistics
// Or enable in code:
arView.debugOptions = [
.showStatistics, // Entity count, draw calls, FPS
.showPhysics, // Collision shapes
.showAnchorOrigins, // Anchor positions
.showAnchorGeometry // Detected plane geometry
]
#endif
If you can't see collision shapes with .showPhysics, your CollisionComponent is missing or misconfigured. Fix collision before debugging gestures or physics.
Symptom 1: Entity Not Visible
Time saved: 30-60 min → 2-5 min
Entity added but nothing appears
│
├─ Is the entity added to the scene?
│ └─ NO → Add to RealityView content:
│ content.add(entity)
│ ✓ Entities must be in the scene graph to render
│
├─ Does the entity have a ModelComponent?
│ └─ NO → Add mesh and material:
│ entity.components[ModelComponent.self] = ModelComponent(
│ mesh: .generateBox(size: 0.1),
│ materials: [SimpleMaterial(color: .red, isMetallic: false)]
│ )
│ ✓ Bare Entity is invisible — it's just a container
│
├─ Is the entity's scale zero or nearly zero?
│ └─ CHECK → Print: entity.scale
│ USD models may import with unexpected scale.
│ Try: entity.scale = SIMD3(repeating: 0.01) for meter-scale models
│
├─ Is the entity behind the camera?
│ └─ CHECK → Print: entity.position(relativeTo: nil)
│ In RealityKit, -Z is forward (toward screen).
│ Try: entity.position = SIMD3(0, 0, -0.5) (half meter in front)
│
├─ Is the entity inside another object?
│ └─ CHECK → Move to a known visible position:
│ entity.position = SIMD3(0, 0, -1)
│
├─ Is the entity's isEnabled set to false?
│ └─ CHECK → entity.isEnabled = true
│ Also check parent: entity.isEnabledInHierarchy
│
├─ Is the entity on an untracked anchor?
│ └─ CHECK → Verify anchor is tracking:
│ entity.isAnchored (should be true)
│ If using plane anchor, ensure surface is detected first
│
└─ Is the material transparent or OcclusionMaterial?
└─ CHECK → Inspect material:
If using PhysicallyBasedMaterial, check baseColor is not black
If using blending = .transparent, check opacity > 0
Quick Diagnostic
func diagnoseVisibility(_ entity: Entity) {
print("Name: \(entity.name)")
print("Is enabled: \(entity.isEnabled)")
print("In hierarchy: \(entity.isEnabledInHierarchy)")
print("Is anchored: \(entity.isAnchored)")
print("Position (world): \(entity.position(relativeTo: nil))")
print("Scale: \(entity.scale)")
print("Has model: \(entity.components[ModelComponent.self] != nil)")
print("Children: \(entity.children.count)")
}
Symptom 2: Anchor Not Tracking
Time saved: 20-45 min → 3-5 min
AR content not appearing or floating
│
├─ Is the AR session running?
│ └─ For RealityView on iOS 18+, AR runs automatically
│ For ARView, check: arView.session.isRunning
│
├─ Is SpatialTrackingSession configured? (iOS 18+)
│ └─ CHECK → Ensure tracking modes requested:
│ let config = SpatialTrackingSession.Configuration(
│ tracking: [.plane, .object])
│ let result = await session.run(config)
│ if let notSupported = result {
│ // Handle unsupported modes
│ }
│
├─ Is the anchor type appropriate for the environment?
│ ├─ .plane(.horizontal) → Need a flat surface visible to camera
│ ├─ .plane(.vertical) → Need a wall visible to camera
│ ├─ .image → Image must be in "AR Resources" asset catalog
│ ├─ .face → Front camera required (not rear)
│ └─ .body → Full body must be visible
│
├─ Is minimumBounds too large?
│ └─ CHECK → Reduce minimum bounds:
│ AnchorEntity(.plane(.horizontal, classification: .any,
│ minimumBounds: SIMD2(0.1, 0.1))) // Smaller = detects sooner
│
├─ Is the device supported?
│ └─ CHECK → Plane detection requires A12+ chip
│ Face tracking requires TrueDepth camera
│ Body tracking requires A12+ chip
│
└─ Is the environment adequate?
└─ CHECK → AR needs:
- Adequate lighting (not too dark)
- Textured surfaces (not blank walls)
- Stable device position during initial detection
Symptom 3: Gesture Not Responding
Time saved: 15-30 min → 2-3 min
Tap/drag on entity does nothing
│
├─ Does the entity have a CollisionComponent?
│ └─ NO → Add collision shapes:
│ entity.generateCollisionShapes(recursive: true)
│ // or manual:
│ entity.components[CollisionComponent.self] = CollisionComponent(
│ shapes: [.generateBox(size: SIMD3(0.1, 0.1, 0.1))])
│ ✓ Collision shapes are REQUIRED for gesture hit testing
│
├─ [visionOS] Does the entity have InputTargetComponent?
│ └─ NO → Add it:
│ entity.components[InputTargetComponent.self] = InputTargetComponent()
│ ✓ Required on visionOS for gesture input
│
├─ Is the gesture attached to the RealityView?
│ └─ CHECK → Gesture must be on the view, not the entity:
│ RealityView { content in ... }
│ .gesture(TapGesture().targetedToAnyEntity().onEnded { ... })
│
├─ Is the collision shape large enough to hit?
│ └─ CHECK → Enable .showPhysics to see shapes
│ Shapes too small = hard to tap.
│ Try: .generateBox(size: SIMD3(repeating: 0.1)) minimum
│
├─ Is the entity behind another entity?
│ └─ CHECK → Front entities may block gestures on back entities
│ Ensure collision is on the intended target
│
└─ Is the entity enabled?
└─ CHECK → entity.isEnabled must be true
Disabled entities don't receive input
Quick Diagnostic
func diagnoseGesture(_ entity: Entity) {
print("Has collision: \(entity.components[CollisionComponent.self] != nil)")
print("Has input target: \(entity.components[InputTargetComponent.self] != nil)")
print("Is enabled: \(entity.isEnabled)")
print("Is anchored: \(entity.isAnchored)")
if let collision = entity.components[CollisionComponent.self] {
print("Collision shapes: \(collision.shapes.count)")
}
}
Symptom 4: Performance Problems
Time saved: 1-3 hours → 10-20 min
Frame rate dropping or stuttering
│
├─ How many entities are in the scene?
│ └─ CHECK → Print entity count:
│ var count = 0
│ func countEntities(_ entity: Entity) {
│ count += 1
│ for child in entity.children { countEntities(child) }
│ }
│ Under 100: unlikely to be entity count
│ 100-500: review for optimization
│ 500+: definitely needs optimization
│
├─ Are mesh/material resources shared?
│ └─ NO → Share resources across identical entities:
│ let sharedMesh = MeshResource.generateBox(size: 0.05)
│ let sharedMaterial = SimpleMaterial(color: .white, isMetallic: false)
│ // Reuse for all instances
│ ✓ RealityKit batches entities with identical resources
│
├─ Is a System creating components every frame?
│ └─ CHECK → Look for allocations in update():
│ Creating ModelComponent, CollisionComponent, or materials
│ every frame causes GC pressure.
│ Cache resources, only update when values change.
│
├─ Are collision shapes mesh-based?
│ └─ CHECK → Replace generateCollisionShapes(recursive: true)
│ with simple shapes (box, sphere, capsule) for dynamic entities
│
├─ Is generateCollisionShapes called repeatedly?
│ └─ CHECK → Call once during setup, not every frame
│
├─ Are there too many physics bodies?
│ └─ CHECK → Dynamic bodies are most expensive.
│ Convert distant/static objects to .static mode.
│ Remove physics from non-interactive entities.
│
└─ Is the model polygon count too high?
└─ CHECK → Decimate models for real-time use.
Target: <100K triangles total for mobile AR.
Use LOD (Level of Detail) for distant objects.
Symptom 5: Material Looks Wrong
Time saved: 15-45 min → 5-10 min
Colors, lighting, or textures look incorrect
│
├─ Is the scene too dark?
│ └─ CHECK → Missing environment lighting:
│ Add DirectionalLightComponent or EnvironmentResource
│ In AR, RealityKit uses real-world lighting automatically
│ In non-AR, you must provide lighting explicitly
│
├─ Is the baseColor set?
│ └─ CHECK → PhysicallyBasedMaterial defaults to white
│ material.baseColor = .init(tint: .red)
│ If using a texture, verify it loaded:
│ try TextureResource(named: "albedo")
│
├─ Is metallic set incorrectly?
│ └─ CHECK → metallic = 1.0 makes surfaces mirror-like
│ Most real objects: metallic = 0.0
│ Only metals (gold, silver, chrome): metallic = 1.0
│
├─ Is the texture semantic wrong?
│ └─ CHECK → Use correct semantic:
│ .color for albedo/baseColor textures
│ .raw for data textures (metallic, roughness)
│ .normal for normal maps
│ .hdrColor for HDR textures
│
├─ Is the model upside down or inside out?
│ └─ CHECK → Try:
│ material.faceCulling = .none (shows both sides)
│ If that fixes it, the model normals are flipped
│
└─ Is blending/transparency unexpected?
└─ CHECK → material.blending
Default is .opaque
For transparency: .transparent(opacity: ...)
Symptom 6: Physics Not Working
Time saved: 20-40 min → 5-10 min
Objects pass through each other or don't collide
│
├─ Do both entities have CollisionComponent?
│ └─ NO → Both sides of a collision need CollisionComponent
│
├─ Does the moving entity have PhysicsBodyComponent?
│ └─ NO → Add physics body:
│ entity.components[PhysicsBodyComponent.self] = PhysicsBodyComponent(
│ mode: .dynamic)
│
├─ Are collision groups/filters configured correctly?
│ └─ CHECK → Entities must be in compatible groups:
│ Default: group = .default, mask = .all
│ If using custom groups, verify mask includes the other group
│
├─ Is the physics mode correct?
│ ├─ Two .static bodies → Never collide (both immovable)
│ ├─ .dynamic + .static → Correct (common setup)
│ ├─ .dynamic + .dynamic → Both move on collision
│ └─ .kinematic + .dynamic → Kinematic pushes dynamic
│
├─ Is the collision shape appropriate?
│ └─ CHECK → .showPhysics debug option
│ Shape may be too small, offset, or wrong type
│
└─ Are entities on different anchors?
└─ CHECK → "Physics bodies and colliders affect only
entities that share the same anchor" (Apple docs)
Move entities under the same anchor for physics interaction
Symptom 7: Multiplayer Sync Issues
Time saved: 30-60 min → 10-15 min
Entities not appearing on other devices
│
├─ Does the entity have SynchronizationComponent?
│ └─ NO → Add it:
│ entity.components[SynchronizationComponent.self] =
│ SynchronizationComponent()
│
├─ Is the MultipeerConnectivityService set up?
│ └─ CHECK → Verify MCSession is connected before syncing
│
├─ Are custom components Codable?
│ └─ NO → Non-Codable components don't sync
│ struct MyComponent: Component, Codable { ... }
│
├─ Does the entity have an owner?
│ └─ CHECK → Only the owner can modify synced properties
│ Request ownership before modifying:
│ entity.requestOwnership { result in ... }
│
└─ Is the entity anchored?
└─ CHECK → Unanchored entities may not sync position correctly
Use a shared world anchor for reliable positioning
Common Mistakes
| Mistake | Time Cost | Fix |
|---------|-----------|-----|
| No CollisionComponent on interactive entity | 15-30 min | entity.generateCollisionShapes(recursive: true) |
| Missing InputTargetComponent on visionOS | 10-20 min | Add InputTargetComponent() |
| Gesture on wrong view (not RealityView) | 10-15 min | Attach .gesture() to RealityView |
| Entity scale wrong for USD model | 15-30 min | Check units: meters vs centimeters |
| No lighting in non-AR scene | 10-20 min | Add DirectionalLightComponent |
| Storing entity refs in System | 30-60 min crash debugging | Query with EntityQuery each frame |
| Components not registered | 10-15 min | Call registerComponent() in app init |
| Systems not registered | 10-15 min | Call registerSystem() before scene load |
| Physics across different anchors | 20-40 min | Put interacting entities under same anchor |
| Calling generateCollisionShapes every frame | Performance degradation | Call once during setup |
Diagnostic Quick Reference
| Symptom | First Check | Time Saved | |---------|-------------|------------| | Not visible | Has ModelComponent? Scale > 0? | 30-60 min | | No gesture response | Has CollisionComponent? | 15-30 min | | Not tracking | Anchor type matches environment? | 20-45 min | | Frame drops | Entity count? Resource sharing? | 1-3 hours | | Wrong colors | Has lighting? Metallic value? | 15-45 min | | No collision | Both have CollisionComponent? Same anchor? | 20-40 min | | No sync | SynchronizationComponent? Codable? | 30-60 min | | Sim OK, device crash | Metal features? Texture format? | 15-30 min |
Symptom 8: Works in Simulator, Crashes on Device
Time cost: 15-30 min (often misdiagnosed as model issue)
Q1: Is the crash a Metal error (MTLCommandBuffer, shader compilation)?
├─ YES → Simulator uses software rendering, device uses real GPU
│ Common causes:
│ - Custom Metal shaders with unsupported features
│ - Texture formats not supported on device GPU
│ - Exceeding device texture size limits (max 8192x8192 on older)
│ Fix: Check device GPU family, use supported formats
│
└─ NO → Check next
Q2: Is it an out-of-memory crash?
├─ YES → Simulator has more RAM available
│ Common: Large USDZ files with uncompressed textures
│ Fix: Compress textures, reduce polygon count, use LOD
│ Check: USDZ file size (keep < 50MB for reliable loading)
│
└─ NO → Check next
Q3: Is it an AR-related crash (camera, tracking)?
├─ YES → Simulator has no real camera/sensors
│ Fix: Test AR features on device only, use simulator for UI/layout
│
└─ NO → Check device capabilities
- A12+ required for RealityKit
- LiDAR for scene reconstruction
- TrueDepth for face tracking
Resources
WWDC: 2019-603, 2019-605, 2023-10080, 2024-10103
Docs: /realitykit, /realitykit/entity, /realitykit/collisioncomponent, /realitykit/physicsbodycomponent
Skills: axiom-realitykit, axiom-realitykit-ref