Agent Skills: WebGL Expert

WebGL Expert

Expert guide for WebGL (Web Graphics Library) API development, covering both WebGL 1.0 and WebGL 2.0 for high-performance 2D and 3D graphics rendering in web browsers.

Overview

WebGL is a JavaScript API that enables hardware-accelerated 3D graphics rendering within HTML canvas elements without requiring plugins. It closely conforms to OpenGL ES 2.0 (WebGL 1.0) and OpenGL ES 3.0 (WebGL 2.0) standards.

Key capabilities:

• Hardware-accelerated 2D and 3D rendering
• Programmable shader pipeline (GLSL)
• Texture mapping and advanced materials
• Lighting and transformation systems
• High-performance graphics for games and visualizations
• Cross-platform compatibility (all modern browsers)

Core Interfaces

WebGLRenderingContext (WebGL 1.0)

The foundational interface for WebGL operations, obtained via canvas context:

const canvas = document.querySelector('canvas');
const gl = canvas.getContext('webgl') || canvas.getContext('experimental-webgl');

if (!gl) {
    console.error('WebGL not supported');
}

WebGL2RenderingContext (WebGL 2.0)

Enhanced interface with advanced features:

const gl = canvas.getContext('webgl2');

if (!gl) {
    console.log('WebGL 2 not supported, falling back to WebGL 1');
    gl = canvas.getContext('webgl');
}

WebGL 2 exclusive features:

• 3D textures
• Sampler objects
• Uniform Buffer Objects (UBO)
• Transform Feedback
• Vertex Array Objects (VAO) - core feature
• Instanced rendering
• Multiple render targets
• Integer textures and attributes
• Query objects
• Occlusion queries

Rendering Pipeline

1. Shader Creation and Compilation

Shaders are programs written in GLSL (OpenGL Shading Language) that run on the GPU:

Vertex Shader - Processes each vertex:

attribute vec3 aPosition;
attribute vec2 aTexCoord;
uniform mat4 uModelViewProjection;
varying vec2 vTexCoord;

void main() {
    gl_Position = uModelViewProjection * vec4(aPosition, 1.0);
    vTexCoord = aTexCoord;
}

Fragment Shader - Determines pixel colors:

precision mediump float;
varying vec2 vTexCoord;
uniform sampler2D uTexture;

void main() {
    gl_FragColor = texture2D(uTexture, vTexCoord);
}

JavaScript shader setup:

function createShader(gl, type, source) {
    const shader = gl.createShader(type);
    gl.shaderSource(shader, source);
    gl.compileShader(shader);

    if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
        console.error('Shader compilation error:', gl.getShaderInfoLog(shader));
        gl.deleteShader(shader);
        return null;
    }
    return shader;
}

function createProgram(gl, vertexShader, fragmentShader) {
    const program = gl.createProgram();
    gl.attachShader(program, vertexShader);
    gl.attachShader(program, fragmentShader);
    gl.linkProgram(program);

    if (!gl.getProgramParameter(program, gl.LINK_STATUS)) {
        console.error('Program linking error:', gl.getProgramInfoLog(program));
        gl.deleteProgram(program);
        return null;
    }
    return program;
}

2. Buffer Management

Buffers store vertex data (positions, colors, normals, texture coordinates):

// Create buffer
const positionBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);

// Upload data
const positions = new Float32Array([
    -1.0, -1.0, 0.0,
     1.0, -1.0, 0.0,
     0.0,  1.0, 0.0
]);
gl.bufferData(gl.ARRAY_BUFFER, positions, gl.STATIC_DRAW);

// Set up attribute pointer
const positionLocation = gl.getAttribLocation(program, 'aPosition');
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 3, gl.FLOAT, false, 0, 0);

Buffer usage patterns:

• gl.STATIC_DRAW - Data doesn't change
• gl.DYNAMIC_DRAW - Data changes occasionally
• gl.STREAM_DRAW - Data changes every frame

3. Texture Handling

function loadTexture(gl, url) {
    const texture = gl.createTexture();
    gl.bindTexture(gl.TEXTURE_2D, texture);

    // Placeholder until image loads
    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 1, 1, 0, gl.RGBA, gl.UNSIGNED_BYTE,
                  new Uint8Array([255, 0, 255, 255]));

    const image = new Image();
    image.onload = () => {
        gl.bindTexture(gl.TEXTURE_2D, texture);
        gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image);

        // Generate mipmaps if power of 2
        if (isPowerOf2(image.width) && isPowerOf2(image.height)) {
            gl.generateMipmap(gl.TEXTURE_2D);
        } else {
            gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
            gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
            gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
        }
    };
    image.src = url;
    return texture;
}

function isPowerOf2(value) {
    return (value & (value - 1)) === 0;
}

4. Rendering Loop

function render(gl, program) {
    // Clear canvas
    gl.clearColor(0.0, 0.0, 0.0, 1.0);
    gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

    // Enable depth testing
    gl.enable(gl.DEPTH_TEST);
    gl.depthFunc(gl.LEQUAL);

    // Use program
    gl.useProgram(program);

    // Set uniforms
    const projectionMatrix = mat4.create();
    mat4.perspective(projectionMatrix, Math.PI / 4, canvas.width / canvas.height, 0.1, 100.0);

    const uniformLocation = gl.getUniformLocation(program, 'uModelViewProjection');
    gl.uniformMatrix4fv(uniformLocation, false, projectionMatrix);

    // Draw
    gl.drawArrays(gl.TRIANGLES, 0, 3);

    // Animation loop
    requestAnimationFrame(() => render(gl, program));
}

Matrix Mathematics

WebGL uses column-major matrices for transformations. Recommended libraries:

• glMatrix - Fast matrix/vector operations
• three.js - High-level 3D library with built-in math

Common transformations:

// Model matrix (object transform)
const modelMatrix = mat4.create();
mat4.translate(modelMatrix, modelMatrix, [x, y, z]);
mat4.rotate(modelMatrix, modelMatrix, angle, [0, 1, 0]);
mat4.scale(modelMatrix, modelMatrix, [sx, sy, sz]);

// View matrix (camera)
const viewMatrix = mat4.create();
mat4.lookAt(viewMatrix, eyePosition, targetPosition, upVector);

// Projection matrix
const projectionMatrix = mat4.create();
mat4.perspective(projectionMatrix, fov, aspect, near, far);

// Combined MVP matrix
const mvpMatrix = mat4.create();
mat4.multiply(mvpMatrix, projectionMatrix, viewMatrix);
mat4.multiply(mvpMatrix, mvpMatrix, modelMatrix);

Performance Optimization

Best Practices

1. Minimize state changes - Batch draw calls with similar state
2. Use Vertex Array Objects (VAO) - Reduce attribute setup overhead
3. Texture atlases - Combine multiple textures into one
4. Instanced rendering - Draw many similar objects efficiently
5. Frustum culling - Don't render objects outside view
6. Level of Detail (LOD) - Use simpler models at distance
7. Texture compression - Use compressed texture formats (DXT, ETC, ASTC)
8. Minimize shader complexity - Keep fragment shaders simple
9. Use uniform buffers (WebGL 2) - Efficient uniform data sharing
10. Avoid CPU-GPU synchronization - Don't read back data frequently

Instanced Rendering (WebGL 2)

const ext = gl.getExtension('ANGLE_instanced_arrays'); // WebGL 1
// or use gl.drawArraysInstanced directly in WebGL 2

// Set up per-instance attribute
const instanceOffsetBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, instanceOffsetBuffer);
gl.bufferData(gl.ARRAY_BUFFER, offsetData, gl.STATIC_DRAW);

const offsetLocation = gl.getAttribLocation(program, 'aInstanceOffset');
gl.enableVertexAttribArray(offsetLocation);
gl.vertexAttribPointer(offsetLocation, 3, gl.FLOAT, false, 0, 0);
gl.vertexAttribDivisor(offsetLocation, 1); // Advance per instance

// Draw multiple instances
gl.drawArraysInstanced(gl.TRIANGLES, 0, vertexCount, instanceCount);

Extension System

Check for and use extensions to access advanced features:

function getExtension(gl, name) {
    const ext = gl.getExtension(name);
    if (!ext) {
        console.warn(`Extension ${name} not supported`);
    }
    return ext;
}

// Common extensions
const anisotropic = getExtension(gl, 'EXT_texture_filter_anisotropic');
const floatTextures = getExtension(gl, 'OES_texture_float');
const depthTexture = getExtension(gl, 'WEBGL_depth_texture');
const drawBuffers = getExtension(gl, 'WEBGL_draw_buffers');
const loseContext = getExtension(gl, 'WEBGL_lose_context'); // for testing

Important extension categories:

• Texture formats: WEBGL_compressed_texture_s3tc, WEBGL_compressed_texture_etc
• Rendering: WEBGL_draw_buffers, EXT_blend_minmax, EXT_frag_depth
• Precision: OES_texture_float, OES_texture_half_float
• Instancing: ANGLE_instanced_arrays (WebGL 1)
• Debugging: WEBGL_debug_renderer_info, WEBGL_debug_shaders

Context Management

Context Loss Handling

canvas.addEventListener('webglcontextlost', (event) => {
    event.preventDefault();
    console.log('WebGL context lost');
    cancelAnimationFrame(animationId);
}, false);

canvas.addEventListener('webglcontextrestored', () => {
    console.log('WebGL context restored');
    initWebGL(); // Recreate all resources
    render();
}, false);

Context Creation Options

const gl = canvas.getContext('webgl2', {
    alpha: false,                    // No alpha channel (better performance)
    antialias: true,                 // Antialiasing (performance cost)
    depth: true,                     // Depth buffer
    stencil: false,                  // Stencil buffer
    premultipliedAlpha: true,        // Alpha premultiplication
    preserveDrawingBuffer: false,    // Keep buffer after render
    powerPreference: 'high-performance', // GPU preference
    failIfMajorPerformanceCaveat: false  // Fallback to software
});

Common Patterns

Framebuffer Rendering (Render to Texture)

const framebuffer = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer);

const targetTexture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, targetTexture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, gl.UNSIGNED_BYTE, null);

gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, targetTexture, 0);

// Render to framebuffer
gl.bindFramebuffer(gl.FRAMEBUFFER, framebuffer);
gl.viewport(0, 0, width, height);
// ... render scene ...

// Render to canvas
gl.bindFramebuffer(gl.FRAMEBUFFER, null);
gl.viewport(0, 0, canvas.width, canvas.height);

Multiple Render Targets (WebGL 2)

const ext = gl.getExtension('WEBGL_draw_buffers'); // WebGL 1
// Fragment shader outputs to multiple targets
gl.drawBuffers([
    gl.COLOR_ATTACHMENT0,
    gl.COLOR_ATTACHMENT1,
    gl.COLOR_ATTACHMENT2
]);

Common Pitfalls

1. Not checking compilation/linking errors - Always check shader status
2. Forgetting to enable attributes - Call gl.enableVertexAttribArray()
3. Incorrect data types - Use Float32Array, Uint16Array, etc.
4. Not handling context loss - Add event listeners
5. Mixing WebGL 1 and 2 APIs - Check version compatibility
6. Power-of-2 texture assumptions - Handle non-POT textures correctly
7. Z-fighting - Insufficient depth buffer precision
8. Coordinate system confusion - WebGL uses clip space [-1, 1]
9. Premature optimization - Profile before optimizing
10. Not clearing buffers - Call gl.clear() each frame

Debugging Tools

1. Browser DevTools - Check console for WebGL errors
2. WebGL Inspector - Browser extension for frame capture
3. Spector.js - WebGL debugging library
4. gl.getError() - Check for runtime errors
5. WEBGL_debug_shaders - Get translated shader source

// Error checking
const error = gl.getError();
if (error !== gl.NO_ERROR) {
    console.error('WebGL error:', error);
}

Popular Libraries and Frameworks

• three.js - Comprehensive 3D library with scene graph
• Babylon.js - Game engine with physics and VR support
• PlayCanvas - Cloud-based game engine
• Pixi.js - Fast 2D WebGL renderer
• Phaser - 2D game framework
• regl - Functional WebGL wrapper
• twgl - Tiny WebGL helper library
• glMatrix - High-performance matrix/vector library

Learning Resources

• MDN WebGL Tutorial
• WebGL Fundamentals
• The Book of Shaders
• Shadertoy - Shader examples
• WebGL2 Fundamentals

Quick Reference

See reference.md for:

• Complete constant reference
• All WebGL methods
• GLSL built-in functions
• Extension compatibility matrix

See examples for:

• Basic triangle rendering
• Texture mapping
• Lighting models
• Advanced techniques

Version Compatibility

When supporting both WebGL 1 and 2:

function initWebGL(canvas) {
    const gl = canvas.getContext('webgl2');
    let version = 2;

    if (!gl) {
        gl = canvas.getContext('webgl');
        version = 1;
        console.log('Using WebGL 1');
    }

    // Feature detection
    const hasVAO = version === 2 || gl.getExtension('OES_vertex_array_object');
    const hasInstancing = version === 2 || gl.getExtension('ANGLE_instanced_arrays');

    return { gl, version, hasVAO, hasInstancing };
}

Security Considerations

• Cross-origin textures - Use CORS properly
• Shader validation - Validate user-provided shader code
• Resource limits - Don't trust client-reported capabilities
• Timing attacks - Be aware of shader compilation timing
• Context fingerprinting - Users may block WebGL for privacy

When helping users with WebGL:

1. Determine version - Check if WebGL 1 or 2 is needed
2. Check requirements - Browser support, extensions needed
3. Start simple - Basic rendering before advanced features
4. Debug systematically - Check shaders, buffers, state in order
5. Profile performance - Use browser tools to identify bottlenecks
6. Consider libraries - Recommend three.js/Babylon.js for complex projects
7. Validate inputs - Check for null contexts, compilation errors
8. Handle context loss - Always implement recovery
9. Optimize appropriately - Don't over-optimize early
10. Test across devices - GPU capabilities vary significantly