1. Project Vision: Towards a Symbiotic Game Engine

Khora was born from a critical observation: modern game engines, for all their power, are fundamentally rigid. They impose static pipelines, force developers into complex manual optimization, and adapt poorly to the ever-growing diversity of hardware—from high-end PCs to mobile devices and VR platforms.

Our vision is to build an engine that behaves not as a machine, but as a living organism. A symbiotic system where each component is aware of its environment and collaborates to achieve a common goal. Instead of following a fixed set of instructions, Khora observes, learns, and continuously adapts.

The Problem with Rigidity

• Static Resource Allocation: Fixed CPU/GPU budgets are ill-suited to the dynamic complexity of a game scene, leading to either underutilization or performance bottlenecks.
• Laborious Manual Optimization: Developers spend a disproportionate amount of time tuning settings for each target platform, a task that is both tedious and fragile.
• Lack of Contextual Awareness: A traditional engine cannot make intelligent trade-offs. It does not know if a performance dip in the audio system is more or less impactful to the player’s experience than one in the renderer during a critical cinematic sequence.

The Khora Solution: The Symbiotic Adaptive Architecture (SAA)

Khora flips the paradigm. It replaces a rigid, top-down orchestrator with a council of intelligent, collaborating agents.

• Automated Self-Optimization: The engine autonomously detects performance bottlenecks and reallocates resources to resolve them.
• Strategic Flexibility: The rendering subsystem can dynamically switch from a fast, low-fidelity technique to a high-quality one based on system load and performance targets, without any direct developer intervention.
• Goal-Oriented Decision Making: All adaptations are driven by high-level goals, such as “maintain 90fps in VR,” “prioritize visual quality during cinematics,” or “conserve battery on mobile.”

The ultimate goal is to empower creators to focus entirely on their artistic vision, trusting the engine to handle the complex technical challenge of delivering a smooth, optimal, and resilient experience on any platform.

2. Core Concepts: The Symbiotic Adaptive Architecture (SAA)

The Symbiotic Adaptive Architecture (SAA) is the philosophical and conceptual framework of Khora. It is built on seven key pillars that work in symbiosis to create a truly adaptive engine.

1. Dynamic Context Core (DCC) - The Central Nervous System

The DCC is the engine’s center of awareness. It does not command subsystems directly; instead, it maintains a constantly updated situational model of the entire application state. It acts as a central hub, aggregating telemetry from across the engine to understand the “big picture”:

• Hardware Load: Real-time utilization of CPU cores, GPU, VRAM, and memory bandwidth.
• Game State: Scene complexity, entity counts, light sources, physics interactions, network status.
• Performance Goals: The currently active objectives, such as target framerate, maximum input latency, or power consumption budget.

2. Intelligent Subsystem Agents (ISAs) - The Specialists

Every major engine subsystem (Rendering, Physics, Audio, AI, Assets) is designed as an Intelligent Subsystem Agent. An ISA is not a passive library; it is a semi-autonomous component with a deep understanding of its own domain.

• Self-Assessment: It constantly measures its own performance and resource consumption.
• Multi-Strategy: It possesses multiple algorithms to accomplish its task, each with different performance characteristics (e.g., a precise but slow physics solver vs. a fast but approximate one).
• Cost Estimation: It can accurately predict the resource cost (CPU time, memory) of each of its strategies under the current conditions.

3. Goal-Oriented Resource Negotiation & Allocation (GORNA) - The Council Protocol

GORNA is the formal communication protocol used by the DCC and the ISAs to dynamically allocate resources. This negotiation process replaces static, pre-defined budgets.

1. Request: ISAs submit their desired resource needs to the DCC, often specifying the strategy they intend to use (e.g., “The Rendering Agent requests 8ms of GPU time to execute its High-Fidelity strategy”).
2. Arbitration: The DCC analyzes all incoming requests, comparing them against its global situational model and the active performance goals.
3. Allocation: The DCC grants a final budget to each ISA. This budget may be less than what was requested.
4. Adaptation: An ISA that receives a reduced budget is responsible for adapting. It must select a less resource-intensive strategy to stay within its allocated budget.

Implementation Status: GORNA v0.3 is fully operational. The DCC runs 9 heuristics each tick (Phase, Thermal, Battery, Frame Time, Stutter, Trend, CPU/GPU Pressure, Death Spiral), the GornaArbitrator resolves multi-agent resource conflicts, and the RenderAgent implements cost-based negotiation with VRAM-aware filtering. The PhysicsAgent is the second fully GORNA-compliant ISA. An initial GORNA round fires automatically on the first tick after agent registration, ensuring baseline budgets are assigned immediately. All lanes across the engine implement the unified Lane trait with LaneContext-based dispatch (see Chapter 4 and Chapter 10). See Chapter 11 and Chapter 12 for the complete specification.

4. Adaptive Game Data Flows (AGDF) - The Living Data

AGDF is the principle that not only algorithms but also the very structure of data should be dynamic. This advanced concept is realized through our custom ECS, the CRPECS. Instead of being static, an entity’s data layout can be fundamentally altered by the SAA in response to the game’s context. For example, the Control Plane can cheaply remove physics components from an entity that is far from the player, and add them back when it gets closer. The CRPECS’s design makes these structural changes extremely low-cost, enabling a deeper level of self-optimization at the memory level.

5. Semantic Interfaces & Contracts - The Common Language

For intelligent negotiation to be possible, all ISAs must speak a common, unambiguous language. These are defined by a set of formal contracts (Rust traits) that specify an ISA’s capabilities and requirements.

• Capabilities: What an ISA can do (“I can render scenes using Forward+ or a Simple Unlit pipeline”).
• Requirements: What data it needs to function (“I require access to all entity positions and meshes”).
• Guarantees: What it promises in return for a given budget (“With a 4ms CPU budget, I guarantee a stable physics simulation for up to 1000 active rigid bodies”).

6. Observability & Traceability - The Glass Box

An intelligent system risks becoming an indecipherable “black box.” To prevent this, Observability is a first-class principle in Khora. Every significant decision made by the DCC is meticulously logged with its complete context—the telemetry, the requests, and the final budget allocation. This allows developers to ask not just “what happened?” but “why did the engine make that choice?”, which is crucial for debugging, tuning, and building trust in the adaptive system.

7. Developer Guidance & Control - A Partnership, Not an Autocracy

The engine’s autonomy serves the developer, it does not replace them. Khora provides clear mechanisms for developers to guide and constrain the SAA.

• Constraints: Developers can define rules or physical volumes in the world to influence decision-making (e.g., “In this zone, physics accuracy is more important than graphical fidelity”).
• Adaptation Modes (planned): The DCC will be able to operate in different modes:
  • Learning: The default, fully dynamic mode where the engine explores strategies to meet its goals.
  • Stable: Uses heuristics learned from the Learning mode but avoids drastic changes. Ideal for profiling and shipping a predictable experience.
  • Manual: Disables the SAA entirely, allowing developers to lock in specific strategies for debugging or benchmarking.

3. The SAA-CLAD Symbiosis: From Philosophy to Practice

The Khora Engine is built on two core architectural concepts: the Symbiotic Adaptive Architecture (SAA) and the CLAD Pattern. It is crucial to understand that these are not two separate architectures; they are two sides of the same coin, representing the vision and its execution.

• SAA is the “Why”: It is our philosophical blueprint. It describes what we want to achieve—a self-optimizing, adaptive engine where intelligent subsystems collaborate to meet high-level goals.
• CLAD is the “How”: It is our concrete implementation strategy in Rust. It provides the strict rules, crate structure, and data flow patterns required to make the SAA vision a high-performance, maintainable reality.

Every abstract concept in the SAA has a direct, physical home within the CLAD structure. This explicit mapping ensures that our code is always a faithful implementation of our vision and provides a clear mental model for development.

graph TD
    subgraph SAA ["Symbiotic Adaptive Architecture (The WHY)"]
        DCC[Dynamic Context Core]
        ISA[Intelligent Subsystem Agents]
        GORNA[GORNA Protocol]
        Strategies[Multiple Strategies]
    end

    subgraph CLAD ["CLAD Crate Pattern (The HOW)"]
        Control[khora-control]
        Agents[khora-agents]
        Lanes[khora-lanes]
        Core[khora-core]
    end

    DCC --> Control
    GORNA --> Control
    ISA --> Agents
    Strategies --> Lanes
    
    Control -.-> |"Orchestrates"| Agents
    Agents -.-> |"Switches"| Lanes
    Lanes -.-> |"Uses Traits"| Core

The SAA-CLAD Mapping

This clear separation of concerns is the key to resolving the classic conflict between complexity and performance. It allows Khora to be highly intelligent and dynamic in its Control Plane (control, agents) while being uncompromisingly fast and predictable in its Data Plane (lanes, data).

4. Technical Architecture: The CLAD Pattern

The CLAD (Control-Lane-Agent-Data) pattern is the concrete Rust implementation of the SAA philosophy. It is an architectural style designed for extreme performance and maintainability by enforcing a strict separation of concerns.

Core Principle: The Hot/Cold Path Split

The entire engine is architected around isolating the Control Plane (Cold Path) from the Data Plane (Hot Path).

sequenceDiagram
    participant DCC as khora-control (DCC)
    participant Agent as khora-agents (ISA)
    participant Lane as khora-lanes (Lane)
    participant Data as khora-data (CRPECS)

    Note over DCC,Agent: Cold Path (Control Plane)
    DCC->>Agent: Allocate Budget & Strategy
    Agent->>Lane: Configure & Dispatch
    
    Note over Lane,Data: Hot Path (Data Plane)
    loop Every Frame
        Lane->>Data: Query Transversal Data
        Data-->>Lane: Contiguous Component Pages
        Lane->>Lane: Execute SIMD/Optimized Work
    end
    
    Lane-->>DCC: Telemetry (Cost/Perf)

• Cold Path (Control Plane):

  • Purpose: Complex, stateful decision-making.
  • Components: khora-control, khora-agents.
  • Frequency: Ticks at a lower, non-critical rate (e.g., 10-60 Hz).
  • Characteristics: Permitted to use complex logic, perform memory allocations, write logs, and analyze data. Its execution time does not directly impact the rendering time of a single frame.

• Hot Path (Data Plane):

  • Purpose: Raw, deterministic work execution.
  • Components: khora-lanes, khora-data.
  • Frequency: Must execute within a strict frame budget (e.g., < 16.67ms for 60fps).
  • Characteristics: Optimized for raw speed. Aims for zero heap allocations, cache-friendly data access (SoA), SIMD operations, and minimal branching.

The CLAD Crates: Roles and Responsibilities

khora-core - The Foundation

The bedrock of the engine. It contains only abstract traits, universal data types (like math primitives), and interface contracts. It has zero dependencies on other khora-* crates and defines the universal "language" of the engine.

khora-data - The Data Layer ([D]ata)

The heart of Khora’s data management strategy. This crate’s primary responsibility is the implementation of our custom, high-performance CRPECS. It provides the concrete foundation for the AGDF concept, containing the specialized memory allocators, page structures, and the query engine that drives the entire Data Plane.

khora-lanes - The Hot Path ([L]ane)

This crate contains the performance-critical, "dumb" execution pipelines. A Lane is a specific implementation of an ISA’s strategy (e.g., a rendering pass, a physics solver). They are optimized for linear data processing and contain no complex branching logic.

khora-agents - The Tactical Brains ([A]gent)

This crate is the home of the ISAs. Each agent is an intelligent wrapper around one or more Lanes. It is responsible for reporting its status to the Control Plane, estimating the cost of its strategies, and, upon receiving a budget, configuring and dispatching the appropriate Lane.

khora-control - The Strategic Brain ([C]ontrol)

The highest level of decision-making. This crate contains the DCC and the GORNA protocol implementation. It consumes telemetry, evaluates the overall situation against high-level goals, and orchestrates the Agents by allocating their resource budgets.

khora-telemetry - The Nervous System

A dedicated service for collecting, aggregating, and exposing engine-wide metrics. It gathers raw data from khora-infra (e.g., VRAM usage) and the Lanes (e.g., execution time) and provides it in a structured format to khora-control and debugging tools.

khora-infra - The Bridge to the World

This crate contains all concrete implementations that interact with external systems: GPU backends (WGPU), windowing (Winit), filesystem I/O, networking libraries, etc. It primarily implements the generic traits defined in khora-core.

khora-sdk - The Public Facade

A simple, stable API designed for game developers. It hides the complexity of the internal CLAD architecture and provides ergonomic, easy-to-use entry points for building an application with Khora. It owns the ECS World and Assets internally and exposes them through the GameWorld facade — users never interact with raw engine types.

khora-macros - Development Ergonomics

A procedural macro crate that provides derive macros to reduce boilerplate and improve developer experience. This crate implements compile-time code generation to automatically implement common traits throughout the engine. Currently provides the #[derive(Component)] macro for automatically implementing the khora_data::ecs::Component trait while enforcing its required bounds (Clone, Send, Sync, 'static). As the engine evolves, this crate will house additional derive macros and attribute macros to streamline development across all CLAD layers.

khora-plugins - Extensions and Packages

A specialized crate that packages independent strategies, lanes, or pre-configured systems into modular plugins. This allows the engine to be extended with new capabilities (e.g., specific rendering techniques, AI behaviors) without bloating the core architecture.

khora-editor - The Engine Tooling

An interactive graphical application built on top of the khora-sdk utilizing egui. It serves as the primary visual tool for interacting with the engine, providing a real-time viewport, scene hierarchy panel, and context visualization for the DCC’s decisions.

The Lane Abstraction Layer

At the center of the CLAD pattern sits the Lane abstraction: a unified trait, a typed context mechanism, and a generic registry. Together they allow agents to manage heterogeneous processing strategies without domain-specific coupling.

The Lane Trait (khora-core::lane)

Every processing strategy in the engine — rendering, shadow mapping, physics, audio mixing, asset loading, serialization, ECS maintenance — implements Lane:

#![allow(unused)]
fn main() {
pub trait Lane: Send + Sync {
    fn strategy_name(&self) -> &'static str;
    fn lane_kind(&self) -> LaneKind;
    fn estimate_cost(&self, ctx: &LaneContext) -> f32;
    fn on_initialize(&self, ctx: &mut LaneContext) -> Result<(), LaneError>;
    fn execute(&self, ctx: &mut LaneContext) -> Result<(), LaneError>;
    fn on_shutdown(&self, ctx: &mut LaneContext);
    fn as_any(&self) -> &dyn Any;
    fn as_any_mut(&mut self) -> &mut dyn Any;
}
}

The lifecycle is linear: on_initialize → [execute]* → on_shutdown. Agents call these methods — lanes never call each other directly.

LaneKind classifies each lane so agents can route execution correctly:

LaneContext — The Type-Map

LaneContext is a HashMap<TypeId, Box<dyn Any>> that serves as the only data channel between agents and lanes. The agent populates it before dispatching; each lane reads what it needs and writes results for downstream consumers.

#![allow(unused)]
fn main() {
let mut ctx = LaneContext::new();
ctx.insert(device.clone());                // Arc<dyn GraphicsDevice>
ctx.insert(ColorTarget(view_id));          // Typed key
ctx.insert(Slot::new(&mut render_world));  // Mutable borrow wrapper
}

This design eliminates domain-specific trait signatures (no fn render(&self, world: &RenderWorld, encoder: &mut dyn CommandEncoder, …)) and keeps each lane loosely coupled to its agent.

Slot<T> and Ref<T> — Borrow Wrappers

For data the agent borrows rather than owns, two wrappers erase the borrow lifetime so the value can be stored in the type-map:

Safety is guaranteed by the stack-scoped context pattern: the LaneContext is created in the agent’s frame method, passed to one lane at a time, and dropped before the function returns. The Slot/Ref never outlive the original reference.

LaneRegistry — Generic Lane Storage

Agents store their lanes in a LaneRegistry instead of using domain-specific vectors:

#![allow(unused)]
fn main() {
pub struct LaneRegistry {
    lanes: Vec<Box<dyn Lane>>,
}
}

Key operations:

This allows developers to add custom lanes without modifying agent code — simply register the lane and the agent will discover it through the registry.

Context Keys (khora-core::lane::context_keys)

Standard typed keys used across the engine:

Agents and lanes agree on these types at compile time — no string-based lookup, no runtime key negotiation.

5. Project and Crate Structure

The Khora project is organized as a Cargo workspace to enforce modularity, enable efficient compilation, and reflect our CLAD architecture. This document provides a high-level overview of the repository’s layout.

Top-Level Directory Structure

• .github/: Contains GitHub-specific configurations like CI workflows and issue templates.
• crates/: The heart of the engine. Contains all the core khora-* source code, organized into modular crates.
• docs/: Contains all project documentation, including the source for this book.
• examples/: Engine usage examples and testbeds, with sandbox being our primary test application demonstrating khora-sdk usage without internal khora-core dependencies.
• resources/: Runtime configuration files, such as default profiles for the DCC.
• xtask/: A dedicated crate for build automation and scripting tasks (cargo-xtask).
• Cargo.toml: The root workspace definition, specifying members and compilation profiles.

Detailed Crate Layout

The following is a representative layout of the crates/ directory, illustrating the internal structure of our CLAD implementation.

crates/
├── khora-core/      # FOUNDATIONAL: Traits, core types, interface contracts.
│   └── src/
│       ├── lane/            # Lane trait, LaneContext, LaneRegistry, Slot/Ref, context_keys
│       ├── math/
│       ├── platform/
│       ├── renderer/
│       └── ...
│
├── khora-control/   # [C]ONTROL: DCC and GORNA implementation.
│   └── src/
│
├── khora-data/      # [D]ATA: CRPECS, resources, and other data containers.
│   └── src/
│       ├── ecs/
│       └── ...
│
├── khora-lanes/     # [L]ANES: Hot-path execution pipelines (all implement Lane).
│   └── src/
│       ├── asset_lane/      # PackLoadingLane (VFS pack file streaming)
│       ├── audio_lane/
│       │   └── mixing/      # SpatialMixingLane (3D audio mixing)
│       ├── ecs_lane/        # CompactionLane (archetype memory defragmentation)
│       ├── physics_lane/    # StandardPhysicsLane, VerletPhysicsLane
│       ├── render_lane/
│       │   ├── shaders/     # WGSL: lit_forward.wgsl, shadow_depth.wgsl, etc.
│       │   ├── simple_unlit_lane.rs
│       │   ├── lit_forward_lane.rs
│       │   ├── forward_plus_lane.rs
│       │   ├── shadow_pass_lane.rs
│       │   ├── extract_lane.rs
│       │   └── world.rs     # RenderWorld, ExtractedLight, ExtractedMesh
│       └── scene_lane/      # ArchetypeLane, SnapshotLane (serialization strategies)
│
├── khora-agents/    # [A]GENTS: Intelligent wrappers driving the Lanes.
│   └── src/
│       ├── render_agent/        # RenderAgent — GPU rendering ISA (GORNA ✅)
│       ├── physics_agent/       # PhysicsAgent — physics simulation ISA (GORNA ✅)
│       ├── audio_agent/         # AudioAgent — spatial audio mixing
│       ├── asset_agent/         # AssetAgent — async asset loading
│       ├── serialization_agent/ # SerializationAgent — scene persistence
│       └── ecs_agent/           # GarbageCollectorAgent — ECS maintenance
│
├── khora-telemetry/ # Central service for metrics and monitoring.
│   └── src/
│
├── khora-infra/     # Concrete implementations of external dependencies.
│   └── src/
│       ├── audio/
│       ├── graphics/    # wgpu backend (device, command encoding, pipelines)
│       ├── physics/
│       ├── platform/
│       └── telemetry/
│
├── khora-sdk/       # [USER-FACING] The stable, public-facing API for game developers.
│   └── src/         # No internal engine data structures are exposed; only traits and Vessels.
│
├── khora-editor/    # [TOOLING] The engine's editor GUI using egui.
│   └── src/
│
├── khora-plugins/   # [EXTENSIONS] Packaged strategies and extensions.
│   └── src/
│
└── khora-macros/    # Procedural macros for code generation (derive macros).
    └── src/

6. ECS Architecture: The CRPECS

This document details the architecture of Khora’s custom Chunked Relational Page ECS (CRPECS). It was designed from the ground up to be the high-performance backbone of the SAA and to enable its most advanced concept: Adaptive Game Data Flows (AGDF).

Philosophy: Performance Through Intelligent Compromise

Modern ECS architectures typically force a difficult choice:

• Archetypal (e.g., bevy_ecs): Delivers extremely fast iteration speeds due to perfect data locality, but incurs a very high performance cost when an entity’s component structure changes (an “archetype move”).
• Sparse Set (e.g., specs): Offers very fast structural changes, but iteration can be slower for entities with many components due to increased pointer indirection and poorer data locality.

Khora’s SAA vision requires the best of both worlds: fast, predictable iteration for the Data Plane (the Lanes), and cheap, frequent structural changes driven by the Control Plane (the Agents and Control crates). The CRPECS resolves this conflict by creating a new set of trade-offs perfectly aligned with our goals.

Its core principle is to completely dissociate an entity’s logical identity from the physical storage location of its component data.

graph LR
    subgraph Metadata ["Entity Registry (Metadata)"]
        E1[Entity 1]
        E2[Entity 2]
    end

    subgraph Pages ["Component Pages (SoA)"]
        subgraph PhysicsDomain ["Physics Domain Page"]
            P1[Pos: x,y,z]
            V1[Vel: x,y,z]
        end
        subgraph RenderDomain ["Render Domain Page"]
            M1[Mesh Handle]
            Mat1[Material Handle]
        end
    end

    E1 -.-> |"Pointer"| P1
    E1 -.-> |"Pointer"| M1
    E2 -.-> |"Pointer"| V1

Core Principles

1. Component Pages: Semantic Data Grouping

Instead of storing components based on the entity’s complete structure (its archetype), we group them by semantic domain into fixed-size memory blocks called Pages (e.g., 16KB).

• A PhysicsPage would store the contiguous Vecs for Position, Velocity, and Collider.
• A RenderPage would store the contiguous Vecs for HandleComponent<Mesh>, MaterialComponent, etc.

Within each Page, component data is stored as a Structure of Arrays (SoA), guaranteeing optimal cache performance for any system iterating within that domain.

2. Entity Metadata: The Relational Hub

An entity’s identity is maintained in a central table, completely separate from its component data. Each entry in this table is a small struct containing pointers that map the entity’s ID to the physical location of its data across all relevant Pages.

#![allow(unused)]
fn main() {
// A logical pointer to a piece of component data.
struct PageIndex {
    page_id: u32,   // Which page holds the data?
    row_index: u32, // Which row within that page's SoA?
}

// The central "table of contents" for a single entity.
struct EntityMetadata {
    // A map from a component's semantic domain to its physical location.
    locations: HashMap<SemanticDomain, PageIndex>,
}
}

3. Data Dissociation: The Key to Flexibility

The data for a single entity is physically scattered across multiple Pages, but it is logically unified by its EntityMetadata. This is the architectural key that unlocks our required flexibility.

How Key Operations Work

Iteration (e.g., Query<(&mut Position, &Velocity)>)

• Performance: Near-Optimal.
• The query understands that Position and Velocity both belong to the Physics semantic domain. It therefore only needs to iterate through all active PhysicsPages, accessing their contiguous Vecs. This achieves performance nearly identical to a pure Archetypal ECS for domain-specific queries.

Structural Changes: Fast Logic, Asynchronous Cleanup

• add_component<C>(): This is a fast operation that handles the addition of a new component to an entity. It migrates the entity’s existing components for the relevant SemanticDomain to a new ComponentPage that matches the new component layout. Crucially, it does not clean up the “hole” left in the old page. Instead, it orphans the data at the old location and queues it for garbage collection.
• remove_component_domain<C>(): This is an extremely high-performance O(1) operation. It removes all components in a given SemanticDomain from an entity by simply deleting an entry from the locations HashMap in its EntityMetadata. Like add_component, this operation orphans the actual component data and queues it for cleanup by the garbage collector.

The Garbage Collection Process

• The actual component data from add and remove operations is now “orphaned” and will be cleaned up later by a low-priority, asynchronous garbage collection process, ensuring that performance-critical code is not blocked by expensive cleanup operations.

The Intelligent Compromise: Transversal Queries

Khora now provides a full implementation for transversal queries—queries that access data from different semantic domains simultaneously (e.g., Query<(&Position, &RenderTag)>).

Mechanism: The Domain Join

Such a query cannot iterate linearly over a single set of Pages. It performs a “join” across domains using a Driver Domain strategy.

sequenceDiagram
    participant Q as QueryIterator
    participant W as World / Registry
    participant D as Driver Domain (e.g. Render)
    participant M as EntityMetadata
    participant P as Peer Domains (e.g. Spatial)

    Q->>W: Request Plan (Driver = Render)
    W-->>Q: QueryPlan + Matching Pages [P1, P2...]
    loop For each Entity in Driver Page
        Q->>Q: Get EntityId from Driver Row
        Q->>Q: Check Bitset Intersection (Fast Skip)
        alt Bitset Set
            Q->>M: Lookup Peer PageIndex
            M-->>Q: Page P1, Row 5
            Q->>P: Fetch Peer Component
            P-->>Q: Return Joined Item
        else Bitset Clear
            Q->>Q: Skip to next row
        end
    end

Bitset-Guided Optimization

To minimize the cost of metadata lookups, Khora uses a Signature-Based Bitset Join. Before starting iteration, the World computes the bitwise intersection of the DomainBitsets for all domains involved in the query.

• Fast-Skip: The iterator uses this pre-computed bitset to instantly skip any entity that does not exist in all required domains.
• Result: This avoids expensive pointer lookups (Metadata HashMap access) for entities that are guaranteed to fail the join, making transversal queries highly efficient even in sparse scenarios.

Dynamic Strategy & Cache

The query system uses a Stateful Strategy Plan. Even though it resides in the Data Plane, it distinguishes between the heavy architectural analysis and the lightweight per-call execution.

graph TD
    subgraph Init ["1. Strategy Analysis (First-Call)"]
        Analyze[Analyze Query Tuple] --> Domains[Identify Domains]
        Domains --> Mode{Transversal?}
        Mode -- No --> Native[Native Plan]
        Mode -- Yes --> SelectDriver[Select Driver Domain]
        SelectDriver --> Trans[Transversal Plan]
    end

    subgraph Dynamic ["2. Dynamic Resolution (Per-Call)"]
        Native --> ReFind[Re-evaluate Page Indices]
        Trans --> ReFind
        ReFind --> Bitset[Compute/Fetch Bitset Intersection]
        Bitset --> Iterate[Return Iterator]
    end

    subgraph HotLoop ["3. Iteration (The Hot Loop)"]
        Iterate --> RowCheck{Row/Bitset Check}
        RowCheck -- Set --> Fetch[Fetch Across Domains]
        RowCheck -- Clear --> Skip[Skip Row]
        Fetch --> RowCheck
    end

    Cache[(Strategy Cache)]
    Trans -.-> |Store| Cache
    Native -.-> |Store| Cache
    Cache -.-> |Reuse Strategy| ReFind

• Strategy Memoization: The execution strategy (which domain drives, which are peers) is memoized to avoid repeating the architectural analysis.
• Per-Call Correctness: While the strategy is cached, the matching page indices and bitset intersections are re-evaluated dynamically on every call. This ensures the engine remains stable and correct even if the world’s archetype layout changes frequently.

Memory Layout: The Transversal View

A transversal join essentially creates a virtual, temporary “bridge” between physically disjoint SoA pages.

graph LR
    subgraph Driver ["Spatial Domain (Driver)"]
        DP["Page 0"]
        DR1["Row 0: ID 101"]
        DR2["Row 1: ID 102"]
    end

    subgraph Bitset ["Selection Filter"]
        BS["Bit 101: [1]<br/>Bit 102: [0]"]
    end

    subgraph Peer ["Render Domain (Peer)"]
        PP["Page 5"]
        PR1["Row 12: MeshA"]
        PR2["Row 13: MeshB"]
    end

    DR1 --> BS
    BS -- "Match!" --> Metadata[Metadata Lookup]
    Metadata --> PR1
    PR1 --> Result["Joined (Pos, Mesh)"]
    
    DR2 --> BS
    BS -- "NO MATCH" --> Skip["Fast Skip"]

Architectural Benefit

This is a deliberate design choice. While transversal joins are highly optimized, they remain slightly slower than native, domain-specific iteration. This creates a natural “architectural gravity” that encourages developers to group related data into the same semantic domain, leading to cleaner and more performant systems.

Integration with CLAD and SAA

The CRPECS is the cornerstone of the khora-data crate and the ultimate implementation of the [D]ata in CLAD.

• It provides the perfect foundation for AGDF, as the SAA’s Control Plane can cheaply and frequently alter data layouts by modifying EntityMetadata using the add_component and remove_component_domain methods.
• The garbage collection and page compaction process has been implemented as a prime example of the CLAD and SAA philosophy. A GarbageCollectorAgent ([A]), acting as an ISA, makes strategic decisions about when and how much to clean. It dispatches this work to a dedicated CompactionLane ([L]), which performs the heavy lifting of modifying the component page [D]ata. This makes the ECS itself a living, self-optimizing part of the SAA.

07 - Asset Architecture: The Virtual File System (VFS)

The Problem: Assets as a Strategic Resource

In a traditional engine, an asset system is a simple loader: you ask for a file path, it gives you back data. This is insufficient for the Symbiotic Adaptive Architecture (SAA). To make intelligent decisions, the AssetAgent ISA needs to understand assets not as files on disk, but as strategic resources with costs, dependencies, and alternatives.

Khora’s solution is the Virtual File System (VFS). It is not a filesystem in the OS sense; it’s an intelligent database that abstracts the physical storage of assets and enriches them with metadata, turning them into a queryable resource for the entire engine.

Core Concepts

1. Logical Identification: AssetUUID and AssetHandle

The engine logic never references a physical file path. Instead, it uses two central types:

• AssetUUID: A unique and stable identifier that represents the “idea” of an asset, regardless of its location or filename. In our production pipeline, it is generated deterministically from the asset’s source path.
• AssetHandle<T>: A lightweight, cloneable smart pointer (Arc<T>). This is the type that engine systems manipulate. This indirection is critical: it allows the AssetAgent to manage the asset’s lifecycle in the background without ever affecting the game logic that holds the handle.

2. The Abstract Data Source: The AssetSource Enum

To support both development and production workflows, we need an abstraction over where asset data comes from. The AssetSource enum fulfills this role in a type-safe manner:

• AssetSource::Path(PathBuf): Used by the editor (future). Indicates that the asset is a loose file on disk.
• AssetSource::Packed { offset, size }: Used by the runtime. Indicates the exact location of the asset’s data within a data.pack file.

3. Rich Metadata and a Hybrid Indexing Strategy

The core intelligence of the VFS lies in the AssetMetadata. To create and use this metadata, Khora employs a hybrid strategy:

• For Production (What we just built): Packfiles

  • A build tool (xtask) runs the assets pack command.
  • This tool scans source directories (defined in Assets.toml), generates a stable AssetUUID for each asset, and produces two files:
    1. index.bin: A binary file containing a list of all AssetMetadata.
    2. data.pack: A binary file containing the raw, contiguous data of all assets for optimal I/O.
  • Goal: Maximum loading performance in release builds.

• For Development (Future Issue #175): The Real-Time Asset Database

  • A service integrated into the editor will watch the asset directories.
  • Changes will trigger an incremental update to an on-disk database, which will contain the same AssetMetadata but with AssetSource::Path variants.
  • Goal: A seamless and iterative development experience.

The Lifecycle of an Asset Request (Runtime Mode)

This diagram illustrates how the CLAD components we built interact to load an asset from packfiles:

sequenceDiagram
    participant GameLogic as "Game Logic"
    participant AssetAgent as "AssetAgent (Control Plane)"
    participant VFS as "VirtualFileSystem (Data)"
    participant PackLane as "PackLoadingLane (Data Plane)"
    participant Loader as "AssetLoader (e.g., PngLoader)"

    GameLogic->>+AssetAgent: load::<Texture>(texture_uuid)
    AssetAgent->>+VFS: get_metadata(&texture_uuid)
    VFS-->>-AssetAgent: Return AssetMetadata
    Note right of AssetAgent: Decision: Use "default" variant.<br/>Source: AssetSource::Packed { offset, size }
    AssetAgent->>+PackLane: load_asset_bytes(source)
    PackLane-->>-AssetAgent: Return Result<Vec<u8>>
    Note right of AssetAgent: Finds correct loader ("png")<br/>in its LoaderRegistry.
    AssetAgent->>+Loader: load(&bytes)
    Loader-->>-AssetAgent: Return Result<Texture>
    AssetAgent->>AssetAgent: Create AssetHandle<Texture>
    AssetAgent-->>-GameLogic: Return Result<AssetHandle<Texture>>

Conclusion: Why This Architecture is Essential for SAA

This VFS architecture is a critical foundation for the SAA philosophy. It elevates the AssetAgent from a simple file loader to a true Intelligent Subsystem Agent. By providing it with rich metadata via the VFS, we give the agent the context it needs to make meaningful, strategic decisions—such as choosing a lower-quality asset variant to stay within a VRAM budget—that align with the engine’s global performance goals.

08. Serialization Architecture: SAA-Serialize

Core Principle: Intent-Driven Persistence

In Khora Engine, scene serialization is not a simple, monolithic operation. It is a direct application of the Symbiotic Adaptive Architecture (SAA) philosophy, designed to be flexible, performant, and future-proof. We call this system SAA-Serialize.

The core principle is that there is no single “best” way to save and load a scene. The optimal method depends entirely on the developer’s goal. Saving a level for a shipping build requires maximum loading speed. Saving data for debugging requires a human-readable format. Saving a game state for a long-term project needs a format that will not be invalidated by future engine updates.

SAA-Serialize solves this by separating the intent of serialization from its implementation. The developer expresses their goal, and an intelligent agent selects the best strategy to achieve it.

A Note on RON (Rusty Object Notation)

Throughout this document, we refer to RON as the format of choice for human-readable output. RON is a data serialization format designed specifically for the Rust ecosystem. Think of it as a superset of Rust’s struct syntax, making it instantly familiar to any Rust developer.

Key features of RON include:

• Human-Readability: Its syntax is clean, allows for comments, and is generally less verbose than formats like XML.
• Expressiveness: It natively supports all of Rust’s data types, including enums and tuples, which are often awkward to represent in formats like JSON.
• Ecosystem: It is a well-supported serde format, making its integration into Khora seamless.

Whenever a SerializationGoal prioritizes debuggability or readability, RON will be the underlying text format used by the chosen strategy.

Advantages of the SAA-Serialize Approach

• Maximum Flexibility: The engine is never locked into a single format. It can use the best tool for every job, seamlessly switching between binary, text, stable, or raw-memory formats.
• Optimized Performance: For production builds, the system can use highly optimized strategies to achieve the fastest possible loading times, without compromising the stability of developer tools.
• Guaranteed Stability: For archival and team collaboration, the system can use a stable, versioned format completely decoupled from the engine’s internal memory layout, ensuring files remain valid across engine versions.
• Future-Proof Extensibility: The architecture is designed to be extended. New serialization techniques can be added to the engine as new Lanes without ever breaking the public API.

Built-in Serialization Strategies (Lanes)

Khora Engine ships with a core set of distinct serialization strategies. The SerializationAgent selects from these Lanes based on the SerializationGoal specified by the developer.

Note

Current Status: The SerializationAgent uses its own internal HashMap<String, Box<dyn SerializationStrategy>> registry rather than the standard LaneRegistry + LaneContext pattern. It does not yet implement the Agent trait and does not participate in GORNA negotiation. Migrating it to full CLAD compliance is planned.

1. The Definition Strategy (The Stable Lane)

• Primary Goal: SerializationGoal::LongTermStability, SerializationGoal::HumanReadableDebug (using the RON format).
• Principle: This strategy converts the live World into a simple, stable, intermediate representation—a SceneDefinition—before writing it to disk. This format describes the scene in logical terms (“this entity has a Transform component and a Material component”) rather than in terms of memory layout.
• Key Characteristics:
  • Pro: Extremely robust. Changes to component memory layout or ECS internals will not invalidate scene files. This is the safest choice for long-term asset stability.
  • Pro: Can be written in a human-readable format like RON, making it perfect for debugging and version control.
  • Con: This is the slowest strategy, as it requires an intermediate memory allocation for the SceneDefinition and a full data conversion.

2. The Recipe Strategy (The Dynamic Lane)

• Primary Goal: SerializationGoal::HumanReadableDebug, and serves as a foundation for editor tools.
• Principle: This strategy represents the scene not as a snapshot of data, but as a sequential list of commands—a “recipe”—that can be executed to reconstruct the world. The file contains instructions like SpawnEntity(ID), AddComponent(EntityID, ComponentData), etc.
• Key Characteristics:
  • Pro: A highly flexible format, ideal for implementing editor features like undo/redo, scene patching (prefabs), and potentially content streaming.
  • Pro: Can also be serialized to a text format, making the scene’s construction process transparent and easy to debug.
  • Con: Loading can be less performant than bulk data loading due to the overhead of executing a long series of individual commands.

3. The Archetype Strategy (The Performance Lane)

• Primary Goal: SerializationGoal::FastestLoad.
• Principle: A pure, data-oriented performance strategy. It bypasses all logical abstractions and serializes the CRPECS memory as directly as possible. It takes a snapshot of each archetype’s component arrays and writes them to disk as contiguous blocks of data.
• Key Characteristics:
  • Pro: By far the fastest possible loading method. Deserialization is nearly equivalent to a direct memory copy (memcpy), minimizing CPU work.
  • Con: Extremely fragile. The file is tightly coupled to the exact component memory layout, engine version, compiler, and target architecture. The slightest change will invalidate it. This is only suitable for final, “cooked” game builds for a specific platform.
  • Con: The format is an opaque binary blob, impossible to read or debug.

4. The Delta Strategy (The Efficiency Lane) — Planned

• Primary Goal: SerializationGoal::SmallestFileSize.
• Principle: This strategy only saves the differences (“deltas”) between the current World state and a reference World state (e.g., the base level scene).
• Key Characteristics:
  • Pro: Ideal for game saves or content patches where only a few elements have changed, resulting in very small files. This is also the foundational concept for network state replication.
  • Con: The algorithm to compute the difference between two scenes is complex and can be slow. Loading requires first loading the base scene and then applying the delta patch.

Developer Extensibility: Creating Custom Lanes

The SAA-Serialize system is designed to be open. The engine will export the public SerializationStrategy trait, allowing developers to implement and register their own custom Lanes with the SerializationAgent.

This allows a studio to:

• Integrate with proprietary, in-house file formats.
• Create Lanes that export to standard formats like JSON or XML for interoperability with external tools.
• Develop highly specialized binary formats tailored to their game’s unique needs or specific hardware.

This extensibility ensures that developers are never limited by the built-in strategies and can adapt Khora’s persistence system to fit any production pipeline.

11. Dynamic Context Core (DCC) Architecture

The Dynamic Context Core (DCC) is the strategic “brain” of the Symbiotic Adaptive Architecture (SAA). It is responsible for maintaining a holistic model of the application’s state, analyzing performance metrics, and orchestrating the Intelligent Subsystem Agents (ISAs) via the GORNA protocol.

Note

The DCC operates on the Control Plane (Cold Path). It runs asynchronously from the main render loop, typically at a lower frequency (e.g., 10-60 Hz), allowing it to perform complex analysis without stalling frame generation.

1. Core Responsibilities

The DCC has three primary directives:

1. Observe: Aggregate telemetry from all subsystems and hardware sensors to build a Context model.
2. Evaluate: Compare the current state against high-level goals (e.g., “Target 60 FPS”, “Conserve Battery”).
3. Orchestrate: Negotiate with Agents to adjust their resource budgets and strategies.

2. State Representation (The Context Model)

The DCC maintains a living model of the world, split into three categories:

2.1 Hardware State

Describes the physical constraints of the device.

• ThermalStatus: Cool | Warm | Throttling | Critical.
  • Source: OS sensors / Driver events.
  • Impact: If throttling, the DCC must aggressively reduce GPU/CPU budgets to prevent OS-level frequency clamping.
• BatteryLevel: Mains | High | Low | Critical.
  • Source: OS power events.
  • Impact: In Low battery, the DCC might cap the frame rate to 30 FPS or disable expensive post-processing.
• LoadAverage: Rolling average of CPU/GPU utilization.

2.2 Execution Phase (Engine Context)

Describes what the engine is currently doing, which dictates priority.

• Boot: The engine is initializing.
  • Goal: Maximize loading speed. Uncap CPU usage for compression/IO.
• Menu: The user is in a menu.
  • Goal: Responsiveness but efficiency. Cap FPS (e.g., 60), reduce GPU clocks if possible.
• Simulation: Active gameplay.
  • Goal: Stable frame times. strict adherence to budgets.
• Background: The window is minimized or lost focus.
  • Goal: Absolute minimum footprint. Cap FPS to 1-5, pause simulation lanes.

2.3 Performance Metrics

Aggregated data from khora-telemetry.

• Timing:
  • CpuFrameTime: Total time spent in the CPU logic loop.
  • GpuFrameTime: Total time spent by the GPU to render the frame.
  • FrameVariance: Standard deviation of the last 60 frames (stutter detection).
• Memory:
  • SystemRAM: Resident Set Size (RSS).
  • VRAM: Video memory usage.
  • PageFaults: Hard/Soft page faults (detecting thrashing).
• Rendering:
  • DrawCalls: Number of draw calls per frame.
  • TriangleCount: Total geometry complexity.
  • ShaderSwitches: Pipeline state changes (sort efficiency).
• Logic (ECS):
  • EntityCount: Total active entities in the world.
  • SystemTime: Time spent in specific hot systems (e.g., Physics).
• Streaming:
  • IoReadBytesPerSec: Disk bandwidth usage.
  • PendingAsyncLoads: Number of assets waiting to load (detecting bottlenecks).

3. Metric Aggregation Strategy

To make informed decisions, the DCC needs accurate data. However, collecting data must not impact the Hot Path.

3.1 Lock-Free Ingestion

Telemetry events are sent from the Hot Path (Lanes) to the DCC via crossbeam channels (Multi-Producer, Single-Consumer). This ensures that a rendering thread never locks or waits when reporting a metric.

3.2 The MetricStore

The DCC stores metrics in a strictly contiguous, cache-friendly structure.

• Data Structure: Vec<RingBuffer<f32, N>> indexed by MetricId.
• Ring Buffers: Fixed-size arrays (e.g., storing the last 60 samples). This avoids allocation during runtime.
• Analysis: The DCC analyzes these buffers to compute:
  • Trends: Is memory usage rising rapidly? (Slope analysis).
  • Stability: Is frame time variance high? (Standard deviation).

4. The Decision Loop (The Brain)

The DCC runs its own loop, decoupled from the frame rate.

graph TD
    subgraph Hot_Path [Hot Path Data Plane]
        Lanes[Lanes Rendering/Physics]
        Perif[Sensors OS/Hardware]
        App[Application Logic/ECS]
    end

    subgraph Telemetry_System [Telemetry System]
        Tel[khora-telemetry]
    end

    subgraph DCC [DCC Control Plane]
        Ingest[Metric Ingest]
        Store[(Metric Store)]
        State[Context State Model]
        Brain[GORNA Solver]
    end

    subgraph Agents [Agents ISAs]
        RenderAgent
        PhysicsAgent
    end

    %% Data Flow
    Lanes -- Metrics --> Tel
    Perif -- Status --> Tel
    
    Tel -- "Channel" --> Ingest
    Ingest --> Store
    Store --> State
    State --> Brain
    
    Brain -- "1. Negotiate" --> RenderAgent
    Brain -- "2. Apply Budget" --> RenderAgent
    
    App -- "3. Tactical Coordination" --> RenderAgent
    RenderAgent -- "Extract" --> App
    RenderAgent -.->|Configure| Lanes

Step 1: Ingest

Drain the telemetry channel. New events are pushed into their respective RingBuffer in the MetricStore.

Step 2: Analysis & Heuristics

Update the Context model.

• Example: If avg_frame_time > 16ms for the last 60 frames, set a PerformanceWarning flag.
• Example: If thermal_event received, update HardwareState.thermal to Throttling.

Step 3: GORNA Evaluation

Compare the current Context against the defined goals. The GORNA (Goal-Oriented Resource Negotiation & Allocation) protocol solves the resource distribution.

The HeuristicEngine runs 9 checks each tick:

Step 3.5: Initial GORNA Round

On the first tick after agents are registered, the DCC forces an initial GORNA negotiation round, regardless of whether any heuristic triggers. This ensures every agent receives a baseline ResourceBudget before telemetry-driven arbitration begins.

Without this initial round, agents would remain in their default state indefinitely on healthy systems where no heuristic threshold is ever crossed.

Step 4: Act (Arbitration & Application)

The DCC runs the GornaArbitrator to resolve resource conflicts and issues budgets to the agents.

• Negotiate: The DCC asks agents for strategy options based on the current target_latency.
• Apply Budget: The DCC selects the optimal strategy and issues a ResourceBudget to each agent.

Step 5: Tactical Coordination (Agent::update)

Beyond strategic budgeting, agents perform high-frequency tactical work. This is triggered by the main engine loop (e.g., in RedrawRequested) via the DccService::update_agents method.

• Mechanism: The engine provides an EngineContext containing type-erased (Any) pointers to the ECS World and Assets.
• Action: Agents (like the RenderAgent) downcast these pointers to extract data, prepare GPU meshes, and coordinate multi-threaded data flows between the Data Plane and the Data Lane.
• Telemetry Emission: After tactical work, agents with a wired Sender<TelemetryEvent> emit performance reports (e.g., GpuReport with draw calls, triangle count) back to the DCC channel.

5. SDK Integration & Phase Lifecycle

The DccService is fully integrated into khora-sdk. End users interact with it implicitly through the Application trait — no manual DCC configuration is required.

5.1 Automatic Wiring (in Engine::run)

sequenceDiagram
    participant SDK as khora-sdk (Engine::run)
    participant DCC as DccService (Background Thread)
    participant RA as RenderAgent (ISA)
    participant TEL as TelemetryService

    SDK->>DCC: new(DccConfig::default())
    SDK->>TEL: new().with_dcc_sender(dcc.event_sender())
    SDK->>RA: RenderAgent::new().with_telemetry_sender(dcc.event_sender())
    SDK->>DCC: register_agent(render_agent)
    SDK->>DCC: start(rx)
    SDK->>DCC: PhaseChange("boot")
    
    Note over DCC: DCC thread starts at 20Hz
    Note over DCC: First tick → initial GORNA negotiation
    DCC->>RA: negotiate(NegotiationRequest)
    RA-->>DCC: NegotiationResponse (strategies)
    DCC->>RA: apply_budget(ResourceBudget)

5.2 Execution Phase Lifecycle

The SDK automatically emits PhaseChange events to the DCC at key lifecycle transitions:

5.3 Per-Frame Tactical Update

On every RedrawRequested, the SDK calls dcc.update_agents(&mut context) using its internally owned GameWorld:

1. The GameWorld (owned by the SDK’s EngineState) provides its World and Assets<Mesh> through a pub(crate) method.
2. These are type-erased into EngineContext and passed to each agent’s update().
3. The RenderAgent downcasts the context, runs MeshPreparationSystem and ExtractRenderablesLane, then emits a GpuReport telemetry event.
4. The user’s Application::update(&mut self, world: &mut GameWorld) is called after agent updates, giving the game logic a typed, controlled view of the ECS without ever touching World directly.

5.4 Communication Channels

• Inbound (Hot Path → DCC): Receiver<TelemetryEvent> from khora-telemetry monitors and from agents’ telemetry senders.
• Outbound (DCC → Agents): Direct Arc<Mutex<dyn Agent>> method calls (negotiate, apply_budget) on the background thread.

12 - GORNA Protocol Specification

The Goal-Oriented Resource Negotiation & Allocation (GORNA) protocol is the core mechanism by which KhoraEngine achieves engine-wide adaptivity. It moves away from hard-coded resource usage towards a marketplace-like negotiation between the engine core and its subsystems.

1. The Core Philosophy

GORNA operates on three fundamental principles:

1. Utility-Based: Subsystems don’t just “request memory”; they offer “quality levels” (Strategies) and their associated costs.
2. Global Optimization: Only the DCC has the global view (Thermal, Battery, Phase) to decide which subsystem gets priority.
3. Soft-Real-Time Contracts: A “Budget” is a contract. If an Agent accepts a 16ms budget, it must guarantee execution within that time.

Architectural Overview

graph TB
    subgraph "DCC - Strategic Brain (Control Plane)"
        DCC[Dynamic Context Core]
        GRN[GORNA Arbitrator]
        HS[Heuristics Engine]
        DCC -->|Situational Model| HS
        HS -->|Alert/Opportunity| GRN
    end

    subgraph "ISAs - Tactical Managers (Control Plane)"
        RA[Renderer Agent]
        PA[Physics Agent]
        AA[Asset Agent]
    end

    subgraph "Lanes - Execution (Data Plane)"
        RL[Renderer Lanes]
        PL[Physics Lanes]
        AL[Asset Lanes]
    end

    GRN <-->|Negotiation Protocol| RA
    GRN <-->|Negotiation Protocol| PA
    GRN <-->|Negotiation Protocol| AA

    RA -->|Apply Budget| RL
    PA -->|Apply Budget| PL
    AA -->|Apply Budget| AL

    RL -.->|Telemetry| DCC
    PL -.->|Telemetry| DCC
    AL -.->|Telemetry| DCC

2. Protocol Phases

The GORNA loop is a continuous cycle of sensing and adapting.

stateDiagram-v2
    [*] --> InitialRound
    InitialRound --> Awareness: Baseline Budgets Issued
    Awareness --> Analysis: Telemetry Ingested
    Analysis --> Negotiation: Threshold Tripped
    Negotiation --> Arbitration: Options Submitted
    Arbitration --> Application: Budget Issued
    Application --> Awareness: Context Updated

Phase 0: Initial Round (Boot)

On the first DCC tick after agents are registered, the DCC forces a full GORNA negotiation round regardless of heuristic state. This ensures every agent receives a baseline ResourceBudget before telemetry-driven arbitration takes over.

Without this initial round, agents would remain throttled at their defaults indefinitely on healthy systems where no performance threshold is ever crossed.

Phase A: Awareness (Telemetry)

Hot Paths (Lanes) emit TelemetryEvents (frametime, VRAM usage, draw calls). These are ingested by the DccService. Additionally, agents with a wired Sender<TelemetryEvent> (e.g., the RenderAgent) push GpuReport events directly into the DCC channel after their tactical update() work.

Phase B: Analysis (DCC Heuristics)

The DCC analyzes trends.

• Ex: “GPU usage is at 98% and the device is warning of thermal pressure.”
• Result: The DCC triggers a “Negotiation Event”.

Phase C: Negotiation (The Handshake)

The DCC sends a NegotiationRequest to all registered Agents.

sequenceDiagram
    participant DCC as DCC / GORNA
    participant AG as Agent (ISA)
    participant LN as Lanes (Hot Path)

    Note right of DCC: A: Awareness & Analysis
    LN-->>DCC: TelemetryEvent (Performance Drop)

    Note right of DCC: B: Negotiation
    DCC->>AG: negotiate(NegotiationRequest)
    AG-->>DCC: NegotiationResponse (Strategies: [High, Mid, Low])

    Note right of DCC: C: Arbitration
    Note over DCC: Logic: Balanced fit for 16.6ms
    DCC->>AG: apply_budget(ResourceBudget: Mid)

    Note right of AG: D: Application
    AG->>LN: reconfigure_strategy("Mid")

The Agent responds with a NegotiationResponse containing a list of StrategyOptions:

• Strategy A (Ultra): 14ms, 4GB VRAM.
• Strategy B (Balanced): 8ms, 2GB VRAM.
• Strategy C (Low): 4ms, 1GB VRAM.

Phase D: Arbitration (GORNA Logic)

The DCC runs the Arbitrator.

1. Sum Costs: If all “Ultra” strategies exceed the 16.6ms frame budget, the DCC must downgrade some agents.
2. Prioritize: Based on Phase (e.g., in Simulation, Physics gets its requested budget, Renderer is downgraded to “Balanced”).
3. Issue Budgets: The DCC calls apply_budget(ResourceBudget) on each Agent.

Phase E: Application (Strategy Switch)

The Agent reconfigures its Lanes. This might involve:

• Switching a shader variant.
• Changing LOD thresholds.
• Reducing the frequency of a simulation step.

3. Data Structures (Rust Pseudocode)

#![allow(unused)]
fn main() {
pub struct ResourceBudget {
    pub agent_id: AgentId,
    pub strategy_id: StrategyId,
    pub time_limit: Duration,
    pub memory_limit: Option<u64>,
}

pub struct AgentStatus {
    pub agent_id: AgentId,
    pub current_strategy: StrategyId,
    pub health_score: f32, // 0.0 - 1.0 (How well it adheres to budget)
    pub is_stalled: bool,
    pub message: String,
}
}

4. Advanced Heuristics & Conflict Resolution

The “Death Spiral” Prevention

If multiple ISAs fail to meet their budgets, GORNA Enforces an “Emergency Stop” strategy across all agents to prevent engine-wide hangs.

Thermal Pullback

As a device heats up, the DCC gradually reduces the GlobalBudgetMultiplier.

• Cooling: 1.0x
• Warm: 0.9x
• Throttling: 0.6x (Strategic degradation of all non-critical ISAs).

5. Implementation Roadmap

• v0.1: Static priority arbitration (Cold Path only).
• v0.2: Dynamic weight calculation based on ExecutionPhase, situational hierarchy, and Hardware awareness (Thermal/CPU).
• v0.3: Cost-based negotiation with lane.estimate_cost(), VRAM-aware filtering, health score reporting, and GpuReport telemetry integration.
• v0.4: Migrate AudioAgent, SerializationAgent, and AssetAgent to full GORNA compliance (Agent trait + LaneRegistry + LaneContext).
• v1.0: Predictive arbitration using MetricStore trends and multi-agent resource bargaining.

Current GORNA Compliance

09 - Roadmap & Issue Tracker

This document outlines the phased development plan for Khora. It integrates all open and proposed tasks into a structured series of milestones.

Phase 1: Foundational Architecture

Goal: Establish the complete, decoupled CLAD crate structure and render a basic scene through the SDK. (Complete)

With the successful abstraction of command recording and submission, the core architectural goals for the foundational phase are now met. The engine is fully decoupled from the rendering backend.

Phase 2: Scene, Assets & Basic Capabilities

Goal: Build out the necessary features to represent and interact with a game world, starting with the implementation of our revolutionary ECS.

[Rendering Capabilities, Physics, Animation, AI & Strategy Exploration]

• #101 [Feature] Implement Skeletal Animation System
• #162 [Feature] Implement SkinnedMesh ComputeLane
• #104 [Feature] Implement Basic AI System (Placeholder Behaviors, e.g., Simple State Machine)

Phase 3: The Adaptive Core

Goal: Implement the “magic” of Khora: the DCC, ISAs, and GORNA, proving the SAA concept.

[Intelligent Subsystem Agents (ISA) v1 & Basic Adaptation]

• #176 [Feature] Evolve AssetAgent into a full ISA (Depends on #174)
• #83 [Task] Refactor a Second Subsystem as ISA v0.1

Phase 4: Tooling, Usability & Scripting

Goal: Make the engine usable and debuggable by humans. Build the editor, provide observability tools, and integrate a scripting language.

[Editor GUI, Observability & UI]

• #52 [Feature] Choose & Integrate GUI Library
• #53 [Feature] Create Editor Layout
• #54 [Feature] Implement Render Viewport
• #55 [Feature] Implement Scene Hierarchy Panel
• #56 [Feature] Implement Inspector Panel (Basic Components)
• #57 [Feature] Implement Performance/Context Visualization Panel
• #58 [Feature] Implement Basic Play/Stop Mode
• #77 [Feature] Visualize Full Context Model in Editor Debug Panel
• #102 [Feature] Implement In-Engine UI System
• #164 [Feature] Implement UiRenderLane
• #165 [Feature] Implement a “Decision Tracer” for DCC/GORNA in the editor
• #166 [Feature] Implement a Timeline Scrubber for the Context Visualization Panel

[Editor Polish, Networking & Manual Control]

• #175 [Feature] Implement Real-time Asset Database for Editor (Depends on #41)
• #177 [Feature] Implement DeltaSerializationLane for Game Saves & Undo/Redo (Depends on #45)
• #66 [Feature] Implement Asset Browser (Depends on #175)
• #67 [Feature] Implement Material Editor
• #68 [Feature] Implement Gizmos
• #167 [Feature] Implement EditorGizmo RenderLane
• #69 [Feature] Implement Undo/Redo Functionality
• #70 [Feature] Implement Editor Panels for Fine-Grained System Control
• #103 [Feature] Implement Basic Networking System

[Scripting v1]

• #168 [Research] Evaluate and choose a scripting language
• #169 [Feature] Implement Scripting Backend and Bindings
• #170 [Feature] Make the Scripting VM an ISA (ScriptingAgent)

[Maturation, Optimization & Packaging]

• #94 [Task] Extensive Performance Profiling & Optimization
• #95 [Task] Documentation Overhaul (Including SAA concepts)
• #96 [Task] Build & Packaging for Target Platforms

[Milestone: API Ergonomics & DX (Developer Experience)]

• #173 [Feature] Implement a Fluent API for Entity Creation

Phase 5: Advanced Intelligence & Future Capabilities

Goal: Build upon the stable SAA foundation to explore next-generation features and specializations.

[Advanced Adaptivity & Specialization (AGDF, Contracts)]

• #89 [Research] Design Semantic Interfaces & Contracts v1
• #90 [Research] Investigate Adaptive Game Data Flow (AGDF) Feasibility & Design
• #91 [Prototype] Implement basic AGDF for a specific component type
• #92 [Research] Explore using Specialized Hardware (ML cores?)
• #129 [Feature] Metrics System - Advanced Features (Labels, Histograms, Export)

[DCC v2 - Developer Guidance & Control]

• #93 [Feature] Implement more Sophisticated DCC Heuristics / potentially ML-based Decision Model
• #171 [Feature] Implement Engine Adaptation Modes (Learning, Stable, Manual)
• #172 [Feature] Implement Developer Hints and Constraints System (PriorityVolume)

[Core XR Integration & Context]

• #59 [Feature] Integrate OpenXR SDK & Bindings
• #60 [Feature] Implement XR Instance/Session/Space Management
• #61 [Feature] Integrate Graphics API with XR
• #62 [Feature] Implement Stereo Rendering Path
• #63 [Feature] Implement Head/Controller Tracking
• #64 [Feature] Integrate XR Performance Metrics
• #65 [Task] Display Basic Scene in VR with Performance Overlay

Phase 6: Next-Generation Custom Physics

Goal: Replace the 3rd-party solver with a native Khora solver implementing cutting-edge physical simulation research.

[Pillar 1: Unified Simulation & Material Point Method]

• #300 [Research] Unified Simulation (MLS-MPM)

  Tip

  Implement “MLS-MPM: Moving Least Squares Material Point Method” for unified simulation of snow, sand, and fluids. Target: Pure algorithmic interaction between disparate materials.

• #301 [Research] Sparse Volume Physics (NanoVDB)

  Note

  Integrate NanoVDB (OpenVDB) for GPU-accelerated sparse volume simulation (fire, smoke, large-scale explosions).

[Pillar 2: Robust Constraints & Collision (The End of Clipping)]

• #302 [Research] Incremental Potential Contact (IPC)

  Important

  Integrate “Incremental Potential Contact (Li et al. 2020)” to guarantee intersection-free and inversion-free simulation. Focus: Eliminating clipping in soft-bodies and high-speed collisions.

• #303 [Research] Stable Constraints (XPBD & ADMM)

  Note

  Combine XPBD for stability with ADMM optimization for complex, hard constraints and heterogeneous materials.

[Pillar 3: Soft-Body & Gaussian Dynamics]

• #304 [Research] High-Speed Soft Bodies (Projective Dynamics)

  Note

  Study Projective Dynamics for real-time muscle and “flesh” simulation with implicit stability.

• #305 [Research] Differentiable & Gaussian Physics

  Note

  Explore PhysGaussian and DiffTaichi for physics-integrated Gaussian splatting and differentiable simulation.

[Pillar 4: Intelligent Characters & Neural Simulation]

• #306 [Research] Learning-Based Character Motion (DeepMimic)

  Tip

  Research DeepMimic (Arxiv) for physics-based character animation using Reinforcement Learning.

• #307 [Research] Graph Network Simulation (DeepMind)

  Note

  Analysis of “Learning to Simulate Complex Physics with Graph Networks” for complex particle-based interactions.

[Implementation & Transition]

• #308 [Feature] Implement Custom Khora-Solver v1 (Rigid Body + XPBD Core)
• #309 [Feature] Transition PhysicsAgent & Lanes to Native Solver
• #310 [Task] Performance Match & Exceed against previous 3rd-party backend

Closed Issues (Historical Reference)

[Core Foundation & Basic Window]

• #1 [Feature] Setup Project Structure & Cargo Workspace
• #2 [Feature] Implement Core Math Library (Vec3, Mat4, Quat) - Design for DOD/Potential ADF
• #3 [Feature] Choose and Integrate Windowing Library
• #4 [Feature] Implement Basic Input System (Feed events into core)
• #5 [Feature] Create Main Application Loop Structure
• #6 [Task] Display Empty Window & Basic Stats (FPS, Mem)
• #7 [Task] Setup Basic Logging & Event System
• #8 [Task] Define Project Coding Standards & Formatting
• #18 [Feature] Design Core Engine Interfaces & Message Passing (Thinking about ISAs & DCC)
• #19 [Feature] Implement Foundational Performance Monitoring Hooks (CPU Timers)
• #20 [Feature] Implement Basic Memory Allocation Tracking

[Core Foundation & Context Hooks]

• #21 [Feature] Setup Project Structure & Cargo Workspace
• #22 [Feature] Implement Core Math Library (Vec3, Mat4, Quat) - Design for DOD/Potential AGDF
• #23 [Feature] Design Core Engine Interfaces & Message Passing (Thinking about ISAs & DCC)
• #24 [Feature] Implement Basic Logging & Event System
• #25 [Feature] Implement Foundational Performance Monitoring Hooks (CPU Timers)
• #26 [Feature] Implement Basic Memory Allocation Tracking
• #27 [Feature] Choose and Integrate Windowing Library
• #28 [Feature] Implement Basic Input System (Feed events into core)
• #29 [Feature] Create Main Loop Structure (placeholder for future DCC control)
• #30 [Task] Display Empty Window & Basic Stats (FPS, Mem)

[Rendering Primitives & ISA Scaffolding]

• #31 [Feature] Choose & Integrate Graphics API Wrapper
• #32 [Feature] Design Rendering Interface as potential ISA (Clear inputs, outputs, potential strategies)
• #33 [Feature] Implement Graphics Device Abstraction
• #34 [Feature] Implement Swapchain Management
• #35 [Feature] Implement Basic Shader System
• #36 [Feature] Implement Basic Buffer/Texture Management (Track VRAM usage)
• #37 [Feature] Implement GPU Performance Monitoring Hooks (Timestamps)
• #110 [Feature] Implement Robust Graphics Backend Selection (Vulkan/DX12/GL Fallback)
• #118 [Feature] Implement Basic Rendering Pipeline System
• #121 [Feature] Develop Custom Bitflags Macro for Internal Engine Use
• #123 [Feature] Implement Core Metrics System Backend v1 (In-Memory)
• #124 [Task] Integrate VRAM Tracking into Core Metrics System
• #125 [Task] Integrate System RAM Tracking into Core Metrics System
• #38 [Task] Render a Single Triangle/Quad with Performance Timings
• #135 [Enhancement] Advanced GPU Performance & Resize Heuristics
• #140 [Feature] Implement Basic Command Recording & Submission

[Scene Representation, Assets & Data Focus]

• #39 [Research & Design] Define Khora’s ECS Architecture
• #154 [Task] Implement Core ECS Data Structures (CRPECS v1)
• #155 [Task] Implement Basic Entity Lifecycle (CRPECS v1)
• #156 [Task] Implement Native Queries (CRPECS v1)
• #40 [Feature] Implement Scene Hierarchy & Transform System (Depends on #156)
• #41 [Design] Design Asset System with VFS & Define Core Structs
• #174 [Feature] Implement VFS Packfile Builder & Runtime (Depends on #41)
• #42 [Feature] Implement Texture Loading & Management (Depends on #174)
• #43 [Feature] Implement Mesh Loading & Management (Depends on #174)
• #44 [Task] Render Loaded Static Model with Basic Materials (Depends on #40, #42, #43)
• #157 [Task] Implement Component Removal & Basic Garbage Collection (CRPECS v1)
• #45 [Feature] Implement Basic Scene Serialization
• #99 [Feature] Implement Basic Audio System (Playback & Management)

[Rendering Capabilities, Physics, Animation, AI & Strategy Exploration]

• #159 [Feature] Implement SimpleUnlit RenderLane
• #46 [Feature] Implement Camera System & Uniforms
• #47 [Feature] Implement Material System
• #48 [Feature] Implement Basic Lighting Models (Track shader complexity/perf)
• #160 [Feature] Implement Forward+ Lighting RenderLane
• #49 [Feature] Implement Depth Buffering
• #50 [Research] Explore Alternative Rendering Paths/Strategies (e.g., Forward vs Deferred concept)
• #158 [Feature] Implement Transversal Queries (CRPECS v1)
• #100 [Feature] Implement Basic Physics System (Integration & Collision Detection) (Depends on #40)
• #161 [Feature] Define and Implement Core PhysicsLanes (Broadphase, Solver)

[Intelligent Subsystem Agents (ISA) v1 & Basic Adaptation]

• #75 [Feature] Design Initial ISA Interface Contract v0.1
• #76 [Task] Refactor one Subsystem to partially implement ISA v0.1 (RenderAgent Base)
• #78 [Feature] Implement Multiple Strategies for one key ISA (RenderAgent: Unlit, LitForward, ForwardPlus, Auto)
• #79 [Feature] Refine ISA Interface Contract (Agent trait: negotiate, apply_budget, report_status)
• #80 [Feature] Implement DCC Heuristics Engine v1 (9 heuristics in khora-control)
• #81 [Feature] Implement DCC Command System to trigger ISA Strategy Switches (GornaArbitrator → apply_budget flow)
• #82 [Task] Demonstrate Automatic Renderer Strategy Switching (Auto mode + GORNA negotiation, 16 tests)
• #224 [Feature] Implement RenderLane Resource Ownership (Pipelines, buffers, bind groups; proper on_shutdown)
• #225 [Feature] Implement Light Uniform Buffer System (UniformRingBuffer in khora-core, persistent GPU ring buffers for camera/lighting uniforms)

[Goal-Oriented Resource Negotiation (GORNA) v1]

• #84 [Research] Design GORNA Protocol
• #85 [Feature] Implement Resource Budgeting in DCC
• #86 [Feature] Enhance ISAs to Estimate Resource Needs per Strategy (estimate_cost + VRAM-aware negotiate)
• #88 [Task] Demonstrate Dynamic Resource Re-allocation under Load

[Dynamic Context Core (DCC) v1 - Awareness]

• #71 [Feature] Design DCC Architecture
• #72 [Feature] Implement DCC Core Service
• #73 [Feature] Integrate Performance/Resource Metrics Collection into DCC
• #74 [Feature] Implement Game State Monitoring Hook into DCC
• #128 [Feature] DCC v1 Integration with Core Metrics System (MetricStore, RingBuffer, GpuReport ingestion)
• #163 [Feature] Make CRPECS Garbage Collector an ISA
• #116 [Research/Refactor] Evaluate Abstraction for Windowing/Platform System

10. Rendering Architecture

This document describes the Rendering ISA architecture: how the RenderAgent manages polymorphic rendering strategies through the unified Lane trait, participates in GORNA negotiation, performs shadow mapping, and adapts at runtime.

1. Polymorphic Rendering in SAA

In a conventional engine the rendering pipeline (Forward, Deferred, etc.) is a project-wide configuration. Khora’s Symbiotic Adaptive Architecture (SAA) treats rendering as a polymorphic service: multiple rendering paths coexist in memory and the engine switches between them transparently, driven by the GORNA protocol.

This approach delivers:

• Zero-downtime switching — GPU resources for all lanes stay alive; only the active strategy pointer changes.
• Budget-aware rendering — the DCC can force the agent onto a cheaper lane when the frame time budget is tight.
• Shadow integration — shadow lanes run before any render lane, injecting depth-atlas data into the shared LaneContext.

2. The Unified Lane Trait

All lanes in KhoraEngine — rendering, shadow, physics, audio, etc. — implement the same Lane trait defined in khora-core::lane. There is no domain-specific trait such as RenderLane or ShadowLane; the unified interface is sufficient:

#![allow(unused)]
fn main() {
pub trait Lane: Send + Sync {
    fn strategy_name(&self) -> &'static str;
    fn lane_kind(&self) -> LaneKind;
    fn estimate_cost(&self, ctx: &LaneContext) -> f32;
    fn on_initialize(&self, ctx: &mut LaneContext) -> Result<(), LaneError>;
    fn execute(&self, ctx: &mut LaneContext) -> Result<(), LaneError>;
    fn on_shutdown(&self, ctx: &mut LaneContext);
    fn as_any(&self) -> &dyn Any;
    fn as_any_mut(&mut self) -> &mut dyn Any;
}
}

Important

on_shutdown() must release every GPU resource the lane owns. ForwardPlusLane, for example, destroys its render pipeline, light buffer, light-index buffer, light-grid buffer, and culling-uniforms buffer. ShadowPassLane destroys its pipeline, bind-group layouts, atlas texture, and atlas view.

LaneKind Classification

#![allow(unused)]
fn main() {
pub enum LaneKind {
    Render,   // Main scene rendering (forward, deferred, …)
    Shadow,   // Shadow map generation
    Physics,  // Physics simulation
    Audio,    // Audio mixing and spatialization
    Asset,    // Asset loading and processing
    Scene,    // Serialization / deserialization
    Ecs,      // ECS maintenance (compaction, GC)
}
}

The agent uses LaneKind to route execution: shadow lanes run before render lanes, physics lanes run in the physics tick, etc.

3. Available Lanes

3.1 Render Lanes (LaneKind::Render)

3.2 Shadow Lanes (LaneKind::Shadow)

3.3 Future Concepts

4. LaneContext: The Data Bridge

Agents and lanes communicate through LaneContext, a HashMap<TypeId, Box<dyn Any>> type-map. The agent populates the context before dispatching lanes; each lane reads what it needs and writes results for downstream lanes.

4.1 Standard Context Keys

These typed keys live in khora-core::lane::context_keys:

4.2 Slot / Ref Wrappers

For data that the agent borrows (not owns), Slot<T> (mutable) and Ref<T> (shared) erase the borrow lifetime so the value can be stored in the type-map:

#![allow(unused)]
fn main() {
// Agent side
let mut ctx = LaneContext::new();
ctx.insert(Slot::new(encoder));     // mutable borrow
ctx.insert(Slot::new(render_world));

// Lane side
let encoder = ctx.get::<Slot<dyn CommandEncoder>>()
    .ok_or(LaneError::missing("Slot<dyn CommandEncoder>"))?
    .get();  // → &mut dyn CommandEncoder
}

Safety is guaranteed by the stack-scoped context pattern: the LaneContext is created by the agent, passed to one lane at a time, and dropped before the next frame.

5. The RenderAgent ISA

The RenderAgent (khora-agents::render_agent) is an Intelligent Subsystem Agent implementing the Agent trait. It sits on the Control Plane (Cold Path) and drives both shadow and render Lanes on the Data Plane (Hot Path).

5.1 Internal State

#![allow(unused)]
fn main() {
pub struct RenderAgent {
    render_world: RenderWorld,
    gpu_meshes: Arc<RwLock<Assets<GpuMesh>>>,
    mesh_preparation_system: MeshPreparationSystem,
    extract_lane: ExtractRenderablesLane,
    lanes: LaneRegistry,             // All lanes (render + shadow) in one registry
    strategy: RenderingStrategy,      // Current active strategy enum
    current_strategy: StrategyId,     // GORNA strategy ID
    device: Option<Arc<dyn GraphicsDevice>>,
    render_system: Option<Arc<Mutex<Box<dyn RenderSystem>>>>,
    telemetry_sender: Option<Sender<TelemetryEvent>>,
    // --- Performance Metrics ---
    last_frame_time: Duration,
    time_budget: Duration,            // Assigned by GORNA
    draw_call_count: u32,
    triangle_count: u32,
    frame_count: u64,
}
}

On construction, RenderAgent::new() registers four default lanes:

#![allow(unused)]
fn main() {
let mut lanes = LaneRegistry::new();
lanes.register(Box::new(SimpleUnlitLane::new()));
lanes.register(Box::new(LitForwardLane::new()));
lanes.register(Box::new(ForwardPlusLane::new()));
lanes.register(Box::new(ShadowPassLane::new()));
}

5.2 Strategy Selection

The RenderingStrategy enum governs which render lane is active:

In Auto mode the agent evaluates the scene every frame, choosing the most appropriate lane without GORNA intervention. GORNA can override this by issuing a specific strategy via apply_budget().

6. GORNA Protocol Integration

6.1 Phase C: Negotiation — negotiate()

When the DCC triggers a GORNA round, it calls negotiate(NegotiationRequest) on the agent.

Algorithm:

1. Build a minimal LaneContext with Slot<RenderWorld> and Arc<RwLock<Assets<GpuMesh>>>.
2. For each LaneKind::Render lane, call lane.estimate_cost(&ctx) to get a real cost factor.
3. Convert cost to estimated GPU time: estimated_time = cost × COST_TO_MS_SCALE / 1000 (clamped to ≥ 0.1 ms).
4. Compute VRAM estimates:
   • Base: mesh_count × 100 KB (vertex + index buffers).
   • LitForward overhead: +512 B/mesh + 4 KB (uniform buffers).
   • ForwardPlus overhead: +512 B/mesh + 4 KB + 8 MB (compute culling buffers).
5. Filter out strategies exceeding request.constraints.max_vram_bytes.
6. Always guarantee at least one LowPower fallback (1 ms, base VRAM).

Example response (3 meshes, no VRAM constraint):

6.2 Phase E: Application — apply_budget()

The DCC selects a strategy and issues a ResourceBudget. The agent:

1. Maps StrategyId → RenderingStrategy:
   • LowPower → Auto — lets the agent pick SimpleUnlit when the scene has no lights, but automatically escalate to LitForward when lights are present so shadows and lighting work correctly.
   • Balanced → LitForward
   • HighPerformance → ForwardPlus
2. Stores budget.time_limit for health reporting.
3. Does not destroy or recreate any lane — only the active strategy enum changes.

Note

Custom strategy IDs fall back to Balanced with a warning log.

6.3 Health Reporting — report_status()

The agent reports health to the DCC every tick:

#![allow(unused)]
fn main() {
AgentStatus {
    agent_id: AgentId::Renderer,
    health_score,       // min(1.0, time_budget / last_frame_time)
    current_strategy,
    is_stalled,         // true if frame_count == 0 && device is initialized
    message,            // "frame_time=X.XXms draws=N tris=N lights=N"
}
}

• health_score = 1.0 when at or under budget, < 1.0 when over budget.
• is_stalled = true signals a potential initialization failure (device present but no frames rendered).

6.4 Telemetry

The agent emits TelemetryEvent::GpuReport each frame:

The DCC’s MetricStore ingests these into ring buffers for trend analysis and heuristic evaluation.

7. Frame Lifecycle

sequenceDiagram
    participant EL as Engine Loop
    participant RA as RenderAgent
    participant EX as ExtractLane
    participant MPS as MeshPrepSystem
    participant SH as ShadowPassLane
    participant RL as Active Render Lane
    participant GPU as GPU

    EL->>RA: update(EngineContext)
    RA->>EX: extract(World) → RenderWorld
    RA->>MPS: prepare_meshes(World, Device)

    Note over RA: Build LaneContext (device, meshes, encoder, targets)
    RA->>SH: execute(&mut LaneContext)
    SH->>GPU: depth-only draw calls → shadow atlas
    SH-->>RA: ctx now contains ShadowAtlasView + ShadowComparisonSampler

    RA->>RL: execute(&mut LaneContext)
    RL->>GPU: scene draw calls (with shadow sampling)
    RA->>RA: record metrics

7.1 Tactical Update (Agent::update)

Called by the engine loop via DccService::update_agents:

1. Cache device: On first call, obtain Arc<dyn GraphicsDevice> from the ServiceRegistry, then initialize all lanes via lane.on_initialize(&mut ctx).
2. Extract scene: Downcast EngineContext to World, run MeshPreparationSystem (CPU → GPU mesh upload), run ExtractRenderablesLane (ECS → RenderWorld).
3. Extract camera: Push the camera view to the cached RenderSystem.

7.2 Rendering (render_with_encoder closure)

Inside the render system’s encoder scope:

1. Build LaneContext: Insert Arc<dyn GraphicsDevice>, Arc<RwLock<Assets<GpuMesh>>>, Slot<dyn CommandEncoder>, Slot<RenderWorld>, ColorTarget, DepthTarget, ClearColor.
2. Shadow pass: Execute all LaneKind::Shadow lanes. The ShadowPassLane:
   • Renders depth-only draw calls into the 2048×2048 4-layer atlas.
   • Patches each ExtractedLight with shadow_view_proj and shadow_atlas_index.
   • Inserts ShadowAtlasView and ShadowComparisonSampler into the context.
3. Render pass: Execute the selected LaneKind::Render lane (via lanes.get(selected_name)). The lit lanes read shadow data from the context and build a 3-entry bind group (uniform buffer + shadow atlas + comparison sampler).
4. Metrics: Record last_frame_time, draw_call_count, triangle_count; increment frame_count.

8. Shadow Mapping Pipeline

8.1 Architecture

Shadow mapping is a first-class, integrated subsystem — not an optional post-process. The ShadowPassLane runs before any render lane in every frame where lights are present.

Shadow Atlas: A Depth32Float 2D-array texture (2048 × 2048 × 4 layers). Each shadow-casting light is assigned one layer.

8.2 ShadowPassLane::execute() — Three Phases

8.3 Shadow Sampling (WGSL)

The lit_forward.wgsl shader performs 3×3 PCF (Percentage-Closer Filtering):

fn sample_shadow_pcf(light_space_pos: vec4<f32>, atlas_index: i32) -> f32 {
    let proj = light_space_pos.xyz / light_space_pos.w;
    let uv   = proj.xy * vec2(0.5, -0.5) + 0.5;
    let texel = 1.0 / 2048.0;
    var shadow = 0.0;
    for (var y = -1; y <= 1; y++) {
        for (var x = -1; x <= 1; x++) {
            shadow += textureSampleCompareLevel(
                shadow_atlas, shadow_sampler,
                uv + vec2(f32(x), f32(y)) * texel,
                atlas_index, proj.z
            );
        }
    }
    return shadow / 9.0;
}

8.4 Resource Lifecycle

9. Integration with CLAD

The rendering subsystem is the canonical embodiment of the CLAD Pattern:

10. Implementation Status

Completed

• Four coexisting lanes: SimpleUnlitLane, LitForwardLane, ForwardPlusLane, ShadowPassLane.
• Unified Lane trait — no domain-specific render/shadow traits.
• LaneRegistry for generic lane storage and lookup by name/kind.
• LaneContext type-map for zero-coupling data passing between agent and lanes.
• Slot<T> / Ref<T> wrappers for safe borrow-through-context patterns.
• Full GORNA protocol: negotiate(), apply_budget(), report_status().
• Cost-based negotiation via lane.estimate_cost(&LaneContext).
• VRAM-aware strategy filtering.
• Shadow atlas (2048×2048, 4 layers, Depth32Float) with 3×3 PCF sampling.
• Shadow data flow: atlas → LaneContext → bind group in render lanes.
• Per-frame performance metrics tracking.
• Health score computation (budget vs. actual frame time).
• Stall detection for the DCC watchdog.
• GpuReport telemetry event integration with MetricStore.
• Tiled Forward+ compute culling and fragment shaders.
• 17 GORNA integration tests (negotiate, apply_budget, report_status, telemetry, full cycle).

Known Limitations & Future Work

• Auto-mode uses a simple light-count threshold; could leverage DCC heuristics instead.
• Shadow atlas is hardcoded to 4 layers — should scale based on shadow-casting light count.
• extract_lane material query uses .nth(entity_id) which may not match actual material IDs.
• LitForwardLane allocates uniform buffers per-frame (should use a persistent ring buffer).
• Vertex layout assumptions differ between lanes (not currently validated).
• MaterialUniforms struct is hardcoded; should derive from material properties dynamically.

11. Future Research Areas

• NPR (Non-Photorealistic Rendering) Lanes: Cell-shading or oil-painting as selectable strategies.
• Hardware-Specific Lanes: Paths for Ray Tracing (DXR/Vulkan RT) or Mesh Shaders.
• Autonomous Streaming Lanes: Lane-managed “just-in-time” geometry/texture streaming (Nanite/Virtual Textures).
• DeferredLane: G-Buffer rendering for massive light counts, decoupling lighting from geometry cost.
• Predictive Strategy Switching: Using MetricStore trend analysis to pre-emptively switch strategies before budget violations occur.
• Cascaded Shadow Maps: Multiple shadow cascades for better depth precision over large view distances.

13. The Khora SDK Engine API

As the Khora Engine grew in complexity with its intricate SAA (Symbiotic Adaptive Architecture) and CLAD (Control, Lanes, Agents, Data) pattern, it became clear that exposing the internal systems directly to game developers would be overwhelming and architecturally impure.

To solve this, we introduced the khora-sdk crate.

The SDK Principle

The khora-sdk crate is designed to be the absolute single point of entry for any external project or game using the Khora Engine.

Warning

External projects (like the sandbox example) should never declare direct dependencies on internal crates like khora-core, khora-data, or khora-lanes. Doing so breaks the engine’s architectural encapsulation.

If a developer needs access to a core type (like a Vec3 from khora_core, or a Transform from khora_data), the khora-sdk crate is responsible for re-exporting those types.

The prelude Module

The SDK provides a highly convenient prelude module that re-exports the most commonly used types across the entire engine architecture.

#![allow(unused)]
fn main() {
use khora_sdk::prelude::*;
use khora_sdk::prelude::math::{Vec3, Quaternion, LinearRgba};
use khora_sdk::prelude::ecs::{EntityId, Transform, GlobalTransform, Camera, Light};
use khora_sdk::prelude::materials::StandardMaterial;
}

This ensures that the underlying implementation crates (khora-core, khora-data) can be completely refactored or replaced without ever breaking the user’s game code, as long as the SDK continues to export the expected interface.

The Vessel Abstraction

To interact with the ECS (Entity Component System) without needing to understand the underlying archetype storage or direct component manipulation, the SDK provides the Vessel abstraction.

A Vessel is a high-level, builder-pattern wrapper around an ECS entity. It guarantees that every object in your game world has a basic physical presence (a Transform for local coordinates and a GlobalTransform for rendering).

Example Usage

#![allow(unused)]
fn main() {
// Spawning a 3D Sphere in the world with a PBR material
let gold_material = StandardMaterial {
    base_color: LinearRgba::new(1.0, 0.8, 0.4, 1.0),
    metallic: 1.0,
    roughness: 0.2,
    ..Default::default()
};

let material_handle = world.add_material(Box::new(gold_material));

khora_sdk::spawn_sphere(world, 1.0, 32, 16)
    .at_position(Vec3::new(0.0, 5.0, -10.0))
    .with_component(material_handle)
    .build();
}

The GameWorld and Application Traits

The SDK provides a standard Application trait that developers must implement. This trait isolates game logic from the low-level render loop and OS windowing events. The GameWorld is provided as a mutable parameter to setup and update loops, serving as the interface to spawn entities, define lighting, and react to inputs.

API Reference

The complete auto-generated Rust API documentation for all khora-* crates is available at:

eraflo.github.io/KhoraEngine/api

This reference is generated with cargo doc and covers every public type, trait, function, and module in the workspace.