Cod3xDocs
guide

Design Genealogy: StaticCollection

How S3maphore changed representation, scheduling, and ownership before optimizing the collection loop.

The problem

S3maphore's playlist rules need facts about the world around the player: which records, types, and content files are present, whether hostile actors are nearby, and whether the exterior area has changed.

The naive representation is a retained list of game objects. That is convenient at first. It also keeps engine-backed values alive, repeats expensive interpretation at query time, and makes the lifetime of the collected objects part of every rule's problem.

Change the representation first

The important early move was not a cleverer loop. It was changing what the collector retained.

Commit 5d608a26 made that first move toward cached string and count primitives instead of retaining game objects as the main representation. The current staticCollection.lua exposes the result as counts by record, type, and content file.

That choice changes the problem. Playlist rules can ask semantic questions about counts and identities without repeatedly crossing into object userdata. The collector owns the engine-bound traversal; the consumer receives data shaped for the decision it has to make.

Then make the work incremental

The representation change made incremental work possible:

  • 342d77dd then tracked cell and combat-target changes with events, removing more repeated work from the collection path;
  • 8d574d40 moved presence tracking into a coroutine-driven global sweep;
  • 76358648 removed the redundant staticList representation;
  • b202141c added per-cell presence so the system could choose the appropriate scope;
  • d7f38bb5 applied a large optimization pass after the shape and ownership were understood.

The sequence is more important than any one commit:

The representation sequence

The collection became cheaper only after its data shape, invalidation, deferred work, and lifetime were made explicit.

  1. engine

    Engine objects

    Rich live handles are expensive to traverse and retain.

  2. state

    String / count representation

    Keep the facts the rules actually need.

  3. service

    Event-aware changes

    React when relevant world state changes.

  4. component

    Incremental coroutine work

    Spread a large scan across bounded slices of time.

  5. boundary

    Explicit generation and lifetime rules

    Reject deferred results that no longer belong to the current world.

  6. result

    Optimization

    Tune the proven shape only after the ownership story is honest.

How the pieces relate
  • Engine objects String / count representation
  • String / count representation Event-aware changes
  • Event-aware changes Incremental coroutine work
  • Incremental coroutine work Explicit generation and lifetime rules
  • Explicit generation and lifetime rules Optimization

The generation-counter scar belongs in this genealogy too. Once work can be deferred, b148ee31 gives a result an identity so an old sweep cannot quietly publish into a newer world. See Generations for Deferred Work.

Why this is a crown jewel

This one system ties together several performance truths without reducing them to “Lua is slow”:

  • representation determines what must cross the engine boundary;
  • counts and strings can outlive a traversal more safely than live object handles;
  • incremental work makes a large scan fit within a frame budget;
  • events reduce needless recomputation but increase invalidation responsibility;
  • deferred work needs an authority check before it publishes;
  • optimization comes after the data shape and ownership are honest.

That is why this belongs late in the suggested path. The pieces are simple individually, but the genealogy shows how they become necessary rather than fashionable.

What to steal

Before optimizing a loop, ask whether it is iterating the right representation. Before making work incremental, define what data may be published and when it becomes stale.

Do not begin with a coroutine, a cache, or a generation counter. Begin by deciding what the consumer actually needs and who owns the engine traversal.

When not to use this

If the collection is small, short-lived, and not queried by multiple consumers, retain the simplest local representation. Do not build a global incremental collector to avoid a cheap direct lookup. This design earns its machinery through world size, repeated queries, engine-boundary cost, or a real frame budget.