一个时间旅行的叙事框架

In a live game, the pause between the player's action and the world's response has a narrow window of tolerance. With a too-slow reply, the moment loses its impact and the player picks up their phone or switches tabs. A good human DM keeps the table moving even while they're thinking. They might stall with a bit of description while they check their notes, or get the players talking among themselves, but they don't make everyone sit in feedback-less silence for a minute. At the same time, an instant response is not necessary—players are willing to wait a little, within reason, for a great result.

Our narrative world model means the story can track its own world state and stay coherent across a multi-hour session, but that isn't worth much if every turn takes a minute to arrive. So what does the model need to be thinking about? And how do we fit it all in?

The secret is that not all latency feels the same. Shipping a product that uses machine learning of many kinds means working in a multidimensional space where product design, machine learning, and software engineering all influence one another’s constraints and possibilities. Each one can be cleverly deployed to help sidestep problems in the others with that classic problem-solving trick, “simply make it so the problem doesn’t need to be solved.”

In this post, we’ll cover how our turn architecture makes room for the agentic story harness that keeps story quality high, how careful product choices around what the player actually waits for can buy valuable time for background reasoning, and how the ML side of things can help smooth over tricky engineering challenges (like state reconciliation that makes programmatic but not narrative sense) and enable product features that would otherwise be difficult to pull off.

Perceived latency and double-dipping on time

Co-creating a satisfying story requires enormous amounts of thought, no matter who (or what) the DM is. A model's thinking is measured in tokens, and so is the input context the model needs to make good decisions. Deciding what each character wants, when a secret is ready to come out, and what plot beats or reveals a scene still needs to pay off before ending are judgement calls, and they rely on sifting through a bunch of information to determine what’s most relevant. They need strong models and room to reason, and increasingly, room to use tools to curate their own information. Stuffing everything into one prompt doesn't help either way. It's slower, and the model gets distracted by whatever's loudest or most recent (see the 5 bad habits of LLM storytellers in Choose your own demise.)

In the basic case of an AI story experience you’d get out of a custom prompt in a chat tool or a character-chat app, you’re interacting with one model, with linear input and output, operating on one timeline. All the thinking happens inside the call as the player waits on each turn, which is why it tends to be either quick and shallow or thoughtful and slow. Even with common tools like character cards and context summaries, it tends to grow less coherent over time as the story history grows.

At Hidden Door, we get to have it both ways. Not only does our narrative world model manage long-running stories with structured information, memory, and differential information, every turn run by our agentic story harness runs on two timelines to keep story quality high and latency low.

The fast timeline is the one the player waits on when they submit their action. It responds to the player's intent, narrates what happens, works out and applies the resulting world-state changes, styles the text, and suggests next moves for the player to choose from. The fast timeline contains only the tasks that this reply can't do without, and it acts on guidance prepared on the previous turn. And our narrative world model means that as gameplay goes on, this step doesn’t lose coherence to bloated context → compaction cycles.

The slow timeline is the one charged with longer-horizon thinking, planning, and story quality. This is the heart of Hidden Door's AI DM within the platform. Instead of getting everything stuffed into the prompt, it uses tools to retrieve structured context just in time, at an appropriate level of detail for the given task or decision: scene history, the player's current focus, secrets, the chapter's plan, the shared history between two characters. It manages each NPC's goals and beliefs, follows situational playbooks, judges whether the scene is ready to end, adjusts pacing and tension, and writes a brief providing guidance for the next fast reply. This is the narrative planning half of the job, and it's where most of the model calls in a turn go, anywhere from a few up to 20. Alongside the planning task, we run smaller upkeep jobs: a running summary of the scene, tidying story variables and game state, tracking secrets and reveals, and keeping descriptions current.

The trick is that the player almost never waits for these slower-thinking tasks.

The tasks in the slow timeline kick off in the background as the fast reply goes out, and they run in the seconds the player was going to spend anyway, reading the story beats and deciding what to do next. That allows us to effectively double-dip on time because the player perceives no extra latency unless they submit their next turn before the slow tasks are complete. In practice, that rarely happens; over a recent two-week window, only 0.5% of player-submitted turns had to wait any length of time on an in-flight background task.

Of course, there are some tradeoffs. The deeper planning tasks must advise the next turn without knowing the player's next move in advance. Here, we’re inspired by the way a human DM needs to prepare, both ahead of a session and in between turns: adjusting to where the player is heading, adapting the plot arc and how it might conclude, what hints could be dropped, which character needs a spotlight moment.

None of that depends on predicting the player's exact next move. A human DM who prepped the dungeon and watches the party head for the tavern instead doesn't drag them back underground. The ambush, the clue, and the spotlight moment come along to the tavern, sometimes with a little narrative sleight of hand. In our systems, the slow timeline's guidance is treated as just that, so if the player makes a surprising choice, it bends instead of doubling down on incoherence or throwing away the thoughtful plan that helps guide toward high-quality story.

Differential knowledge rides along with differential timing

The less obvious payoff of splitting the turn is that the system writing the turn-by-turn reply should know less than the more comprehensive, slower planning step. A single model that knows everything tends to use everything even when instructed not to. The innkeeper who is supposed to be hiding the heir will casually leak that secret over your first drink, your fellow bar patrons react to the news as if it's common knowledge, and the betrayal planned for the story arc’s finale either gets forgotten or plays out within a few turns (see eagerness to resolve tension, one of AI storytelling's worst habits). Even frontier models are really bad at holding back knowledge they aren't supposed to act on yet, and in a long context, instructions about it tend to get overridden by whatever is most relevant to the plot in the moment.

So we deliberately keep secrets from the fast writer. It gets a short, focused brief about what this moment needs, and it can put all its attention on the moment.

The same trouble shows up in what characters know. If your character just tumbled into a magical world and from our mundane one, your roommate should freak out the first time you cast a spell, but the locals of the magical world shouldn't blink. Give a simple AI tool enough context to understand the magic system, and it tends to treat magic as expected and unremarkable for everyone, everywhere. Tracking who knows what is a memory problem as much as a timing one!

The same split lets us feed the fast writer precisely the context required for the decisions we know it may need to make—so if the player character casts a spell in front of their unsuspecting roommate, thus revealing the existence of magic to them, the system is prepared to handle it…but it also can’t leak deeper secrets about the growing plot to overthrow the ruling clan in the magical world, because we don’t tell it.

Wrapping up

It's tempting to approach latency as just an engineering problem, chasing faster providers, smaller models, and fewer calls. And all of those matter! But our biggest wins have come from making architecture, machine learning, and product choices that complement each other. Double-dipping on time with those tools makes room for the intensive thinking, planning, and upkeep necessary for good storytelling—all, from the player’s perspective, for free.