Streaming an Open-World City in Godot Without Hitches

Driving across town. Everything in this shot was streamed in around the player, a 32-meter square at a time.

The city in Philadelphia Fiefdom is cut into 32-meter squares, and each one is built ahead of time and streamed in only when someone's near it. Every chunk comes in two halves: what the simulation needs, streamed in around every player, and what the screen draws, streamed in around each screen's own player. Both are read on worker threads, nearest first, so the main thread only ever spends a fraction of a millisecond putting a chunk into the world.

Last time I rebuilt the game for online co-op. One of its rules is that everything happens near any player, never near the camera, and streaming the city is where that rule earns its keep.

What's in a chunk?

A chunk is a 32 by 32 meter square of city. The weather bake samples it at 16 texels a meter, so each chunk's weather tile is 512 by 512 texels. Into one chunk goes:

The ground. Asphalt, curbs, sidewalks, alleys, road markings, crossings.
Buildings. Each building goes in the chunk its middle stands in, whole.
What's solid. The boxes and ground triangles things collide with.
Walkways. The navigation mesh people walk on.
The street furniture. Lamps, poles, signals, hydrants, bins and drains.
Weather tiles. How high the cover is over every spot, so the rain knows where it can reach, and where water stands: the gutters running to the drains, the dips in the road.

Why 32 meters?

When I first set up streaming, I tested a few chunk sizes for how smoothly they streamed in, and 32 meters was the sweet spot: the one that came in with the fewest hitches. The city has been built on it ever since. Here's what a chunk that size costs:

Draw calls. A chunk is a handful of draws, and on a desktop screen about 16 chunks are drawn at once.
Putting one in. On my PC, the worst chunk takes a few tenths of a millisecond to put on screen and under a millisecond to make solid.
Small jobs. A chunk is small enough to build on a worker in a couple of milliseconds, and to put into the world a few pieces a frame.

How did the first version work?

The city first got chunks on October 5, and loading it went from 43 seconds to 4. Most of those 43 seconds had been snow: the game walked every triangle of every building at load, looking for sills and roofs to put snow caps on. With chunks, each one got its own caps, built while snow was about, a few meshes a frame.

A streamer read chunks on a thread around the view's focus and dropped them behind. The first bench swept the view across the city at 20 meters a second for 200 meters: 27 chunks in, 26 out, an average frame of 3.2 ms, the worst 1% at 4.1 ms. That looked fine. It wasn't.

What made it hitch?

Driving fast showed the problems. Every fix came from a bench run, a profile, and a number:

Snow caps, built live. In snow, building each chunk's caps on the fly cost about 2 seconds of main thread for every 8 seconds of driving, with frames up to 80 ms. They're baked per chunk when the city's built now, and loaded on a thread.
What a chunk brings to life. Profiling showed the cost wasn't reading a chunk, it was waking up what was in it. Every lit window searched the whole scene for the sun, about 2 ms a window. Now it's found once and shared.
Nearest first. Chunks are asked for and put in nearest the view first. With that and the window fix, frames over 16.7 ms on the bench went from 26-35 to 12, none over 33.
A few pieces a frame. A chunk goes in as ground and walkways first, then its buildings, furniture and lights a few at a time, within 1.5 ms a frame.
Off the main thread. Chunks are built on a worker thread, 1 to 2.5 ms each, and only handed to the main thread once done. A building's ivy, the posters on a pole and a manhole's cover are set up there too. Spikes on a drive went from 9-18 to 4.
Going out, too. A chunk freed all at once cost 5-11 ms. It's taken down a few pieces a frame now.
A slow machine. Running the streamer five times slower, as a stand-in for a handheld, the ground was missing at the screen's edge in 172 frames of 721. Putting each chunk's ground in first, building two chunks at once, and spending 5 ms a frame instead of 1.5 while a chunk in view is unfinished brought that to 14-18.

What changed in the rebuild?

When I rebuilt the game, streaming the city was the second thing in, half an hour after the first commit. It's a different design now, built for co-op.

Each chunk is split in two. The simulation's half holds what's solid and the walkways. The drawing half holds the meshes and the weather tiles. They're separate files. The host keeps the city solid round a guest across town without drawing any of it, and each screen draws only round its own player.

This demo needs JavaScript.

Try it: drag your player round the city. The blue ring keeps the city solid round both of us, the orange one draws only round you, and the nearest chunks come in first, a couple a frame. Walk back and forth over a chunk's edge and nothing reloads; switch to all at once and watch the frame time spike.

Everything is baked offline. One build bakes the whole city ahead of time, chunk by chunk: the meshes, the collision, the walkways, the weather tiles and the furniture. Nothing is generated while you play. The old game drew an overhead map of the weather live and read it back from the graphics card. The new one reads nothing back.

No nodes. A chunk on screen is one rendering-server instance, not a tree of scene nodes. Its solid half goes into the physics world as one static body.

Fewer draws. A chunk used to draw once per surface, about 16 surfaces on average and up to 43. Now every vertex carries its surface's number, the shader looks the colors up in one table, and a chunk draws in a handful of calls. The whole screen, with shadows, went from about 400 draw calls to about 40.

Baking the buildings into the chunks made the city's files about 14 times bigger. That's the trade I'm taking for a chunk that goes in with nothing to make.

What was the 55 ms hitch?

The bench caught frames of up to 75 ms on a drive. It was the walkways. Each chunk's navigation mesh was being forced to sync on the main thread the moment chunks streamed in: 55-77 ms every time. Now it syncs on the navigation server's own threads, and someone with no path yet just waits a moment. The worst frame on the bench went from 45-138 ms to 5.1-7.6 ms, and frames over 16.7 ms went from up to 17 a scene to none.

What went wrong at the edges?

Chunks have edges, and bugs love edges:

A step in the snow. Snow is simulated in a window of tiles round the player. A tile newly filled started bare next to hours of history, so there was a step along every chunk's edge. New tiles carry on from their neighbors now.
Walkways that didn't meet. In the first version, the walkway bake stopped short of every chunk's edge. Now each chunk's walkways are baked a person's width past its edge, so neighbors overlap and join.
Two bins on one corner. Furniture was planned chunk by chunk, so two chunks meeting at a corner would each put a bin there. Now it's planned once, for the whole city.
A wire that popped in. A 45 m wire across a junction belonged to its far chunk, so it appeared out of nowhere. Wires now live in the chunk under their middle.
Every leaf at once. The first chunks in at the start were never told they were ground, so no fallen leaves lay on them until a later chunk arrived, and then every leaf popped in at once.
A building for one frame. A building arriving in its chunk was drawn whole for a frame before the cutaway noticed it was in the way. Now the cutaway decides the frame it arrives.
Falling out of the world. You could walk into a chunk that wasn't in yet. Now anything with a body waits on ground that hasn't loaded, and doesn't fall or step.
Long shadows. At sunset a block past the drawn chunks should still shade the street. Each chunk that isn't drawn stands its buildings in as plain blocks, drawn only into the shadows.

There's more to say about streaming on a Steam Deck, but that's its own post.

Quick answers

Is there a loading screen? Only when the game starts. After that the city streams in as you go, on foot or by car.

Does a guest across town slow the host down? The host keeps the city solid round the guest, but doesn't draw it. Joining mid-game streams on a worker thread too, so the host's frame doesn't stall.

Why not generate the city as you go? Baking it ahead means a chunk does no work when it arrives, and every player gets exactly the same city. The game checks that when someone joins.

Follow along

Philadelphia Fiefdom is an online co-op sandbox set in a city laid out after Philadelphia. Last time was why I rebuilt the game for online co-op. Next, the fists come out: fighting the freeflow way. Join the playtest list to play it early, or come say hi on Discord (opens in a new tab).