AI-assisted stage design in Final Freeway

I recently released an update of my mobile game Final Freeway on App Store and Play Store. This update adds a new stage "Paris Sonnet" which was designed with the aid of AI.

Given the right context an LLM (Claude Sonnet 3.5 in this case) was able to produce a stage that was good enough and that only needed a few corrections to my taste.

The new stage is available in the paid version of the game. Look for "Paris Sonnet" in the time attack mode, or play Course B in the regular mode.

• Final Freeway on App Store
• Final Freeway on Google Play

NOTE: Feel free to skip to the "Track definition and LLMs" section if you're not interested in the backstory and other details.

Backstory

I originally released Final Freeway in 2009 as an early iPhone mobile game, built from the ancient code base of my 2001 Java Applet JavaKazRace.

The goal was always to make a one-man game with minimal effort. Graphics were a mix between pictures of matchbox car models, outdoor pictures and stock images, with some rough clipping and editing with Paint.NET and Photoshop.

Most of the original images from 2001 where clipped from photos I took myself around the Pacific Coast Highway in Huntington Beach, California, with my Ricoh RDC-300Z digital camera with a resolution of 640x480 pixels.

For the track definition, I went for an assembly-like compact text format that allows to define sections of the road with curves, hills, valleys, number of lanes, colors and placement of roadside items, which make the setting.

Having a simple text format spared me the work of building visual tool, something very common in modern games that makes it easier to iterate on the design of the stages, but that requires additional development work.

The game does have an edit mode, consisting in the ability to quickly reload a stage while running and to jump anywhere in the track without having to drive to that point.

Track/stage design

As it stands, creating a stage for the game is not difficult, and in fact, two of the better-looking stages of the original game were made by a friend that was able to quickly pick up the format.

The work to make a new stage comes down to choosing a theme, building the sprites, and then generating the track definition with a text editor.

This process is much simpler than for most other games, but it still requires some time and commitment to go through the full process.

# Curved shopping passage
LR +0 so_lightpole_paris 16
L +4 so_cafe_green_paris 32
R +3 so_kiosk_newspaper_flower_paris 32 16
LR +1 so_people_03 8 4
LR +2 so_tree_paris_blvd 16 8
LR +1 so_tulips_yellow 8 6
200 30 50 ...
200 -30 -50

This format expects first the setup of the road-side objects, followed by definition of the road itself.

The road-side objects are defined with a line like this:

<cmd> <x_distance> <sprite_name> <frequency> <offset>

Where:

• cmd is the command, L for left, R for right, LR for both sides (add an F to fill the side, useful for grass fields)
• x_offset is the distance from the side of the road
• sprite_name is the name of the sprite
• frequency is the frequency at which the object repeats
• z_offset is the offset within the frequency

The definition of the road sectors is comprised of 3 numbers:

• The length of the sector in meters (or slabs)
• The angle of the curve
• The angle of the inclination

The optional ellipsis (...) is used to keep the same road-side objects for the next sector. When there are no ellipsis, the road-side objects are reset.

There are also more commands to set the number and color of the lanes, the background panoramic image and other minor details.

Making the sprites

Images, or sprites, used in the stages were originally extracted from photos. For this new stage I instead obviously opted to use generative AI, using a few image models available today.

This saved some time, but there is still the manual work of doing a relatively precise clipping of the objects to extract them from the full images, as current models are focused on generating whole scenes, not just individual objects.

For the future, an asset pipeline leveraging a segmentation model to extract objects from images may help here, but I didn't try that yet.

Track definition and LLMs

Although sprites are fundamental for a stage, what I wanted to focus on was the automation of the track design, including the placement of roadside objects.

To do this was necessary to instruct an LLM like I would instruct a person, giving a list of available assets (usable images), and a general guide on how to build a stage.

For this, I created a Python script that builds the context for a generative prompt.

This context file includes:

• Description of the track format
• A list of available sprites, such as trees, buildings, panoramic backgrounds
• Definitions of existing tracks, to use as a reference/inspiration
• General notes and best practices for better results

Here's a snippet of the context file. The actual file is over 2000 lines long. This is generated by a Python script that reads the available sprites in the game's assets folder and the existing track definitions.

The first two lines are meant to be edited with any kind of detail specific to the stage to be created:

Please create a stage about <INSERT_HERE>.
Be creative and have fun with it. This is your chance to be a game designer !


The context below includes sections wrapped around XML-style tags.
The <images> section includes the list of images used in the game, divided by type.
The <track_spec> section includes the syntax and format of the track definition files.
The <roadtracks> section includes the list of road tracks, each wrapped around a <track_def> tag.
These tracks are meant as an example of how to define a track.

<images>

# Background images
bg_pharaon (880 x 100)
bg_green_hills (1024 x 34)
bg_arizona (1024 x 142)
bg_sunset_miami (1024 x 218)
bg_paris (1013 x 254)
bg_snow_mountains (1024 x 46)
bg_safari (1024 x 218)
bg_sea_cloudy (2048 x 178)

# Side-objects images
so_sphinx (476 x 512)
so_elephant (175 x 150)
so_orange_condo (412 x 256)
so_bboard_pizza_world (124 x 256)
[...]

With the full context, I could then simply ask to generate a stage, and in most cases the response was a fully usable track definition. This can save a good chunk of work, and I plan to use similar approaches for other games in the future.

Paris Sonnet

To create the Paris stage, I first brainstormed in a chat with an LLM on which objects may make sense to have in, such as typical structures and styles. I then prepared those sprites and named the files in an intuitive way.

I then updated the context file for the prompt, so that it included the new assets for the stage, and asked an LLM to generate a new track.

I did not point the LLM to use specific sprites. The model did the guessing by picking obvious objects with "paris" in the name, and even less obvious ones named "so_jeanne_darc".

Recursive help

I should also note that because the stage format was never fully documented, and even I forgot exactly how it worked, I had an LLM do the job of decoding it from the source code of the parser and from some stage examples. The result was almost perfect, with only a minor correction needed.

This highlights the importance of trying to offload as much work as possible to an AI at every level of the process, including the creation of the supporting documentation to build the final system prompt. It's always a good idea to get into a recursive mindset to extract more efficiency.

Which model ?

For the selection of which LLM to use, I experimented with all the major ones, and they all gave plausible results, but I ultimately went with Anthropic's Sonnet 3.5, in part because I liked the initial results, in part because I've learned to depend on it with code and I probably have a decent intuition of what it can and cannot do.

For the future, this type of dynamic generation could move from the development phase, with large and powerful models, to the end user phase, with smaller and more affordable models that will eventually run locally on a mobile device.

On that vein I currently have RogueLLM generating game narrative dynamically via an LLM. In that case I rely on gpt-4o-mini, which can't run locally, but it's a step forward towards using smaller models to achieve the same results.

Eye of the beholder

Because the LLMs can't quite picture the results by themselves, it took a little coaching to give the perspective of the human viewer on what's pleasant to the eye, especially in the context of an old-school arcade game whose rendering is at the crossroads between 2D and 3D.

This highlights how a human-in-the-loop is still required to get feedback of how the animation feels like. A multimodal LLM will surely be able to deal with these details in the future, but it'll have to happen at least 30 fps and also have a sense of what's pleasant even if it's not realistic, e.g. 2.5D racing games are pleasant because of the unrealistic distribution of the roadside objects, with many fixed frequency repetitions that give a high sense of speed and the pleasure of recurring patterns, which are probably somewhat hypnotic...

I suppose an AI could understand and implement all that, but to directly go to that it would take a special kind of creativity and inspiration that comes from direct experience with the "human hardware": eyes, organic brain, natural predispositions, dopamine, memories, nostalgia, display devices (small phone screens and CRTs are quite forgiving), and so on.

Why delegating design ?

To be fair, most game developers probably envision a future where they get to dictate the design, while the AI relieves them from the burden of writing the actual game code and other annoying bits that come with development and publishing of a finished title (my kingdom for not having to deal with the Android NDK and the Play Store console anymore !).

Who will do which task is something that is still not clear and will depend on the developer and on the abilities of the AI models to come.

A well designed stage is also something that communicates something to the player, it's a shared experience between the developer and the players, and then among the players themselves.

I suspect that many players in the future will still favor games that are designed by human authors, even if those games may not be as stimulating as the ones designed by an all-powerful AI relentlessly optimizing for engagement... ouch.