Spawn Point is an open-source game generator: a fine-tuned 7B model turns a sentence ("a platformer where a ninja rescues his mother") into a JSON game design, and plain Python draws every sprite and assembles a playable Godot 4 game. I recently shipped model v5.1 and Spawn Point 0.3.3. This post is about what I learned fixing a regression, and about a decision to make the project pixel art only.
The regression I shipped on purpose
v5 added side-scrolling shooters and a per-genre output format. It was clearly better on the new prompts, but a bit worse than v4 on my older hand-written sets (30 real prompts: 87% to 80%). The cause was my last training patch. It taught the model about enemies that are implied rather than named ("a tank shooting monsters"), and the model over-corrected: when a prompt named the player only indirectly, it answered with the default "Hero".
I released v5 anyway, wrote the weakness into the model card, and planned a patch.
Patch 1: more data, same learning rate. Mixed results.
My first v5.1 attempt added about 730 new examples (half of them naming the hero indirectly) on top of 800 replayed old rows, at the learning rate I'd used before. Result: the 30-prompt set and two dev sets improved a little, but my sealed shooter set, the cleanest benchmark I have, fell from 88% to 69%, and a second sealed set slipped too. Three prompts broke, and not because of the Hero problem: a wrong archetype, a genre confusion, a renamed rescue target. It was not a win, so I didn't ship it.
Patch 2: less of everything
The second attempt used about 370 new examples, 1,080 replayed rows, and half the learning rate (3e-5). Fully-right prompts, hand-written English and Vietnamese, whole pipeline:
| set | v4 | v5 | v5.1 |
|---|---|---|---|
| 30 real-world prompts | 87% | 80% | 87% |
| older sealed set (12) | 67% | 58% | 75% |
| set of 16 (farms, tower defense...) | 75% | 75% | 94% |
| sealed shooter set (16) | 56% | 88% | 75% |
| newest set (12) | 92% | 92% | 83% |
Two honest caveats. Run-to-run noise at temperature 0.4 is one to three prompts per set, so a single row proves little; look at the pattern. And the shooter and newest sets still dip at full precision. The 4-bit AWQ build you actually download scored 88% on the sealed shooter set and 80% on the 30 prompts, which says as much about noise as about the model.
What I'd take from it:
- A patch can fix one behavior and break another. Re-run every evaluation set after every patch, not just the one you were targeting.
- Gentler beat bigger. Fewer new rows and a lower learning rate recovered the old sets without losing the new ones.
- Keep a sealed set you don't look at while tuning. It is the only reason I caught that patch 1 looked fine on the dev sets.
Pixel art only
The other decision: Spawn Point used to offer two looks, pixel art and a smooth "cartoon" style. I'm dropping the second as a goal and focusing on pixel art. A prompt that asks for cartoon art now still gets pixel art, and the cartoon engine stays only for hand-written graphs. The practical reason is time: every new monster or item had to be drawn twice. The better reason is that a small project looks more coherent with one strong style.
So 0.3.3 is a pixel art release:
- Six new monsters: a biting plant, a mushroom, a reaper, a flame spirit, a snail and an alien, each with idle, run and hurt animations. Attack and death are derived from existing frames, so a new creature gets them for free.
-
Names pick the art:
PiranhaPlant,Mushroom,Reaper,FireSpirit,GiantSnailandAlienroute to the new sprites. -
spawnpoint sprites paraderenders an animated sheet of heroes, monsters or items at a large scale. - Closer, finer demos: games use the finer pixel mode and zoom in, and every demo GIF is re-recorded.
I also checked that adding the new nouns to the model's prompt didn't hurt it: the sealed shooter set held at 94% with the served model, and the 30 real prompts moved from 80% to 77%, which is one prompt of noise.
Try it
pip install -U spawnpoint-studio
spawnpoint llm up # downloads the v5.1 model (12 GB NVIDIA GPU + Docker, or CPU via Ollama)
spawnpoint create "a platformer where a ninja rescues his mother, with mushrooms and snails"
You can also play nine generated games in the browser, with touch controls on a phone: leluong141996-dev.github.io/Spawn-Point.
- Repo: https://github.com/leluong141996-dev/Spawn-Point
- Model (AWQ): https://huggingface.co/smoky1496/SpawnPoint-Coder-7B-AWQ
- Model (GGUF for Ollama): https://huggingface.co/smoky1496/SpawnPoint-Coder-7B-GGUF
A small ask
The model files have been downloaded a few hundred times and the repo has no stars yet. I suspect most people run it, see what it does, and move on, which is fair. If you tried it and it did something useful or interesting, a star on GitHub or a like on the Hugging Face page helps other people find it. And if it did something wrong, an issue with the prompt you used helps me more than a star: my evaluation sets only grow from prompts that broke.
What would you want a model like this to draw next?
Top comments (2)
The 1,080 replayed rows: same pool as the sealed shooter set, or only the older hand-written prompts you already watch? A sealed prompt that shows up in the replay stops being a holdout, and the 75% on that set would not mean what the table says.
Separate pools: replay is sampled from the synthetic v5 set only, and the sealed prompts are hand-written in their own files. I grep'd the v5.1 training data against every eval prompt after reading this: no hits. The leak risk I actually worry about is softer: the synthetic templates and my hand-written prompts come from the same head, so the sealed sets are "unseen" but not "out of distribution." Something to fix with prompts written by other people.