One character that stays the same across scenes

Grawwww — a small engineering studio

Transparency: we sell generative image work to teams and we use the system we describe. Every image in this article is AI-generated. Except for the base-model comparison, which uses no LoRA, all come from our own LoRA, trained on our own synthetic dataset. The offer is at the end, labelled as such. The rest stands without it.

An illustrated book needs the same character in hundreds of drawings: same face, same armour, same ink line and crosshatching. With a prompt alone, the character drifts from one image to the next. We checked: with nothing trained, the character's name means nothing to the model and it returns a generic knight.

We took Don Quixote, a public-domain character, and built a reusable ComfyUI rig. Here is how we picked the base, how we made the dataset, what we trained, what came out and what still fails.

Character sheet with six new scenes of a tall thin knight drawn in ink with crosshatching — The same character in six new scenes.

Choosing the base

We tested three base models, no LoRA, with 8 subjects and 2 seeds each (48 images, no failures), using the same 19th-century pen-and-ink prompt, on the same RTX 4090.

Base	Licence	Steps	Warm s/image	Peak VRAM (whole GPU)
SDXL 1.0	OpenRAIL++-M, with conditions	30	4.29	11.2 GiB
Z-Image Turbo	Apache-2.0	8	4.02	23.2 GiB
Third model	Its licence does not fit client delivery	8	6.29	20.2 GiB

We chose Z-Image Turbo for three reasons. Its licence is Apache-2.0, per the model card. It gives clean black line on white, with shading made of hatching rather than wash. And at 8 steps it takes about 4 s per image warm on a single RTX 4090.

SDXL came out greyer, with cream paper and filled backgrounds even when we asked for white. It needs about half the GPU memory, so it stays an option on a smaller card. The third model had the finest hatching, but its licence does not fit delivery to a client. We used it for comparison only.

Measurement note: peak VRAM is for the whole GPU, including about 2 GiB from other processes. Z-Image's peak is close to the 24 GB limit.

Two columns of ink-drawn knights on horseback from the same prompt: SDXL on the left, Z-Image Turbo on the right — SDXL (left) and Z-Image Turbo (right), no LoRA, two seeds per model. The prompt, in words: a gaunt old knight in improvised armour on a thin horse, 19th-century pen-and-ink engraving.

The dataset: we generated it

The obvious move was to train on scanned drawings from a 1906 book. We did not. Those drawings are mostly scenes with several small figures, and we wanted one large, recognisable character.

Instead we wrote a character sheet: long thin face, pointed grey beard, drooping moustache, barber's-basin helmet, dented breastplate. From that sheet we generated 236 candidates, varying framing (full figure, knees-up, half figure, close-up, profile), pose and view. We kept 32.

Rejections had concrete reasons. One funnel-shaped hat instead of the helmet. Four images with brown colour leaking into leather and skin, only visible by checking pixel values. Each image has a caption that describes what is visible (framing, pose, action, view) and never the face, beard or armour: the character's trigger word has to carry those.

Known gaps: no half figures, no genuine laughing face, nothing seated at a table, almost everything on white backgrounds; a few outdoor scenes.

Training

One LoRA (rank 32) on Z-Image Turbo, with ai-toolkit. We saved a checkpoint every 500 steps and compared five variants (no LoRA; step 1000 at strength 0.7 and 0.9; step 1500; step 2500) on 8 new scenes with 2 seeds: 80 images.

We kept the step-1000 checkpoint at strength 0.9. It takes about 30 minutes on one RTX 4090 (estimated: 1,000 steps at ~1.8 s per step; the full 2,500-step session took 83 min). At step 1500 close-up requests came out as half figures; at 2500 this did not repeat in the sheet we reviewed. Also the lance, which appears in 18 of the 22 full-figure images in the dataset, began to show up unasked. Step 1000 respects framing and identity. Step 2500 was not clearly better and carries more bias.

Results

Six new scenes, all with the same prompt opening (face visible, visor open) and 8 steps:

Charging a windmill with his lance, on a thin horse.
Laughing close-up, three-quarter view.
Writing a letter by candlelight, in an inn.
Talking to a short stout squire on a donkey.
Kneeling in a chapel, keeping vigil over his armour at night.
Asleep under a tree with his lance beside him.

Gaunt knight on a thin horse charging a windmill with his lance, ink drawing — Windmill.

Close-up of an old man with grey beard and drooping moustache laughing, three-quarter view — Laughing close-up.

Knight at a rough wooden table writing a letter by candlelight in an inn — Letter at the inn, by candlelight, with a goose-feather quill.

Knight talking to a short stout squire on a donkey on a dusty road — With the squire.

Knight kneeling in a chapel keeping vigil over his armour at night — Armour vigil.

Knight asleep under a tree with his lance beside him — Sleeping.

For each scene we generated 4 variants with different seeds and picked one. These are the best of four, not every output (the inn scene: best of eight, with the quill requested in the prompt).

Fixing an anachronism

Our first version of the inn scene came out with a modern fountain pen in the middle of the 17th century. We tried swapping it by inpainting and the quill came out looking pasted on, like a cut-out. So we regenerated the scene with a goose quill and an inkwell in the prompt (best of 8 seeds, same LoRA, same character).

Two versions of the inn scene: on the left the knight writes with a modern fountain pen, on the right with a goose quill and an inkwell; below, a zoom on the hand in each — Before: modern fountain pen in a 17th-century scene; after: regenerated asking for a goose quill and an inkwell.

Repairing a defect: LaMa plus the LoRA

In the close-up, the model engraved random letter-like marks on the armour. Asking this distilled model to inpaint the hole directly invented things in it (faces, hands, helmets) or left flat grey patches; we tried several rounds. What works is two steps. First, LaMa (an inpainting model that fills from the surrounding texture without inventing a subject) removes the marks. Then a light pass with the character LoRA (denoise 0.3, on 1024-px crops) restores the hatching. The result is pasted through the mask grown by 4 px, and outside it not one pixel changes (verified: 14,781 px changed, 0 outside). LaMa's code is Apache-2.0.

Before, mask and after: letter-like marks engraved on the breastplate, the red mask only over those marks, and clean steel with the hatching continuing — Before: marks on the armour. Mask: only the pixels of the defect. After: clean steel; everything else identical.

What still fails

Closed visor in dark scenes. In dim or candle-lit scenes, several images show a closed helmet hiding the face, even at the last checkpoint. The model's generic "knight" beats the character. That is why our prompts ask for "visor open, face visible". We did not count this systematically.
Grey interiors and night. Backgrounds drift from the dataset's clean white to busy, grey ones with stray decoration.
A judgement by eye. Choosing step 1000 is a judgement from 16 seed-prompt pairs, not a metric.
The repaired area is slightly grainier. At high zoom, the repaired patch of armour looks slightly grainier than its surroundings.
Two test sheets not reviewed (4 and 6) out of 8.
For v2 we would add half figures, laughter, seated scenes and interiors to the dataset.

What a client gets

The same rig with your drawings in place of ours. One ComfyUI graph to generate the character; a script that downloads each model from its official source and checks its hash; and each model's licence, listed for you to review. It runs locally with Docker; the rented-GPU version is part of the delivery.

Screenshot of the ComfyUI node graph that generates the character with the LoRA — Character generation graph. The graph we deliver.

Each model's licence is listed for you to review. Some we left out on purpose because of their licence (for example, certain non-commercial preprocessors and upscalers). We do not guarantee copyright over AI-generated images; it depends on each country.

What we do for teams

Commercial section. Grawwww sells the work this article describes.

One character or house style, consistent across your book, game or brand. You send your drawings or your character sheet. We build the rig with your style, show you new scenes, and hand you the rig to run yourself. A senior engineer reviews and answers for every delivery.

Custom models, from €450 excl. VAT. It is a starting price: the proposal fixes scope and price after the diagnostic (€300 excl. VAT, deducted if we go ahead).

Send us your case: https://grawwww.xyz/en/teams/

Every image in this article is AI-generated; except the base-model comparison, with our own LoRA.

Another article: Agents don't review themselves →