ComfyUI Visual Stories — The Python Scripts

This post covers the course's other operational path: not the node-based GUI — explained in The Node Graph — but Python and bash scripts that automate the generation (essentially patcher.py and sweep.py)¹, useful for handling things like loops, multiple variants and the repetition of node clusters — hard to manage through graphs alone. In this post we'll see these scripts applied to 14 shots from the Sotto l'Asfalto story, with the two recurring characters John and Elena.

The Node Graph — a pipeline shown on its own page — runs to 50+ nodes in ComfyUI. Scaling that kind of graph to generate 14 shots means replicating identical sub-graphs fourteen times — that is, multiplying around ten nodes by 14 within a single graph. This means handling prompts, weights and image references, shot by shot, inside the JSON. A process that quickly becomes unworkable.² This is the problem the scripts described here solve.

1. 01

   Vibe-code: creating without writing code

   Vibe-coding is the practice of using an AI assistant — Claude, GPT, Gemini and others — to generate working code by describing the problem in natural language. You don't write Python directly; you describe the problem to the LLM, which writes the code — code you can either run yourself or have the model run, simulating the Python interpreter. The result behaves exactly as if a developer had built it for you. When changes are needed, you work through a configuration file that describes the properties of the vibe-coded functions: in principle, changing these configuration values is enough to get different runs of the code. This is AI's great revolution in programming: reducing developer jargon to natural language, and reshaping how teams are organised.

   In this pipeline the division of roles is clear. The AI handles the technical layer — generating the node-graph JSON, writing the patching and sweep scripts.³ The human operator handles the creative layer — describing what each shot looks like (lighting, focal lengths, camera placement, etc.), setting the weights that balance facial identity, choosing which generated frames to keep, etc.

   Coming back to this project, the workflow uses five files: a) a base graph JSON (generated once by Claude from a natural-language description of the pipeline), b) a configuration file you can easily edit before each generation run, c) two automation scripts (also generated by Claude and reusable across other projects), d) the character reference images. No syntax to learn. One command to run. Filling in the config file and running a script is the entire technical interaction.

2. 02

   The project

   The example used throughout this post is a 14-shot storyboard called bar_14shots, set in a wine bar with two recurring characters — Jim and Barbara. Their reference images (full-body turnaround, face crop, two-character reference) are the same files described in the setup page. Each shot is photorealistic: Flux.1-dev as the base diffusion model, PuLID for face identity locking, XLabs IPAdapter for body and clothing consistency, ControlNet depth for spatial structure.

   Before writing any configuration or running any script, it helps to describe the project in natural language — because that description is precisely what you will hand to Claude to generate the base graph JSON. A clear project brief covers: the number of shots; the two characters; which shots feature one character and which feature both (shots with two characters need a doubled PuLID + IPAdapter chain in the graph); the bar interior setting, anchored by a depth reference image called wall1.png; and the overall visual register — in this case photorealistic and cinematic.

   Each shot can vary independently in framing (medium shot, wide shot, close-up), focal length, camera angle, and props. All of that variation is controlled through the configuration file — not through ComfyUI's interface, and not by editing the graph. Once the graph is generated by Claude, it becomes fixed infrastructure and is never touched again.

Want the concrete, hands-on steps — the five files, the run sequence, and the post-processing pass? See Instructions for writing and running this section's workflow.

1. The name echoes monkey patching: modifying or extending classes, functions or objects at runtime without touching the original source code. Here it applies to the ComfyUI graph: patcher.py takes the base graph (never modified) and injects the configuration file's values into it at runtime — prompts, weights, image references — producing the PATCHED files ready for generation. sweep.py automates the generation itself: it varies a single parameter across a list of values and fires one API call per value, so several results can be compared quickly. ↩
2. Although it is theoretically possible, with vibe-coding, to multiply sub-graphs and chain them indefinitely, in practice this runs into problems of visual readability and of actually running graphs with too many nodes. It is better to plan the project by splitting it into several distinct graphs, properly feeding the output of one as the input of the next. ↩
3. Internally they're simple: patcher.py loads the graph JSON as a Python dictionary, looks up the target nodes by their ID (always the same, since the graph is fixed), and overwrites their parameter fields with the values read from shots_config.py — then saves the result to a new file. sweep.py does the same job as patcher.py in a loop: for each value in the list, it injects that value into the node, sends the patched JSON to the ComfyUI API endpoint, and saves the returned image under a name that includes the value used. ↩