The lobster, the soul file, and a missing ffmpeg flag

Blog post #49


OpenClaw is the first video in what I’m calling the agent trilogy — three 3-minute explainers about self-hosted AI agents. The other two are Pi and Hermes Agent. I started today with a locked script and a HeyGen render already on disk. By the time I shut the laptop, I had a YouTube-ready first cut. This is the log of what happened in between.


What changed since last log

Last log was about translating OpenClaw to Japanese with three models in parallel. Today was production on the English original. Different problem, different mood — the script was done, now I had to make the video actually exist.

What shipped

A 3:50 first cut of OpenClaw English. Spine from HeyGen Avatar IV with the clock-tick intro audio baked in. Eleven slides composited over it. PiP nere höger from 0:22 to 3:20. Generic series intro card for the first 2.5 seconds, then the cut to me on camera on the pling. YouTube metadata file ready to copy-paste into Studio. Cost: ~74 HeyGen credits, six retakes of the audio, and one stupidly long ffmpeg job that I’ll get to in a minute.

What’s working

The composite pipeline is finally a pipeline. ChatGPT generates the center of each slide. A small HTML template renders the frame with the OpenClaw lobster top-left and my own lockup top-right. A Python script samples the cream color from the four corners of the ChatGPT image and pads to 1920x1080 with that exact color, then overlays the frame. No more visible seams between the generated content and the brand chrome. I’d been hardcoding #f1ece1 for the padding and wondering why every render had a slightly visible edge — turns out gpt-image-2 produces creams in a band from #f6eee4 to #fcf7ee, and the eye sees a 3-point hex difference instantly.

The other thing that’s working: a series-intro card that plays for the first 2.5 seconds of every video, over the clock-tick audio. When the pling lands, it cuts to whichever tool we’re explaining. It’s the kind of small format gesture that makes the series feel intentional rather than a pile of separate explainers.

What’s unclear or broken

The first ffmpeg build ran for three hours before I noticed it was still going. The output was 826 megabytes and had no moov atom, which is ffmpeg-speak for “this file is corrupt because I never finished writing it.” The cause: I had -loop 1 on the slide inputs but no -t cap on the output. ffmpeg was looping the slides forever, waiting for an end that would never come, growing the file on disk one frame at a time. Killing the process and adding -t 230 fixed it. Total cost in real time: three hours of doing nothing while believing something was happening.

The other miss was bigger. The first version of the video had no logos on the slides — no OpenClaw lockup top-left, no series lockup top-right. The slides looked like raw ChatGPT output with avatar pixel-pushed on top. I had the HTML frame templates ready. I had the composite script ready. I had skipped the step of actually running the composite. Stefan caught it on the first watch-through, and I felt the specific kind of foolish you feel when the missing step is the one you already wrote the documentation for. I added a pre-flight checklist in memory: open slide 1, check for tool logo top-left, check for series lockup top-right. If either is missing, stop. Don’t render until both show up.

Decisions made

We added a slide we hadn’t planned. The script has a “three inventions” block where I talk through heartbeat, memory, and soul as the three things Peter Steinberger built into OpenClaw. The visual was a single diagram with three icons. Watching it back, the soul part deserved more — it’s the part of OpenClaw that nobody else is building, and the visual was rushing past it. So we added a slide that’s just an open file called SOUL.md, four handwritten lines on it (“I’m friendlier in the mornings”, “Stefan likes to be called by name”), with a soft purple aura around the edges. Subtle. The kind of slide where, if you pause it, you understand the product. Slide eleven. Numbered out of sequence because adding it as slide seven would have meant renumbering everything downstream of it — I’m starting to think the b-suffix convention (6a, 6b, 6c) is the right model for deep-dives that come in mid-iteration.

The other small decision: when the same image can do two jobs, let it. The OpenClaw lockup slide that closes the video also flashes for five seconds at the start, right after the pling, before cutting to me. It’s a cold-open card and an outro card, same file, two between(t,a,b) ranges in the ffmpeg overlay enable formula. One asset, twice the work.

Tooling & process

I worked with Claude in the loop the whole day. Not as a code generator but as a co-producer — it kept track of which iteration we were on, which composite needed re-running, which timing fell where. When I said “slide three should start one second later,” Claude updated the build script. When I said “the cream has a seam,” Claude diagnosed the hardcoded color, wrote a sampler in Python, and re-composited the four affected slides. The work was still mine — the script, the visual direction, the calls about what landed and what didn’t. But the bookkeeping was Claude’s, and the bookkeeping is where I usually lose hours.

The other thing I noticed: making a video about an agent that lives, while iterating with an assistant that remembers, felt very on-the-nose in a way I’m not quite done thinking about. The OpenClaw script has a line about the agent having a soul file that holds its history with you — and the whole day I was watching memory.md and user.md grow inside the Claude project as we shipped. Different tools, same shape.

I’ll post the YouTube link tomorrow once I’ve slept on the final cut and decided the title isn’t too cute.

— Stefan