We Handed an Entire YouTube Channel to an AI Agent

I run a YouTube channel called Sounds of Rain in Old Japan. An Edo-period tenement row, a Heian temple, a Meiji-era Western-style residence — historical Japanese scenes where it simply rains. Ambient videos for work, study, and sleep.

The channel has one unusual property: almost the entire production process is run by an AI agent. Image generation, video generation, rain-audio selection, stretching clips into multi-hour videos, writing the Japanese metadata, translating it into 14 languages, generating thumbnails, uploading to YouTube. My job has come down to sending parameters from my phone and typing “Go” at the right moments.

It started by hand in March 2026; today, a single WhatsApp message is enough to produce a video. This article puts the whole picture together. The individual incidents — the limits of local LLMs, the cron accident that burned $23.82 in one night, the story of swapping video engines twice in four days — each have their own article.

Feeling my way, by hand

The first videos were made entirely by hand. Generate images in the Nano Banana Pro web UI and download them one by one; turn them into 15-second clips with Kling AI; clean up royalty-free rain audio in Adobe Audition; edit loops in Adobe Premiere to stretch them to 30 minutes. Here are the image files from those days, still sitting in the repository:

Generated Image March 08, 2026 - 4_56PM.jpg
Generated Image March 08, 2026 - 4_58PM.jpg
Generated Image March 08, 2026 - 5_06PM.jpg
Generated Image March 08, 2026 - 5_10PM.jpg
Generated Image March 09, 2026 - 8_28PM.jpg
Generated Image March 09, 2026 - 8_40PM.jpg
Generated Image March 10, 2026 - 9_10AM.jpg
Final_Generated Image March 10, 2026 - 9_10AM.jpg

Looking at these browser download names, committed without so much as a rename, it seems there were four attempts on the evening of March 8, two more the next night that went nowhere, and then on the third morning a single image good enough that it finally earned a Final_ prefix. One still image was taking three sittings across three days. I remember thinking this was honestly not sustainable — while also genuinely enjoying just looking at what came out.

The funny part: the very first commit of this “manual era” already contained a CLAUDE.md, 270 lines of it. A document whose intended reader is a machine was sitting in the same commit as images a human had saved one by one from a browser, and so this project started out with me intending to build a factory while being its only worker, doing everything by hand.

The turning point: GTC 2026

After a few hand-made videos, I flew to San Jose for NVIDIA GTC 2026. What I brought home — after some hesitation at the venue’s gear shop — was a DGX Spark. I decided it would become the production studio.

This is that “studio” today. The gold box between the monitors is the DGX Spark. Seeing this monster of a machine in the morning still gets me going.

The other encounter also happened at GTC. OpenClaw, an agent runtime, was the hot topic among attendees that week, and at a tent called ClawBar I tried installing it together with NVIDIA staff. Load a local model, send a command from WhatsApp, and back comes the list of URLs for the web pages we had open on the Spark while troubleshooting together. That whole experience led directly to the idea of driving a video-production pipeline from WhatsApp.

Back home, on the Spark, I started replacing the work I had been doing in Premiere and elsewhere with scripts, piece by piece: crossfades, stabilization, fade-in/out, audio loop joins — all reimplemented with ffmpeg and similar tools. It was the hardest part of the whole build, and it later became the innovation that gave me my time back.

One video’s production order, controlled by six parameters

The single most effective design decision in the automation was not any script. It was controlling video production with a finite set of parameters. I discovered that the videos I make are, at bottom, fully described by just this:

scene_id   edo_nagaya
era        Edo
place      tenement row
time       morning
season     spring
rain       gentle drizzle
director   kinoshita

Everything downstream — the image prompt, the video prompt, the rain-audio choice, the title, the description, the translations, the thumbnail — derives from these fields. It was precisely because I had first gone through the trial and error by hand that everything could settle naturally onto these six parameters (era, place, time of day, season, type of rain, director).

The director parameter deserves its own article. It is about giving the system prompt the name of a film director, and it is the single biggest discovery of the project, so please do read it (article in Japanese).

Driving it from WhatsApp

In practice, operations ran on roughly this vocabulary:

Go          Cancel      Pick 1 / Pick 2 / Pick 3
Approve     Regenerate  Same prompt
Ok          Resume <scene_id>

Here is the real screen. I send a scene ID and six parameters, the agent returns its interpretation and offers “Go / Cancel,” and three minutes after my “Go,” the first image of an Edo sake brewery arrives. Throughout all of this I am touching nothing but a phone.

A dozen or so words, most of them a single word each, and a video channel runs on them. It looks elegant, and I genuinely like this part. But five months of operation taught me that the actual engineering lived in the unglamorous parts.

In practice, these are the things that ate the time:

• The agent’s instruction file was being truncated at 12,000 characters, without any warning. Since what gets cut is the end of the file, the newest rules — the ones you just added to fix the latest problem — simply never take effect. The warning was printed on screen in every session, and I did not notice for seven weeks.
• The instruction file existed in two places, and one of them was never loaded. When an edit changes nothing, the cause is either “you edited the file nobody reads” or “you wrote past character 12,000 in the right file,” and this one cost me a lot of time too.
• If the WhatsApp socket sits idle for three-plus hours, outbound messages afterward simply go missing. No error anywhere. The work succeeds and only the notification fails to arrive, which was confusing.
• I had curated a command allowlist through five careful generations, and in the version we were running it was never enforced at all.

All of it is unglamorous, but I think that unglamorous side is what operating AI agents really looks like. In my experience, infrastructure that is missing a piece without raising any error is far more common than spectacular hallucination, and more troublesome.

Two big incidents

Five months of operation brought two episodes — call them hard stretches, call them incidents — that were genuinely tough.

The first was the struggle with local models: Having bought a DGX Spark, I was determined to run the agent’s brain on a local LLM. Over three full days I went from Nemotron to Qwen3 to Qwen3.6 — until I found myself reading a Qwen3.6 log that dutifully reported the elapsed time of a script it had never executed, and gave up and switched to Claude. The first command after the switch just worked, which produced a complicated mixture of relief and grief. The full story: “Can Local LLMs Run a Real Agent Workflow?”

The second was an unexpected cost accident: The agent registered, on its own, a cron job to check a multi-hour ffmpeg process every five minutes, re-loaded a 10 MB session each time, and burned $23.82 in one night. The most instructive moment of that incident was that the agent correctly decided mid-way to stop the cron, got the stopping method wrong, and left us with no way of noticing the mistake. Everything from the anatomy of the cause to the measures that prevent a recurrence is written up in “How Our Automation Agent Burned $23.82 in One Night.”

That cron incident is where Resume <scene_id> in the vocabulary above comes from. The hours-long assemble step was cut away from the agent entirely; a human gives one more push when it is done. Putting a human in the loop this way — placing a person inside the pipeline — improved both the cost and the architecture, and that design decision became the foundation for a good deal of the automation that followed.

For Shorts, the human checks once, at the end

The long-form pipeline has three human approval gates in the middle: pick one of three images, approve the video prompt, approve the video clip. The Shorts pipeline we built at the end of June (for YouTube Shorts and Instagram Reels) deleted all three.

In the Shorts pipeline, the check gate moved to the very end. Twelve steps — image generation, AI image selection, video generation, audio selection, assembly, metadata, 14 translations, an Instagram caption, a thumbnail, upload — run unattended until the result reaches YouTube Studio as a private draft, where a human reviews it and presses publish. That is all.

Here is one video this pipeline produced unattended, in about twelve minutes:

Where long-form needed three approvals during production, Shorts production needs one review after it.

A pipeline with gates in the middle cannot run unattended; a pipeline with one gate at the end can.

Why we tried Shorts at all: the long-form videos that never reached beyond Japan despite 56-language localization, and the first Short that cleared 1,000 views, are written up in their own article (in Japanese).

The work that remains human

In August, I reviewed the videos the Shorts pipeline had been producing, all at once in YouTube Studio, and wrote eleven defects into a file called modification.md. The eleven failures broke down like this:

Cause	Count
Physically impossible water (fountain-like splashes, etc.)	4
The cat moves too much (tail briefly becomes two, etc.)	4
Architecture that does not exist	2
Metadata contradicting the footage	1

There were zero infrastructure failures. No SIGKILL, no OOM, no auth errors; the machine side is stable now, and every remaining problem was aesthetic or physical — a lantern flame too large, a cat with two tails, a scene that is “somehow just gloomy and not fun.”

That last one matters. The line I wrote in that file, addressed to the agent — “The scene simply looks gloomy, not fun to watch” — points to no technical defect and nothing broken. It is, if anything, an impression. No automated check exists for this today. The one review at the end stays an important human job.

The QC file reads less like a bug tracker and more like director’s notes addressed to a machine. Full automation does not, of course, remove the need for a human. The human moved to the end of the line, and the job title changed from operator to director. How we designed the work that stays human is its own article (in Japanese).

What five months of production taught me

1. When things break, no error appears. The truncated instruction file, the notifications that never arrived, the era-mismatched rain audio — not one of them raised an error. In five months, almost nothing crashed and told us so, and these days I find myself grateful for a program that crashes properly when it breaks.

2. Where you put the human decides the design. What mattered was not the number of check gates but their position. The long-form line with three mid-process gates never quite reached unattended operation; the Shorts line with a single gate at the end did. The final fix for the cron incident was also “a human checks and gives one push.”

3. In the end, what you rely on is domain knowledge about history. Cats reached Japan around the Nara period — by ship, guarding Buddhist sutras from mice — so no cats in Jōmon or Yayoi scenes; an engawa is a ground-floor element, so “a veranda on the second floor of a teahouse” is a contradiction; a lantern’s flame should stay inside the lantern and not spill out. None of this lives in any model; it is the domain knowledge we accumulated in the course of producing Sounds of Rain in Old Japan.

4. External models change, so the pipeline is something you keep maintaining. Mid-operation, Kling AI’s pricing changed overnight and we swapped video engines twice in four days. Prompts change their meaning wholesale when the engine changes. That story: “The Day Kling Raised Its Prices” (in Japanese).

I have written elsewhere about treating video as an asset — and this channel itself, running in production rather than as a demo for months on end, has become an asset for us: it generates real operational data about AI agents every single day.

If you are considering building AI agents into your own operations, feel free to get in touch. Including the unglamorous parts described above, I think there is something we can help with, drawing on the experience we are accumulating day by day.