Darryl Schaefer
← All projects

Case study

Media generation platform

Orchestrating image, text, and audio into repeatable long-form video production.

2021–presentFull-stack / platform
Next.jsTypeScriptDockerFFmpegPuppeteerCloudflare R2Vast.aiOpenAIInworld TTS

In 2021, I committed full-time to a product thesis: generative AI would make text, narration, and imagery dramatically cheaper, but turning those assets into complete long-form media would still require reliable orchestration, rendering, and distribution. I built the platform around that missing production layer, using audiobook, read-along, and educational YouTube channels as the initial testing ground.

A Scandal in Bohemia

Illustrated full-cast Sherlock Holmes; text highlights in time with the narration.

Video crossword

Themed mini-crossword solved on a timer.

The Problem

Testing a YouTube concept requires a catalog, not a single demonstration. Producing enough long-form videos manually to evaluate an idea makes experimentation slow and expensive.

I built the platform to reduce the path from concept to 30 published videos, then reuse the same orchestration and rendering infrastructure across books, language learning, games, and other formats.

Product Walkthrough

This walkthrough follows two audiobook projects from scheduling through a shared production template, persisted production contracts, parallel containerized GPU workers, visual review, and YouTube delivery.

Two projects from queue to delivery

The operator queues a two-project batch; the choreography layer creates a production contract for each project, triggers remote renderers, tracks completion, previews synchronized output, and delivers the publishing assets.

Pipeline Architecture

The platform separates production planning from execution. The choreography application turns a configured project into a production contract, stores that contract and its assets in Cloudflare R2, and launches a remote Docker renderer to complete the order.

Progress is persisted throughout the workflow. If a local process or remote instance stops, the system can inspect the existing project state and continue from completed work rather than rebuilding the project from the beginning.

Choreography

The application validates the project configuration, prepares the required text, audio, imagery, timing, and publishing data, and assembles them into a production contract. That contract defines what the finished video should contain without prescribing how every frame must be rendered.

Once the contract is ready, the application uploads it to R2 and starts a remote rendering job.

Rendering

The Docker container downloads the production contract, checks which outputs already exist, and distributes the remaining work across Node, FFmpeg, and Puppeteer workers.

For frame-based formats, Puppeteer renders the visual state for each frame while FFmpeg handles audio, video assembly, and muxing. The completed video and supporting assets are uploaded, and the publishing workflow can deliver the result to YouTube.

Recovery and Reuse

Because project state and intermediate artifacts are stored outside the rendering instance, a failed instance can be replaced without restarting the entire order. Retaining reusable chunks also makes it possible to create alternate edits, remove intros or outros, and assemble longer compilations without regenerating the source material.

Case Study: Learn American English

I created an all-new channel concept — Learn American English — featuring read-along text designed for English B1 learners who want to learn while being exposed to American ideas.

I queued all 30 projects in a single session. The platform produced 8 hours and 51 minutes of finished media within 24 hours, averaging 17 minutes and 42 seconds per video. The completed videos were scheduled for publication between July 2 and August 1.

The run required approximately 12 minutes of hands-on operation: the initial queueing session and one brief recovery session. No editor manually recorded narration, synchronized text, or keyframed the videos.

Videos produced
30
Finished media
8h 51m
Production time
Within 24 hours
Metered spend
$21.47
Cost per video
$0.72
Hands-on operation
≈12 minutes

Midway Mud — production output

A finished read-along excerpt from the 30-video production run. Narration timing is converted into a frame-deterministic contract that drives synchronized highlighting and passage progression without manual editing.

What the Stress Test Revealed

Running so many videos through at once was the first time this concept was stress-tested at that scale.

A subset of projects did not complete delivery on their first remote instance and required a later restart. Because project state and completed artifacts were persisted, the affected jobs could continue without rebuilding the entire project. I did not retain enough run-level telemetry to report an exact retry count, which exposed an observability gap I would address in the next version.

  • Durable orchestration telemetry — Persist per-project attempts, host failures, recovery time, and spend so reliability can be analyzed after a run rather than reconstructed from transient logs.
  • Push more processing to Docker — We normalize audio locally today. That isn't intensive for a few projects, but with a half-dozen running at once it jams up. Uploading earlier and letting Docker handle normalization alongside everything else would have removed most of the local background processing time.

Cost

Measured provider costs for the 30-video run:

  • OpenAI — $9.66 batch · $0.32 / video
  • Illustration (Replicate) — $4.80 batch · $0.16 / video
  • Inworld TTS — $4.92 batch · $0.16 / video
  • Vast.ai remote compute — $2.09 batch · $0.07 / video
  • Total — ~$21.47 batch · ~$0.72 / video

Total metered API and remote-compute spend was $21.47—approximately $0.72 per video or $2.43 per finished hour. The figure includes the measured OpenAI, image-generation, TTS, and Vast.ai charges associated with the batch.

Distribution Outcome

As of August 24, the original 30 videos had generated 287 views, 13.6 watch hours, and 10,509 impressions. The catalog continued receiving approximately 10–20 views every 48 hours, but it did not achieve commercially meaningful traction. The experiment validated production throughput and economics; it did not establish product-market fit.