Week 4.2: Systematizing AI Video Workflows

Week 4.2 Resources: Systematizing AI Video Workflows

Overview

In this session, we take the conversational workflow established in Week 4.1 and turn it into a robust, repeatable system using Markdown-based workflow files. We break down the process into modular sub-workflows (text assets, images, audio, assembly) and create a master workflow to orchestrate them all in parallel.

Required Tools & APIs

Ensure you have the following set up from previous weeks:

Claude Code: The CLI agent we are using to build and execute the system.
FFmpeg: Required for video assembly. (If not installed, ask Claude: install ffmpeg using chocolatey).
ElevenLabs API: For text-to-speech generation.
Image Generation API: (e.g., OpenAI/DALL-E 3) configured in your tools file.

Workflow File Structure

We are moving away from single-file prompts to a structured folder system.

Folder Structure:

workflows/
- create_short_form_video/
  - create_text_assets.md
  - generate_images.md
  - generate_audio.md
  - assemble_video.md
- create_short_form_video.md (The Main Controller)

Class Prompts & Workflow Definitions

You can copy these workflow definitions into your project files.

1. Create Text Assets Workflow

File: workflows/create_short_form_video/create_text_assets.md


# Inputs
project_name: $project_name

# Steps
1. READ docs/tools.md
2. READ docs/short_form_video.md
3. [data_folder]: projects/[project_name]/data/

4. Based on the research doc, choose the 5 most interesting facts and save those to [data_folder]/chosen-topics.md
5. Based on our chosen topics write a short form video script. 1 sentence per line.
  - save it to [data_folder]/script.md
6. Now create [data_folder]/segments.yaml. It will be a list of segment objects...
  - include image prompts relative to project root
  - use "image" for the key

2. Generate Images Workflow

File: workflows/create_short_form_video/generate_images.md


# Inputs
project_name: $project_name

# Read First
- projects/[project_name]/data/segments.yaml

# Steps
1. For each segment in segments.yaml
  - create projects/[project_name]/media/[id].png
  - aspect ratio 9:16
2. Update projects/[project_name]/data/segments.yaml
  - add the relative path to the image relative to project root
  - use "image" for the key

3. Generate Audio Workflow

File: workflows/create_short_form_video/generate_audio.md


# Inputs
project_name: $project_name

# Read First
- projects/[project_name]/data/script.md

# Steps
1. For each line in the script
  - use your eleven labs tools to create audio for that script
  - save them to projects/[project_name]/media/[line number][first few words].mp3
2. create projects/[project_name]/data/lines.yaml
  - a list of objects that include
    - the line text
    - path to the audio relative to project root

4. Assemble Video Workflow

File: workflows/create_short_form_video/assemble_video.md


# Inputs
project_name: $project_name

# Read First
- projects/[project_name]/data/segments.yaml
- projects/[project_name]/data/lines.yaml

# Steps
1. use ffmpeg to combine the images and audio into a video
  - combine all the line audios in order
  - display the image for each segment for the combined duration...
  - make sure the dimensions of the video match the dimensions of the images
  - use a codec that works with windows
  - save result to output/[project_name].mp4

5. Main Controller Workflow

File: workflows/create_short_form_video.md


# Inputs
topic: $ARGUMENTS

# Steps
1. run workflows/create_short_form_video/create_text_assets.md
  - give the agent the $project_name

2. run 2 parallel subagents
  1. The first one will run workflows/create_short_form_video/generate_images.md
    - give the agent the $project_name
  2. The second one will run workflows/create_short_form_video/generate_audio.md
    - give the agent the $project_name

3. wait for the agents to finish their work

4. with another subagent, run workflows/create_short_form_video/assemble_video.md
  - give the agent the $project_name

Terminal Commands to Execute

To run the full system, use the slash command in Claude Code pointing to your main workflow file and pass in your topic.

Run the Generator:


/create_short_form_video mermaids

(Note: Replace “mermaids” with any topic you wish to generate).

To Fix Audio/Video Codec Issues (If video won’t play):


convert it to a codec that works with windows

Git Commands:


commit

Source Code

The complete source code for Week 4 (including the tools, workflows, and .claude folders) is available for download.

Download Week 4 Files

After downloading, extract the zip file and create a .env file with your API keys.