Week 3.2: Advanced Video Generation & Editing

Week 3.2 Resources: Advanced Video Generation & Editing

Required Software & Accounts

Before starting, ensure you have the following tools and accounts ready.

• ChatGPT: Used for generating storyboards and character sheets.
  • Visit ChatGPT 
• Aidolons: The primary platform for video generation and storyboarding.
  • Aidolons 
• CapCut: Used for editing and splicing the generated video clips.
  • Download CapCut 
• Instagram: Used for posting the final video.
  • Instagram 

Video Generation Models

We experiment with several models in this lesson. Here is a breakdown of their capabilities and costs:

• MiniMax H3 Turbo:
  • Pros: Very fast, cheap. Good for rough drafts and simple prompts.
  • Cons: Struggles with complex prompts and multi-character coherence.
• Seedance 2.0 Mini:
  • Pros: Slightly more capable than MiniMax for certain tasks.
  • Cons: Often produces messy or incoherent results for complex scenes.
• Wan 3.0:
  • Pros: Good balance of quality and cost. Can generate up to 30-second clips.
  • Cons: More expensive than MiniMax.
• Seedance 2.5:
  • Pros: Best visual style, good sound effects, understands complex prompts and jokes better than the others.
  • Cons: The most expensive option.

Class Prompts

You can copy and paste these prompts directly into ChatGPT or Aidolons to replicate the lesson.

1. Generating Character Sheets (ChatGPT)

Prompt to extract frames and create character sheets:

grab multiple frames from this video, then create 2 character sheets, one for the man, another for the woman, both should be 16:9, front back left right, same angles for face close ups, no text, blank white background

Refining the character sheet (if the model gets it wrong):

no, a younger couple. I've included references of the characters

Refining the frames used for extraction:

the woman doesn't look quite right. grab the frames at 4 and 7 seconds and feed those into the image model and try again

2. Storyboard Prompts (ChatGPT)

Initial prompt for a dramatic story:

Give me a 15 second dramatic story featuring a man and a woman in their living room, and make me cry.

Refining the story to focus on the relationship:

try again, but make it about their relationship

Fixing a bad line of dialogue:

"No. 'It' was every night I waited for you to notice I was lonely." - this line sucks

Explaining why the line is bad (if the model is confused):

so that's it" the implication is more like "so we're really broken up now?" so when she says no, it's confusing

3. Rendering Prompts (Aidolons)

Rendering the revised scene with MiniMax:

Read the changes I made, then render with minimax h3

Rendering a specific scene with Seedance 2.5:

render again in seedance 2.5

Rendering a specific scene with different parameters:

do one more in seedance 2.5, 9:16 this time, and make it 20 seconds