Short Form Video Pipeline

Short-Form Video Clip Pipeline — Skill

by ericosiu · MIT license · ★ 3,615 on GitHub

python3 telemetry/versioncheck.py 2>/dev/null || true

# Short-Form Video Clip Pipeline — Skill

## Preamble (runs on skill start)

```bash
# Version check (silent if up to date)
python3 telemetry/version_check.py 2>/dev/null || true

# Telemetry opt-in (first run only, then remembers your choice)
python3 telemetry/telemetry_init.py 2>/dev/null || true
```

> **Privacy:** This skill logs usage locally to `~/.ai-marketing-skills/analytics/`. Remote telemetry is opt-in only. No code, file paths, or repo content is ever collected. See `telemetry/README.md`.

---

Extract viral short-form clips (TikTok, Reels, Shorts) from long-form YouTube videos. Handles download, transcription, AI segmentation, cutting, vertical cropping, and caption burn-in.

Overview

Extract viral short-form clips (TikTok, Reels, Shorts) from long-form YouTube videos. Handles download, transcription, AI segmentation, cutting, vertical cropping, and caption burn-in.

What you get

1080×1920resolution (9:16 vertical)
H.264 + AACencoding
Word-highlighted captionsburned in
Ready for direct upload to TikTok, Reels, or Shorts

Manual install (npx, ZIP)

Claude Codenpx skills add ericosiu/ai-marketing-skills --skill short-form-pipeline -g -a claude-code

Claude Code (degit)npx degit ericosiu/ai-marketing-skills/short-form-pipeline#main ~/.claude/skills/short-form-pipeline

Claude.ai / desktopZip the folder and upload it under Settings → Capabilities → Skills.

Codex, Cursor, other agentsPoint them at SKILL.md. The other files are plain text.

Connects with Agent Alley MCP · agentalley.io/mcp

Files included

SKILL.md3.9 KB

Read SKILL.md here

ericosiu/main1 file shown

SKILL.md3.9 KB

Browse every file on GitHub↗

SKILL.md

Show the full text103 lines

short-form-pipeline/SKILL.md103 lines · 3.9 KB

PreviewCode

Outline

RawView on GitHub

Short-Form Video Clip Pipeline — Skill

Preamble (runs on skill start)

# Version check (silent if up to date)
python3 telemetry/version_check.py 2>/dev/null || true

# Telemetry opt-in (first run only, then remembers your choice)
python3 telemetry/telemetry_init.py 2>/dev/null || true

Privacy: This skill logs usage locally to ~/.ai-marketing-skills/analytics/. Remote telemetry is opt-in only. No code, file paths, or repo content is ever collected. See telemetry/README.md.

Extract viral short-form clips (TikTok, Reels, Shorts) from long-form YouTube videos. Handles download, transcription, AI segmentation, cutting, vertical cropping, and caption burn-in.

Prerequisites

yt-dlp and ffmpeg installed
ANTHROPIC_API_KEY environment variable set
Python dependencies from requirements.txt installed
Optional: mediapipe and opencv-python for face-detected smart crop

Quick Start

Single video → clips

python3 scripts/shortform_pipeline.py \
  --url "https://www.youtube.com/watch?v=VIDEO_ID" \
  --max-clips 3 \
  --output-dir ./output

Standalone clipper (no Claude, heuristic scoring)

python3 scripts/video_clipper.py --url "https://www.youtube.com/watch?v=VIDEO_ID"

Pipeline Overview

Download — yt-dlp fetches video + auto-generated VTT captions
Transcribe — Whisper generates word-level timestamps (falls back to YouTube captions)
Segment — Claude identifies 2–5 best 30–60s moments with hook scoring ≥7/10
Cut Verification — Second Claude pass verifies each clip ends on a complete thought
Cut — FFmpeg extracts each clip from the source video
Vertical Crop — Layout-aware 16:9 → 9:16 conversion with face detection
Caption Burn — TikTok-style word-highlighted captions (ASS format) burned in

Key Files

File	Purpose
`scripts/shortform_pipeline.py`	Full pipeline: download → segment → cut → crop → caption
`scripts/video_clipper.py`	Standalone clipper with heuristic scoring (no Claude needed)
`scripts/clip_sender.py`	Helper for clip delivery and review workflow

Layout-Aware Cropping

The pipeline handles four video layouts differently:

talking_head — Face-detected center crop using MediaPipe; audio panning fallback
screen_share_overlay — Stacks screen content on top, webcam bubble on bottom
side_by_side — Stacks screen on top, presenter face on bottom
gallery_view — Crops to active speaker quadrant

Claude outputs a layout_hint for each segment during segmentation.

Customization

Voice patterns

Edit VOICE_PATTERNS in video_clipper.py to match your creator's speech patterns. These boost clip scoring for authentic-sounding segments.

Segmentation prompt

The Claude prompt in shortform_pipeline.py can be customized:

Adjust hook_strength minimum (default: 7/10)
Change target duration range (default: 30–60s)
Modify layout hint options

Crop tuning

In video_clipper.py:

scale_factor — Zoom level for single face (default: 1.08)
desired_face_y — Target face position in frame (default: upper 35%)

Output

Each clip is output as:

1080×1920 resolution (9:16 vertical)
H.264 + AAC encoding
Word-highlighted captions burned in
Ready for direct upload to TikTok, Reels, or Shorts

Troubleshooting

FFmpeg filter_complex error: Don't use -c:v copy with -filter_complex. Only -c:a copy is safe.
Wrong output resolution: Always crop before scaling. Verify with ffprobe -show_entries stream=width,height.
Caption sync issues: Run Whisper on the cut clip, not the source episode.
TikTok upload fails: Ensure H.264 + AAC encoding. Add -c:v libx264 -c:a aac if needed.
Clip too long: Claude sometimes overshoots. The pipeline auto-trims clips >90s to 75s.

1	# Short-Form Video Clip Pipeline — Skill
2
3	## Preamble (runs on skill start)
4
5	```bash
6	# Version check (silent if up to date)
7	python3 telemetry/version_check.py 2>/dev/null \|\| true
8
9	# Telemetry opt-in (first run only, then remembers your choice)
10	python3 telemetry/telemetry_init.py 2>/dev/null \|\| true
11	```
12
13	> Privacy: This skill logs usage locally to `~/.ai-marketing-skills/analytics/`. Remote telemetry is opt-in only. No code, file paths, or repo content is ever collected. See `telemetry/README.md`.
14
15	---
16
17	Extract viral short-form clips (TikTok, Reels, Shorts) from long-form YouTube videos. Handles download, transcription, AI segmentation, cutting, vertical cropping, and caption burn-in.
18
19	## Prerequisites
20
21	- `yt-dlp` and `ffmpeg` installed
22	- `ANTHROPIC_API_KEY` environment variable set
23	- Python dependencies from `requirements.txt` installed
24	- Optional: `mediapipe` and `opencv-python` for face-detected smart crop
25
26	## Quick Start
27
28	### Single video → clips
29
30	```bash
31	python3 scripts/shortform_pipeline.py \
32	--url "https://www.youtube.com/watch?v=VIDEO_ID" \
33	--max-clips 3 \
34	--output-dir ./output
35	```
36
37	### Standalone clipper (no Claude, heuristic scoring)
38
39	```bash
40	python3 scripts/video_clipper.py --url "https://www.youtube.com/watch?v=VIDEO_ID"
41	```
42
43	## Pipeline Overview
44
45	1. Download — yt-dlp fetches video + auto-generated VTT captions
46	2. Transcribe — Whisper generates word-level timestamps (falls back to YouTube captions)
47	3. Segment — Claude identifies 2–5 best 30–60s moments with hook scoring ≥7/10
48	4. Cut Verification — Second Claude pass verifies each clip ends on a complete thought
49	5. Cut — FFmpeg extracts each clip from the source video
50	6. Vertical Crop — Layout-aware 16:9 → 9:16 conversion with face detection
51	7. Caption Burn — TikTok-style word-highlighted captions (ASS format) burned in
52
53	## Key Files
54
55	\| File \| Purpose \|
56	\|------\|---------\|
57	\| `scripts/shortform_pipeline.py` \| Full pipeline: download → segment → cut → crop → caption \|
58	\| `scripts/video_clipper.py` \| Standalone clipper with heuristic scoring (no Claude needed) \|
59	\| `scripts/clip_sender.py` \| Helper for clip delivery and review workflow \|
60
61	## Layout-Aware Cropping
62
63	The pipeline handles four video layouts differently:
64
65	- `talking_head` — Face-detected center crop using MediaPipe; audio panning fallback
66	- `screen_share_overlay` — Stacks screen content on top, webcam bubble on bottom
67	- `side_by_side` — Stacks screen on top, presenter face on bottom
68	- `gallery_view` — Crops to active speaker quadrant
69
70	Claude outputs a `layout_hint` for each segment during segmentation.
71
72	## Customization
73
74	### Voice patterns
75	Edit `VOICE_PATTERNS` in `video_clipper.py` to match your creator's speech patterns. These boost clip scoring for authentic-sounding segments.
76
77	### Segmentation prompt
78	The Claude prompt in `shortform_pipeline.py` can be customized:
79	- Adjust `hook_strength` minimum (default: 7/10)
80	- Change target duration range (default: 30–60s)
81	- Modify layout hint options
82
83	### Crop tuning
84	In `video_clipper.py`:
85	- `scale_factor` — Zoom level for single face (default: 1.08)
86	- `desired_face_y` — Target face position in frame (default: upper 35%)
87
88	## Output
89
90	Each clip is output as:
91	- 1080×1920 resolution (9:16 vertical)
92	- H.264 + AAC encoding
93	- Word-highlighted captions burned in
94	- Ready for direct upload to TikTok, Reels, or Shorts
95
96	## Troubleshooting
97
98	- FFmpeg filter_complex error: Don't use `-c:v copy` with `-filter_complex`. Only `-c:a copy` is safe.
99	- Wrong output resolution: Always crop before scaling. Verify with `ffprobe -show_entries stream=width,height`.
100	- Caption sync issues: Run Whisper on the cut clip, not the source episode.
101	- TikTok upload fails: Ensure H.264 + AAC encoding. Add `-c:v libx264 -c:a aac` if needed.
102	- Clip too long: Claude sometimes overshoots. The pipeline auto-trims clips >90s to 75s.
103