Communitygithub.com

Cetc2211/AcTR-app

使用 z-ai-web-dev-sdk 實現專業影片理解功能。當使用者需要分析影片內容、理解動作與時間序列、從影格中提取資訊、描述影片場景或進行基於影片的 AI 分析時,使用此技能。最佳化支援 MP4、AVI、MOV 及其他常見影片格式。

AcTR-app 是什麼?

此技能利用 z-ai-web-dev-sdk 套件提供專業的影片理解功能,讓 AI 模型能夠分析、描述並從影片內容中提取資訊,包括動作、時間序列及場景變化。它同時支援命令列介面(CLI)和軟體開發套件(SDK)兩種使用方式:CLI 適合快速影片摘要、一次性分析或簡單自動化;SDK 則適合多輪對話、複雜處理流程或生產環境應用。技能涵蓋影片場景理解、動作與行為偵測、事件時間線提取、內容分類、品質評估、內容審核及視覺逐字稿生成等進階用例。支援 MP4、AVI、MOV、WebM、MKV 及 FLV 等常見格式,提供靈活且強大的影片分析能力。

相容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/Cetc2211/AcTR-app/tree/HEAD/skills/video-understand

在你喜歡的 AI 中提問

開啟一個已預先載入此 Agent Skill 的新對話。

說明文件

影片理解技能

此技能使用 z-ai-web-dev-sdk 套件提供專業的影片理解功能,讓 AI 模型能夠分析、描述並從影片內容中提取資訊,包括動作、時間序列及場景變化。

技能路徑

技能位置:{project_path}/skills/video-understand

此技能位於你專案中的上述路徑。

參考腳本:在 {Skill Location}/scripts/ 目錄中提供了範例測試腳本,供快速測試與參考。請參閱 {Skill Location}/scripts/video-understand.ts 取得可運作的範例。

概述

影片理解專注於影片內容分析,提供以下功能:

  • 影片場景理解與描述
  • 動作與行為偵測
  • 時間序列分析
  • 影片中的事件偵測
  • 影片內容摘要
  • 場景變化偵測
  • 跨影格的人物與物體追蹤
  • 視聽內容分析(適用時)

重要:z-ai-web-dev-sdk 必須僅在後端程式碼中使用。切勿在客戶端程式碼中使用。

先決條件

z-ai-web-dev-sdk 套件已安裝。請依下方範例所示方式匯入。

命令列介面使用方式(適用於簡單任務)

對於快速的影片分析任務,您可以使用 z-ai 命令列介面替代編寫程式碼。這適用於簡單的影片描述、測試或自動化。

基本影片分析

# 從 URL 分析影片
z-ai vision --prompt "Summarize what happens in this video" --image "https://example.com/video.mp4"

# 注意:影片 URL 也請使用 --image 旗標
z-ai vision -p "Describe the key events" -i "https://example.com/presentation.mp4"

分析本機影片

# 分析本機影片檔案
z-ai vision -p "What activities are shown in this video?" -i "./recording.mp4"

# 將回應儲存至檔案
z-ai vision -p "Provide a detailed summary" -i "./meeting.mp4" -o summary.json

進階影片分析

# 包含思考過程的複雜場景理解
z-ai vision \
  -p "Analyze this video and identify: 1) Main events, 2) People and their actions, 3) Timeline of key moments" \
  -i "./event.mp4" \
  --thinking \
  -o analysis.json

# 動作偵測
z-ai vision \
  -p "Identify all actions performed by people in this video" \
  -i "./sports.mp4" \
  --thinking

串流輸出

# 串流影片分析結果
z-ai vision -p "Describe this video content" -i "./video.mp4" --stream

命令列參數

  • --prompt, -p <text>:必要 - 關於影片的問題或指令
  • --image, -i <URL or path>:選用 - 影片 URL 或本機檔案路徑(雖然名稱是 image,但也適用於影片)
  • --thinking, -t:選用 - 啟用鏈式思考推理以進行複雜分析(預設:停用)
  • --output, -o <path>:選用 - 輸出檔案路徑(JSON 格式)
  • --stream:選用 - 即時串流回應

支援的影片格式

  • MP4 (.mp4) - 最廣泛支援的格式
  • AVI (.avi) - Audio Video Interleave
  • MOV (.mov) - QuickTime 格式
  • WebM (.webm) - 網頁最佳化格式
  • MKV (.mkv) - Matroska 格式
  • FLV (.flv) - Flash Video 格式

何時使用命令列介面 vs 軟體開發套件

使用命令列介面時機:

  • 快速影片摘要
  • 一次性影片分析
  • 測試影片理解能力
  • 簡單自動化腳本
  • 產生影片描述

使用軟體開發套件時機:

  • 多輪對話討論影片
  • 複雜的影片處理流程
  • 具錯誤處理的正式環境應用
  • 自訂整合影片處理邏輯
  • 批次影片處理與自訂工作流程

建議做法

為獲得更好的本機影片效能與可靠性,請考慮:

  1. 將影片上傳至 CDN 並使用 URL
  2. 對於較短的影片,將關鍵影格轉換為圖片以加速分析
  3. 對於較長的影片,考慮分段或以間隔取樣

基本影片理解實作

單一影片分析

import ZAI from 'z-ai-web-dev-sdk';

async function analyzeVideo(videoUrl, prompt) {
  const zai = await ZAI.create();

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: prompt
          },
          {
            type: 'video_url',
            video_url: {
              url: videoUrl
            }
          }
        ]
      }
    ],
    thinking: { type: 'disabled' }
  });

  return response.choices[0]?.message?.content;
}

// 使用範例
const summary = await analyzeVideo(
  'https://example.com/presentation.mp4',
  'Summarize the key points presented in this video'
);

const actionDetection = await analyzeVideo(
  'https://example.com/sports.mp4',
  'Identify and describe all athletic actions performed in this video'
);

影片場景理解

import ZAI from 'z-ai-web-dev-sdk';

async function understandVideoScenes(videoUrl) {
  const zai = await ZAI.create();

  const prompt = `Analyze this video and provide:
1. Overall summary of the video content
2. Main scenes or segments (with approximate timestamps if possible)
3. Key people or characters and their roles
4. Important actions or events in chronological order
5. Setting and environment description
6. Overall mood or tone`;

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'enabled' } // 啟用以進行詳細分析
  });

  return response.choices[0]?.message?.content;
}

// 使用
const sceneAnalysis = await understandVideoScenes(
  'https://example.com/documentary.mp4'
);

動作與行為偵測

import ZAI from 'z-ai-web-dev-sdk';

async function detectActions(videoUrl, specificAction = null) {
  const zai = await ZAI.create();

  const prompt = specificAction
    ? `Identify all instances of "${specificAction}" in this video. For each instance, describe when it occurs and provide details about how it's performed.`
    : 'Identify and describe all significant actions and movements in this video. Include who is performing them and when they occur.';

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'enabled' }
  });

  return response.choices[0]?.message?.content;
}

// 使用
const runningActions = await detectActions(
  'https://example.com/sports.mp4',
  'running'
);

const allActions = await detectActions(
  'https://example.com/activity.mp4'
);

事件時間線提取

import ZAI from 'z-ai-web-dev-sdk';

async function extractTimeline(videoUrl) {
  const zai = await ZAI.create();

  const prompt = `Create a detailed timeline of events in this video:
- Identify key moments and transitions
- Note approximate timing (beginning, middle, end or specific timestamps if visible)
- Describe what happens at each key point
- Identify any cause-and-effect relationships between events

Format as a chronological list.`;

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'enabled' }
  });

  return response.choices[0]?.message?.content;
}

影片內容分類

import ZAI from 'z-ai-web-dev-sdk';

async function classifyVideo(videoUrl) {
  const zai = await ZAI.create();

  const prompt = `Classify this video content:
1. Primary category (e.g., educational, entertainment, sports, news, tutorial)
2. Sub-category or genre
3. Target audience
4. Content style (professional, casual, documentary, etc.)
5. Key themes or topics
6. Suggested tags (10-15 keywords)

Format your response as structured JSON.`;

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'disabled' }
  });

  const content = response.choices[0]?.message?.content;
  
  try {
    return JSON.parse(content);
  } catch (e) {
    return { rawResponse: content };
  }
}

進階使用案例

多輪影片對話

import ZAI from 'z-ai-web-dev-sdk';

class VideoConversation {
  constructor() {
    this.messages = [];
  }

  async initialize() {
    this.zai = await ZAI.create();
  }

  async loadVideo(videoUrl, initialQuestion) {
    this.messages.push({
      role: 'user',
      content: [
        { type: 'text', text: initialQuestion },
        { type: 'video_url', video_url: { url: videoUrl } }
      ]
    });

    return this.getResponse();
  }

  async askFollowUp(question) {
    this.messages.push({
      role: 'user',
      content: [
        { type: 'text', text: question }
      ]
    });

    return this.getResponse();
  }

  async getResponse() {
    const response = await this.zai.chat.completions.createVision({
      messages: this.messages,
      thinking: { type: 'disabled' }
    });

    const assistantMessage = response.choices[0]?.message?.content;
    
    this.messages.push({
      role: 'assistant',
      content: assistantMessage
    });

    return assistantMessage;
  }
}

// 使用
const conversation = new VideoConversation();
await conversation.initialize();

const initial = await conversation.loadVideo(
  'https://example.com/lecture.mp4',
  'What is the main topic of this lecture?'
);

const followup1 = await conversation.askFollowUp(
  'Can you explain the key concepts mentioned?'
);

const followup2 = await conversation.askFollowUp(
  'What examples were used to illustrate these concepts?'
);

影片品質評估

import ZAI from 'z-ai-web-dev-sdk';

async function assessVideoQuality(videoUrl) {
  const zai = await ZAI.create();

  const prompt = `Assess the quality of this video:
1. Visual quality (resolution, clarity, lighting) - Rate 1-10
2. Audio quality (if audio is present) - Rate 1-10
3. Camera work (stability, framing, composition) - Rate 1-10
4. Production value (editing, transitions, effects) - Rate 1-10
5. Content clarity (is the message clear?) - Rate 1-10
6. Pacing (too fast, too slow, just right)
7. Technical issues (artifacts, blur, audio sync, etc.)
8. Overall rating - 1-10
9. Specific recommendations for improvement

Provide detailed feedback for each criterion.`;

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'enabled' }
  });

  return response.choices[0]?.message?.content;
}

影片內容審核

import ZAI from 'z-ai-web-dev-sdk';

async function moderateVideo(videoUrl) {
  const zai = await ZAI.create();

  const prompt = `Review this video for content moderation:
1. Check for any inappropriate or sensitive content
2. Identify any potential safety concerns
3. Note any content that might violate common community guidelines
4. Assess age-appropriateness
5. Identify any copyrighted material visible (logos, brands, music)
6. Overall safety rating: Safe / Caution / Review Required

Provide specific examples for any concerns identified.`;

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'enabled' }
  });

  return response.choices[0]?.message?.content;
}

影片逐字稿生成(視覺描述)

import ZAI from 'z-ai-web-dev-sdk';

async function generateVisualTranscript(videoUrl) {
  const zai = await ZAI.create();

  const prompt = `Generate a detailed visual transcript of this video:
- Describe what's happening in each scene
- Note any text that appears on screen
- Describe important visual elements
- Mention any scene changes or transitions
- Include descriptions of people's actions and expressions

Format as a time-based narrative (e.g., "At the beginning...", "Then...", "Finally...").`;

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'disabled' }
  });

  return response.choices[0]?.message?.content;
}

運動影片分析

import ZAI from 'z-ai-web-dev-sdk';

async function analyzeSportsVideo(videoUrl, sport = null) {
  const zai = await ZAI.create();

  const prompt = sport
    ? `Analyze this ${sport} video in detail:
1. Identify players and their positions
2. Describe key plays and strategies
3. Note scoring events or important moments
4. Assess player performance
5. Identify any rule violations or fouls
6. Describe the pace and flow of the game`
    : `Analyze this sports video:
1. Identify the sport being played
2. Describe the key actions and plays
3. Note any scoring or significant events
4. Describe player movements and strategies
5. Overall assessment of the game or match`;

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'enabled' }
  });

  return response.choices[0]?.message?.content;
}

教學影片分析

import ZAI from 'z-ai-web-dev-sdk';

async function analyzeEducationalVideo(videoUrl) {
  const zai = await ZAI.create();

  const prompt = `Analyze this educational video:
1. Subject matter and difficulty level
2. Key learning objectives
3. Teaching methods used (lecture, demonstration, animation, etc.)
4. Clarity of explanations
5. Visual aids and their effectiveness
6. Logical flow of information
7. Suggested prerequisites for viewers
8. Target audience appropriateness
9. Overall educational value (1-10)
10. Suggestions for improvement`;

  const response = await zai.chat.completions.createVision({
    messages: [
      {
        role: 'user',
        content: [
          { type: 'text', text: prompt },
          { type: 'video_url', video_url: { url: videoUrl } }
        ]
      }
    ],
    thinking: { type: 'enabled' }
  });

  return response.choices[0]?.message?.content;
}

影片處理錯誤處理

import ZAI from 'z-ai-web-dev-sdk';

async function analyzeVideoWithFallback(videoUrl, prompt) {
  try {
    const zai = await ZAI.create();
    
    const response = await zai.chat.completions.createVision({
      messages: [
        {
          role: 'user',
          content: [
            { type: 'text', text: prompt },
            { type: 'video_url', video_url: { url: videoUrl } }
          ]
        }
      ],
      thinking: { type: 'disabled' }
    });
    
    return response.choices[0]?.message?.content;
  } catch (error) {
    console.error('Video analysis failed:', error.message);
    
    // 嘗試使用簡化的提示詞重試
    try {
      console.log('Retrying with simplified prompt...');
      const zai = await ZAI.create();
      
      const simplifiedResponse = await zai.chat.completions.createVision({
        messages: [
          {
            role: 'user',
            content: [
              { type: 'text', text: 'Describe the video key points' },
              { type: 'video_url', video_url: { url: videoUrl } }
            ]
          }
        ]
      });
      
      return simplifiedResponse.choices[0]?.message?.content;
    } catch (retryError) {
      return `Analysis failed: ${error.message}. Retry also failed: ${retryError.message}`;
    }
  }
}

使用影片分析的最佳實踐

  1. 影片長度:較短的影片(5 分鐘以下)可獲得最佳效果。較長的影片應考慮分段。
  2. 影片品質:解析度高、光線充足、穩定的影片可產生更準確的分析。
  3. 明確的提示詞:使用具體、結構化的提示詞以獲得詳細的分析。
  4. 時間參考:在提示詞中要求時間參考,以取得結構化的回應。
  5. 結合鍵盤:對於複雜分析,啟用 thinking: { type: 'enabled' }。
  6. 本地影片:對於本地影片,上傳至 CDN 或將其轉換為較小區塊。
  7. 錯誤處理:始終實作錯誤處理與重試邏輯,尤其是在正式環境中。
  8. 效能:在提示詞中要求較短的摘要,以縮短回應時間。

Individual skills in this repo

This repo contains 7 individual skills — each has its own dedicated page.

Cetc2211/AcTR-app

Generate a single-file Chinese atelier-style portfolio website (horizontal-row list + hover-reveal thumb interaction, art lishu typography "Alimama DaoLiTi" via CDN, light/dark theme, hash-routed detail pages). Takes a natural-language description of the studio (name, director, discipline, works) and outputs ONE working .html file. Runs in BOTH Claude Code (writes to ~/Desktop) and GLM / online chat (outputs HTML inline). Trigger phrases include "做个中式工作室作品集", "做个隶书作品集", "lishu-portfolio-gen ...", "做个白屋风格的作品集", or when user drags this skill folder into chat asking for an atelier-style site.

Cetc2211/AcTR-app

Premium calm-tech SaaS landing template — bilingual (CN/EN) literary voice over a dark forest-night palette with warm gold accent, Cormorant + Noto Serif SC, Three.js procedural forest hero, station-based scrolling narrative, custom cursor. Built for ambient-AI / creative-tool / philosophical SaaS, not loud commercial launch.

Cetc2211/AcTR-app

Pro-camera HUD mockup template — full-viewport black canvas wrapping a real video with monospace overlays (corner brackets, top status bar, side AF/exposure columns, bottom timecode + aperture/shutter/ISO row, vertical 8-filter wheel, animated reticle). For cinematographer portfolios, camera-app demos, LUT pack previews, filter-look auditions. NOT a TikTok/Reels mockup, NOT a storyboard, NOT a shot list, NOT a script document. The chrome wraps real footage — never a div placeholder.

Cetc2211/AcTR-app

Premium pre-launch waitlist page template — dark charcoal canvas with subtle vignette, drifting portrait video-thumbnail cards, fade-up cascades, glass nav, live-updating join counter, single email form with success/error states. Built for creative-tool / AI / niche-SaaS invite-only launches with hushed exclusive tone, not loud growth-marketing.

Cetc2211/AcTR-app

Route design-related HTML artifact tasks to the right artifact skill, reference, design system generation, or export skill.

Cetc2211/AcTR-app

Implement text-to-speech (TTS) capabilities using the z-ai-web-dev-sdk. Use this skill when the user needs to convert text into natural-sounding speech, create audio content, build voice-enabled applications, or generate spoken audio files. Supports multiple voices, adjustable speed, and various audio formats.

Cetc2211/AcTR-app

Implement AI-powered video generation capabilities using the z-ai-web-dev-sdk. Use this skill when the user needs to generate videos from text prompts or images, create video content programmatically, or build applications that produce video outputs. Supports asynchronous task management with status polling and result retrieval.

相關技能