[conteúdo truncado devido ao tamanho máximo]
Teste e benchmarking de agentes LLM, incluindo testes comportamentais, avaliação de capacidade, métricas de confiabilidade e monitoramento de produção – onde mesmo os melhores agentes atingem menos de 50% em benchmarks do mundo real.
Esta skill fornece uma estrutura abrangente para testar, avaliar e monitorar agentes de IA, particularmente aqueles baseados em Modelos de Linguagem de Grande Escala (LLMs). Ela é crucial para desenvolvedores que constroem agentes autônomos, pois aborda a natureza estocástica e frequentemente imprevisível desses sistemas. A skill oferece metodologias como Avaliação por Teste Estatístico, Teste de Contrato Comportamental e Teste Adversarial, permitindo que os usuários vão além de simples verificações de aprovação/reprovação para entender a confiabilidade, consistência e comportamento de fronteira de seus agentes. Projetada para plataformas como Claude Code, Cursor e Codex, ela integra ferramentas como AgentBench, τ-bench e Langsmith, ajudando a preencher a lacuna entre o desempenho do mundo real e a avaliação de laboratório, onde mesmo agentes de ponta frequentemente falham em benchmarks complexos. Ao implementar esses padrões, os desenvolvedores podem detectar degradação de desempenho, garantir a segurança do agente e criar sistemas de IA mais robustos e dignos de confiança.
npx skills add https://github.com/SamuelChien/mega-skills-collection/tree/HEAD/agent-evaluation[conteúdo truncado devido ao tamanho máximo]
This repo contains 20 individual skills — each has its own dedicated page.
Security audit, hardening, threat modeling (STRIDE/PASTA), Red/Blue Team, OWASP checks, code review, incident response, and infrastructure security for any project.
Arquitecto de Soluciones Principal y Consultor Tecnológico de Andru.ia. Diagnostica y traza la hoja de ruta óptima para proyectos de IA en español.
Ingeniero de Sistemas de Andru.ia. Diseña, redacta y despliega nuevas habilidades (skills) dentro del repositorio siguiendo el Estándar de Diamante.
Estratega de Inteligencia de Dominio de Andru.ia. Analiza el nicho específico de un proyecto para inyectar conocimientos, regulaciones y estándares únicos del sector. Actívalo tras definir el nicho.
2D game development principles. Sprites, tilemaps, physics, camera.
3D game development principles. Rendering, shaders, physics, cameras.
Expert in building 3D experiences for the web - Three.js, React Three Fiber, Spline, WebGL, and interactive 3D scenes. Covers product configurators, 3D portfolios, immersive websites, and bringing depth to web experiences.
Accessibility audit skill for scanning, fixing, and verifying WCAG 2.2 Level A and AA compliance across React, Next.js, Vue, Angular, Svelte, and plain HTML codebases. Use when auditing accessibility, fixing a11y violations, checking color contrast, generating compliance reports, or integrating accessibility checks into CI/CD pipelines.
Ab Test Analyzer - Auto-activating skill for Data Analytics. Triggers on: ab test analyzer, ab test analyzer Part of the Data Analytics skill category.
A B Test Config Creator - Auto-activating skill for ML Deployment. Triggers on: a b test config creator, a b test config creator Part of the ML Deployment skill category.
When the user wants to plan, design, or implement an A/B test or experiment. Also use when the user mentions "A/B test," "split test," "experiment," "test this change," "variant copy," "multivariate test," "hypothesis," "conversion experiment," "statistical significance," or "test this." For tracking implementation, see analytics-tracking.
Acceptance Criteria Creator - Auto-activating skill for Enterprise Workflows. Triggers on: acceptance criteria creator, acceptance criteria creator Part of the Enterprise Workflows skill category.
Use when a coding task should be driven end-to-end from issue intake through implementation, review, deployment, and acceptance verification with minimal human re-intervention.
Accessibility Audit Runner - Auto-activating skill for Frontend Development. Triggers on: accessibility audit runner, accessibility audit runner Part of the Frontend Development skill category.
You are an accessibility expert specializing in WCAG compliance, inclusive design, and assistive technology compatibility. Conduct audits, identify barriers, and provide remediation guidance.
Web accessibility patterns for WCAG 2.2 compliance including ARIA, keyboard navigation, screen readers, and testing
Manage Discord channel access — approve pairings, edit allowlists, set DM/group policy. Use when the user asks to pair, approve someone, check who's allowed, or change policy for the Discord channel.
Create unified specification packages across Business, Development, and Design teams. Staged elaboration (L0 Vision → L1 Requirements → L2 Team Detail → L3 Acceptance Criteria) to build shared understanding. Does not write code.
Research a company or person and get actionable sales intel. Works standalone with web search, supercharged when you connect enrichment tools or your CRM. Trigger with "research [company]", "look up [person]", "intel on [prospect]", "who is [name] at [company]", or "tell me about [company]".
Validate messaging consistency across website, GitHub repos, and local documentation generating read-only discrepancy reports. Use when checking content alignment or finding mixed messaging. Trigger with phrases like "check consistency", "validate documentation", or "audit messaging".
ElevenLabs text-to-speech with mac-style say UX.
Oracle CLI second-model review/debug/refactor/design with selected files, dry-run token checks, API or browser engine.
Capture and automate macOS UI with the Peekaboo CLI.
Generate multiple radically different interface designs for a module using parallel sub-agents. Use when user wants to design an API, explore interface options, compare module shapes, or mentions "design it twice".
Build a throwaway prototype to flesh out a design — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route.
Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".