Aidez les utilisateurs à créer et exécuter des évaluations d'IA. À utiliser lorsque quelqu'un construit des évaluations pour des produits LLM, mesure la qualité des modèles, crée des cas de test, conçoit des rubriques ou tente de mesurer systématiquement la qualité des sorties d'IA.
Cette compétence, refoundai/ai-evals, est conçue pour aider les développeurs et les utilisateurs d'agents IA à créer et exécuter des évaluations d'IA. Elle est particulièrement utile lors du développement de produits basés sur des modèles de langage (LLM), où la mesure systématique de la qualité des sorties est cruciale. La compétence prend en charge la création de cas de test, la conception de rubriques d'évaluation et l'exécution de tests automatisés pour évaluer la performance des modèles. Basée sur la documentation SKILL.md, elle fonctionne en fournissant un cadre structuré pour définir des métriques de qualité, configurer des expériences répétables et interpréter les résultats d'évaluation. Cela permet aux équipes de développement de réduire les biais subjectifs et d'améliorer la fiabilité de leurs systèmes d'IA. La compétence est compatible avec divers outils d'évaluation d'IA et peut être intégrée dans des pipelines CI/CD pour une validation continue.
npx skills add https://github.com/refoundai/lenny-skills/tree/main/skills/ai-evalsAidez les utilisateurs à créer et exécuter des évaluations d'IA. Utilisez cette compétence lorsque quelqu'un construit des évaluations pour des produits LLM, mesure la qualité des modèles, crée des cas de test, conçoit des rubriques ou tente de mesurer systématiquement la qualité des sorties d'IA.
This repo contains 20 individual skills — each has its own dedicated page.
Help users define AI product strategy. Use when someone is building an AI product, deciding where to apply AI in their product, planning an AI roadmap, evaluating build vs buy for AI capabilities, or figuring out how to integrate AI into existing products.
Help users synthesize and act on customer feedback. Use when someone is analyzing NPS responses, processing support tickets, reviewing user research, synthesizing feedback from multiple channels, or trying to identify patterns in customer input.
Help users apply behavioral science to product design. Use when someone is designing for habit formation, reducing friction, applying psychology to UX, increasing retention through behavioral principles, or using nudges to influence user behavior.
Help users craft compelling brand narratives. Use when someone is defining brand strategy, writing company positioning, creating pitch narratives, developing messaging frameworks, or trying to make their company story more memorable.
Help users get promoted at work. Use when someone is preparing for a promotion conversation, building their case for advancement, trying to understand what's blocking their promotion, or figuring out how to get to the next level in their career.
Help users build and scale their sales organization. Use when someone is hiring their first salespeople, deciding when to bring on sales leadership, structuring sales compensation, or transitioning from founder-led sales.
Help users build and maintain strong team culture. Use when someone is defining team values, creating psychological safety, onboarding to a new team, navigating cultural change, or building distributed team norms.
Help users build effective AI applications. Use when someone is building with LLMs, writing prompts, designing AI features, implementing RAG, creating agents, running evals, or trying to improve AI output quality.
Help users navigate career changes and pivots. Use when someone is considering a new role, transitioning into product management, evaluating job offers, taking a sabbatical, or feeling stuck in their current position.
Help users develop and coach product managers. Use when someone is managing PMs, creating development plans, running performance reviews, or trying to level up their PM team's capabilities.
Help users build and grow product communities. Use when someone is starting a community, scaling an ambassador program, driving community-led growth, or choosing between user, developer, or partner communities.
Help users understand and respond to competition. Use when someone is positioning against competitors, evaluating market threats, running competitive war games, or deciding how much to focus on competitors versus customers.
Help users conduct effective hiring interviews. Use when someone is designing an interview loop, crafting interview questions, evaluating candidates in real-time, or building a structured interview process.
Help users run better customer and user interviews. Use when someone is preparing for user research, planning discovery interviews, writing interview questions, analyzing interview findings, or trying to understand customer needs.
Help users build content marketing strategies. Use when someone is starting a blog, building SEO, creating thought leadership content, or deciding on content formats and distribution channels.
Help users work effectively across functions. Use when someone is navigating PM-engineering relationships, resolving cross-team conflicts, building product trios, or improving handoffs between design, engineering, and product.
Help users create compelling product visions. Use when someone is writing a vision statement, defining a long-term product direction, aligning teams on the future state, or distinguishing vision from strategy.
Help users delegate effectively. Use when someone is struggling to let go of tasks, deciding what to delegate, building team autonomy, or balancing being hands-on vs hands-off.
Help users understand and build design engineering capabilities. Use when someone is creating a design engineering function, hiring design engineers, or bridging the gap between design and engineering teams.
Help users design and optimize growth loops. Use when someone is building viral mechanics, designing referral programs, creating product-led acquisition, or figuring out how to make their product grow itself.
ElevenLabs text-to-speech with mac-style say UX.
Oracle CLI second-model review/debug/refactor/design with selected files, dry-run token checks, API or browser engine.
Capture and automate macOS UI with the Peekaboo CLI.
Generate multiple radically different interface designs for a module using parallel sub-agents. Use when user wants to design an API, explore interface options, compare module shapes, or mentions "design it twice".
Build a throwaway prototype to flesh out a design — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route.
Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".