Hilft Benutzern, KI-Evaluierungen zu erstellen und auszuführen. Verwenden Sie dies, wenn jemand Bewertungen für LLM-Produkte erstellt, die Modellqualität misst, Testfälle erstellt, Rubriken entwirft oder versucht, die Qualität von KI-Ausgaben systematisch zu messen.
Dieses Skill 'refoundai/ai-evals' unterstützt Entwickler und KI-Agenten-Nutzer dabei, strukturierte Evaluierungen für Sprachmodelle (LLMs) zu erstellen, durchzuführen und auszuwerten. Es ist besonders nützlich für Qualitätssicherung von LLM-Produkten, die Messung der Modellqualität, das Erstellen von Testfällen, die Entwicklung von Bewertungsrubriken und die systematische Erfassung der KI-Ausgabequalität. Das Skill funktioniert als Teil eines KI-Agenten-Workflows, indem es dem Agenten klare Anweisungen und Kontext gibt, wann und wie es angewendet werden sollte – etwa bei der Arbeit mit Claude Code, Cursor oder Codex. Es ermöglicht die Definition von Evaluierungsmetriken, die Erstellung von Testdatensätzen und die Analyse von Ergebnissen, um wiederholbare und objektive Bewertungen der Modellleistung zu gewährleisten. Durch die Integration in den Entwicklungsprozess hilft es, Qualitätsengpässe frühzeitig zu erkennen und die Modellqualität kontinuierlich zu verbessern.
npx skills add https://github.com/refoundai/lenny-skills/tree/main/skills/ai-evalsHilft Benutzern, KI-Evaluierungen zu erstellen und auszuführen. Verwenden Sie dies, wenn jemand Bewertungen für LLM-Produkte erstellt, die Modellqualität misst, Testfälle erstellt, Rubriken entwirft oder versucht, die Qualität von KI-Ausgaben systematisch zu messen.
This repo contains 20 individual skills — each has its own dedicated page.
Help users define AI product strategy. Use when someone is building an AI product, deciding where to apply AI in their product, planning an AI roadmap, evaluating build vs buy for AI capabilities, or figuring out how to integrate AI into existing products.
Help users synthesize and act on customer feedback. Use when someone is analyzing NPS responses, processing support tickets, reviewing user research, synthesizing feedback from multiple channels, or trying to identify patterns in customer input.
Help users apply behavioral science to product design. Use when someone is designing for habit formation, reducing friction, applying psychology to UX, increasing retention through behavioral principles, or using nudges to influence user behavior.
Help users craft compelling brand narratives. Use when someone is defining brand strategy, writing company positioning, creating pitch narratives, developing messaging frameworks, or trying to make their company story more memorable.
Help users get promoted at work. Use when someone is preparing for a promotion conversation, building their case for advancement, trying to understand what's blocking their promotion, or figuring out how to get to the next level in their career.
Help users build and scale their sales organization. Use when someone is hiring their first salespeople, deciding when to bring on sales leadership, structuring sales compensation, or transitioning from founder-led sales.
Help users build and maintain strong team culture. Use when someone is defining team values, creating psychological safety, onboarding to a new team, navigating cultural change, or building distributed team norms.
Help users build effective AI applications. Use when someone is building with LLMs, writing prompts, designing AI features, implementing RAG, creating agents, running evals, or trying to improve AI output quality.
Help users navigate career changes and pivots. Use when someone is considering a new role, transitioning into product management, evaluating job offers, taking a sabbatical, or feeling stuck in their current position.
Help users develop and coach product managers. Use when someone is managing PMs, creating development plans, running performance reviews, or trying to level up their PM team's capabilities.
Help users build and grow product communities. Use when someone is starting a community, scaling an ambassador program, driving community-led growth, or choosing between user, developer, or partner communities.
Help users understand and respond to competition. Use when someone is positioning against competitors, evaluating market threats, running competitive war games, or deciding how much to focus on competitors versus customers.
Help users conduct effective hiring interviews. Use when someone is designing an interview loop, crafting interview questions, evaluating candidates in real-time, or building a structured interview process.
Help users run better customer and user interviews. Use when someone is preparing for user research, planning discovery interviews, writing interview questions, analyzing interview findings, or trying to understand customer needs.
Help users build content marketing strategies. Use when someone is starting a blog, building SEO, creating thought leadership content, or deciding on content formats and distribution channels.
Help users work effectively across functions. Use when someone is navigating PM-engineering relationships, resolving cross-team conflicts, building product trios, or improving handoffs between design, engineering, and product.
Help users create compelling product visions. Use when someone is writing a vision statement, defining a long-term product direction, aligning teams on the future state, or distinguishing vision from strategy.
Help users delegate effectively. Use when someone is struggling to let go of tasks, deciding what to delegate, building team autonomy, or balancing being hands-on vs hands-off.
Help users understand and build design engineering capabilities. Use when someone is creating a design engineering function, hiring design engineers, or bridging the gap between design and engineering teams.
Help users design and optimize growth loops. Use when someone is building viral mechanics, designing referral programs, creating product-led acquisition, or figuring out how to make their product grow itself.
ElevenLabs text-to-speech with mac-style say UX.
Oracle CLI second-model review/debug/refactor/design with selected files, dry-run token checks, API or browser engine.
Capture and automate macOS UI with the Peekaboo CLI.
Generate multiple radically different interface designs for a module using parallel sub-agents. Use when user wants to design an API, explore interface options, compare module shapes, or mentions "design it twice".
Build a throwaway prototype to flesh out a design — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route.
Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".