Communitygithub.com

A/B-Test-Setup

Hilft bei der Planung, Gestaltung und Durchführung von A/B-Tests und Experimenten, um statistisch valide und umsetzbare Ergebnisse zu erzielen.

Was ist B-Test-Setup?

Diese Fähigkeit unterstützt dich als KI-Agent bei der Konzeption und Durchführung von A/B-Tests und Experimenten. Sie ist nützlich, wann immer ein Benutzer zwei Ansätze vergleichen und messen möchte, welche Variante besser abschneidet – sei es bei der Optimierung einer Webseite, eines E-Mail-Textes oder eines Produktfeatures. Die Fähigkeit folgt einem strukturierten Prozess, der mit einer fundierten Hypothese beginnt, die Testvariablen begrenzt und statistische Strenge fordert. Sie enthält Vorlagen für Hypothesen, Entscheidungshilfen zur Stichprobengröße, eine Anleitung zur Auswahl von Primär-, Sekundär- und Sicherheitsmetriken sowie Tipps zur Implementierung (client- oder serverseitig). Die Fähigkeit ist plattformunabhängig einsetzbar und funktioniert mit Tools wie PostHog, Optimizely, VWO, LaunchDarkly oder Split. Sie betont die Vermeidung typischer Fehler wie vorzeitiges Anhalten eines Tests oder das Verändern von Varianten während des Laufs. Durch die strukturierte Herangehensweise hilft die Fähigkeit, falsch-positive Ergebnisse zu vermeiden und nachvollziehbare, geschäftsrelevante Entscheidungen zu treffen.

Funktioniert mit~Claude Code~Codex CLI~Cursor
Part ofDesign & UI
npx skills add https://github.com/LeoYeAI/openclaw-marketing-skills/tree/main/skills/ab-test-setup
Original anzeigen→Alle Skills durchsuchen

In Ihrer bevorzugten KI fragen

Öffnet einen neuen Chat, in dem dieser Agent-Skill bereits geladen ist.

OpenAIChatGPTClaudeClaudeGeminiGeminiGrokGrokPerplexityPerplexityDeepSeekDeepSeek

Dokumentation

Was macht B-Test-Setup?

Du bist ein Experte für Experimente und A/B-Tests. Dein Ziel ist es, Tests zu entwerfen, die statistisch valide und umsetzbare Ergebnisse liefern.

Erste Einschätzung

Prüfe zuerst den Produkt-Marketing-Kontext: Wenn .agents/product-marketing-context.md existiert (oder .agents/product-marketing-context.md in älteren Setups), lies es, bevor du Fragen stellst. Nutze diesen Kontext und frage nur nach Informationen, die nicht bereits abgedeckt oder für diese Aufgabe spezifisch sind.

Bevor du einen Test entwirfst, verstehe:

  1. Testkontext – Was möchtest du verbessern? Welche Änderung erwägst du?
  2. Aktueller Zustand – Basis-Conversion-Rate? Aktuelles Traffic-Volumen?
  3. Einschränkungen – Technische Komplexität? Zeitplan? Verfügbare Tools?

Kernprinzipien

1. Beginne mit einer Hypothese

  • Nicht nur „mal sehen, was passiert“
  • Spezifische Vorhersage des Ergebnisses
  • Basierend auf Überlegungen oder Daten

2. Teste eine Sache

  • Eine Variable pro Test
  • Sonst weißt du nicht, was gewirkt hat

3. Statistische Strenge

  • Stichprobengröße vorab festlegen
  • Nicht spicken und vorzeitig abbrechen
  • Sich der Methodik verpflichten

4. Messe, was zählt

  • Primäre Metrik, die mit dem Geschäftswert verknüpft ist
  • Sekundäre Metriken für den Kontext
  • Sicherheitsmetriken, um Schaden zu verhindern

Hypothesen-Rahmenwerk

Struktur

Weil [Beobachtung/Daten],
glauben wir, dass [Änderung]
zu [erwartetem Ergebnis]
für [Zielgruppe] führt.
Wir erkennen dies, wenn [Metriken].

Beispiel

Schwach: „Die Buttonfarbe zu ändern, könnte die Klicks erhöhen.“

Stark: „Weil Nutzer laut Heatmaps und Feedback Schwierigkeiten haben, den CTA zu finden, glauben wir, dass ein größerer Button mit kontrastierender Farbe die CTA-Klicks bei neuen Besuchern um 15%+ steigert. Wir messen die Klickrate von Seitenaufruf bis zum Start der Anmeldung.“

Testtypen

TypBeschreibungBenötigter Traffic
A/BZwei Versionen, eine ÄnderungMittel
A/B/nMehrere VariantenHöher
MVTMehrere Änderungen in KombinationSehr hoch
Split-URLUnterschiedliche URLs für VariantenMittel

Stichprobengröße

Kurzreferenz

Basis10% Lift20% Lift50% Lift
1%150k/Variante39k/Variante6k/Variante
3%47k/Variante12k/Variante2k/Variante
5%27k/Variante7k/Variante1,2k/Variante
10%12k/Variante3k/Variante550/Variante

Rechner:

Für detaillierte Stichprobengrößentabellen und Laufzeitberechnungen: Siehe references/sample-size-guide.md

Metrikenauswahl

Primäre Metrik

  • Die eine Metrik, die am meisten zählt
  • Direkt mit der Hypothese verknüpft
  • Wird verwendet, um den Test zu bewerten

Sekundäre Metriken

  • Unterstützen die Interpretation der primären Metrik
  • Erklären, warum/wie die Änderung gewirkt hat

Sicherheitsmetriken

  • Dinge, die nicht schlechter werden sollten
  • Test abbrechen, wenn signifikant negativ

Beispiel: Preisseitentest

  • Primär: Auswahlrate des Tarifs
  • Sekundär: Verweildauer, Tarifverteilung
  • Sicherheitsmetriken: Support-Tickets, Rückerstattungsrate

Varianten entwerfen

Was variieren

KategorieBeispiele
Überschriften/TextNachrichtenwinkel, Wertversprechen, Spezifität, Ton
Visuelles DesignLayout, Farbe, Bilder, Hierarchie
CTAButton-Text, Größe, Platzierung, Anzahl
InhaltEnthaltene Informationen, Reihenfolge, Umfang, sozialer Beweis

Best Practices

  • Einzelne, bedeutungsvolle Änderung
  • Kühn genug, um einen Unterschied zu machen
  • Der Hypothese treu

Traffic-Verteilung

AnsatzAufteilungWann verwenden
Standard50/50Standard für A/B
Konservativ90/10, 80/20Risiko einer schlechten Variante begrenzen
RampingKlein starten, steigernTechnische Risikominderung

Überlegungen:

  • Konsistenz: Nutzer sehen bei Rückkehr die gleiche Variante
  • Ausgewogene Verteilung über Tages-/Wochenzeit

Implementierung

Client-Seite

  • JavaScript ändert die Seite nach dem Laden
  • Schnell zu implementieren, kann zu Flackern führen
  • Tools: PostHog, Optimizely, VWO

Server-Seite

  • Variante wird vor dem Rendern bestimmt
  • Kein Flackern, erfordert Entwicklungsarbeit
  • Tools: PostHog, LaunchDarkly, Split

Testdurchführung

Checkliste vor dem Start

  • Hypothese dokumentiert
  • Primäre Metrik definiert
  • Stichprobengröße berechnet
  • Varianten korrekt implementiert
  • Tracking verifiziert
  • QA für alle Varianten abgeschlossen

Während des Tests

TUN:

  • Auf technische Probleme überwachen
  • Segmentqualität prüfen
  • Externe Faktoren dokumentieren

VERMEIDEN:

  • Ergebnisse ansehen und vorzeitig abbrechen
  • Änderungen an Varianten vornehmen
  • Traffic aus neuen Quellen hinzufügen

Das Spick-Problem

Das Ansehen von Ergebnissen vor Erreichen der Stichprobengröße und vorzeitiges Abbrechen führt zu falsch-positiven Ergebnissen und falschen Entscheidungen. Lege die Stichprobengröße vorab fest und vertraue dem Prozess.

Ergebnisse analysieren

Statistische Signifikanz

  • 95% Konfidenz = p-Wert < 0,05
  • Bedeutet <5% Wahrscheinlichkeit, dass das Ergebnis zufällig ist
  • Keine Garantie – nur ein Schwellenwert

Analyse-Checkliste

  1. Stichprobengröße erreicht? Wenn nicht, ist das Ergebnis vorläufig
  2. Statistisch signifikant? Konfidenzintervalle prüfen
  3. Effektgröße bedeutungsvoll? Vergleiche mit MDE, Projektauswirkung
  4. Sekundäre Metriken konsistent? Unterstützen sie die primäre?
  5. Sicherheitsmetriken bedenklich? Ist etwas schlechter geworden?
  6. Segmentunterschiede? Mobil vs. Desktop? Neu vs. Wiederkehrend?

Ergebnisse interpretieren

ErgebnisSchlussfolgerung
Signifikanter GewinnerVariante implementieren
Signifikanter VerliererKontrolle behalten, lernen warum
Kein signifikanter UnterschiedMehr Traffic oder kühneren Test benötigt
Gemischte SignaleTiefer graben, ggf. segmentieren

Dokumentation

Dokumentiere jeden Test mit:

  • Hypothese
  • Varianten (mit Screenshots)
  • Ergebnissen (Stichprobe, Metriken, Signifikanz)
  • Entscheidung und Erkenntnissen

Für Vorlagen: Siehe references/test-templates.md

Häufige Fehler

Testdesign

  • Zu kleine Änderung testen (nicht erkennbar)
  • Zu viele Dinge testen (kann nicht isolieren)
  • Keine klare Hypothese

Durchführung

  • Vorzeitig abbrechen
  • Dinge während des Tests ändern
  • Implementierung nicht prüfen

Analyse

  • Konfidenzintervalle ignorieren
  • Segmente herauspicken
  • Nicht schlüssige Ergebnisse überinterpretieren

Aufgabenbezogene Fragen

  1. Wie hoch ist deine aktuelle Conversion-Rate?
  2. Wie viel Traffic erhält diese Seite?
  3. Welche Änderung erwägst du und warum?
  4. Was ist die kleinste Verbesserung, die es zu erkennen gilt?
  5. Welche Tools hast du für Tests?
  6. Hast du diesen Bereich schon einmal getestet?

Verwandte Fähigkeiten

  • page-cro: Zum Generieren von Testideen basierend auf CRO-Prinzipien
  • analytics-tracking: Zum Einrichten der Testmessung
  • copywriting: Zum Erstellen von Variantentexten

Individual skills in this repo

This repo contains 20 individual skills — each has its own dedicated page.

LeoYeAI/openclaw-marketing-skills

When the user wants to generate, iterate, or scale ad creative — headlines, descriptions, primary text, or full ad variations — for any paid advertising platform. Also use when the user mentions 'ad copy variations,' 'ad creative,' 'generate headlines,' 'RSA headlines,' 'bulk ad copy,' 'ad iterations,' 'creative testing,' 'ad performance optimization,' 'write me some ads,' 'Facebook ad copy,' 'Google ad headlines,' 'LinkedIn ad text,' or 'I need more ad variations.' Use this whenever someone needs to produce ad copy at scale or iterate on existing ads. For campaign strategy and targeting, see paid-ads. For landing page copy, see copywriting.

LeoYeAI/openclaw-marketing-skills

When the user wants to optimize content for AI search engines, get cited by LLMs, or appear in AI-generated answers. Also use when the user mentions 'AI SEO,' 'AEO,' 'GEO,' 'LLMO,' 'answer engine optimization,' 'generative engine optimization,' 'LLM optimization,' 'AI Overviews,' 'optimize for ChatGPT,' 'optimize for Perplexity,' 'AI citations,' 'AI visibility,' 'zero-click search,' 'how do I show up in AI answers,' 'LLM mentions,' or 'optimize for Claude/Gemini.' Use this whenever someone wants their content to be cited or surfaced by AI assistants and AI search engines. For traditional technical and on-page SEO audits, see seo-audit. For structured data implementation, see schema-markup.

LeoYeAI/openclaw-marketing-skills

When the user wants to set up, improve, or audit analytics tracking and measurement. Also use when the user mentions "set up tracking," "GA4," "Google Analytics," "conversion tracking," "event tracking," "UTM parameters," "tag manager," "GTM," "analytics implementation," "tracking plan," "how do I measure this," "track conversions," "attribution," "Mixpanel," "Segment," "are my events firing," or "analytics isn't working." Use this whenever someone asks how to know if something is working or wan

LeoYeAI/openclaw-marketing-skills

When the user wants to reduce churn, build cancellation flows, set up save offers, recover failed payments, or implement retention strategies. Also use when the user mentions 'churn,' 'cancel flow,' 'offboarding,' 'save offer,' 'dunning,' 'failed payment recovery,' 'win-back,' 'retention,' 'exit survey,' 'pause subscription,' 'involuntary churn,' 'people keep canceling,' 'churn rate is too high,' 'how do I keep users,' or 'customers are leaving.' Use this whenever someone is losing subscribers o

LeoYeAI/openclaw-marketing-skills

Write B2B cold emails and follow-up sequences that get replies. Use when the user wants to write cold outreach emails, prospecting emails, cold email campaigns, sales development emails, or SDR emails. Also use when the user mentions "cold outreach," "prospecting email," "outbound email," "email to leads," "reach out to prospects," "sales email," "follow-up email sequence," "nobody's replying to my emails," or "how do I write a cold email." Covers subject lines, opening lines, body copy, CTAs, p

LeoYeAI/openclaw-marketing-skills

When the user wants to create competitor comparison or alternative pages for SEO and sales enablement. Also use when the user mentions 'alternative page,' 'vs page,' 'competitor comparison,' 'comparison page,' '[Product] vs [Product],' '[Product] alternative,' 'competitive landing pages,' 'how do we compare to X,' 'battle card,' or 'competitor teardown.' Use this for any content that positions your product against competitors. Covers four formats: singular alternative, plural alternatives, you v

LeoYeAI/openclaw-marketing-skills

When the user wants to plan a content strategy, decide what content to create, or figure out what topics to cover. Also use when the user mentions "content strategy," "what should I write about," "content ideas," "blog strategy," "topic clusters," "content planning," "editorial calendar," "content marketing," "content roadmap," "what content should I create," "blog topics," "content pillars," or "I don't know what to write." Use this whenever someone needs help deciding what content to produce,

LeoYeAI/openclaw-marketing-skills

When the user wants to edit, review, or improve existing marketing copy. Also use when the user mentions 'edit this copy,' 'review my copy,' 'copy feedback,' 'proofread,' 'polish this,' 'make this better,' 'copy sweep,' 'tighten this up,' 'this reads awkwardly,' 'clean up this text,' 'too wordy,' or 'sharpen the messaging.' Use this when the user already has copy and wants it improved rather than rewritten from scratch. For writing new copy, see copywriting.

LeoYeAI/openclaw-marketing-skills

When the user wants to write, rewrite, or improve marketing copy for any page — including homepage, landing pages, pricing pages, feature pages, about pages, or product pages. Also use when the user says "write copy for," "improve this copy," "rewrite this page," "marketing copy," "headline help," "CTA copy," "value proposition," "tagline," "subheadline," "hero section copy," "above the fold," "this copy is weak," "make this more compelling," or "help me describe my product." Use this whenever s

LeoYeAI/openclaw-marketing-skills

When the user wants to create or optimize an email sequence, drip campaign, automated email flow, or lifecycle email program. Also use when the user mentions "email sequence," "drip campaign," "nurture sequence," "onboarding emails," "welcome sequence," "re-engagement emails," "email automation," "lifecycle emails," "trigger-based emails," "email funnel," "email workflow," "what emails should I send," "welcome series," or "email cadence." Use this for any multi-email automated flow. For cold out

LeoYeAI/openclaw-marketing-skills

When the user wants to optimize any form that is NOT signup/registration — including lead capture forms, contact forms, demo request forms, application forms, survey forms, or checkout forms. Also use when the user mentions "form optimization," "lead form conversions," "form friction," "form fields," "form completion rate," "contact form," "nobody fills out our form," "form abandonment," "too many fields," "demo request form," or "lead form isn't converting." Use this for any non-signup form tha

LeoYeAI/openclaw-marketing-skills

When the user wants to plan, evaluate, or build a free tool for marketing purposes — lead generation, SEO value, or brand awareness. Also use when the user mentions "engineering as marketing," "free tool," "marketing tool," "calculator," "generator," "interactive tool," "lead gen tool," "build a tool for leads," "free resource," "ROI calculator," "grader tool," "audit tool," "should I build a free tool," or "tools for lead gen." Use this whenever someone wants to build something useful and give

LeoYeAI/openclaw-marketing-skills

Connect GetXAPI to pull public X/Twitter marketing signals into OpenClaw workflows. Use when the user wants social listening, tweet search, reply search, user lookup, or monitor setup driven by a Bearer-auth search backend. Enhances social-content, paid-ads, ad-creative, content-strategy, launch-strategy, and competitor-alternatives with real public conversation data.

LeoYeAI/openclaw-marketing-skills

Connect to Google Ads API to pull real account data — campaign performance, keyword health, wasted spend, search terms, impression share. Use when the user wants to audit their Google Ads account, find wasted budget, optimize keywords, or get data-driven recommendations. Enhances paid-ads and ad-creative skills with real numbers instead of guesswork. Triggers on: 'audit my Google Ads', 'find wasted spend', 'why are my ads not converting', 'Google Ads performance', 'which keywords to pause', 'Goo

LeoYeAI/openclaw-marketing-skills

When the user wants to plan a product launch, feature announcement, or release strategy. Also use when the user mentions 'launch,' 'Product Hunt,' 'feature release,' 'announcement,' 'go-to-market,' 'beta launch,' 'early access,' 'waitlist,' 'product update,' 'how do I launch this,' 'launch checklist,' 'GTM plan,' or 'we're about to ship.' Use this whenever someone is preparing to release something publicly. For ongoing marketing after launch, see marketing-ideas.

LeoYeAI/openclaw-marketing-skills

When the user wants to create, plan, or optimize a lead magnet for email capture or lead generation. Also use when the user mentions "lead magnet," "gated content," "content upgrade," "downloadable," "ebook," "cheat sheet," "checklist," "template download," "opt-in," "freebie," "PDF download," "resource library," "content offer," "email capture content," "Notion template," "spreadsheet template," or "what should I give away for emails." Use this for planning what to create and how to distribute

LeoYeAI/openclaw-marketing-skills

When the user needs marketing ideas, inspiration, or strategies for their SaaS or software product. Also use when the user asks for 'marketing ideas,' 'growth ideas,' 'how to market,' 'marketing strategies,' 'marketing tactics,' 'ways to promote,' 'ideas to grow,' 'what else can I try,' 'I don't know how to market this,' 'brainstorm marketing,' or 'what marketing should I do.' Use this as a starting point whenever someone is stuck or looking for inspiration on how to grow. For specific channel execution, see the relevant skill (paid-ads, social-content, email-sequence, etc.).

LeoYeAI/openclaw-marketing-skills

When the user wants to apply psychological principles, mental models, or behavioral science to marketing. Also use when the user mentions 'psychology,' 'mental models,' 'cognitive bias,' 'persuasion,' 'behavioral science,' 'why people buy,' 'decision-making,' 'consumer behavior,' 'anchoring,' 'social proof,' 'scarcity,' 'loss aversion,' 'framing,' or 'nudge.' Use this whenever someone wants to understand or leverage how people think and make decisions in a marketing context.

LeoYeAI/openclaw-marketing-skills

Connect to Meta Marketing API to pull real Facebook and Instagram ad data — campaign performance, creative fatigue, audience saturation, ROAS by ad set, frequency analysis, and Learning Phase status. Use when the user wants to audit their Meta/Facebook/Instagram ads, diagnose creative fatigue, find wasted budget, or get data-driven optimization. Enhances paid-ads and ad-creative skills with real Meta data. Triggers on: 'audit my Facebook ads', 'Instagram ads performance', 'Meta ads not convertin

LeoYeAI/openclaw-marketing-skills

When the user wants to optimize post-signup onboarding, user activation, first-run experience, or time-to-value. Also use when the user mentions "onboarding flow," "activation rate," "user activation," "first-run experience," "empty states," "onboarding checklist," "aha moment," "new user experience," "users aren't activating," "nobody completes setup," "low activation rate," "users sign up but don't use the product," "time to value," or "first session experience." Use this whenever users are si

Verwandte Skills

steipete/sag

ElevenLabs text-to-speech with mac-style say UX.

community

steipete/oracle

Oracle CLI second-model review/debug/refactor/design with selected files, dry-run token checks, API or browser engine.

community

steipete/peekaboo

Capture and automate macOS UI with the Peekaboo CLI.

community

mattpocock/design-an-interface

Generate multiple radically different interface designs for a module using parallel sub-agents. Use when user wants to design an API, explore interface options, compare module shapes, or mentions "design it twice".

community

mattpocock/prototype

Build a throwaway prototype to flesh out a design — a runnable terminal app for state/business-logic questions, or several radically different UI variations toggleable from one route.

community

mattpocock/grill-me

Interview the user relentlessly about a plan or design until reaching shared understanding, resolving each branch of the decision tree. Use when user wants to stress-test a plan, get grilled on their design, or mentions "grill me".

community