spark-optimization 是做什么的？
优化Apache Spark作业，涵盖分区、缓存、Shuffle优化及内存调优。适用于提升Spark性能、调试慢速作业或扩展数据处理管道。
优化Apache Spark作业，涵盖分区、缓存、Shuffle优化及内存调优。适用于提升Spark性能、调试慢速作业或扩展数据处理管道。
该技能专注于优化Apache Spark作业性能，通过调整分区策略、合理设置缓存、优化Shuffle过程以及精细调节内存配置来显著提升Spark应用的运行效率。它特别适用于处理慢速作业调试、大规模数据处理管道的扩展以及高负载场景下的性能瓶颈分析。兼容所有主流Spark版本（包括Databricks、Amazon EMR、Cloudera等环境），并能够根据作业的输入数据量、集群资源（CPU/内存）和计算模式自动推荐最佳实践参数。其工作原理是结合Spark的物理执行计划分析、Spark UI指标监控以及常见反模式检测，提供可执行的优化建议，包括动态分区裁剪、列式存储格式选择、广播Hash Join与Sort-Merge Join的切换、数据序列化方式优化等。使用该技能可以显著减少作业运行时间、降低资源消耗并提高集群的稳定性。
npx skills add https://github.com/wshobson/agents/tree/main/skills/spark-optimizationInstalled? Explore more 生产力与协作 skills: steipete/gemini, steipete/gh-issues, steipete/skill-creator · View all 6 →
优化Apache Spark作业，涵盖分区、缓存、Shuffle优化及内存调优。适用于提升Spark性能、调试慢速作业或扩展数据处理管道。
This repo contains 20 individual skills — each has its own dedicated page.
Implement WCAG 2.2 compliant interfaces with mobile accessibility, inclusive design patterns, and assistive technology support. Use when auditing accessibility, implementing ARIA patterns, building for screen readers, or ensuring inclusive user experiences.
Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.
Migrate from AngularJS to Angular using hybrid mode, incremental component rewriting, and dependency injection updates. Use when upgrading AngularJS applications, planning framework migrations, or modernizing legacy Angular code.
Understand anti-reversing, obfuscation, and protection techniques encountered during software analysis. Use this skill when analyzing malware evasion techniques, when implementing anti-debugging protections for CTF challenges, when reverse engineering packed binaries, or when building security research tools that need to detect virtualized environments.
Master REST and GraphQL API design principles to build intuitive, scalable, and maintainable APIs that delight developers. Use when designing new APIs, reviewing API specifications, or establishing API design standards.
Write and maintain Architecture Decision Records (ADRs) following best practices for technical decision documentation. Use when documenting significant technical decisions, reviewing past architectural choices, or establishing decision processes.
Implement proven backend architecture patterns including Clean Architecture, Hexagonal Architecture, and Domain-Driven Design. Use this skill when designing clean architecture for a new microservice, when refactoring a monolith to use bounded contexts, when implementing hexagonal or onion architecture patterns, or when debugging dependency cycles between application layers.
Master Python asyncio, concurrent programming, and async/await patterns for high-performance applications. Use when building async APIs, concurrent systems, or I/O-bound applications requiring non-blocking operations.
Build comprehensive attack trees to visualize threat paths. Use when mapping attack scenarios, identifying defense gaps, or communicating security risks to stakeholders.
Master authentication and authorization patterns including JWT, OAuth2, session management, and RBAC to build secure, scalable access control systems. Use when implementing auth systems, securing APIs, or debugging security issues.
Audit and rewrite prose so it stops reading as machine-generated. Use this skill when asked to remove AI-isms, clean up AI writing, edit a draft for AI tells, audit a README, changelog, release note, PR description, or blog post for machine-sounding prose, or make text sound less like AI. Supports a detect-only mode, a rewrite mode, and an edit-in-place mode, with optional voice and context profiles.
Build robust backtesting systems for trading strategies with proper handling of look-ahead bias, survivorship bias, and transaction costs. Use when developing trading algorithms, validating strategies, or building backtesting infrastructure.
Master defensive Bash programming techniques for production-grade scripts. Use when writing robust shell scripts, CI/CD pipelines, or system utilities requiring fault tolerance and safety.
Master Bash Automated Testing System (Bats) for comprehensive shell script testing. Use when writing tests for shell scripts, CI/CD pipelines, or requiring test-driven development of shell utilities.
Optimize Bazel builds for large-scale monorepos. Use when configuring Bazel, implementing remote execution, or optimizing build performance for enterprise codebases.
Pre-build product and feature risk review for founders, product managers, and AI-assisted builders. Use this skill when the user is about to build a landing page, MVP, SaaS product, internal tool, agent workflow, or major feature and needs to check demand, positioning, monetization, retention, trust, distribution, and adoption risk before implementation starts.
Build automated billing systems for recurring payments, invoicing, subscription lifecycle, and dunning management. Use when implementing subscription billing, automating invoicing, or managing recurring payment systems.
Master binary analysis patterns including disassembly, decompilation, control flow analysis, and code pattern recognition. Use when analyzing executables, understanding compiled code, or performing static analysis on binaries.
Configure a PreToolUse hook to prevent AI agents from skipping git pre-commit hooks with --no-verify and other bypass flags. Use when setting up Claude Code projects that enforce commit quality gates.
Brand-first landing page designer — runs a brand-identity interview (colors, typography, shape language), then generates and iterates on a polished landing page via Stitch with deployment-ready HTML. Use when the user asks to create, design, or build a landing page, homepage, or marketing page and has no established visual direction. Skip when they have a design mockup, need a dashboard or app UI, are working at component level, building a multi-page app, or restyling with known design tokens — use frontend-design instead.
Gemini CLI one-shot prompts, summaries, generation, skills, hooks, MCP, or Gemma routing.
Fetch GitHub issues, select candidates, spawn background fix agents, open PRs, and optionally process PR review comments.
Create, edit, audit, tidy, validate, or restructure AgentSkills and SKILL.md files.
Search, install, update, sync, or publish agent skills with the ClawHub CLI and registry.
Delegate coding work to Codex, Claude Code, or OpenCode as background workers; not simple edits or read-only code lookup.
List, configure, authenticate, call, and inspect MCP servers/tools with mcporter over HTTP or stdio.