Communitygithub.com

可观测性与服务等级目标(SLO)

在添加日志记录、指标、链路追踪、健康检查、SLO 或告警时使用——或者在构建需要可运维和可调试的服务表面时使用。关键词——结构化日志、OpenTelemetry、关联 ID、RED 指标、存活探针、就绪探针、SLI、SLO、错误预算、告警。

可观测性与服务等级目标(SLO) 是什么?

该技能为 AI 代理(如 Claude Code、Coder 或 Codex)提供了在构建可观测系统时应用的结构化方法和原则。其核心是强制实施三大支柱:结构化日志、指标(特别是 RED——速率、错误、持续时间)和链路追踪(使用 OpenTelemetry)。该技能要求在构建任何新服务表面时就内建可观测性,而不是事后附加。它强调“黄金信号”的仪表化、将 liveness/readiness 健康检查作为独立端点暴露,以及基于服务等级目标(SLO)定义关键用户旅程,并配合服务等级指标(SLI)和错误预算进行测量。告警设计为有症状触发(例如 SLO 燃烧速率、就绪探针失败),而不是对内部误报警的简单重复。该技能支持的平台包括 Azure Monitor/Application Insights、Prometheus/Grafana 和 Kubernetes 探针。此外,它还包含实际机制:将结构化 JSON 日志输出到 stdout、通过 OpenTelemetry 导出链路追踪和指标、传播关联 ID、RED 指标、Postgres 慢查询日志、前端 Web Vitals 报告以及基于燃烧速率的告警。对于早期项目或本地项目,该技能提供了渐进式实施路径:先实施结构化日志和健康端点,直到有真正用户需要保护时才全面部署 SLO 和高级告警。

兼容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/lguidolin/agent-skills/tree/main/skills/observability-and-slos

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

可观测性与服务等级目标(SLO) 是做什么的?

概述

你无法运维、调试或改进你看不见的东西。可观测性是设计时就融入的,而不是在第一次故障后才匆忙补救。每次事故的第一个问题都是“什么变了?谁受到了影响?”——一个无法回答这个问题系统,会把五分钟就能解决的修复变成一场考古挖掘。

原则(通用)

  • 三大信号。 发出结构化日志(机器可解析,而非自由文本)、指标(计数、速率、持续时间)和链路追踪(请求在服务间的路径)。一个请求从头到尾携带一个关联 ID/链路追踪 ID。
  • 为每个服务表面仪表化黄金信号:每个端点/操作的速率、错误、持续时间(RED),以及资源饱和度。一个新端点直到变得可观测才算“完成”。
  • 健康是一个端点,而不是猜测。 将存活探针(“我还在运行吗?”)和就绪探针(“我能处理流量吗?”)作为独立的检查暴露出来。
  • 为关键用户旅程定义 SLO,配合衡量它们的SLI和错误预算,使可靠性与功能之间的权衡成为明确、数据驱动的选择。
  • 根据症状触发告警,而不是原因。 当用户受到影响时(SLO 燃烧、就绪探针失败)才呼叫值班人员,而不是对每个内部告警都响应。每个告警必须是可操作的——一个无人响应的告警训练人们忽略所有告警。

机制(此技术栈)

  • 结构化 JSON 日志输出到 stdout,由平台收集(例如 AKS → Azure Monitor / Log Analytics)。服务中禁止自由文本日志。
  • 应用服务器和 PostGraphile 中使用 OpenTelemetry;将链路追踪和指标导出到 Azure Monitor / Application Insights(或 Prometheus/Grafana)。
  • 关联 ID 在整个数据路径中传播——在边缘生成,通过 Browser → App → PostGraphile 传递,携带上下文头部,附加到每个日志行和 Span 上。
  • 每个 HTTP 路由和 GraphQL 操作的 RED 指标;Postgres 慢查询日志 + pg_stat_statements。
  • 存活/就绪端点连接到 Kubernetes 探针(参见 cloud-delivery-aks)。就绪探针反映真实的依赖健康状态(能否访问数据库?),而不仅仅是进程在运行。
  • 前端: 将 Web Vitals 和客户端错误报告到同一个后端。
  • 每个用户旅程的 SLO,配合仪表板和燃烧速率告警。

何时扩展此实践

一个预发布/本地项目应先实施结构化日志和健康端点,推迟 SLO/告警,直到有真实用户需要保护时才实施。在第一个决策记录中声明当前模式。

完整理论基础:宪法第 IX 条和第 XVI 条,位于 engineering-constitution/references/engineering-constitution.md。

Individual skills in this repo

This repo contains 18 individual skills — each has its own dedicated page.

lguidolin/change-hygiene-and-code-craft

Use when writing or refactoring code, structuring a commit or PR, or deciding whether to abstract duplication. Symptoms — mixing reorg with logic changes, a PR doing several things at once, a file growing large, the second copy of similar code, or unsure whether to DRY something up.

lguidolin/cloud-delivery-aks

Use when deploying to Kubernetes or Azure Kubernetes Service (AKS), configuring cloud secrets, setting up progressive rollout/canary, per-PR ephemeral environments, or k8s health probes. Keywords — Kubernetes, AKS, Key Vault, Argo Rollouts, Flagger, canary, blue-green, liveness, readiness, PodDisruptionBudget, HPA, rollback, GHCR.

lguidolin/commit-history-rewrite

Use when an existing repository has messy commit history that needs to conform to conventional commits before adopting release-please, or when intermediate WIP/fixup/merge commits need to be cleaned up.

lguidolin/conventional-commits-and-releases

Use when committing, writing a commit message, opening a PR that will be squash-merged, or configuring automated versioning/changelogs. Keywords — conventional commits, release-please, semver, feat/fix/chore, breaking change, changelog.

lguidolin/defense-in-depth-security

Use when handling untrusted input, secrets, authentication/authorization, or dependencies — or threat-modeling a new surface. Keywords — STRIDE, threat model, least privilege, secrets management, supply chain, dependency scanning, input validation, audit log, defense in depth.

lguidolin/designing-before-building

Use when starting a feature, fixing a non-trivial bug, or about to write implementation code — before any code exists. Symptoms you need this: "this is simple, I'll just code it", reaching for the editor before a design is approved, or an idea that hasn't been turned into a spec and plan.

lguidolin/engineering-constitution

Use when starting work in a project that follows the engineering constitution, orienting to its rules, or deciding which engineering practice applies to a task — spec writing, commits, testing, security, deploys, database, or UI work.

lguidolin/graphql-contract-testing

Use when writing a GraphQL query/mutation that the UI and a test will share, or building route/schema contract or smoke tests. Symptoms — copying a query into a test, a test asserting on query text, schema change that didn't break the UI build, or RLS/permission drift. Keywords — graphql-codegen, typed document, contract test, route smoke test.

lguidolin/init-repo-CI

Use when setting up a new repository with conventional commits, release-please, and CI automation, or when retrofitting an existing repository that lacks automated versioning and PR validation workflows.

lguidolin/interface-craft-and-accessibility

Use when building or styling UI — components, layouts, forms, design tokens — or making accessibility decisions. Keywords — a11y, WCAG, keyboard navigation, focus state, contrast, design system, minimalist UI, component reuse, ARIA, semantic HTML.

lguidolin/merge-gates-and-automation

Use when setting up or changing CI, pre-push hooks, or a task runner, or deciding what must pass before merge. Symptoms — tempted to put authoritative checks only in a local hook, skip CI, bypass with --no-verify, or unsure what gates a merge vs. runs locally.

lguidolin/performance-and-scale

Use when working on hot paths, list endpoints, pagination, data-access in loops, or public interfaces/schemas. Symptoms — unbounded queries, N+1 access, no latency budget, optimizing without measuring, or changing an interface many consumers depend on. Keywords — pagination, N+1, Hyrum's Law, performance budget, bundle size.

lguidolin/postgres-postgraphile-rls-and-sql

Use when writing PostgreSQL, PostGraphile config, Row-Level Security policies, SQL schema files, or working on the Browser→App→PostGraphile→Postgres data path. Keywords — RLS, SECURITY DEFINER, search_path, pgSettings, grants, roles, GraphQL depth limit, query cost, statement_timeout, SQL file organization.

lguidolin/recording-decisions

Use when a design or architecture decision has been made and needs to be captured — writing a decision record or ADR, updating a decision index, noting a deferred idea, or superseding a past decision. Keywords — ADR, decision record, rationale, rejected alternatives, dependency index.

lguidolin/resilience-and-deploy-safety

Use when planning a deploy, designing a rollback, or responding to an incident or writing a postmortem. Keywords — deploy safety, rollback, immutable artifact, progressive delivery, canary, blast radius, incident response, blameless postmortem, error budget.

lguidolin/ship-it

Use when the user wants to ship work — push, PR, archive decision records, merge, and clean up. Handles the full lifecycle from committing final changes through post-merge cleanup including converting specs/plans to compact decision records.

lguidolin/tests-as-a-control

Use when writing or modifying tests, when a test breaks during a refactor, or when testing permission/role rules. Symptoms — tempted to edit a test to make it pass, testing only the happy path, a deny-test that started passing, flaky tests, or unsure what to assert.

lguidolin/zero-downtime-migrations

Use when changing a database schema where data must survive the change — adding/removing/renaming columns, constraints, indexes, or backfilling. Symptoms — a destructive migration bundled with a code deploy, a NOT NULL column with a backfill, a table-locking UPDATE, or a rename. Keywords — expand/contract, parallel change, backfill, NOT VALID, CREATE INDEX CONCURRENTLY, graphile-migrate.

相关技能