YepAPI/skills

Background Jobs

Rules

• Offload slow work to background jobs: emails, image processing, webhooks, reports, AI calls
• BullMQ for self-hosted Redis queues — Inngest or Trigger.dev for serverless/managed
• Every job must be idempotent — safe to retry without side effects (use idempotency keys)
• Retry with exponential backoff: 3-5 attempts, delays of 1s, 10s, 60s, 5min
• Dead letter queue (DLQ): after max retries, move to DLQ for investigation — never silently drop
• Set concurrency limits per queue — prevent worker overload (e.g., concurrency: 5)
• Job payloads should be small: pass IDs and references, not full objects
• Separate queues by priority: critical (payments), default (emails), low (analytics)
• Monitor queue depth and processing time — alert when queue backs up

Patterns

// BullMQ pattern
import { Queue, Worker } from "bullmq";
const queue = new Queue("emails", { connection: redis });

await queue.add("welcome", { userId }, {
  attempts: 3,
  backoff: { type: "exponential", delay: 1000 },
});

const worker = new Worker("emails", async (job) => {
  await sendWelcomeEmail(job.data.userId);
}, { connection: redis, concurrency: 5 });

// Inngest serverless pattern
import { inngest } from "./client";

export const sendEmail = inngest.createFunction(
  { id: "send-welcome-email", retries: 3 },
  { event: "user/created" },
  async ({ event }) => { await sendWelcomeEmail(event.data.userId); }
);

Avoid

• Processing slow tasks in API request handlers — respond fast, process in background
• Non-idempotent jobs — duplicate execution will cause double charges, double emails
• Unbounded concurrency — will exhaust database connections and memory
• Storing large payloads in the job queue — pass references instead