Dify Performance Optimization SKILL

What does Dify Performance Optimization SKILL do?

Overview

This SKILL provides a comprehensive guide for optimizing the performance of Dify workflows and plugins, covering core topics such as graph structure optimization, LLM cost control, Worker pool tuning, variable pool optimization, parallel processing, and performance monitoring.

Applicable Scenarios

Use this SKILL when you encounter the following situations:

• ⏱️ Workflow execution time is too long and requires performance optimization
• 💰 LLM API costs are too high and need to reduce token consumption
• 🚀 Concurrent processing capacity is insufficient and requires throughput improvement
• 📊 System resource utilization is low and needs configuration optimization
• 📈 Performance bottlenecks with large-scale data processing
• 🔍 Performance diagnosis and monitoring are necessary

Core Value

• Quantifiable Optimization Results: Parallel branch execution reduces time by 40-60%, LLM cache reduces calls by 30-50%
• Driven by Real-World Cases: 5 real optimization cases with before-and-after comparisons
• Comprehensive Optimization Strategies: Covers all levels: workflows, LLMs, plugins, and monitoring
• Rapid Diagnostic Tools: A performance diagnostic checklist with 5 dimensions

---

Rapid Diagnostic Checklist

Before starting optimization, use the following checklist to quickly locate performance issues:

📊 Graph Structure Diagnosis

• Are there too many serial dependencies?
• Can parallel branches be used? (v0.8.0+)
• Are there unnecessary nodes and connections?
• Does the number of nodes exceed 50?
• Is the appropriate design pattern being used?

🤖 LLM Usage Diagnosis

• Is the model choice appropriate? (Lightweight tasks use lightweight models)
• Is the prompt too long? (Can it be streamlined?)
• Is LLM response caching enabled?
• Is structured output being used? (JSON Schema)
• Is token consumption within budget?

⚙️ Worker Pool Diagnosis

• Is the Worker count configuration reasonable? (2-8 threads)
• Is the queue depth too large?
• Is there idle Worker waste?
• Is the scaling strategy reasonable?

💾 Variable Pool Diagnosis

• Are large objects stored in the variable pool?
• Are variables cleaned up promptly?
• Are references used instead of copies?
• Is the variable scope appropriate?

🔄 Concurrent Processing Diagnosis

• Is parallel_nums configured in the iterator?
• Is streaming response being used?
• Are asynchronous tasks configured appropriately?
• Are there any blocking operations?

---

Part 1: Graph Structure Optimization

1.1 Reduce Serial Dependencies

Problem: Too many serial dependencies cause workflow execution time to grow linearly.

Optimization Strategy:

1. Identify parallel nodes: Analyze dependencies between nodes to find nodes that can be executed in parallel
2. Use parallel branches: Dify v0.8.0+ supports parallel branch functionality
3. Refactor workflows: Restructure serial flows into a parallel + aggregation pattern

Example:

# Before optimization: Serial execution (Total time = 3 + 2 + 4 = 9 seconds)
Start → API Call 1(3s) → API Call 2(2s) → API Call 3(4s) → End

# After optimization: Parallel execution (Total time = max(3,2,4) = 4 seconds)
Start → [API Call 1(3s), API Call 2(2s), API Call 3(4s)] → Variable Aggregator → End

Optimization Result: Execution time reduced by 40-60%

1.2 Parallel Branch Feature (v0.8.0+)

Dify v0.8.0 introduces the parallel branch feature, allowing multiple branches to execute simultaneously.

Usage:

1. Create multiple branches in the workflow editor
2. Ensure there are no data dependencies between branches
3. Use a variable aggregator to collect results

Applicable Scenarios:

• Parallel calls to multiple HTTP Request nodes
• Parallel execution of multiple Tool nodes
• Parallel processing of Knowledge Retrieval + API calls
• Parallel inference with multiple models (model comparison scenarios)

Configuration Example:

# Parallel branch configuration
parallel_branches:
  - branch_1:
      nodes: [http_request_1, code_1]
  - branch_2:
      nodes: [http_request_2, code_2]
  - branch_3:
      nodes: [tool_1, llm_1]

1.3 Avoid Unnecessary Nodes

Common Issues:

• Redundant data transformation nodes
• Unnecessary intermediate variable nodes
• Excessive conditional node usage

Optimization Suggestions:

• Merge similar processing logic
• Use code nodes to replace multiple simple nodes
• Simplify conditional logic

1.4 Graph Structure Design Patterns

Choose an appropriate design pattern based on task type:

1. Sequential Pipeline: Start → LLM → End (Simple tasks)
2. Conditional Branches: Start → LLM → If-Else → Multiple paths (Decision tasks)
3. Iterative Processing: Start → Code → Iteration → LLM (Batch tasks)
4. Knowledge Retrieval: Start → Knowledge-Retrieval → LLM (RAG applications)
5. Multi-Source Aggregation: Multiple Tool/HTTP → Variable-Aggregator → LLM (Data aggregation)

Detailed Documentation: See workflow-optimization/graph-structure.md

---

Part 2: LLM Optimization Strategy

2.1 Model Selection Strategy

Choosing the right model is key to cost optimization.

Model Selection Matrix:

Task Type	Recommended Model	Token Cost	Applicable Scenarios
Simple Classification	Qwen-Turbo, Gemini Flash	Low	Text classification, sentiment analysis
Text Generation	GPT-3.5-turbo, Qwen-Plus	Medium	Content generation, summarization
Complex Reasoning	GPT-4, Claude Opus	High	Complex analysis, decision making
Local Deployment	Ollama (Llama 3, Qwen)	Zero API cost	Privacy-sensitive scenarios

Cost Comparison:

• GPT-4: $0.03/1K tokens (input) + $0.06/1K tokens (output)
• GPT-3.5-turbo: $0.0015/1K tokens (input) + $0.002/1K tokens (output)
• Qwen-Turbo: ¥0.0008/1K tokens (approximately 20x cost difference)

Optimization Suggestions:

• Use lightweight models (Qwen/Gemini) for lightweight tasks
• Use powerful models (GPT-4/Claude) for complex tasks
• Consider using local models (Ollama) to reduce costs

2.2 Prompt Optimization Techniques

Streamline Prompt:

# Before optimization (150 tokens)
"""
You are a professional customer service assistant. Please read the following customer question carefully and provide a detailed, accurate, and friendly response based on our product knowledge base. When answering, please note the following points:
1. Maintain a professional and polite tone
2. Provide specific solutions
3. If unsure, state it clearly
4. Reference relevant product documentation

Customer Question: {query}
"""

# After optimization (50 tokens)
"""
As a customer service assistant, answer the customer's question based on the knowledge base, maintaining professionalism and politeness.

Question: {query}
"""

Token Savings: 66% (150 → 50 tokens)

Other Optimization Techniques:

• Remove redundant explanations and examples
• Use concise instructions
• Avoid repeated context
• Use structured output to reduce parsing costs

2.3 Caching Strategy

LLM Response Cache:

Use Redis to cache LLM responses and avoid repeated calls.

Implementation:

import hashlib
import redis

# Connect to Redis
r = redis.Redis(host='localhost', port=6379, db=0)

def get_llm_response_with_cache(prompt, model="gpt-3.5-turbo"):
    # Generate cache key
    cache_key = f"llm:{model}:{hashlib.md5(prompt.encode()).hexdigest()}"

    # Check cache
    cached = r.get(cache_key)
    if cached:
        return cached.decode('utf-8')

    # Call LLM
    response = call_llm_api(prompt, model)

    # Store cache (TTL: 1 hour)
    r.setex(cache_key, 3600, response)

    return response

Caching Strategies:

• LLM Response Cache: Return cached results for identical prompts
• Knowledge Base Retrieval Cache: Return cached vectors for identical queries
• Node Result Cache: Avoid re-executing identical nodes

Optimization Result: API calls reduced by 30-50%

2.4 Structured Output

Use JSON Schema to ensure LLM output stability, reducing parsing errors and retries.

Configuration Example:

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "enum": ["Technical", "Sales", "After-Sales"]
    },
    "priority": {
      "type": "integer",
      "minimum": 1,
      "maximum": 5
    },
    "summary": {
      "type": "string",
      "maxLength": 100
    }
  },
  "required": ["category", "priority", "summary"]
}

Optimization Result:

• Reduces output token count
• Avoids parsing errors and retries
• Improves downstream node processing efficiency

Detailed Documentation: See llm-optimization/

---

Part 3: Worker Pool Tuning

3.1 Dynamic Worker Thread Pool

Dify uses a dynamic Worker pool to execute nodes in parallel, with a default configuration of 2-8 worker threads.

Worker Pool Architecture:

GraphEngine
├── ReadyQueue (Ready Node Queue)
├── EventQueue (Event Queue)
├── Dispatcher (Event Dispatcher)
└── WorkerPool (2-8 worker threads)
    ├── Worker 1
    ├── Worker 2
    ├── ...
    └── Worker N

3.2 Configuration Parameters

Key Configuration:

Parameter	Default	Description	Optimization Suggestion
min_workers	2	Minimum Worker count	Set based on minimum concurrency requirements
max_workers	8	Maximum Worker count	Set based on the number of CPU cores
queue_depth_threshold	10	Scale-up trigger threshold	Adjust based on average queue depth
idle_timeout	60s	Worker idle timeout	Adjust based on task frequency

Scaling Strategy:

# Scale-up condition
if queue_depth > queue_depth_threshold and workers < max_workers:
    add_worker()

# Scale-down condition
if worker_idle_time > idle_timeout and workers > min_workers:
    remove_worker()

3.3 Performance Baseline

Test Scenario: Workflow with 100 nodes

Worker Count	Execution Time	Throughput	CPU Usage
2	45s	2.2 nodes/s	40%
4	25s	4.0 nodes/s	70%
8	15s	6.7 nodes/s	95%

Optimization Result: Increasing Worker count from 2 to 8 improves throughput by 3x

3.4 Optimization Suggestions

1. Set max_workers based on CPU cores: Typically set to 1-2 times the number of CPU cores
2. Monitor queue depth: If queue depth remains high, increase max_workers
3. Avoid over-scaling: Too many Workers can lead to context switching overhead
4. Use async I/O: For I/O-intensive tasks, use asynchronous processing to improve efficiency

Detailed Documentation: See workflow-optimization/parallel-execution.md

---

Part 4: Variable Pool Optimization

4.1 Avoid Storing Large Objects

Problem: Storing large objects (e.g., large files, long text) in the variable pool consumes significant memory.

Optimization Strategy:

# ❌ Not recommended: Storing large objects
variables = {
    "large_file": open("large_file.pdf", "rb").read(),  # 10MB
    "long_text": "..." * 100000  # 1MB
}

# ✅ Recommended: Storing references
variables = {
    "file_path": "/path/to/large_file.pdf",
    "text_summary": summarize(long_text)  # Store only the summary
}

4.2 Use References Instead of Copies

Problem: Copying variables increases memory usage and processing time.

Optimization Strategy:

# ❌ Not recommended: Copying variables
result = process_data(data.copy())

# ✅ Recommended: Using references
result = process_data(data)  # Use references for read-only operations

4.3 Clean Up Variables Timely

Problem: Unused variables occupy memory.

Optimization Strategy:

# Clean up unused variables in code nodes
def main(args):
    # Process data
    large_data = process_large_data(args["input"])
    result = extract_result(large_data)

    # Clean up large objects
    del large_data

    return {"result": result}

4.4 Variable Scope Management

Variable Types:

• System Variables: sys.query, sys.files, sys.conversation_id
• Node Variables: {{#node_id.output#}}
• Environment Variables: Configuration and secret management
• Iteration Variables: Current item and index in a loop

Optimization Suggestions:

• Use the smallest possible scope
• Avoid global variable pollution
• Release temporary variables promptly

Detailed Documentation: See workflow-optimization/variable-pool.md

---

Part 5: Concurrent Processing and Streaming Responses

5.1 Iterator Parallel Processing

Use the iterator's parallel_nums parameter to process array data in parallel.

Configuration Example:

iteration_node:
  type: iteration
  inputs:
    items: "{{#code_node.output.items#}}"
  config:
    parallel_nums: 5  # Process 5 items in parallel

Performance Comparison:

Data Size	Serial Processing	Parallel Processing (5)	Improvement
10 items	20s	5s	4x
50 items	100s	25s	4x
100 items	200s	50s	4x

5.2 Streaming Responses

Use streaming responses to improve user experience and reduce Time to First Byte (TTFB).

Implementation:

# Enable streaming in API calls
response = requests.post(
    "https://api.dify.ai/v1/workflows/run",
    headers={"Authorization": f"Bearer {api_key}"},
    json={"inputs": {...}, "response_mode": "streaming"},
    stream=True
)

for line in response.iter_lines():
    if line:
        event = json.loads(line)
        print(event["data"])

Optimization Result:

• TTFB reduced by 80%
• User-perceived response time reduced by 50%

5.3 Asynchronous Task Processing

Use Celery to process long-running tasks asynchronously.

Architecture:

API Server → Celery Queue → Worker Pool → Task Execution

Applicable Scenarios:

• Large file processing
• Batch data import
• Long-running API calls
• Scheduled tasks

5.4 Event-Driven Architecture Optimization

Dify uses an event-driven architecture where node execution generates an event stream.

Optimization Strategies:

• Use non-blocking event processing
• Batch process event queues
• Optimize event distribution logic

Detailed Documentation: See workflow-optimization/parallel-execution.md

---

Part 6: Performance Optimization Cases

Case 1: Database Query Optimization (N+1 Problem)

Applicable Scenarios:

• Need to query multiple related data points in a workflow
• Loop-based database queries exist
• Execution time grows linearly with data size

Problem Characteristics:

• Number of queries = 1 + N (N is the number of data items)
• Frequent database connection establishment and closure
• Long execution time

Before Optimization:

# Query user list
users = query("SELECT * FROM users LIMIT 100")

# Query orders for each user (N+1 problem)
for user in users:
    orders = query(f"SELECT * FROM orders WHERE user_id = {user.id}")
    user.orders = orders

• Number of queries: 101 (1 + 100)
• Execution time: 10.5s
• Database load: High

Optimization Plan:

# Query all data at once
users = query("SELECT * FROM users LIMIT 100")
user_ids = [user.id for user in users]

# Batch query orders
orders = query(f"SELECT * FROM orders WHERE user_id IN ({','.join(map(str, user_ids))})")

# Associate data in memory
orders_by_user = {}
for order in orders:
    if order.user_id not in orders_by_user:
        orders_by_user[order.user_id] = []
    orders_by_user[order.user_id].append(order)

for user in users:
    user.orders = orders_by_user.get(user.id, [])

After Optimization Result:

• Number of queries: 2 (reduced by 98%)
• Execution time: 2.1s (reduced by 80%)
• Database load: Low

Expected Benefits:

• Short-term: 80% reduction in execution time, lower database load
• Long-term: Support for larger data sizes, improved system scalability
• ROI: 2 hours of development cost, saves 30% monthly server costs

---

Case 2: Large-Scale Node Rendering Optimization (100+ Nodes)

Applicable Scenarios:

• Workflow contains 100+ nodes
• Frontend rendering is slow
• Editor response is sluggish

Problem Characteristics:

• Page load time > 5s
• Significant lag when dragging nodes
• High browser memory usage

Before Optimization:

• Number of nodes: 150
• Page load time: 8.5s
• Memory usage: 500MB
• User experience: Severe lag

Optimization Plan:

1. Virtual Scrolling: Render only visible nodes
2. Node Lazy Loading: Load node details on demand
3. Canvas Optimization: Use Canvas instead of DOM rendering
4. State Management Optimization: Reduce unnecessary re-renders

After Optimization Result:

• Page load time: 2.1s (reduced by 75%)
• Memory usage: 150MB (reduced by 70%)
• User experience: Smooth

Expected Benefits:

• Short-term: Significant improvement in user experience
• Long-term: Supports more complex workflow designs
• ROI: 1 week of development cost, 40% increase in user satisfaction

---

Case 3: Workflow Log Deletion Performance Optimization

Applicable Scenarios:

• Need to periodically clean up historical logs
• Large volume of log data (millions)
• Long deletion operation time

Problem Characteristics:

• Deletion operations block the database
• Affects normal business operations
• Execution time is unpredictable

Before Optimization:

• Number of logs: 5 million
• Deletion time: 45 minutes
• Database lock: Long table lock
• Business impact: Severe

Optimization Plan:

# Before optimization: Delete all at once
DELETE FROM workflow_logs WHERE created_at < '2024-01-01';

# After optimization: Batch deletion
batch_size = 10000
while True:
    deleted = DELETE FROM workflow_logs
              WHERE created_at < '2024-01-01'
              LIMIT batch_size;

    if deleted < batch_size:
        break

    # Short sleep to release database lock
    sleep(0.1)

After Optimization Result:

• Deletion time: 50 minutes (slightly increased, but no business blocking)
• Database lock: Short-term lock (0.1s)
• Business impact: None

Expected Benefits:

• Short-term: Business operations are unaffected
• Long-term: Improved database stability
• ROI: 4 hours of development cost, avoids business interruption losses

---

Case 4: TiDB Distributed Storage Integration

Applicable Scenarios:

• Large knowledge base data (TB level)
• Performance bottleneck with single-machine PostgreSQL
• Requires horizontal scaling capability

Problem Characteristics:

• Query response time > 5s
• Database CPU usage > 90%
• Cannot scale further

Before Optimization:

• Database: PostgreSQL (single machine)
• Data volume: 2TB
• Query response time: 8.5s
• Scaling capability: Limited

Optimization Plan:

1. Migrate to TiDB: Distributed SQL database
2. Vector Index Optimization: Use TiDB Vector to accelerate retrieval
3. Read/Write Separation: Distribute read requests to multiple replicas
4. Table Partitioning: Partition by time to improve query efficiency

After Optimization Result:

• Query response time: 1.2s (reduced by 86%)
• Database CPU: 40% (reduced by 50%)
• Scaling capability: Supports 10x data volume

Expected Benefits:

• Short-term: Significant improvement in query performance
• Long-term: Supports rapid business growth
• ROI: 2 weeks of migration cost, supports 10x business growth

---

Case 5: Higress AI Gateway High-Availability Deployment

Applicable Scenarios:

• Production environment requires high availability
• Multi-LLM model load balancing
• Requires failover capability

Problem Characteristics:

• Single point of failure risk
• Unstable LLM API calls
• Cannot handle traffic spikes

Before Optimization:

• Architecture: Single Dify instance
• Availability: 95%
• Fault recovery time: 10 minutes
• Traffic spikes: Cannot handle

Optimization Plan:

# Higress AI Gateway configuration
apiVersion: networking.higress.io/v1
kind: McpBridge
metadata:
  name: dify-gateway
spec:
  replicas: 3  # 3 instances
  loadBalancer:
    algorithm: round-robin
  healthCheck:
    enabled: true
    interval: 10s
  failover:
    enabled: true
    maxRetries: 3

Architecture:

User Request → Higress Gateway → [Dify Instance 1, Dify Instance 2, Dify Instance 3]
                ↓
          Load Balancing + Failover

After Optimization Result:

• Availability: 99.9%
• Fault recovery time: < 1 second (automatic switch)
• Traffic spikes: Supports 10x traffic

Expected Benefits:

• Short-term: Significant improvement in system stability
• Long-term: Supports business-critical applications
• ROI: 3 days of deployment cost, avoids business interruption losses

---

Part 7: Performance Testing and Monitoring

7.1 Performance Indicator System

Workflow Execution Metrics:

Metric	Target Value	Description
Average Execution Time	< 2s	Complete workflow execution time
P95 Execution Time	< 5s	Execution time for 95% of requests
P99 Execution Time	< 10s	Execution time for 99% of requests
Node Success Rate	> 99%	Percentage of successful node executions
API Call Success Rate	> 99.5%	External API call success rate
Concurrent Processing Capacity	> 100 QPS	Requests processed per second

Resource Usage Metrics:

Metric	Target Value	Description
CPU Usage	< 70%	Average CPU usage
Memory Usage	< 80%	Average memory usage
Database Connection Count	< 80%	Connection pool usage
Redis Memory	< 70%	Redis memory usage

Cost Metrics:

Metric	Target Value	Description
Token Usage Cost	< Budget	LLM API cost
Infrastructure Cost	< Budget	Server, database costs
Cost per User	Continuously decreasing	Cost per user

7.2 Monitoring Tools

LLMOps Monitoring:

# Dify built-in monitoring
from dify import Monitor

monitor = Monitor()

# Log workflow execution
monitor.log_workflow_execution(
    workflow_id="xxx",
    execution_time=2.5,
    node_count=10,
    token_usage=1500,
    cost=0.003
)

# Query statistics
stats = monitor.get_workflow_stats(
    workflow_id="xxx",
    time_range="7d"
)

Execution Tracing:

• Workflow execution history
• Node run step statistics
• LLM usage statistics
• Error logs and retry records

Resource Monitoring:

# Prometheus + Grafana
# Monitor CPU, memory, network, disk

# Example query
rate(dify_workflow_execution_total[5m])  # Workflow execution rate
histogram_quantile(0.95, dify_workflow_duration_seconds)  # P95 execution time

7.3 Performance Testing Methods

Load Testing:

# Use Locust for load testing
from locust import HttpUser, task, between

class DifyUser(HttpUser):
    wait_time = between(1, 3)

    @task
    def run_workflow(self):
        self.client.post(
            "/v1/workflows/run",
            headers={"Authorization": f"Bearer {self.api_key}"},
            json={"inputs": {"query": "Test question"}}
        )

# Run test
# locust -f load_test.py --users 100 --spawn-rate 10

Stress Testing:

# Use Apache Bench
ab -n 1000 -c 50 -H "Authorization: Bearer xxx" \
   -p request.json \
   https://api.dify.ai/v1/workflows/run

Benchmarking:

# Performance benchmarking
import time

def benchmark_workflow(workflow_id, iterations=100):
    times = []
    for i in range(iterations):
        start = time.time()
        run_workflow(workflow_id)
        end = time.time()
        times.append(end - start)

    return {
        "avg": sum(times) / len(times),
        "p50": sorted(times)[len(times) // 2],
        "p95": sorted(times)[int(len(times) * 0.95)],
        "p99": sorted(times)[int(len(times) * 0.99)]
    }

7.4 Performance Analysis Tools

Python Profiler:

import cProfile