---
name: cost-aware-llm-pipeline
description: Cost optimization for AI development pipelines — model routing, budget tracking, token optimization. Use when managing multi-step AI workflows, reducing API costs, or selecting models for tasks.
---

# Cost-Aware LLM Pipeline

Based on ECC cost-aware-llm-pipeline patterns.

## Model Selection Strategy

Match model capability to task complexity:

| Task Type | Model Tier | Examples |
|-----------|-----------|----------|
| Simple (summarize, classify, format) | Small/Fast | Claude Haiku, GPT-4o-mini |
| Medium (code fixes, analysis, rewrite) | Mid | Claude Sonnet, GPT-4o |
| Complex (architecture, debugging, planning) | High | Claude Opus, GPT-4o, Gemini Pro |
| Creative (design, brainstorming) | Variable | Depends on breadth needed |

## Token Optimization Techniques

1. **Trim system prompts**: Remove redundant instructions. Keep only task-specific rules.
2. **Compress context**: Use summaries instead of full files. Skip irrelevant code.
3. **Background processes**: Run long tasks (test suites, builds) async. Don't waste tokens waiting.
4. **Chunk large files**: Read only relevant sections with offset/limit.
5. **Avoid loops**: Don't poll in a loop. Use proper timeouts and wait mechanisms.

## Budget Tracking

```bash
# Check session cost (if available)
session_status

# Monitor token usage
# Track: input_tokens, output_tokens, cost_cents
```

## Cost Reduction Rules

- Use the cheapest model that still works
- Batch similar requests together
- Cache responses when possible
- Avoid regenerating the same output
- Set explicit max_tokens for generation
- Use structured output (JSON) to reduce retries
- Prefer targeted file reads over broad "read everything"

## Pipeline Design

When building multi-step AI workflows:
1. Step 1: Plan/analyze with mid-tier model
2. Step 2: Implement with appropriate model for code
3. Step 3: Verify/test with cheapest model
4. Step 4: Review with high-tier model only if needed