1.9 KiB
1.9 KiB
name, description
| name | description |
|---|---|
| cost-aware-llm-pipeline | Cost optimization for AI development pipelines — model routing, budget tracking, token optimization. Use when managing multi-step AI workflows, reducing API costs, or selecting models for tasks. |
Cost-Aware LLM Pipeline
Based on ECC cost-aware-llm-pipeline patterns.
Model Selection Strategy
Match model capability to task complexity:
| Task Type | Model Tier | Examples |
|---|---|---|
| Simple (summarize, classify, format) | Small/Fast | Claude Haiku, GPT-4o-mini |
| Medium (code fixes, analysis, rewrite) | Mid | Claude Sonnet, GPT-4o |
| Complex (architecture, debugging, planning) | High | Claude Opus, GPT-4o, Gemini Pro |
| Creative (design, brainstorming) | Variable | Depends on breadth needed |
Token Optimization Techniques
- Trim system prompts: Remove redundant instructions. Keep only task-specific rules.
- Compress context: Use summaries instead of full files. Skip irrelevant code.
- Background processes: Run long tasks (test suites, builds) async. Don't waste tokens waiting.
- Chunk large files: Read only relevant sections with offset/limit.
- Avoid loops: Don't poll in a loop. Use proper timeouts and wait mechanisms.
Budget Tracking
# Check session cost (if available)
session_status
# Monitor token usage
# Track: input_tokens, output_tokens, cost_cents
Cost Reduction Rules
- Use the cheapest model that still works
- Batch similar requests together
- Cache responses when possible
- Avoid regenerating the same output
- Set explicit max_tokens for generation
- Use structured output (JSON) to reduce retries
- Prefer targeted file reads over broad "read everything"
Pipeline Design
When building multi-step AI workflows:
- Step 1: Plan/analyze with mid-tier model
- Step 2: Implement with appropriate model for code
- Step 3: Verify/test with cheapest model
- Step 4: Review with high-tier model only if needed