The AI Orchestrator: Software Engineering's New Role

Andrej Karpathy, former Tesla AI director and OpenAI researcher, has been articulating a vision for the future of software engineering that’s worth understanding. He calls it “LLM orchestration”: the discipline of directing AI models to build software rather than writing code directly.

This isn’t about AI replacing developers. It’s about developers shifting from writing code to orchestrating AI systems that write code. The skills that matter are changing, and the engineers who adapt early will have significant advantages.

Here’s what the AI Orchestrator role looks like in practice.

What Is AI Orchestration?

The Traditional Developer Role

Traditional software engineering involves:

Activity	Time Allocation
Writing code	40%
Reading code	20%
Debugging	20%
Planning/designing	15%
Communication	5%

The primary activity is directly producing code.

The Orchestrator Role

AI orchestration shifts the allocation:

Activity	Time Allocation
Prompting and directing AI	25%
Reviewing AI output	30%
Debugging and verification	20%
Planning/designing	20%
Communication	5%

The primary activity is directing and verifying AI-generated code.

The Key Difference

Aspect	Traditional	Orchestrator
Output mechanism	Write code	Prompt AI to write code
Quality control	Self-review	Verify AI output
Expertise needed	Syntax, patterns, libraries	System design, requirements, judgment
Scalability	Limited by typing speed	Limited by orchestration ability

The orchestrator doesn’t need to know every API detail. They need to know what they want and verify they got it.

Skills for AI Orchestration

Core Capabilities

Skill	Why It Matters	How to Develop
Requirements decomposition	AI needs clear direction	Practice breaking down features into atomic prompts
System architecture	AI generates code, not systems	Study design patterns, trade-offs
Code review	AI makes subtle errors	Practice catching AI-specific mistakes
Prompt engineering	Better prompts, better output	Experiment with different prompting strategies
Verification design	Trust requires proof	Build comprehensive test suites

The Prompting Skill

Effective prompting is more nuanced than asking nicely:

Prompt Element	Purpose	Example
Context	Frame the problem	”In a Next.js app using App Router…”
Constraints	Define boundaries	”Use TypeScript, no external dependencies”
Examples	Show desired pattern	”Follow this pattern: [example code]“
Edge cases	Specify handling	”Handle null values by returning empty array”
Quality bar	Set expectations	”Include error handling and JSDoc comments”

A well-structured prompt produces better output than a vague request.

The Verification Skill

AI-generated code requires different verification than human-written code:

Verification Type	What to Check	Why AI Needs It
Functional correctness	Does it do what was asked?	AI may misunderstand requirements
API accuracy	Do imported functions exist?	AI hallucinates APIs
Edge cases	Does it handle corner cases?	AI often misses edge cases
Security	Any vulnerabilities?	AI trained on vulnerable code
Performance	Will it scale?	AI optimizes for correctness, not performance

The Orchestration Workflow

A Practical Example

Building a feature with AI orchestration:

Step 1: Decompose the Feature

Instead of prompting for the whole feature, break it down:

Feature: User notification preferences

Components needed:
1. Database schema for preferences
2. API endpoints (GET, PUT)
3. React form component
4. Integration with notification service
5. Tests for each component

Step 2: Prompt Each Component

Create a PostgreSQL schema for user notification preferences.

Requirements:
- User ID (foreign key to users table)
- Email notifications (boolean, default true)
- Push notifications (boolean, default true)
- Notification categories (JSON for flexibility)
- Created/updated timestamps

Output: SQL migration file with comments

Step 3: Review Output

-- AI output
CREATE TABLE notification_preferences (
  id SERIAL PRIMARY KEY,
  user_id INTEGER REFERENCES users(id) ON DELETE CASCADE,
  email_enabled BOOLEAN DEFAULT true,
  push_enabled BOOLEAN DEFAULT true,
  categories JSONB DEFAULT '{}',
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

-- Review: Looks correct, but missing unique constraint on user_id
-- Fix: Add UNIQUE constraint to prevent duplicate preferences

Step 4: Iterate and Verify

Continue for each component, verifying as you go. Run tests after each piece.

Orchestrator vs. Writer Output

Metric	Direct Writing	Orchestration
Time to first draft	Longer	Shorter
Time to production	Variable	Often faster (with good verification)
Code consistency	Variable	Higher (AI is consistent)
Bug types	Mixed	Subtle, harder to find
Knowledge captured	In developer’s head	In prompts and documentation

The Economics of Orchestration

Productivity Calculation

For a typical feature:

Approach	Write Time	Review Time	Debug Time	Total
Direct writing	4 hours	1 hour	1 hour	6 hours
Orchestration	1 hour	2 hours	1.5 hours	4.5 hours

Orchestration wins when:

• Review skills are strong
• Verification is systematic
• Prompts are well-structured

When Orchestration Loses

Scenario	Why Orchestration Struggles
Novel algorithms	AI doesn’t know the approach
Domain-specific logic	AI lacks context
Legacy system integration	AI doesn’t know your codebase history
Performance-critical code	AI optimizes for correctness
Security-sensitive code	AI may introduce vulnerabilities

The orchestrator must recognize when to write directly.

Building an Orchestrator Team

Team Composition

Role	Count (10-person team)	Focus
Senior Orchestrators	4	Architecture, complex features
Mid-level Orchestrators	4	Standard features, verification
Junior Developers	2	Learning, simple tasks (limited)

The pyramid inverts toward senior-heavy teams.

Interview Criteria

When hiring for orchestration skills:

Assess For	How to Test
Decomposition	Give vague feature, evaluate breakdown
Prompting	Review their prompts for an AI-assisted task
Verification	Debug AI-generated code with subtle errors
Judgment	When would they write directly vs. orchestrate?

Training Existing Developers

Phase	Focus	Duration
1: Tool familiarization	Basic AI tool usage	1-2 weeks
2: Prompting practice	Effective prompt construction	2-4 weeks
3: Verification rigor	Catching AI errors	2-4 weeks
4: Workflow integration	Full orchestration workflow	1-2 months

The Future of the Role

Near Evolution (1-2 Years)

Change	Impact
Better AI context	Less decomposition needed
Integrated tools	Smoother workflow
Standardization	Established best practices
Certification	Formal recognition of skills

Medium Evolution (3-5 Years)

Change	Impact
Agent-based development	AI handles more autonomously
Reduced code review	Higher AI reliability
Shift to design focus	Less code, more architecture
New specializations	Domain-specific orchestration

Long Evolution (5+ Years)

The role may converge with product management, system design, or split into new specializations entirely. The one constant: human judgment remains valuable even as implementation becomes automated.

Common Mistakes

Mistake	Why It Hurts
Trusting AI completely	Subtle bugs reach production
Poor prompting	More revision time negates speed gains
Skipping verification	AI errors compound
Orchestrating everything	Some code is faster to write directly
Not documenting prompts	Knowledge isn’t captured

---

AI orchestration is becoming a core engineering skill. If you’re looking for a development partner who understands both the power and limitations of AI-assisted development, book a consultation. We’ve built orchestration into our workflow while maintaining the verification rigor production systems require.