Parallel Stages

Parallel stages (v0.3+)

Stages without depends_on run concurrently (implicit DAG edges).

Implicit parallelism

stages:
  - id: analyze-code
    provider: openai
    prompt: "Analyze: {{ inputs.code }}"

  - id: analyze-style
    provider: openai
    prompt: "Style: {{ inputs.code }}"

  - id: synthesize
    depends_on: [analyze-code, analyze-style]
    provider: openai
    prompt: "Synthesize..."

analyze-code and analyze-style run in parallel; synthesize waits for both.

Fan-out over collections: parallel.of

- id: scan-endpoints
  parallel:
    of: "{{ inputs.endpoints }}"
    max_concurrent: 5
    body:
      provider: openai
      prompt: "Scan {{ item.url }}..."

Concurrency controls

Streaming interleave

Parallel stages with streaming.enabled: true emit to a shared event bus. Consumers subscribe by event_pipeline tag. Backpressure: pause slow consumer, fast producers queue.

Behavioral oracle

• packages/pipeline-runtime/tests/parallel.test.ts — R10.4 topological sort + Promise.all: independent stages execute concurrently; depends_on enforced; one failure does not abort siblings (status reported per-stage, caller decides)
• packages/pipeline-runtime/tests/integration-r10-x.test.ts — end-to-end through runPipelineV2: 4-way fan-out, dependency-merge stage waits for all, partial-failure observed in provenance
• Composition (R13) is documented in the next section: Composition: uses: accepts pipeline IDs