Env Var Substitution
Pipelines reference environment variables via bash-style `${VAR}` / `${VAR:-default}`.
Environment variable substitution (v0.3+)
Pipelines reference environment variables via bash-style ${VAR} / ${VAR:-default}.
Substitution runs before yaml.load (text preprocessing), so ${VAR} works inside
any string position — top-level scalars, array elements, nested map values, multi-line
block scalars. Inspired by 12-factor (bash/zsh) convention.
Syntax
| Pattern | env present | env absent |
|---|---|---|
${VAR} |
env value | throws MissingEnvVarError |
${VAR:-default} |
env value (default ignored) | default value |
${VAR:-} |
env value (incl. empty string) | empty string |
Var-name regex: [A-Z_][A-Z0-9_]* — uppercase letter or underscore start, then uppercase
letters / digits / underscores. Lowercase first char not matched (left literal as ${lowercase}).
Single-pass substitution (not recursive).
3-layer defense (where substitution fires)
| Layer | Entry-point | Error response |
|---|---|---|
| 1. validate CLI | agentsmarket validate pipeline.yaml |
exit 1: ✗ Environment variable "X" is required (no default specified) |
| 2. run CLI | agentsmarket run pipeline.yaml |
exit 1: same text |
| 3. server publish-pipeline | POST /v1/pipelines |
HTTP 400 { code: 'MISSING_ENV_VAR', var: 'X', error } |
The /v1/pipelines/:id/invoke route does not re-substitute — spec_yaml stored in D1
is already substituted at publish-time, and re-substituting at invoke would be wrong
(env may have changed between publish and invoke).
Where env vars come from
- validate / run CLI:
process.envof the CLI process (shell,.env, Docker-e, CI secrets) - Server publish-pipeline: server’s
process.envat request-time (never trust caller-provided secrets) - Hosted operator runs: hosted runner env
Examples (5 example pipelines demonstrate)
All 5 examples declare at least one ${VAR:-default} literal so they remain
runnable without any env set; override env to customize tone/persona:
parse_logs.system: ${BUG_TRIAGE_TONE:-You are an SRE on-call engineer. Triage with speed and precision.}
analyze.system: ${EXTRA_GUIDANCE:-You are a senior code reviewer. Identify improvement opportunities. Be concise and specific.}
per_article_summary.system: ${NEWS_READER_TONE:-You write for a busy professional. Skip boilerplate, name tradeoffs, lead with new facts.}
per_review.system: ${FOCUS_AREA:-You analyze customer reviews for overall satisfaction, themes, and concrete improvement signals.}
analyze.system: ${ANALYSIS_DEPTH:-You analyze source code for issues, complexity, and lines of code.}Pre-existing interpolation patterns (not affected)
| Pattern | Engine | Example |
|---|---|---|
{{var}} |
stage-input interpolation | prompt: "Hello {{name}}" |
$input.X |
input literal | prompt: "$input.user_query" |
$stages.X.output |
stage-output chaining | prompt: "Prev: $stages.analyse.output" |
These use different syntax (double-brace for inputs, single-dollar-no-brace for input/stage
references) and do not conflict with ${VAR}.
JSON output mode (validate CLI)
{
"ok": false,
"error": "Environment variable \"MISSING_VAR\" is required (no default specified)",
"var": "MISSING_VAR"
}Behavioral oracle (tests = ground truth for documented behavior)
packages/pipeline-runtime/tests/resolve-vars.test.ts— R7.1 (13) + R7.6 (6): substitution unit-tests covering arrays / nested maps / block scalars / special chars in defaults / empty defaults / recursive throw across YAML tree.packages/pipeline-runtime/tests/examples.test.ts— R7.5 (11): example pipelines declare${VAR:-default}, parse with empty env, env-override propagates into stage.packages/cli/tests/validate.test.ts— R7.2: validate CLI fail-fast on missing var.packages/cli/tests/run-registry.test.ts— R7.3: run CLI fail-fast + env-override visible in startup log.packages/server/tests/publish-pipeline-r6.test.ts— R7.4: server 400MISSING_ENV_VAR.
If this doc contradicts tests → tests are the source of truth (PR a fix to this page).