Skip to content

Mock Vs Real Execution

The `--mock` flag in `agentsmarket run` intercepts LLM API calls (returns canned responses) but **does not** intercept marketplace HTTP calls for `uses:` skill

v0.4.0concept

Mock vs real execution (CLI --mock flag)

The --mock flag in agentsmarket run intercepts LLM API calls (returns canned responses) but does not intercept marketplace HTTP calls for uses: skill stages.

Why mock does not cover uses:

uses: stages fetch skills from https://api.agentsmarket.world/v1/skills/:id (or self-hosted URL via AGENTSMARKET_URL env var). Mock intercepts only the LLM provider layer; marketplace HTTP is a separate concern.

Implication for offline runs

agentsmarket run pipeline.yaml --mock requires:

  • All LLM stages OK (mocked, no network)
  • All uses: stages still need network access (or self-hosted marketplace endpoint)

For fully-offline runs: --offline-skills flag is planned for v0.4 with skills.lock content-hash resolved at execute time. Not in v0.3.

What this means for tests

CI smoke tests run validate pipeline.yaml (validate-only, no uses: fetches). Pipeline-level integration tests that exercise uses: require network or self-hosted marketplace endpoint — see R11 docker image + R10.4 parallel integration suite.

Behavioral oracle (post-R11)

  • Docker image with --mock works for validate + LLM-only stages (R11 docker-publish.yml).
  • CI smoke tests: .github/workflows/ci.yml lines exercising docker run ... agentsmarket-runtime:dev validate ....