Agent Validation Workforce

The agent validation workforce is a GitHub Actions release gate for cross-platform swarmhost agent checks. Agent-impacting pull requests and pushes to main run the default bootstrap/live Ubuntu Docker gate automatically. Operators can also dispatch the workflow manually for a selected branch, commit, or prebuilt artifact URL.

The workflow plans the supported platform/runtime matrix, prepares the candidate agent artifact and metadata, runs per-target worker jobs, uploads sanitized evidence, and prepares backlog findings for failed required targets.

The workflow is intentionally conservative: current required entries must pass, while platform/runtime combinations that are still roadmap work are represented as pending or unsupported with their tracking issue. Matrix rows also expose a CI gate_status: required, optional, or expected-failure. Only required target failures block the gate. In live mode, a required target fails if no live validation driver is configured.

Workflow

Automatic triggers run on PRs and main pushes that touch the swarmhost agent, installer scripts, served artifact routes, runtime contracts, deployment orchestration, file gateway, backup paths, support diagnostics, or the validation workflow itself.

Manual run:

Actions -> agent-validation-workforce -> Run workflow

Inputs:

Input Default Use
candidate_ref workflow ref Branch, tag, or SHA to validate.
candidate_artifact_url empty Optional prebuilt swarm-agent artifact URL. The URL is hashed in evidence and is never printed.
candidate_version empty Optional expected agent_version for a supplied artifact.
matrix_profile bootstrap bootstrap, linux, fedora-rhel, opensuse-arch, windows, or full.
validation_depth preflight preflight records readiness; live requires runtime drivers.
runner_labels_json ["self-hosted","swarmhosts","codex-workspace","agent-validation"] Runner labels for the expensive jobs.
target_filter empty Comma-separated target IDs within the selected profile for a focused run.
build_candidate_agent true Build downloads/swarm-agent and pass it to target jobs when no artifact URL is supplied.

Automatic PR and main push runs use:

  • candidate_ref: the workflow commit SHA.
  • matrix_profile: bootstrap.
  • target_filter: linux-ubuntu-lts-docker.
  • validation_depth: live.
  • runner_labels_json: the Codex workspace self-hosted runner labels.
  • build_candidate_agent: true.

That default is the currently runnable live release gate for the shared codex-workspace-agent-validation runner. The Ubuntu Docker target uses the checked-in Linux live driver directly on the self-hosted runner, installs the candidate artifact, registers a fresh host, creates a disposable Minecraft deployment, proves Java status/RCON/logs/lifecycle/cleanup evidence, and uploads the sanitized target bundle. Manual dispatch with an empty target_filter runs the full selected profile, including Debian, Podman, openSUSE/Arch, and Windows rows that require target-specific SSH/live-driver configuration.

Manual dispatch with candidate_artifact_url skips the build and downloads the artifact to the candidate bundle. Provide candidate_version when the artifact must be compared with live heartbeat agent_version evidence.

The bootstrap profile currently includes:

  • linux-ubuntu-lts-docker: required Linux Docker baseline.
  • linux-debian-stable-docker: required Debian-family Docker baseline for #158.
  • linux-ubuntu-lts-podman-rootful: required Ubuntu rootful Podman baseline after #156.
  • linux-debian-stable-podman-rootful: required Debian rootful Podman baseline for #158.
  • windows-native-foundation: required live target for Windows service, registration, heartbeat, and capability parity.

Use linux, fedora-rhel, opensuse-arch, windows, or full for broader parity runs. The Windows profile includes windows-native-foundation as required, windows-native-minecraft as an intentionally retired fallback boundary, and windows-linux-container-runtime as optional until the Linux-container runtime work records proof.

Windows validation prerequisites and known gaps

For preflight and release planning, treat each Windows target as a separate parity gate:

  • windows-native-foundation: registration, service lifecycle, and shared control-plane contract must be stable before any Windows deployment parity claim.
  • windows-native-minecraft: retained as a retired fallback path. It is not part of support parity and should remain excluded from release-gate evidence.
  • windows-linux-container-runtime remains pending until a host can prove docker info returns Linux-container readiness, relay/public endpoint behavior, and file-manager/log/diagnostic parity for supported runtimes.

Known gaps before parity: firewall rule persistence in complex port-sharing scenarios, live file-manager action proof across world paths, and reusable relayed endpoint proof for Windows runtimes remain tracked in #152 and #153.

The fedora-rhel profile is the focused issue #159 matrix. It includes:

  • linux-fedora-docker: required Docker proof on current stable Fedora.
  • linux-rhel-compatible-docker: required Docker proof on Rocky Linux, AlmaLinux, or another approved RHEL-compatible representative.
  • linux-fedora-podman-rootful and linux-rhel-compatible-podman-rootful: required rootful Podman proof now that #156 added Podman runtime support.
  • linux-fedora-podman-rootless and linux-rhel-compatible-podman-rootless: conditional targets with #159 live proof on Fedora 44 and Rocky 10. Keep them out of the default release gate unless the validation host has high host ports, subordinate UID/GID ranges, working rootless networking, and cleanup through the rootless Podman user namespace.

Set target-specific SSH or live-driver environment variables on the self-hosted runner before dispatching this profile. For example:

export SWARM_VALIDATION_LINUX_FEDORA_PODMAN_ROOTFUL_SSH_TARGET='fedora-validation.example'
export SWARM_VALIDATION_LINUX_FEDORA_PODMAN_ROOTFUL_LIVE_COMMAND='scripts/lab/validate-fedora-podman.sh'
export SWARM_VALIDATION_LINUX_RHEL_COMPATIBLE_PODMAN_ROOTFUL_SSH_TARGET='rocky-validation.example'
export SWARM_VALIDATION_LINUX_RHEL_COMPATIBLE_PODMAN_ROOTFUL_LIVE_COMMAND='scripts/lab/validate-rocky-podman.sh'

When dedicated Linux validation hosts are not already available, create disposable normal Linux guests on win11-validation, attach them to the external LAN switch, and assign static addresses from the approved 192.168.1.26-36 range. Install the selected runtime inside the guest, then point the target-specific SSH_TARGET and live-driver variables at that VM. Do not use Hyper-V NAT for these runtime checks because relay and public reachability proof should exercise the same routable LAN path as a real swarm host.

The opensuse-arch profile is the focused issue #160 matrix. It includes:

  • linux-opensuse-docker: required Docker proof on openSUSE Leap, with Tumbleweed acceptable as additional rolling evidence.
  • linux-arch-rolling-docker: required Docker proof on Arch Linux, with Manjaro or EndeavourOS acceptable as derivative evidence when documented.
  • linux-opensuse-podman-rootful and linux-arch-rolling-podman-rootful: required rootful Podman proof now that #156 added Podman runtime support.
  • linux-opensuse-podman-rootless and linux-arch-rolling-podman-rootless: conditional targets that also need rootless Podman safety proof before support is claimed.

Set target-specific SSH or live-driver environment variables on the self-hosted runner before dispatching this profile. For example:

export SWARM_VALIDATION_LINUX_OPENSUSE_DOCKER_SSH_TARGET='opensuse-validation.example'
export SWARM_VALIDATION_LINUX_OPENSUSE_DOCKER_LIVE_COMMAND='scripts/lab/validate-opensuse-docker.sh'
export SWARM_VALIDATION_LINUX_ARCH_ROLLING_DOCKER_SSH_TARGET='arch-validation.example'
export SWARM_VALIDATION_LINUX_ARCH_ROLLING_DOCKER_LIVE_COMMAND='scripts/lab/validate-arch-docker.sh'

The target filter is strict. A typoed target ID fails planning instead of silently producing an empty validation matrix.

Podman rootless remains conditional rather than a required bootstrap gate. Treat it as product-safe only after a distro-specific live run proves subordinate UID/GID ranges, rootless networking, high published ports, reboot/user-service survival, relay-backed Minecraft status, RCON, and cleanup.

Self-Hosted Runner

Expensive matrix jobs default to the Codex workspace runner at 192.168.1.208 instead of GitHub-hosted runners. Register one repository self-hosted runner on that host with these labels:

self-hosted,swarmhosts,codex-workspace,agent-validation

Suggested runner name:

codex-workspace-agent-validation

Install path:

/home/openclaw/actions-runner/swarmhosts-agent-validation

Setup outline on 192.168.1.208:

mkdir -p /home/openclaw/actions-runner/swarmhosts-agent-validation
cd /home/openclaw/actions-runner/swarmhosts-agent-validation

# Download the current Linux x64 runner release from GitHub's official
# self-hosted runner setup page for this repository.
tar xzf actions-runner-linux-x64-*.tar.gz

./config.sh \
  --url https://github.com/DillonHansen71/ai-agent-swarm-hosts \
  --token <RUNNER_REGISTRATION_TOKEN> \
  --name codex-workspace-agent-validation \
  --labels swarmhosts,codex-workspace,agent-validation \
  --unattended

sudo ./svc.sh install openclaw
sudo ./svc.sh start
./svc.sh status

Do not commit or paste the registration token. It is short-lived and should come from GitHub's runner registration UI or an operator-approved token request.

To disable or remove the runner:

cd /home/openclaw/actions-runner/swarmhosts-agent-validation
sudo ./svc.sh stop
sudo ./svc.sh uninstall
./config.sh remove --token <RUNNER_REMOVAL_TOKEN>

Live Drivers

Live validation is intentionally delegated to target-specific commands on the self-hosted runner so lab SSH keys, local VM names, and staging credentials stay out of the repository. The workflow passes these environment variables to the driver:

Variable Meaning
SWARM_VALIDATION_TARGET_ID Target ID, such as linux-ubuntu-lts-docker.
SWARM_VALIDATION_PLATFORM Target platform.
SWARM_VALIDATION_RUNTIME Runtime ID.
SWARM_VALIDATION_AGENT_ARTIFACT Path to the candidate agent artifact.
SWARM_VALIDATION_ARTIFACT_DIR Directory for sanitized output.
SWARM_VALIDATION_RESULT_JSON Optional JSON file path for structured results.

Configure a live driver with a target-specific environment variable on the self-hosted runner:

export SWARM_VALIDATION_LINUX_UBUNTU_LTS_DOCKER_LIVE_COMMAND='scripts/lab/validate-ubuntu-docker.sh'

linux-ubuntu-lts-docker has a built-in live driver enabled in the workflow by SWARM_VALIDATION_ENABLE_BUILTIN_LINUX_LIVE=1. It runs scripts/validation/linux_live_runtime_validation.py --local on the self-hosted runner, using $RUNNER_TEMP for the validation install/data roots. The workflow passes repository secrets named APP_BASE_URL, SWARM_API_TOKEN, DOCKER_HUB_REPO, and registry credential variants into the job. If a target-specific LIVE_COMMAND is set, it takes precedence over the built-in driver. The default built-in validation range is 32600-32699, which avoids the existing broad Windows lab range on the shared public IP. Override it with SWARM_VALIDATION_PORT_RANGE_START and SWARM_VALIDATION_PORT_RANGE_END only after checking for registered-host overlap.

For the Windows lab host, configure the Windows targets to call the parity validator. The validator can run directly on win11-validation, or through an operator-owned wrapper that copies/invokes it on that host:

export SWARM_VALIDATION_WINDOWS_NATIVE_FOUNDATION_LIVE_COMMAND='pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/validation/windows_agent_parity.ps1 -ValidationDepth live'
export SWARM_VALIDATION_WINDOWS_NATIVE_MINECRAFT_LIVE_COMMAND='pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/validation/windows_agent_parity.ps1 -ValidationDepth live -CreateMinecraftDeployment -Cleanup'

The command must install or update the candidate agent artifact, register against an isolated test control plane or controlled staging target, run the standard live checklist, and write safe JSON to $SWARM_VALIDATION_RESULT_JSON.

When candidate metadata includes a version, the workforce wrapper exports SWARM_VALIDATION_EXPECTED_AGENT_VERSION. Required Linux live targets must report a matching agent_version in sanitized evidence, usually under host.agent_version, before the target is treated as passing.

Expected JSON shape:

{
  "deployment_id": "dep-redacted-or-safe-id",
  "host_id": "host-redacted-or-safe-id",
  "runtime_id": "docker:container-id-or-safe-summary",
  "live_validation": {
    "install": {"status": "pass", "detail": "candidate artifact installed"},
    "registration": {"status": "pass", "detail": "host registered"},
    "heartbeat": {"status": "pass", "detail": "heartbeat accepted"},
    "capability_matrix": {"status": "pass", "detail": "Windows support posture recorded"},
    "runtime_readiness": {"status": "pass", "detail": "native or container runtime ready"},
    "deployment_lifecycle": {"status": "pass", "detail": "create/start/stop/restart/delete passed"},
    "minecraft_status": {"status": "pass", "detail": "Minecraft Java status handshake passed"},
    "rcon": {"status": "pass", "detail": "RCON command delivered"},
    "logs": {"status": "pass", "detail": "logs collected and sanitized"},
    "cleanup": {"status": "pass", "detail": "no orphaned process/container/listener"},
    "support_report": {"status": "pass", "detail": "support report safe to paste"}
  }
}

Never print registration tokens, host credentials, kubeconfigs, SSH private keys, or registry tokens. The orchestrator redacts common token-shaped strings, but drivers should avoid emitting secrets in the first place.

Windows Lab Command

Run the Windows parity validator from an elevated PowerShell session on win11-validation:

.\scripts\validation\windows_agent_parity.ps1 -ValidationDepth live -CreateMinecraftDeployment -Cleanup -ArtifactDir C:\ProgramData\SwarmHosts\validation

Inputs can also come from environment variables:

Variable Meaning
SWARM_VALIDATION_SERVER_URL Control-plane URL, defaulting to https://swarmhosts.com.
SWARM_API_TOKEN User API token used only when -CreateMinecraftDeployment queues and cleans up a disposable Minecraft deployment.
SWARM_USER_TOKEN Optional first-registration token used only when -Install configures the service.
SWARM_VALIDATION_RESULT_JSON Exact sanitized evidence JSON path for workflow ingestion.
SWARM_WINDOWS_LAB_TARGET_HOST Lab hostname label, defaulting to win11-validation.

Without -CreateMinecraftDeployment, the script still validates PowerShell syntax, agent binary presence, service status, persisted identity, and a real heartbeat POST using the host token from identity.json. With -CreateMinecraftDeployment, it queues a disposable Minecraft deployment, waits for running, waits for the control-plane public Minecraft Java status proof, observes the deployment as stable for 300 seconds by default, sends a Minecraft game-command/RCON action, collects logs, and deletes the deployment when -Cleanup is set. Disposable deployment creation is retried twice by default; pass -DeploymentAttempts 1 when using an explicit -DeploymentId or when a single-attempt failure is required for troubleshooting. Pass -StableSeconds 0 only for targeted troubleshooting where the 5-minute stability proof is intentionally skipped.

Artifacts

Each target uploads:

  • target.json: machine-readable target evidence.
  • target.md: pasteable Markdown summary.
  • live-stdout.log and live-stderr.log when a live driver runs.

The candidate artifact job uploads:

  • swarm-agent
  • candidate-agent.json: candidate source, ref, SHA, version, workflow run id, and artifact URL hash when an artifact URL was used.

The summary job uploads:

  • summary.json
  • summary.md
  • findings.json
  • findings.md

For manual and main push runs, failed required targets create or update duplicate-safe GitHub backlog findings. Pull request runs do not create issues; they upload the same findings.json and findings.md artifacts for review.

The workflow should be treated as a release gate only when required targets run with validation_depth=live and all required live checklist steps pass.