Game Server Validation Workforce

The game-server-validation-workforce GitHub Actions workflow validates supported game server catalog, image, and version changes before they become customer-impacting runtime regressions.

It is intentionally Minecraft-first. Minecraft Java is the required live target because it is the flagship path. Other public launch games are explicit optional targets until each game has its own protocol proof driver. Credential-gated, hidden, and unvalidated catalog entries do not block the required gate unless the catalog readiness metadata promotes them.

When It Runs

The workflow runs for pull requests and pushes to main that touch game server images, the game catalog, game version sync code, publish scripts, the validation workforce script, or related tests.

It can also be dispatched manually:

Actions -> game-server-validation-workforce -> Run workflow

Manual inputs:

Input Use
candidate_ref Branch, tag, or SHA to validate. Defaults to the workflow SHA.
game_filter Comma-separated game ids, such as minecraft or minecraft,css.
candidate_version Managed game version to request when creating the deployment.
candidate_image Candidate image reference to record in evidence. The public API still validates managed catalog images.
validation_depth preflight checks catalog contracts; live creates disposable deployments.
runner_labels_json Self-hosted runner label list for live jobs.

Target Selection

Automatic runs inspect changed paths:

  • images/games/<game>/..., docs/games/<game>.md, or that game's publish script select the matching game.
  • Shared catalog, game-version automation, workflow, or workforce-script changes select the required gate, currently Minecraft.
  • Credential-gated or unvalidated game-specific changes select no required target and pass with a no-target summary.

Catalog readiness controls gate status:

Catalog status Gate behavior
flagship and public_ready=true Required live target.
validated and public_ready=true Optional target; evidence is useful but non-blocking.
credential_gated, needs_validation, needs_repair, or hidden Excluded from the required gate.

Live Minecraft Proof

For Minecraft live validation, the runner needs a control-plane user token and a validation target:

APP_BASE_URL=https://swarmhosts.com
SWARM_API_TOKEN=<secret user token>
SWARM_HOST_ID=<system-owned validation swarm host id>

SWARM_GAME_VALIDATION_HOST_ID may be used instead of SWARM_HOST_ID. SWARM_GAME_VALIDATION_DEPLOYMENT_TARGET_ID may be used when the workflow should select a host from a managed deployment target.

The live proof creates a disposable deployment through POST /api/deployments with relay_mode=automatic, waits for running with a concrete container id, checks startup logs, runs a Minecraft Java status handshake through the user-facing endpoint, queues a structured Minecraft Broadcast command template through the RCON-backed Admin Actions path, observes stability, and deletes the deployment.

Useful optional variables:

Variable Default
SWARM_GAME_VALIDATION_STABILITY_SECONDS 300
SWARM_GAME_VALIDATION_POLL_SECONDS 8
SWARM_GAME_VALIDATION_DEPLOYMENT_TIMEOUT_SECONDS 900
SWARM_GAME_VALIDATION_MINECRAFT_TIMEOUT_SECONDS 420
SWARM_GAME_VALIDATION_MEMORY_MB 2048
SWARM_GAME_VALIDATION_CPU_LIMIT 2.0

Candidate Control-Plane Runtime Proof

The live harnesses normally point at https://swarmhosts.com. That is correct for deployed behavior, but it is a false-negative path when an MR changes control-plane/API behavior and the expected response only exists in the candidate backend commit.

When a runtime issue is in status:manager-review or a blocked rescope hold with validation:failed, first classify the failure instead of rerunning the same production-only proof:

.venv/bin/python scripts/codex/factory_validation_router.py \
  --issue 303 \
  --issue-labels "status:manager-review,validation:failed,lane:runtime,needs:runtime-validation" \
  --factory-evidence-file /tmp/mr-body.md \
  --failure-text-file /tmp/live-validation-failure.txt \
  --mr-url https://git.murican.io/swarm-hosts/ai-agent-swarm-hosts/-/merge_requests/13 \
  --mr-state opened \
  --mr-merge-status can_be_merged \
  --pipeline-status success

If a candidate/staging control-plane URL exists, pass --candidate-control-plane-url and rerun the live proof with APP_BASE_URL/SWARMHOSTS_BASE_URL pointed at that URL.

If staging is not available, use the serialized post-merge proof gate:

  1. Add status:validation, gate:validation, and validation:post-merge-proof.
  2. Remove status:manager-review, status:blocked, scope:needs-rewrite, and validation:failed so the independent validator does not repeat the same production-only run.
  3. The finalizer merges one ready MR, deploys that exact control-plane commit, runs the live proof, and records sanitized live-result evidence before any done/final label is applied.
  4. If the live proof fails after deploy, requeue with evidence, revert, or block with the exact failure. Unit tests, CI, and successful deploy health checks do not replace runtime proof.

For #303 / MR !13, the restart path after this gate is:

PATH="$HOME/.local/node-v22/bin:$PATH" \
  /data/repos/ai-agent-swarm-hosts/.venv/bin/python \
  scripts/validation/minecraft_give_item_live_validation.py \
  --install-bot-deps \
  --validate-advanced-template \
  --artifact-dir /tmp/swarmhosts-issue303-runtime-fallback

The expected proof is advanced-template preview, queued RCON-backed Give Item task, successful task/history/audit semantics, stability observation, cleanup, and live-result.json. A production response such as server_catalog_required before the MR backend is deployed should remain classified as a candidate control-plane gap, not as a reason to repeatedly retry production.

Live Minecraft Give Item Proof

The structured Give Item admin-action safety-policy flow has a dedicated live validation harness. It exercises the production user API and a disposable Minecraft deployment with relay networking enabled:

set -a
source .env
set +a
.venv/bin/python scripts/validation/minecraft_give_item_live_validation.py

The harness starts controlled offline-mode test players with scripts/dev/minecraft-player-bots, refreshes cached online-player discovery with a harmless RCON list command, validates invalid and unauthenticated API rejections, updates the deployment Give Item policy, proves required-reason, blocked-item, and over-limit quantity rejections without queueing tasks, queues one-player, multi-player, and partial-failure Give Item tasks with reasons, checks task history and audit logs for policy decisions, observes stability, and deletes the validation deployment. It writes sanitized evidence to /tmp/swarmhosts-issue-298-minecraft-give-item-policy/<run>/live-result.json. The partial-failure proof uses two cache-backed validation players, then disconnects one controlled bot before queueing so the runtime task records one successful recipient and one failed recipient.

By default, this harness creates a dedicated validation server with VERSION=1.21.11 and ONLINE_MODE=false so the controlled offline-mode bots can join without Mojang/Microsoft credentials. Do not point it at customer servers or retained personal worlds.

If the local bot dependencies are missing, install them before the run or pass --install-bot-deps to let the harness run npm ci in scripts/dev/minecraft-player-bots. The bot runner requires Node.js 22.

Live Minecraft Companion Item Registry Proof (Issue #301)

301 adds companion-side item-registry discovery so Give Item workflows can

validate server-reported item IDs (including non-minecraft namespaces) and surface item metadata for modded deployments.

Current live proof prerequisites for this feature set are:

  • APP_BASE_URL, SWARM_API_TOKEN, and a validation-capable host target (SWARM_HOST_ID or SWARM_GAME_VALIDATION_HOST_ID).
  • A modded deployment in a stable running state with working structured Admin Actions / RCON delivery and shared minecraft_status protocol checks.
  • A server-side companion item-registry report that includes at least one non- minecraft: item namespace and metadata fields required by the control-plane preview path (display_name, namespace, source, and tags).
  • API and UI checks covering /api/deployments/<id>/minecraft/admin-actions/give-item/items and .../give-item/preview with a non-minecraft: sample item.
  • Queueing of one Give Item task for a modded item ID and task/audit history evidence showing acceptance/rejection semantics when stale or unavailable catalog data is used.

The existing Give Item live harness can be extended for the companion portion by passing a non-minecraft: item that the controlled modded server reports:

.venv/bin/python scripts/validation/minecraft_give_item_live_validation.py \
  --companion-item-id examplemod:ruby_ingot \
  --companion-item-query ruby

When these flags are present, the harness requests a companion catalog refresh, requires the item search response to be fresh server-verified catalog data with mod/source metadata and tags, previews the selected item through the server catalog, queues one Give Item task for it, and includes that task in the history and audit-log checks.

Known gaps before this feature can be promoted in the required gate:

  • The optional harness path now covers fresh server catalog refresh/search, preview, live delivery, task history, and audit evidence for one configured non-minecraft: item; stale catalog markers and unavailable-catalog rejection semantics still need dedicated runtime cases.
  • Existing Give Item live proof validates immediate minecraft:* policy and dispatch paths, but does not yet prove server-aware discovery, metadata persistence, or stale-while-offline behavior.
  • Runtime proof should include discovery freshness, catalog-backed preview/queue correctness, and sanitized task/audit artifacts before this row can be promoted to "implemented in runtime evidence."

Live Minecraft Saved Kit Proof (Issue #302)

302 adds Saved Kit and item preset lifecycle support through the shared admin-action

queue, including role-guarded CRUD and versioned kit metadata stored with queued actions.

Current live proof prerequisites for this feature set are:

  • APP_BASE_URL, SWARM_API_TOKEN, and a validation-capable host target (SWARM_HOST_ID or SWARM_GAME_VALIDATION_HOST_ID).
  • A deployment in a stable running state with working structured Admin Actions / RCON path and shared minecraft_status protocol checks.
  • Controlled online players and one or more queued Saved Kit grants that exercise kit-to-player dispatch and snapshot-bearing task details.

Run the dedicated harness after the Saved Kit web/control-plane code is deployed to the target environment:

set -a
source .env
set +a
.venv/bin/python scripts/validation/minecraft_saved_kit_live_validation.py

The harness creates a disposable offline-mode Minecraft validation deployment, starts controlled players, creates a two-item starter kit, validates API rejections for unsafe item IDs, missing confirmation, raw command sibling payloads, and unauthenticated access, previews the kit across two players, queues a one-player Saved Kit grant, verifies delivery from the structured per-item task receipt, updates/duplicates/deletes the kit, and proves the queued task/audit history still contains the original versioned kit snapshot. Sanitized artifacts are written to /tmp/swarmhosts-issue-302-minecraft-saved-kit/<run>/live-result.json.

Known gaps before this feature can be promoted further:

  • The harness validates owner/admin CRUD plus unauthenticated rejection, but it does not yet create a second API token to prove view/operator/admin share-role transitions against the live deployment.
  • Existing Give Item live proof still owns the broader policy matrix for blocked items, quantity thresholds, and partial offline-player failures; Saved Kit proof focuses on kit snapshot integrity and multi-item RCON dispatch.

Live Minecraft Advanced Item Template Proof (Issue #303)

303 adds optional structured metadata templates for Give Item workflow fields

(display_name, lore, enchantments, durability) while keeping the existing simple item grant path as the default. It depends on stable server metadata and template-family validation in the shared control-plane queue.

Current live proof prerequisites for this feature set are:

  • APP_BASE_URL, SWARM_API_TOKEN, and a validation-capable host target (SWARM_HOST_ID or SWARM_GAME_VALIDATION_HOST_ID).
  • A deployment in a stable running state with working structured Admin Actions / RCON path and shared minecraft_status protocol checks.
  • Template-compatible command preview coverage to validate structured metadata, rejection reasons, and version/platform-aware capability guards in sanitized task and deployment snapshots.

Run the existing Give Item harness with the issue #303 advanced-template gate enabled after the web/control-plane code is deployed to the validation target:

.venv/bin/python scripts/validation/minecraft_give_item_live_validation.py \
  --validate-advanced-template

The advanced-template step verifies server-side preview generation for display_name, lore, enchantments, durability.damage, and unbreakable, then queues the generated Give Item task and checks task history plus audit-log summaries. Before previewing or queueing the advanced item, the harness requests a companion catalog refresh and checks the Give Item policy capability payload. It exits with code 2, writes result=blocked, and records advanced_template_blocker when the target deployment does not expose either a detected Minecraft Java 1.20.5+ runtime version for vanilla item-component syntax or a fresh companion item catalog with capabilities.advanced_item_data=true for server-specific item data. That blocked result is a runtime capability blocker, not a successful advanced item grant.

Live Minecraft Admin Action Scheduling and Collaboration Proof (Issue #306, child of #302 and #305)

306 adds scheduled and recurring admin actions (Give Item and Saved Kit),

approval-needed flows, pause/resume/cancel controls, and task comments/activity. Its parent, #305, extends this area into safe command templates and broader action families. PR-level validation scripts cover the scheduling surface in a dedicated test path, while #305-level breadth still needs one-click coverage before the matrix can be promoted from “implemented in runtime evidence.”

Current live proof prerequisites for this feature set are:

  • APP_BASE_URL, SWARM_API_TOKEN, and a validation-capable host target (SWARM_HOST_ID or SWARM_GAME_VALIDATION_HOST_ID).
  • A deployment in a stable running state with working structured Admin Actions / RCON path and shared minecraft_status protocol checks.
  • Two users (requester + approver) when validating approval_required_* policy transitions.
  • scripts/dev/minecraft-player-bots or controlled players for offline-policy edge-case checks.

Known gaps before this feature can be promoted in the required gate:

  • No dedicated one-click harness currently validates scheduled one-shot, recurring, offline-policy (fail / retry / run_when_available), collaborator, and broader parent #305 template-family workflows end-to-end.
  • Existing Give Item live proof covers immediate task policy and immediate dispatch, but does not validate scheduled follow-up records, approval notifications, collaboration history visibility, broader #305-safe-template families, or #302 Saved Kit variants.
  • Runtime proof should include task lifecycle transitions (scheduled, paused, pending_approval, expired, rejected) and corresponding user-facing evidence in task detail records before the matrix row is promoted to "implemented in runtime evidence."

Live Minecraft Offline Player Delivery Proof (Issue #304, child of #306)

304 adds pending offline Give Item grants with UUID-keyed targets and delivery when

those players next join after receiving an action, extending the #306 scheduler surface.

Current live proof prerequisites for this feature set are:

  • APP_BASE_URL, SWARM_API_TOKEN, and a validation-capable host target (SWARM_HOST_ID or SWARM_GAME_VALIDATION_HOST_ID).
  • A deployment in a stable running state with working structured Admin Actions / RCON path and shared minecraft_status protocol checks.
  • Controlled offline/online player workflows where queued pending targets can be validated through controlled name/UUID tracking and run_when_available policy.
  • Sanitized evidence collection for player-target snapshots, pending-to-terminal task transitions, and cleanup.

Known gaps before this feature can be promoted in the required gate:

  • No dedicated one-click harness currently validates the #304 lifecycle (pending -> scheduled -> partially_succeeded/succeeded) when a player is offline at queue-time and delivered on join.
  • Existing Give Item and player-metadata live harnesses validate immediate online grants, stale-target rejection, and offline-policy handling, but do not exercise join-time replay, cancel/expire, or delivery idempotence for UUID-keyed pending grants in one pass.
  • Runtime proof should include pending-grant task-history visibility, scheduled follow-up transitions, and user-facing delivery artifacts before the matrix row is promoted to "implemented in runtime evidence."

Artifacts

Every run uploads sanitized artifacts:

Artifact Contents
game-server-validation-plan Matrix plan, selected games, readiness status, gate status, and changed paths.
game-server-validation-<game> Per-target JSON/Markdown evidence and live result payload when live validation runs.
game-server-validation-summary Required failure counts and per-target status summary.
game-server-validation-findings Machine-readable backlog finding payloads for required failures.

Artifacts are intended to be safe to paste after a final scan. They redact token-like values and only record deployment ids, image references, protocol status, log tails, and failure details needed for triage.

Failure Findings

When a required target fails on a non-PR workflow event, the workflow creates or updates a GitHub issue keyed by:

game-server-validation/<game_id>

Findings include the game id/version, failing phase, sanitized log/probe excerpts, likely layer, workflow run URL, and restart instructions. Labels route the finding to the serialized runtime lane.

Promoting A Game Into The Gate

To make another game block the required validation gate:

  1. Add or reuse a protocol-aware live driver for that game in scripts/validation/game_server_validation_workforce.py.
  2. Update catalog readiness so the game is public-ready and has a required validation posture.
  3. Add tests that prove the target is required, its driver runs, and hidden or credential-gated games still do not block.
  4. Update this page with the new live proof steps and required runner variables.

Do not promote a game based only on TCP reachability if that game's user-facing protocol needs a stronger query or client proof.