Understand the system.
Show the evidence.
A practical guide to the software evolution lab: from a website brief to browser checks, or a Python candidate to its final audit.
From one answer to an experiment.
GENESIS turns a programming goal into a population of competing Python programs. It executes them against frozen tests, selects verified survivors, and uses their code and development feedback to generate another generation.
The important output is more than source code: it is the record of what ran, under which constraints, against which inputs, and with what results. Evolution can improve a solution, but improvement is measured rather than assumed.
A real program, a parent-child relationship, an enforced constraint, an execution receipt, and a final independent check. An archived run is always identified as recorded.
A clear boundary between creation and judgment.
The browser presents experiments. FastAPI owns orchestration, tests, scoring, and persistence. AI creates candidate source. CreateOS executes that source in isolated microVMs; it does not decide the score.
Expected answers remain in the backend. The sandbox receives candidate source and input only. The engine compares actual stdout with expected JSON and saves the result after each test.
Stripe Projects provisioning and public hosting are deferred. The current application runs locally with PostgreSQL; it does not claim a completed Stripe deployment.
Generate. Execute. Evaluate. Evolve.
- Freeze the challenge and tests.
Preset benchmarks use deterministic reference oracles and a stored seed. Custom challenges use a supplied suite or an AI evaluator followed by a blind verifier. Tests are validated and hashed before candidate generation.
- Create genuinely different programs.
The creator receives the goal, constraints, a generation strategy, parent code, and development feedback. Duplicate source hashes are retried up to three times. All programs use the Python standard library and JSON stdin/stdout.
- Execute and select.
Each candidate runs in its own CreateOS sandbox. The engine records every case, computes development and holdout grades separately, and marks a candidate verified only if both groups pass completely. Source that exceeds the byte limit is rejected before execution, without a score.
- Build the next generation.
Elitism copies and retests the best verified parent. Mutation works from one parent. Crossover uses two verified parents when available. If there are no verified parents, the engine creates new candidates.
- Adapt strategies, then audit.
Strategy trials rank actual holdout results while reserving the last candidate slot for exploration. Changing constraints resets these trials and preserves their history. A preset winner receives a fresh seeded final audit after selection; audit results never feed back into evolution.
Constraint updates increment the revision and apply at the next generation. The engine snapshots that generation’s limits. Once the final generation begins, new limits require a new experiment. Survivors are retested, and current dashboard metrics exclude candidates from older revisions.
Correctness first. Performance second.
Preset sorting, stable deduplication, and interval merging use six development cases and three holdout cases. Cases include boundary conditions and seeded random inputs. Custom suites allow 4–24 development and 2–12 holdout cases, with no shared inputs between groups.
90 × passed / total + speed bonusSpeed bonus = 10 / (1 + median execution ms / timeout ms), awarded only when every development test passes.
A pass requires valid finite JSON equal to the expected result, a successful exit, no timeout, and bounded output. A high development score alone does not make a survivor: every holdout case must also pass. Interrupted evaluation and source-limit rejection have no fitness score.
The creator receives development failures, not holdout expected answers. Because strategy adaptation uses holdout results, holdout is not a permanently untouched benchmark. The fresh final audit is the separate post-selection check for presets.
CreateOS execution time includes interpreter startup and runner overhead. It is not a calibrated CPU benchmark. AI evaluator agreement reduces mistakes but does not prove arbitrary program correctness. Check exact receipts before claiming an improvement.
Generated code executes outside the app.
- CreateOS provides a microVM boundary with closed ingress and a host-enforced non-routable outbound allowlist.
- Candidate Python runs with isolated interpreter mode, UID/GID 65534, a clean environment, and read-only uploaded source.
- The runner imposes wall time, CPU, memory, process, file size, open-file, and output limits. API keys and the lab token never enter the candidate environment.
- Tests, expected answers, and grading remain outside the microVM. Candidate stdout is treated as data.
- Each sandbox is destroyed after evaluation. Failed cleanup preserves its ID for retry; auto-pause is not deletion.
Draft prompts and the private lab token are kept in React memory, never in URLs or browser storage. Reloading or opening another tab requires unlocking again. Local setup saves provider credentials in an owner-only, ignored environment file; public hosting requires HTTPS and hosting secrets.
Goals, parent source, and development feedback go to the selected AI provider. Source and inputs go to CreateOS. Avoid putting secrets or personal data in challenges.
Your brief, a real website, and visible progress.
Choose Website on the home page or a new workspace project. Select Developer portfolio to try a complete frontend brief. Unlock the workspace, review Build settings, and choose Build website. The default makes one candidate in each of two passes. Website builds use the saved AI connection and a separate browser worker; they do not need CreateOS or Stripe.
- Plan.
The AI publishes a concise design plan and freezes a section checklist for this request. AI progress shows real timestamped actions and results, not private model reasoning.
- Build and check.
The AI generates a self-contained HTML/CSS/JavaScript website. Isolated Chromium renders the source at desktop and phone sizes, captures real screenshots, checks overflow at 1280, 390, and 320 pixels, scans automated WCAG A/AA rules, and tests mobile navigation and JavaScript errors.
- Refine.
The next pass receives the actual browser findings and selected parent source. Every version remains saved, including failures. The chosen version has the most passing checks; visual quality still needs your review.
- Make it yours.
Use the sidebar to request a change from the selected version. Preview supports interactive desktop/phone views and saved screenshots. Code includes a file explorer, package list, startup guide and parent diff. Checks explains observed results. Versions lets you switch iterations. Download saves a ZIP with index.html and a README; Evidence exports the project record.
A self-contained frontend with working local interactions, such as menus, filters, and accordions. It does not generate a server, database, payments, or a React/npm project. A contact form cannot send messages without an integration. Automated checks are not a full accessibility audit, comprehensive security scan, or proof that every interaction works.
Preview and browser boundaries
Preview scripts run in an opaque-origin sandboxed frame. Network requests, forms, popups, and workspace access are restricted. The browser worker has no provider keys or host mounts and uses a private Docker network. Its container has resource limits and a read-only root filesystem. Chromium's additional OS sandbox is not enabled; this local configuration is not a hardened public multi-tenant service.
Progress refreshes about every 1.5 seconds. Individual AI calls may take up to 90 seconds. Stop retains saved versions; a restart marks active work interrupted. Screenshots persist in a separate artifact volume. Ten follow-up revision requests are allowed per project.
A working app, with evidence.
Choose Full-stack to build a task board with real login, saved tasks, search, status filtering and role-based access. The AI generates the frontend against a supplied trusted backend. This release supports this task-board blueprint.
- Build and preview.
Start with one candidate and one pass. The preview signs in with a short-lived app owner session. Create a task and change its status: records persist in PostgreSQL.
- Inspect the backend.
Open Backend to create editor/viewer accounts. Sign out inside the app to test a user. Viewer accounts cannot write, including through direct API calls. Generated code cannot change the permission rules.
- Check and compare.
Checks use expiring real test accounts and database records. Compare opens mobile captures for full-stack projects and separates repaired failures, newly verified checks and regressions. Use Expand workspace for a larger view and Escape to return. CreateOS proof opens clearly labelled recorded Python execution evidence. Inspect failed and unchecked results before presenting.
- Download and run.
Download exports a runnable project ZIP with the selected frontend, trusted API, setup guide and Dockerfile. Install Python 3.11 or newer, unzip the download, then double-click Start.command (macOS) or Start.bat (Windows). The launcher installs packages, checks them, helps create the first account and opens the app. Existing local users and tasks are retained on later starts. It excludes saved users, data, keys and sessions. Evidence exports the separate verification record.
Generation jobs are saved and serialized. Resume can recheck saved web source without generating it again. Python jobs require explicit cleanup/restart. AI request limits and conservative token reservations persist across revisions; actual provider token counts remain separate from estimates. Currency pricing is not configured.
The generated preview has an opaque origin and no direct network access. A narrow validated bridge uses app-scoped sessions; workspace/provider secrets never reach generated code. Browser isolation currently relies on the container; the extra Chromium OS sandbox is unavailable on this host. Automated tests do not certify complete security.
Workspace access supports administrator-created accounts with private project histories. The first administrator account can access existing owner projects. Reloading clears the in-memory session; sign in again.
Record a user test · Tester guide · Blank feedback form · Presentation outline
Working in GENESIS.
| Surface | What to do / show |
|---|---|
| Prompt & examples | Describe a Python task or choose an example. The homepage carries your draft into the workspace without starting execution. Unlock the workspace, review settings, then submit explicitly. Editing a preset goal switches it to a custom challenge. |
| Preview | Start with a recorded execution, edit JSON input, and explicitly run the selected Python source in a fresh sandbox. Data, JSON, and Console show actual results. HTML strings can be viewed as inert static markup; React builds are not supported. Each fresh run consumes provider quota and saves its own receipt without affecting evolution. |
| Verification | Inspect selected-candidate tests, source and suite fingerprints, preset expected answers, receipt consistency, final audit, and project-wide cleanup. Configured runtime protections are distinct from verified evidence. Unperformed vulnerability and application-login checks remain unchecked. |
| Run settings | Choose a preset or custom goal. Set candidates, generations, time, source-byte, and memory limits. During an active run, limits can be queued for the next generation until the final generation begins. |
| Evolution | Follow the real program lineage. Select a node to open Code. Edges identify parents; operation labels distinguish creation, elite retest, mutation, and crossover. Expand the graph for presentations; Escape returns to the workspace. |
| Code & Evidence | Select a candidate in the sidebar. Code shows source; Diff compares its first parent. Evidence expands exact inputs, expected answers, actual output, exit status, and timing. Sandbox shows allocation and cleanup receipts. |
| Activity | Read saved agent and test events, fitness trajectory, and current-revision strategy trials. Small-sample strategy performance is experimental, not a general self-improvement guarantee. |
| Report & Export | Compare candidates in the current revision, inspect constraint history and the suite hash, check final audit and sandbox cleanup, and export the full run. |
| Projects sidebar | New project clears the current view and starts a blank prompt. Choose a saved project explicitly; it is recorded evidence, not a replay. An active run resumes after unlock only when there is no incoming prompt. |
Recorded examples create no sandbox. Fresh previews accept 16 KiB of finite JSON input, use one allocation, and retain up to 20 receipts per project. Cancelled or interrupted runs preserve sandbox IDs for cleanup retry. Preview results never change grading. Arbitrary custom inputs have no reference check unless they match a frozen test case.
Rendered HTML is an inert static iframe. Links, scripts, forms, external resources, and parent-page access are disabled. React generation and a web development server are not provided. Verification describes saved evidence and implemented protections; it is not a vulnerability or generated-app login audit.
Start the lab with your own credentials.
Requirements: Python 3.11+, uv, Node 22+, npm, and Docker Desktop. Run these commands from the repository root. The backend uses one worker and supports one active experiment.
uv sync --directory backend
npm ci --prefix frontend
python3 scripts/secure_setup.pyOpen http://127.0.0.1:8765, enter the provider keys, AI model ID, and HTTPS API base URL. Save the displayed lab token privately and stop the setup server after saving. Never paste keys into chat or commit them.
python3 scripts/start_lab.pyOpen http://localhost:3000, choose Workspace, then Unlock workspace. Review the prompt and Run settings, then submit explicitly to begin. Unlocking alone does not start execution. The launcher starts PostgreSQL and both services. Its Ctrl+C stops the app processes while retaining the database volume. For the container stack, after setup and password generation, use docker compose up --build. With Docker, change saved credentials through secure setup and restart the backend.
Custom challenge contract
Programs read one JSON value and print one JSON value. A supplied suite needs 4–24 development and 2–12 holdout cases; each case has input, expected, and an optional label. The optional input_type declares array, object, string, number, boolean, null, or any. Development and holdout inputs must differ.
{
"input_type": "array",
"cases": [
{"input": [], "expected": 0},
{"input": [1], "expected": 1},
{"input": [-2], "expected": -2},
{"input": [2, -1, 3], "expected": 4}
],
"holdout": [
{"input": [-5, -1], "expected": -1},
{"input": [3, -4, 5], "expected": 5}
]
}This small suite illustrates maximum-subarray input. Add broader trusted cases before making a correctness claim. When no suite is supplied, evaluator and blind-verifier agreement is required before tests freeze.
Let the evidence tell the story.
Before presenting, confirm credentials, provider quota, sandbox capacity, and submission eligibility. Keep a completed real preset run ready in the archive. Live provider latency can exceed the presentation window.
- Define the experiment
“We generate competing programs, execute them independently, and evolve verified survivors.” Show the goal and constraints; choose population 3 and two generations.
- Show the programs and execution
Show distinct algorithms in the chamber. Open a candidate’s evidence receipt and explain how stdout is checked against an external expected answer.
- Introduce a constraint
During a live run, lower the byte or time limit for the next generation. In a recorded demo, expand constraint history and explicitly explain the saved change.
- Follow a parent into its child
Select a mutation or crossover node, inspect its parent diff, and explain elite retesting. A rejected candidate is evidence that constraints were enforced.
- Close with the proof
Show current-revision comparison, fresh final audit, and destroyed sandboxes. Export the run. State the pending Stripe deployment and any unverified flow clearly.
Use the completed archive and say “This is a recorded real experiment.” It contains real programs and executions, but opening it does not start a new run. Do not present saved evidence as fresh activity.
What is proven, and what is still pending.
Live verification recorded on October 14, 2026:
- A real sorting run completed two generations with six candidates: five executed and verified; the crossover candidate exceeded the revised byte limit and was rejected without scoring.
- 45 candidate tests and nine fresh final-audit tests ran in CreateOS. The final audit passed. All six experiment sandboxes were destroyed.
- A live constraint update changed 1500 ms / 12,000 bytes / 128 MiB to 1000 ms / 650 bytes / 64 MiB. PostgreSQL retained both generation snapshots.
- Sandbox checks verified unprivileged execution, secret-free candidate environments, output and timeout limits, protected source, and non-routable egress.
- Automated backend tests cover contracts, fitness, lineage, revisions, authentication, cleanup, and error handling. Frontend type checks, production build, and dependency audit pass.
Real sorting evolution and retained execution evidence.
The real custom run completed six candidates and 78 executions, including a verified crossover. All six sandboxes were destroyed.
Account access is unresolved. The official event listing says to start from an empty directory; pre-event code requires organizer approval.
Repeatable release check
From the repository root, inspect the recorded real experiment without creating sandboxes:
uv run --directory backend python ../scripts/verify_release.py --experiment e678e548-76e0-4fe7-a2e7-e640be5965e4The checker validates source and suite hashes, reference answers, recorded stdout and pass status, fitness, lineage, strategy measurements, final audit, and cleanup. It checks internal consistency; exported receipts alone do not independently authenticate the provider.
To repeat verification using six new sandboxes, explicit live mode runs the real maximum-subarray custom challenge and requires a verified crossover:
uv run --directory backend python ../scripts/verify_release.py --runLive mode consumes actual AI requests and up to six sandbox allocations. It preserves incomplete evidence and attempts to stop on cancellation or deadline failure. Failure never becomes a success claim.
The official event listing says builders start from an empty directory and submit through Stripe Projects. The schedule is October 17, 12:00–19:00 Pacific, or October 18, 00:30–07:30 India time. Registration requires approval. Treat this prebuilt repository as preparation until organizers authorize its use.
The app is a local workspace. Restarted active runs are marked interrupted, not automatically resumed. Workspace accounts have private project histories. Jobs use a persistent lease with one backend worker. Web source can be rechecked through explicit recovery; automatic billed Python replay and automatic worker failover are excluded. Quota/capacity failures preserve evidence and report failure; there is no simulated fallback.
The experiment is the durable record.
All /api endpoints require the lab bearer token. The browser uses a same-origin proxy; default local API access is http://localhost:8000. Health is public and returns no credentials.
| Method / path | Purpose |
|---|---|
GET /health | Service liveness. |
GET /api/status | Provider configuration and database status; configured is not a live health guarantee. |
GET /api/experiments | Experiment summaries. |
GET /api/experiments/{id} | Complete saved experiment and evidence. |
POST /api/experiments | Start a validated challenge; rejects a second active experiment. |
PATCH /api/experiments/{id}/constraints | Queue limits for the next generation of an active run. |
POST /api/experiments/{id}/stop | Cancel evaluation while preserving evidence and attempting cleanup. |
POST /api/experiments/{id}/cleanup | Retry undeleted sandbox cleanup after a run ends. |
POST /api/experiments/{id}/revisions | Website follow-up changes: {message, candidate_id}. |
GET /api/experiments/{id}/candidates/{candidate}/screenshots/{desktop|mobile} | Authenticated source-matched browser screenshot. |
GET /api/experiments/{id}/candidates/{candidate}/verification | Saved evidence checks and configured runtime protections. |
POST /api/experiments/{id}/candidates/{candidate}/previews | Start a background preview; body is {input: JSON}. |
POST /api/experiments/{id}/previews/{preview}/stop | Cancel preview and attempt sandbox cleanup. |
POST /api/credentials | Loopback-only runtime credentials; hosting uses environment secrets. |
Exports include the challenge, seed, frozen suite and hash, generation revisions, candidate source and hash, parent IDs, operations, development/holdout grades, stdout/stderr, exit status, timing, lifecycle receipts, events, strategy trials, and any final audit. Exports contain user-provided goals and inputs; review them before sharing.
Repository map
frontend/app | Workspace, guide, styling, and backend proxy. |
|---|---|
frontend/components | Run report and shared data types. |
backend/genesis/engine.py | Generation loop, selection, scoring, audit, cleanup. |
backend/genesis/providers.py | AI contracts and trusted CreateOS runner. |
backend/genesis/benchmarks.py | Seeded test inputs and preset reference oracles. |
backend/genesis/models.py | Challenge, test, and constraint validation. |
backend/genesis/store.py | Persistent snapshots and restart recovery. |
backend/genesis/api.py | Authenticated HTTP interface and request limits. |
scripts | Private setup, local launcher, and repeatable evidence / live release verification. |
docs | Architecture, deployment, verification, eligibility, and demo notes. |
Verify a change
cd backend && uv run pytest -q
cd ../frontend
npm test
npm run typecheck
npm run build
npm audit --omit=devUnit tests use provider doubles; they do not inject demo data into the dashboard. Live receipts and automated tests establish different kinds of evidence.