Wish Course

Tests, CI repair, and failure reproduction

How Hermon Code should move from a failing symptom to a verified fix.

Domain hubEvaluation snapshot

Learning Objectives

What this lesson should make precise

01

Name the core abstraction and its failure modes.

02

Translate the concept into a Maple/Hermon proposal contract.

03

Define at least one evaluation case that can fail the model safely.

Tutorial Flow

How this lesson becomes a demo and training target

Each tutorial is written as a user education path and a model-improvement artifact. The diagram shows how the idea moves into a lab, a typed contract, an eval gate, and then a Hermon/MapleAI demo route.

01

Concept

Reproduce first

02

Applied Lab

Applied lab: Tests, CI repair, and failure reproduction

03

Output Contract

failure_summary, files_to_inspect, proposed_changes, tests

04

Eval Gate

Does the answer separate model proposal from deterministic execution?

05

Demo Route

Wish training and public model checks

01

Reproduce first

If the failure can be reproduced locally, do that before patching. If not, record the limitation and use the closest deterministic check.

  • Capture error.
  • Run targeted test.
  • Avoid blind patching.

02

Patch with verification

Every proposed change should name the test that proves it. A UI change may need build, typecheck, screenshot, or smoke test.

  • Unit test.
  • Typecheck.
  • Public smoke.

03

CI-specific evidence

For CI repair, the model should identify job name, failing command, relevant log excerpt, suspected cause, and post-fix check.

  • Job name.
  • Failing command.
  • Post-fix proof.

Lab

Applied lab: Tests, CI repair, and failure reproduction

Create a CI repair proposal for a failed Next.js build after adding a dynamic route.

Expected result

  • A typed JSON-style proposal rather than free-form advice.
  • Clear authority boundaries and denied operations.
  • A test or rubric that decides whether the proposal is deployable.

Evaluation

How Maple would grade this work

Rubric

  • Does the answer expose assumptions instead of hiding them?
  • Does the answer separate model proposal from deterministic execution?
  • Does the answer produce artifacts that can be tested, reviewed, and rolled back?

Output contract

failure_summary, files_to_inspect, proposed_changes, tests, ci_checks

Use this lesson as training direction

A strong lesson gives users a mental model and gives Hermon a sharper target for examples, probes, and demo prompts.

View evalsDomain hub