M4.3 — Lifecycle & build-your-own boot test

What you'll learn

• The test node lifecycle: how a UnrealTestNode progresses from start to verdict.
• The hooks you override — start, tick, stop — and where the verdict is set.
• How to assemble a working boot test incrementally and reason about its pass/fail.

How it applies (QA)

The lifecycle is where "what does passed mean" becomes executable. Most custom-test bugs you'll own live here: a success condition that never fires, a verdict set too early, a tick that doesn't actually check anything. Building one boot test by hand inoculates you against all three.

Concepts

The lifecycle, in hooks

TestExecutor drives every node through the same shape (ITestNode's contract). On a UnrealTestNode the hooks you care about:

1. GetConfiguration() — declare roles/args/duration (M4.2). Called first.
2. StartTest() — the base launches the UnrealSessionInstance (your roles come up). Do one-time setup here.
3. TickTest() — called repeatedly while the session runs. This is your monitoring loop: inspect role state/logs, decide whether your success or failure condition has been met, and when it has, set the result and mark the test complete.
4. StopTest() — teardown hook; the base kills processes and collects artifacts.

Between start and stop the node moves through internal states (installing, running, complete, etc.); you don't manage those directly — you react inside TickTest() and let the base advance the machine.

Where the verdict is set

The node carries a result you set — conceptually a TestResult (Passed / Failed / others). The pattern inside TickTest():

• success condition observed → set result Passed, mark complete.
• failure condition observed (or a disqualifying crash the base detected) → result Failed, mark complete.
• neither yet, and within MaxDuration → return and wait for the next tick.
• MaxDuration elapses with no verdict → the framework fails it as a timeout.

That result becomes the process exit code CI reads (M2.4). The base contributes crash/ensure detection automatically; your job is the intended success/failure logic on top.

Build it incrementally

Step 1 — config onlyStep 2 — add a success conditionStep 3 — explicit failure + honesty

public override MyBootTestConfig GetConfiguration()
{
    var Config = base.GetConfiguration();
    Config.RequireRole(UnrealTargetRole.Client);
    Config.MaxDuration = 120;
    return Config;
}

Right now the test launches a client and… waits 120s, then times out. It has no success condition yet — deliberately. Run it and you'd see a timeout failure. That's the baseline.

public override void TickTest()
{
    // Has the client logged that it reached the menu?
    if (RoleReachedMarker("Reached MainMenu"))   // pseudo-helper; see note
    {
        SetUnrealTestResult(TestResult.Passed);
        MarkTestComplete();
        return;
    }
    base.TickTest();   // let the base advance state / detect crashes
}

Now the test passes as soon as the client logs the boot marker, instead of running the full 120s. The timeout becomes the failure path, not the normal path.

public override void TickTest()
{
    if (RoleReachedMarker("Reached MainMenu"))
    {
        SetUnrealTestResult(TestResult.Passed);
        MarkTestComplete();
        return;
    }
    if (RoleReachedMarker("Fatal error"))        // belt-and-suspenders; base also detects crashes
    {
        SetUnrealTestResult(TestResult.Failed);
        MarkTestComplete();
        return;
    }
    base.TickTest();
}

Verify on a real build: method names (TickTest, SetUnrealTestResult, MarkTestComplete), the TestResult enum, and especially RoleReachedMarker (a stand-in — real log inspection goes through the role/log-parser APIs) differ by version. The shape — tick, check a condition, set result, mark complete, else wait — is the durable pattern.

The three classic bugs

• No condition that fires → always times out (you saw it in Step 1). Your marker is wrong, stripped by configuration (M2.2), or never emitted.
• Verdict set too early → passes before the thing it claims to verify actually happened (e.g. marking pass on "client launched" instead of "client reached menu").
• Tick that doesn't check → calls base.TickTest() and nothing else, so the test only ever passes/fails on the base's crash/timeout behavior, not your intent.

Pitfall: 'passes' that proves nothing

A success condition that's too shallow (or accidentally always true) yields a green test that certifies nothing — the worst kind, because it builds false confidence. When you write a pass condition, ask: what specific, late-enough observable does this require, and could it be true when the feature is actually broken?

Worked example — reading the incremental test's behavior

Version	Client reaches menu in 8s	Client hangs forever	Client crashes at 3s
Step 1 (no condition)	times out at 120s (red, wrong)	times out (red, right)	base detects crash (red)
Step 2 (success only)	green at ~8s (right)	times out (red, right)	base detects crash (red)
Step 3 (success + fail)	green at ~8s	times out (red)	fails fast on crash (red)

The progression shows why Step 1's green is impossible (no success path) and why Step 2 is already a correct boot test — Step 3 mainly fails faster and more explicitly.

Lab — Extend the boot test to 'reached first playable'

Modify the Step 2/3 TickTest() so the test passes only when the client reaches gameplay, not just the menu. Specify: (1) the new marker/observable you'd require, (2) why menu-reached is too early for this test's claim, (3) what you'd raise MaxDuration to and why, (4) one false-pass your new condition still permits.

Exercise 1 — Diagnose from behavior

A custom test always reports timeout failure, even on builds a human confirms boot fine in 6s. Give the two most likely causes and how you'd tell them apart.

Exercise 2 — Where's the verdict?

In one sentence each: what does StartTest() do, what does TickTest() do, and where is Passed set?

Self-check — answers

Lab: (1) e.g. a PlayLevelLoaded / PossessedPawn marker emitted at first playable. (2) menu-reached proves the shell loaded but not that a level streamed in and the player spawned — the test claims "playable," so it must observe playable. (3) Higher than the menu test — level load adds seconds on a console (e.g. 180s), sized to the slowest target. (4) It still passes if the player spawns but, say, into a broken/black level — "reached playable" isn't "playable is correct"; you'd need a screenshot/perf check (M5.3) to tighten it.

Exercise 1: (a) The success marker never fires — wrong string, or compiled out in this -configuration (M2.2); (b) TickTest() doesn't actually check (only calls base). Tell apart: grep the role log for the marker — present in the log but test still times out → your tick/check is wrong; absent from the log → the marker/emit/configuration is the problem.

Exercise 2: StartTest() — the base launches the session (roles come up) and you do one-time setup; TickTest() — runs repeatedly to monitor and evaluate your success/failure condition; Passed is set inside TickTest() (via SetUnrealTestResult) when the success condition is observed, then MarkTestComplete().

Done when

• [ ] You can name the lifecycle hooks and what each is for.
• [ ] You can write a TickTest() that sets a verdict on a success condition and marks complete.
• [ ] You can explain why a no-condition test always times out.
• [ ] You can identify a shallow/false-pass success condition and tighten it.

---

Next: M4.4 — TestExecutor.