Detect repetition
Compare the latest attempt with prior failures to catch strategy oscillation.
Reflex notices when an agent repeats a failed strategy, then changes its working context using evidence from earlier attempts.
A MongoDB-powered harness for persistent, measurable software development. Built for agents that need to recover, not just retry.
git clone https://github.com/OlegSavchuk/reflex-harness.git && cd reflex-harnessCLONEpython -m reflex_harness run --task sem-dev-01 --arm memoryRUN DEV TASK
Ilya Sutskever described the failure pattern plainly: one fix introduces another bug, and repairing that bug brings the first one back.
You tell the model can you please fix the bug — and it introduces a second bug. Then you tell it you have a second bug and it brings back the first. You can alternate between those.
Compare the latest attempt with prior failures to catch strategy oscillation.
Select from context configurations with evidence of success on similar failures.
Store attempts and outcomes so each decision can be inspected and measured.
Every attempt is observed, recorded, and used to select the next context. The loop runs until a verified fix lands or the budget is spent.
Diagnostics, protected tests, and the static gaming check all passed.
Two held-out tasks per family, repeated five times. Repeats share tasks, so effective N is 2 per family. This is feasibility evidence, not a significance claim.
Fixed fallback cannot expose family B’s hidden helper. The fair random-choice reference (about 3.3/10) is analytical; it was not run.
Gate 8 measures strategy selection. It does not establish repeated-strategy detection on semantic-repetition tasks.
Selects a context configuration from prior outcomes on similar failures.
→Receives the assembled context and returns a patch.
→Runs pytest and records passes, failures, and regressions.
→Checks whether a changed diff repeats a failed strategy.
→Stores attempts and retrieves the evidence for the next selection.
Attempt memory · retrieval · selection
Semantic embeddings
Model routing

Semantic loop detection
Test oracle
Reflex is an open source hackathon project. See the repository for setup details, current requirements, and implementation notes.
Repository ↗