Abstract
AI-assisted entitlement research must do more than produce a plausible explanation. A project team needs to examine the sources behind a conclusion, understand which facts remain unresolved, and determine when professional judgment is required. Prebreak has implemented a research workflow that separates evidence collection, parcel assessment, and review. Its execution harness coordinates tools and state, records evidence incrementally, and limits correction attempts. This methods note describes those controls and the role of adversarial agent review. It also sets out an evaluation agenda for reliability, coverage, and operating cost. The implementation is a basis for testing these outcomes, not proof that they have improved.
Method and implementation scope
This note is based on a source-code and engineering-documentation review of Prebreak's research implementation at revision d468d56, assessed September 10, 2026. No new live model runs or performance experiments were conducted for this publication. Implementation claims refer to the reviewed code; deployment configuration and execution outcomes must be verified separately. The registered state-specific research coverage in that implementation is California. 1
We distinguish three forms of evidence: implemented execution controls, observations documented during engineering work, and hypotheses requiring comparative evaluation. Public literature provides context for the design choices. It does not independently validate Prebreak's system.
1. Make the evidence inspectable
The research workflow assembles a structured parcel dossier rather than treating a final narrative as its only artifact. The dossier represents sources, findings, relationships, conflicts, and gaps in examined authority. Research can delegate bounded investigations and request additional evidence when assessment identifies missing information. An unavailable source is meant to remain a coverage gap rather than silently becoming evidence that a constraint does not exist. 1
Evidence is recorded in small validated updates during execution. This avoids requiring the agent to reproduce the entire accumulated record in one final response. Accepted updates receive a receipt after a local snapshot write. Uploaded, normalized document text also has a hash-linked input snapshot and a ledger of the chunks returned to agents. These mechanisms support inspection of what material was available and what was recorded. They do not prove that a model correctly interpreted it, and local recording is not a guarantee of recovery after every form of infrastructure failure. 1
For a reviewer, the intended result is a shorter path from a conclusion to the evidence and assumptions supporting it. That is a concrete property to evaluate: can a person reconstruct the basis for a material finding without repeating the whole investigation?
2. Separate assessment from review
The system assigns different responsibilities to evidence collection, assessment, and review. Assessments distinguish the application of rules from observed records and judgment. This distinction matters because a computation can be internally correct even when the wrong standard or an incomplete factual record was supplied to it. 1
The workflow can be summarized as follows. Additional evidence gathering may occur between assessment and final review.
- Project inputs and sources
- Research and bounded investigations
- Structured evidence dossier
- Parcel assessment
- Judge review
- Publication checks
- Report or unresolved outcome
A separate evaluator that critiques an initial output is an established agent-workflow pattern. Its usefulness depends on clear criteria and measurable improvement, rather than the presence of an extra model call alone. That principle informs our decision to keep review roles specific and to assess their contribution separately. 2
3. Use adversarial review to expose weaknesses
Prebreak's review lenses examine grounding, legal reasoning, and uncertainty. They are intended to surface unsupported claims, faulty reasoning, and conclusions that exceed the available evidence. In the default configuration, the panel runs in an advisory comparison mode and the judge controls the decision. Panel execution can be limited by budget or failures. A stronger panel decision mode exists, but should not be confused with the default behavior. 1
Separating roles creates an opportunity for disagreement; it does not make reviewers statistically independent. Models can share blind spots, and multiple confident responses can still repeat the same mistake. Research on language-model judges has documented position, verbosity, and self-enhancement biases, along with reasoning limitations. Those results motivate human-grounded evaluation of a review system, not a general conclusion that automated review is either sufficient or useless. 3
Accordingly, we treat review outcomes as structured evidence about potential weaknesses. We do not interpret an internal confidence value as a calibrated probability of factual correctness or project approval. Mechanical checks for review receipts establish recorded coverage, not independently verified truth. 1
4. The harness is part of the research method
An agent harness is the execution layer around a model: it provides tools, maintains state, records results, handles errors, and determines when work should stop. Prebreak has implemented these controls for its research workflow. The model reasons within an application that constrains tool use, validation, correction attempts, and cancellation. This is application infrastructure around external models, not a claim to a new foundation model. 1
One documented engineering lesson concerned final submission size. An earlier execution repeatedly attempted to submit output that exceeded its response limit. Subsequent changes added explicit truncation handling and incremental evidence recording. That incident supports a design lesson about preserving research state and recognizing transport failures. It does not establish an accuracy gain. Fixture-based regressions exercise these failure paths, while live research quality remains a separate evaluation problem. 1
The broader experimental question is which combination of context management, tool design, review roles, and stopping rules produces useful research at acceptable cost. This is consistent with evaluation-driven tool development: test realistic tasks, inspect failures, and retain complexity when it earns its place. 4
5. An evaluation agenda tied to decisions
Our next evaluation should compare configurations on the same permission-cleared project inputs and a versioned reference set reviewed by qualified practitioners. Keep the evidence-gathering task, available sources, and relevant limits comparable. When historical outcomes are used, distinguish facts available at the decision date from information learned afterward.
| Dimension | Proposed measure | Failure the measure should reveal |
|---|---|---|
| Grounding | Human-reviewed support for material claims | Citations that exist but do not support the conclusion |
| Coverage | Known material requirements found, missed, or explicitly unresolved | A confident answer built on an incomplete investigation |
| Review contribution | Errors caught and correct claims incorrectly challenged | A review layer that adds noise or merely agrees |
| Operational reliability | Completion, correction, and terminal failure outcomes | Repeated work, lost evidence, or an unusable handoff |
| Efficiency | Time and cost per accepted, reviewable result | Lower apparent cost achieved by omitting necessary work |
The repository contains evaluation machinery and execution tests. This note does not report a completed comparative benchmark. Publishing an accuracy or performance claim will require a documented dataset, labels, comparison conditions, exclusions, and results. Comparing a baseline judge with an added panel, while changing one factor at a time, would help establish whether the panel contributes enough value to justify its cost.
Implications and limitations
The technical thesis is that an entitlement research product should preserve the basis of its conclusions and make incomplete work visible. Source relationships, structured state, bounded execution, and review checkpoints make that thesis concrete enough to inspect and test.
The current evidence does not establish exhaustive coverage, legal correctness, improved project approval outcomes, or superior performance over another harness. Human professional review remains necessary for consequential project decisions. The next research contribution should be a reproducible evaluation showing where the system succeeds, where it fails, and what changes those outcomes.
References
Reference 1
Prebreak. Source-code audit and internal engineering records, revision d468d56, September 10, 2026. Evidence covers orchestration, dossier schema and recording, document-read receipts, review configuration, and failure-handling tests. Implementation evidence, not third-party validation.
Reference 2
Anthropic. Building effective agents. December 2024. Evaluator-optimizer and agent execution patterns. Accessed September 10, 2026.
Reference 3
Zheng, L., et al. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. 2023. Findings concern the evaluated models and benchmarks, not Prebreak's entitlement tasks.
Reference 4
Anthropic. Writing effective tools for AI agents. September 2025. Evaluation-driven tool development. Accessed September 10, 2026.