Real operational data or a simulation for AI evaluation?

Start with the decision the evaluation must support

Before looking for a dataset, write down the failure you need to detect. Is the question whether an agent can follow a known process, join information across systems, handle an exception, or make a decision whose outcome appears later? Then name what a passing answer or action would look like and who would review it.

A realistic task does not always require licensed company records. Argo-Bench, submitted October 1, 2026, builds a simulated enterprise warehouse and grades the consequences of agent actions. That is one research design, not proof that simulations answer every operational question. State the decision first; choose the source type second.

Use a public or simulated setting when it can test the failure

A public benchmark or simulation may be enough when the important relationships, choices, and outcomes can be represented without the details of a particular business. It also gives a team a repeatable starting point for comparing systems. Record which process rules and edge cases are modeled, which are assumptions, and whether a model might already have seen public examples.

Argo-Bench uses public material to construct a synthetic business setting because actual enterprise warehouses are sensitive. OpenAI’s GDPval, published September 25, 2025, uses expert-built tasks with reference materials and work products, and describes limits in how much of a longer workflow its first version captures. These examples show different ways to create useful evaluations. Neither creates a general requirement to buy private records.

Identify the gap that only an authorized source could fill

Consider real operational history if the evaluation depends on source-specific variation that a constructed task cannot faithfully represent: changing policies, incomplete notes, multiple systems, human handoffs, exceptions, or outcomes recorded after the decision. Explain which of those features is essential, how it affects the score, and why a public or simulated alternative is insufficient for this particular question.

EnterpriseClawBench, submitted June 22, 2026, derives tasks from proprietary workplace sessions but does not release the internal data. That illustrates both the research value and the access boundary of real work. It does not mean a comparable source is available through Rahvs or that its owner permits a new use.

Compare source options with the same acceptance test

Make a small comparison before approaching a business:

1. Task fidelity: Which events, constraints, and exceptions must the case preserve?
2. Outcome evidence: Is the correct result observable, reviewed, or only assumed?
3. Access and rights: Who controls the material, and are the proposed use, recipients, retention, and onward sharing permitted?
4. Reproducibility: Can reviewers rerun the case, separate test data from development data, and detect contamination?
5. Preparation: What joins, exclusions, review, and secure handling would be needed before a usable sample exists?

Mark each answer as known, to verify, or unavailable. A large real export with no reliable outcome may serve the task less well than a smaller constructed evaluation. A simulation with a neat outcome may miss the variation the team actually needs to study.

Try a hypothetical service-workflow comparison

Illustrative example only; Rahvs is not offering these records. Suppose a team wants to evaluate an assistant that recommends a next step after equipment inspection. A simulated case can test whether the assistant follows a specified policy and handles a designed fault. If the research question instead concerns ambiguous notes, repeat visits, technician judgment, and later resolution, the team should specify which linked events and reviewed outcomes it needs before asking whether any owner can describe such a history.

The brief could require one unit linking an initial request, inspection findings, recommendation, completed action, and follow-up result. It should say how missing follow-up is treated and which customer, employee, or third-party fields must be excluded. Whether an owner has those links, can authorize their use, or can prepare an acceptable sample remains to be checked.

Only source after the gap and permission are clear

If a public or simulated source meets the evaluation need, use it and record its limits. If a real source is necessary, first write the task, minimum linked unit, time span, quality threshold, intended use, exclusions, reviewer, and approval path. Ask for a non-sensitive source description before any files or identities are shared. A sample requires owner approval, an agreed scope, and handling terms.

Use the six-line buyer worksheet to turn that decision into a reviewable request, then describe the requirement to Rahvs. Rahvs explores potential sources and coordinates owner-approved conversations. Availability, preparation, delivery, and licensing terms are confirmed for each opportunity.