Repeating prompts until patterns hold

In a composite benchmark example, a farm-supply cooperative in Nakuru can be named in one answer, skipped in another, blurred into “agricultural supplier” in a third, and displaced by a Nairobi company in the fourth. Kivuli Index Lab begins there, with the visible behaviour of the answer. An observation is a recorded state: named, omitted, inaccurately described or replaced by a different reference. Only after the same kind of behaviour appears across comparable prompts, engines or sector groups does the lab move from note-taking to conclusion.

A sample starts with a practical Kenyan question rather than a claim of national coverage. The team builds sets around sectors, regions, business forms and query types: tourism, fintech, agriculture, professional services, informal trade, cooperatives, SACCOs and mobile-first commerce. Nairobi examples are placed beside non-Nairobi examples. English prompts are placed beside Swahili wording. Formal firms are compared with jua kali enterprises and social-commerce-first sellers where public evidence may be thinner or shaped differently. The point is descriptive coverage, carefully marked, not a pretend census of every enterprise in the country.

A repeatable run has enough small pieces left on the table for another reader to follow. The lab records the prompt type, sector, region, engine, language, answer date and classification logic. It does not expect identical answers on every rerun; answer engines change their interfaces, their source mix and sometimes their phrasing. Still, the test path should be legible. If a reader cannot see how a coastal tourism query was compared with a Nairobi tourism query, or how an English prompt differed from a Swahili one, the result is too soft to carry much weight.

A benchmark frame is built from several kinds of weakness, because one score would flatten the useful part. Presence matters, but so do omission, inaccuracy, regional skew, language divergence, freshness problems and business-form mismatch. A business may appear by name and still be badly compressed. A county may be mentioned while its active local operators vanish. A cooperative may be treated as if it were a conventional private company. These are different failures, and the lab keeps them separate so that businesses, trade bodies and county offices can see which problem they are actually facing.

A working Kenyan enterprise may have limited public records, a quiet website, a WhatsApp-first sales path or seasonal information that is current locally but weak online. The lab treats that as part of the infrastructure problem, not as a flaw in the business. It also states the limits plainly: AI answers are unstable, interfaces change, and public evidence is uneven across counties and business forms. Forecasts are marked as uncertainty notes. When review scarcity, mobile-money-first selling, county data, licence renewal language or seasonality looks likely to affect future visibility, the lab names the signal without dressing it up as certainty.