Commit graph

1 commit

Author SHA1 Message Date
Yiorgis Gozadinos
7ae395a107
Generate the multi-database corpus and build its databases
The acceptance dataset for lancedb.databases needs a corpus where the
expected database is known per question, which no existing dataset gives.
Three databases: northern and southern hold station reports on one schema,
equipment holds spec sheets that share the vocabulary and answer nothing.

Names are invented throughout. A real station lets the model answer from
priors, which would measure memorisation rather than retrieval.

Three near-name pairs and one shared entity carry the attribution cases.
Pair members are identical apart from the name and the numbers, since a
difference in instrument or technician would hand the model a free
discriminator. Elevations and years are unique across the corpus so a
number identifies one station, and document counts differ per database so
a count cannot be right by luck while attribution is wrong.

Questions and gold answers both derive from STATIONS and INSTRUMENTS, so
they cannot drift apart. Readings are derived with SHA-256 over database
and name: hash() is salted per process, and keying on the name alone gave
the two Station Auk reports identical tables.

evaluations run populates one database and refuses a configured set, so
the builder opens each database itself and the run is --skip-db. It
asserts no single chunk holds all twelve monthly readings, because
without that S3 collapses into a search question and the guarantee has to
survive a chunker change.
2026-08-27 09:26:52 +03:00