Research / Open directions
Research, evaluation and open directions
Models that learn the path to evidence
We are studying small specialized models for navigating UKS/TRAMA and planning queries on Trino/lakehouse systems. The aim is to choose where to look, which relations to follow and which execution steps to propose. This is not fine-tuning a model to memorize enterprise knowledge: that knowledge remains in the connected systems, with its permissions and updates.
This is a research direction. No specialized Ando model is currently offered as an available feature, and no cost or quality advantage is established by the evaluations on this page. Any experiment must compare answer correctness and completeness relative to the question and authorized sources, latency and cost per correct answer.
What we build
Ando Engine connects enterprise knowledge and selects context for AI applications. This site describes the UKS and TRAMA architecture and offers access to the private beta. It is a product under active development.
How to read the published numbers
The landing page contains internal measurements and examples. These are not independent certifications or a promise of performance on your data. Workload, corpus, model, hardware, concurrency and access policies can change the result. A public downloadable dataset and a complete reproducibility report are not currently available here.
What a reproducible retrieval evaluation must include
Specify the corpus and document versions, representative queries and relevance judgments, permissions, retrieval settings, model versions and hardware. Compare identical workloads and report relevance (for example recall@k and nDCG), citation correctness, p50/p95 latency, cost per query, freshness and unauthorized-result rate. Publish sample size, failure cases and the exact evaluation procedure.
Evaluate Engine on your own workload
Request beta access and agree on a representative dataset and acceptance criteria before making a purchase decision. Include negative cases, changed or deleted documents and users with different permissions. We do not claim a universal ranking against all RAG or retrieval products.
Internal results and public references
- HERB (n=125): Hit@5 0.512 → 0.608 (+18.8%); MRR@10 0.378 → 0.412 (~9%); Hit@1 0.296 → 0.272 (−8.1%). Graph fusion improves some retrieval metrics but worsens first-result accuracy. HERB.
- FinQA (n=500): MRR@5 0.782. The literature reference 0.618 is an overall result on a different query population (23,088 queries), not a matched FinQA comparison or proof of superiority. From BM25 to Corrective RAG, Table VI.
- BRIGHT/pony (n=20): MRR@10 0.239 → 0.338. Exploratory internal test in one domain; not a result on the full BRIGHT benchmark. BRIGHT.
The linked datasets and papers are external references, not independent validations of Ando. The aggregates above come from internal evaluations; raw query-level results, exact run configuration and a public reproducibility package are not published here yet.