Case study · Vireon Labs · 2023–24

Ori Optic Deep Research

How I took Ori, a fast AI research tool for investors, and grew it into Deep Research: a product that reads up to 40 sources, checks every citation, and gets the answer right far more often.

Product Lead for Ori’s research surface · 10 features in 12 months

answer relevance (premium tier): ~65% → ~92%
real user questions mined: 10K+
beta testers across 11 cohorts: 127

Ori Optic Deep Research

Reads up to 40 sources and writes a sectioned briefing, checking every number against the other sources before it reaches you.

Time to answer: About 60–95 seconds
Answer length: About 1,400 words
Sources: Up to 40 read

01 · Listen

What people couldn’t get answered

When I joined, Ori already worked well for quick questions. But our most engaged users kept asking for more: deeper reasoning, and more sources behind every answer.

To find out exactly where Ori fell short, I paired 15 tester interviews with SQL mining of more than 10,000 real questions. The pattern was clear. Our core users cared about stock moves, earnings reports and SEC filings, and those were the questions Ori answered too thinly.

10K+real questions mined, plus 15 interviews

What I learned: the questions people ask tell you more than the features they request.

What 10,000 questions were really about

Tap a theme

Most common

“Why did Nvidia move today?”

Then: Ori gave a quick summary, but often from only a few sources.

What it shaped: deeper reading, so the “why” came with evidence.

Bubble size shows how often each theme came up, from “most common” to “asked less often”.

02 · Define

One number to steer by

With a backlog in hand, the team needed one number that told us whether we were winning. I chose answer relevance on the premium tier: would an investor find this answer useful enough to act on?

Speed and answer count were easy to grow and easy to game. Relevance kept us focused on the reason people came to Ori. Underneath it sat three numbers we could move directly.

~92%where relevance ended up

What I learned: a good north star is one the team can’t improve by accident.

The north star and what moved it

Tap the star

Tap the north star, then an input, to see which feature moved it.

03 · Decide

Depth versus cost

Deep research is expensive by design: every extra page Ori reads costs money, and our research budget was already running low. The easy fix was to make every answer shallower.

I pushed for something else: match the depth to the question. A quick ticker lookup reads nothing extra, a comparison reads a handful of pages, and a deep news question reads the most. We also put a lighter, cheaper step in front of the heavy one, so the expensive writing only ever saw the key facts.

The trade-off was real. Some questions get a shallower answer than Ori could produce, and I chose that ceiling on purpose.

What I learned: the best cost decision is often a product decision.

How deep Ori reads, by question

Tap a question type

30pages read in full

Reads the most, so every claim has several sources behind it.

We gainedDeep answers where they matter, at a cost the business could sustain.

We gave upSome answers stay lighter than the engine could make them.

04 · Prove

The testing loop

The 10,000 mined questions did double duty: they became our test set. After every change, we re-ran them and re-scored the answers.

Some changes were to the prompts, some to how Ori grounded an answer in what it had retrieved, and some to how deeply it reasoned. Relevance on the premium tier climbed from about 65% to about 92%. For users, that meant answers broken into clear sections, with the sources behind each claim right there to check.

~65% → ~92%answer relevance

What I learned: real questions make the most honest test set.

Run the loop

Real questions → change one thing → re-score

We rewrote how Ori was asked to answer, then re-ran the same real questions to see if relevance moved.

Only the start and end numbers are real; the gauge doesn’t invent the steps in between.

05 · Validate

Testing with real users

Before anything reached everyone, it went to a cohort of beta testers. Over the year, 127 testers took part across 11 cohorts, each one trying a specific feature before release.

Small, focused groups meant we found problems early and could tell quickly whether a feature was worth shipping.

127testers · 11 cohorts

What I learned: test with the smallest group that can still surprise you.

127 testers, 11 cohorts

Group sizes shown evenly

Time to validate a feature30% faster

Feature adoption+20%

Bars are relative to the “before” baseline.

06 · Ship

Every source is real

An AI research tool is only useful if you can check its work. So I shipped citation verification: the backend, not the model, decides which sources appear; every citation is tied to a document Ori actually read; and a final check blocks anything it can’t verify.

We held the product’s own progress to the same rule. The Deep Research badge reports what the system really did, like how many articles it read, instead of asking the model to describe its own work. It took more engineering than a one-line prompt, and it kept the product honest.

Up to 40sources read, every citation checked

What I learned: trust is a feature you have to build, not claim.

Watch a Deep Research answer come together

Pick a sample question

Ori OpticDeep Research

Finding sources for the question
Reading 36 of up to 40 sources
Noting the key figures
Verifying every citation
Writing the briefing

A simplified sketch with sample questions. Every source is checked before it’s shown; the crossed-out one couldn’t be verified, so it never reached the reader.

Read the PRD behind Ori Optic Deep Research →A short, high-level version of the requirements I wrote in 2024.