Skip to content

USOpenloop Health · 2026

AI Assist for a telehealth funnel builder

An AI editing assistant on Vertex AI that safely edits live funnel pages, with the production model picked through a 315-call evaluation harness.

Team Lead · owner from spike to production

eval calls to choose the model
315eval calls to choose the model
shipped with 100+ new tests
6 PRsshipped with 100+ new tests
Gemini tool definitions
4Gemini tool definitions

Context

The situation

Openloop runs a white-label telehealth platform serving 600+ insurance plans and hundreds of client brands. Marketers build patient funnels in an in-house page builder.

The problem. Marketers wanted to change copy and layout by describing the change, but the builder’s page structure is strict: one bad edit can break a live funnel that patients are using right now. The feature needed privacy review before any patient-facing content touched a model.

Approach

Key decisions

  1. 01

    Choose the model on evidence, not taste

    Built an evaluation harness of 21 real utterances across 5 model and prompt variants — 315 calls — and picked the production setup from the results.

  2. 02

    Tools instead of free text

    The model answers through 4 Gemini tool definitions behind a tRPC endpoint, so every suggestion is a structured operation rather than HTML to paste.

  3. 03

    A patch engine as the safety boundary

    Suggested operations go through a patch engine that validates and applies them to live page content, so the page structure cannot be corrupted.

  4. 04

    Scope before code

    Ran the technical research and privacy review first, then split the work into 10 tickets so it could ship incrementally.

How it works

  1. 1

    Marketer prompt

    Plain-language change request in the builder

  2. 2

    tRPC endpoint

    Page context + privacy-reviewed payload

  3. 3

    Gemini on Vertex AI

    4 tool definitions → structured operations

  4. 4

    Patch engine

    Validates and applies operations safely

  5. 5

    Live page

    Accept, reject or undo in the panel

Request flow (simplified, no client data)

Outcome

Results

  • Shipped in 6 PRs with 100+ new tests.
  • Panel UI with 4 response states, progressive loading, rejection handling and undo — marketers accept or reject edits without touching structure.
  • Same period: release gatekeeper across sprints, 40+ code reviews, 2 production hotfixes.

Next case study

AI question reviewer for exam content

Quill Learn