How We Built a HIPAA-Aligned AI Scribe

The problem: three tools, zero shared record

Before this build, the practice ran on three disconnected systems: charting in one, imaging in a second, patient communication in a third. None of them talked to each other.

The cost showed up during the visit itself. Providers spent chair time typing notes instead of treating patients, because the only way to get an exam finding into the chart was to type it in by hand while the patient waited. Radiographs lived in a separate imaging system that the chart never queried, so a provider reviewing a patient's history had to open a second application to see the films that justified the treatment plan on screen. Clinical data and imaging data never met, which meant nothing in the record connected a diagnosis to the scan that supported it.

That is a workflow problem first and a compliance problem second. Every extra system a patient's protected health information passes through is another system that needs its own access controls, its own audit trail, and its own answer to "who can see this and why." Three disconnected tools meant three separate compliance surfaces to reason about, for a single patient encounter.

The architecture: one API, two models, one record

The fix was not a fourth tool. It was collapsing charting, imaging, and the clinical AI layer into a single practice-management platform, with two purpose-built model calls sitting inside the clinical loop rather than bolted on as a chatbot.

The system has three moving pipelines behind one API surface:

1. Voice-to-chart. A provider dictates during or immediately after the exam. The transcript goes to GPT-4o with a structured-output schema, and a SOAP-format entry lands directly in the patient's chart record.
2. Vision-assisted radiograph review. A radiograph is uploaded or pulled from the imaging hub, and GPT-4o reads it as a vision input, returning findings that a provider reviews and signs off on.
3. CBCT imaging hub. Cone-beam 3D scans move through an intake-to-analysis pipeline, get associated with the patient record, and become queryable from the chart instead of sitting in a separate imaging silo.

All three pipelines write into the same patient record through a Fastify API layer, which is what makes the "connected" part real instead of aspirational. A finding from the vision pipeline and a SOAP entry from the voice pipeline land in the same chart, timestamped against the same encounter, visible from the same provider screen.

Voice-to-chart: turning dictation into a structured SOAP entry

The voice pipeline does not summarize freeform text into a paragraph and call it a note. It maps a transcript to a fixed schema, so the chart gets a Subjective, Objective, Assessment, and Plan section every time, in a shape the rest of the platform (billing, claims, scheduling) can read programmatically rather than parse out of prose.

A simplified illustration of that pattern, written to show the shape rather than reproduce the client's code or prompt:

import { z } from "zod";
import OpenAI from "openai";

const client = new OpenAI();

const SoapNoteSchema = z.object({
 subjective: z.string().describe("Patient-reported symptoms and history, in the provider's words"),
 objective: z.object({
 findings: z.array(z.string()),
 toothNumbers: z.array(z.number()).optional(),
 }),
 assessment: z.string(),
 plan: z.object({
 treatments: z.array(z.string()),
 followUpDays: z.number().nullable(),
 }),
});

export async function transcriptToSoapNote(transcript: string, encounterId: string) {
 const response = await client.responses.parse({
 model: "gpt-4o",
 input: [
 {
 role: "system",
 content:
 "You convert a dental provider's dictated encounter transcript into a structured SOAP note. " +
 "Use only what the provider stated. Do not infer a diagnosis the transcript does not support.",
 },
 { role: "user", content: transcript },
 ],
 text: { format: { type: "json_schema", schema: SoapNoteSchema } },
 });

 const note = response.output_parsed;

 // Written into the chart as a reviewed draft, not an auto-signed entry.
 return {
 encounterId,
 note,
 status: "pending_provider_review" as const,
 };
}

The pending-review state matters as much as the schema. The model drafts the note. The provider reads it, edits it, and signs it before it becomes part of the legal record. That review step is the pattern any AI-assisted output goes through on a build like this: a named human owns the final artifact.

The vision pipeline: routing a radiograph through GPT-4o

The radiograph pipeline follows the same shape, just with an image input instead of a transcript, and it lives behind its own endpoint in the Fastify layer so the imaging hub, the provider portal, and the billing system can each call it independently. Again, a simplified illustration of the endpoint pattern, not the shipped route:

import type { FastifyPluginAsync } from "fastify";
import OpenAI from "openai";

const client = new OpenAI();

const radiographRoutes: FastifyPluginAsync = async (app) => {
 app.post("/api/radiographs/:id/analyze", async (request, reply) => {
 const { id } = request.params as { id: string };
 const radiograph = await app.db.radiograph.findUniqueOrThrow({ where: { id } });

 const analysis = await client.responses.create({
 model: "gpt-4o",
 input: [
 {
 role: "user",
 content: [
 {
 type: "input_text",
 text:
 "Review this dental radiograph. List findings by tooth number and region. " +
 "Flag anything that warrants provider attention. Do not state a definitive diagnosis.",
 },
 { type: "input_image", image_url: radiograph.signedUrl },
 ],
 },
 ],
 });

 const findings = await app.db.radiographFinding.create({
 data: {
 radiographId: id,
 modelOutput: analysis.output_text,
 reviewedByProviderId: null,
 },
 });

 return reply.send({ findingsId: findings.id, status: "pending_review" });
 });
};

export default radiographRoutes;

Two details in that pattern carry the compliance weight. First, the image reaches the model through a scoped, short-lived reference rather than as raw bytes pasted into something that gets logged. Second, the reviewer field starts empty and the record is not treatment-actionable until a provider closes it. The model reads the film. It does not sign off on it.

What "HIPAA-aligned" means for a pipeline that touches PHI twice

A voice-to-chart pipeline and a vision pipeline both put protected health information directly in front of a model call, twice per encounter. That is exactly the situation where "is this HIPAA compliant" stops being a marketing question and becomes an engineering one, and it is worth being precise about the answer, because there is no such thing as a HIPAA certification to point to. HIPAA has no certifying body. The honest claim is a signed Business Associate Agreement plus a documented set of controls, which is what we describe on our security page rather than a badge.

The controls that make this build defensible:

• The model sits behind a single interface. Every call to GPT-4o for either the voice or the vision pipeline goes through one internal client, which means the underlying model is swappable without touching the pipeline logic that handles the PHI itself.
• No training on client data. The provider's dictation and the patient's radiograph are not used to train or fine-tune anything, by default and by contract.
• Deployed inside the practice's own environment. The API, the database, and the imaging store run in the client's own cloud account, not a shared multi-tenant Asaasin service. If we disappeared tomorrow, the system keeps running.
• BAAs signed on request, covering the model vendor relationship and our own engineering access to the environment.
• AI-assisted code goes through the same gate as any other code: a pull request in the client's repository, reviewed by a named engineer, typed contracts (the zod schema above is exactly that), and tests in CI before a schema change or a new endpoint ships.

A build with this many concurrent surfaces runs as a pod rather than a single hire, because the voice pipeline, the vision pipeline, the imaging hub, and the API surface all had to move in the same weeks rather than in sequence. This is the same posture we describe in more depth for is ChatGPT HIPAA compliant and what HIPAA compliant software actually requires: the model vendor's own terms and a signed BAA get PHI legally into the pipeline, but the controls around access, review, and where the data lives are what actually keep it defensible. A raw ChatGPT session with no BAA and no audit trail is a different thing entirely from a scoped API call behind a reviewed endpoint, even when the underlying model is the same.

The result: two models, one chart, 30+ provider surfaces

The build replaced three disconnected tools with one platform. Voice-to-chart SOAP generation and radiograph analysis both run in production, and the practice's providers work from more than 30 distinct provider-facing surfaces spanning scheduling, charting, billing and claims, and batch insurance verification, all reading and writing to the same Fastify API.

Metric	What it measures
80+ REST endpoints	The Fastify API surface behind both portals and both model pipelines
30+ provider pages	Scheduling, charting, billing/claims, and batch insurance verification surfaces
2 AI models live	Voice-to-chart (GPT-4o) and vision-based radiograph analysis (GPT-4o)

The 80+ endpoints figure is not incidental scope creep. It is what "one system" costs when charting, imaging, billing, claims, and two AI pipelines all need to read from and write to a single patient record instead of three siloed ones. A dictation drafts into the same chart a radiograph finding lands in, which is the entire point: clinical data finally meets the imaging pipeline, in the same request lifecycle, reviewed by the same provider.

The short version

A developmental-dentistry practice network went from three disconnected tools to one practice-management platform with two AI models in the clinical loop: voice-to-chart SOAP notes and radiograph analysis, both running through a Fastify API with 80+ endpoints, serving 30+ provider surfaces across scheduling, charting, billing, claims, and insurance verification. The HIPAA-alignment case rests on where the data lives and who reviews the output, not on a certification that does not exist: a signed BAA, a model call behind one swappable interface, no training on client data, deployment inside the practice's own environment, and a named provider closing the loop on every AI-drafted note and every AI-read radiograph before it becomes part of the record.