How to Add a Usage Dashboard to an AI Built App

Nobody notices a usage spike until the invoice arrives. A founder checks their AI provider bill on the first of the month and finds it is four times higher than usual, with no error logs, no outage, and no obvious culprit. This is what happens when you add a usage dashboard to an AI built app too late instead of too early.

A usage dashboard is the layer that shows you, in near real time, who is using your app, how much they are costing you in model tokens, and which requests are the expensive ones. It sits between "the app works" and "the app is sustainable to run." This guide walks through what to track, how to build it, and how to read the numbers once you have them.

Why Request Counts Alone Don't Work for AI Apps

If you built a usage dashboard the way you would for a normal web app, you would log a row every time someone hits an endpoint and count requests per user per day. That works fine for a CRUD app where every request costs roughly the same amount of server time.

It falls apart for an AI built app because the cost of a single request is not fixed. A request that sends a 40-token question to a small model and gets a one-sentence answer back is nearly free. A request that uploads a 30,000-token document and asks for a full rewrite can cost 50 to 100 times more, on the same endpoint, from the same user, within the same hour.

Anthropic's own pricing table makes this concrete: Claude Haiku 4.5 runs $1 per million input tokens and $5 per million output tokens, while Claude Opus 5 runs $5 per million input tokens and $25 per million output tokens (published API pricing). Two requests that look identical in your request logs, same endpoint, same status code, can differ in real cost by two orders of magnitude depending on the model, the input length, and the output length.

A dashboard that only counts requests will miss this completely. It will tell you User A made 20 requests and User B made 20 requests, and make them look equally cheap, right up until User B's requests turn out to be long-document summaries on a bigger model that quietly cost 40 times more.

What a Usage Dashboard Actually Needs to Track

At minimum, log one row per model call, not per user action. A single "generate report" click might trigger three model calls; each one gets its own row. Each row should carry enough detail to reconstruct exact cost after the fact.

Compute cost_usd at the moment you log the row, using whatever the provider charged for that exact model and token count at that time. Providers change prices; storing a pre-computed dollar figure means old rows stay accurate even after a price change, and you are not re-deriving history from a pricing table that has since moved on.

Building It: A Concrete Walkthrough

The mechanics are the same regardless of stack. You need three pieces: a logging call around every model request, a table to store it in, and queries or a small UI on top that roll the rows up into something a human can read.

1. Log every model call, not every user request

Wrap your model client call in a function that captures the token counts the provider returns in its response (most APIs, including Claude's Messages API, return input and output token counts in the response usage object) and writes one row per call.

async function callModelAndLog({ userId, feature, model, messages }) {
  const response = await modelClient.send({ model, messages });

  const { input_tokens, output_tokens } = response.usage;
  const costUsd = computeCost(model, input_tokens, output_tokens);

  await db.usage_events.insert({
    user_id: userId,
    feature,
    model,
    input_tokens,
    output_tokens,
    cost_usd: costUsd,
    created_at: new Date(),
  });

  return response;
}

function computeCost(model, inputTokens, outputTokens) {
  const rates = PRICING_TABLE[model]; // e.g. { input: 1.00, output: 5.00 } per MTok
  return (inputTokens / 1_000_000) * rates.input
       + (outputTokens / 1_000_000) * rates.output;
}

This is pseudocode. The exact shape of the response object and the field names for token counts differ by provider and SDK version, so check your provider's current API reference before wiring this up.

2. Roll the rows up into views that answer real questions

Raw rows are not a dashboard. Aggregate them into a small number of views that map to decisions you actually need to make:

1. Cost per user, per day: sort descending and the top of the list is where your spend concentrates.

2. Cost per feature: shows whether a chatbot, a summarizer, or a batch export is driving the bill.

3. Cost per model: useful once you have more than one model in production, so you can see if a cheaper model would do.

4. 7-day trend per user: a flat line that suddenly steps up is worth a look before it becomes a pattern.

3. Surface it somewhere you will actually look

A table nobody opens is not a dashboard. At a minimum, put daily total spend and the top five users by cost on a page you check as often as you check signups. Many teams put this inside an existing internal admin view rather than building a separate tool.

A Worked Example With Real Numbers

Say your app runs mostly on Claude Haiku 4.5 for short interactions, at $1 per million input tokens and $5 per million output tokens. A typical request sends 800 input tokens and gets back 400 output tokens:

(800 / 1,000,000) x $1 + (400 / 1,000,000) x $5 = $0.0008 + $0.002 = $0.0028 per request.

At 200 such requests a day for a normal user, that is $0.56 a day, or roughly $17 a month, which is easy to absorb. Now say one user's workflow uploads a long document and asks for analysis: 25,000 input tokens and 3,000 output tokens, still on Haiku 4.5:

(25,000 / 1,000,000) x $1 + (3,000 / 1,000,000) x $5 = $0.025 + $0.015 = $0.04 per request.

That single request costs about 14 times more than the typical one. If that same user runs it 50 times a day, their daily cost is $2, which is more than three times what 200 normal users' worth of typical requests would cost combined. A dashboard built on request counts would show this user as "active," not as the reason your model bill doubled. A dashboard built on logged token cost shows it immediately, because it is sorted by dollars, not by clicks.

Setting Alerts and Thresholds

Once cost per user is visible, set a simple threshold and alert on it rather than waiting to notice a trend by eye. A reasonable starting point for a small app is to flag any single user who crosses $5 of model spend in a rolling 24-hour window, or whose daily spend is more than 5 times their trailing 7-day average. Both numbers are starting points, not rules. Tune them against your own margins once you have a few weeks of real data.

Treat the alert as a signal to look, not necessarily to block. A support agent handling a genuinely long document is a good user having a legitimate day. A script hitting your API in a loop is a different story, and that is a job for request-rate limiting, not for the usage dashboard itself.

How This Relates to Rate Limiting

A usage dashboard and a rate limiter solve related but different problems, and it helps to be clear about which is which. Rate limiting caps how often someone can call your API, which stops scraping and brute-force abuse. It does not know or care how expensive any individual request is, so a user can stay under a request-rate limit while still running up an enormous model bill through a handful of very large calls. If you have not set that layer up yet, how to add rate limiting to an AI built app walks through a concrete token bucket implementation with real numbers.

The usage dashboard is what tells you where to actually set those limits, and where a request-rate cap alone will not be enough because the danger is cost per request, not request frequency. For more background on how AI providers structure their own request-rate limits, AI API Rate Limits Explained covers how those provider-side limits work.

Where to Put the Dashboard in Your App

If you already have, or are planning, an internal control panel for your app, a usage view belongs there rather than in a standalone tool. See how to add an admin dashboard to an AI built app for the general pattern of building that panel, and add a usage tab to it once it exists.

Keep in mind that your model provider's own console shows total account-level spend, not a breakdown by your app's individual end users. That attribution has to happen in your own application code, because the provider has no visibility into which of your users triggered which call. This is the core reason the logging step described above is not optional if you want per-user numbers.

This guide assumes you already have a working app in production. If you are still putting the basics together, how to build an app with AI covers the fundamentals this usage layer sits on top of.

Questions

How do I calculate the cost of a single AI API request?

Multiply the input token count by the provider's per-token input price, multiply the output token count by the output price, and add the two together. Most model APIs return the exact token counts used in the response object for each call, so you do not need to estimate.

Should I show users their own usage inside the app?

For apps with metered or tiered pricing, yes, a simple usage-this-month view builds trust and reduces support questions. For internal tools or apps without usage-based billing, the dashboard can stay purely internal.

Is a usage dashboard a replacement for rate limiting?

No. They address different failure modes. Rate limiting stops a high volume of requests from overwhelming your app or your API quota. A usage dashboard catches the case where request volume looks normal but a small number of unusually expensive requests are driving up cost. Most production AI apps need both.

What counts as a token, in simple terms?

A token is a chunk of text, roughly three to four characters in English on average, that a model processes as one unit. Providers price and meter usage in tokens because that is the unit the model actually consumes, not characters or words.

Do I need a separate database table, or can I reuse existing logs?

A dedicated table is worth the extra setup. General application logs are usually not structured for fast aggregation by user and by day, and they often get rotated or deleted on a schedule that is too short for month-over-month cost analysis.