How not to double-bill a dead phone

A parent taps “Create,” nothing visibly happens for ten seconds, so they tap it again. That’s the moment I want to talk about.

My bedtime-story app generates a whole story from an LLM. That call is slow (60 to 120 seconds) and it costs real money every single time. You can’t hold an HTTP request open for two minutes across a flaky mobile network and a phone OS that will happily suspend your app to save battery. So generation is asynchronous: the client POSTs, the server returns a 202 with a job_id, and the client polls until the story is done.

Going async buys you a lot. It also hands you a problem you never had with a synchronous call: retries.

The network blips and the client resends. The parent double-taps because nothing happened. The phone dies mid-generation, the app relaunches, and it has no idea whether it already kicked off a story. Every one of those is a chance to bill the LLM twice for one bedtime story, and to leave a parent with two near-identical stories they didn’t ask for.

Every one of them has to converge on exactly one LLM call. Not “usually one.” Exactly one.

The contract, and why it’s async

The shape of the thing is deliberately boring:

• POST /stories/generate returns 202 Accepted with { job_id, status: "pending" }
• the client polls GET /stories/jobs/:id every couple of seconds
• the job walks pending → running → completed | failed, and on completed it carries a story_id

No SSE, no streaming, no held-open connection. An in-process FIFO worker with single concurrency and a bounded queue picks the job up and does the expensive LLM call out of band:

// job-queue.ts: bounded FIFO, single concurrency
enqueue(item: T) {
  if (this.buf.length >= this.opts.max) throw new TooManyRequests();
  this.buf.push(item);
  if (this.handler && !this.running) void this.loop();
}

Nothing exotic. The interesting part is everything guarding the entry to this queue, because that’s where money gets spent.

The failure modes I actually had to defend against

Once the operation is “slow, costs money, retryable,” the threat list writes itself:

1. The double-tap. Parent taps Create, sees no instant feedback, taps again. Two POSTs, milliseconds apart.
2. The network retry. The first POST succeeded on the server, but the 202 never made it back to the phone. The HTTP client retries. The server sees what looks like a brand-new request.
3. The dead phone. Generation is running server-side, the app gets killed. On relaunch the client has forgotten everything. Does it start a new story? If yes: double bill.
4. The impatient reload. User backgrounds and foregrounds the app over and over while waiting, each time triggering a resume path.

The fix, thankfully, is boring in the best way: an idempotency key the client mints once, a single INSERT ... ON CONFLICT DO NOTHING RETURNING on the server, and a little local storage on the client so a relaunched app resumes the job it already started instead of starting a new one.

The server half: an idempotency key and ON CONFLICT

The client generates a UUIDv4 and sends it as a header. The server rejects a missing or malformed one with a 400 Bad Request:

// stories.controller.ts
@Post('generate') @HttpCode(202)
async generate(
  @Headers('idempotency-key') key: string | undefined,
  @Body() body: unknown,
  @CurrentUser() u: { userId: number; blocked: boolean; dailyStoryLimit: number | null },
) {
  if (!key || !UUID.safeParse(key).success)
    throw new BadRequest('valid Idempotency-Key (uuid v4) required');
  // ...
}

That key is the deduplication identity for the whole operation. But the actual guarantee lives one layer down, in a single statement:

// stories.service.ts: claimJobWithQuota()
const inserted = await tx.insert(storyJobs).values({
  userId: opts.userId, idempotencyKey: opts.idempotencyKey,
  kidId: opts.kidId ?? null, params, status: 'pending',
}).onConflictDoNothing({ target: storyJobs.idempotencyKey }).returning();

if (!inserted.length) {
  const [existing] = await tx.select().from(storyJobs)
    .where(eq(storyJobs.idempotencyKey, opts.idempotencyKey));
  return { job: existing, created: false };   // race on same key, no quota consumed
}

idempotency_key has a unique constraint. The first request wins the insert and gets created: true. Any concurrent or later request carrying the same key hits the conflict, inserts nothing, reads back the row that already exists, and returns created: false. The failed insert and the follow-up SELECT both run on the same tx, inside one transaction, so there’s no window for a race between them. The controller only enqueues the worker when created is true:

if (created) this.queue.enqueue(job.id);
return { job_id: job.id, status: job.status };

So a double-tap and a network retry both resolve to the same job_id, and the worker runs once. The client can’t tell whether it was the winner or a replay, and it doesn’t need to. It got a job_id to poll either way.

You might reach for a heavier tool here. There’s a well-known pattern for exactly-once mutations: a two-transaction mutation_log protocol (record intent, do the work, record completion). It’s the right tool when you have many different mutation points that all need dedup. But here there is exactly one paid mutation point: story generation. A single INSERT ... ON CONFLICT on a uniquely-constrained key gives the same guarantee with one statement and no second table. Reaching for the mutation-log machinery would be gold-plating a door with one lock on it.

The sharp detail: check the replay before the rate limiter

This is the part that’s easy to get subtly wrong, and it’s the whole reason retries are actually free.

The AI endpoints are rate-limited per user (a sliding window, default 6 requests per minute; this.limiter is an injected token-bucket / sliding-window limiter) to bound spend. The naive ordering is: check the limiter, then dedup. Fine, until you think about a client that’s retrying precisely because it never got the first response.

• Request 1 arrives, passes the limiter, creates the job. The response is lost on the way back.
• The client retries. Request 2 arrives, and if the limiter runs first, retry #2 consumes a token against the user’s window, even though it’s the same logical operation.

Under a burst of retries you could rate-limit a user out of their own single story. The retry, the thing they didn’t choose to do, eats the budget meant to stop abuse.

So the ordering is inverted. Look at the controller flow:

const params = GenerateBody.parse(body);
const blocked = checkInputsBlocklist(params);
if (!blocked.ok) throw new ContentBlocked(blocked.reason);

const replay = await this.svc.findJobByKey(u.userId, key);   // <-- replay FIRST
if (replay) return { job_id: replay.id, status: replay.status };

if (u.blocked) throw new AccessSuspended();
if (!this.limiter.tryAcquire(u.userId)) throw new RateLimited();   // <-- limiter AFTER
// ... only now do we claim the job + quota and enqueue

findJobByKey is a cheap read-only lookup:

/** Read-only idempotency-replay lookup, lets the controller skip rate limiting for retries. */
async findJobByKey(userId: number, idempotencyKey: string) {
  const [row] = await getDb().select().from(storyJobs)
    .where(and(eq(storyJobs.idempotencyKey, idempotencyKey), eq(storyJobs.userId, userId)));
  return row ?? null;
}

If the key is already known, we short-circuit and return the existing job before touching the limiter, the quota, or the LLM. A retry is idempotent all the way down: no token consumed, no daily quota decremented, no provider call. Only a genuinely new key gets past the limiter and into claimJobWithQuota, where the per-user daily quota is incremented in the same transaction as the job insert. So a lost 202 never leaves a phantom quota charge: a same-key race returns before the quota row is ever touched.

The same replay-before-limit shape is repeated deliberately on every paid endpoint: regenerate, continue (series episodes), and even audio narration:

// requestAudio(): mirrors /generate
// Replay before rate limit: a repeat request for already-claimed audio
// returns the existing job without burning a token (mirrors /generate).
const existing = await this.svc.findReusableAudio(u.userId, parseId(id));
if (existing) return { audio_id: existing.id, status: existing.status };
if (!this.limiter.tryAcquire(u.userId)) throw new RateLimited();

Once you internalize the rule it collapses to one line: dedup is not abuse, so check it before you charge for abuse.

The client half: persist and resume, so a dead phone can’t double-bill

Server-side idempotency handles double-taps and network retries, because those resend the same key. The dead-phone case is different. If the app forgets its key, relaunches, and mints a new one, the server has no way to know it’s the same intent. New key, new job, new bill. So the client has to remember.

The moment the 202 comes back, the controller writes a tiny record to local storage: a UUID, the job id, and a timestamp. No secrets, so shared_preferences rather than secure storage, and it needs to be readable on the first frame:

// generate_controller.dart
final key = const Uuid().v4();
final res = await repo.generate(key, p);
final rec = InFlightRecord(
  idempotencyKey: key,
  jobId: res.jobId,
  startedAt: DateTime.now().toUtc(),
);
await store.save(rec);                 // persist the instant we have a job_id
state = GenerateRunning(jobId: res.jobId, startedAt: rec.startedAt);
_startPoll(res.jobId);

It’s cleared only on a terminal status, completed or failed, inside the poll loop:

if (j.status == JobStatus.completed && j.storyId != null) {
  _timer?.cancel();
  await ref.read(inFlightStoreProvider).clear();
  state = GenerateDone(j.storyId!);
} else if (j.status == JobStatus.failed) {
  _timer?.cancel();
  await ref.read(inFlightStoreProvider).clear();
  state = GenerateFailed(j.errorText ?? 'unknown');
}

And here’s the payoff. On relaunch the app resumes the existing job instead of minting a new key:

Future<void> resumeIfAny() async {
  final store = ref.read(inFlightStoreProvider);
  final rec = await store.load();
  if (rec != null) {
    state = GenerateRunning(jobId: rec.jobId, startedAt: rec.startedAt);
    _startPoll(rec.jobId);            // poll the OLD job_id, no new POST, no new key
  }
}

The relaunched app slots straight back into polling the job the dead phone had already started. No second POST, so no second bill. And if it did re-POST with the same persisted key anyway, the server’s ON CONFLICT would still catch it. That’s the belt-and-suspenders split, and the ordering matters: the server’s ON CONFLICT is the actual guarantee, and client-side persistence is the optimization on top, avoiding a redundant request rather than being the thing that keeps you honest.

Two small robustness touches in the store are worth stealing:

// in_flight_store.dart: self-healing load()
final age = DateTime.now().difference(record.startedAt).inSeconds;
if (age > 3600) { await p.remove(_k); return null; }   // stale record → drop it
// ... and a jsonDecode try/catch that clears corrupt data instead of wedging launch

A stale record (a job older than an hour, well past any real generation) gets dropped rather than resumed into a job that will never complete. A corrupt blob clears itself instead of crashing every launch forever. Persisted state you resume from needs an escape hatch, or your double-bill fix quietly becomes a stuck-forever bug.

What I’d tell you to take from this

• Slow plus costs money means async means retryable means it needs idempotency. The moment you return 202 and poll, you’ve signed up to handle retries. Design for exactly-once from the start, not after the first double-bill lands in your logs.
• A client-minted idempotency key and a unique constraint is the whole trick. One INSERT ... ON CONFLICT DO NOTHING RETURNING gives you dedup with no second table and no distributed-lock ceremony. Don’t reach for a mutation-log protocol when you have a single paid mutation point.
• Check the replay before the rate limiter and the quota. Otherwise a client’s retries, which it didn’t choose, burn the budget meant to stop abuse, and a flaky network can rate-limit a user out of their own single request.
• The dead-phone case needs client memory, not just server dedup. Persist {key, job_id, started_at} the instant you get the job id, and on relaunch resume the old job instead of minting a new key. A forgetful client defeats server-side idempotency by sending a fresh key every time.

That said, the point isn’t that every operation needs this machinery. A cheap, fast, read-only endpoint retrying itself into oblivion costs you nothing, and wiring up idempotency keys and persisted in-flight records there would be effort spent guarding a door nobody’s trying to walk through twice. It’s that the moment an operation is slow, paid, and async, retries stop being a rare edge case and become the normal weather. You either make that operation safe to retry, or you make it unsafe to build a real app on top of.

Make the guarantee authoritative, not the client, and a dead battery costs you one story instead of two.

← all writing