How do you design an API endpoint for a long-running operation (e.g. video processing) — sync, polling, or webhooks?

#Why synchronous doesn't work here

Every layer between a client and your application has a timeout ceiling: browsers, HTTP client libraries, load balancers, and API gateways commonly cut a request off somewhere between 30 and 120 seconds. A video transcode, a large report export, or a bulk data import can easily run for minutes. If you try to hold the HTTP connection open for the whole duration, you're betting that none of those intermediate timeouts fire, you're tying up a server thread or worker for the entire job instead of freeing it to serve other requests, and you're giving the client zero visibility into progress — it either gets the final answer or a timeout, with nothing in between.

The fix is to stop treating "submit the work" and "get the result" as one HTTP exchange, and instead accept the job immediately, then let the client come back for the answer.

#The pattern: 202 Accepted plus a status resource

The client kicks off the job with a normal POST, but the server responds the moment the job is queued, not when it's done:

POST /videos/9f2c/transcode HTTP/1.1
Host: api.example.com
Content-Type: application/json

{ "profile": "1080p", "format": "mp4" }

HTTP/1.1 202 Accepted
Location: /videos/9f2c/transcode/jobs/b7e1
Retry-After: 5
Content-Type: application/json

{
  "jobId": "b7e1",
  "status": "queued",
  "statusUrl": "/videos/9f2c/transcode/jobs/b7e1"
}

202 Accepted is the right status here, and it's worth distinguishing from its neighbors: 200 OK says "here's your finished result right now," 201 Created says "the resource now exists exactly as returned," and 202 Accepted says "I took the request, processing hasn't necessarily finished, and I'm not promising it will succeed." The Location header points at a resource representing the job itself, and the Retry-After header (seconds, or an HTTP date) tells a well-behaved client how long to wait before checking again instead of hammering the endpoint.

The client then polls that job resource:

curl -s https://api.example.com/videos/9f2c/transcode/jobs/b7e1

{
  "jobId": "b7e1",
  "status": "processing",
  "progress": 42,
  "statusUrl": "/videos/9f2c/transcode/jobs/b7e1"
}

...until it eventually reaches a terminal state:

{
  "jobId": "b7e1",
  "status": "completed",
  "result": {
    "downloadUrl": "https://cdn.example.com/videos/9f2c-1080p.mp4",
    "durationMs": 812345
  }
}

If the job fails, the same resource should report it — and this is a natural place to reuse the RFC 7807 Problem Details shape, so a failed async job reports errors the same way a failed synchronous request would:

Wrap long lines
{
  "jobId": "b7e1",
  "status": "failed",
  "error": {
    "type": "https://api.example.com/errors/unsupported-codec",
    "title": "Unsupported Codec",
    "detail": "Source file uses a video codec that cannot be transcoded to the requested profile."
  }
}

#Polling vs. webhooks vs. staying synchronous

Polling is simple and works everywhere — the client doesn't need to accept inbound connections, which matters a lot for anything running behind NAT, in a serverless function, or on someone's laptop. Its cost is wasted requests when nothing has changed yet, and a notification delay bounded by however aggressively the client polls (which is exactly what Retry-After is for — telling the client how aggressively it's allowed to poll instead of guessing).

Webhooks flip the direction: instead of the client asking, your server calls the client's URL the instant the job finishes. That's lower latency and less wasted traffic, but it pushes real complexity onto both sides — your server needs retry logic with exponential backoff for when the client's endpoint is briefly down, both sides need to handle the same delivery arriving more than once (so the client's webhook handler must be idempotent?), and the client needs some way to verify the callback actually came from you and not from an attacker who guessed the URL:

const crypto = require('crypto');

function verifyWebhookSignature(rawBody, signatureHeader, secret) {
  const expected = crypto
    .createHmac('sha256', secret)
    .update(rawBody)
    .digest('hex');

  const provided = signatureHeader.replace('sha256=', '');
  return (
    provided.length === expected.length &&
    crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(provided))
  );
}

In practice, the systems people hold up as well-designed (Stripe, most cloud video/AI processing APIs) offer both: a webhook if the client configured one, with polling always available as the fallback and as what local development uses when there's no public URL to receive a callback at all.