How we run a GTmetrix-class testing stack on one small VPS

RankVitals runs throttled Lighthouse tests with waterfalls, filmstrips and letter grades; a 57-check SEO and AI-readiness crawl; uptime monitoring with scheduled re-tests and alerting; AI-generated fix plans; and an MCP server that lets an AI agent drive all of it. That is the feature list of a product you would expect to be running on a cluster.

It runs on one small VPS. Not as a stunt, and not because a bigger box was unaffordable — because when we costed the pieces honestly, almost none of the distributed machinery was buying us anything. The genuine constraint in this product is a single number, and it has nothing to do with the queue, the cache layer or the orchestrator.

This is the whole architecture, including the parts that are unimpressive on purpose and the parts we would change. Every claim below is something you could verify by reading the code.

The one number that decides the architecture

A headless Chrome running a Lighthouse test holds roughly 500 MB. The worker container is capped at 2 GB. That is the entire capacity model — you can fit one comfortably, two if nothing else goes wrong, and the third one is an outage.

Everything downstream follows from that. If a box can only run one or two tests concurrently, there is no throughput problem for a queue to solve. A queue that can dispatch ten thousand jobs a second in front of a worker that can start one job every forty seconds is not infrastructure, it is decoration. Once you accept the Chrome number, most of the usual stack stops earning its place.

We also launch Chrome fresh for every single run and kill it afterwards, in a finally block, unconditionally. Partly that is memory hygiene — a leaked instance is 500 MB gone until something notices. Mostly it is result integrity: a reused browser accumulates warm HTTP caches, resolved DNS and JIT state that make the second test of a site quietly faster than the first. A performance product that reports optimistic numbers because of its own caching is worse than no product.

We deleted Redis and made Postgres the queue

Version one used BullMQ on Redis, which is the default answer for Node job queues and worked exactly as advertised. Then we read the Redis dashboard. Our managed free tier meters commands per month, and BullMQ workers poll constantly — delayed-job checks, stalled-job checks, heartbeats. A worker that had processed almost nothing had consumed most of the monthly quota asking "anything yet?" into an empty queue.

The realisation was that every test already lived in a Postgres row with a status column. The queue was a second, redundant copy of state we were already keeping. So the tests table became the queue, and claiming work became one function:

create or replace function public.claim_next_test(p_locations text[] default null)
returns setof public.tests
language sql security definer
as $$
  update public.tests t
  set status = 'running', started_at = now()
  where t.id = (
    select id from public.tests
    where status = 'queued'
      and (p_locations is null
           or coalesce(config->>'location', 'default') = any (p_locations))
    order by created_at asc
    for update skip locked
    limit 1
  )
  returning t.*;
$$;

FOR UPDATE SKIP LOCKED is the whole trick: two workers racing for work lock different rows rather than blocking on the same one, so adding a second worker box later needs no code change at all. A partial index on created_at where status = 'queued' keeps the claim cheap however large the table gets. The p_locations argument arrived later, when we wanted region-scoped workers — a worker lists the locations it serves and claims only those rows, and a NULL keeps the old single-region behaviour intact.

Crash recovery is a second function rather than a heartbeat protocol: anything still marked running after fifteen minutes is dead, so fail it. The worker calls it hourly. No stalled-job events, no lease renewal, just a timestamp comparison you can also run by hand in a SQL console when you are trying to understand what happened at 3am.

What we gave up is real: no pub/sub, no push notification when a job finishes, no per-job priority beyond created_at ordering, and a poll every four seconds instead of an instant wake-up. What we got back was one fewer service to run, one fewer bill, one fewer thing to monitor, and a queue that is inspectable with SELECT. For a workload whose unit of work takes forty seconds, four seconds of scheduling latency is a rounding error.

Shutting down without losing a test

The worker restarts on every deploy, and a restart in the middle of a four-minute Lighthouse run is the obvious way to lose someone's test. On SIGTERM it stops claiming new work and gives the in-flight run up to 45 seconds to finish. If it finishes, everything is normal. If it does not, the worker kills Chrome and sets that row back to queued with started_at cleared — so the next worker picks it up immediately instead of the user waiting out the fifteen-minute reaper.

The update is conditioned on the row still being in the running state, which is what makes it safe to race against anything else touching the row. This is the kind of code that looks like paranoia until the first time you deploy during a busy hour.

// requeue instead of leaving the row for the 15-minute reaper
await db
  .from('tests')
  .update({ status: 'queued', started_at: null })
  .eq('id', currentTestId)
  .eq('status', 'running');

The artifact pipeline, and the ordering rule that matters

A finished Lighthouse run produces more than scores. We keep four kinds of artifact: the full Lighthouse JSON, a HAR file, the filmstrip frames, and a final screenshot.

The HAR is the interesting one, because we do not capture it with a proxy or a browser extension. Lighthouse already records a DevTools protocol log of every network event during the run; the HAR is reconstructed from that log after the fact. The waterfall you see in a report is therefore a rendering of exactly what Chrome did during the measured load — not a second, separate crawl that might have hit a different cache state. The filmstrip and the final screenshot come out of the Lighthouse result the same way, as base64 JPEG frames with their capture timings.

The rule we care most about is the ordering. The test row is marked completed with its scores and metrics FIRST, before a single artifact upload is attempted. Artifacts upload best-effort afterwards, and a storage hiccup means one tab in the report shows its empty state — never a failed test with the credit burned. Getting this backwards is an easy and expensive mistake: it converts every blip in your object storage into a customer-visible failure of a run that actually succeeded.

The one wrinkle is the window between "completed" and "artifacts uploaded", where a report would otherwise claim there is no waterfall when one is seconds away. An artifacts_ready flag written in the same update as the status claims that window, and is flipped true when the upload pass ends — whether or not every upload succeeded, because at that point whatever landed is all there will ever be.

Failures, retries and who pays for them

Credits are spent when a test starts, not when it finishes, so the only question at failure time is who caused it. Every failure message is classified into one of three buckets, and the bucket decides both whether we try again and whether the credit comes back:

• transient — a timeout, a socket error, Chrome refusing to launch or dying mid-run, a Lighthouse run that produced nothing. Requeued for another attempt; two attempts total by default.
• user — the request itself was bad: an invalid URL, a private or internal address, a hostname that does not resolve. Terminal and not refunded. Retrying a domain that does not exist just spends another four-minute Chrome run learning the same thing twice.
• infra — everything else, including our own bugs and storage failures. Terminal and refunded. Unrecognised messages land here deliberately: nobody should pay for a failure we cannot explain.

The refund is not issued by the worker. Writing the failure class onto the row is what triggers it, in the same transaction as the status change — so a worker that dies immediately after marking a test failed still refunds, and so do the reaper's SQL-only failures where no worker is involved at all. Putting the money logic in the database rather than the process that might be dying is the sort of decision you make once and never regret.

What else the box is doing

The same process runs several independent loops beside the test queue, which is another thing a single box makes easy and a fleet makes complicated:

• The SEO and AEO crawler — 57 checks over fetched HTML, no browser involved, which is why it can share a box with something as heavy as Chrome. It is what produced our starter-template study.
• Uptime checks and the monitor scheduler, which enqueue scheduled re-tests through the same queue as manual ones — a monitored run is not a special code path, it is a row.
• Alert evaluation after every monitored test, with delivery to Slack, Discord, Teams, PagerDuty and generic webhooks.
• A weekly digest, and a daily retention pass that deletes storage objects before their rows so a crash never orphans a blob.
• A separate job loop for AI fix plans and PDF exports, so a slow model call can never block the test queue.

Deploying with no CI

There is no build server. A deploy is git archive of a committed tree, piped over SSH, extracted into a directory on the box, and rebuilt with docker compose. The tree must be clean — the script refuses a dirty working directory, because a tarball of uncommitted work is a build nobody can ever reproduce.

The consequence is that the deployed directory has no git metadata, so "what is actually running?" is not a question you can answer with git log on the box. Two things answer it instead: the commit SHA is written to a VERSION file at extract time, and it is also baked into the image as an environment variable that the worker logs at boot and republishes in its heartbeat. Those can disagree — extracted but not rebuilt — and when they do, that disagreement is the bug you are looking for.

The worker writes a heartbeat row every thirty seconds carrying its uptime, tests completed, running commit and how many scheduled monitor windows it has skipped. A cron job hits a watchdog endpoint every ten minutes and fails loudly if that heartbeat has gone stale. That is the entire alerting stack for the backend, and for a one-person product it has been enough.

The honest tradeoffs

What this design costs, stated plainly:

• Tests are serial. Under load, tests queue. SKIP LOCKED means scaling is "add another box", but until we do, a busy hour means waiting.
• One region. If your origin is far from our worker, your TTFB carries that distance. The claim function already supports region-scoped workers; the second region is a deployment step, not a rewrite.
• One run per test, with simulated throttling. Single runs have variance — our own 60-run study had ten runs that needed a second attempt. Treat trends across runs as signal and any single run as an estimate.
• No push. The MCP server is stateless by design, so agents poll for results. Fine for a job that takes under two minutes; wrong for anything long-running. That architecture is its own post.
• A single box is a single point of failure. The mitigation today is that a crash loses nothing durable — queued rows stay queued, running rows get reaped and retried — not that the box is redundant.

None of this is clever, and that is the point we would most like to land. The interesting decisions were all subtractions: no Redis, no orchestrator, no build server, no session store, no message bus. What is left is a Postgres table, one Chrome at a time, a tarball over SSH, and a heartbeat. Every component can be inspected with curl or a SELECT, which means that when something breaks at 3am the debugging tool is understanding rather than archaeology.

If you want to see what the stack actually produces, the fastest route is to run a test on your own URL — a throttled Lighthouse run with the waterfall and filmstrip, plus the 57-check SEO and AEO crawl in the same result. It is free, it does not need a card, and it takes about as long as reading one more section of this post. The tools are free too, and the docs cover the API and MCP server if you would rather drive it from an agent than a browser.

FAQ

Would you use Postgres as a queue again?

For this workload, without hesitation. The condition that makes it right is a low job rate with an expensive unit of work — a queue that handles tens of jobs a minute in front of workers that take a minute each. If we were dispatching thousands of short jobs a second, the polling cost and the row churn would eventually make a purpose-built queue worth its operational weight. Start with the table; you will know when you have outgrown it.

Why kill Chrome after every run instead of reusing the browser?

Two reasons, and the second matters more. A leaked Chrome holds around 500 MB and two of them on a small box is an outage. But the real reason is measurement integrity: a reused browser carries warm caches, resolved DNS and JIT state into the next run, which makes later tests of the same site look faster than the site actually is. A performance product cannot afford to be optimistic about itself.

How do you know what commit is running if the deploy has no git metadata?

Two independent stamps that can disagree. The deploy writes the SHA to a VERSION file when it extracts the tarball, and the same SHA is baked into the container image as an environment variable that the worker logs at boot and publishes in its heartbeat. VERSION says what was extracted; the heartbeat says what is executing. When they differ, the image was not rebuilt — which is exactly the failure you would otherwise spend an hour not noticing.

What happens to my test if you deploy while it is running?

The worker stops claiming new work and gives the in-flight run 45 seconds to finish. If it does not finish, Chrome is killed and the row goes back to queued with its start time cleared, so the next worker picks it up straight away rather than the test waiting out the fifteen-minute stale-test reaper. You see a slightly slower test, not a failed one.