How to monitor RPC latency (p50 / p95 / p99) for Solana bots

TL;DR

→  The average latency number on your dashboard is lying to you. RPC latency distributions are right-skewed—a few slow requests drag the mean, and your bot rarely experiences the mean. Track percentiles instead.

→  p50 tells you the typical case, p95 tells you the bad case, p99 tells you the case that loses trades. For competitive Solana bots, p99 is the number that maps to whether you land in the target slot during congestion.

→  Instrument four segments separately, not one blended number: RPC call latency, data freshness lag (slot behind tip), submission latency, and time-to-land. When one degrades, you need to know which.

→  Compute percentiles in the metrics layer using histograms (Prometheus histogram, HDR histogram, OpenTelemetry). Never average pre-averaged percentiles—that's statistically meaningless.

→  Alert on p99 breaching a threshold for N minutes, not on single spikes. A one-off p99 blip is noise; sustained p99 > 100 ms during a trading window is a page-worthy incident.

→  When p99 spikes but p50 stays flat, it's a tail problem: rate limiting, a noisy neighbor on a shared node, or an overloaded endpoint. When both rise, it's systemic—network congestion or a far-away region.

→  The fix for tail-latency problems is usually infrastructure, not code. Dedicated nodes remove noisy-neighbor variance; co-location removes geographic latency; staked submission paths remove congestion drops.

‍

Here's a scenario every Solana bot operator eventually hits. Your dashboard shows an average RPC latency of 40 ms. Looks healthy. Green across the board. Meanwhile, your bot is quietly missing entries during every memecoin launch, your landing rate has slid from 92% to 70%, and you have no idea why—because the metric you're watching is the one metric that hides the problem.

The average is the wrong tool. Not because averages are bad, but because RPC latency doesn't follow the kind of distribution where an average tells you anything useful. A small fraction of requests are dramatically slower than the rest, and those slow requests are exactly the ones that decide whether your transaction lands in a competitive slot. Averaging them into a single number throws away the signal you actually need.

This guide covers how to monitor Solana RPC latency properly: which percentiles to track, which four latency segments to instrument separately, how to build the monitoring stack, how to set alerts that fire on real problems and not on noise, and how to read the numbers to diagnose what's actually wrong when things degrade. 

Why the average lies

Start with what an RPC latency distribution actually looks like. If you log the response time of every RPC call your bot makes over a busy hour and plot the histogram, you don't get a nice bell curve centered on the average. You get a right-skewed distribution: a tall cluster of fast responses on the left, and a long tail of slow ones stretching to the right.

Look at where the mean falls versus the percentiles. The mean is pulled to the right by the slow tail, so it lands somewhere between the median and p95—a latency that relatively few individual requests actually experience. When you tune a bot's timeout, retry logic, or fee strategy against the average, you're tuning against a number that doesn't correspond to a real operating condition.

Percentiles fix this by telling you about specific points in the distribution:

• p50 (median): half your requests are faster than this, half slower. This is the typical experience—what a request looks like on a normal day with no congestion.
• p95: 95% of requests are faster than this. This is the bad-but-not-rare case. One in twenty requests is at least this slow.
• p99: 99% of requests are faster than this. This is the tail. One in a hundred requests is at least this slow—and during congestion, when everyone's bot is hammering the network, your competitive transactions cluster in exactly this region.

Google's engineering teams formalized this thinking in a well-known paper, "The Tail at Scale," which argues that in any system making many dependent requests, tail latency—not average latency—determines user-facing performance. A Solana bot is precisely this kind of system: it makes many RPC calls per opportunity, and the slowest call in the chain gates the whole operation. Your effective latency is closer to your p99 than your p50.

Why p99 specifically matters for Solana bots

On Solana, slots are ~400 ms. When you're competing for an entry—a token launch, an arbitrage gap, a liquidation—the window to land in the right slot is a fraction of that. If your RPC call to fetch the latest blockhash, or your sendTransaction submission, lands in the p99 tail instead of near the median, you've spent 200+ ms on a single operation that should have taken 15. That's half a slot gone, and in a competitive launch, half a slot is the difference between the first fill and watching the price move 3x without you.

The brutal part is that these tail events cluster exactly when they hurt most. During quiet hours, your p50 and p99 are close together—everything's fast. During a memecoin launch, when landing the trade actually matters, the tail blows out. Your p50 might stay at 15 ms while your p99 jumps to 400 ms. If you're watching the average, your dashboard barely twitches. If you're watching p99, you see the problem immediately.

What to measure: four segments, not one number

"RPC latency" is not a single quantity. It's at least four distinct things, each with a different failure mode and a different fix. Blending them into one number means that when it degrades, you can't tell which part broke. Instrument them separately.

1. RPC call latency

The round-trip time for a read call—getLatestBlockhash, getAccountInfo, getMultipleAccounts, getSlot. This is the most basic measurement: how long from when you send the request to when you get the response. Track it per method, because a getAccountInfo and a getProgramAccounts have very different cost profiles and blending them hides problems. Target for a competitive setup: p99 under 15 ms for lightweight calls.

2. Data freshness lag

How many slots behind the chain tip is the state you just read? This is not a time measurement—it's a slot delta. You can have a fast RPC call that returns stale data, which is arguably worse than a slow call that returns fresh data, because you don't know it's stale. Measure it by comparing the slot in your response against getSlot with processed commitment. Target: under 1 slot of lag. Two or more slots behind means you're reacting to a version of the chain that's already gone.

3. Submission latency

The round-trip time for sendTransaction—from when you fire the submission to when the RPC acknowledges receipt. This is distinct from RPC call latency because the submission path is different from the read path: it involves the transaction reaching the leader's TPU, which is where staked connections and SWQoS matter. Target: p99 under 30 ms.

4. Time-to-land

The full loop: from submission to the transaction being confirmed in a block. This is measured in slots, not milliseconds, and it's the metric that most directly reflects real execution quality. Measure it by subscribing to your transaction's signature and recording the slot delta between submission and confirmation. Target: under 2 slots. This is the number that correlates most tightly with your landing rate and, ultimately, your P&L.

These four are related but independent. A rate-limit problem shows up in segment 1. A slow-syncing node shows up in segment 2. A non-staked submission path shows up in segment 3. Network congestion shows up in segment 4. The Solana RPC method reference documents the calls; your job is to wrap each one in a timer and record the distribution.

How to instrument it correctly

The single most common mistake in latency monitoring is computing percentiles wrong. You cannot average percentiles. If one node reports a p99 of 40 ms and another reports a p99 of 200 ms, the combined p99 is not 120 ms—that number is meaningless. Percentiles must be computed from the underlying distribution, which means you need to record the distribution, not pre-computed summaries.

The right way to do this is with histograms. Instead of storing every raw latency value (expensive) or storing just an average (lossy), you store counts in latency buckets. A histogram with buckets at 5, 10, 20, 50, 100, 200, 500, 1000 ms captures the shape of the distribution cheaply, and you can compute any percentile from it after the fact—and correctly aggregate across nodes by summing bucket counts.

A minimal instrumentation pattern

Here's the shape of it in TypeScript, wrapping an RPC call with a histogram timer. The specifics depend on your metrics library, but the pattern is universal: record start, make the call, record the elapsed time into a histogram tagged with the method name.

// Prometheus-style histogram with method + endpoint labels
const rpcLatency = new Histogram({
  name: 'solana_rpc_latency_ms',
  help: 'RPC call latency in milliseconds',
  labelNames: ['method', 'endpoint', 'region'],
  buckets: [5, 10, 15, 20, 30, 50, 100, 200, 500, 1000],
});
 
async function timedCall(method, endpoint, region, fn) {
  const start = performance.now();
  try {
    return await fn();
  } finally {
    const elapsed = performance.now() - start;
    rpcLatency.labels(method, endpoint, region).observe(elapsed);
  }
}
 
// usage
const blockhash = await timedCall(
  'getLatestBlockhash', 'rpcfast-us-east', 'us-east',
  () => connection.getLatestBlockhash('processed')
);

Three things make this pattern work. First, the finally block ensures you record latency even when the call throws—failed calls have latency too, and a call that times out at 5000 ms is critical signal you don't want to drop. Second, the labels (method, endpoint, region) let you slice the data later: p99 for getLatestBlockhash specifically, on the us-east endpoint specifically. Third, the bucket boundaries are chosen around your targets—dense where you care (5–50 ms) and sparse where you just need to catch outliers (500–1000 ms).

For the histogram implementation itself, Prometheus client histograms are the most common choice and aggregate correctly across instances. If you need high-precision percentiles with low memory overhead, HDR Histogram is the gold standard—it's what latency-sensitive systems use when they need accurate p99.9 and beyond. OpenTelemetry metrics work well if you're already in that ecosystem.

The chart above is the whole argument for percentile monitoring in one image. Across a trading day, the median latency barely moves—it sits calmly around 15 ms whether the network is quiet or on fire. But p99 tells a completely different story: it spikes to 300, 200, and 450 ms during a memecoin launch, an NFT mint, and a liquidation cascade respectively. Those are the exact moments your bot is trying to compete, and those are the moments a mean-based dashboard tells you everything is fine.

Building the monitoring stack

A production latency monitoring setup has three layers: instrumentation inside your bot, a collection layer that aggregates the metrics, and a visualization and alerting layer on top. You don't need anything exotic—the standard observability stack does the job—but you do need to wire the layers together correctly.

Layer 1—instrumentation

Inside the bot, wrap every RPC call and submission in a histogram timer, as shown above. Record the slot deltas for data freshness and time-to-land. Tag everything with method, endpoint, and region so you can slice later. This is the only layer that touches your hot path, so keep it cheap—a histogram observe is a few nanoseconds, which is negligible next to a network call.

Layer 2—collection

Prometheus scrapes a /metrics endpoint your bot exposes, pulling histogram data at a fixed interval. If you prefer push over pull, StatsD or an OpenTelemetry collector work. In parallel, emit structured per-request logs (JSON with latency, slot, method, endpoint) to a log sink—these are invaluable for post-incident forensics when you need to reconstruct exactly what happened during a specific 30-second window. Prometheus histogram aggregation handles the cross-instance math correctly, which is the whole reason to use it rather than rolling your own.

Layer 3—visualization and alerting

Grafana panels showing p50/p95/p99 per endpoint and region give you the at-a-glance view. Grafana's histogram_quantile function computes percentiles from Prometheus histogram buckets at query time. Alertmanager (or your paging tool of choice) fires when p99 breaches a threshold for a sustained period. The key design decision is in the alerting logic, which is worth its own section.

A note on computing percentiles correctly

This bears repeating because it's the error that silently corrupts most homegrown dashboards: percentiles do not average. If you have three nodes and you want the fleet-wide p99, you cannot take each node's p99 and average them. You must sum the histogram buckets across all three nodes, then compute the p99 from the combined distribution. Prometheus and OpenTelemetry do this correctly if you use histograms. If you're storing pre-computed percentiles and averaging them, your numbers are wrong in a way that gets worse exactly when the fleet is under uneven load—which is precisely when you need them to be right.

Setting alerts that fire on real problems

A latency alert that fires on every single p99 spike will page you constantly and train you to ignore it. A latency alert that only fires on sustained degradation will page you when something is actually wrong. The difference is in the alerting window.

The pattern that works: alert when p99 exceeds your threshold for a sustained duration, not on any single scrape. For example, page when the 5-minute p99 for sendTransaction exceeds 100 ms for at least 3 consecutive minutes. A one-off spike from a single slow request is noise. Three minutes of elevated p99 is a real endpoint problem, a rate-limit situation, or a network event—something you can act on.

Reasonable starting thresholds for a competitive Solana setup, to be tuned against your own baseline:

These are starting points, not universal truths. The right thresholds depend on your strategy, your region, and your baseline. The discipline that matters is: establish your normal p99 during quiet hours, then alert on meaningful deviation from it, sustained over a window that filters out single-request noise.

Diagnosing a latency spike

When the alert fires, the percentile structure tells you where to look. The first and most useful question: did p50 rise too, or did only p99 spike? That single distinction splits the diagnosis into two very different branches.

Only p99 rose—a tail problem

If p50 stayed flat and only p99 climbed, your typical request is still fast but a growing fraction are slow. This is almost always one of two things. Rate limiting: you're hitting your RPS cap, and excess requests get queued or 429'd, showing up as tail latency—check your request rate against your plan's limit. Or a noisy neighbor: on a shared RPC endpoint, someone else's traffic surge is starving your requests intermittently. The tell is that the slowness is erratic and uncorrelated with network-wide events. The fix for both is the same: dedicated infrastructure removes the shared-tenancy variance entirely.

p50 rose too—a systemic problem

If the whole distribution shifted right—p50, p95, and p99 all climbed together—the problem is systemic, not tail-specific. Either the network itself is congested (check slot lag and network-wide failure rate; if everyone is slow, it's the chain, not you) or your endpoint is fundamentally overloaded or geographically distant from the current leader. The fixes here are failover to a healthier endpoint and co-location closer to validators, so your baseline latency is low enough that even a systemic shift stays manageable.

This is why you instrument the four segments separately. A tail spike in submission latency (segment 3) with flat RPC call latency (segment 1) points straight at the submission path—a non-staked connection getting deprioritized during congestion. A freshness lag spike (segment 2) with everything else flat points at a node falling behind in sync. The segment that moved tells you which subsystem to investigate.

Where infrastructure solves the problem

A recurring pattern in latency diagnosis is that the fix lives below your code. You can optimize your bot's logic all day, but if the tail latency comes from a shared endpoint's noisy neighbors or a non-staked submission path, no amount of application-level tuning removes it. These are infrastructure problems with infrastructure fixes.

• Noisy-neighbor tail latency → dedicated nodes. Single-tenant infrastructure means your p99 reflects only your own traffic, not the aggregate load of everyone sharing the endpoint.
• Geographic baseline latency → co-location. Nodes positioned near validators in US East, EU, and APAC cut the physical round-trip that no code change can touch.
• Submission-path congestion drops → SWQoS staked identity. A staked connection to the leader's TPU is prioritized during exactly the congestion windows where public paths get deprioritized.
• Data freshness lag → Yellowstone gRPC with ShredStream. Streaming state at processed commitment, filtered server-side, keeps your freshness lag under a slot instead of polling a node that's a few hops behind.

This is the layer RPC Fast operates: dedicated bare-metal Solana nodes co-located with validators, Yellowstone gRPC with filtered streams, Jito ShredStream enabled by default, SWQoS-enabled transaction paths, and sub-50 ms automated failover. The point isn't that infrastructure replaces good monitoring—it's that good monitoring frequently reveals problems that only infrastructure can fix. You measure the tail, you find it's noisy-neighbor variance, and the honest fix is to stop sharing the node.

What good looks like: reference latency numbers

Once you're measuring correctly, you need a reference for what healthy looks like. These are RPC Fast internal benchmarks across roughly 40,000 sampled calls in Q1 2026, spanning US-East, EU, and APAC regions, as a calibration point for your own dashboards:

Numbers are illustrative aggregates and vary by region, method, and network conditions. Use them as a rough calibration, not a guarantee. The point is the shape: on healthy dedicated infrastructure, p99 should be a small multiple of p50, not 10–40x higher the way it is on a congested public endpoint. If your p99 is orders of magnitude above your p50, that gap is your problem to chase—and usually it's living in the infrastructure layer, not your code.

What actually matters

Monitoring RPC latency for a Solana bot comes down to a few disciplines. Measure percentiles, not averages, because the average hides the tail and the tail is where you lose trades. Instrument the four segments separately, so a degradation tells you which subsystem broke. Compute percentiles from histograms in the metrics layer, never by averaging pre-averaged numbers. Alert on sustained p99 breaches, not single spikes. And when you diagnose a spike, let the p50-vs-p99 structure route you to the cause.

Most importantly: when the numbers point at the infrastructure layer—noisy neighbors, non-staked paths, geographic latency—accept that the fix lives there, not in another round of code optimization. The best-instrumented bot in the world can't tune its way out of a shared endpoint's tail latency. Good monitoring's real value is telling you, precisely and early, when the problem is one you can't code around.