Almost every integration that fails in production failed at the same place: somebody wrote a for loop over a list of cases, ran it against a staging key with eleven records in the list, watched it pass, and then pointed it at forty thousand matters on a Monday morning. Two minutes later the logs are a wall of 429s and the job has quietly stopped somewhere nobody can identify.
The failure is not really about rate limiting. It is about having designed for the wrong unit of work. A loop assumes that the only thing standing between you and the data is time. A queue assumes that throughput is a budget you are spending, and that the budget is small, fixed, and shared with everything else your product is doing. Court data work is squarely in the second world, and pretending otherwise is the most common architectural mistake in Indian legal tech.
This article sets out what the limits actually are on a production Indian court data API, what those numbers mean when you do the arithmetic on a real backfill, and the handful of patterns that turn a fragile script into something you can leave running over a weekend. The goal is not to squeeze past the limit. It is to build something whose behaviour at the limit is boring.
The Actual Numbers, and Why They Are Lower Than You Expected
On the CourtMesh API at https://research.courtmesh.ai/api/v1/prod, an API key is allowed 10 requests per minute. That is the number. It applies per key, across every endpoint, in a fixed window rather than a rolling one. Session traffic from the product itself runs at a much higher ceiling, 200 requests per minute, but that is not the lane an integrator is in and it is not a number to plan against.
Ten per minute surprises people who have integrated consumer APIs, where four figures per minute is unremarkable. It stops surprising them once they look at what a single call does. A keyword search runs against an index of roughly 310 million case records assembled from official government portals, and it is allowed up to ninety seconds before it times out. A semantic search runs a filter extraction model, a query expansion model, an embedding, and then a vector search, and the endpoint holds the connection open for as long as ten minutes. These are not lookups. They are units of computation, and they are priced in throughput accordingly.
The number that should drive your architecture
Ten requests per minute is 600 an hour and 14,400 a day if you never sleep and never waste one. Every design decision below follows from that single figure. If your plan requires more calls than that, the answer is not a faster client. It is fewer calls, better filters, and a longer schedule.
What a breach actually returns
When you cross the limit you get HTTP 429 with a JSON body carrying four things: an error of Rate limit exceeded, a human readable message, a retryAfter expressed in whole seconds, and a resetTime as an ISO timestamp. Both of the last two are telling you the same thing in two formats, and both are more reliable than any interval you invent yourself.
You do not have to wait for a 429 to know where you stand. Every response, successful or not, carries three headers: X-RateLimit-Limit, which is your ceiling, X-RateLimit-Remaining, which is what is left in the current window, and X-RateLimit-Reset, which is when the window rolls. A client that reads those headers can slow itself down before it is ever refused. A client that ignores them is choosing to learn about the limit by hitting it.
| Signal | Where it appears | What to do with it |
|---|---|---|
| X-RateLimit-Limit | Every response | Read it once at startup rather than hardcoding a number. If the ceiling changes, your client adapts instead of misbehaving. |
| X-RateLimit-Remaining | Every response | The primary input to a client side throttle. When it falls to one or two, stop issuing new work until the window resets rather than racing to spend the last of it. |
| X-RateLimit-Reset | Every response | An ISO timestamp for the start of the next window. Schedule against it rather than against your own clock, which may drift from the server's. |
| 429 with retryAfter | Only when you have already breached | Sleep for exactly that many seconds, then resume. Do not retry sooner, and do not retry the whole batch, only the refused call. |
| 408 on keyword search | When a query exceeds the 90 second server budget | Not a rate limit. It means the query was too broad. Narrow it by court, by year, or by date range and try again rather than retrying the same query harder. |
| 403 on AI endpoints | Semantic search and the analyze endpoints | The AI allowance is exhausted, which is a credit problem rather than a throughput problem. Backing off will not fix it. Stop the AI lane and alert a human. |
Do the Arithmetic Before You Write the Client
Most rate limit pain is avoidable by spending five minutes with a calculator before writing any code. The exercise is simple: count the calls your feature actually needs, divide by ten, and look honestly at the number of minutes that comes out.
Take a realistic backfill. Suppose you want the full record for fifty thousand matters in a practice area, and for each one you intend to fetch the case detail, the related documents, and the stored analysis. That is three calls per matter, so one hundred and fifty thousand calls. At ten per minute that is fifteen thousand minutes, which is two hundred and fifty hours, which is a little over ten days of continuous running with no failures, no retries and no other traffic on the key. If you had budgeted an afternoon, you were out by a factor of sixty.
Now redesign it. A keyword search returns up to one hundred results per page, and each result already carries the substantive case fields. Pull the population with paged searches rather than one fetch per case, and fifty thousand matters becomes five hundred search calls, roughly fifty minutes. Fetch the detail endpoint only for the matters a human or a rule has actually selected, which in practice is a few hundred. The related and analysis calls become lazy, made when someone opens the record. The same feature, designed twice, differs by two orders of magnitude in call count.
The fastest client is the one that does not make the call. Everything else is negotiation with a queue.
The Patterns That Actually Hold Up
There are perhaps five techniques worth knowing here, and they compose. None of them is exotic, and all of them are the difference between a job that survives an overnight run and one that does not.
Put a token bucket in front of the client, not a sleep in the loop
A fixed sleep of six seconds between calls appears to respect ten per minute and then breaks the moment two workers run at once, or a retry fires, or a user triggers an interactive search on the same key. A shared token bucket refilling at ten tokens per minute is the only construct that holds when concurrency is greater than one. Every call, interactive or batch, takes a token from the same bucket, because the server is counting them all in the same window.
Back off exponentially, with jitter, and honour retryAfter
On a 429, sleep for the retryAfter value the server gave you. On a 502 or a network error, back off exponentially: one second, then two, four, eight, capped at a minute, with a random jitter added so that a fleet of workers recovering from the same outage does not synchronise into a thundering herd. Cap the attempts at four or five and then move the item to a dead letter queue rather than retrying forever.
Separate the AI lane from the cheap lane
Semantic search and the two analyze endpoints behave differently from everything else. They are slower, they draw AI credit, and they can fail with 403 for reasons no backoff will cure. Run them through a separate queue with a lower concurrency and a separate alert. Mixing a ten minute semantic call into the same worker pool as a two hundred millisecond judge lookup means the cheap work waits behind the expensive work for no reason.
Make every unit of work idempotent and checkpointed
Each job should be safe to run twice. Key it on the case id or the search slice, write the result with an upsert, and record completion in your own store. Then a crash costs you one item rather than the run. For paged pulls, persist the cursor and the last completed page after every page, not at the end.
Schedule the batch against the calendar, not against optimism
If the arithmetic says a backfill takes four days, plan four days. Run it as a durable background job with a resume path, run it in off hours if interactive users share the key, and instrument it so somebody can answer where it has got to without reading logs. A job you can pause and resume is a job you can live with.
One key per workload
The limit is enforced per key. If your nightly backfill and your interactive product share a single key, the backfill will starve your users and the users will make the backfill unpredictable. Issue a separate key per workload and per environment. It costs nothing, it isolates the throughput, and it means you can revoke the noisy one without taking the product down.
Six Ways to Need Fewer Calls
Throughput engineering is mostly demand reduction. Before you tune the client, cut the work.
- Page at 100, not at 20. The limit parameter on keyword search defaults to 20 and accepts up to 100. Leaving it at the default means five times as many calls for the same result set, which is five times the throughput budget spent on nothing.
- Filter server side rather than client side. Passing court, caseType, year, fromDate and toDate narrows the result set before it is paged. Pulling a broad set and discarding most of it in your own code spends your budget on records you throw away.
- Cache aggressively, because judgments do not change. A decided case is immutable. Once you have fetched the detail for a case id, there is no reason to fetch it again this quarter. Pending matters are the only records with a genuine freshness requirement, and even those move on the timescale of hearings, not minutes.
- Read stored analysis before you generate it. GET on the analysis endpoint tells you whether analysis already exists via a hasAnalysis flag. Calling the analyze endpoint blindly, or worse with force set, turns a cheap read into an expensive write and burns credit as well as throughput.
- Prefer the related endpoint over per document fetches. One call to the related endpoint returns every stored document sharing a case number along with an assembled timeline, capped at fifty. Reconstructing the same thing with individual fetches costs you a call per document.
- Do not poll what you can pull on intent. Timeline jobs should start when a user asks for a hearing history, not on every page load. A poll loop attached to a page that renders a hundred times an hour is a rate limit incident waiting for a busy day.
Why the Limit Is Still Cheaper Than the Alternative
There is always a moment in this conversation where somebody says that ten per minute is inconvenient and asks whether it would be simpler to pull the data from the source portals directly. It is worth answering that honestly rather than defensively.
The eCourts services portal, the NJDG dashboards and the individual High Court sites are built for human beings. They carry CAPTCHAs, session tokens, per state establishment codes and layouts that change without notice. A scraper against them is not a one week project, it is a standing maintenance obligation across the Supreme Court, twenty five High Courts, hundreds of District Court establishments and a set of tribunals that each do things their own way. The failure mode is worse than a 429, because a broken parser usually returns something rather than nothing, and a silently wrong record is more expensive than a refused request.
There is also a posture question that sits above the engineering. Court records being public is one thing. Bypassing an access control, ignoring a portal's terms of use, or putting sustained automated load on government infrastructure are separate questions, and they are questions for your counsel rather than for your engineering lead. A published rate limit, by contrast, is a term you can read, plan against and comply with. Being told exactly how fast you may go is a feature, because it is the difference between a contract and a risk.
The failure that costs the most
The expensive incident is rarely the 429. It is the job that hit a 429, swallowed the exception, logged nothing useful, and reported success with sixty percent of the records missing. Nobody notices for a fortnight, and by then the gap is in a diligence report. Make partial completion loud. Count what you attempted, count what you wrote, and alert on the difference.
A Working Shape for a Bulk Pull
Putting it together, here is the architecture that survives contact with a real corpus. It is not clever, and that is the point.
A durable queue, not a script
Work items live in a table or a queue with a status. A worker claims an item, does one API call's worth of work, writes the result, and marks it done. Restarting the worker costs one item. Adding a second worker changes nothing except that both draw from the same token bucket.
Slices narrow enough to finish
Partition the corpus by court and by year, or by date range where a court is large. Each slice is a paged keyword search with a cursor. Narrow slices finish inside the 90 second server budget, avoid deep paging entirely, and give you a natural resume point.
Two lanes, two speeds
Cheap deterministic calls in one lane at the full ten per minute. Semantic search and analysis in a slower lane with lower concurrency, its own retry policy, and a hard stop on 403 rather than a backoff.
Observability that answers one question
How far along is it, and what has failed. Attempted, succeeded, retried, dead lettered, and the current cursor per slice. If an engineer has to grep logs to answer that, the job is not operable.
None of this is specific to legal data, but legal data is where getting it wrong is most expensive, because the output is not a dashboard tile. It is a diligence report, a limitation calculation, or a list of matters a partner will rely on. A missing page in a search result set is not a performance problem. It is a professional one.
So treat the limit as the first line of the specification rather than the last obstacle in the way. Ten per minute is a modest budget, and it is entirely enough for almost every product anybody actually wants to build, provided the product is designed by someone who knew the number before they started. The endpoint reference and the current parameters are at the API documentation, and the coverage that budget buys is set out at the API overview.
Build for the budget you actually have
Court data work is throughput constrained by design, because a single search runs against roughly 310 million records drawn from official government portals and an AI search runs models before it retrieves anything. A key on the CourtMesh API is allowed 10 requests per minute, every response tells you what is left in the window, and a 429 tells you exactly how long to wait. Read the endpoint and parameter reference at the API documentation, check what a call costs at API pricing, and design the queue before you write the loop.
Explore CourtMesh
