
Library Microservice
Books, customers and orders as separate services, run with Docker Compose.
I'm Vaibhav. I build services and APIs in Go, Java and Python.
Tap the badge to flip it
My contest rating over time and a year of daily practice. See my profile
A few things I've built.

Books, customers and orders as separate services, run with Docker Compose.



The ones that placed.
A blockchain document suite on IPFS.
A blog recommender built in 48 hours.
Product recommendations; I built the frontend.
Notes on backend engineering, distributed systems, Go, Java and problem solving.
Thanks for reading. Questions or corrections are welcome on LinkedIn.
Every public API eventually meets a client that sends far more requests than it should. Sometimes it's a bug in a retry loop, sometimes a scraper, sometimes a perfectly legitimate customer having a big day. A rate limiter decides which of those requests you serve and which get a 429 Too Many Requests, so one noisy caller can't take the service down for everyone else.
There are four classic algorithms: fixed window, sliding log, sliding window counter and token bucket. In practice two of them cover almost everything, so this post builds both in Go and then looks at what changes when you run more than one server.
Picture a bucket that holds up to capacity tokens and refills at rate tokens per second. Each request takes one token. If the bucket is empty, the request is rejected. A client that has been quiet can burst up to the bucket's capacity, but over time it can't exceed the refill rate.
You don't need a background goroutine to refill. Store the time of the last update and add the tokens that would have accumulated since then whenever a request arrives:
This is the same idea behind golang.org/x/time/rate. In real code, use that package: rate.NewLimiter(rate.Limit(10), 20) gives you 10 requests per second with bursts of 20, plus Wait and Reserve for callers that would rather queue than fail.
A fixed window ("100 requests per minute, reset at :00") is simple, but it lets a client send 100 requests at 12:00:59 and another 100 at 12:01:00: 200 in two seconds. A sliding log fixes that by storing every timestamp, which costs memory per request.
The sliding window counter is the compromise. Keep counts for the current and previous fixed windows, and estimate the rolling count by weighting the previous window by how much of it still overlaps:
It's an approximation because it assumes the previous window's requests were evenly spread, but it uses two integers per client and the error is small in practice.
You almost always want a limiter per client (per API key, user or IP), not one global limiter. A minimal middleware:
Two things to fix before production. First, the map grows forever, so evict idle clients (a TTL cache or a periodic sweep works). Second, behind a load balancer RemoteAddr is the balancer, not the client. Key on the API key or a trusted forwarded header instead.
In-memory limiters are per instance. With 10 instances behind a round-robin balancer, a "100 per minute" limit quietly becomes about 1,000. The usual options:
INCR a key like rl:{client}:{minute} and set an expiry. Do both atomically (a Lua script or MULTI), otherwise a crash between the two leaves a key that never expires.| If you need | Use |
|---|---|
| Bursty traffic with a capped average, like most APIs | Token bucket |
| A strict "N per window" quota that's easy to explain to customers | Sliding window counter |
| Exact counts and low volume | Sliding log |
Whichever you pick, send a Retry-After header with the 429. Well-behaved clients will back off, and the badly behaved ones were going to hit the limit anyway.
A client sends POST /payments. The server charges the card, writes to the database and starts sending the response, and then the connection drops. The client never sees a reply. Did the payment go through?
The client can't know. If it retries, it might charge the customer twice. If it doesn't, the order might never be paid. Networks fail like this all the time, so any API that moves money, books seats or sends messages needs an answer. The standard one is the idempotency key.
An operation is idempotent if doing it twice has the same effect as doing it once. GET, PUT and DELETE are meant to be idempotent; POST usually isn't. Idempotency keys make a POST behave idempotently:
Idempotency-Key: 6f1c….Stripe popularised this pattern, and there's now an IETF draft standardising the Idempotency-Key header.
All you need is a table with a unique constraint on the key:
Storing a hash of the request body lets you catch a client that reuses a key for a different request, which is always a bug on their side.
The trick is to claim the key before doing any work, using the database's unique constraint as the lock:
Each branch maps to a clear outcome:
| Situation | Response |
|---|---|
| New key | Run the operation, store and return the result |
| Same key, same body, finished | Return the stored response, no new work |
| Same key, same body, still running | 409 Conflict: tell the client to retry shortly |
| Same key, different body | 422: the client reused a key by mistake |
Crashes mid-operation. If the server dies after claiming the key but before finishing, the row stays in_progress forever. Add a locked_at timestamp and let a later request take over a key that has been stuck longer than your timeout.
Side effects outside your database. If the operation calls a payment processor, your table alone can't make that call idempotent. Pass an idempotency key downstream too, so the processor dedupes on its side. Idempotency has to hold at every hop, not just the first.
Which responses to store. Store successes and client errors (4xx), because retrying them gives the same answer. Don't permanently store 5xx results caused by transient failures, or a temporary outage turns into a permanent failure for that key.
Expiry. Keys don't need to live forever, just longer than any reasonable retry window. A day is a common choice. Clean old rows up with a scheduled job.
Retries are not optional in distributed systems. Timeouts, dropped connections and redeploys guarantee that some requests will be sent twice. Idempotency keys turn "did that go through?" from a support ticket into a non-event, and they cost one table and about fifty lines of code.
After 1,000+ problems, the biggest change isn't that I know more algorithms. It's that I read the problem statement differently. Most problems announce their pattern in the first two sentences if you know what to listen for. This post is that list of signals.
| The problem says | Reach for |
|---|---|
| Sorted array, find a pair or triplet | Two pointers |
| Longest or shortest contiguous subarray or substring that satisfies X | Sliding window |
| Count subarrays whose sum equals k | Prefix sums and a hash map |
| "Minimum possible maximum", or "smallest X such that it's possible" | Binary search on the answer |
| Next greater or smaller element | Monotonic stack |
| Top k, kth largest, merge k sorted lists | Heap |
| Shortest path, unweighted | BFS (Dijkstra if weights are non-negative) |
| Groups, connected components, "are these linked?" | Union-find or DFS |
| Tasks with prerequisites | Topological sort |
| All combinations, permutations or subsets | Backtracking |
| Count the ways, or min/max cost, with choices at each step | Dynamic programming |
| Overlapping intervals | Sort by start, then sweep |
The table won't solve the problem for you, but it turns a blank page into a short list of things to try.
Grow the window on the right, and shrink it from the left when it breaks the rule. Every index enters and leaves once, so it's O(n). Here it is on "longest substring without repeating characters":
This is the most underrated pattern. If you can check "is answer X good enough?" quickly, and the answer is monotonic (if X works, X+1 also works), you can binary search over the answers instead of the input. Koko eating bananas is the classic example:
Once you see it, it shows up everywhere: shipping packages within D days, splitting an array to minimise the largest sum, placing routers as far apart as possible.
Patterns get you to a working solution. Talking clearly about trade-offs is what turns that into a great one.
Say you run four cache servers and decide which one holds a key with hash(key) % 4. It works perfectly until traffic grows and you add a fifth server. Now every key is routed with % 5, and most of them land somewhere new. Your cache hit rate falls off a cliff and the database behind it takes the full load at the worst possible moment.
Consistent hashing is the fix, and it's one of those ideas that shows up everywhere once you know it: cache clients, Cassandra's token ring, load balancers like Envoy's ring hash, and sharded databases.
I hashed 100,000 keys (user:0 to user:99999) with CRC32 and counted how many changed server when going from 4 servers to 5:
| Strategy | Keys that moved |
|---|---|
hash % N | 80.0% |
| Consistent hashing, 1 point per server | 98.0% |
| Consistent hashing, 150 virtual nodes per server | 21.5% |
The ideal is 20%: the new server should take its fair fifth and nothing else should move. The middle row is the interesting one, and we'll come back to it.
Imagine the hash space (0 to 232−1) bent into a circle. Hash each server onto the circle. To find a key's server, hash the key and walk clockwise until you hit a server.
When you add a server, it lands at one spot on the ring and takes over only the keys between it and the previous server. Everything else stays put. Removing a server is the mirror image: its keys move to the next server clockwise.
With one point per server, the arcs between servers are random lengths, so some servers own huge slices of the ring and others almost nothing. In my test, with short, similar server names, one server ended up owning about 98% of the keys. That's also why the 1-point version moved almost everything: the new server happened to land in the giant arc.
The fix is to put each server on the ring many times, as cache-0#0, cache-0#1 and so on. With 150 virtual nodes per server, each of the 5 servers held between 18.9% and 21.5% of the keys. More points means smoother balance, at the cost of a bigger sorted array.
A complete ring in Go is about 30 lines. Lookups are a binary search over the sorted virtual node hashes:
Usage is what you'd expect:
The takeaway: if you shard anything by key and the number of shards can change, don't use % N. Use a ring with virtual nodes, or a library that does.
Retries feel like free reliability. A call fails, you try again, it works. But a retry is also extra load, sent at exactly the moment a service is struggling. Done carelessly, retries turn a brief slowdown into a full outage. Done well, they're invisible.
This post covers the three pieces that make retries safe: exponential backoff, jitter, and circuit breakers.
Only retry errors that might succeed next time: timeouts, connection resets, 503, 429. Never retry a 400 or a validation error; it will fail identically forever.
And only retry operations that are safe to repeat. Reads are fine. Writes need an idempotency key, or a retry can charge someone twice.
Retrying immediately just adds load to a system that's already failing. Instead, wait longer after each failure: 100ms, 200ms, 400ms, 800ms, up to a cap. That gives the downstream service room to recover.
Backoff alone has a problem. If 10,000 clients all failed at the same moment, they all retry at 100ms, then all at 200ms, then all at 400ms. You've built a synchronised wave that hits the recovering service at each step.
Jitter adds randomness so retries spread out. The simplest strong version, often called "full jitter" after an AWS Architecture Blog post, sleeps a random time between zero and the backoff value:
The context check matters. If the caller has given up, stop retrying immediately instead of burning work nobody will see.
Here's the trap in microservices. Service A calls B, B calls C, C calls the database. If every layer retries 3 times, one failing database call can turn into 3 × 3 × 3 = 27 attempts at the bottom. Under load, retries multiply exactly where the system is weakest.
Two defences:
If a dependency is clearly down, every request still waits for a timeout before failing. That ties up threads and connections, and the slowness spreads upstream. A circuit breaker notices repeated failures and fails fast instead:
This version is deliberately small: in half-open state several concurrent requests can slip through, and it counts consecutive failures rather than a failure rate over a time window. For production, a library like sony/gobreaker handles those details.
| Tool | Protects against |
|---|---|
| Timeouts | Waiting forever on a slow call |
| Retries with backoff and jitter | Brief, transient failures |
| Retry budget | Retries multiplying across layers |
| Circuit breaker | Hammering a dependency that's down |
| Idempotency keys | Retries causing duplicate side effects |
None of these is complicated on its own. Reliability comes from using all of them together, so that a struggling dependency gets less traffic, not more.
Go makes concurrency so cheap that the first instinct is to start a goroutine per item. For 10 items that's fine. For 100,000 database rows it means 100,000 open connections, a rate-limited API returning 429s, and possibly an out-of-memory crash. The fix is bounded concurrency: process many items, but only N at a time.
If each task can fail and you want to stop everything on the first error, golang.org/x/sync/errgroup does almost all the work:
Three things happen for free here. SetLimit(8) makes g.Go block once 8 tasks are running, so you never have more than 8 in flight. WithContext cancels the context when any task fails, so the others can stop early. And g.Wait() returns the first error.
Writing into results[i] from several goroutines is safe because each goroutine writes a different index, and it keeps the output in input order.
When work arrives as a stream rather than a slice, or workers need long-lived state like a connection, use a fixed set of workers reading from a channel:
The goroutine at the end is the detail people forget. Someone has to close results once every worker is done, or the consumer ranging over it waits forever.
for _, x := range items { go work(x) } with no limit. Fine in a demo, an incident in production.ctx.Done() case, as in the pool above.id := id).process doesn't take and respect ctx, cancelling the group does nothing for work already running.For CPU-bound work, start near runtime.NumCPU(). For I/O-bound work, the limit is usually set by whatever you're calling: database pool size, an API's rate limit, or a downstream service's capacity. Make it configurable, measure, and adjust. The right number is rarely the first guess.
I led a Java 8 to 17 upgrade at Barclays. Plenty of codebases are still on 8, and the jump looks scarier than it is. Most of the breakage falls into a handful of predictable buckets. Here's the checklist I'd use again.
Java 17 is a long-term support release with nine years of improvements on top of 8: records, switch expressions, text blocks, var, sealed classes, NullPointerException messages that name the null variable, and better garbage collectors. Newer frameworks also require it; Spring Boot 3 needs Java 17 as a minimum.
The JDK ships a tool for exactly this:
It lists every class that depends on JDK internals such as sun.misc and suggests replacements. Since Java 17, these internals are strongly encapsulated and the old --illegal-access escape hatch is gone, so this list is your to-do list.
Java 11 removed the Java EE and CORBA modules from the JDK. Code that compiled fine on 8 suddenly can't find javax.xml.bind (JAXB), javax.annotation or the JAX-WS classes. They still exist; they're just separate dependencies now:
Old versions of the tools around your code break before your code does:
| Tool | What to check |
|---|---|
| Maven | Compiler plugin with <release>17</release> |
| Gradle | 7.3 or later to run on Java 17 |
| Lombok | 1.18.22 or later |
| JaCoCo | 0.8.8 or later; older versions can't read Java 17 class files |
| Mockito, ByteBuddy | Recent versions; old ones can't instrument Java 17 classes |
| Spring Boot | A 2.x release that supports Java 17 (2.5.5+), or 3.x |
Startup scripts are a quiet source of failures. Java 9 replaced the old GC logging flags with unified logging, and some old flags now stop the JVM from starting at all. -XX:+PrintGCDateStamps is the classic example; replace the old set with -Xlog:gc*. The CMS collector was removed in Java 14, and G1 has been the default since Java 9, so check any tuning you did for the old defaults.
ScriptEngine for JS needs a replacement.InaccessibleObjectException. Upgrade the library; use --add-opens only as a last, documented resort.The single biggest factor in how smooth this goes is test coverage. On the Barclays project we also pushed the JUnit suite to 100% coverage, and that's what makes an upgrade like this safe: a compatibility break shows up as a failing test, not a production incident. If your coverage is thin, write tests around the risky areas (serialisation, reflection, TLS calls, date handling) before you switch.
jdeps and list everything that needs changing.Done in that order, each step is small, testable and easy to roll back.