Vasyl’s Dev Notes

TL;DR: A cache trades freshness for speed. In-memory cache is the fastest but lives in one process. Distributed cache (Redis) is shared between servers but needs a network call. TTL decides how stale data can get. Invalidation is the hard part. In .NET 9+, HybridCache gives you both levels and stampede protection in one API.

Why this post

“The page is slow, let’s add a cache” is one of the most common performance fixes. It is also one of the most common sources of strange bugs: users see old prices, one server shows different data than another, and after a deploy the database falls over.

So here is a short refresh, built around one small refactoring.

Before: every request hits the database

public async Task<Product?> GetProductAsync(int id) =>
    await _db.Products.FindAsync(id);

The product page is opened 500 times per second. The product changes maybe once a day. We ask the database the same question 43 million times to get the same answer.

Step 1: in-memory cache (cache-aside)

The basic pattern is cache-aside: look in the cache, if not there, load from the DB and put it in the cache.

builder.Services.AddMemoryCache();

public async Task<Product?> GetProductAsync(int id)
{
    return await _cache.GetOrCreateAsync($"product:{id}", async entry =>
    {
        entry.AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5);
        return await _db.Products.FindAsync(id);
    });
}

Very fast, nanoseconds, no network. But:

• Every server has its own copy. With 4 servers, you have 4 caches. Update a product, and for up to 5 minutes server A shows the new price and server B shows the old one.
• Restart = empty cache. After each deploy, all requests go to the DB at once.
• Memory. The cache lives in your app’s memory. Without a limit, it grows until the GC suffers. Set SizeLimit and give entries a Size.

Step 2: distributed cache (Redis)

Move the cache out of the process, so all servers share it:

builder.Services.AddStackExchangeRedisCache(o =>
    o.Configuration = builder.Configuration.GetConnectionString("Redis"));

Now IDistributedCache stores bytes in Redis. One copy for all servers, survives deploys.

The price: every read is a network call (around a millisecond), and you need to serialize objects. Still much faster than a heavy SQL query, but slower than memory.

Step 3: HybridCache, both levels in one

In .NET 9 and later there is HybridCache (package Microsoft.Extensions.Caching.Hybrid). It keeps a small in-memory cache (L1) in front of the distributed one (L2):

builder.Services.AddStackExchangeRedisCache(...);   // optional L2
builder.Services.AddHybridCache();

public async Task<Product?> GetProductAsync(int id, CancellationToken ct)
{
    return await _cache.GetOrCreateAsync(
        $"product:{id}",
        async token => await _db.Products.FindAsync([id], token),
        new HybridCacheEntryOptions
        {
            Expiration = TimeSpan.FromMinutes(5),            // L2
            LocalCacheExpiration = TimeSpan.FromMinutes(1)   // L1
        },
        tags: ["products"],
        cancellationToken: ct);
}

And it solves a problem that the code in Step 1 has.

The cache stampede

A popular key expires. In the next millisecond, 500 requests miss the cache at the same time. All 500 go to the database with the same query. The database slows down, requests time out, users retry, it gets worse. This is a cache stampede (or “thundering herd”).

IMemoryCache.GetOrCreateAsync does not protect you: every request that misses runs the factory. HybridCache does: for one key, only one caller runs the factory, and the others wait for its result.

Other tricks: add some random jitter to TTLs, so many keys don’t expire at the same second, and warm up important keys after a deploy.

TTL: how stale is OK?

• Absolute expiration: the entry dies 5 minutes after it was created. Predictable. Good default.
• Sliding expiration: the entry lives 5 minutes after the last read. A popular entry can live forever and never refresh. If you use sliding, add an absolute limit too.

Choose the TTL from the business question: “how old can this data be?” Product description: an hour is fine. Stock count on a checkout page: maybe don’t cache at all.

Invalidation: the hard part

“There are only two hard things in computer science: cache invalidation and naming things.”

Options, from simple to complex:

1. Just TTL. Accept that data is up to N minutes old. Simplest, often enough.
2. Remove on write. When you update the product, remove the key.
3. Versioned keys. Put a version in the key: product:42:v7. On change, bump the version. Old entries just expire.

Remove on write looks like this:

await _db.SaveChangesAsync(ct);
await _cache.RemoveAsync($"product:{id}", ct);

With HybridCache you can also remove a whole group by tag: await _cache.RemoveByTagAsync("products", ct);

Two traps here:

• Write the DB first, then remove the key. If you remove first, another request can load the old value back into the cache before your DB write finishes.
• L1 copies on other servers. Removing from Redis doesn’t clear the in-memory copy on the other 3 servers. Keep LocalCacheExpiration short, or use a backplane (like Redis pub/sub) to notify them.

Don’t cache these

• Data that must be exactly right: balances, stock at checkout, permissions.
• Per-user data with a key that doesn’t include the user. cart instead of cart:{userId} = one user sees another user’s cart.
• Things that are already fast. A cache adds code, bugs and memory. Measure first.

Quick comparison

	In-memory	Distributed (Redis)	HybridCache
Speed	fastest	network hop	L1 fast, L2 shared
Shared between servers	no	yes	yes (L2)
Survives deploy	no	yes	yes (L2)
Stampede protection	no	no	yes

The 30-second summary

Caching trades freshness for speed, usually with the cache-aside pattern. In-memory cache is the fastest but each server has its own copy and it is lost on restart. A distributed cache like Redis is shared and survives deploys, but costs a network call. In .NET 9+, HybridCache combines both and prevents cache stampede, where many requests miss at once and all hit the database. TTL decides how stale data can be, absolute is safer than sliding. For invalidation, remove the key after the database write, or use tags or versioned keys, and remember in-memory copies on other servers.

That’s it. If you have a favorite caching bug, tell me in comments. 🙂