ASP.NET: Caching and cache locking for a high traffic website migration

Here we tell the story of migrating a major client's website over to new servers. Much faster servers, which by the end of the project reduced average page server time from 600ms to 60ms.

The challenge

During testing, we kept seeing CPU spikes, database locks, and eventually lots of timeouts. It turned out that the cause was the way the ASP.NET cache was implemented.

The implementation (similar to most implementations) was similar to this:

[code lang=csharp]
public static class CacheManager
{
public static ExpensiveObject GetExpensiveObject()
{
var cache = HttpContext.Current.Cache;
var obj = cache[THE_KEY] as ExpensiveObject;

if (obj == null)
{
obj = GetItFromSource();
}

return obj;
}
}
[/code]

The problem was that if there ever was a database slowdown, a long query, table lock, etc, thousands of requests per second would hit this method. Because the first request was still trying to get the expensive object from source, it wasn't in the cache yet. Which means every other request was trying to get it from source.

The result? A tiny blip in DB performance turned into a much larger problem of 503 errors, servers started sweating, and everyone got mad.

The solution

At the root of the issue is the fact that we want all other requests to wait nicely in line while the first one gets the expensive object from source, instead of all trying to pummel the database at the same time. So we needed to somehow lock the cache.

However, locking is fraught with dangers. With high traffic and thousands of concurrent website hits, we had to get this right.

After extensive research and testing, this was our solution:

[code language=csharp]

public static class CacheManager {
private static ConcurrentDictionary < string, object > cacheLocks =
new ConcurrentDictionary < string, object > ();

public static ExpensiveObject GetExpensiveObject() {
var cache = HttpContext.Current.Cache;
var obj = cache[THE_KEY] as ExpensiveObject;

if (obj == null) {
//get a lock
var lockObject = GetCacheLock(THE_KEY);
//check again
if (obj == null) {
lock(lockObject) {
obj = GetItFromSource();
}
}
}

return obj;
}

private static object GetCacheLock(string cacheKey) {
var lockObject = cacheLocks.GetOrAdd(cacheKey,
o => new object());
return lockObject;
}
}

[/code]

What can we learn?

The finished code looked a little different, but the sample shows the main points:

• You should use your own objects to lock, not lock the whole cache or lock(this)
• You should lock at the lowest level possible, in this case we use the cache key. Any higher than this will damage scalability
• You should always use the check-lock-check pattern, in case another thread put it in the cache in the time it took for the lock to be established.

Need help with your website migration?

We have vast experience of migrating websites, from small scale to high traffic websites. 

We're a software development agency specialising in Umbraco, an ASP.NET Content Management System (CMS). Need help migrating your website to Umbraco from Wordpress, or migrating to Umbraco 8?

As an Umbraco registered partner, we can help. Let's talk.