Locking pattern for proper use of NET MemoryCache

Question

I assume this code has concurrency issues   const string CacheKey    CacheKey   static string GetCachedData         string expensiveString  null      if  MemoryCache Default Contains CacheKey                 expensiveString   MemoryCache Default CacheKey  as string            else               CacheItemPolicy cip   new CacheItemPolicy                         AbsoluteExpiration   new DateTimeOffset DateTime Now AddMinutes 20                      expensiveString   SomeHeavyAndExpensiveCalculation            MemoryCache Default Set CacheKey  expensiveString  cip             return expensiveString      The reason for the concurrency issue is that multiple threads can get a null key and then attempt to  insert data into cache   What would be the shortest and cleanest way to make this code concurrency proof  I like to follow a good pattern across my cache related code  A link to an online article would be a great help   UPDATE   I came up with this code based on  Scott Chamberlain s answer  Can anyone find any performance or concurrency issue with this  If this works  it would save many line of code and errors    using System  using System Collections Generic  using System Linq  using System Text  using System Threading Tasks  using System Runtime Caching   namespace CachePoc       class Program               static object everoneUseThisLockObject4CacheXYZ   new object            const string CacheXYZ    CacheXYZ           static object everoneUseThisLockObject4CacheABC   new object            const string CacheABC    CacheABC            static void Main string   args                        string xyzData   MemoryCacheHelper GetCachedData lt string gt  CacheXYZ  everoneUseThisLockObject4CacheXYZ  20  SomeHeavyAndExpensiveXYZCalculation               string abcData   MemoryCacheHelper GetCachedData lt string gt  CacheABC  everoneUseThisLockObject4CacheXYZ  20  SomeHeavyAndExpensiveXYZCalculation                      private static string SomeHeavyAndExpensiveXYZCalculation    return  Expensive            private static string SomeHeavyAndExpensiveABCCalculation    return  Expensive             public static class MemoryCacheHelper                       public static T GetCachedData lt T gt  string cacheKey  object cacheLock  int cacheTimePolicyMinutes  Func lt T gt  GetData                  where T   class                                 Returns null if the string does not exist  prevents a race condition where the cache invalidates between the contains check and the retreival                  T cachedData   MemoryCache Default Get cacheKey  null  as T                   if  cachedData    null                                        return cachedData                                     lock  cacheLock                                          Check to see if anyone wrote to the cache while we where waiting our turn to write the new value                      cachedData   MemoryCache Default Get cacheKey  null  as T                       if  cachedData    null                                                return cachedData                                               The value still did not exist so we now write it in to the cache                      CacheItemPolicy cip   new CacheItemPolicy                                                 AbsoluteExpiration   new DateTimeOffset DateTime Now AddMinutes cacheTimePolicyMinutes                                              cachedData   GetData                        MemoryCache Default Set cacheKey  cachedData  cip                       return cachedData

User · Answer

public interface ILazyCacheProvider   IAppCache            lt summary gt          Get data loaded - after allways throw cached result  even when data is older then needed  but very fast           lt  summary gt           lt param name  key  gt  lt  param gt           lt param name  getData  gt  lt  param gt           lt param name  slidingExpiration  gt  lt  param gt           lt typeparam name  T  gt  lt  typeparam gt           lt returns gt  lt  returns gt      T GetOrAddPermanent lt T gt  string key  Func lt T gt  getData  TimeSpan slidingExpiration           lt summary gt      Initialize LazyCache in runtime      lt  summary gt  public class LazzyCacheProvider  CachingService  ILazyCacheProvider       private readonly Logger  logger   LogManager GetLogger  MemCashe        private readonly Hashtable  hash   new Hashtable        private readonly List lt string gt    reloader   new List lt string gt         private readonly ConcurrentDictionary lt string  DateTime gt   lastLoad   new ConcurrentDictionary lt string  DateTime gt             T ILazyCacheProvider GetOrAddPermanent lt T gt  string dataKey  Func lt T gt  getData  TimeSpan slidingExpiration                var currentPrincipal   Thread CurrentPrincipal          if   ObjectCache Contains dataKey   amp  amp    hash Contains dataKey                          hash dataKey    null               logger Debug    dataKey  - first start                 lastLoad dataKey    DateTime Now               hash dataKey      object GetOrAdd dataKey  getData  slidingExpiration   CloneObject                 lastLoad dataKey    DateTime Now              logger Debug    dataKey  - first                      else                       if    ObjectCache Contains dataKey      lastLoad dataKey  AddMinutes slidingExpiration Minutes   lt  DateTime Now   amp  amp   hash dataKey     null                  Task Run      gt                                        if   reloader Contains dataKey   return                      lock   reloader                                                if  ObjectCache Contains dataKey                                                         if  lastLoad dataKey  AddMinutes slidingExpiration Minutes   gt  DateTime Now                                  return                               lastLoad dataKey    DateTime Now                              Remove dataKey                                                      reloader Add dataKey                           Thread CurrentPrincipal   currentPrincipal                           logger Debug    dataKey  - reload start                             hash dataKey      object GetOrAdd dataKey  getData  slidingExpiration   CloneObject                             logger Debug    dataKey  - reload                             reloader Remove dataKey                                                               if   hash dataKey     null  return  T    hash dataKey              logger Debug    dataKey  - dummy start            var data   GetOrAdd dataKey  getData  slidingExpiration            logger Debug    dataKey  - dummy            return  T   object data  CloneObject

User · Answer

It is difficult to choose which one is better  lock or ReaderWriterLockSlim  You need real world statistics of read and write numbers and ratios etc   But if you believe using  lock  is the correct way  Then here is a different solution for different needs  I also include the Allan Xu s solution in the code  Because both can be needed for different needs   Here are the requirements  driving me to this solution    You don t want to or cannot supply the  GetData  function for some reason  Perhaps the  GetData  function is located in some other class with a heavy constructor and you do not want to even create an instance till ensuring it is unescapable   You need to access the same cached data from different locations tiers of the application  And those different locations don t have access to same locker object   You don t have a constant cache key  For example  need of caching some data with the sessionId cache key    Code   using System  using System Runtime Caching  using System Collections Concurrent  using System Collections Generic   namespace CachePoc       class Program               static object everoneUseThisLockObject4CacheXYZ   new object            const string CacheXYZ    CacheXYZ           static object everoneUseThisLockObject4CacheABC   new object            const string CacheABC    CacheABC            static void Main string   args                          Allan Xu s usage             string xyzData   MemoryCacheHelper GetCachedDataOrAdd lt string gt  CacheXYZ  everoneUseThisLockObject4CacheXYZ  20  SomeHeavyAndExpensiveXYZCalculation               string abcData   MemoryCacheHelper GetCachedDataOrAdd lt string gt  CacheABC  everoneUseThisLockObject4CacheXYZ  20  SomeHeavyAndExpensiveXYZCalculation                  My usage             string sessionId   System Web HttpContext Current Session  CurrentUser SessionId   ToString                string yvz   MemoryCacheHelper GetCachedData lt string gt  sessionId               if  string IsNullOrWhiteSpace yvz                                 object locker   MemoryCacheHelper GetLocker sessionId                   lock  locker                                        yvz   MemoryCacheHelper GetCachedData lt string gt  sessionId                       if  string IsNullOrWhiteSpace yvz                                                 DatabaseRepositoryWithHeavyConstructorOverHead dbRepo   new DatabaseRepositoryWithHeavyConstructorOverHead                            yvz   dbRepo GetDataExpensiveDataForSession sessionId                           MemoryCacheHelper AddDataToCache sessionId  yvz  5                                                                             private static string SomeHeavyAndExpensiveXYZCalculation     return  Expensive             private static string SomeHeavyAndExpensiveABCCalculation     return  Expensive              public static class MemoryCacheHelper                         Allan Xu s solution             public static T GetCachedDataOrAdd lt T gt  string cacheKey  object cacheLock  int minutesToExpire  Func lt T gt  GetData  where T   class                                 Returns null if the string does not exist  prevents a race condition where the cache invalidates between the contains check and the retreival                  T cachedData   MemoryCache Default Get cacheKey  null  as T                   if  cachedData    null                      return cachedData                   lock  cacheLock                                          Check to see if anyone wrote to the cache while we where waiting our turn to write the new value                      cachedData   MemoryCache Default Get cacheKey  null  as T                       if  cachedData    null                          return cachedData                       cachedData   GetData                        MemoryCache Default Set cacheKey  cachedData  DateTime Now AddMinutes minutesToExpire                        return cachedData                                                region  My Solution               readonly static ConcurrentDictionary lt string  object gt  Lockers   new ConcurrentDictionary lt string  object gt                 public static object GetLocker string cacheKey                                CleanupLockers                     return Lockers GetOrAdd cacheKey  item   gt   cacheKey  new object                                 public static T GetCachedData lt T gt  string cacheKey  where T   class                               CleanupLockers                     T cachedData   MemoryCache Default Get cacheKey  as T                  return cachedData                             public static void AddDataToCache string cacheKey  object value  int cacheTimePolicyMinutes                                CleanupLockers                     MemoryCache Default Add cacheKey  value  DateTimeOffset Now AddMinutes cacheTimePolicyMinutes                               static DateTimeOffset lastCleanUpTime   DateTimeOffset MinValue              static void CleanupLockers                                 if  DateTimeOffset Now Subtract lastCleanUpTime  TotalMinutes  gt  1                                        lock  Lockers   maybe a better locker is needed                                                try  bypass exceptions                                                       List lt string gt  lockersToRemove   new List lt string gt                                 foreach  var locker in Lockers                                                                if   MemoryCache Default Contains locker Key                                       lockersToRemove Add locker Key                                                              object dummy                              foreach  string lockerKey in lockersToRemove                                  Lockers TryRemove lockerKey  out dummy                                lastCleanUpTime   DateTimeOffset Now                                                    catch  Exception                                                                                                  endregion                      class DatabaseRepositoryWithHeavyConstructorOverHead               internal string GetDataExpensiveDataForSession string sessionId                        return  Expensive data from database

User · Answer

I ve solved this issue by making use of the AddOrGetExisting method on the MemoryCache and the use of Lazy initialization    Essentially  my code looks something like this   static string GetCachedData string key  DateTimeOffset offset        Lazy lt String gt  lazyObject   new Lazy lt String gt       gt  SomeHeavyAndExpensiveCalculationThatReturnsAString         var returnedLazyObject   MemoryCache Default AddOrGetExisting key  lazyObject  offset        if  returnedLazyObject    null         return lazyObject Value      return   Lazy lt String gt   returnedLazyObject  Value      Worst case scenario here is that you create the same Lazy object twice  But that is pretty trivial  The use of AddOrGetExisting guarantees that you ll only ever get one instance of the Lazy object  and so you re also guaranteed to only call the expensive initialization method once

User · Answer

I assume this code has concurrency issues    Actually  it s quite possibly fine  though with a possible improvement   Now  in general the pattern where we have multiple threads setting a shared value on first use  to not lock on the value being obtained and set can be    Disastrous - other code will assume only one instance exists  Disastrous - the code that obtains the instance is not can only tolerate one  or perhaps a certain small number  concurrent operations  Disastrous - the means of storage is not thread-safe  e g  have two threads adding to a dictionary and you can get all sorts of nasty errors   Sub-optimal - the overall performance is worse than if locking had ensured only one thread did the work of obtaining the value  Optimal - the cost of having multiple threads do redundant work is less than the cost of preventing it  especially since that can only happen during a relatively brief period    However  considering here that MemoryCache may evict entries then    If it s disastrous to have more than one instance then MemoryCache is the wrong approach  If you must prevent simultaneous creation  you should do so at the point of creation  MemoryCache is thread-safe in terms of access to that object  so that is not a concern here    Both of these possibilities have to be thought about of course  though the only time having two instances of the same string existing can be a problem is if you re doing very particular optimisations that don t apply here    So  we re left with the possibilities    It is cheaper to avoid the cost of duplicate calls to SomeHeavyAndExpensiveCalculation    It is cheaper not to avoid the cost of duplicate calls to SomeHeavyAndExpensiveCalculation      And working that out can be difficult  indeed  the sort of thing where it s worth profiling rather than assuming you can work it out   It s worth considering here though that most obvious ways of locking on insert will prevent all additions to the cache  including those that are unrelated   This means that if we had 50 threads trying to set 50 different values  then we ll have to make all 50 threads wait on each other  even though they weren t even going to do the same calculation   As such  you re probably better off with the code you have  than with code that avoids the race-condition  and if the race-condition is a problem  you quite likely either need to handle that somewhere else  or need a different caching strategy than one that expels old entries      The one thing I would change is I d replace the call to Set   with one to AddOrGetExisting    From the above it should be clear that it probably isn t necessary  but it would allow the newly obtained item to be collected  reducing overall memory use and allowing a higher ratio of low generation to high generation collections   So yeah  you could use double-locking to prevent concurrency  but either the concurrency isn t actually a problem  or your storing the values in the wrong way  or double-locking on the store would not be the best way to solve it    If you know only one each of a set of strings exists  you can optimise equality comparisons  which is about the only time having two copies of a string can be incorrect rather than just sub-optimal  but you d want to be doing very different types of caching for that to make sense  E g  the sort XmlReader does internally      Quite likely either one that stores indefinitely  or one that makes use of weak references so it will only expel entries if there are no existing uses

User · Answer

Console example of MemoryCache   How to save get simple class objects   Output after launching and pressing  Any key except Esc    Saving to cache  Getting from cache  Some1 Some2      class Some               public String text   get  set             public Some String text                        this text   text                     public override string ToString                         return text                       public static MemoryCache cache   new MemoryCache  cache         public static string cache name    mycache        static void Main string   args                 Some some1   new Some  some1            Some some2   new Some  some2             List lt Some gt  list   new List lt Some gt             list Add some1           list Add some2            do                if  cache Contains cache name                                 Console WriteLine  Getting from cache                     List lt Some gt  list c   cache Get cache name  as List lt Some gt                   foreach  Some s in list c  Console WriteLine s                             else                               Console WriteLine  Saving to cache                     cache Set cache name  list  DateTime Now AddMinutes 10                                                while  Console ReadKey true  Key    ConsoleKey Escape

User · Answer

Somewhat dated question  but maybe still useful  you may take a look at FusionCache    which I recently released  The feature you are looking for is described here  and you can use it like this  const string CacheKey    quot CacheKey quot   static string GetCachedData         return fusionCache GetOrSet          CacheKey              gt  SomeHeavyAndExpensiveCalculation            TimeSpan FromMinutes 20            You may also find some of the other features interesting like fail-safe  advanced timeouts with background factory completion and support for an optional  distributed 2nd level cache  If you will give it a chance please let me know what you think   shameless-plug

User · Answer

There is an open source library  disclaimer  that I wrote   LazyCache that IMO covers your requirement with two lines of code   IAppCache cache   new CachingService    var cachedResults   cache GetOrAdd  CacheKey           gt  SomeHeavyAndExpensiveCalculation       It has built in locking by default so the cacheable method will only execute once per cache miss  and it uses a lambda so you can do  get or add  in one go  It defaults to 20 minutes sliding expiration   There s even a NuGet package

User · Answer

Its a bit late  however    Full implementation        HttpGet      public async Task lt HttpResponseMessage gt  GetPageFromUriOrBody RequestQuery requestQuery                log nameof GetPageFromUriOrBody   nameof requestQuery            var responseResult   await  requestQueryCache GetOrCreate              nameof GetPageFromUriOrBody                requestQuery                x    gt  getPageContent x  Result           return Request CreateResponse System Net HttpStatusCode Accepted  responseResult             static MemoryCacheWithPolicy lt RequestQuery  string gt   requestQueryCache   new MemoryCacheWithPolicy lt RequestQuery  string gt       Here is getPageContent signature   async Task lt string gt  getPageContent RequestQuery requestQuery     And here is the MemoryCacheWithPolicy implementation   public class MemoryCacheWithPolicy lt TParameter  TResult gt        static ILogger  nlogger   new AppLogger   Logger      private MemoryCache  cache   new MemoryCache new MemoryCacheOptions                    Size limit amount  this is actually a memory size limit value          SizeLimit   1024                    lt summary gt          Gets or creates a new memory cache record for a main data         along with parameter data that is assocciated with main main           lt  summary gt           lt param name  key  gt Main data cache memory key  lt  param gt           lt param name  param  gt Parameter model that assocciated to main model  request result   lt  param gt           lt param name  createCacheData  gt A delegate to create a new main data to cache  lt  param gt           lt returns gt  lt  returns gt      public async Task lt TResult gt  GetOrCreate object key  TParameter param  Func lt TParameter  TResult gt  createCacheData                   this key is used for param cache memory          var paramKey   key   nameof param            if    cache TryGetValue key  out TResult cacheEntry                            key is not in the cache  create data through the delegate              cacheEntry   createCacheData param               createMemoryCache key  cacheEntry  paramKey  param                 nlogger Warn   cache is created                       else                          data is chached so far     check if param model is same  or changed               if   cache TryGetValue paramKey  out TParameter cacheParam                                   exception  this case should not happened                             if   cacheParam Equals param                                    request param is changed  create data through the delegate                  cacheEntry   createCacheData param                   createMemoryCache key  cacheEntry  paramKey  param                    nlogger Warn   cache is re-created  param model has been changed                                else                                nlogger Trace   cache is used                                      return await Task FromResult lt TResult gt  cacheEntry             MemoryCacheEntryOptions createMemoryCacheEntryOptions TimeSpan slidingOffset  TimeSpan relativeOffset                   Cache data within  slidingOffset  seconds              request new result after  relativeOffset  seconds          return new MemoryCacheEntryOptions                   Size amount  this is actually an entry count per                 key limit value  not an actual memory size value               SetSize 1                  Priority on removing when reaching size limit  memory pressure               SetPriority CacheItemPriority High                  Keep in cache for this amount of time  reset it if accessed               SetSlidingExpiration slidingOffset                  Remove from cache after this time  regardless of sliding expiration              SetAbsoluteExpiration relativeOffset                        void createMemoryCache object key  TResult cacheEntry  object paramKey  TParameter param                   Cache data within 2 seconds              request new result after 5 seconds          var cacheEntryOptions   createMemoryCacheEntryOptions              TimeSpan FromSeconds 2                TimeSpan FromSeconds 5                Save data in cache           cache Set key  cacheEntry  cacheEntryOptions               Save param in cache           cache Set paramKey  param  cacheEntryOptions             void checkCacheEntry lt T gt  object key  string name                 cache TryGetValue key  out T value            nlogger Fatal  Key   0   Name   1   Value   2    key  name  value             nlogger is just nLog object to trace MemoryCacheWithPolicy behavior  I re-create the memory cache if request object  RequestQuery requestQuery  is changed through the delegate  Func lt TParameter  TResult gt  createCacheData  or re-create when sliding or absolute time reached their limit  Note that everything is async too

User · Answer

This is my 2nd iteration of the code  Because MemoryCache is thread safe you don t need to lock on the initial read  you can just read and if the cache returns null then do the lock check to see if you need to create the string  It greatly simplifies the code   const string CacheKey    CacheKey   static readonly object cacheLock   new object    private static string GetCachedData            Returns null if the string does not exist  prevents a race condition where the cache invalidates between the contains check and the retreival      var cachedString   MemoryCache Default Get CacheKey  null  as string       if  cachedString    null                return cachedString             lock  cacheLock                  Check to see if anyone wrote to the cache while we where waiting our turn to write the new value          cachedString   MemoryCache Default Get CacheKey  null  as string           if  cachedString    null                        return cachedString                       The value still did not exist so we now write it in to the cache          var expensiveString   SomeHeavyAndExpensiveCalculation            CacheItemPolicy cip   new CacheItemPolicy                                                                     AbsoluteExpiration   new DateTimeOffset DateTime Now AddMinutes 20                                            MemoryCache Default Set CacheKey  expensiveString  cip           return expensiveString              EDIT  The below code is unnecessary but I wanted to leave it to show the original method  It may be useful to future visitors who are using a different collection that has thread safe reads but non-thread safe writes  almost all of classes under the System Collections namespace is like that    Here is how I would do it using ReaderWriterLockSlim to protect access  You need to do a kind of  Double Checked Locking  to see if anyone else created the cached item while we where waiting to to take the lock   const string CacheKey    CacheKey   static readonly ReaderWriterLockSlim cacheLock   new ReaderWriterLockSlim    static string GetCachedData           First we do a read lock to see if it already exists  this allows multiple readers at the same time      cacheLock EnterReadLock        try                 Returns null if the string does not exist  prevents a race condition where the cache invalidates between the contains check and the retreival          var cachedString   MemoryCache Default Get CacheKey  null  as string           if  cachedString    null                        return cachedString                      finally               cacheLock ExitReadLock                 Only one UpgradeableReadLock can exist at one time  but it can co-exist with many ReadLocks     cacheLock EnterUpgradeableReadLock        try                 We need to check again to see if the string was created while we where waiting to enter the EnterUpgradeableReadLock         var cachedString   MemoryCache Default Get CacheKey  null  as string           if  cachedString    null                        return cachedString                       The entry still does not exist so we need to create it and enter the write lock         var expensiveString   SomeHeavyAndExpensiveCalculation            cacheLock EnterWriteLock      This will block till all the Readers flush          try                       CacheItemPolicy cip   new CacheItemPolicy                                 AbsoluteExpiration   new DateTimeOffset DateTime Now AddMinutes 20                              MemoryCache Default Set CacheKey  expensiveString  cip               return expensiveString                    finally                        cacheLock ExitWriteLock                        finally               cacheLock ExitUpgradeableReadLock

User · Answer

To avoid the global lock  you can use SingletonCache to implement one lock per key  without exploding memory usage  the lock objects are removed when no longer referenced  and acquire release is thread safe guaranteeing that only 1 instance is ever in use via compare and swap    Using it looks like this   SingletonCache lt string  object gt  keyLocks   new SingletonCache lt string  object gt      const string CacheKey    CacheKey   static string GetCachedData         string expensiveString  null      if  MemoryCache Default Contains CacheKey                 return MemoryCache Default CacheKey  as string                double checked lock     using  var lifetime   keyLocks Acquire url                 lock  lifetime Value                       if  MemoryCache Default Contains CacheKey                              return MemoryCache Default CacheKey  as string                           cacheItemPolicy cip   new CacheItemPolicy                              AbsoluteExpiration   new DateTimeOffset DateTime Now AddMinutes 20                            expensiveString   SomeHeavyAndExpensiveCalculation               MemoryCache Default Set CacheKey  expensiveString  cip              return expensiveString                            Code is here on GitHub  https   github com bitfaster BitFaster Caching   Install-Package BitFaster Caching   There is also an LRU implementation that is lighter weight than MemoryCache  and has several advantages - faster concurrent reads and writes  bounded size  no background thread  internal perf counters etc   disclaimer  I wrote it

[c#] Locking pattern for proper use of .NET MemoryCache

Examples related to c#

Examples related to .net

Examples related to multithreading

Examples related to memorycache