Is there a decorator to simply cache function return values

Question

Consider the following    property def name self        if not hasattr self    name               expensive calculation         self  name   1   1      return self  name   I m new  but I think the caching could be factored out into a decorator  Only I didn t find one like it     PS the real calculation doesn t depend on mutable values

User · Answer

class memorize dict       def   init   self  func           self func   func      def   call   self   args           return self args       def   missing   self  key           result   self key    self func  key          return result   Sample uses    gt  gt  gt   memorize     def foo a  b           return a   b  gt  gt  gt  foo 2  4  8  gt  gt  gt  foo   2  4   8   gt  gt  gt  foo  hi   3   hihihi   gt  gt  gt  foo   2  4   8    hi   3    hihihi

User · Answer

functools cache is released in Python 3 9  docs   from functools import cache   cache def factorial n       return n   factorial n-1  if n else 1  In previous versions  one of the early answers is still a valid solution using lru cache as an ordinary cache without limit and lru feature   docs   If maxsize is set to None  the LRU feature is disabled and the cache can grow without bound   Here is a prettier version of it  cache   lru cache maxsize None    cache def func param1      pass

User · Answer

Python 3 8 functools cached property decorator  https   docs python org dev library functools html functools cached property  cached property from Werkzeug was mentioned at  https   stackoverflow com a 5295190 895245 but a supposedly derived version will be merged into 3 8  which is awesome   This decorator can be seen as caching  property  or as a cleaner   functools lru cache for when you don t have any arguments   The docs say     functools cached property func        Transform a method of a class into a property whose value is computed once and then cached as a normal attribute for the life of the instance  Similar to property    with the addition of caching  Useful for expensive computed properties of instances that are otherwise effectively immutable       Example   class DataSet      def   init   self  sequence of numbers           self  data   sequence of numbers       cached property     def stdev self           return statistics stdev self  data        cached property     def variance self           return statistics variance self  data        New in version 3 8       Note This decorator requires that the dict attribute on each instance be a mutable mapping  This means it will not work with some types  such as metaclasses  since the dict attributes on type instances are read-only proxies for the class namespace   and those that specify slots without including dict as one of the defined slots  as such classes don   t provide a dict attribute at all

User · Answer

Try joblib http   pythonhosted org joblib memory html  from joblib import Memory memory   Memory cachedir cachedir  verbose 0   memory cache     def f x           print  Running f  s     x          return x

User · Answer

If you are using Django Framework  it has such a property to cache a view or response of API s using  cache page time  and there can be other options as well   Example    cache page 60   15  cache  special cache   def my view request             More details can be found here

User · Answer

It sounds like you re not asking for a general-purpose memoization decorator  i e   you re not interested in the general case where you want to cache return values for different argument values    That is  you d like to have this   x   obj name    expensive y   obj name    cheap   while a general-purpose memoization decorator would give you this   x   obj name      expensive y   obj name      cheap   I submit that the method-call syntax is better style  because it suggests the possibility of expensive computation while the property syntax suggests a quick lookup    Update  The class-based memoization decorator I had linked to and quoted here previously doesn t work for methods   I ve replaced it with a decorator function    If you re willing to use a general-purpose memoization decorator  here s a simple one   def memoize function     memo        def wrapper  args       if args in memo        return memo args      else        rv   function  args        memo args    rv       return rv   return wrapper   Example usage    memoize def fibonacci n     if n  lt  2  return n   return fibonacci n - 1    fibonacci n - 2    Another memoization decorator with a limit on the cache size can be found here

User · Answer

Starting from Python 3 2 there is a built-in decorator    functools lru cache maxsize 100  typed False    Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls  It can save time when an expensive or I O bound function is periodically called with the same arguments    Example of an LRU cache for computing Fibonacci numbers    lru cache maxsize None  def fib n       if n  lt  2          return n     return fib n-1    fib n-2    gt  gt  gt  print  fib n  for n in range 16     0  1  1  2  3  5  8  13  21  34  55  89  144  233  377  610    gt  gt  gt  print fib cache info    CacheInfo hits 28  misses 16  maxsize None  currsize 16      If you are stuck with Python 2 x  here s a list of other compatible memoization libraries    functools32   PyPI   Source code repoze lru   PyPI   Source code pylru   PyPI   Source code backports functools lru cache   PyPI   Source code

User · Answer

I implemented something like this  using pickle for persistance and using sha1 for short almost-certainly-unique IDs  Basically the cache hashed the code of the function and the hist of arguments to get a sha1 then looked for a file with that sha1 in the name  If it existed  it opened it and returned the result  if not  it calls the function and saves the result  optionally only saving if it took a certain amount of time to process    That said  I d swear I found an existing module that did this and find myself here trying to find that module    The closest I can find is this  which looks about right  http   chase-seibert github io blog 2011 11 23 pythondjango-disk-based-caching-decorator html  The only problem I see with that is it wouldn t work well for large inputs since it hashes str arg   which isn t unique for giant arrays   It would be nice if there were a unique hash   protocol that had a class return a secure hash of its contents  I basically manually implemented that for the types I cared about

User · Answer

Werkzeug has a cached property decorator  docs  source

User · Answer

There is yet another example of a memoize decorator at Python Wiki   http   wiki python org moin PythonDecoratorLibrary Memoize  That example is a bit smart  because it won t cache the results if the parameters are mutable   check that code  it s very simple and interesting

User · Answer

DISCLAIMER  I m the author of kids cache   You should check kids cache  it provides a  cache decorator that works on python 2 and python 3  No dependencies   100 lines of code  It s very straightforward to use  for instance  with your code in mind  you could use it like this   pip install kids cache   Then  from kids cache import cache     class MyClass object                cache               lt -- That s all you need to do      property     def name self           return 1   1    supposedly expensive calculation   Or you could put the  cache decorator after the  property  same result    Using cache on a property is called lazy evaluation  kids cache can do much more  it works on function with any arguments  properties  any type of methods  and even classes      For advanced users  kids cache supports cachetools which provides fancy cache stores to python 2 and python 3  LRU  LFU  TTL  RR cache    IMPORTANT NOTE  the default cache store of kids cache is a standard dict  which is not recommended for long running program with ever different queries as it would lead to an ever growing caching store  For this usage you can plugin other cache stores using for instance   cache use cachetools LRUCache maxsize 2   to decorate your function property class method

User · Answer

lru cache is not perfect with default function values  my mem decorator   import inspect   def get default args f       signature   inspect signature f      return           k  v default         for k  v in signature parameters items           if v default is not inspect Parameter empty         def full kwargs f  kwargs       res   dict get default args f       res update kwargs      return res   def mem func       cache   dict        def wrapper  args    kwargs           kwargs   full kwargs func  kwargs          key   list args          key extend kwargs values            key   hash tuple key           if key in cache              return cache key          else              res   func  args    kwargs              cache key    res             return res     return wrapper   and code for testing   from time import sleep    mem def count a   x  z 10       sleep 2      x   list x      x append z      x append a      return sum x    def main        print count 1 2 3 4 5       print count 1 2 3 4 5       print count 1 2 3 4 5  z 6       print count 1 2 3 4 5  z 6       print count 1       print count 1  z 10     if   name         main         main     result - only 3 times with sleep   but with  lru cache it will be 4 times  because this   print count 1   print count 1  z 10     will be calculated twice  bad working with defaults

User · Answer

Along with the Memoize Example I found the following python packages    cachepy  It allows to set up ttl and or the number of calls for cached functions  Also  one can use encrypted file-based cache    percache

User · Answer

If you are using Django and want to cache views  see Nikhil Kumar s answer      But if you want to cache ANY function results  you can use django-cache-utils   It reuses Django caches and provides easy to use cached decorator   from cache utils decorators import cached   cached 60  def foo x  y 0       print  foo is called      return x y

User · Answer

Ah  just needed to find the right name for this   Lazy property evaluation    I do this a lot too  maybe I ll use that recipe in my code sometime

User · Answer

There is fastcache  which is  C implementation of Python 3 functools lru cache  Provides speedup of 10-30x over standard library    Same as chosen answer  just different import   from fastcache import lru cache  lru cache maxsize 128  typed False  def f a  b       pass   Also  it comes installed in Anaconda  unlike functools which needs to be installed

User · Answer

I coded this simple decorator class to cache function responses  I find it VERY useful for my projects   from datetime import datetime  timedelta   class cached object       def   init   self   args    kwargs           self cached function responses              self default max age   kwargs get  default cache max age   timedelta seconds 0        def   call   self  func           def inner  args    kwargs               max age   kwargs get  max age   self default max age              if not max age or func not in self cached function responses or  datetime now   - self cached function responses func   fetch time    gt  max age                   if  max age  in kwargs  del kwargs  max age                   res   func  args    kwargs                  self cached function responses func      data   res   fetch time   datetime now                return self cached function responses func   data           return inner   The usage is straightforward   import time   cached def myfunc a       print  in func      return  a  datetime now      cached default max age   timedelta seconds 6   def cacheable test a       print  in cacheable test        return  a  datetime now      print cacheable test 1 max age timedelta seconds 5   print cacheable test 2 max age timedelta seconds 5   time sleep 7  print cacheable test 3 max age timedelta seconds 5

[python] Is there a decorator to simply cache function return values?

Examples related to python

Examples related to caching

Examples related to decorator

Examples related to memoization