Python · Syntax · Advanced

functools: partial, lru_cache, and more

10 tasks

Master the functools module: `partial`, `wraps`, `lru_cache`, `cached_property`, `total_ordering`, and `singledispatch`.

lru_cache, cache, and cached_property in depth

#
**`functools` — the functional programming toolkit** The `functools` module provides higher-order functions — functions that operate on or return other functions. These patterns appear constantly in professional Python code: caching, partial application, composing comparisons. **`@lru_cache` — memoization with a size limit** LRU (Least Recently Used) cache stores the results of function calls. When the same arguments are passed again, the cached result is returned instantly instead of re-computing: ```python from functools import lru_cache @lru_cache(maxsize=128) def fibonacci(n): if n < 2: return n return fibonacci(n - 1) + fibonacci(n - 2) fibonacci(100) # computed once; subsequent calls instant fibonacci.cache_info() # CacheInfo(hits=98, misses=101, maxsize=128, currsize=101) fibonacci.cache_clear() # invalidate the cache ``` The cache key is built from the arguments, so all arguments must be hashable (no lists or dicts). `maxsize=None` means unlimited — equivalent to `@cache`. **`@cache` (Python 3.9+) — unbounded memoization** ```python from functools import cache @cache def count_ways(n, k): if n == 0: return 1 if n < 0 or k == 0: return 0 return count_ways(n - k, k) + count_ways(n, k - 1) ``` `@cache` is simpler and slightly faster than `@lru_cache(maxsize=None)` because it doesn't track LRU order. **`@cached_property` — compute once per instance** A regular `@property` re-runs its getter every time. `@cached_property` runs the getter once and stores the result as an instance attribute: ```python from functools import cached_property class DataSet: def __init__(self, raw): self.raw = raw @cached_property def sorted_data(self): print('computing...') return sorted(self.raw) ds = DataSet([3, 1, 2]) ds.sorted_data # 'computing...' printed ds.sorted_data # nothing printed — cached as ds.__dict__['sorted_data'] ``` Note: `cached_property` only works on instances that have a `__dict__`. Classes using `__slots__` must use `lru_cache` on methods instead.

partial, reduce, compose pattern, cmp_to_key, and @wraps

#
**`partial` — fix some arguments of a function** `functools.partial` creates a new callable with some arguments pre-filled. It's cleaner than writing a lambda for simple cases: ```python from functools import partial def power(base, exp): return base ** exp square = partial(power, exp=2) cube = partial(power, exp=3) square(5) # 25 cube(3) # 27 # Common use: creating specialized versions of general utilities import json dump_pretty = partial(json.dumps, indent=4, sort_keys=True) dump_pretty({'b': 2, 'a': 1}) # '{\n "a": 1,\n "b": 2\n}' ``` **`reduce` — fold a sequence into a single value** `reduce(func, iterable)` applies `func` cumulatively to the elements — `func(func(func(a, b), c), d)` for `[a, b, c, d]`: ```python from functools import reduce from operator import mul, add numbers = [1, 2, 3, 4, 5] product = reduce(mul, numbers) # 1*2*3*4*5 = 120 total = reduce(add, numbers) # 1+2+3+4+5 = 15 # With initial value: reduce(add, [], 0) # 0 (avoids TypeError on empty list) # reduce to build a pipeline: def compose(*funcs): return reduce(lambda f, g: lambda x: g(f(x)), funcs) pipeline = compose(str.strip, str.lower, str.title) pipeline(' hello world ') # 'Hello World' ``` **`cmp_to_key` — adapt an old-style comparison function** Python 3 dropped the `cmp` argument from `sort()`. If you have a comparison function that returns negative/zero/positive, `cmp_to_key` wraps it: ```python from functools import cmp_to_key def compare_versions(a, b): # returns negative if a < b, 0 if equal, positive if a > b av = tuple(int(x) for x in a.split('.')) bv = tuple(int(x) for x in b.split('.')) return (av > bv) - (av < bv) versions = ['1.10.0', '1.9.0', '2.0.0', '1.10.1'] sorted(versions, key=cmp_to_key(compare_versions)) # ['1.9.0', '1.10.0', '1.10.1', '2.0.0'] ``` **`@wraps` — preserving metadata through decorators** When you write a decorator, the wrapper function hides the original's `__name__`, `__doc__`, etc. `@wraps` copies them: ```python from functools import wraps def logged(func): @wraps(func) # copies __name__, __doc__, __annotations__ def wrapper(*args, **kwargs): print(f'calling {func.__name__}') return func(*args, **kwargs) return wrapper @logged def add(a, b): '''Add two numbers.''' return a + b add.__name__ # 'add' (not 'wrapper') add.__doc__ # 'Add two numbers.' ```

total_ordering, singledispatch, and full functools reference

#
**`@total_ordering` — define one comparison, get the rest** If you define `__eq__` and one of `__lt__`, `__le__`, `__gt__`, `__ge__`, `@total_ordering` fills in the remaining comparison methods: ```python from functools import total_ordering @total_ordering class Version: def __init__(self, major, minor): self.major = major self.minor = minor def __eq__(self, other): return (self.major, self.minor) == (other.major, other.minor) def __lt__(self, other): # only need to define __lt__ return (self.major, self.minor) < (other.major, other.minor) v1 = Version(1, 9) v2 = Version(1, 10) v1 < v2 # True v1 > v2 # False — generated by @total_ordering v2 >= v1 # True — generated by @total_ordering sorted([v2, v1]) # [Version(1,9), Version(1,10)] ``` **`@singledispatch` — function overloading by type** `singledispatch` lets you write different implementations of a function that are chosen based on the type of the first argument: ```python from functools import singledispatch @singledispatch def process(value): raise NotImplementedError(f'No handler for {type(value)}') @process.register(int) def _(value): return value * 2 @process.register(str) def _(value): return value.upper() @process.register(list) def _(value): return [process(item) for item in value] process(5) # 10 process('hello') # 'HELLO' process([1, 'a', 2]) # [2, 'A', 4] ``` **Quick reference — all functools** | Function | Purpose | |---|---| | `@lru_cache(maxsize=N)` | Cache results; evicts LRU when full | | `@cache` | Same but unbounded (3.9+) | | `@cached_property` | Compute once per instance | | `partial(func, *args, **kw)` | Pre-fill arguments | | `reduce(func, iterable)` | Fold sequence to one value | | `cmp_to_key(cmp_fn)` | Adapt old comparison to key function | | `@wraps(func)` | Copy metadata through decorators | | `@total_ordering` | Generate comparison methods from eq + one | | `@singledispatch` | Type-based function overloading |
01

functools.partial — pre-fill arguments

#

Use `functools.partial` to create two specialized functions from `power(base, exp)`: `square(n)` which is `power(n, 2)`, and `cube(n)` which is `power(n, 3)`. Also create `add5 = partial(operator.add, 5)` that adds 5 to any number.

import functools
import operator

def power(base: int, exp: int) -> int:
    return base ** exp


# Create square, cube, and add5 using partial


print(square(4))   # 16
print(cube(3))     # 27
print(add5(10))    # 15
Solution
import functools
import operator

def power(base: int, exp: int) -> int:
    return base ** exp

square = functools.partial(power, exp=2)
cube = functools.partial(power, exp=3)
add5 = functools.partial(operator.add, 5)


print(square(4))   # 16
print(cube(3))     # 27
print(add5(10))    # 15
02

lru_cache for memoization

#

Use `@functools.lru_cache(maxsize=None)` to memoize a recursive `fib(n)` function. Print `fib(35)`. Then use `fib.cache_info()` to show how many hits vs misses occurred. Without caching, `fib(35)` would make ~29 million calls.

import functools

# Add lru_cache decorator here
def fib(n: int) -> int:
    if n < 2:
        return n
    return fib(n - 1) + fib(n - 2)


print(fib(35))          # 9227465
print(fib.cache_info()) # CacheInfo(hits=..., misses=36, ...)
Solution
import functools

@functools.lru_cache(maxsize=None)
def fib(n: int) -> int:
    if n < 2:
        return n
    return fib(n - 1) + fib(n - 2)


print(fib(35))          # 9227465
print(fib.cache_info()) # CacheInfo(hits=33, misses=36, maxsize=None, currsize=36)
03

cached_property for expensive attributes

#

Create a class `Circle` with `radius: float`. Add an `area` attribute using `@functools.cached_property` that computes `math.pi * radius ** 2`. Verify that accessing `area` twice returns the same value without recomputing (add a `print('computing...')` inside to see it only prints once).

import math
import functools

class Circle:
    def __init__(self, radius: float) -> None:
        self.radius = radius

    # Add cached_property for area here


c = Circle(5)
print(c.area)   # computing... then 78.539...
print(c.area)   # 78.539... (no 'computing...' this time)
Solution
import math
import functools

class Circle:
    def __init__(self, radius: float) -> None:
        self.radius = radius

    @functools.cached_property
    def area(self) -> float:
        print('computing...')
        return math.pi * self.radius ** 2


c = Circle(5)
print(c.area)   # computing... then 78.539...
print(c.area)   # 78.539... (no 'computing...')
04

total_ordering — fill in comparison methods

#

Use `@functools.total_ordering` to create a `Version` class with `major, minor, patch` fields. Define only `__eq__` and `__lt__`. The decorator will auto-generate `__le__`, `__gt__`, `__ge__`. Test that all comparisons work.

import functools

@functools.total_ordering
class Version:
    def __init__(self, major: int, minor: int, patch: int) -> None:
        self.major = major
        self.minor = minor
        self.patch = patch

    def _tuple(self):
        return (self.major, self.minor, self.patch)

    def __eq__(self, other) -> bool:
        pass

    def __lt__(self, other) -> bool:
        pass


v1 = Version(1, 2, 3)
v2 = Version(1, 3, 0)
print(v1 < v2)   # True
print(v1 > v2)   # False
print(v1 <= v1)  # True
print(v1 == Version(1, 2, 3))  # True
Solution
import functools

@functools.total_ordering
class Version:
    def __init__(self, major: int, minor: int, patch: int) -> None:
        self.major = major
        self.minor = minor
        self.patch = patch

    def _tuple(self):
        return (self.major, self.minor, self.patch)

    def __eq__(self, other) -> bool:
        return self._tuple() == other._tuple()

    def __lt__(self, other) -> bool:
        return self._tuple() < other._tuple()


v1 = Version(1, 2, 3)
v2 = Version(1, 3, 0)
print(v1 < v2)   # True
print(v1 > v2)   # False
print(v1 <= v1)  # True
print(v1 == Version(1, 2, 3))  # True
05

singledispatch — type-based dispatch

#

Use `@functools.singledispatch` to write a `describe(value)` function that has different implementations for `int`, `str`, `list`, and a default for other types. int: `f'Integer: {value}'`, str: `f'String of length {len(value)}'`, list: `f'List with {len(value)} items'`, default: `f'Unknown type: {type(value).__name__}'`.

import functools

@functools.singledispatch
def describe(value) -> str:
    pass  # default implementation

# Register int, str, list implementations here


print(describe(42))          # Integer: 42
print(describe('hello'))     # String of length 5
print(describe([1, 2, 3]))   # List with 3 items
print(describe(3.14))        # Unknown type: float
Solution
import functools

@functools.singledispatch
def describe(value) -> str:
    return f'Unknown type: {type(value).__name__}'

@describe.register(int)
def _(value: int) -> str:
    return f'Integer: {value}'

@describe.register(str)
def _(value: str) -> str:
    return f'String of length {len(value)}'

@describe.register(list)
def _(value: list) -> str:
    return f'List with {len(value)} items'


print(describe(42))          # Integer: 42
print(describe('hello'))     # String of length 5
print(describe([1, 2, 3]))   # List with 3 items
print(describe(3.14))        # Unknown type: float
06

functools.reduce

#

Use `functools.reduce` to implement three operations without built-in functions: `product(nums)` — product of all numbers, `my_max(nums)` — maximum value, `flatten_str(words)` — join words with a space.

import functools

nums = [2, 3, 4, 5]
words = ['Hello', 'world', 'from', 'reduce']

product = functools.reduce(lambda a, b: a * b, nums)
print(product)    # 120

my_max = functools.reduce(lambda a, b: a if a > b else b, nums)
print(my_max)     # 5

flatten_str = functools.reduce(lambda a, b: a + ' ' + b, words)
print(flatten_str)  # Hello world from reduce
Solution
import functools

nums = [2, 3, 4, 5]
words = ['Hello', 'world', 'from', 'reduce']

product = functools.reduce(lambda a, b: a * b, nums)
print(product)    # 120

my_max = functools.reduce(lambda a, b: a if a > b else b, nums)
print(my_max)     # 5

flatten_str = functools.reduce(lambda a, b: a + ' ' + b, words)
print(flatten_str)  # Hello world from reduce
07

functools.wraps in a decorator

#

Write a decorator `log_calls` that prints `'Calling <func_name>'` before calling the wrapped function. Use `@functools.wraps(func)` inside so that the wrapper preserves the original function's `__name__` and `__doc__`. Verify that `decorated.__name__` is the original name, not `'wrapper'`.

import functools

def log_calls(func):
    # Apply @functools.wraps here
    def wrapper(*args, **kwargs):
        print(f'Calling {func.__name__}')
        return func(*args, **kwargs)
    return wrapper


@log_calls
def greet(name: str) -> str:
    '''Say hello.'''
    return f'Hello, {name}!'

print(greet('Alice'))       # Calling greet / Hello, Alice!
print(greet.__name__)       # greet (not 'wrapper')
print(greet.__doc__)        # Say hello.
Solution
import functools

def log_calls(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        print(f'Calling {func.__name__}')
        return func(*args, **kwargs)
    return wrapper


@log_calls
def greet(name: str) -> str:
    '''Say hello.'''
    return f'Hello, {name}!'

print(greet('Alice'))       # Calling greet / Hello, Alice!
print(greet.__name__)       # greet
print(greet.__doc__)        # Say hello.
08

lru_cache with bounded size

#

Write a function `slow_square(n)` that simulates a slow computation (print `f'Computing {n}^2'` then return `n*n`). Apply `@lru_cache(maxsize=3)` so only the 3 most recent results are cached. Call `slow_square` with 1, 2, 3, 1 (cached), 4 (evicts 2), 2 (recomputed).

import functools

# Add @lru_cache(maxsize=3) here
def slow_square(n: int) -> int:
    print(f'Computing {n}^2')
    return n * n


print(slow_square(1))  # Computing 1^2 -> 1
print(slow_square(2))  # Computing 2^2 -> 4
print(slow_square(3))  # Computing 3^2 -> 9
print(slow_square(1))  # cached -> 1
print(slow_square(4))  # Computing 4^2, evicts 2 -> 16
print(slow_square(2))  # Computing 2^2 again -> 4
Solution
import functools

@functools.lru_cache(maxsize=3)
def slow_square(n: int) -> int:
    print(f'Computing {n}^2')
    return n * n


print(slow_square(1))  # Computing
print(slow_square(2))  # Computing
print(slow_square(3))  # Computing
print(slow_square(1))  # cached
print(slow_square(4))  # Computing — evicts 2 (least recently used)
print(slow_square(2))  # Computing again
09

partial as a sort key

#

Use `functools.partial` to create a reusable `sort_by_field(items, field)` function. Then create `sort_by_age = partial(sort_by_field, field='age')` and `sort_by_name = partial(sort_by_field, field='name')`. Apply both to a list of dicts.

import functools

def sort_by_field(items, field):
    return sorted(items, key=lambda x: x[field])

sort_by_age = functools.partial(sort_by_field, field='age')
sort_by_name = functools.partial(sort_by_field, field='name')


people = [
    {'name': 'Charlie', 'age': 30},
    {'name': 'Alice', 'age': 25},
    {'name': 'Bob', 'age': 35},
]

for p in sort_by_age(people):
    print(p['name'], p['age'])
# Alice 25 / Charlie 30 / Bob 35

for p in sort_by_name(people):
    print(p['name'])
# Alice / Bob / Charlie
Solution
import functools

def sort_by_field(items, field):
    return sorted(items, key=lambda x: x[field])

sort_by_age = functools.partial(sort_by_field, field='age')
sort_by_name = functools.partial(sort_by_field, field='name')


people = [
    {'name': 'Charlie', 'age': 30},
    {'name': 'Alice', 'age': 25},
    {'name': 'Bob', 'age': 35},
]

for p in sort_by_age(people):
    print(p['name'], p['age'])

for p in sort_by_name(people):
    print(p['name'])
10

Combining functools tools

#

Combine `@singledispatch` and `@lru_cache` to write a cached `process(value)` function. The `int` implementation returns `value * 2`, the `str` implementation returns `value.upper()`, and the default returns `str(value)`. Cache the str and int implementations with `maxsize=32`.

import functools

@functools.singledispatch
def process(value):
    return str(value)

@process.register(int)
@functools.lru_cache(maxsize=32)
def _(value: int):
    print(f'processing int {value}')
    return value * 2

@process.register(str)
@functools.lru_cache(maxsize=32)
def _(value: str):
    print(f'processing str {value}')
    return value.upper()


print(process(5))        # processing int 5 -> 10
print(process(5))        # cached -> 10
print(process('hello'))  # processing str hello -> HELLO
print(process(3.14))     # default -> 3.14
Solution
import functools

@functools.singledispatch
def process(value):
    return str(value)

@process.register(int)
@functools.lru_cache(maxsize=32)
def _(value: int):
    print(f'processing int {value}')
    return value * 2

@process.register(str)
@functools.lru_cache(maxsize=32)
def _(value: str):
    print(f'processing str {value}')
    return value.upper()


print(process(5))
print(process(5))        # from cache
print(process('hello'))
print(process(3.14))