Python `with` Statement

By: Cam Wohlfeil
Published: 2018-09-03 2330 EDT
Category: Programming
Tags: python

with, and by extension context managers, are one of the things about Python that I've used but never really understood. At my current job it's used quite frequently for an API I have to access, and I'm also trying to square where it fits in relation to decorators, so it's time for me to understand it. The Python docs were surprisingly helpful in trying to understand, despite their usual verbosity.

with is used to properly handle resources. Rather than having a try-catch-finally block every single time you access a resource to make sure it's properly closed, this common pattern was added in to the language. Without this there's no guarantee some error, exception, or poor programming will prevent your resource from being properly closed. The syntax is very straightforward:

with something_that_returns_a_context_manager() as my_resource:
    do_something(my_resource)

with almost acts like a function, allowing you to assign a variable to the resource with as and cleaning up its context and variables once it is out of scope. Of course the ability to use with requires whatever you are calling to return a context manager, but I don't think I've run in to anything yet that doesn't do so where ot makes sense.

with statements can also be nested:

with A() as a, B() as b:
    do_something(a, b)

# is equivalent to

with A() as a:
    with B() as b:
        do_something(a, b)

Beyond handling files and connections, context managers are useful for handling mutexes in multi-threaded programs:

from threading import Lock
lock = Lock()

def do_something_dangerous():
    with lock:
        raise Exception('oops I forgot this code could raise exceptions')

try:
    do_something_dangerous()
except:
    print('Got an exception')
lock.acquire()
print('Got here')

No need to worry about deadlocks since with guarantees the lock will be released once it goes out of scope, and you'd have to try really hard to get around this. You can use the contextlib standard library module to make your own context managers. The easiest way to do this is to use the @contextmanager decorator, which must be used on a generator function. Everything before yield is the code for the __enter__() dunder method, everything after is the code for the __exit__() dunder method.

from contextlib import contextmanager

@contextmanager
def open_file(path, mode):
    file = open(path, mode)
    yield file
    file.close()

References

Python with Statement

Python `with` Statement