How exactly do context managers work?

Context managers (PEP 343) are pretty important in Python. You probably use one every time you open a file:

But how well do you understand what’s going on behind the scenes?

Context manager classes

It’s actually quite simple. A context manager is a class that implements an __enter__ and an __exit__ method.

Let’s imagine you want to you print a line of text to the console surrounded with asterisks. Here’s a context manager to do it:

The __exit__ method takes three arguments apart from self. Those arguments contain information about any errors that occurred inside the with block.

You can use asterisks in the same way as any of the built-in context managers:

Accessing the context inside the with block

If you need to get something back and use it inside the with block – such as a file descriptor – you simply return it from __enter__:

myopen works identically to the built-in open:

The contextmanager decorator

Thankfully, you don’t have to implement a class every time. The contextlib package has a contextmanager decorator that you can apply to generators to automatically transform them into context managers:

The code before yield corresponds to __enter__ and the code after yield corresponds to __exit__. A context manager generator should have exactly one yield in it.

It works the same as the class version:

Roll your own contextmanager decorator

The implementation in contextlib is complicated, but it’s not hard to write something that works similarly with the exception of a few edge cases:

It’s not as robust as the real implementation, but it should be understandable. Here are the key points:

  • The inner function instantiates a copy of the nested CMWrapper class with a handle on the generator passed into the decorator.
  • __enter__ calls next() on the generator and returns the yielded value so it can be used in the with block.
  • __exit__ calls next() again and catches the StopIteration exception that the generator throws when it finishes.

That’s it for now. If you want to learn more about context managers, I recommend you take a look at the code for contextlib.

Download Mastering Decorators

Mastering_decorators_cover

Enjoyed this article? Join the newsletter and get Mastering Decorators - a gentle 22-page introduction to one of the trickiest parts of Python.

Weekly-ish. No spam. Unsubscribe any time. Powered by ConvertKit