Python descriptors made simple

Descriptors, introduced in Python 2.2, provide a way to add managed attributes to objects. They are not used much in everyday programming, but it’s important to learn them to understand a lot of the “magic” that happens in the standard library and third-party packages.

The problem

Imagine we are running a bookshop with an inventory management system written in Python. The system contains a class called Book  that captures the author, title and price of physical books.

Our simple Book class works fine for a while, but eventually bad data starts to creep into the system. The system is full of books with negative prices or prices that are too high because of data entry errors. We decide that we want to limit book prices to values between 0 and 100. In addition, the system contains a Magazine class that suffers from the same problem, so we want our solution to be easily reusable.

This tutorial is pretty long. Want a PDF?

Just type in your email address and I'll send a PDF version to your inbox.

Powered by ConvertKit

The descriptor protocol

The descriptor protocol is simply a set of methods a class must implement to qualify as a descriptor. There are three of them:

  • __get__(self, instance, owner)
  • __set__(self, instance, value)
  • __delete__(self, instance)

__get__ accesses a value stored in the object and returns it.

__set__ sets a value stored in the object and returns nothing.

__delete__ deletes a value stored in the object and returns nothing.

Using these methods, we can write a descriptor called Price that limits the value stored in it to between 0 and 100.

A few details in the implementation of Price deserve mentioning.

An instance of a descriptor must be added to a class as a class attribute, not as an instance attribute. Therefore, to store different data for each instance, the descriptor needs to maintain a dictionary that maps instances to instance-specific values. In the implementation of Price, that dictionary is self.values.

A normal Python dictionary stores references to objects it uses as keys. Those references by themselves are enough to prevent the object from being garbage collected. To prevent Book instances from hanging around after we are finished with them, we use the WeakKeyDictionary from the weakref standard module. Once the last strong reference to the instance passes away, the associated key-value pair will be discarded.

Using descriptors

As we saw in the last section, descriptors are linked to classes, not to instances, so to add a descriptor to the Book class, we must add it as a class variable.

The price constraint for books is now enforced.

How descriptors are accessed

So far we’ve managed to implement a working descriptor that manages the price attribute on our Book class, but how it works might not be clear. It all feels a bit too magical, but not to worry. It turns out that descriptor access is quite simple:

  • When we try to evaluate b.price and retrieve the value, Python recognizes that price is a descriptor and calls Book.price.__get__.
  • When we try to change the value of the price attribute, e.g. b.price = 23 , Python again recognizes that price is a descriptor and substitutes the assignment with a call to Book.price.__set__.
  • And when we try to delete the price attribute stored against an instance of Book, Python automatically interprets that as a call to Book.price.__delete__.

The number 1 descriptor gotcha

Unless we fully understand the fact that descriptors are linked to classes and not to instances, and therefore need to maintain their own mapping of instances to instance-specific values, we might be tempted to write the Price descriptor as follows:

But once we start instantiating multiple Book instances, we’re going to have a problem.

The key is to understand that there is only one instance of Price for Book, so every time the value in the descriptor is changed, it changes for all instances. That behaviour in itself is useful for creating managed class attributes, but it is not what we want in this case. To store separate instance-specific values, we need to use the WeakRefDictionary.

The property built-in function

Another way of building descriptors is to use the property built-in function. Here is the function signature:

fget, fset and fdel are methods to get, set and delete attributes, respectively. doc is a docstring.

Instead of defining a single class-level descriptor object that manages instance-specific values, property works by combining instance methods from the class. Here is a simple example of a Publisher class from our inventory system with a managed name property. Each method passed into property has a print statement to illustrate when it is called.

If we make an instance of Publisher and access the name attribute, we can see the appropriate methods being called.

That’s it for this basic introduction to descriptors. If you want a challenge, take what you have learned and try to reimplement the @property decorator. There is enough information in this post to allow you to figure it out.

Download Mastering Decorators

Mastering_decorators_cover

Enjoyed this article? Join the newsletter and get Mastering Decorators - a gentle 22-page introduction to one of the trickiest parts of Python.

Weekly-ish. No spam. Unsubscribe any time. Powered by ConvertKit
  • Pingback: Python descriptors made simple – SmallSur...()

  • Primeq

    interesting and I think well written. However I miss how this (unless the example is specifically contrived for clarity) is better than the following. What is below has behavior that is quite similar to the descriptor thing, but since the price-value is declared at instance level in the constructor it’s simpler

    class Price(object):
    def __init__(self, cost):
    if cost0′)
    self.dollarCost = cost

    def __set__(self, instance, value):
    if value 100:
    raise ValueError(“Price must be between 0 and 100.”)
    self.values[instance] = value

    def __delete__(self, instance):
    del self.values[instance]

    class Book(object):

    def __init__(self, author, title, p):
    self.author = author
    self.title = title
    self.price = Price(p)

    def __str__(self):
    return “{0} – {1} – {2}”.format(self.author, self.title, self.price.dollarCost)

    if __name__ == ‘__main__’:

    x= Book(‘Tolstoy’, ‘W&P’, 13)
    print(x)
    print(x.price.dollarCost)

    • Supi Mon

      when you do “self.price = Price(p)” for the instance, that creates an instance of the Price class for every instance of Book class
      – that would render this line of code “self.values[instance] = value” in the __set__() in the Price class useless (because the __set__() method will always be called on a single instance anyway)
      – the purpose of the Price class, for e.g., in the above scenario is for validations and validations are usually achieved via singleton(i mean why would you want a single dedicated instance of the Price class for each Book instance when they all just do the same thing?)

  • Primeq

    oops – all python indents are mangled by the comment system – screenshot attached

  • Anton

    As for the beginner I think the article is poorly written.
    1) After the block of Price descriptor declaration there is the following passage:
    “A few details in the implementation of Price deserve mentioning.”
    So I’m prepared to see something about this descriptor block of code. Next you’re writing:”An instance of a descriptor must be added to a class as a class attribute, not as an instance attribute.” So my question is how can I infer this info looking at the descriptor declaration? It just misguides and adds frustration. And in the next section you write “As we saw in the last section, descriptors are linked to classes, not to instances, ” but we didn’t see it at all.
    2) The mechanics of descriptor is still not understandable.
    Looking at the Price example I still have a lot of questions. The main is: how does the first price value of 12 get to the descriptor dictionary? At the class Book declaration we add new class variable price=Price() which is a descriptor. But next there still is the Book’s __init__ method that takes price argument. So when we create instance of the Book class with:
    b1 = Book(“William Faulkner”, “The Sound and the Fury”, 12)
    the __init__ method is run and self.price is assigned with 12. Otherwise it would have been lost somewhere. And I still can’t figure out how 12 goes to the dictionary of descriptor.

    It would be good that author made more clear explanations.

  • Serge Mosin

    Very nice intro for beginners. I’d say, it’s a shorter simplified version of Raymond Hettinger’s article https://docs.python.org/2/howto/descriptor.html .