Bits of Py.

If the implementation is hard to explain, it's a bad idea

Fri 10 August 2018

The descriptor protocol in Python

Posted by marodrig in blog   

Python Descriptors

Python descriptors were introduced in Python version 2.2; however, they are not commonly known or used by developers. The main advantage of descriptors is that they allow the developer to create managed attributes, protecting the attribute from changes or automatically updating the attribute if it has a dependency.

The descriptors protocol allows the developer to improve coding quality by implementing getter and setters methods in a Pythonic Way.

What are Descriptors

Java uses setter and getter public functions when interacting with a private class attribute. Private functions or attributes are not part of the Pythonic Way. This is where Descriptors come in to save the day and allow developers to bind behavior when accessing an attribute while keeping the Pythonic Way. The descriptor protocol requires us to implement three methods for a class, these are the __get__, __set__, and __delete__ methods. If any of these three methods is defined for a class, this class is said to be a descriptor.

One last point, descriptors are assigned to a Class. Not an instance. This means that modifying the Class overwrites or deletes the descriptor itself, rather than triggering the descriptor.

The signatures for these three methods is as follows:

__get__(self, instance, owner)
__set__(self, instance, value)
__delete__(self, instance)

Here:

  • __get__: Accesses the attribute, returning the attribute value.

  • __set__: Assignment operation of the attribute. Doesn't return any value.

  • __delete__: Delete operation for the attribute. Doesn't return any value.

Why and when to use Descriptors

Think of a phone number attribute for a class, or a Social Security Number. These two numbers follow a specific format that validates their correctness. Without descriptors, an invalid Social Security Number could be assigned, and we wouldn't know. On the other hand, using descriptors would allow us to use regular expressions before we assign an attribute, raise an error and protect access to an attribute of our class.

How to implement Descriptors

We will go over three main ways to implement descriptors:

  • Override any (or all) of the descriptor methods in a class. This method is best when the same descriptor is used in more than one class and attribute.

  • Use a property type to create a descriptor. This is more flexible way of implementing descriptors.

  • Use property decorators!

Implement Descriptors by Overriding descriptor methods

We start by creating our own descriptor class by overriding __get__, __set__, and __delete__ methods as we do in the following code fragment:

class PyDescriptor(object):

    def __init__(self):
        self._name = None

    def __get__(self, instance, owner):
        print("Getting: {}".format(self._name))
        return self._name

    def __set__(self, instance, value):
        print("Setting: {}".format(self._name))
        print("To value: {}".format(value))
        self._name = value

    def __delete__(self, instance):
        print("Deleting: {}".format(self._name))
        del self._name

We can then use an instance of the PyDescriptor class in another class, applying our new behavior to the property of the new class like so:

class User(object):
    name = PyDescriptor()

This way we can create an object of our User class, and the name attribute will implement the Descriptor protocol, adding the simple behavior of print statements for each of the methods we overwrote. In this case it's simple print statements, and nothing special.

>>> py_user = User()
>>> py_user.name
Getting: None
>>> py_user.name = 'Harry'
Setting: None
To value: Harry
>>> del py_user.name
Deleting: Harry

Implement Descriptors by using a property type

An alternative method when implementing descriptors is to use the property type. Using property() we can easily create an descriptor for any attribute. Signature for property: property(fget = None, fset = None, fdel = None, doc = None).

Where:

  • fget: Method to get the attribute.

  • fset: Method to set the attribute.

  • fdel: Method to delete the attribute.

  • doc: docstring for this attribute.

import re


class User(object):

    def __init__(self):
        self._phone_number = ''

    def fget(self):
        return self._phone_number

    def fset(self, value):
        if re.match(r'^(\d{3})-(\d{3})-(\d{4})', value):
            print('Adding phone number for user: {}'.format(value))
            self._phone_number = value
        else:
            raise ValueError('Phone Number format does not match.')

    def fdel(self):
        print("Deleting: {}".format(self._phone_number))
        del self._phone_number

    phone_number = property(fget, fset, fdel, 'I am the phone number property.')

We have a property for the phone number of our user. We have added a simple regular expression used to validate the phone number before we set the attribute to the value given. We raise a ValueError if this doesn't match the regular expression. And yes, it's a simple expression that doesn't cover all the valid phone numbers, and is not intended to cover all of them.

Let's see our additional behavior work it's magic:

>>> py_user = User()
>>> py_user.phone_number = '123'
ValueError: Phone Number format does not match.
>>> py_user.phone_number = '800-123-4567'
Adding phone number for user: 800-123-4567
>>> print(py_user.phone_number)
800-123-4567
>>> del py_user.phone_number
Deleting: 800-123-4567

Implement Descriptors by using property decorators

A more Pythonic Way of implementing descriptors is by using decorators to create a property, and bind behavior to it.

class User(object):

    def __init__(self):
        self._phone_number = ''

    @property
    def phone_number(self):
        return self._phone_number

    @phone_number.setter
    def phone_number(self, value):
        if re.match(r'^(\d{3})-(\d{3})-(\d{4})', value):
            print('Adding phone number for user: {}'.format(value))
            self._phone_number = value
        else:
            raise ValueError('Phone Number format does not match.')

    @phone_number.deleter
    def phone_number(self):
        print("Deleting: {}".format(self._phone_number))
        del self._phone_number

We have implemented the same behavior as before, but now we are using decorators. Let's see it in action:

>>> py_user = User()
>>> py_user.phone_number
>>> py_user.phone_number = '123'
ValueError: Phone Number format does not match.
>>> py_user.phone_number = '800-123-4567'
Adding phone number for user: 800-123-4567
>>> del py_user.phone_number
Deleting: 800-123-4567

Creating read-only attributes

In order to create a read-only attribute the developer can simply not define a setter method. The following example uses property decorators when creating a read-only attribute:

class Item(object):

    def __init__(self):
        self._item_id = 100

    @property
    def item_id(self):
        return self._item_id

Let's see what happens if we try to set this attribute:

>>> my_item = Item()
>>> my_item.item_id = 201

AttributeError: can't set attribute

We get an AttributeError, and the attribute is unchanged.

try:
    my_item.item_id = 201
except AttributeError as ae:
    print(ae)

print("ID: {}".format(my_item.item_id))

Bonus: Alternative way to create a read-only attribute using the property type:

class Item(object):

    def __init__(self):
        self._item_id = 100

    def fset(self):
        return self._item_id

    def fdel(self):
        del self._item_id

    item_id = property(fset, None, fdel, 'I am a read-only attribute')

Summary

Python descriptors are a flexible way for attribute management. Combined with decorators, they make for a pythonic programming style, allowing to bind behavior (we could validate the attribute by value or type) when accessing attributes of a class, as well as read-only attributes. But remember to avoid adding unnecessary code complexity when using descriptors from overriding the normal behaviors of an object.

Sample code Repository

References - Links