Notes on "Python's Class Development Toolkit"

By: Cam Wohlfeil
Published: 2019-03-07 1105 EST
Category: Programming
Tags: python, oop

Adapted from a Jupyter notebook, these are my notes of the the Raymond Hettinger PyCon talk "Python's Class Development Toolkit", linked at the end.

Include a Module DocString for your module

'''Circuituous, LLC --
   An Advanced Circle Analytics Company
'''

Document your class and methods

class Circle:
    'An advanced circle analytic toolkit'

Inherit from object

Make a new style class to get extra capablilities. This is advanced technology (sarcasm :P).

class Circle(object):            # new-style class
    'An advanced circle analytic toolkit'

Initialize instance variables

Init isn't a constructor. It's job is to initialize the instance variables. Init takes an existing instance 'self' and populates it with instance variables.

class Circle(object):            # new-style class
    'An advanced circle analytic toolkit'

    def __init__(self, radius):
        self.radius = radius         # instance variable

Regular method

Regular methods have "self" as first argument. "self" is a python convention but you can use any argument name you want. e.g. "this" , "that" etc.

class Circle(object):  
    'An advanced circle analytic toolkit'

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        'Perform quadrature on a shape of uniform radius'
        return 3.14 * self.radius ** 2.0

What about 3.14? Use math.pi

import math

class Circle(object): 
    'An advanced circle analytic toolkit'

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        'Perform quadrature on a shape of uniform radius'
        return math.pi * self.radius ** 2.0

Class variables for shared data among all instances

import math

class Circle(object): 
    'An advanced circle analytic toolkit'

    version = '0.1'                  # class variable


    def __init__(self, radius):
        self.radius = radius

    def area(self):
        'Perform quadrature on a shape of uniform radius'
        return math.pi * self.radius ** 2.0

Create a tutorial for your class

# Tutorial

print('Circuituous version', Circle.version)
c = Circle(10)
print('A circle of radius', c.radius)
print('has an area of', c.area())

First customer: Academia

from random import random, seed

seed(8675309)                      # for reproduceable results
print('Using Circuituous version', Circle.version)
n = 10
circles = [Circle(random()) for i in range(n)]
print('The average area of', n, 'random circles')
avg = sum([c.area() for c in circles]) / n
print('is %.5f' % avg)

Next customer wants a perimeter method

import math

class Circle(object): 
    'An advanced circle analytic toolkit'
    version = '0.2'

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        return math.pi * self.radius ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

Second customer: Rubber sheet company

How do we feel about exposing the radius attribute? The customer changed the radius value. If you expose an attribute expect the user to do all sorts of things with it.

cuts = [0.1, 0.7, 0.8]
circles = [Circle(r) for r in cuts]
for c in circles:
    print('A circle with a radius of ', c.radius)
    print('has a perimeter of', c.perimeter())
    print('and a cold area of', c.area())
    c.radius *= 1.1
    print('and a warm area of', c.area())

Do things the Python way. Python is a consenting adults language and we leave all of the variables exposed. You don't use getter and setter methods in Python, there is no protected and private variables in like in Java and C++. Don't impose Java rules when you are coding in python and vice versa.

Third customer: National tire chain

Change functionality through subclassing. If the parent gets called in the subclass method then it is called extending. If the parent does not get called in the subclass method then it is called overriding.

class Tire(Circle):
    'Tires are circles with a corrected perimeter'

    def perimeter(self):
        'Circumference corrected for the rubber'
        return Circle.perimeter(self) * 1.25

t = Tire(22)
print('A tire of radius', t.radius)
print('has an inner area of', t.area())
print('and an odometer corrected perimeter of'),
print('%.1f' % t.perimeter())

Next customer: National graphics company

The API is awkward. A converter function is always needed. Perhaps change the constructor signature? Need a new constructor to initialize Circle with bounding box diagonal. Several different users want different signatures for the constructor. If there is a constructor war, everyone should get their wish.

def bbd_to_radius(bbd):
    v = (bbd // math.sqrt(2.0)) // 2.0
    return v
bbd = 25.1
c = Circle(bbd_to_radius(bbd))
print('A circle with a bbd of %d' % bbd)
print('has a radius of', c.radius)
print('and an area of %.2f' % c.area())

Need an alternate constructor. We do it all the time.

from datetime import datetime 

print(datetime(2015, 3, 16))
print(datetime.fromtimestamp(1264383616))
print(datetime.fromordinal(734000))
print(datetime.now())

print(dict.fromkeys(['raymond', 'rachel', 'mathew']))

Class methods create alternative constructors. Use @classmethod decorator.

import math

class Circle(object): 
    'An advanced circle analytic toolkit'
    version = '0.3'

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        return math.pi * self.radius ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    @classmethod                     # alternative constructor (wrong)
    def from_bbd(cls, bbd):
        'Construct a circle from a bounding box diagonal'
        radius = bbd / 2.0 / math.sqrt(2.0)
        return Circle(radius) # wroooooong, use cls, see later on
<br > Client code: National graphics company

c = Circle.from_bbd(25.1)
print('A circle with a ddb of 25.1')
print('has a radius of', c.radius)
print('and and area of', c.area())

It should also work for subclasses. This code doesn't work. It makes a perimeter from Circle and not Tire.

class Tire(Circle):
    'Tires are circles with a corrected perimeter'

    def perimeter(self):
        'Circumference corrected for the rubber'
        return Circle.perimeter(self) * 1.25

t = Tire.from_bbd(45)
print('A tire of radius', t.radius)
print('has an inner area of', t.area())
print('and an odometer corrected perimeter of')
print(t.perimeter())

Alternative constructors need to anticipate subclassing. We need to change the return type of class method from_bbd from Circle to cls. Alternative constructors have a parameter cls and be sure to use that parameter because it will support the subclassing.

import math

class Circle(object): 
    'An advanced circle analytic toolkit'
    version = '0.3'

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        return math.pi * self.radius ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    @classmethod                     # alternative constructor
    def from_bbd(cls, bbd):
        'Construct a circle from a bounding box diagonal'
        radius = bbd / 2.0 / math.sqrt(2.0)
        return cls(radius) #correctly using cls

class Tire(Circle):
    'Tires are circles with a corrected perimeter'

    def perimeter(self):
        'Circumference corrected for the rubber'
        return Circle.perimeter(self) * 1.25

t = Tire.from_bbd(45)
print('A tire of radius', t.radius)
print('has an inner area of', t.area())
print('and an odometer corrected perimeter of')
print(t.perimeter()) #look how this has changed value now

New customer request: add a function angle_to_grade

Will this also work for the Sphere class and the Hyperbolic class? Can people even find this code? This function is just sitting out there for anyone to use. Need to add it to the class.

import math

class Circle(object): 
    'An advanced circle analytic toolkit'
    version = `0.3a`

    def __init__(self, radius):
        self.radius = radius

    def angle_to_grade(self, angle):
        'Convert angle in degrees to a percentage grade'
        return math.tan(math.radians(angle)) * 100.0    

    def area(self):
            return math.pi * self.radius ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    @classmethod                     # alternative constructor
    def from_bbd(cls, bbd):
        'Construct a circle from a bounding box diagonal'
        radius = bbd / 2.0 / math.sqrt(2.0)
        return cls(radius)

Well, findability has been improved and it won't be called in the wrong context. Really? You have to create an instance just to call this function? We passed in self but it is not used in the function.

Move function to a static method

The purpose of static method is to attach functions to classes. You do it to improve the findability of the function and to make sure that people are using the function in the appropriate context.

import math

class Circle(object): 
    'An advanced circle analytic toolkit'
    version = '0.4'

    def __init__(self, radius):
        self.radius = radius

    @staticmethod                    # attach functions to classes 
    def angle_to_grade(angle):
        'Convert angle in degrees to a percentage grade'
        return math.tan(math.radians(angle)) * 100.0    

    def area(self):
            return math.pi * self.radius ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    @classmethod                     # alternative constructor
    def from_bbd(cls, bbd):
        'Construct a circle from a bounding box diagonal'
        radius = bbd / 2.0 / math.sqrt(2.0)
        return cls(radius)

Government request: ISO-11110 compliance

Tell you what your product should do and how it should do it (micro-management). ISO standard says you are not allowed to use the radius in the perimeter calculation. Assume some legitimate reason for requiring this rule.

But the following is the wrong way:

import math

class Circle(object): 
    'An advanced circle analytic toolkit'

    version = '0.5b'

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        p = self.perimeter()
        r = p / math.pi / 2.0
        return math.pi * r ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

But self is you (a Circle instance) and your children classes instances (e.g. a Tire instance). This messes up the Tire company code. Its perimeter (modified) scales up the radius. We have broken their area method. They should be free to override any method they want without breaking any of the other methods.

class Tire(Circle):
    'Tires are circles with a corrected perimeter'

    def perimeter(self):
        'Circumference corrected for the rubber'
        return Circle.perimeter(self) * 1.25
c = Circle(10)
t = Tire(10)
print(c.perimeter(), t.perimeter()) #notice the desired different perimeter
print(c.area(), t.area()) #notice the undesired different area!!!
# 62.83185307179586 78.53981633974483
# 314.1592653589793 490.8738521234052

Class local reference: Keep a spare copy

Possible solution:

import math

class Circle(object): 
    'An advanced circle analytic toolkit'

    version = '0.5b'

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        p = self._perimeter()
        r = p / math.pi / 2.0
        return math.pi * r ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    _perimeter = perimeter


class Tire(Circle):
    'Tires are circles with a corrected perimeter'

    def perimeter(self):
        'Circumference corrected for the rubber'
        return Circle.perimeter(self) * 1.25
c = Circle(10)
t = Tire(10)
print(c.perimeter(), t.perimeter()) #notice the desired different perimeter
print(c.area(), t.area()) #notice the desired equal area!!!
# 62.83185307179586 78.53981633974483
# 314.1592653589793 314.1592653589793

But a subclass could do the same thing.They will protect their reference to their perimeter by doing the same thing, and hence break the area method again.

class Tire(Circle):
    'Tires are circles with a corrected perimeter'

    def perimeter(self):
        'Circumference corrected for the rubber'
        return Circle._perimeter(self) * 1.25

    _perimeter = perimeter

c = Circle(10)
t = Tire(10)
print(c.perimeter(), t.perimeter()) #notice the desired different perimeter
print(c.area(), t.area()) #notice that the area is again different. Bad!!!
# 62.83185307179586 78.53981633974483
# 314.1592653589793 490.8738521234052

Double underscore (dunder)

A way around this is to use double underscore. The purpose of double underscore is to make self refer to you and not your children. It is for a class local reference. It is not about privacy. The variables in python are free and open to all. It is about allowing your subclass to override any method without breaking the others.

import math

class Circle(object): 
    'An advanced circle analytic toolkit'

    version = '0.5b'

    def __init__(self, radius):
        self.radius = radius

    def area(self):
        p = self.__perimeter()
        r = p / math.pi / 2.0
        return math.pi * r ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    __perimeter = perimeter


class Tire(Circle):
    'Tires are circles with a corrected perimeter'

    def perimeter(self):
        'Circumference corrected for the rubber'

        #return Circle.__perimeter(self) * 1.25
        #this would get an error because it would call Circle._Tire__perimeter

        return Circle.perimeter(self) * 1.25

    __perimeter = perimeter #we are also storing a spare copy of Tire's perimeter method


c = Circle(10)
t = Tire(10)
print(c.perimeter(), t.perimeter()) #notice the desired different perimeter
print(c.area(), t.area()) #notice that the area is equal (good!)
# 62.83185307179586 78.53981633974483
# 314.1592653589793 314.1592653589793

Government request: ISO-22220

You're not allowed to store the radius, you must store the diameter instead! Solution is to use @property decorator for radius. You can do this 'after the fact' in Python classes and it will store the .radius parameter in the diameter attribute. The property method for radius will get called instead of storing the parameter in the radius attribute.

This is a big big win for the Python language. When you design a class you don't have to put getters and setters in it, you can just use property if you later need to make changes. It will make your classes shorter and make them run faster and make the API's more beautiful. You can do this in a dynamic language but not in a compiled language.

import math

class Circle(object): 
    'An advanced circle analytic toolkit'
    version = '0.6'

    def __init__(self, radius):
        self.radius = radius

    @property       # convert dotted access to method calls
    def radius(self):
        'Radius of a circle'
        return self.diameter / 2.0

    @radius.setter
    def radius(self, radius):
        self.diameter = radius * 2.0

    @staticmethod                    # attach functions to classes 
    def angle_to_grade(angle):
        'Convert angle in degrees to a percentage grade'
        return math.tan(math.radians(angle)) * 100.0    

    def area(self):
            return math.pi * self.radius ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    @classmethod                     # alternative constructor
    def from_bbd(cls, bbd):
        'Construct a circle from a bounding box diagonal'
        radius = bbd / 2.0 / math.sqrt(2.0)
        return Circle(radius)

User request: Many circles

Major memory problem: Circle instances are over 300 bytes each!

from random import random, seed
from datetime import datetime

n = 10000000

start_time = datetime.now()
seed(8675309)
print('Using Circuituous(tm) version', Circle.version)

circles = [Circle(random()) for i in range(n)] #this takes long
print('The average area of', n, 'random circles')
avg = sum([c.area() for c in circles]) / n #this takes long too
print('is %.4f' % avg)
print(datetime.now() - start_time)
# Using Circuituous(tm) version 0.6
# The average area of 10000000 random circles
# is 1.0470
# 0:00:18.442537
```python

How can you shrink down the size of the circles when all we are storing is a radius for each one? By using "slots". This is called the flyweight design pattern: when you have many many instances then make them lightweight. That slots will do is allocate just one pointer for the diameter and nothing else, no dictionary. You lose the ability to inspect the dictionary and the ability to add additional attributes. So this is just an optimization, you save it for last only when you are scaling up to millions of instances. Your users can now add millions of circles on their machine without eating up all of the memory.

```python
import math

class Circle(object): 
    'An advanced circle analytic toolkit'

    # flyweight design pattern suppresses
    # the instance dictionary

    __slots__ = ['diameter']
    version = '0.7'

    def __init__(self, radius):
        self.radius = radius

    @property       # convert dotted access to method calls
    def radius(self):
        'Radius of a circle'
        return self.diameter / 2.0

    @radius.setter
    def radius(self, radius):
        self.diameter = radius * 2.0

    @staticmethod                    # attach functions to classes 
    def angle_to_grade(angle):
        'Convert angle in degrees to a percentage grade'
        return math.tan(math.radians(angle)) * 100.0    

    def area(self):
            return math.pi * self.radius ** 2.0

    def perimeter(self):
        return 2.0 * math.pi * self.radius

    @classmethod                     # alternative constructor
    def from_bbd(cls, bbd):
        'Construct a circle from a bounding box diagonal'
        radius = bbd / 2.0 / math.sqrt(2.0)

        return Circle(radius)

from datetime import datetime

n = 10000000

seed(8675309)
print('Using Circuituous(tm) version', Circle.version)
start_time = datetime.now()
circles = [Circle(random()) for i in range(n)]
print('The average area of', n, 'random circles')
avg = sum([c.area() for c in circles]) / n
print('is %.1f' % avg)
print(datetime.now() - start_time)
# Using Circuituous(tm) version 0.7
# The average area of 10000000 random circles
# is 1.0
# 0:00:15.326014

Summary: Toolset for New-Style Classes

Inherit from object
Instance variables for information unique to an instance
Class variables for data shared among all instances
Regular methods need "self" to operate on instance data
Class methods implement alternative constructors. They need "cls" so they can create subclass instances as well.
Static methods attach functions to classes. They don't need either "self" or "cls". Static methods improve discoverability and require context to be specified.
A property() lets getter and setter methods be invoked automatically by attribute access. This allows Python classes to freely expose their instance variables.
Double underscore make self refer to this class and not its children. It allows the subclass to override any method without breaking the others.
slots is flyweight design pattern used to shrink class size.

References

YouTube: Python's Class Development Toolkit (Raymond Hettinger, PyCon 2013)