Documenting the public interface

This library provides two decorators that document the public visibility of the names in your module. They keep your module’s __all__ in sync so you don’t have to.

Also included is a function that you can put at the bottom of your module to simply infer all the public names, and populate the __all__ for you.

Background

__all__ is great. It has both functional and documentation purposes.

The functional purpose is that it directly controls which module names are imported by the from <module> import * statement. In the absence of an __all__, when this statement is executed, every name in <module> that does not start with an underscore will be imported. This often leads to importing too many names into the module. That’s a good enough reason not to use from <module> import * with modules that don’t have an __all__.

In the presence of an __all__, only the names specified in this list are imported by the from <module> import * statement. This in essence gives the <module> author a way to explicitly state which names are for public consumption.

And that’s the second purpose of __all__; it serves as module documentation, explicitly naming the public objects it wants to export. You can print a module’s __all__ and get an explicit declaration of its public API.

The problem with __all__

__all__ has two problems.

First, it separates the declaration of a name’s public export semantics from the implementation of that name. Usually the __all__ is put at the top of the module, although this isn’t required, and in some cases it’s actively prohibited. So when you’re looking at the definition of a function or class in a module, you have to search for the __all__ definition to know whether the function or class is intended for public consumption.

This leads to the second problem, which is that it’s too easy for the __all__ to get out of sync with the module’s contents. Often a function or class is renamed, removed, or added without the __all__ being updated. Then it’s difficult to know what the module author’s intent was, and it can lead to an exception when a string appearing in __all__ doesn’t match an existing name in the module. The standard library has had this bug too. Some tools like Sphinx will complain when names appear in __all__ don’t appear in the module. Linters can do some checking of the names in __all__ but they can’t easily guess the author’s intent. All of this points to the root problem; it should be easy to keep __all__ in sync!

@public

This package provides a way to declare a name’s public visibility right at the point of its declaration, and to infer the name to export from that definition. In this way, a module’s author never explicitly sets the __all__ so there’s no way for it to get out of sync.

This package provides just such a solution [1], in the form of a function that can be used as either a decorator (@public) or a function (public()).

>>> from public import public

You’ll usually use this as a decorator, for example:

>>> @public
... def tune():
...    pass

or:

>>> @public
... class Cello:
...     pass

After these code snippets run, the __all__ has both names in it:

>>> print(__all__)
['tune', 'Cello']

Note

You do not need to initialize __all__ in the module, since public() will do it for you. Of course, if your module already has an __all__, it will append any new names to the existing list.

The requirements to use the @public decorator are simple: the decorated thing must have a __name__ attribute. Since you’ll overwhelmingly use it to decorate functions and classes, this will almost always be the case.

Function call form

public() is just a function, so you can also call it directly. There are two flavors: you can pass it a single positional argument, or you can pass it keyword arguments.

Either way, public() adds the name to the __all__ of the module where the call appears. It never touches the __all__ of the module where the object happens to have been defined.

Single argument

The most common reason to reach for this form is re-exporting: the thing you want to declare public isn’t defined in your module at all, it’s imported from somewhere else.

>>> from strings import Bass
>>> public(Bass)
<class 'strings.Bass'>
>>> print(__all__)
['Bass']

Note that Bass was added to your module’s __all__. The strings module is left completely alone; it doesn’t even grow an __all__:

>>> import strings
>>> hasattr(strings, '__all__')
False

If you rename something as you import it, the name you bound locally is the one that gets exported, since that’s the name that has to work for from <module> import *:

>>> from reeds import Harmonica as Harp
>>> public(Harp)
<class 'reeds.Harmonica'>
>>> print(__all__)
['Bass', 'Harp']

Modules work too, which is handy for a package that wants to export one of its submodules:

>>> from woodwinds import piccolo
>>> public(piccolo)
<module 'woodwinds.piccolo' from '...'>
>>> print(__all__)
['Bass', 'Harp', 'piccolo']

There’s one case where public() can’t read your mind. If you bind the same object to two names in your own module, the name it was defined with wins:

>>> class Fiddle:
...     pass
>>> Violin = Fiddle
>>> public(Violin)
<class '...Fiddle'>
>>> print(__all__)
['Fiddle']

Both names refer to the same object, and by the time public() runs there’s no way to tell which name it should use. The definition name is used (Fiddle in this case), and from <module> import * works either way. If you specifically want Violin exported, use the keyword form described below:

>>> public(Violin=Fiddle)
<class '...Fiddle'>
>>> print(__all__)
['Fiddle', 'Violin']

Finally, you can pass a string, which does exactly what appending to __all__ yourself would do, except that public() creates the __all__ if it doesn’t exist yet:

>>> public('Tuba')
'Tuba'
>>> print(__all__)
['Tuba']

Reach for the string form only when you have to. A string is just a string; nothing checks it against your module’s contents, so it’s precisely the kind of thing that goes stale, which is what this library exists to prevent. It’s good for names that don’t exist as source code, such as dynamic bindings:

orchestra = {'Bassoon': dict, 'Flute': list}
for name, factory in orchestra.items():
    globals()[name] = factory()
    public(name)

>>> print(__all__)
['Bassoon', 'Flute']

public() rejects anything it can’t reliably turn into a name. Constants and instances don’t have a __name__, and looking them up by identity isn’t dependable, because small integers and interned strings are shared throughout the interpreter. You get a TypeError pointing you at the keyword form:

>>> public(7)
Traceback (most recent call last):
...
TypeError: Cannot infer a name from: <class 'int'>; use the keyword argument form

A module has to actually be bound in your namespace. import xml.dom binds only xml, so there’s no name there to export:

>>> import xml.dom
>>> public(xml.dom)
Traceback (most recent call last):
...
TypeError: Module is not bound in the calling namespace: xml.dom

And a string has to look like a name:

>>> public('not an identifier')
Traceback (most recent call last):
...
ValueError: Not a valid Python identifier: 'not an identifier'

Reserved words are rejected as well. They pass Python’s identifier test, but nothing can ever be bound to them, so a reserved word in __all__ is guaranteed to be a name that doesn’t exist:

>>> public('class')
Traceback (most recent call last):
...
ValueError: Cannot use a Python keyword as a name: 'class'

Soft keywords are fine, because they can be bound:

>>> public('match')
'match'
>>> print(__all__)
['match']

Keyword arguments

There’s one other common use case that isn’t covered by the @public decorator. Sometimes you want to declare simple constants or instances as publicly available. You can’t use the @public decorator for two reasons: constants don’t have a __name__ and Python’s syntax doesn’t allow you to decorate such constructs.

To solve this use case, public() also accepts keyword arguments, where the key is used as the name.

>>> public(TEMPO=120)
120
>>> public(a_cello=Cello())
<...Cello object ...>

The module’s __all__ now contains both names:

>>> print(__all__)
['TEMPO', 'a_cello']

The module also contains name bindings for these constants:

>>> print(TEMPO)
120
>>> print(a_cello)
<....Cello object at ...>

Multiple keyword arguments are allowed:

>>> public(ROOT=1, FIFTH=5)
(1, 5)
>>> print(__all__)
['TEMPO', 'a_cello', 'ROOT', 'FIFTH']

>>> print(ROOT)
1
>>> print(FIFTH)
5

You’ll notice that the functional form of public() returns the values in keyword argument order. This is to help with a use case where some linters complain because they can’t see that public() binds the names in the global namespace. In the above example they might report erroneously that ROOT and FIFTH aren’t defined. To work around this, when public() is used in its functional form, it will return the values in the order they are seen [2] and you can simply assign them to local variable names explicitly.

>>> second, third, seventh = public(second=2, third=3, seventh=7)
>>> print(__all__)
['TEMPO', 'a_cello', 'ROOT', 'FIFTH', 'second', 'third', 'seventh']
>>> print(second, third, seventh)
2 3 7

It also works if you bind only a single value.

>>> ninth = public(ninth=9)
>>> print(__all__)
['TEMPO', 'a_cello', 'ROOT', 'FIFTH', 'second', 'third', 'seventh', 'ninth']
>>> print(ninth)
9

@private

You might also want to be explicit about your private, i.e. non-public, names. This library provides an @private decorator for this purpose. While it mostly serves for documentation purposes, this decorator also ensures that the decorated object’s name does not appear in the __all__.

>>> from public import private

>>> @public
... def tune(): pass

>>> print(__all__)
['tune']

>>> @private
... def tune(): pass

>>> print(__all__)
[]

You can see here that tune has been removed from the __all__. It’s okay if the name doesn’t appear in __all__ at all:

>>> @private
... class Timpani:
...     pass

>>> print(__all__)
[]

In this case, Timpani does not appear in __all__.

Unlike @public, @private never creates an __all__. If your module doesn’t have one there is nothing for the name to be missing from, and an empty __all__ is not the same thing as no __all__ at all: the first exports nothing, while the second exports every name that doesn’t start with an underscore. Creating one would silently change what from <module> import * gives you.

>>> @private
... class Oboe:
...     pass

>>> '__all__' in globals()
False

If the __all__ does exist, it must be a list, just as it must be for @public.

private() accepts the same single argument call form as public() does, resolving the name the same way, and removing it from the __all__ of the module where the call appears:

>>> from strings import Bass
>>> public(Bass)
<class 'strings.Bass'>
>>> print(__all__)
['Bass']

>>> private(Bass)
<class 'strings.Bass'>
>>> print(__all__)
[]

There is no keyword argument form for private().

Inferring __all__

If you don’t like using the decorators, you can instead infer and populate the contents of __all__ by calling the populate_all() function at the bottom of your module. This uses heuristics to pick out some names from the module, adding them to __all__ if they meet the following criteria:

• The name does not start with an underscore.

• The name is not bound to a module object. This prevents imported modules from being added [3].

• The object the name is bound to does not appear to be defined in some other module. This prevents most from-imports from being added, but note that this can be fooled if you import simple types (such as an int or a str) from another module (e.g. from sys import abiflags), because simple types don’t have a __module__ attribute.

For example, if your Python module looks like this:

# example.py
from public import populate_all

def tune():
    pass

class Bass:
    pass

tempo: int = 120
_muted: bool = False

populate_all()

when you import this module, the __all__ will be populated with the names matching the above heuristics.

>>> example.__all__
['tune', 'Bass', 'tempo']

In this case, you can see that the module has an __all__ set to ['tune', 'Bass', 'tempo'] but note that neither _muted nor populate_all are added.

If the inferencing misses some names you want to publicly export, you can always add them explicitly by using the function call form public().

Note

populate_all() only adds new names to __all__.

Installation

The package is on PyPI under the name atpublic:

$ pip install atpublic

If you want, you can install @public and @private into your builtins module so they’re always available without having to import them first.

The easy way to do this is to depend on the atpublic[install] dependency (i.e. with the install extra) instead of just atpublic:

$ pip install atpublic[install]

The extra pulls in the companion atpublic-install package, which adds the decorators to builtins as the interpreter starts up, so every module can use them without importing anything.

Alternatively, you can call the install method manually:

from public import install
install()

Caveats

There are some important usage restrictions you should be aware of:

• Only use @public and @private on top-level objects. Specifically, don’t try to use either decorator on a class method name. While the declaration won’t fail, you will get an exception when you attempt to from <module> import * because the name pulled from __all__ won’t be in the module’s globals.

• If you explicitly set __all__ in your module, be sure to set it to a list. Some style guides require __all__ to be a tuple, but since that’s immutable, as soon as @public tries to append to it, you will get an exception. Best practice is to not set __all__ explicitly; let @public and @private do it!

• If you still want __all__ to be immutable, put the following at the bottom of your module:

  __all__ = tuple(__all__)
  

[1]

PEP 844, and Python issue 26632 propose adding this to Python itself.

[2]

This ordering is guaranteed by PEP 468.

[3]

Thus for so called “hub” modules (where names in submodules are exported in a package’s top level __init__.py), you would need to explicitly add the imported modules you want to export. The single argument call form does exactly this, e.g. public(submodule).