================================== Documenting the public interface ================================== This library provides two decorators that document the public visibility of the names in your module. They keep your module's ``__all__`` in sync so you don't have to. Also included is a function that you can put at the bottom of your module to simply infer all the public names, and populate the ``__all__`` for you. Background ========== ``__all__`` is great. It has both `functional `__ and documentation purposes. The functional purpose is that it `directly controls`_ which module names are imported by the ``from import *`` statement. In the absence of an ``__all__``, when this statement is executed, every name in ```` that does not start with an underscore will be imported. This often leads to importing too many names into the module. That's a good enough reason not to use ``from import *`` with modules that don't have an ``__all__``. In the presence of an ``__all__``, only the names specified in this list are imported by the ``from import *`` statement. This in essence gives the ```` author a way to explicitly state which names are for public consumption. And that's the second purpose of ``__all__``; it serves as module documentation, explicitly naming the public objects it wants to export. You can print a module's ``__all__`` and get an explicit declaration of its public API. The problem with __all__ ======================== ``__all__`` has two problems. First, it separates the declaration of a name's public export semantics from the implementation of that name. Usually the ``__all__`` is put at the top of the module, although this isn't required, and in some cases it's `actively prohibited`_. So when you're looking at the definition of a function or class in a module, you have to search for the ``__all__`` definition to know whether the function or class is intended for public consumption. This leads to the second problem, which is that it's too easy for the ``__all__`` to get `out of sync`_ with the module's contents. Often a function or class is renamed, removed, or added without the ``__all__`` being updated. Then it's difficult to know what the module author's intent was, and it can lead to an exception when a string appearing in ``__all__`` doesn't match an existing name in the module. The standard library has had `this bug`_ too. Some tools like Sphinx_ will complain when names appear in ``__all__`` don't appear in the module. Linters can do *some* checking of the names in ``__all__`` but they can't easily guess the author's intent. All of this points to the root problem; it should be easy to keep ``__all__`` in sync! @public ======= This package provides a way to declare a name's public visibility right at the point of its declaration, and to infer the name to export from that definition. In this way, a module's author never explicitly sets the ``__all__`` so there's no way for it to get out of sync. This package provides just such a solution [#]_, in the form of a function that can be used as either a decorator (``@public``) or a function (``public()``). .. code-block:: pycon >>> from public import public You'll usually use this as a decorator, for example: .. code-block:: pycon >>> @public ... def tune(): ... pass or: .. code-block:: pycon >>> @public ... class Cello: ... pass After these code snippets run, the ``__all__`` has both names in it: .. code-block:: pycon >>> print(__all__) ['tune', 'Cello'] .. note:: You do not need to initialize ``__all__`` in the module, since ``public()`` will do it for you. Of course, if your module *already* has an ``__all__``, it will append any new names to the existing list. The requirements to use the ``@public`` decorator are simple: the decorated thing must have a ``__name__`` attribute. Since you'll overwhelmingly use it to decorate functions and classes, this will almost always be the case. Function call form ================== ``public()`` is just a function, so you can also call it directly. There are two flavors: you can pass it a single positional argument, or you can pass it keyword arguments. Either way, ``public()`` adds the name to the ``__all__`` of the module where the *call* appears. It never touches the ``__all__`` of the module where the object happens to have been defined. Single argument --------------- The most common reason to reach for this form is re-exporting: the thing you want to declare public isn't defined in your module at all, it's imported from somewhere else. .. invisible-code-block: pycon >>> reset() >>> make_module('strings', """ ... class Bass: ... pass ... """) >>> make_module('reeds', """ ... class Harmonica: ... pass ... """) >>> make_module('woodwinds.piccolo') .. code-block:: pycon >>> from strings import Bass >>> public(Bass) >>> print(__all__) ['Bass'] Note that ``Bass`` was added to *your* module's ``__all__``. The ``strings`` module is left completely alone; it doesn't even grow an ``__all__``: .. code-block:: pycon >>> import strings >>> hasattr(strings, '__all__') False If you rename something as you import it, the name you bound locally is the one that gets exported, since that's the name that has to work for ``from import *``: .. code-block:: pycon >>> from reeds import Harmonica as Harp >>> public(Harp) >>> print(__all__) ['Bass', 'Harp'] Modules work too, which is handy for a package that wants to export one of its submodules: .. code-block:: pycon >>> from woodwinds import piccolo >>> public(piccolo) >>> print(__all__) ['Bass', 'Harp', 'piccolo'] There's one case where ``public()`` can't read your mind. If you bind the same object to two names in your own module, the name it was defined with wins: .. invisible-code-block: pycon >>> reset() .. code-block:: pycon >>> class Fiddle: ... pass >>> Violin = Fiddle >>> public(Violin) >>> print(__all__) ['Fiddle'] Both names refer to the same object, and by the time ``public()`` runs there's no way to tell which name it should use. The definition name is used (``Fiddle`` in this case), and ``from import *`` works either way. If you specifically want ``Violin`` exported, use the keyword form described below: .. code-block:: pycon >>> public(Violin=Fiddle) >>> print(__all__) ['Fiddle', 'Violin'] Finally, you can pass a string, which does exactly what appending to ``__all__`` yourself would do, except that ``public()`` creates the ``__all__`` if it doesn't exist yet: .. invisible-code-block: pycon >>> reset() .. code-block:: pycon >>> public('Tuba') 'Tuba' >>> print(__all__) ['Tuba'] Reach for the string form only when you have to. A string is just a string; nothing checks it against your module's contents, so it's precisely the kind of thing that goes stale, which is what this library exists to prevent. It's good for names that don't exist as source code, such as dynamic bindings: .. invisible-code-block: pycon >>> reset() .. code-block:: python orchestra = {'Bassoon': dict, 'Flute': list} for name, factory in orchestra.items(): globals()[name] = factory() public(name) .. code-block:: pycon >>> print(__all__) ['Bassoon', 'Flute'] ``public()`` rejects anything it can't reliably turn into a name. Constants and instances don't have a ``__name__``, and looking them up by identity isn't dependable, because small integers and interned strings are shared throughout the interpreter. You get a ``TypeError`` pointing you at the keyword form: .. code-block:: pycon >>> public(7) Traceback (most recent call last): ... TypeError: Cannot infer a name from: ; use the keyword argument form A module has to actually be bound in your namespace. ``import xml.dom`` binds only ``xml``, so there's no name there to export: .. code-block:: pycon >>> import xml.dom >>> public(xml.dom) Traceback (most recent call last): ... TypeError: Module is not bound in the calling namespace: xml.dom And a string has to look like a name: .. code-block:: pycon >>> public('not an identifier') Traceback (most recent call last): ... ValueError: Not a valid Python identifier: 'not an identifier' Reserved words are rejected as well. They pass Python's identifier test, but nothing can ever be bound to them, so a reserved word in ``__all__`` is guaranteed to be a name that doesn't exist: .. code-block:: pycon >>> public('class') Traceback (most recent call last): ... ValueError: Cannot use a Python keyword as a name: 'class' Soft keywords are fine, because they *can* be bound: .. invisible-code-block: pycon >>> reset() .. code-block:: pycon >>> public('match') 'match' >>> print(__all__) ['match'] Keyword arguments ----------------- There's one other common use case that isn't covered by the ``@public`` decorator. Sometimes you want to declare simple constants or instances as publicly available. You can't use the ``@public`` decorator for two reasons: constants don't have a ``__name__`` and Python's syntax doesn't allow you to decorate such constructs. To solve this use case, ``public()`` also accepts keyword arguments, where the key is used as the name. .. invisible-code-block: pycon >>> reset() .. code-block:: pycon >>> public(TEMPO=120) 120 >>> public(a_cello=Cello()) <...Cello object ...> The module's ``__all__`` now contains both names: .. code-block:: pycon >>> print(__all__) ['TEMPO', 'a_cello'] The module also contains name bindings for these constants: .. code-block:: pycon >>> print(TEMPO) 120 >>> print(a_cello) <....Cello object at ...> Multiple keyword arguments are allowed: .. code-block:: pycon >>> public(ROOT=1, FIFTH=5) (1, 5) >>> print(__all__) ['TEMPO', 'a_cello', 'ROOT', 'FIFTH'] >>> print(ROOT) 1 >>> print(FIFTH) 5 You'll notice that the functional form of ``public()`` returns the values in keyword argument order. This is to help with a use case where some linters complain because they can't see that ``public()`` binds the names in the global namespace. In the above example they might report erroneously that ``ROOT`` and ``FIFTH`` aren't defined. To work around this, when ``public()`` is used in its functional form, it will return the values in the order they are seen [#]_ and you can simply assign them to local variable names explicitly. .. code-block:: pycon >>> second, third, seventh = public(second=2, third=3, seventh=7) >>> print(__all__) ['TEMPO', 'a_cello', 'ROOT', 'FIFTH', 'second', 'third', 'seventh'] >>> print(second, third, seventh) 2 3 7 It also works if you bind only a single value. .. code-block:: pycon >>> ninth = public(ninth=9) >>> print(__all__) ['TEMPO', 'a_cello', 'ROOT', 'FIFTH', 'second', 'third', 'seventh', 'ninth'] >>> print(ninth) 9 @private ======== You might also want to be explicit about your private, i.e. non-public, names. This library provides an ``@private`` decorator for this purpose. While it mostly serves for documentation purposes, this decorator also ensures that the decorated object's name does *not* appear in the ``__all__``. .. invisible-code-block: pycon >>> reset() .. code-block:: pycon >>> from public import private >>> @public ... def tune(): pass >>> print(__all__) ['tune'] >>> @private ... def tune(): pass >>> print(__all__) [] You can see here that ``tune`` has been removed from the ``__all__``. It's okay if the name doesn't appear in ``__all__`` at all: .. invisible-code-block: pycon >>> reset() .. code-block:: pycon >>> @private ... class Timpani: ... pass >>> print(__all__) [] In this case, ``Timpani`` does not appear in ``__all__``. Unlike ``@public``, ``@private`` never *creates* an ``__all__``. If your module doesn't have one there is nothing for the name to be missing from, and an empty ``__all__`` is not the same thing as no ``__all__`` at all: the first exports nothing, while the second exports every name that doesn't start with an underscore. Creating one would silently change what ``from import *`` gives you. .. invisible-code-block: pycon >>> del __all__ .. code-block:: pycon >>> @private ... class Oboe: ... pass >>> '__all__' in globals() False If the ``__all__`` does exist, it must be a list, just as it must be for ``@public``. ``private()`` accepts the same single argument call form as ``public()`` does, resolving the name the same way, and removing it from the ``__all__`` of the module where the call appears: .. invisible-code-block: pycon >>> reset() .. code-block:: pycon >>> from strings import Bass >>> public(Bass) >>> print(__all__) ['Bass'] >>> private(Bass) >>> print(__all__) [] There is no keyword argument form for ``private()``. Inferring __all__ ================= If you don't like using the decorators, you can instead infer and populate the contents of ``__all__`` by calling the ``populate_all()`` function at the bottom of your module. This uses heuristics to pick out some names from the module, adding them to ``__all__`` if they meet the following criteria: * The name does not start with an underscore. * The name is not bound to a module object. This prevents imported modules from being added [#]_. * The object the name is bound to does not appear to be defined in some other module. This prevents most from-imports from being added, but note that this can be fooled if you import simple types (such as an ``int`` or a ``str``) from another module (e.g. ``from sys import abiflags``), because simple types don't have a ``__module__`` attribute. For example, if your Python module looks like this: .. literalinclude:: popall_example.py :language: python :linenos: when you import this module, the ``__all__`` will be populated with the names matching the above heuristics. .. invisible-code-block: pycon >>> example = import_example('popall_example.py') .. code-block:: pycon >>> example.__all__ ['tune', 'Bass', 'tempo'] In this case, you can see that the module has an ``__all__`` set to ``['tune', 'Bass', 'tempo']`` but note that neither ``_muted`` nor ``populate_all`` are added. If the inferencing misses some names you want to publicly export, you can always add them explicitly by using the function call form ``public()``. .. note:: ``populate_all()`` only adds new names to ``__all__``. Installation ============ The package is on PyPI under the name ``atpublic``: .. code-block:: console $ pip install atpublic If you want, you can install ``@public`` and ``@private`` into your :mod:`builtins` module so they're always available without having to import them first. The easy way to do this is to depend on the ``atpublic[install]`` dependency (i.e. with the ``install`` extra) instead of just ``atpublic``: .. code-block:: console $ pip install atpublic[install] The extra pulls in the companion ``atpublic-install`` package, which adds the decorators to :mod:`builtins` as the interpreter starts up, so every module can use them without importing anything. Alternatively, you can call the install method manually: .. code-block:: python from public import install install() Caveats ======= There are some important usage restrictions you should be aware of: * Only use ``@public`` and ``@private`` on top-level objects. Specifically, don't try to use either decorator on a class method name. While the declaration won't fail, you will get an exception when you attempt to ``from import *`` because the name pulled from ``__all__`` won't be in the module's globals. * If you explicitly set ``__all__`` in your module, be sure to set it to a list. Some style guides require ``__all__`` to be a tuple, but since that's immutable, as soon as ``@public`` tries to append to it, you will get an exception. Best practice is to not set ``__all__`` explicitly; let ``@public`` and ``@private`` do it! * If you still want ``__all__`` to be immutable, put the following at the bottom of your module: .. code-block:: python __all__ = tuple(__all__) .. [#] `PEP 844 `_, and Python `issue 26632`_ propose adding this to Python itself. .. [#] This ordering is guaranteed by `PEP 468 `_. .. [#] Thus for so called "hub" modules (where names in submodules are exported in a package's top level ``__init__.py``), you would need to explicitly add the imported modules you want to export. The single argument call form does exactly this, e.g. ``public(submodule)``. .. _`issue 26632`: https://bugs.python.org/issue26632 .. _builtins: https://docs.python.org/3/library/builtins.html .. _`directly controls`: https://docs.python.org/3/tutorial/modules.html#importing-from-a-package .. _`actively prohibited`: https://pep8.readthedocs.io/en/latest/intro.html?highlight=e402#error-codes .. _`out of sync`: https://bugs.python.org/issue23883 .. _`this bug`: https://bugs.python.org/issue22247#msg225637 .. _Sphinx: https://www.sphinx-doc.org/en/master/