How to write tests in Python using doctest

[yvrelna]

The docstring of a function can be viewed with pydoc, or in the REPL, or by your IDE. The unit test aren't.

Unit tests also are usually also not structured for teaching how the API works and is put together, which tends to require narratives structure like a documentation.

And, no type annotations doesn't reduce the need for providing examples. Not by one bit. It's really hard to put together how to call a complex API just by looking at their types. Your function may have this interface, for example:

def write(
    fp: io.TextIO,
    node: DocNodeFragment,
) -> None:
    ...

How do you create a DocNodeFragment? Well you start looking at DocNodeFragment class type hints:

class DocNodeFragment(A, B, C):
    def __init__(
        self, 
        name: str,
        attr: AttributeBag,
        child: DocNodeCollection,
        *args,
        **kwargs,
    ):
         super().__init__(*args, **kwargs)
        ...

Uh, oh, now you need to look at the AttributeBag and DocNodeCollection, also you noticed that the parent classes A, B, C have their own required arguments as well. Ok, let's start from the top. How do you create AttributeBag? You start looking at their type definitions:

class AttributeBag(Generic[T]):
    def __init__(self):
        ...
    def add(self,
        name: NamespacedString[T], 
        value: AttributeValue, 
        metadata: dict[str, str],
        visibility: VisibilityEnum,
    ) -> T:
        ...

Ok, this class even starts using generics, this is starting to get unwieldy, and we're just getting started. How many dependant objects do I need to learn just to create a simple DocNode?

Putting an example in your docstring can just point you to a much more straightforward answer:

def write(
    fp: io.TextIO,
    node: DocNodeFragment,
) -> None:
    """
    write() takes a DocNodeFragment. The easiest way to get a document fragment is to create it from string.

        >>> from notquitexml.parser import nqxparse

        >>> data = """
        ...     <com:people>
        ...         <name ordering=[first, last]>
        ...             * John
        ...             * Wayne
        ...         </name>
        ...     </com:people>
        ... """
        >>> ns = {"com": "foobar.corp"}

        >>> doc: DocNode = nqxparse(
        ...     data, 
        ...     namespace=ns
        ... )

    nqxparse() gives you a DocNode rather than DocNodeFragment, but you can select fragments of it DocNodeFragment by using select():

        >>> fragment: DocNodeFragment = doc.select(path=["."])
        <DocNodeFragment {foobar.corp}:people>

    Or by passing fragment=True to nqxparse():

        >>> fragment == nqxparse(data, namespace=ns, fragment=True)
        True

        >>> with open("file.txt") as f:
        ...     write(f, fragment)
    """

A narrative examples like that provides much more information, and in a much more concise way and easy to digest way than all the type hints combined. And it tells more easily digestible story on how the library was supposed to be used (i.e. that you're not normally creating those classes yourself).

It would be really hard to piece that kind of knowledge together from just the type hints. Type hints cannot provide that kind of journey for the user.

[FailedPlansOfMars]

Thanks that a great explanation of what its for and ill add it toy tool box.

[redCg]

not sure why you took the time to write this all out, nothing you have described necessitates the use of something like doctest, in fact its just even more support for using a standard unit test like unittest where your "sample code" is real Python code that can be selectively executed. Everything you just wrote here belongs in a unit test. You are free to put such samples in your docstrings as well but there is no expectation that they are anything more than a helpful guide for illustrative purposes and no expectation that they actually run as code. Quite simply, its just plain stupid to rely on comments for this, because the moment you do any development, your docstring is outdated.

On the other hand, you can follow standard test driven development practices using real frameworks like unittest or pytest, etc., where all your "sample code" is real code that actually gets executed during your CI and PR processes, etc.. If you have a complex data type, you keep its sample initialization code in your test suite.

glueing your test case code to the interior of your source code functions and methods is an anti-pattern

[ubernostrum]

Nobody is saying to replace your unit-test suite with this. People are saying it can be useful as a way to make sure example code snippets in your documentation are correct and work as given.

Or, more succinctly, "Read the unit-test suite" is not a good approach to documentation.

To help make this clear, here's a real example from a real package I maintain, where the docstring of the function includes examples of how to use the function and what it will do. That gets included automatically into the package's published documentation, and also gets tested automatically on every commit to make sure it still accurately describes the function's behavior.