Many coders are aware of Python documentation methods and converting python code to documentation. I’m sure you’ve encountered the old saying, a picture is worth a thousand words. Programs with high quality have documentations of their codes. Python allows developers to convert python code to documentation that ultimately helps other understand its structure and working of code.
Documenting Python Code: A Complete Guide tells you everything you need to know to document your Python code. You will learn how to document both the API and your project, how to use the standard tools in Python for manipulating those docs and write them yourself. Documentation is an important part of software development, but it’s boring and hard. This book makes it easy.
Learning how to code can be the best choice you’ll ever make. However if you want your code to be able to communicate with other programmers, you need to learn how to document it properly. Python code documentation is a method of using spacing, formatting and commenting in your python code to create easily readable documentation. If you have any questions about python documentation, please feel free to ask in the comment section!
Python style is now a popular choice for many projects. It’s readable, concise, and not overly verbose. You can also use the python code documentation to keep track of changes in your project.
Where Do I Start?
The documentation of projects have a simple progression:
- No Documentation
- Some Documentation
- Complete Documentation
- Good Documentation
- Great Documentation
If you’re at a loss about where to go next with your documentation, look at where your project is now in relation to the progression above. Do you have any documentation? If not, then start there. If you have some documentation but are missing some of the key project files, get started by adding those.
pdoc3 — the semi-automatic solution
Python package
pdocprovides types, functions, and a command-line interface for accessing public documentation of Python modules, and for presenting it in a user-friendly, industry-standard open format […]
pdocextracts documentation of:– modules (including submodules),
– functions (including methods, properties, coroutines …),
– classes, and
– variables (including globals, class variables, and instance variables)
pdoconly extracts public API documentation ([…] if their identifiers don’t begin with an underscore ‘_’)
So pdoc takes you code (modules, functions/methods, classes, variables) and creates a browsable (html/plaintext) documentation. It’s semi-automatic because it uses your code to create the main docs, but it will add more useful info if you have docstrings.
Other features:
- Docstrings for objects can be disabled, overridden, or whitelisted with a special module-level dictionary
__pdoc__ - Supports multiple docstring formats: pure Markdown (with extensions), numpydoc, Google-style and some reST directives
- LaTeX math syntax is supported when placed between recognized delimiters
- Linking to other identifiers in your modules
- Programmatic usage — control
pdocusing Python - Custom templates – override the built-in HTML/CSS or plaintext
- With CLI params you can: change the output directory, omit the source code preview, target documentation at specific modules, filter identifiers that will be documented
- Create output formatted in Markdown-Extra, compatible with most Markdown-(to-HTML-)to-PDF converters
- Local HTTP server (*it was throwing exceptions for me)
- Requires Python 3.5+
- License GNU AGPL-3.0 (*make sure you double-check how you use pdoc3 in a commercial product, read more)
Installation:
pip install pdoc3
Usage (inside your Python project):
pdoc --html .
This will create a directory called html containing another directory (named the same way as your project dir) and inside you will find .html files with your Python modules documented. Here is the output of pdoc ran on my example Python code
The index.htmlfile:
We have all our modules indexed on the left. What is important to notice pdoc also documented code without docstrings. This is huge — you don’t need to have docstrings and you will still benefit from pdoc.
No docstring code:
module_variable = 1
class NoDocStrings:
class_variable = 2
def __init__(self):
self.instance_variable = 3
def foo(self):
pass
def _private_method(self):
pass
def __name_mangled_method(self):
pass
def module_function():
pass
The result:
A class with docstrings:
class Foo:
"""
This is a docstring of class Foo
"""
class_variable = 3
"""This is a docstring for class_variable"""
def __init__(self):
self.instance_var_1 = 1
"""This is a docstring for instance_var_1"""
self.instance_var_2 = 2
def foo_method(self):
"""
This is a docstring for foo_method.
:param self:
:return:
"""
def bar_method(self):
"""
This is a docstring for bar_method.
:return:
"""
def _private_method(self):
"""
This is a docstring for _private_method
:return:
"""
def __name_mangled_method(self):
"""
This is a docstring for __name_mangled_method
:return:
"""
The result:
A class which has no docstrings, but inherits from a class with a docstring:
class InheritedFoo(Foo):
def foo_method(self):
pass
def bar_method(self):
"""This is an overwritten docstring for bar_method"""
pass
The result:
The bar_method docstring was overwritten.
Private and name-mangled methods are not documented but you can see them when you click “Expand source code”:
Nested classes:
class Baz:
"""
This is a docstring for class Baz
"""
class BazInner:
"""
This is a docstring for BazInner
"""
The result:
Inner class “BazInner” was indexed as a variable, wish pdoc would also indicate that it is a class 😉
Module-level variables and functions:
"""
This is a docstring for module_c
"""
module_variable = 100
"""
This is a docstring for module_variable
"""
def module_function():
"""
This is a docstring for module_function
:return:
"""
function_variable = 10
"""
This is a docstring for function_variable
"""
def _private_module_function():
"""
This is a docstring for _private_module_function
:return:
"""
def __name_mangled_function():
"""
This is a docstring for __name_mangled_function
:return:
"""
The result:
You can see more examples on pdoc3 docs page — they documented their own code with pdoc 🙂
MkDocs — the manual solution
MkDocs is a fast, simple and downright gorgeous static site generator that’s geared towards building project documentation. Documentation source files are written in Markdown, and configured with a single YAML configuration file.
Out of the 3 tools I’m describing this one is the least automatic, it only autogenerates a nice-looking documentation website. All of the content is created manually.
Features:
- Source files are written in Markdown
- Preview site with a dev server
- Host anywhere like GitHub pages or Amazon S3
- Plugins
- Custom themes (also check out the community themes — GitBook is my favourite)
- Python versions 3.5, 3.6, 3.7, 3.8, and pypy3
- MkDocs License — BSD, each theme may have its own license, i.e, ReadTheDocs theme is MIT-licensed.
Installation:
pip install mkdocs
Usage:
mkdocs new mkdocs_test
Result:
- mkdocs.yml — configuration
- index.md — the default docs page
To run the dev server:
mkdocs serve
And go to http://127.0.0.1:8000 (by default)
The server will auto-reload the page whenever you change the configuration or documented pages.
Adding a new page:
Create a .md file in docs/ dir and link it in the configuration file in nav section:
nav:
- Home: index.md
- About: about.md
You also get “search”, “previous”, “next” buttons for free.
Changing the theme is as easy as (in the config file):
theme: readthedocs
Building the site (in cli):
mkdocs build
This will create a static html site located int site directory.
The documentation site that you just built only uses static files so you’ll be able to host it from pretty much anywhere. GitHub project pages and Amazon S3 may be good hosting options, depending upon your needs.
Alternatives:
What other tools are available in the Python ecosystem that help with documentation:
- The offical Python documentation pages use reStructuredText (as markup language) and Sphinx, (*I find Markdown a bit simpler than rST but it’s a personal choice)
- Doxygen —generates documentation from annotated sources
- Portray — Python3 command-line tool and library that helps you create great documentation websites for your Python projects with as little effort as possible
- Pycco — Python port of Docco: the original quick-and-dirty, hundred-line-long, literate-programming-style documentation generator. It produces HTML that displays your comments alongside your code.
If you are creating an API then Swagger-UI is a must.
With very little effort you can create module/class/function documentation using pdoc3. If the developers write docstrings then you will benefit even more.
Writing manual documentation takes more time, but things like architecture overview, installation etc should be (at least briefly) described. MkDocs makes it easy to create simple and beautiful documentation.
Just remember that having some documentation is not an excuse for creating bad code. Self-documenting code is an absolute priority.
reStructuredText
Most Python documentation is written with reStructuredText. It’s like Markdown, but with all the optional extensions built in.
The reStructuredText Primer and the reStructuredText Quick Reference should help you familiarize yourself with its syntax.
Code Documentation Advice
Comments clarify the code and they are added with purpose of making the code easier to understand. In Python, comments begin with a hash (number sign) (#).
In Python, docstrings describe modules, classes, and functions:
def square_and_rooter(x):
"""Return the square root of self times self."""
...
In general, follow the comment section of PEP 8#comments (the “Python Style Guide”). More information about docstrings can be found at PEP 0257#specification (The Docstring Conventions Guide).
Commenting Sections of Code
Do not use triple-quote strings to comment code. This is not a good practice, because line-oriented command-line tools such as grep will not be aware that the commented code is inactive. It is better to add hashes at the proper indentation level for every commented line. Your editor probably has the ability to do this easily, and it is worth learning the comment/uncomment toggle.
Docstrings and Magic
Some tools use docstrings to embed more-than-documentation behavior, such as unit test logic. Those can be nice, but you won’t ever go wrong with vanilla “here’s what this does.”
Tools like Sphinx will parse your docstrings as reStructuredText and render it correctly as HTML. This makes it very easy to embed snippets of example code in a project’s documentation.
Additionally, Doctest will read all embedded docstrings that look like input from the Python commandline (prefixed with “>>>”) and run them, checking to see if the output of the command matches the text on the following line. This allows developers to embed real examples and usage of functions alongside their source code. As a side effect, it also ensures that their code is tested and works.
def my_function(a, b):
"""
>>> my_function(2, 3)
6
>>> my_function('a', 3)
'aaa'
"""
return a * b
Docstrings versus Block comments
These aren’t interchangeable. For a function or class, the leading comment block is a programmer’s note. The docstring describes the operation of the function or class:
# This function slows down program execution for some reason.
def square_and_rooter(x):
"""Returns the square root of self times self."""
...
Unlike block comments, docstrings are built into the Python language itself. This means you can use all of Python’s powerful introspection capabilities to access docstrings at runtime, compared with comments which are optimized out. Docstrings are accessible from both the __doc__ dunder attribute for almost every Python object, as well as with the built in help() function.
While block comments are usually used to explain what a section of code is doing, or the specifics of an algorithm, docstrings are more intended towards explaining other users of your code (or you in 6 months time) how a particular function can be used and the general purpose of a function, class, or module.
Writing Docstrings
Depending on the complexity of the function, method, or class being written, a one-line docstring may be perfectly appropriate. These are generally used for really obvious cases, such as:
def add(a, b):
"""Add two numbers and return the result."""
return a + b
The docstring should describe the function in a way that is easy to understand. For simple cases like trivial functions and classes, simply embedding the function’s signature (i.e. add(a, b) -> result) in the docstring is unnecessary. This is because with Python’s inspect module, it is already quite easy to find this information if needed, and it is also readily available by reading the source code.
In larger or more complex projects however, it is often a good idea to give more information about a function, what it does, any exceptions it may raise, what it returns, or relevant details about the parameters.
For more detailed documentation of code a popular style used, is the one used by the NumPy project, often called NumPy style docstrings. While it can take up more lines than the previous example, it allows the developer to include a lot more information about a method, function, or class.
def random_number_generator(arg1, arg2):
"""
Summary line.
Extended description of function.
Parameters
----------
arg1 : int
Description of arg1
arg2 : str
Description of arg2
Returns
-------
int
Description of return value
"""
return 42
The sphinx.ext.napoleon plugin allows Sphinx to parse this style of docstrings, making it easy to incorporate NumPy style docstrings into your project.
At the end of the day, it doesn’t really matter what style is used for writing docstrings; their purpose is to serve as documentation for anyone who may need to read or make changes to your code. As long as it is correct, understandable, and gets the relevant points across then it has done the job it was designed to do.
For further reading on docstrings, feel free to consult PEP 257
Other Tools
You might see these in the wild. Use Sphinx.PyccoPycco is a “literate-programming-style documentation generator” and is a port of the node.js Docco. It makes code into a side-by-side HTML code and documentation.RonnRonn builds Unix manuals. It converts human readable textfiles to roff for terminal display, and also to HTML for the web.EpydocEpydoc is discontinued. Use Sphinx instead.MkDocsMkDocs is a fast and simple static site generator that’s geared towards building project documentation with Markdown.
Reusing signatures and docstrings with autodoc
To use autodoc, first add it to the list of enabled extensions:docs/source/conf.py
extensions = [
'sphinx.ext.duration',
'sphinx.ext.doctest',
'sphinx.ext.autodoc',
]
Next, move the content of the .. py:function directive to the function docstring in the original Python file, as follows:lumache.py
def get_random_ingredients(kind=None):
"""
Return a list of random ingredients as strings.
:param kind: Optional "kind" of ingredients.
:type kind: list[str] or None
:raise lumache.InvalidKindError: If the kind is invalid.
:return: The ingredients list.
:rtype: list[str]
"""
return ["shells", "gorgonzola", "parsley"]
Finally, replace the .. py:function directive from the Sphinx documentation with autofunction:docs/source/usage.rst
you can use the ``lumache.get_random_ingredients()`` function: .. autofunction:: lumache.get_random_ingredients
If you now build the HTML documentation, the output will be the same! With the advantage that it is generated from the code itself. Sphinx took the reStructuredText from the docstring and included it, also generating proper cross-references.
You can also autogenerate documentation from other objects. For example, add the code for the InvalidKindError exception:lumache.py
class InvalidKindError(Exception):
"""Raised if the kind is invalid."""
pass
And replace the .. py:exception directive with autoexception as follows:docs/source/usage.rst
or ``"veggies"``. Otherwise, :py:func:`lumache.get_random_ingredients` will raise an exception. .. autoexception:: lumache.InvalidKindError
And again, after running make html, the output will be the same as before.
Generating comprehensive API references
While using sphinx.ext.autodoc makes keeping the code and the documentation in sync much easier, it still requires you to write an auto* directive for every object you want to document. Sphinx provides yet another level of automation: the autosummary extension.
The autosummary directive generates documents that contain all the necessary autodoc directives. To use it, first enable the autosummary extension:docs/source/conf.py
extensions = [ 'sphinx.ext.duration', 'sphinx.ext.doctest', 'sphinx.ext.autodoc', 'sphinx.ext.autosummary', ]
Next, create a new api.rst file with these contents:docs/source/api.rst
API === .. autosummary:: :toctree: generated lumache
Remember to include the new document in the root toctree:docs/source/index.rst
Contents -------- .. toctree:: usage api
Finally, after you build the HTML documentation running make html, it will contain two new pages:
api.html, corresponding todocs/source/api.rstand containing a table with the objects you included in theautosummarydirective (in this case, only one).generated/lumache.html, corresponding to a newly created reST filegenerated/lumache.rstand containing a summary of members of the module, in this case one function and one exception.
Each of the links in the summary page will take you to the places where you originally used the corresponding autodoc directive, in this case in the usage.rst document.
Note
The generated files are based on Jinja2 templates that can be customized, but that is out of scope for this tutorial.
Conclusion
Generating HTML Documentation from a single source file can be more useful because it puts all of the relevant information in one place. Doxygen is the most popular solution for generating documentation from annotated C++ sources, but it also supports other popular programming languages such as C, Objective-C, C
Nowadays, Python is the most popular language for coding. No wonder that scripters prefer it to implement any application or website. In fact, Python has long become the standard programming language which comes with a wide range of features and functionality. However, not so many people are aware of Python code documentation.