You have no idea how much time I wasted on Python documentation. I knew what the method did but couldn’t remember the name of it. 80% of my time was spent on figuring out the method names, without wasting a lot of time writing code and then figuring out the method names again. This article details how you can convert python code to documentation.
Have you ever been curious about how to convert python code to documentation or about python documentation methods? Well here’s your chance. I’m going to go over the best ways to manually convert your python code into documentation. Since that sounds rather technical, we’ll cover a very specific case that should give you a better understanding on how you can do it yourself–for yourself!
Document your Python code in a readable, but more compact way. Use your Python files to create beautiful, presentable documentation. Pydoc-Markdown generates a Markdown file from Python source code, allowing beautiful documentation to be generated via MkDocs or MkDocs Material using the Markdown file written by this tool.
Writing documentation is one of the most challenging and technical projects for software developers. There are numerous tools available, but none that fully support the way teams write Python code. In this article, we’ll teach you how to convert Python code to documentation automatically.
Commenting vs Documenting Code
Before we can go into how to document your Python code, we need to distinguish documenting from commenting.
In general, commenting is describing your code to/for developers. The intended main audience is the maintainers and developers of the Python code. In conjunction with well-written code, comments help to guide the reader to better understand your code and its purpose and design:
“Code tells you how; Comments tell you why.”
— Jeff Atwood (aka Coding Horror)
Documenting code is describing its use and functionality to your users. While it may be helpful in the development process, the main intended audience is the users. The following section describes how and when to comment your code.
pdoc
Auto-generate API documentation for Python projects
Ever finished a neat Python script only to see it later lay unused by anyone at all because the only documenting effort you gave it was the petty README.md? Ever found Sphinx too cumbersome to use for the small project on your hands or have repeatedly found reST syntax for links too volatile to touch-type, whereas you had always loved and adored Markdown? Many have. Documentation is a PITA to maintain. Real programmers don’t document their programs — documentation is for simpletons who can’t read object code from memory dumps. Right? Wrong.
Documentation is like sex: when it is good, it is very, very good;
and when it is bad, it is better than nothing.— Dick B.
Enter pdoc, the perfect documentation generator for small-to-medium-sized, tidy Python projects. It generates documentation simply from your project’s already-existing public modules’ and objects’ docstrings, like sphinx-apidoc or sphinx.ext.autodoc, but without the hassle of these tools. Minimal and lightweight. Guaranteed 99% correct magic out of the box!
- Compatible with Python 3+Python 3.6 and up. Best choice for new or forward-looking projects.
- Common docstring formatsOut-of-the-box support for markdown, numpydoc, Google-style docstrings, Napoleon, doctests, and some common reST directives.
- Zero configurationPdoc works automatically, following common Python programming conventions. Simply run the command line script, ecstatically nodding in agreement.
- Docs hierarchy follows module hierarchyUse module docstrings to introduce topics. Literate programming as Guido intended it.
- Customizable templatesPdoc uses fast mako templating engine. The default templates should suit most cases, but they can easily be customized, extended or overridden.
- Easy cross-linking of identifiersSimply wrap object references in backticks and let the technology do all the rest.
- Integrated dev web serverInstant feedback on your doc writing skills — see it as you write it.
- DependableUnmatched identifiers and source errors raise loud warnings. Reliably build documentation in your CI pipeline.
- Plain text, HTML, or PDF outputFunctionally laid out for every kind of eye.
How to generate a documentation using Python?
Documentation improves the readability of the code. There are many tools that help us to create documentations. One such tool is pdoc to write documentation for python projects.
Installation: Run the following pip command on the terminal.
pip3 install pdoc3
Now navigate (through command line) to the folder where our Python program is kept. Let’s say the name of our program is Numbers.py. It is a simple class that has 2 functions, one to find the factorial of the number and another to check whether the number is a prime number or not.
Python3
class Numbers : """ This class will be used to perform operations on numbers. To find out their factorial, check whether the number is prime """ def isPrime(self, number) : ''' A method to check whether a number is prime ''' i = 2 while(i <= number / 2) : if number % i == 0: return False i += 1 return True def factorial(self, number) : """ A function that finds out the factorial of numbers """ result = 1 while(number >= 2) : result *= number number -= 1 return result |
Adding docstrings in classes and methods provide a convenient way of associating documentation with Python modules, functions, classes, and methods. Docstrings are an important part of the documentation.
After navigating to the folder(through command line) where our code (Numbers.py) is kept. We have to type this command in the terminal:
pdoc --html Numbers.py
Directory structure
After firing the –html command shown above, an HTML folder will be created in the directory where Numbers.py is kept. Inside that HTML folder, a file called Numbers.html will be created. Navigate to HTML folder and open Numbers.html.
Numbers.html
Numbers.html should be looking like this. We just created proper documentation of our class Numbers. In the left column(Index) of Number.html, classes and it’s methods are seen. Remember the docstrings which we wrote, they are available below their respective classes and methods.
Five Tips for Automatic Python Documentation
Create beautiful Python documentation in MkDocs & Material with these five automation steps and pre-commit Git hooks
In this story, you will learn how to automatically generate documentation from Python modules with a bit of magic in our custom functions, the package mkgendocs, pre-commit Git hooks, and MkDocs. We will touch upon the following elements
- MkDocs & Material installation
- Automate type-hints to docstrings
- Automate docstrings to MkDocs with
mkgendocs - Automate the documentation of new Python functions
- Tie everything together in pre-commit Git hooks
MkDocs & Material installation
MkDocs is a static site generator for building project documentation and together with the Material framework, it simply looks gorgeous. First, we need to install a heap of packages in order to use all of the functionalities of MkDocs. All of these packages are pip-installable.
MkDocs uses a configuration file mkdocs.yml, where you can enable all of the functionalities and packages installed above. Please find mine here. It includes references to the /docs and /docs_assets folders with the theme.
Automate type-hints to docstrings
Previously, I wrote on the importance of writing docstrings, with a focus on Sphinx documentation.
Docstrings are an essential tool to document your functions. Python 3.5+ introduced type-hints, a way to assign static types to variables directly in the function arguments.
Several IDEs such as Pycharm, Visual Studio, and Sublime Text support automatic docstring generation. They do not however infer variable types from type-hints yet, which means that you have to fill both the variable type and descriptions in the docstrings.
Shown above is the implementation in Pycharm with Google-style docstrings. You are free to use other styles (such as reStructuredText/Sphinx or NumPy), but I found a package that exclusively works with Google-style docstrings for our next automation steps.
Automate docstrings to MkDocs
The package mkgendocs automatically translates Google-style docstrings into pages with the description of Python functions. It uses a configuration file mkgendocs.yml. The configuration file looks like this
sources_dir: docs
templates_dir: docs/templates
repo: https://github.com/LouisdeBruijn/Medium
version: masterpages:
- page: "scripts/base/base.md"
source: "Python tips/base.py"
functions:
- parse_arguments
- print_df
- unescape_html
- equal_array_items
Two manual steps for the use of this package are
- Add the pages, sources, and functions to be documented to this
mkgendocs.ymlfile. - Run
$ gendocs --config mkgendocs.ymlto create the static MkDocs pages with the documentation of these functions.
Next up, we will automate both steps by first creating a script to pre-fill our configurations file, and next attach both steps in a pre-commit Git hook.
Automate the documentation of new functions
First, I wrote a module automate.py with a function automate_mkdocs_from_docstring() to fill the mkgendocs.yml configurations file with all the Python functions in modules (scripts) in a repository.
automate_mkdocs_from_docstring() uses Pathlib to read the Python scripts in a directory and extract the function names. It saves both the module and the function names in a dictionary and uses this to overwrite the mkgendocs.yml. This way we can automatically fil the configurations file for the mkgendocs package.
Second, we will include both the execution of automate.py and the $ gendocs --config mkgendocs.yml in a pre-commit Git hook.
Tie everything together in pre-commit Git hooks
Our last step is to add each automation step into a Git hook in the pre-commit-config.yaml.
We’re adding the two mandatory steps for the automatic generation of MkDocs pages with the mkgendocs package in a pre-commit Git hook. These hooks allow us to execute a Python script with an entry bash command.
This way, when we create or remove Python functions anywhere in our repository, they will automatically be added to the mkgendocs configuration file, automatically creating static MkDocs pages and building the documentation website.
The only thing left to do is to run our MkDocs site with $ mkdocs serve and find our documentation at http://127.0.0.1:8000/.
Documentation Tools and Resources
Documenting your code, especially large projects, can be daunting. Thankfully there are some tools out and references to get you started:
| Tool | Description |
|---|---|
| Sphinx | A collection of tools to auto-generate documentation in multiple formats |
| Epydoc | A tool for generating API documentation for Python modules based on their docstrings |
| Read The Docs | Automatic building, versioning, and hosting of your docs for you |
| Doxygen | A tool for generating documentation that supports Python as well as multiple other languages |
| MkDocs | A static site generator to help build project documentation using the Markdown language |
| pycco | A “quick and dirty” documentation generator that displays code and documentation side by side. Check out our tutorial on how to use it for more info. |
Along with these tools, there are some additional tutorials, videos, and articles that can be useful when you are documenting your project:
- Carol Willing – Practical Sphinx – PyCon 2018
- Daniele Procida – Documentation-driven development – Lessons from the Django Project – PyCon 2016
- Eric Holscher – Documenting your project with Sphinx & Read the Docs – PyCon 2016
- Titus Brown, Luiz Irber – Creating, building, testing, and documenting a Python project: a hands-on HOWTO – PyCon 2016
- reStructuredText Official Documentation
- Sphinx’s reStructuredText Primer
Conclusion
One of the trick factors to software development is the documentation. Documented code makes it easier for new programmers to understand the code they will be working with. While using Python, if you really want to write good documentation, then you need a set of tools and methods to convert your Python code into point wise instructions.
A lot of beginner python developers make a mistake of not documenting their code properly. They don’t add the documentation string in the python code and leave it at that. The python documentation strings are really really helpful because they provide a quick reminder on what a function does or how to use a class. They are also really easy to read and write as well.