Visual Studio Code Auto Documentation Python

Python code auto documentation has been a popular feature request for several years. A lot of us would like to have it out of the box, but there is a reason why it needs to be turned off by default. There are two kinds of python code auto documentation and they do not work 100% reliably. This article describes both, how they work and what you can do to generate your own docs from code at least in one way.

Python auto documentation is a tool that creates documentation (using Sphinx, an open source writing tool) from all the comments in the code itself. Python auto documentation means that we can have code documentation generated by the program itself. What this means is that once we have some procedure to help us add comments to our python code, python generate docs from code might not be far behind.

In this article I will be discussing python code auto documentation which is a technique in software development that creates supplementary documentation from the code of programs written in the Python programming language. This framework is designed to provide assistance to new users and users unfamiliar with a particular library or module. The main principle of the framework is explaining how to use existing libraries and giving a guide on how to maintain different aspects of it.

Python code can be very well documented through docstrings, but sometimes you want to go a step further. You want to generate your own API documentation for your API endpoints. If this is the case, then you have come to the right place. This is a guide on how to auto-generate your API documentation using Python.

Five Tips for Automatic Python Documentation

Create beautiful Python documentation in MkDocs & Material with these five automation steps and pre-commit Git hooks

In this story, you will learn how to automatically generate documentation from Python modules with a bit of magic in our custom functions, the package mkgendocs, pre-commit Git hooks, and MkDocs. We will touch upon the following elements

• MkDocs & Material installation
• Automate type-hints to docstrings
• Automate docstrings to MkDocs with mkgendocs
• Automate the documentation of new Python functions
• Tie everything together in pre-commit Git hooks

MkDocs & Material installation

MkDocs is a static site generator for building project documentation and together with the Material framework, it simply looks gorgeous. First, we need to install a heap of packages in order to use all of the functionalities of MkDocs. All of these packages are pip-installable.

MkDocs uses a configuration file mkdocs.yml, where you can enable all of the functionalities and packages installed above. Please find mine here. It includes references to the /docs and /docs_assets folders with the theme.

Automate type-hints to docstrings

Previously, I wrote on the importance of writing docstrings, with a focus on Sphinx documentation.

Docstrings are an essential tool to document your functions. Python 3.5+ introduced type-hints, a way to assign static types to variables directly in the function arguments.

Several IDEs such as Pycharm, Visual Studio, and Sublime Text support automatic docstring generation. They do not however infer variable types from type-hints yet, which means that you have to fill both the variable type and descriptions in the docstrings.

Shown above is the implementation in Pycharm with Google-style docstrings. You are free to use other styles (such as reStructuredText/Sphinx or NumPy), but I found a package that exclusively works with Google-style docstrings for our next automation steps.

Automate docstrings to MkDocs

The package mkgendocs automatically translates Google-style docstrings into pages with the description of Python functions. It uses a configuration file mkgendocs.yml. The configuration file looks like this

sources_dir: docs
templates_dir: docs/templates
repo: https://github.com/LouisdeBruijn/Medium
version: masterpages:
  - page: "scripts/base/base.md"
    source: "Python tips/base.py"
    functions:
      - parse_arguments
      - print_df
      - unescape_html
      - equal_array_items

Two manual steps for the use of this package are

1. Add the pages, sources, and functions to be documented to this mkgendocs.yml file.
2. Run $ gendocs --config mkgendocs.yml to create the static MkDocs pages with the documentation of these functions.

Next up, we will automate both steps by first creating a script to pre-fill our configurations file, and next attach both steps in a pre-commit Git hook.

Automate the documentation of new functions

First, I wrote a module automate.py with a function automate_mkdocs_from_docstring() to fill the mkgendocs.yml configurations file with all the Python functions in modules (scripts) in a repository.

automate_mkdocs_from_docstring() uses Pathlib to read the Python scripts in a directory and extract the function names. It saves both the module and the function names in a dictionary and uses this to overwrite the mkgendocs.yml. This way we can automatically fil the configurations file for the mkgendocs package.

Second, we will include both the execution of automate.py and the $ gendocs --config mkgendocs.yml in a pre-commit Git hook.

Tie everything together in pre-commit Git hooks

Our last step is to add each automation step into a Git hook in the pre-commit-config.yaml.

We’re adding the two mandatory steps for the automatic generation of MkDocs pages with the mkgendocs package in a pre-commit Git hook. These hooks allow us to execute a Python script with an entry bash command.

This way, when we create or remove Python functions anywhere in our repository, they will automatically be added to the mkgendocs configuration file, automatically creating static MkDocs pages and building the documentation website.

The only thing left to do is to run our MkDocs site with $ mkdocs serve and find our documentation at http://127.0.0.1:8000/.

Pros of creating good documentation:

1. Increases information exchange between team members— this single reason is just so powerful!
2. Decreases onboarding time of new members

3. Helps to organize big projects (helps to see the big picture)

4. Increases team member awareness of how the whole project is organized

5. Increases development speed — finding information is faster and thus development is faster

6. Promotes standards and consistency

Cons of creating documentation?

1. Requires time (and money) — sometimes a project can’t afford to spend time on documentation.

2. It’s hard to keep it up-to-date, especially in startup projects with rapid changes

3. Creating documentation is not a “pleasant” activity for the developers (compared to creating code) — some developers don’t like to create the documentation, they will be demotivated when asked to do it.

Cons of bad documentation?

1. “Out-of-date” documentation can lead to misunderstandings and slower development

2. Can get fragmented — it’s hard to maintain one, consistent documentation.

Taking the pros and cons into account it would seem sensible to either create good, up-to-date documentation or not create it at all. But there are tools that can help and decrease the human factor. Autogenerated documentation tools require less effort from people and can create valuable documentation automatically.

 Three types of autogenerated documentation:

By autogenerated I mean documentation which is built with some tool (maybe cli) and the output is instantly browsable (in a form of a web page, a pdf etc..). We can distinguish 3 types of these:

• fully automatic — no need to do anything manually
• semi-automatic — it works without doing anything manually, but it can be better when you put in some extra effort
• manual — you write the documentation by yourself, the output webstie/pdf is generated automatically

Swagger UI — the fully automatic solution

Swagger delivers multiple tools:

• Swagger Editor — helps design an API
• Swagger UI — creates an API documentation
• Swagger Codegen — generates server stubs and client SDKs for an API

All are licensed under Apache 2.0 license.

I’ll focus only on Swagger UI. What does it do?

Swagger UI allows anyone — be it your development team or your end consumers — to visualize and interact with the API’s resources without having any of the implementation logic in place. It’s automatically generated from your OpenAPI (formerly known as Swagger) Specification, with the visual documentation making it easy for back end implementation and client side consumption.

So it automatically documents, visualizes and helps interact with an API. Try the Petstore demo page

Overview of the API:

You can see what API namespaces/groups are defined (pet, store, user) and see if the API handles http/https schemes.

When you click the “Authorize” button:

The “Value” and “client_id” inputs are interactive — you can type a value and click “Authorize” (there is no real authorization behind it, it’s just a stub).

When you click on the group name (“pet”), you get a list of all the endpoints defined:

You can see the URL, HTTP verb and path parameters. The lock icon on the right side highlights authorization options.

The greyed-out and strikethrough endpoint is deprecated.

(* For SEO reasons I would recommend creating URLs with hyphens instead of camelCase)

Expanding a GET endpoint:

The “status” parameter is documented and has available values listed.

The response lists possible HTTP codes (200,400) and an example JSON response.

Clicking “Try it” enables interactive mode in which you can modify the parameter, execute the query and see results, that’s nice!

Models view — you can see the nesting and types:

Swagger creates this beautiful documentation automagically, but your API needs to follow the OpenAPI Specification (originally known as the Swagger Specification).

Installation:

The installation will vary depending on how you serve your API. If you are using a framework, then a ready-to-go library:

• Django REST Swagger
• Flask-RESTPlus
• falcon-swagger-ui
• FastAPI has Swagger-UI built in — no need for external libraries

Or you can try a standalone installation.

Documentation

Readability is a primary focus for Python developers, in both project and code documentation. Following some simple best practices can save both you and others a lot of time.

Project Documentation

A README file at the root directory should give general information to both users and maintainers of a project. It should be raw text or written in some very easy to read markup, such as reStructuredText or Markdown. It should contain a few lines explaining the purpose of the project or library (without assuming the user knows anything about the project), the URL of the main source for the software, and some basic credit information. This file is the main entry point for readers of the code.

An INSTALL file is less necessary with Python. The installation instructions are often reduced to one command, such as pip install module or python setup.py install, and added to the README file.

A LICENSE file should always be present and specify the license under which the software is made available to the public.

A TODO file or a TODO section in README should list the planned development for the code.

A CHANGELOG file or section in README should compile a short overview of the changes in the code base for the latest versions.

Project Publication

Depending on the project, your documentation might include some or all of the following components:

• An introduction should give a very short overview of what can be done with the product, using one or two extremely simplified use cases. This is the thirty-second pitch for your project.
• A tutorial should show some primary use cases in more detail. The reader will follow a step-by-step procedure to set-up a working prototype.
• An API reference is typically generated from the code (see docstrings). It will list all publicly available interfaces, parameters, and return values.
• Developer documentation is intended for potential contributors. This can include code convention and general design strategy of the project.

Conclusion

Automatically generated documentation has been a long desired feature and a cornerstone of many programming languages, however it is often seen as a very complex or expensive addition to a language. In fact, Python is extremely successful in providing documentation from code that can otherwise be manually written with little gain in terms of efficiency or clarity.

Python is a great language for writing multi-purpose scripts, due to its flexibility and power. The documentation can be very clear, but this clarity can sometimes turn into verbosity. Writing documentation for a Python project can be a tedious process, especially if you don’t have any standard code comments in place. For smaller projects it is easy to keep track of what methods are available and their parameters. But as your modules grow it can become increasingly difficult to remember the last time you added a new function, or how exactly it’s meant to work.