Tips and Tricks

The Cookiecutter for Spatial Data Science template has many features that can help you get started quickly and efficiently. Here are some tips and tricks to make the most out of it.

MkDocs

Documentation is built using MkDocs with a few extensions.

MkDocs: Writing Your Docs - this is a great place to start understanding how to write and structure your documentation
MkDocStrings: Usage - Extension creating docstrings directly from docstrings in the Python package built with your project. This is configured to use Google docstring conventions.
MkDocs-Jupyter - Extension enabling inclusion of Notebooks directly in the documentation.
MkDocs-Material - Theme used for the documentation. Useful information for customizing the theme if you want.
Admonitions - How to add Notes, etc.

Documentation layout

Files in the ./docsrc directory are used to build the documentation. The following files are included by default.

mkdocs.yml                    # MkDocs configuration file. This is where navigation is set up.
mkdocs/
    index.md                  # Documentation homepage.
    api.md                    # API (Python package) documentation generated from docstrings using MkDocStrings
    notebooks/                # Directory to put Jupyter Notebooks
    ...                       # Other markdown pages, images and files.

Note

The structure of the documentation pages is derived directly from the way files are organized in this directory. This is well explained in the MkDocs: File Layout documentation.

Notebooks

Any Jupyter Notebooks located in ./docsrc/mkdocs/notebooks will be converted into documentation pages able to be included in your table of contents specified in ./docsrc/mkdocs.yml. You will need to manually move any Jupyter Notebooks you want included in the documentation into this directory.

Note

I used to automatically copy Jupyter Notebooks from ./notebooks into the documentation, but this created two problems. First, a LOT of the notebooks were copied, which were not needed in the documentation. Second, frequently I did something to alter the Notebook I did not really want in the documentation.

Hence, to avoid these two issues, now the template requires deliberately moving the Jupyter Notebooks you want to include in the documentation from ./notebooks to ./docsrc/mkdocs/notebooks.

Logging

As a best practice, it is recommended to set up logging for your application using the :get_logger function. This ensures logging is properly routed to the console, logfile and ArcPy messaging as appropriate. For example, in each module of your application, you should set up a logger like this:

from {{cookiecutter.support_library}}.utils import get_logger

logger = get_logger(__name__, level='DEBUG', add_stream_handler=False)

This ensures logging is consistent across your application and can be easily managed. Then, when you create a script in the scripts diretory, you can configure the root logger as needed for that script's execution context.

import datetime
from pathlib import Path

from {{cookiecutter.support_library}}.utils import get_logger

# get the path to a directory to store logfiles - assuming script is in scripts directory
script_pth = Path(__file__)
dir_prj = script_pth.parent.parent
dir_logs = dir_prj / 'data' / 'logs'

# ensure the log directory exists
if not dir_logs.exists():
    dir_logs.mkdir(parents=True)

# get the name of the scritp without the .py extension
script_name = script_pth.stem

if __name__ == "__main__":

    # define the logfile path with a timestamp - enables unique logfile per execution
    logfile_path = dir_logs / f'{script_name}_{datetime.datetime.now().strftime("%Y%m%dT%H%M%S")}.log'

    # ommitting the name uses the root logger - will output both to console and logfile
    logger = get_logger(level='INFO', add_stream_handler=True, logfile_path=logfile_path)

    # from here on out, use the logger to log messages
    logger.debug('This is a debug message, which will not be shown since the log level is set to INFO.')
    logger.info('This is an informational message, which will be shown in both the console and logfile.')
    logger.warning('This is a warning message, indicating a potential issue.')
    logger.error('This is an error message, indicating a failure in a specific operation.')
    logger.critical('This is a critical message, indicating a severe failure that may stop the program.')

Commands

Here are a few commonly used commands for efficient project configuration and use.

make env - creates a Conda environment in the project directory in ./env with resources needed for project development
make jupyter - run Jupyter notebook with options enabling connecting from another computer on the same network if desired
make data - build data using the file ./scripts/make_data.py using the Conda environment ./env created with the command
make env
make docs - builds documentation in ./docs from resources in ./docsrc.
make docserve - runs live server on http://127.0.0.1:8000/ to see updates to docs in real time. This is extremely useful when building the documentation to see how it will look.

Note

These commands are defined in ./make.cmd if you want to examine, modify or extend this capability.

BumpVersion Cliff Notes

Bump2Version is preconfigured based on hints from this article on Medium.

If you want to...

apply a patch, bumpversion patch
update version with no breaking changes (minor version update), bumpversion minor
update version with breaking changes (major version update), bumpversion major
create a release (tagged in version control - Git), bumpversion --tag release