From 7bd848f4506f26d5a09f047972b601ec179f0b31 Mon Sep 17 00:00:00 2001 From: Glenn Matthews Date: Fri, 16 Jul 2021 14:11:26 -0400 Subject: [PATCH] Doc updates --- FAQ.md | 8 ++ README.md | 124 ------------------ docs/contributing.md | 119 +++++++++++++++++ .../diffsync/adapter_servicenow.py | 2 +- 4 files changed, 128 insertions(+), 125 deletions(-) create mode 100644 docs/contributing.md diff --git a/FAQ.md b/FAQ.md index 318b08d..9697adf 100644 --- a/FAQ.md +++ b/FAQ.md @@ -1 +1,9 @@ # Frequently Asked Questions + +## Why did my job fail with an `IncompleteJSONError`? + +``` +An exception occurred: `IncompleteJSONError: lexical error: invalid char in json text. `. Environment variables `INVOKE_NAUTOBOT_SSOT_SERVICENOW_PYTHON_VER` and `INVOKE_NAUTOBOT_SSOT_SERVICENOW_NAUTOBOT_VER` may be specified to override the default versions. Each command also has its own help `invoke --help` - -#### Docker dev environment - -```no-highlight - build Build all docker images. - debug Start Nautobot and its dependencies in debug mode. - destroy Destroy all containers and volumes. - restart Restart Nautobot and its dependencies. - start Start Nautobot and its dependencies in detached mode. - stop Stop Nautobot and its dependencies. -``` - -#### Utility - -```no-highlight - cli Launch a bash shell inside the running Nautobot container. - create-user Create a new user in django (default: admin), will prompt for password. - makemigrations Run Make Migration in Django. - nbshell Launch a nbshell session. -``` - -#### Testing - -```no-highlight - bandit Run bandit to validate basic static code security analysis. - black Run black to check that Python files adhere to its style standards. - flake8 This will run flake8 for the specified name and Python version. - pydocstyle Run pydocstyle to validate docstring formatting adheres to NTC defined standards. - pylint Run pylint code analysis. - tests Run all tests for this plugin. - unittest Run Django unit tests for the plugin. -``` - -### Project Documentation - -Project documentation is generated by [mkdocs](https://www.mkdocs.org/) from the documentation located in the docs folder. You can configure [readthedocs.io](https://readthedocs.io/) to point at this folder in your repo. For development purposes a `docker-compose.docs.yml` is also included. A container hosting the docs will be started using the invoke commands on [http://localhost:8001](http://localhost:8001), as changes are saved the docs will be automatically reloaded. - ## Questions For any questions or comments, please check the [FAQ](FAQ.md) first and feel free to swing by the [Network to Code slack channel](https://networktocode.slack.com/) (channel #networktocode). diff --git a/docs/contributing.md b/docs/contributing.md new file mode 100644 index 0000000..76f6e5c --- /dev/null +++ b/docs/contributing.md @@ -0,0 +1,119 @@ +# Contributing + +Pull requests are welcomed and automatically built and tested against multiple version of Python and multiple version of Nautobot through TravisCI. + +The project is packaged with a light development environment based on `docker-compose` to help with the local development of the project and to run the tests within TravisCI. + +The project is following Network to Code software development guideline and is leveraging: + +- Black, Pylint, Bandit and pydocstyle for Python linting and formatting. +- Django unit test to ensure the plugin is working properly. + +## Development Environment + +The development environment can be used in 2 ways. First, with a local poetry environment if you wish to develop outside of Docker. Second, inside of a docker container. + +### Invoke tasks + +The [PyInvoke](http://www.pyinvoke.org/) library is used to provide some helper commands based on the environment. There are a few configuration parameters which can be passed to PyInvoke to override the default configuration: + +* `nautobot_ver`: the version of Nautobot to use as a base for any built docker containers (default: develop-latest) +* `project_name`: the default docker compose project name (default: nautobot-ssot-servicenow) +* `python_ver`: the version of Python to use as a base for any built docker containers (default: 3.6) +* `local`: a boolean flag indicating if invoke tasks should be run on the host or inside the docker containers (default: False, commands will be run in docker containers) +* `compose_dir`: the full path to a directory containing the project compose files +* `compose_files`: a list of compose files applied in order (see [Multiple Compose files](https://docs.docker.com/compose/extends/#multiple-compose-files) for more information) + +Using PyInvoke these configuration options can be overridden using [several methods](http://docs.pyinvoke.org/en/stable/concepts/configuration.html). Perhaps the simplest is simply setting an environment variable `INVOKE_NAUTOBOT_SSOT_SERVICENOW_VARIABLE_NAME` where `VARIABLE_NAME` is the variable you are trying to override. The only exception is `compose_files`, because it is a list it must be overridden in a yaml file. There is an example `invoke.yml` in this directory which can be used as a starting point. + +### Local Poetry Development Environment + +1. Copy `development/creds.example.env` to `development/creds.env` (This file will be ignored by git and docker) +2. Uncomment the `POSTGRES_HOST`, `REDIS_HOST`, and `NAUTOBOT_ROOT` variables in `development/creds.env` +3. Create an invoke.yml with the following contents at the root of the repo: + +```shell +--- +nautobot_ssot_servicenow: + local: true + compose_files: + - "docker-compose.requirements.yml" +``` + +3. Run the following commands: + +```shell +poetry shell +poetry install +export $(cat development/dev.env | xargs) +export $(cat development/creds.env | xargs) +``` + +4. You can now run nautobot-server commands as you would from the [Nautobot documentation](https://nautobot.readthedocs.io/en/latest/) for example to start the development server: + +```shell +nautobot-server runserver 0.0.0.0:8080 --insecure +``` + +Nautobot server can now be accessed at [http://localhost:8080](http://localhost:8080). + +### Docker Development Environment + +This project is managed by [Python Poetry](https://python-poetry.org/) and has a few requirements to setup your development environment: + +1. Install Poetry, see the [Poetry Documentation](https://python-poetry.org/docs/#installation) for your operating system. +2. Install Docker, see the [Docker documentation](https://docs.docker.com/get-docker/) for your operating system. + +Once you have Poetry and Docker installed you can run the following commands to install all other development dependencies in an isolated python virtual environment: + +```shell +poetry shell +poetry install +invoke start +``` + +Nautobot server can now be accessed at [http://localhost:8080](http://localhost:8080). + +## CLI Helper Commands + +The project is coming with a CLI helper based on [invoke](http://www.pyinvoke.org/) to help setup the development environment. The commands are listed below in 3 categories `dev environment`, `utility` and `testing`. + +Each command can be executed with `invoke `. Environment variables `INVOKE_NAUTOBOT_SSOT_SERVICENOW_PYTHON_VER` and `INVOKE_NAUTOBOT_SSOT_SERVICENOW_NAUTOBOT_VER` may be specified to override the default versions. Each command also has its own help `invoke --help` + +### Docker dev environment + +```no-highlight + build Build all docker images. + debug Start Nautobot and its dependencies in debug mode. + destroy Destroy all containers and volumes. + restart Restart Nautobot and its dependencies. + start Start Nautobot and its dependencies in detached mode. + stop Stop Nautobot and its dependencies. +``` + +### Utility + +```no-highlight + cli Launch a bash shell inside the running Nautobot container. + create-user Create a new user in django (default: admin), will prompt for password. + makemigrations Run Make Migration in Django. + nbshell Launch a nbshell session. +``` + +### Testing + +```no-highlight + bandit Run bandit to validate basic static code security analysis. + black Run black to check that Python files adhere to its style standards. + flake8 This will run flake8 for the specified name and Python version. + pydocstyle Run pydocstyle to validate docstring formatting adheres to NTC defined standards. + pylint Run pylint code analysis. + tests Run all tests for this plugin. + unittest Run Django unit tests for the plugin. +``` + +## Project Documentation + +Project documentation is generated by [mkdocs](https://www.mkdocs.org/) from the documentation located in the docs folder. You can configure [readthedocs.io](https://readthedocs.io/) to point at this folder in your repo. For development purposes a `docker-compose.docs.yml` is also included. A container hosting the docs will be started using the invoke commands on [http://localhost:8001](http://localhost:8001), as changes are saved the docs will be automatically reloaded. + + diff --git a/nautobot_ssot_servicenow/diffsync/adapter_servicenow.py b/nautobot_ssot_servicenow/diffsync/adapter_servicenow.py index c02bb5f..d154af3 100644 --- a/nautobot_ssot_servicenow/diffsync/adapter_servicenow.py +++ b/nautobot_ssot_servicenow/diffsync/adapter_servicenow.py @@ -93,7 +93,7 @@ def load_record(self, table, record, model_cls, mappings, **kwargs): try: self.add(model) - self.job.log_debug(f"Loaded {modelname} {model.get_unique_id()}") + # self.job.log_debug(f"Loaded {modelname} {model.get_unique_id()}") except ObjectAlreadyExists: # TODO: the baseline data in ServiceNow has a number of duplicate Location entries. For now, continue self.job.log_debug(f"Duplicate object encountered for {modelname} {model.get_unique_id()}")