Technical_Documentation

Technical Documentation

Modules

views.py

Central file to accept and propagate all requests from clients. Those include:

• upload firmware: User either drag and drops file or selects it by hand. For further information see (### file uploading)
  It is passed as FirmwareFile attached to the request which is stored as model in the database and saved packed on the server machine.
• analyse firmware: User fills django form which is send togehter with the request.
  It includes Reference to FirmwareFile, normal emba Options and expert mode Options, which is saved into the DB.
  Both the File and the Metainformation are passed to the BoundedExecuter (see ###)
• service dashboard: Displays currently running Analyzers with status information. Communication is done via asgi and channels. To retrieve the status information the emba log is read asynchronously
• central dashboard: TODO

file uploading

Archiver

The Archiver is basically an extended wrapper around shutils to support all common archive formats.
It supports packing, unpacking and validating of different archive types.
Unittest labeled as test_archiver.

BoundedExecutor

The BoundedExecutor is basically an extended wrapper around pythons ThreadPoolExecutor to support a selectable finite upper bound for running processes.
Internally it uses additional BoundedSemaphore to track the current load, on submit() the semaphore ius decremented on finished process incremented.
On submit_firmware, the Archiver is utilized to unpack the selected firmware to an new directory and also starting the emba.sh process with the blocking subprocess.call.
Unittest labeled as test_boundedExecutor.

inotify log reader

asgi

Database

authentication

models

There are currently 2 different models:

• FirmwareFile:
  Used to track zipped firmwares. Featuring FileField as well as DateTimeField.
• Firmware:
  Used to track Analyzed firmwares. Custom fields provide additional attribute for defining expert field: CharFieldExpertMode, BooleanFieldExpertMode.
  Fields include reference to FirmwareFile, basic fields (e.g. version, vendor, architecture) and expert fields (e.g. web-report, multi-threading, dependencies).
  For further information on available flags consult the script like this: emba.sh -h

forms

The current forms are generated from above defined models. The forms additionally take care of html attributes like expert mode and class on instancing.
See https://docs.djangoproject.com/en/3.2/topics/forms/modelforms/ for further information.

Guidelines

testing

The project uses the django testing environment. To write your own unittest you need a python file labeled test_. Within there need to be class extending TestCase. On test execution all methodes in test classes are invoked an run.
For an example on unittests consult test_archiver and test_boundedExecutor.

There is Pipeline checking regression by running the django test environment: python manage.py test.
You are encouraged to run the tests locally beforehand.

logging

For logging use djangos logging environment.
Configuration can be found in embark/settings.py as dict LOGGING. Logs can be inspected at embark/logs/*.log

Usage (this will create the logfile 'web'):

logger = logging.getLogger('web')

[...]

logger.info("my very own log message")

For further reading see how to logging.

codestyle

Every contributor is obliged to the PEP8 coding style on python code. To check your conformity with pep8 locally: pycodestyle .
To get further information about the violation run: pycodestyle --show-source .
For further setting see pycodestyle documentation