wis2box 1.0b7 released

[tomkralidis]

This is the seventh 1.0 beta release of wis2box. This release includes a number of new features, enhancements and bugfixes. This release introduces a "dataset" driven approach to closer align to core WIS2 concepts. A "dataset" in wis2box is a single record containing both the metadata and the data mappings definition. The separate data-mappings.yml will not longer be used.

This release also introduces the "dataset" editor that supports the creation of WCMP2 discovery metadata and data mappings definitions without requiring the user to edit configuration files directly.

The release also includes the a new "universal" data plugin to support the publishing of data that is produced upstream of wi2box.

Users are strongly encouraged to update to this latest release for WIS2 compliance.

Enhancements/updates

• move data mappings to discovery metadata record, removing data-mappings.yml
• added dataset editor to wis2box-webapp to support WCMP2 publishing and data mappings configuration
• added basic authentication to wis2box-webapp
• add new universal data pipeline
• update station metadata columns to be consistent between station list CSV and station editor, using WIGOS metadata tables
• csv2bufr templates now referenced from external repository
• remove API links from WCMP2 records
• numerous documentation updates, including updates for station metadata as well as security recommendations
• reduced loading-times for wis2box-ui

Bug fixes

• fix incorrect originating centre and sub centre incorrect in BUFR generation
• fix links in WCMP2 metadata

New installation

Please follow the instructions provided at https://docs.wis2box.wis.wmo.int/en/latest/user/getting-started.html

re-using station_list.csv from previous installation

Note that if you want to re-use your station_list.csv from a previous installation you will have to update the content of the columns
facility_type,territory_name,wmo_region to use the WIGOS-metadata codes:

1.0b6:

station_name,wigos_station_identifier,traditional_station_identifier,facility_type,latitude,longitude,elevation,barometer_height,territory_name,wmo_region
BONIFATI (16337-0),0-20000-0-16337,16337,Land (fixed),39.5847222222,15.8913888889,484,485.0,Italy,VI

1.0b7:

station_name,wigos_station_identifier,traditional_station_identifier,facility_type,latitude,longitude,elevation,barometer_height,territory_name,wmo_region
BONIFATI (16337-0),0-20000-0-16337,16337,landFixed,39.5847222222,15.8913888889,484,485.0,ITA,europe

After updating these columns you can copy the file to /metadata/station/station_list.csv and then login to the wis2box-container and run wis2box metadata station publish-collection.

python3 wis2box-ctl.py login
wis2box metadata station publish-collection

To associate all stations to a specific dataset you can use the command 'wis2box metadata station auth-topic', for example:

wis2box metadata station add-topic --territory-name ITA it-roma_met_centre/data/core/weather/surface-based-observations/synop

Migrating existing installations

Migrating existing installations will require you to recreate your discovery metadata records.

Start by downloading the new release and extracting the archive:

cd ~/
wget https://github.com/wmo-im/wis2box/releases/download/1.0b7/wis2box-setup-1.0b7.zip
unzip wis2box-setup-1.0b7.zip

Then, back in your old wis2box-installation , login to the wis2box-management container and unpublish your current discovery metadata:

cd ~/wis2box-1.0b6
python3 wis2box-ctl.py login
wis2box metadata discovery unpublish <metadata-id>

Repeat the wis2box metadata discovery unpublish command until there are no items remaining in https://YOUR_HOST/oapi/collections/discovery-metadata/items

Then stop wis2box in the original installation:

python3 wis2box-ctl.py stop

Delete the api-config volume:

docker volume rm wis2box_project_api-config

Copy your old wis2box.env from the old to the new installation:

cp ~/wis2box-1.0b6/wis2box.env ~/wis2box-1.0b7/wis2box.env

Optionally, add new entries to wis2box.env for environment variables for WIS2BOX_WEBAPP_USERNAME and WIS2BOX_WEBAPP_PASSWORD. If these are not defined, the default credentials are:

WIS2BOX_WEBAPP_USERNAME=wis2box-user
WIS2BOX_WEBAPP_PASSWORD=${WIS2BOX_STORAGE_PASSWORD}

Then start the wis2box from the new release-folder:

cd ~/wis2box-1.0b7
python3 wis2box-ctl.py start

And run the migration script to update your stations to match the new definitions as follows:

python3 wis2box-ctl.py execute python3 /app/migrations/v1_0b6_to_v1_0b7/update_station_definition_v1.0b7.py

Finally use the dataset editor in wis2box-webapp to recreate your datasets, see https://docs.wis2box.wis.wmo.int/en/1.0b7/user/setup.html#adding-datasets.

Note that you can also recreate your metadata by adding data mappings within your wis2box metadata configurations and use the wis2box dataset publish <file-name> command to publish

Documentation

Documentation for this release is available at https://docs.wis2box.wis.wmo.int/en/1.0b7

Feedback

All contributions are always welcome. Feedback is welcome via GitHub issues or discussions.

The wis2box team

https://docs.wis2box.wis.wmo.int

[kornada]

page https://github.com/wmo-im/wis2box/releases/tag/1.0b7
wget https://github.com/wmo-im/wis2box/releases/download/1.0b6/wis2box-setup-1.0b7.zip
incorrect path .../1.0b6/...