add screenshot and ingest related updates

wmo-im · Nov 23, 2023 · 31e57b8 · 31e57b8
1 parent 42ef9d5
commit 31e57b8
Show file tree

Hide file tree

Showing 5 changed files with 28 additions and 23 deletions.
diff --git a/docs/source/_static/minio-new-folder-path.png b/docs/source/_static/minio-new-folder-path.png
diff --git a/docs/source/_static/minio-upload-files.png b/docs/source/_static/minio-upload-files.png
diff --git a/docs/source/_static/winscp_wis2box-ftp_example.png b/docs/source/_static/winscp_wis2box-ftp_example.png
diff --git a/docs/source/reference/storage.rst b/docs/source/reference/storage.rst
@@ -47,47 +47,45 @@ Install the Python boto3 package via `pip`_:
 
 The below example copies a local file (``myfile.csv``) to the ``wis2box-incoming`` bucket with topic ``foo.bar.baz``:
 
-
 .. code-block:: python
 
     import boto3
 
-    endpoint_url = '<your-wis2box-url>'
+    endpoint_url = 'http://localhost:9000'
     filename = 'myfile.csv'
 
     session = boto3.Session(
         aws_access_key_id='wis2box',
-        aws_secret_access_key='Wh00data!'
+        aws_secret_access_key='XXXXX' # your wis2box storage password
     )
 
     s3client = session.client('s3', endpoint_url=endpoint_url)
 
     with open(filename, 'rb') as fh:
         s3client.upload_fileobj(fh, 'wis2box-incoming', f'foo/bar/baz/{filename}')
 
-
 To allow uploading files into MinIO remotely, the ``wis2box-incoming`` bucket is proxied via Nginx. 
 
-For example, to upload the local file (``WIGOS_0-454-2-AWSNAMITAMBO_2021-11-18T0955.csv with topic``) with topic ``mw-mw_met_centre.data.core.weather.surface-based-observations.synop`` via the Nbinx proxy:
+For example, to upload the local file (``WIGOS_0-454-2-AWSNAMITAMBO_2021-11-18T0955.csv with topic``) to the folder 
+``mw-mw_met_centre/data/core/weather/surface-based-observations/synop``:
 
 .. code-block:: python
 
     import boto3
 
-    endpoint_url = 'http://localhost:9000'
-    filename = 'myfile.csv'
+    endpoint_url = '<your-wis2box-url>'
+    filename = 'WIGOS_0-454-2-AWSNAMITAMBO_2021-11-18T0955.csv'
 
     session = boto3.Session(
         aws_access_key_id='wis2box',
-        aws_secret_access_key='Wh00data!'
+        aws_secret_access_key='XXXXX' # your wis2box storage password
     )
 
     s3client = session.client('s3', endpoint_url=endpoint_url)
 
-    filename = 'WIGOS_0-454-2-AWSNAMITAMBO_2021-11-18T0955.csv'
-
-    with open(filename, 'rb') as f:
-        s3client.upload_fileobj(f, 'wis2box-incoming', f'data/core/observations-surface-land/mw/FWCL/landFixed/{filename}')
+    folder = 'mw-mw_met_centre/data/core/weather/surface-based-observations/synop'
+    with open(filename, 'rb') as fh:
+        s3client.upload_fileobj(fh, 'wis2box-incoming', f'{folder}/{filename}')
 
 
 Using the MinIO Python Client
@@ -101,20 +99,28 @@ Install the Python minio module via `pip`_:
 
     pip3 install minio
 
-The below example copies a local file (``myfile.csv``) to the ``wis2box-incoming`` bucket to topic ``foo.bar.baz``:
+The below example copies a local file (``mydata.bin``) to the ``wis2box-incoming`` bucket to topic ``foo.bar.baz``:
 
 .. code-block:: python
 
     from minio import Minio
 
+    filepath = '/home/wis2box-user/local-data/mydata.bin'
+    minio_path = '/it-roma_met_centre/data/core/weather/surface-based-observations/synop/'
+
+    endpoint = 'http://localhost:9000'
+    WIS2BOX_STORAGE_USERNAME = 'wis2box'
+    WIS2BOX_STORAGE_PASSWORD = 'XXXXX' # your wis2box storage password
+
     client = Minio(
-        'localhost:9000',
-        access_key='minio',
-        secret_key='minio123',
-        secure=False
-    )
+        endpoint=endpoint,
+        access_key=WIS2BOX_STORAGE_USERNAME,
+        secret_key=WIS2BOX_STORAGE_PASSWORD,
+        secure=is_secure=False)
+    
+    filename = filepath.split('/')[-1]
+    client.fput_object('wis2box-incoming', minio_path+filename, filepath)
 
-    client.fput_object('wis2box-incoming', 'myfile.csv', '/foo/bar/baz/myfile.csv') 
 
 Using S3cmd
 ^^^^^^^^^^^
@@ -132,8 +138,8 @@ Edit the following fields in ``~/.s3cfg``:
     use_https = False
 
     # Setup access keys
-    access_key = minio
-    secret_key = minio123
+    access_key = wis2box
+    secret_key = XXXXX
     EOF
 
 

diff --git a/docs/source/user/data-ingest.rst b/docs/source/user/data-ingest.rst
@@ -183,8 +183,7 @@ To start the ``wis2box-data-subscriber``, add the following additional variables
 
 .. code-block:: bash
 
-    COUNTRY_ID=zmb  # set country_id used in wis2-topic-hierarchy
-    CENTRE_ID=zmb_met_centre  # set centre_id for wis2-topic-hierarchy
+    CENTRE_ID=zm-zmb_met_centre  # set centre_id for wis2-topic-hierarchy
 
 These variables determine the destination path in the ``wis2box-incoming`` bucket: