nightscout update (#5)

* * Bump version to 14.2.3 * Change packaging to include full moment.js locale library to fix date processing issues for some users * Add `docker-compose` support (nightscout#6903) * Create docker-compose.yml file for easy deployment * Restore invalid password, to force change * New Crowdin updates (nightscout#6917) * New translations en.json (Hungarian) * New translations en.json (Hungarian) * New translations en.json (Czech) * New translations en.json (Hungarian) * New translations en.json (Hungarian) * New translations en.json (Hungarian) * New translations en.json (Hungarian) * New translations en.json (Hungarian) * New translations en.json (Hungarian) * New translations en.json (Russian) * New translations en.json (Romanian) * New translations en.json (Italian) * New translations en.json (Italian) * New translations en.json (Tamil) * New translations en.json (Portuguese) * New translations en.json (Slovenian) * New translations en.json (Slovenian) * New translations en.json (Slovenian) * New translations en.json (Slovenian) * New translations en.json (Slovenian) * New translations en.json (Slovenian) * New translations en.json (Chinese Simplified) * New translations en.json (Norwegian Bokmal) * New translations en.json (French) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Chinese Simplified) * New translations en.json (Chinese Simplified) * Fixing a bad URL that sends the user to docs if they have an open site (nightscout#7005) Fixing a bad URL that sends the user to docs if they have an open site http://nightscout.github.io/nightscout/security/#how-to-turn-off-unauthorized-access Co-authored-by: Sulka Haro <[email protected]> * New translations en.json (German) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Greek) * New translations en.json (Greek) * New translations en.json (Greek) * bump share2nightscout-bridge version Use npm update --save share2nightscout-bridge to update package-lock.json and package.json. This should assist in updates intended to help unify Dexcom Account management. * New translations en.json (Hebrew) * New translations en.json (Arabic) * New translations en.json (Arabic) * New translations en.json (Arabic) * New translations en.json (Arabic) * New translations en.json (Arabic) * New translations en.json (Arabic) * New translations en.json (Arabic) * respect per route body-parser configuration This change allows each route to express policies for interpreting and parsing the request body. Before this change uploads to entries or treatments api would error if they were larger than 100Kb due to the preference being set for the whole server. This change removes the global preference in favor of allowing each route to choose it's own request size limit. We also refactor usage of body-parser to be more consistent throughout the code base. Most routes can use jsonParser, rawPraser, and urlencodedParser provided by the common wares component. Anything doing something else should be called out as such. For example, treatments, activity, and entries allow uploads up to 50Mb. Other v1 endpoints are using the common configuration set to 1Mb. * New translations en.json (Arabic) * New translations en.json (Arabic) * New translations en.json (Arabic) * sanitize x-forwarded-for header (nightscout#7122) Many proxies (NGINX included) will generate a list of IPs in v4 and v6 formats. The forwarded-for library is a well-maintained library for express that sanitizes, checks, and validates trusted proxy IPs. This helps eliminate XSS or other attempts to inject invalid material through the x-forwarded-for header. Co-authored-by: Sulka Haro <[email protected]> * Prep next release - 14.2.4 * Add wares to v2 to support request parsing * Move wares to ctx * npm install diffs make package-lock.json match This patch is intended to make everything match for version 14.2.4. * new dev branch starting post 14.2.4 release * Fix api-secret header name in swagger configuration. The /api-docs/ endpoint will now send the correct api-secret header when API_SECRET is used for authentication. * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Portuguese, Brazilian) * New translations en.json (Polish) * New translations en.json (Polish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Turkish) * New translations en.json (Hebrew) * update minimum version of share2nightscout-bridge Forcing a newer version of share2nightscout-bridge in order to keep up with Dexcom changes. * update minimed-connect-to-nightscout updates Include enhancements on the minimed-connect-to-nightscout plugin to include fetching data from a Careportal account. * Increase accuracy of mg/dl to mmol/l ratio Using a ratio of 18 mg/ml to 1 mmol/l causes NightScout to report different values compared Dexcom app/receivers. A more accurate value is based on 0.0555 mmol/l to 1 mg/dl. The inverse is 18.018018018 mg/dl to 1 mmol/l This puts the value closer to the Dexcom app/receiver. Affected values: |mg/dl| 18|18.018018018| |----:|---:|-----------:| | 109| 6.1| 6.0| | 118| 6.6| 6.5| | 127| 7.1| 7.0| | 136| 7.6| 7.5| | 145| 8.1| 8.0| | 154| 8.6| 8.5| | 163| 9.1| 9.0| | 172| 9.6| 9.5| | 181|10.1| 10.0| | 190|10.6| 10.5| | 199|11.1| 11.0| | 208|11.6| 11.5| | 217|12.1| 12.0| | 226|12.6| 12.5| | 235|13.1| 13.0| | 244|13.6| 13.5| | 253|14.1| 14.0| | 262|14.6| 14.5| | 271|15.1| 15.0| | 280|15.6| 15.5| | 289|16.1| 16.0| | 298|16.6| 16.5| | 307|17.1| 17.0| | 309|17.2| 17.1| | 316|17.6| 17.5| | 318|17.7| 17.6| | 325|18.1| 18.0| | 327|18.2| 18.1| | 334|18.6| 18.5| | 336|18.7| 18.6| | 343|19.1| 19.0| | 345|19.2| 19.1| | 352|19.6| 19.5| | 354|19.7| 19.6| | 361|20.1| 20.0| | 363|20.2| 20.1| | 370|20.6| 20.5| | 372|20.7| 20.6| | 379|21.1| 21.0| | 381|21.2| 21.1| | 388|21.6| 21.5| | 390|21.7| 21.6| | 397|22.1| 22.0| | 399|22.2| 22.1| * Support Dexcom HIGH/LOW when device is share2 Using share2nightscout-bridge, the device property is defined as 'share2' displayBg should honour the Dexcom HIGH and LOW values when the device is share2 This ensures that NightScout returns "LOW" for 39mg/dl and "HIGH" for 401mg/dl as is reported in the Dexcom app/receiver. * Revert "update minimed-connect-to-nightscout updates" This reverts commit 513d155. * use newer version of minimed-connect-to-nightscout This patches uses a newer versin of minimed-connect-to-nightscout that is designed to work under the same version of node as Nightscout. It should result in Nightscout gaining a capability to use Medtronic Careportal accounts for cloud to cloud synchronization. * New translations en.json (Chinese Simplified) * Fixes broken Docker image build The `npm audit fix` in the `Dockerfile` is currently deleting vulnerable packages. Although we do want to have package security, this should be done in a development process, not as part of the Docker build. This commit also reduces the Docker image size and improves security. * New translations en.json (Chinese Simplified) * Generating codacy coverage should not fail build This should force the coverage job to return with a success error code. Codacy changed support for legacy projects. This change should not cause builds to fail and should not influence ability to merge pull requests. Until further work with coverage with codacy, this change emits a warning of "NO COVERAGE" while allowing the build to pass, rather than failing the build. * Synchronise the dexcom collection times to reduce refresh lag When collecting date from Dexcom via share2bridge-dexcom before we would poll at regular intervals. Dexcom transmitters / apps refresh every 5 minutes give/take the inaccuracy in the transmitters clock. If we poll every 5 minutes, we will randomly introduce up to 5 minutes of lag before we refresh. For example, if we poll 4:59s after dexcom refreshed we will face a 4:59s lag seeing the latest data arrive This patch introduces a refresh adjustment. We look at the timestamp of the most recent data item and aim to refresh 5 minutes plus a buffer after that timestamp. for example, 5:20s after the most recent data time. the additional buffer gives dexcom a chance to push the data through their application/infrastructure (after the app sends the data, it appears in testing to be around 20s before it is available in the API) This reduces the lag time to a reliable 20 seconds regardless of when nightscout was started and keeps it in track as it drifts forwards and back. * Update test for new default interval The default interval is increasd from 2.5 minutes to 2.6 minutes This change incorporates the lag between the app uploading and the data being available in share2nightscout-bridge This change also accomodates the optimisation of the refresh interval based on the last collected data timestamp Co-authored-by: Sulka Haro <[email protected]> Co-authored-by: Lennart Goedhart <[email protected]> Co-authored-by: Scott Hanselman <[email protected]> Co-authored-by: Ben West <[email protected]> Co-authored-by: Matthew Dawson <[email protected]> Co-authored-by: cpitchford <[email protected]>
njmo · Jul 27, 2022 · 447be08 · 447be08
1 parent 46418c7
commit 447be08
Show file tree

Hide file tree

Showing 48 changed files with 3,762 additions and 2,054 deletions.
diff --git a/.gitignore b/.gitignore
@@ -30,3 +30,7 @@ npm-debug.log
 /cgm-remote-monitor.sln
 /obj/Debug
 /*.bat
+
+# directories created by docker-compose.yml
+mongo-data/
+letsencrypt/
diff --git a/Dockerfile b/Dockerfile
@@ -2,17 +2,22 @@ FROM node:14.15.3-alpine
 
 LABEL maintainer="Nightscout Contributors"
 
-RUN mkdir -p /opt/app
-ADD . /opt/app
 WORKDIR /opt/app
-RUN chown -R node:node /opt/app
-USER node
+ADD . /opt/app
 
-RUN npm install && \
+# TODO: We should be able to do `RUN npm install --only=production`.
+# For this to work, we need to copy only package.json and things needed for `npm`'s to succeed.
+# TODO: Do we need to re-add `npm audit fix`? Or should that be part of a development process/stage?
+RUN npm install --cache /tmp/empty-cache && \
   npm run postinstall && \
   npm run env && \
-  npm audit fix
+  rm -rf /tmp/*
+  # TODO: These should be added in the future to correctly cache express-minify content to disk
+  # Currently, doing this breaks the browser cache.
+  # mkdir /tmp/public && \
+  # chown node:node /tmp/public
 
+USER node
 EXPOSE 1337
 
 CMD ["node", "lib/server/server.js"]
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -0,0 +1,76 @@
+version: '3'
+
+services:
+  mongo:
+    image: mongo:4.4
+    volumes:
+      - ${NS_MONGO_DATA_DIR:-./mongo-data}:/data/db:cached
+
+  nightscout:
+    image: nightscout/cgm-remote-monitor:latest
+    container_name: nightscout
+    restart: always
+    depends_on:
+      - mongo
+    labels:
+      - 'traefik.enable=true'
+      # Change the below Host from `localhost` to be the web address where Nightscout is running.
+      # Also change the email address in the `traefik` service below.
+      - 'traefik.http.routers.nightscout.rule=Host(`localhost`)'
+      - 'traefik.http.routers.nightscout.entrypoints=websecure'
+      - 'traefik.http.routers.nightscout.tls.certresolver=le'
+    environment:
+      ### Variables for the container
+      NODE_ENV: production
+      TZ: Etc/UTC
+
+      ### Overridden variables for Docker Compose setup
+      # The `nightscout` service can use HTTP, because we use `traefik` to serve the HTTPS
+      # and manage TLS certificates
+      INSECURE_USE_HTTP: 'true'
+
+      # For all other settings, please refer to the Environment section of the README
+      ### Required variables
+      # MONGO_CONNECTION - The connection string for your Mongo database.
+      # Something like mongodb://sally:[email protected]:99999/nightscout
+      # The default connects to the `mongo` included in this docker-compose file.
+      # If you change it, you probably also want to comment out the entire `mongo` service block
+      # and `depends_on` block above.
+      MONGO_CONNECTION: mongodb://mongo:27017/nightscout
+
+      # API_SECRET - A secret passphrase that must be at least 12 characters long.
+      API_SECRET: change_me
+
+      ### Features
+      # ENABLE - Used to enable optional features, expects a space delimited list, such as: careportal rawbg iob
+      # See https://github.com/nightscout/cgm-remote-monitor#plugins for details
+      ENABLE: careportal rawbg iob
+
+      # AUTH_DEFAULT_ROLES (readable) - possible values readable, denied, or any valid role name.
+      # When readable, anyone can view Nightscout without a token. Setting it to denied will require
+      # a token from every visit, using status-only will enable api-secret based login.
+      AUTH_DEFAULT_ROLES: denied
+
+      # For all other settings, please refer to the Environment section of the README
+      # https://github.com/nightscout/cgm-remote-monitor#environment
+
+  traefik:
+    image: traefik:latest
+    container_name: 'traefik'
+    command:
+      - '--providers.docker=true'
+      - '--providers.docker.exposedbydefault=false'
+      - '--entrypoints.web.address=:80'
+      - '--entrypoints.web.http.redirections.entrypoint.to=websecure'
+      - '--entrypoints.websecure.address=:443'
+      - "--certificatesresolvers.le.acme.httpchallenge=true"
+      - "--certificatesresolvers.le.acme.httpchallenge.entrypoint=web"
+      - '--certificatesresolvers.le.acme.storage=/letsencrypt/acme.json'
+      # Change the below to match your email address
+      - '[email protected]'
+    ports:
+      - '443:443'
+      - '80:80'
+    volumes:
+      - './letsencrypt:/letsencrypt'
+      - '/var/run/docker.sock:/var/run/docker.sock:ro'
diff --git a/lib/api/activity/index.js b/lib/api/activity/index.js
@@ -12,23 +12,14 @@ function configure(app, wares, ctx) {
         , api = express.Router();
 
     api.use(wares.compression());
-    api.use(wares.bodyParser({
-        limit: 1048576 * 50
-    }));
     // text body types get handled as raw buffer stream
-    api.use(wares.bodyParser.raw({
-        limit: 1048576
-    }));
+    api.use(wares.rawParser);
     // json body types get handled as parsed json
     api.use(wares.bodyParser.json({
-        limit: 1048576
-        , extended: true
+      limit: '50Mb'
     }));
     // also support url-encoded content-type
-    api.use(wares.bodyParser.urlencoded({
-        limit: 1048576
-        , extended: true
-    }));
+    api.use(wares.urlencodedParser);
     // invoke common middleware
     api.use(wares.sendJSONStatus);
 
@@ -94,9 +85,7 @@ function configure(app, wares, ctx) {
             });
         }
 
-        api.post('/activity/', wares.bodyParser({
-            limit: 1048576 * 50
-        }), ctx.authorization.isPermitted('api:activity:create'), post_response);
+        api.post('/activity/', ctx.authorization.isPermitted('api:activity:create'), post_response);
 
         api.delete('/activity/:_id', ctx.authorization.isPermitted('api:activity:delete'), function(req, res) {
             ctx.activity.remove(req.params._id, function() {

diff --git a/lib/api/alexa/index.js b/lib/api/alexa/index.js
@@ -11,12 +11,12 @@ function configure (app, wares, ctx, env) {
   // invoke common middleware
   api.use(wares.sendJSONStatus);
   // text body types get handled as raw buffer stream
-  api.use(wares.bodyParser.raw());
+  api.use(wares.rawParser);
   // json body types get handled as parsed json
-  api.use(wares.bodyParser.json({
-    limit: 1048576
-    , extended: true
-  }));
+  api.use(wares.jsonParser);
+  // also support url-encoded content-type
+  api.use(wares.urlencodedParser);
+  // text body types get handled as raw buffer stream
 
   ctx.virtAsstBase.setupVirtAsstHandlers(ctx.alexa);
 

diff --git a/lib/api/devicestatus/index.js b/lib/api/devicestatus/index.js
@@ -13,14 +13,12 @@ function configure (app, wares, ctx, env) {
   // invoke common middleware
   api.use(wares.sendJSONStatus);
   // text body types get handled as raw buffer stream
-  api.use(wares.bodyParser.raw());
+  api.use(wares.rawParser);
   // json body types get handled as parsed json
-  api.use(wares.bodyParser.json({
-    limit: 1048576
-    , extended: true
-  }));
+  api.use(wares.jsonParser);
   // also support url-encoded content-type
-  api.use(wares.bodyParser.urlencoded({ extended: true }));
+  api.use(wares.urlencodedParser);
+  // text body types get handled as raw buffer stream
 
   api.use(ctx.authorization.isPermitted('api:devicestatus:read'));
 

diff --git a/lib/api/entries/index.js b/lib/api/entries/index.js
@@ -41,20 +41,18 @@ function configure (app, wares, ctx, env) {
   // invoke common middleware
   api.use(wares.sendJSONStatus);
   // text body types get handled as raw buffer stream
-  api.use(wares.bodyParser.raw());
+  api.use(wares.rawParser);
   // json body types get handled as parsed json
   api.use(wares.bodyParser.json({
-    limit: 1048576
-    , extended: true
+    limit: '50Mb'
   }));
+  // also support url-encoded content-type
+  api.use(wares.urlencodedParser);
+  // text body types get handled as raw buffer stream
   // shortcut to use extension to specify output content-type
   api.use(wares.extensions([
         'json', 'svg', 'csv', 'txt', 'png', 'html', 'tsv'
     ]));
-  // also support url-encoded content-type
-  api.use(wares.bodyParser.urlencoded({
-    extended: true
-  }));
 
   api.use(ctx.authorization.isPermitted('api:entries:read'));
   /**

diff --git a/lib/api/food/index.js b/lib/api/food/index.js
@@ -9,14 +9,13 @@ function configure (app, wares, ctx) {
     // invoke common middleware
     api.use(wares.sendJSONStatus);
     // text body types get handled as raw buffer stream
-    api.use(wares.bodyParser.raw( ));
+    api.use(wares.rawParser);
     // json body types get handled as parsed json
-    api.use(wares.bodyParser.json({
-      limit: 1048576
-      , extended: true
-    }));  
+    api.use(wares.jsonParser);
     // also support url-encoded content-type
-    api.use(wares.bodyParser.urlencoded({ extended: true }));
+    api.use(wares.urlencodedParser);
+    // text body types get handled as raw buffer stream
+    // shortcut to use extension to specify output content-type
 
     api.use(ctx.authorization.isPermitted('api:food:read'));
 

diff --git a/lib/api/googlehome/index.js b/lib/api/googlehome/index.js
@@ -11,9 +11,10 @@ function configure (app, wares, ctx, env) {
   // invoke common middleware
   api.use(wares.sendJSONStatus);
   // text body types get handled as raw buffer stream
-  api.use(wares.bodyParser.raw());
+  api.use(wares.rawParser);
   // json body types get handled as parsed json
-  api.use(wares.bodyParser.json());
+  api.use(wares.jsonParser);
+
 
   ctx.virtAsstBase.setupVirtAsstHandlers(ctx.googleHome);
 

diff --git a/lib/api/index.js b/lib/api/index.js
@@ -6,7 +6,7 @@ function create (env, ctx) {
     ,  app = express( )
     ;
 
-  var wares = require('../middleware/')(env);
+  const wares = ctx.wares;
 
   // set up express app with our options
   app.set('name', env.name);

diff --git a/lib/api/notifications-v2.js b/lib/api/notifications-v2.js
@@ -7,6 +7,13 @@ function configure (app, ctx) {
     , api = express.Router( )
     ;
 
+  api.use(ctx.wares.compression());
+  api.use(ctx.wares.rawParser);
+  api.use(ctx.wares.bodyParser.json({
+      limit: '50Mb'
+  }));
+  api.use(ctx.wares.urlencodedParser);
+
   api.post('/loop', ctx.authorization.isPermitted('notifications:loop:push'), function (req, res) {
     ctx.loop.sendNotification(req.body, req.connection.remoteAddress, function (error) {
       if (error) {

diff --git a/lib/api/profile/index.js b/lib/api/profile/index.js
@@ -9,14 +9,12 @@ function configure (app, wares, ctx) {
     // invoke common middleware
     api.use(wares.sendJSONStatus);
     // text body types get handled as raw buffer stream
-    api.use(wares.bodyParser.raw( ));
+    api.use(wares.rawParser);
     // json body types get handled as parsed json
-    api.use(wares.bodyParser.json({
-        limit: 1048576
-        , extended: true
-    })); 
+    api.use(wares.jsonParser);
     // also support url-encoded content-type
-    api.use(wares.bodyParser.urlencoded({ extended: true }));
+    api.use(wares.urlencodedParser);
+    // text body types get handled as raw buffer stream
 
     api.use(ctx.authorization.isPermitted('api:profile:read'));
 

diff --git a/lib/api/status.js b/lib/api/status.js
@@ -2,6 +2,7 @@
 
 function configure (app, wares, env, ctx) {
   var express = require('express'),
+    forwarded = require('forwarded-for'),
     api = express.Router( )
     ;
 
@@ -21,7 +22,8 @@ function configure (app, wares, env, ctx) {
     var authToken = req.query.token || req.query.secret || '';
 
     function getRemoteIP (req) {
-      return req.headers['x-forwarded-for'] || req.connection.remoteAddress;
+      const address = forwarded(req, req.headers);
+      return address.ip;
     }
 
     var date = new Date();

diff --git a/lib/api/treatments/index.js b/lib/api/treatments/index.js
@@ -13,24 +13,16 @@ function configure (app, wares, ctx, env) {
     , api = express.Router();
 
   api.use(wares.compression());
-  api.use(wares.bodyParser({
-    limit: 1048576 * 50
-    , extended: true
-  }));
+
   // text body types get handled as raw buffer stream
-  api.use(wares.bodyParser.raw({
-    limit: 1048576
-  }));
+  api.use(wares.rawParser);
   // json body types get handled as parsed json
   api.use(wares.bodyParser.json({
-    limit: 1048576
-    , extended: true
+    limit: '50Mb'
   }));
   // also support url-encoded content-type
-  api.use(wares.bodyParser.urlencoded({
-    limit: 1048576
-    , extended: true
-  }));
+  api.use(wares.urlencodedParser);
+
   // invoke common middleware
   api.use(wares.sendJSONStatus);
 
@@ -150,9 +142,7 @@ function configure (app, wares, ctx, env) {
       });
     }
 
-    api.post('/treatments/', wares.bodyParser({
-      limit: 1048576 * 50
-    }), ctx.authorization.isPermitted('api:treatments:create'), post_response);
+    api.post('/treatments/', ctx.authorization.isPermitted('api:treatments:create'), post_response);
 
     /**
      * @function delete_records

diff --git a/lib/api3/security.js b/lib/api3/security.js
@@ -4,11 +4,13 @@ const apiConst = require('./const.json')
   , _ = require('lodash')
   , shiroTrie = require('shiro-trie')
   , opTools = require('./shared/operationTools')
+  , forwarded = require('forwarded-for')
   ;
 
 
 function getRemoteIP (req) {
-  return req.headers['x-forwarded-for'] || req.connection.remoteAddress;
+  const address = forwarded(req, req.headers);
+  return address.ip;
 }
 
 

diff --git a/lib/api3/storageSocket.js b/lib/api3/storageSocket.js
@@ -1,6 +1,7 @@
 'use strict';
 
 const apiConst = require('./const');
+const forwarded = require('forwarded-for');
 
 /**
  * Socket.IO broadcaster of any storage change
@@ -28,7 +29,8 @@ function StorageSocket (app, env, ctx) {
     self.namespace = io.of(NAMESPACE);
     self.namespace.on('connection', function onConnected (socket) {
 
-      const remoteIP = socket.request.headers['x-forwarded-for'] || socket.request.connection.remoteAddress;
+      const address = forwarded(socket.request, socket.request.headers);
+      const remoteIP = address.ip;
       console.log(LOG + 'Connection from client ID: ', socket.client.id, ' IP: ', remoteIP);
 
       socket.on('disconnect', function onDisconnect () {
@@ -142,4 +144,4 @@ function StorageSocket (app, env, ctx) {
   }
 }
 
-module.exports = StorageSocket;
+module.exports = StorageSocket;