From 09f35469f7f696fa8918185bb77dd841eb77472f Mon Sep 17 00:00:00 2001 From: Ashley Roach Date: Mon, 8 Oct 2018 16:41:30 -0600 Subject: [PATCH 1/9] Add inbound parse webhook example --- examples/inbound-parse-docker/.dockerignore | 13 + examples/inbound-parse-docker/.gitignore | 1 + examples/inbound-parse-docker/CONTRIBUTING.md | 225 +++++++ examples/inbound-parse-docker/Dockerfile | 8 + examples/inbound-parse-docker/LICENSE | 15 + examples/inbound-parse-docker/README.md | 21 + examples/inbound-parse-docker/app.js | 27 + .../docker-compose.debug.yml | 13 + .../inbound-parse-docker/docker-compose.yml | 10 + .../inbound-parse-docker/package-lock.json | 555 ++++++++++++++++++ examples/inbound-parse-docker/package.json | 17 + 11 files changed, 905 insertions(+) create mode 100644 examples/inbound-parse-docker/.dockerignore create mode 100644 examples/inbound-parse-docker/.gitignore create mode 100644 examples/inbound-parse-docker/CONTRIBUTING.md create mode 100644 examples/inbound-parse-docker/Dockerfile create mode 100644 examples/inbound-parse-docker/LICENSE create mode 100644 examples/inbound-parse-docker/README.md create mode 100644 examples/inbound-parse-docker/app.js create mode 100644 examples/inbound-parse-docker/docker-compose.debug.yml create mode 100644 examples/inbound-parse-docker/docker-compose.yml create mode 100644 examples/inbound-parse-docker/package-lock.json create mode 100644 examples/inbound-parse-docker/package.json diff --git a/examples/inbound-parse-docker/.dockerignore b/examples/inbound-parse-docker/.dockerignore new file mode 100644 index 000000000..3a78c9d4c --- /dev/null +++ b/examples/inbound-parse-docker/.dockerignore @@ -0,0 +1,13 @@ +node_modules +npm-debug.log +Dockerfile* +docker-compose* +.dockerignore +.git +.gitignore +.env +*/bin +*/obj +README.md +LICENSE +.vscode \ No newline at end of file diff --git a/examples/inbound-parse-docker/.gitignore b/examples/inbound-parse-docker/.gitignore new file mode 100644 index 000000000..b512c09d4 --- /dev/null +++ b/examples/inbound-parse-docker/.gitignore @@ -0,0 +1 @@ +node_modules \ No newline at end of file diff --git a/examples/inbound-parse-docker/CONTRIBUTING.md b/examples/inbound-parse-docker/CONTRIBUTING.md new file mode 100644 index 000000000..53cbbaf56 --- /dev/null +++ b/examples/inbound-parse-docker/CONTRIBUTING.md @@ -0,0 +1,225 @@ +Hello! Thank you for choosing to help contribute to one of the SendGrid open source libraries. There are many ways you can contribute and help is always welcome. We simply ask that you follow the following contribution policies. + +- [CLAs and CCLAs](#cla) +- [Roadmap & Milestones](#roadmap) +- [Feature Request](#feature-request) +- [Submit a Bug Report](#submit-a-bug-report) +- [Improvements to the Codebase](#improvements-to-the-codebase) +- [Understanding the Code Base](#understanding-the-codebase) +- [Codebase Overview](#codebase-overview) +- [Testing](#testing) +- [Style Guidelines & Naming Conventions](#style-guidelines-and-naming-conventions) +- [Creating a Pull Request](#creating-a-pull-request) +- [Code Reviews](#code-reviews) + + +We use [Milestones](https://github.com/sendgrid/sendgrid-nodejs/milestones) to help define current roadmaps, please feel free to grab an issue from the current milestone. Please indicate that you have begun work on it to avoid collisions. Once a PR is made, community review, comments, suggestions and additional PRs are welcomed and encouraged. + + +## CLAs and CCLAs + +Before you get started, SendGrid requires that a SendGrid Contributor License Agreement (CLA) be filled out by every contributor to a SendGrid open source project. + +Our goal with the CLA is to clarify the rights of our contributors and reduce other risks arising from inappropriate contributions. The CLA also clarifies the rights SendGrid holds in each contribution and helps to avoid misunderstandings over what rights each contributor is required to grant to SendGrid when making a contribution. In this way the CLA encourages broad participation by our open source community and helps us build strong open source projects, free from any individual contributor withholding or revoking rights to any contribution. + +SendGrid does not merge a pull request made against a SendGrid open source project until that pull request is associated with a signed CLA. Copies of the CLA are available [here](https://gist.github.com/SendGridDX/98b42c0a5d500058357b80278fde3be8#file-sendgrid_cla). + +When you create a Pull Request, after a few seconds, a comment will appear with a link to the CLA. Click the link and fill out the brief form and then click the "I agree" button and you are all set. You will not be asked to re-sign the CLA unless we make a change. + +There are a few ways to contribute, which we'll enumerate below: + + +## Feature Request + +If you'd like to make a feature request, please read this section. + +The GitHub [issue tracker](https://github.com/sendgrid/sendgrid-nodejs/issues) is the preferred channel for library feature requests, but please respect the following restrictions: + +- Please [**search for existing issues**](https://github.com/search?utf8=%E2%9C%93&q=repo%3Asendgrid%2Fsendgrid-nodejs&type=Issues) in order to ensure we don't have duplicate bugs/feature requests. +- Please be respectful and considerate of others when commenting on issues + + +## Submit a Bug Report + +Note: DO NOT include your credentials in ANY code examples, descriptions, or media you make public. + +A software bug is a demonstrable issue in the code base. In order for us to diagnose the issue and respond as quickly as possible, please add as much detail as possible into your bug report. + +Before you decide to create a new issue, please try the following: + +1. [Check the Github issues tab](https://github.com/sendgrid/sendgrid-nodejs/issues) if the identified issue has already been reported, if so, please add a +1 to the existing post. +2. Update to the latest version of this code and check if issue has already been fixed +3. Copy and fill in the Bug Report Template we have provided below + +### Please use our Bug Report Template + +In order to make the process easier, we've included a [sample bug report template](https://github.com/sendgrid/sendgrid-nodejs/blob/master/.github/ISSUE_TEMPLATE) (borrowed from [Ghost](https://github.com/TryGhost/Ghost/)). The template uses [GitHub flavored markdown](https://help.github.com/articles/github-flavored-markdown/) for formatting. + + +## Improvements to the Codebase + +We welcome direct contributions to the sendgrid-nodejs code base. Thank you! + +### Development Environment ### + +#### Install and Run Locally #### + +##### Prerequisites ##### + +- Node.js version 6, 7 or 8 +- Please see [package.json](https://github.com/sendgrid/sendgrid-nodejs/tree/master/package.json) + +##### Initial setup: ##### + +```bash +git clone https://github.com/sendgrid/sendgrid-nodejs.git +cd sendgrid-nodejs +npm install +``` + +## Environment Variables + +First, get your free SendGrid account [here](https://sendgrid.com/free?source=sendgrid-nodejs). + +You will need to setup the following environment to use the SendGrid examples in the [README.md](https://github.com/sendgrid/sendgrid-nodejs/blob/master/README.md), [USAGE.md](https://github.com/sendgrid/sendgrid-nodejs/blob/master/USAGE.MD) and [USE_CASES.md](https://github.com/sendgrid/sendgrid-nodejs/blob/master/USE_CASES.md) files: + +```bash +echo "export SENDGRID_API_KEY='YOUR_API_KEY'" > sendgrid.env +echo "sendgrid.env" >> .gitignore +source ./sendgrid.env +``` + +##### Execute: ##### + +To run an example: + +```bash +touch example.js +``` + +Copy the desired code into `example.js`. For this example, I'm assuming you create this file in the root of this project. + +Change the path to the Sendgrid library to the relative path, for example: `./packages/mail/mail`. + +```bash +node example.js +``` + + +## Understanding the Code Base + +This repo is organized as a monorepo with the packages residing in the `./packages` directory. Please see the root [README.md](https://github.com/sendgrid/sendgrid-nodejs/blob/master/README.md) for details. + + +## Codebase Overview + +This repo is subdivided in 6 main [packages](https://github.com/sendgrid/sendgrid-nodejs/tree/master/packages). Each package has its own dependencies (internal or external) and its own source code in the `src` folder. Each package also has its isolated ReadME files, use cases and usage.md files. + +To install a particular packages' dependencies. +```bash +cd packages/{NAME} +npm install or yarn install +``` +#### Package List + +**1. Client** +This is a wrapper written on top of the ```request``` module to suite the sendgrid module. All requests made to the sendgrid API are invoked by the `request` function in the `client.js`. + +Type declarations: client.d.ts +Test Cases: client.spec.js + +**2. Mail** +This module exposes the `send` function which send mails via the sdk. This module can be a good starting point to read the source code. + +Type declarations: mail.d.ts +Test Cases: mail.spec.js + +**3. Helpers** +These are a set of utility functions which all the modules use. Some of them are very basic functions and can be an easy starting point for reading the source code. + + +## Testing + +All PRs require passing tests before the PR will be reviewed. + +To run tests, please install Prism first by either running `yarn prism:install` or manually downloading from [the Prism website](https://stoplight.io/platform/prism/). + +Next, start Prism in one console window using `yarn prism`. + +Open a new console window and run `lerna bootstrap`. + +And finally, run `yarn test`, or specific tests e.g. `yarn test:mail` or `yarn test:client`. + + +## Style Guidelines & Naming Conventions + +Generally, we follow the style guidelines as suggested by the official language. However, we ask that you conform to the styles that already exist in the library. If you wish to deviate, please explain your reasoning. + +- [Unofficial Style Guide](https://github.com/felixge/node-style-guide) + +Please run your code through: + +- [ESLint](http://eslint.org/) with the standard style guide. +- [esdoc](https://github.com/sendgrid/sendgrid-nodejs/blob/master/.github/USAGE.md) to check the documentation coverage of your added code. + +## Creating a Pull Request + +1. [Fork](https://help.github.com/fork-a-repo/) the project, clone your fork, + and configure the remotes: + + ```bash + # Clone your fork of the repo into the current directory + git clone https://github.com/sendgrid/sendgrid-nodejs + + # Navigate to the newly cloned directory + cd sendgrid-nodejs + + # Assign the original repo to a remote called "upstream" + git remote add upstream https://github.com/sendgrid/sendgrid-nodejs + ``` + +2. If you cloned a while ago, get the latest changes from upstream: + + ```bash + git checkout + git pull upstream + ``` + +3. Create a new topic branch (off the main project development branch) to + contain your feature, change, or fix: + + ```bash + git checkout -b + ``` + +4. Commit your changes in logical chunks. Please adhere to these [git commit + message guidelines](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html) + or your code is unlikely be merged into the main project. Use Git's + [interactive rebase](https://help.github.com/articles/interactive-rebase) + feature to tidy up your commits before making them public. + + 4a. Create tests. + + 4b. Create or update the example code that demonstrates the functionality of this change to the code. + +5. Locally merge (or rebase) the upstream development branch into your topic branch: + + ```bash + git pull [--rebase] upstream master + ``` + +6. Push your topic branch up to your fork: + + ```bash + git push origin + ``` + +7. [Open a Pull Request](https://help.github.com/articles/using-pull-requests/) + with a clear title and description against the `master` branch. All tests must be passing before we will review the PR. + +If you have any additional questions, please feel free to [email](mailto:dx@sendgrid.com) us or create an issue in this repo. + + +## Code Reviews + +If you can, please look at open PRs and review them. Give feedback and help us merge these PRs much faster! If you don't know how, Github has some [great information on how to review a Pull Request](https://help.github.com/articles/about-pull-request-reviews/). diff --git a/examples/inbound-parse-docker/Dockerfile b/examples/inbound-parse-docker/Dockerfile new file mode 100644 index 000000000..ce85231e0 --- /dev/null +++ b/examples/inbound-parse-docker/Dockerfile @@ -0,0 +1,8 @@ +FROM node:8.9-alpine +ENV NODE_ENV production +WORKDIR /usr/src/app +COPY ["package.json", "package-lock.json*", "npm-shrinkwrap.json*", "./"] +RUN npm install --production --silent && mv node_modules ../ +COPY . . +EXPOSE 3000 +CMD npm start \ No newline at end of file diff --git a/examples/inbound-parse-docker/LICENSE b/examples/inbound-parse-docker/LICENSE new file mode 100644 index 000000000..8fc986624 --- /dev/null +++ b/examples/inbound-parse-docker/LICENSE @@ -0,0 +1,15 @@ +Copyright (c) 2012-2018 SendGrid, Inc. + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated +documentation files (the "Software"), to deal in the Software without restriction, including without limitation +the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, +and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of +the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO +THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF +CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +DEALINGS IN THE SOFTWARE. \ No newline at end of file diff --git a/examples/inbound-parse-docker/README.md b/examples/inbound-parse-docker/README.md new file mode 100644 index 000000000..ba2cd2019 --- /dev/null +++ b/examples/inbound-parse-docker/README.md @@ -0,0 +1,21 @@ +# Introduction + +This is an example project for using a Docker container as a webhook receiver. + +# Prerequisites + +* Docker CE 18 +* SendGrid Account with inbound parse enabled +* Ngrok or Internet accessible URL + +# Usage + +* `cd ` +* `docker-compose build && docker-compose up` + +At the moment, the `app.js` only prints data to the console. You can extend this project by adding more business logic to the `/parse_webhook` route. + +# License + +See [LICENSE](LICENSE) + diff --git a/examples/inbound-parse-docker/app.js b/examples/inbound-parse-docker/app.js new file mode 100644 index 000000000..5a1d46367 --- /dev/null +++ b/examples/inbound-parse-docker/app.js @@ -0,0 +1,27 @@ +const express = require('express') +const app = express() +const formidable = require('express-formidable') +const port = 3000 + +app.use(formidable()) + + + +app.get('/', (req, res) => res.send('Hello World!')) + +app.post('/parse_webhook', (req, res) => { + console.log('---------- RECEIVED WEBHOOK DATA ----------') + // Email data comes in as a form. Using express-formidable to + // handle the form data. Form fields are available in req.fields + // Below, extracting the from and text. + // You can take this data and do something more interesting with it + // such as sending it to a database. + console.log() + console.log('FROM: ' + req.fields.from) + console.log() + console.log('MESSAGE TEXT: ' + req.fields.text) + + res.sendStatus(200) +}) + +app.listen(port, () => console.log(`SendGrid Inbound Parse webhook listener started on port ${port}!`)) \ No newline at end of file diff --git a/examples/inbound-parse-docker/docker-compose.debug.yml b/examples/inbound-parse-docker/docker-compose.debug.yml new file mode 100644 index 000000000..4e56ea477 --- /dev/null +++ b/examples/inbound-parse-docker/docker-compose.debug.yml @@ -0,0 +1,13 @@ +version: '2.1' + +services: + inbound-parse-docker: + image: inbound-parse-docker + build: . + environment: + NODE_ENV: development + ports: + - 3000:3000 + - 9229:9229 + ## set your startup file here + command: node --inspect index.js \ No newline at end of file diff --git a/examples/inbound-parse-docker/docker-compose.yml b/examples/inbound-parse-docker/docker-compose.yml new file mode 100644 index 000000000..c0a1eb970 --- /dev/null +++ b/examples/inbound-parse-docker/docker-compose.yml @@ -0,0 +1,10 @@ +version: '2.1' + +services: + inbound-parse-docker: + image: inbound-parse-docker + build: . + environment: + NODE_ENV: production + ports: + - 3000:3000 \ No newline at end of file diff --git a/examples/inbound-parse-docker/package-lock.json b/examples/inbound-parse-docker/package-lock.json new file mode 100644 index 000000000..cb49e7cde --- /dev/null +++ b/examples/inbound-parse-docker/package-lock.json @@ -0,0 +1,555 @@ +{ + "name": "inbound-parse-docker", + "version": "1.0.0", + "lockfileVersion": 1, + "requires": true, + "dependencies": { + "@sendgrid/helpers": { + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/@sendgrid/helpers/-/helpers-6.3.0.tgz", + "integrity": "sha512-uTFcmhCDFg/2Uhz+z/cLwyLHH0UsblG49hKwdR7nKbWsGKWv4js7W32FlPdXqy2C/plTJ20vcPLgKM1m3F/MjQ==", + "requires": { + "chalk": "^2.0.1", + "deepmerge": "^2.1.1" + } + }, + "@sendgrid/inbound-mail-parser": { + "version": "6.3.0", + "resolved": "https://registry.npmjs.org/@sendgrid/inbound-mail-parser/-/inbound-mail-parser-6.3.0.tgz", + "integrity": "sha512-4DzcoyHRm9yAYMfeuKpmGpXx7pQHtss865rib6uVCimReSDzjVMslbECWCBb8u2rcfdqvlBZGjqheBU9y95joA==", + "requires": { + "@sendgrid/helpers": "^6.3.0", + "mailparser": "^0.6.1" + } + }, + "accepts": { + "version": "1.3.5", + "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.5.tgz", + "integrity": "sha1-63d99gEXI6OxTopywIBcjoZ0a9I=", + "requires": { + "mime-types": "~2.1.18", + "negotiator": "0.6.1" + } + }, + "addressparser": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/addressparser/-/addressparser-1.0.1.tgz", + "integrity": "sha1-R6++GiqSYhkdtoOOT9HTm0CCF0Y=" + }, + "ansi-styles": { + "version": "3.2.1", + "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-3.2.1.tgz", + "integrity": "sha512-VT0ZI6kZRdTh8YyJw3SMbYm/u+NqfsAxEpWO0Pf9sq8/e94WxxOpPKx9FR1FlyCtOVDNOQ+8ntlqFxiRc+r5qA==", + "requires": { + "color-convert": "^1.9.0" + } + }, + "array-flatten": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz", + "integrity": "sha1-ml9pkFGx5wczKPKgCJaLZOopVdI=" + }, + "body-parser": { + "version": "1.18.3", + "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.18.3.tgz", + "integrity": "sha1-WykhmP/dVTs6DyDe0FkrlWlVyLQ=", + "requires": { + "bytes": "3.0.0", + "content-type": "~1.0.4", + "debug": "2.6.9", + "depd": "~1.1.2", + "http-errors": "~1.6.3", + "iconv-lite": "0.4.23", + "on-finished": "~2.3.0", + "qs": "6.5.2", + "raw-body": "2.3.3", + "type-is": "~1.6.16" + }, + "dependencies": { + "qs": { + "version": "6.5.2", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.5.2.tgz", + "integrity": "sha512-N5ZAX4/LxJmF+7wN74pUD6qAh9/wnvdQcjq9TZjevvXzSUo7bfmw91saqMjzGS2xq91/odN2dW/WOl7qQHNDGA==" + } + } + }, + "bytes": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/bytes/-/bytes-3.0.0.tgz", + "integrity": "sha1-0ygVQE1olpn4Wk6k+odV3ROpYEg=" + }, + "chalk": { + "version": "2.4.1", + "resolved": "https://registry.npmjs.org/chalk/-/chalk-2.4.1.tgz", + "integrity": "sha512-ObN6h1v2fTJSmUXoS3nMQ92LbDK9be4TV+6G+omQlGJFdcUX5heKi1LZ1YnRMIgwTLEj3E24bT6tYni50rlCfQ==", + "requires": { + "ansi-styles": "^3.2.1", + "escape-string-regexp": "^1.0.5", + "supports-color": "^5.3.0" + } + }, + "color-convert": { + "version": "1.9.3", + "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-1.9.3.tgz", + "integrity": "sha512-QfAUtd+vFdAtFQcC8CCyYt1fYWxSqAiK2cSD6zDB8N3cpsEBAvRxp9zOGg6G/SHHJYAT88/az/IuDGALsNVbGg==", + "requires": { + "color-name": "1.1.3" + } + }, + "color-name": { + "version": "1.1.3", + "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.3.tgz", + "integrity": "sha1-p9BVi9icQveV3UIyj3QIMcpTvCU=" + }, + "content-disposition": { + "version": "0.5.2", + "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.2.tgz", + "integrity": "sha1-DPaLud318r55YcOoUXjLhdunjLQ=" + }, + "content-type": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/content-type/-/content-type-1.0.4.tgz", + "integrity": "sha512-hIP3EEPs8tB9AT1L+NUqtwOAps4mk2Zob89MWXMHjHWg9milF/j4osnnQLXBCBFBk/tvIG/tUc9mOUJiPBhPXA==" + }, + "cookie": { + "version": "0.3.1", + "resolved": "https://registry.npmjs.org/cookie/-/cookie-0.3.1.tgz", + "integrity": "sha1-5+Ch+e9DtMi6klxcWpboBtFoc7s=" + }, + "cookie-signature": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.6.tgz", + "integrity": "sha1-4wOogrNCzD7oylE6eZmXNNqzriw=" + }, + "debug": { + "version": "2.6.9", + "resolved": "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz", + "integrity": "sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==", + "requires": { + "ms": "2.0.0" + } + }, + "deepmerge": { + "version": "2.2.1", + "resolved": "https://registry.npmjs.org/deepmerge/-/deepmerge-2.2.1.tgz", + "integrity": "sha512-R9hc1Xa/NOBi9WRVUWg19rl1UB7Tt4kuPd+thNJgFZoxXsTz7ncaPaeIm+40oSGuP33DfMb4sZt1QIGiJzC4EA==" + }, + "depd": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/depd/-/depd-1.1.2.tgz", + "integrity": "sha1-m81S4UwJd2PnSbJ0xDRu0uVgtak=" + }, + "destroy": { + "version": "1.0.4", + "resolved": "https://registry.npmjs.org/destroy/-/destroy-1.0.4.tgz", + "integrity": "sha1-l4hXRCxEdJ5CBmE+N5RiBYJqvYA=" + }, + "ee-first": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz", + "integrity": "sha1-WQxhFWsK4vTwJVcyoViyZrxWsh0=" + }, + "encodeurl": { + "version": "1.0.2", + "resolved": "https://registry.npmjs.org/encodeurl/-/encodeurl-1.0.2.tgz", + "integrity": "sha1-rT/0yG7C0CkyL1oCw6mmBslbP1k=" + }, + "encoding": { + "version": "0.1.12", + "resolved": "https://registry.npmjs.org/encoding/-/encoding-0.1.12.tgz", + "integrity": "sha1-U4tm8+5izRq1HsMjgp0flIDHS+s=", + "requires": { + "iconv-lite": "~0.4.13" + } + }, + "escape-html": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz", + "integrity": "sha1-Aljq5NPQwJdN4cFpGI7wBR0dGYg=" + }, + "escape-string-regexp": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz", + "integrity": "sha1-G2HAViGQqN/2rjuyzwIAyhMLhtQ=" + }, + "etag": { + "version": "1.8.1", + "resolved": "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz", + "integrity": "sha1-Qa4u62XvpiJorr/qg6x9eSmbCIc=" + }, + "express": { + "version": "4.16.3", + "resolved": "http://registry.npmjs.org/express/-/express-4.16.3.tgz", + "integrity": "sha1-avilAjUNsyRuzEvs9rWjTSL37VM=", + "requires": { + "accepts": "~1.3.5", + "array-flatten": "1.1.1", + "body-parser": "1.18.2", + "content-disposition": "0.5.2", + "content-type": "~1.0.4", + "cookie": "0.3.1", + "cookie-signature": "1.0.6", + "debug": "2.6.9", + "depd": "~1.1.2", + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "etag": "~1.8.1", + "finalhandler": "1.1.1", + "fresh": "0.5.2", + "merge-descriptors": "1.0.1", + "methods": "~1.1.2", + "on-finished": "~2.3.0", + "parseurl": "~1.3.2", + "path-to-regexp": "0.1.7", + "proxy-addr": "~2.0.3", + "qs": "6.5.1", + "range-parser": "~1.2.0", + "safe-buffer": "5.1.1", + "send": "0.16.2", + "serve-static": "1.13.2", + "setprototypeof": "1.1.0", + "statuses": "~1.4.0", + "type-is": "~1.6.16", + "utils-merge": "1.0.1", + "vary": "~1.1.2" + }, + "dependencies": { + "body-parser": { + "version": "1.18.2", + "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.18.2.tgz", + "integrity": "sha1-h2eKGdhLR9hZuDGZvVm84iKxBFQ=", + "requires": { + "bytes": "3.0.0", + "content-type": "~1.0.4", + "debug": "2.6.9", + "depd": "~1.1.1", + "http-errors": "~1.6.2", + "iconv-lite": "0.4.19", + "on-finished": "~2.3.0", + "qs": "6.5.1", + "raw-body": "2.3.2", + "type-is": "~1.6.15" + } + }, + "iconv-lite": { + "version": "0.4.19", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.19.tgz", + "integrity": "sha512-oTZqweIP51xaGPI4uPa56/Pri/480R+mo7SeU+YETByQNhDG55ycFyNLIgta9vXhILrxXDmF7ZGhqZIcuN0gJQ==" + }, + "raw-body": { + "version": "2.3.2", + "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.3.2.tgz", + "integrity": "sha1-vNYMd9Prk83gBQKVw/N5OJvIj4k=", + "requires": { + "bytes": "3.0.0", + "http-errors": "1.6.2", + "iconv-lite": "0.4.19", + "unpipe": "1.0.0" + }, + "dependencies": { + "depd": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/depd/-/depd-1.1.1.tgz", + "integrity": "sha1-V4O04cRZ8G+lyif5kfPQbnoxA1k=" + }, + "http-errors": { + "version": "1.6.2", + "resolved": "https://registry.npmjs.org/http-errors/-/http-errors-1.6.2.tgz", + "integrity": "sha1-CgAsyFcHGSp+eUbO7cERVfYOxzY=", + "requires": { + "depd": "1.1.1", + "inherits": "2.0.3", + "setprototypeof": "1.0.3", + "statuses": ">= 1.3.1 < 2" + } + }, + "setprototypeof": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.0.3.tgz", + "integrity": "sha1-ZlZ+NwQ+608E2RvWWMDL77VbjgQ=" + } + } + } + } + }, + "express-formidable": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/express-formidable/-/express-formidable-1.0.0.tgz", + "integrity": "sha1-3JIvBFUTIyJFip7BowHYkbP/yo0=", + "requires": { + "formidable": "^1.0.17" + } + }, + "extend": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz", + "integrity": "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==" + }, + "finalhandler": { + "version": "1.1.1", + "resolved": "http://registry.npmjs.org/finalhandler/-/finalhandler-1.1.1.tgz", + "integrity": "sha512-Y1GUDo39ez4aHAw7MysnUD5JzYX+WaIj8I57kO3aEPT1fFRL4sr7mjei97FgnwhAyyzRYmQZaTHb2+9uZ1dPtg==", + "requires": { + "debug": "2.6.9", + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "on-finished": "~2.3.0", + "parseurl": "~1.3.2", + "statuses": "~1.4.0", + "unpipe": "~1.0.0" + } + }, + "formidable": { + "version": "1.2.1", + "resolved": "https://registry.npmjs.org/formidable/-/formidable-1.2.1.tgz", + "integrity": "sha512-Fs9VRguL0gqGHkXS5GQiMCr1VhZBxz0JnJs4JmMp/2jL18Fmbzvv7vOFRU+U8TBkHEE/CX1qDXzJplVULgsLeg==" + }, + "forwarded": { + "version": "0.1.2", + "resolved": "https://registry.npmjs.org/forwarded/-/forwarded-0.1.2.tgz", + "integrity": "sha1-mMI9qxF1ZXuMBXPozszZGw/xjIQ=" + }, + "fresh": { + "version": "0.5.2", + "resolved": "https://registry.npmjs.org/fresh/-/fresh-0.5.2.tgz", + "integrity": "sha1-PYyt2Q2XZWn6g1qx+OSyOhBWBac=" + }, + "has-flag": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/has-flag/-/has-flag-3.0.0.tgz", + "integrity": "sha1-tdRU3CGZriJWmfNGfloH87lVuv0=" + }, + "http-errors": { + "version": "1.6.3", + "resolved": "http://registry.npmjs.org/http-errors/-/http-errors-1.6.3.tgz", + "integrity": "sha1-i1VoC7S+KDoLW/TqLjhYC+HZMg0=", + "requires": { + "depd": "~1.1.2", + "inherits": "2.0.3", + "setprototypeof": "1.1.0", + "statuses": ">= 1.4.0 < 2" + } + }, + "iconv-lite": { + "version": "0.4.23", + "resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.4.23.tgz", + "integrity": "sha512-neyTUVFtahjf0mB3dZT77u+8O0QB89jFdnBkd5P1JgYPbPaia3gXXOVL2fq8VyU2gMMD7SaN7QukTB/pmXYvDA==", + "requires": { + "safer-buffer": ">= 2.1.2 < 3" + } + }, + "inherits": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.3.tgz", + "integrity": "sha1-Yzwsg+PaQqUC9SRmAiSA9CCCYd4=" + }, + "ipaddr.js": { + "version": "1.8.0", + "resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.8.0.tgz", + "integrity": "sha1-6qM9bd16zo9/b+DJygRA5wZzix4=" + }, + "mailparser": { + "version": "0.6.2", + "resolved": "https://registry.npmjs.org/mailparser/-/mailparser-0.6.2.tgz", + "integrity": "sha1-A8SGA5vfTfbNO2rcqqxBB9/bwGg=", + "requires": { + "encoding": "^0.1.12", + "mime": "^1.3.4", + "mimelib": "^0.3.0", + "uue": "^3.1.0" + } + }, + "media-typer": { + "version": "0.3.0", + "resolved": "https://registry.npmjs.org/media-typer/-/media-typer-0.3.0.tgz", + "integrity": "sha1-hxDXrwqmJvj/+hzgAWhUUmMlV0g=" + }, + "merge-descriptors": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-1.0.1.tgz", + "integrity": "sha1-sAqqVW3YtEVoFQ7J0blT8/kMu2E=" + }, + "methods": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/methods/-/methods-1.1.2.tgz", + "integrity": "sha1-VSmk1nZUE07cxSZmVoNbD4Ua/O4=" + }, + "mime": { + "version": "1.4.1", + "resolved": "https://registry.npmjs.org/mime/-/mime-1.4.1.tgz", + "integrity": "sha512-KI1+qOZu5DcW6wayYHSzR/tXKCDC5Om4s1z2QJjDULzLcmf3DvzS7oluY4HCTrc+9FiKmWUgeNLg7W3uIQvxtQ==" + }, + "mime-db": { + "version": "1.36.0", + "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.36.0.tgz", + "integrity": "sha512-L+xvyD9MkoYMXb1jAmzI/lWYAxAMCPvIBSWur0PZ5nOf5euahRLVqH//FKW9mWp2lkqUgYiXPgkzfMUFi4zVDw==" + }, + "mime-types": { + "version": "2.1.20", + "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.20.tgz", + "integrity": "sha512-HrkrPaP9vGuWbLK1B1FfgAkbqNjIuy4eHlIYnFi7kamZyLLrGlo2mpcx0bBmNpKqBtYtAfGbodDddIgddSJC2A==", + "requires": { + "mime-db": "~1.36.0" + } + }, + "mimelib": { + "version": "0.3.1", + "resolved": "https://registry.npmjs.org/mimelib/-/mimelib-0.3.1.tgz", + "integrity": "sha1-eHrdJBXYJ6yzr27EvKHqlZZBiFM=", + "requires": { + "addressparser": "~1.0.1", + "encoding": "~0.1.12" + } + }, + "ms": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/ms/-/ms-2.0.0.tgz", + "integrity": "sha1-VgiurfwAvmwpAd9fmGF4jeDVl8g=" + }, + "negotiator": { + "version": "0.6.1", + "resolved": "https://registry.npmjs.org/negotiator/-/negotiator-0.6.1.tgz", + "integrity": "sha1-KzJxhOiZIQEXeyhWP7XnECrNDKk=" + }, + "on-finished": { + "version": "2.3.0", + "resolved": "https://registry.npmjs.org/on-finished/-/on-finished-2.3.0.tgz", + "integrity": "sha1-IPEzZIGwg811M3mSoWlxqi2QaUc=", + "requires": { + "ee-first": "1.1.1" + } + }, + "parseurl": { + "version": "1.3.2", + "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.2.tgz", + "integrity": "sha1-/CidTtiZMRlGDBViUyYs3I3mW/M=" + }, + "path-to-regexp": { + "version": "0.1.7", + "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-0.1.7.tgz", + "integrity": "sha1-32BBeABfUi8V60SQ5yR6G/qmf4w=" + }, + "proxy-addr": { + "version": "2.0.4", + "resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.4.tgz", + "integrity": "sha512-5erio2h9jp5CHGwcybmxmVqHmnCBZeewlfJ0pex+UW7Qny7OOZXTtH56TGNyBizkgiOwhJtMKrVzDTeKcySZwA==", + "requires": { + "forwarded": "~0.1.2", + "ipaddr.js": "1.8.0" + } + }, + "qs": { + "version": "6.5.1", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.5.1.tgz", + "integrity": "sha512-eRzhrN1WSINYCDCbrz796z37LOe3m5tmW7RQf6oBntukAG1nmovJvhnwHHRMAfeoItc1m2Hk02WER2aQ/iqs+A==" + }, + "range-parser": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.0.tgz", + "integrity": "sha1-9JvmtIeJTdxA3MlKMi9hEJLgDV4=" + }, + "raw-body": { + "version": "2.3.3", + "resolved": "https://registry.npmjs.org/raw-body/-/raw-body-2.3.3.tgz", + "integrity": "sha512-9esiElv1BrZoI3rCDuOuKCBRbuApGGaDPQfjSflGxdy4oyzqghxu6klEkkVIvBje+FF0BX9coEv8KqW6X/7njw==", + "requires": { + "bytes": "3.0.0", + "http-errors": "1.6.3", + "iconv-lite": "0.4.23", + "unpipe": "1.0.0" + } + }, + "safe-buffer": { + "version": "5.1.1", + "resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.1.1.tgz", + "integrity": "sha512-kKvNJn6Mm93gAczWVJg7wH+wGYWNrDHdWvpUmHyEsgCtIwwo3bqPtV4tR5tuPaUhTOo/kvhVwd8XwwOllGYkbg==" + }, + "safer-buffer": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz", + "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==" + }, + "send": { + "version": "0.16.2", + "resolved": "https://registry.npmjs.org/send/-/send-0.16.2.tgz", + "integrity": "sha512-E64YFPUssFHEFBvpbbjr44NCLtI1AohxQ8ZSiJjQLskAdKuriYEP6VyGEsRDH8ScozGpkaX1BGvhanqCwkcEZw==", + "requires": { + "debug": "2.6.9", + "depd": "~1.1.2", + "destroy": "~1.0.4", + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "etag": "~1.8.1", + "fresh": "0.5.2", + "http-errors": "~1.6.2", + "mime": "1.4.1", + "ms": "2.0.0", + "on-finished": "~2.3.0", + "range-parser": "~1.2.0", + "statuses": "~1.4.0" + } + }, + "serve-static": { + "version": "1.13.2", + "resolved": "https://registry.npmjs.org/serve-static/-/serve-static-1.13.2.tgz", + "integrity": "sha512-p/tdJrO4U387R9oMjb1oj7qSMaMfmOyd4j9hOFoxZe2baQszgHcSWjuya/CiT5kgZZKRudHNOA0pYXOl8rQ5nw==", + "requires": { + "encodeurl": "~1.0.2", + "escape-html": "~1.0.3", + "parseurl": "~1.3.2", + "send": "0.16.2" + } + }, + "setprototypeof": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.1.0.tgz", + "integrity": "sha512-BvE/TwpZX4FXExxOxZyRGQQv651MSwmWKZGqvmPcRIjDqWub67kTKuIMx43cZZrS/cBBzwBcNDWoFxt2XEFIpQ==" + }, + "statuses": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/statuses/-/statuses-1.4.0.tgz", + "integrity": "sha512-zhSCtt8v2NDrRlPQpCNtw/heZLtfUDqxBM1udqikb/Hbk52LK4nQSwr10u77iopCW5LsyHpuXS0GnEc48mLeew==" + }, + "supports-color": { + "version": "5.5.0", + "resolved": "https://registry.npmjs.org/supports-color/-/supports-color-5.5.0.tgz", + "integrity": "sha512-QjVjwdXIt408MIiAqCX4oUKsgU2EqAGzs2Ppkm4aQYbjm+ZEWEcW4SfFNTr4uMNZma0ey4f5lgLrkB0aX0QMow==", + "requires": { + "has-flag": "^3.0.0" + } + }, + "type-is": { + "version": "1.6.16", + "resolved": "https://registry.npmjs.org/type-is/-/type-is-1.6.16.tgz", + "integrity": "sha512-HRkVv/5qY2G6I8iab9cI7v1bOIdhm94dVjQCPFElW9W+3GeDOSHmy2EBYe4VTApuzolPcmgFTN3ftVJRKR2J9Q==", + "requires": { + "media-typer": "0.3.0", + "mime-types": "~2.1.18" + } + }, + "unpipe": { + "version": "1.0.0", + "resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz", + "integrity": "sha1-sr9O6FFKrmFltIF4KdIbLvSZBOw=" + }, + "utils-merge": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz", + "integrity": "sha1-n5VxD1CiZ5R7LMwSR0HBAoQn5xM=" + }, + "uue": { + "version": "3.1.2", + "resolved": "https://registry.npmjs.org/uue/-/uue-3.1.2.tgz", + "integrity": "sha512-axKLXVqwtdI/czrjG0X8hyV1KLgeWx8F4KvSbvVCnS+RUvsQMGRjx0kfuZDXXqj0LYvVJmx3B9kWlKtEdRrJLg==", + "requires": { + "escape-string-regexp": "~1.0.5", + "extend": "~3.0.0" + } + }, + "vary": { + "version": "1.1.2", + "resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz", + "integrity": "sha1-IpnwLG3tMNSllhsLn3RSShj2NPw=" + } + } +} diff --git a/examples/inbound-parse-docker/package.json b/examples/inbound-parse-docker/package.json new file mode 100644 index 000000000..2258b7322 --- /dev/null +++ b/examples/inbound-parse-docker/package.json @@ -0,0 +1,17 @@ +{ + "name": "inbound-parse-docker", + "version": "1.0.0", + "description": "A containter to capture webhook data from SendGrid inbound parse", + "main": "app.js", + "dependencies": { + "express": "^4.16.3", + "express-formidable": "^1.0.0" + }, + "devDependencies": {}, + "scripts": { + "start": "node app.js", + "test": "echo \"Error: no test specified\" && exit 1" + }, + "author": "Ashley Roach ", + "license": "MIT" +} From 771a0ffbf701ef116ca055a9f01124058c85711c Mon Sep 17 00:00:00 2001 From: Ashley Roach Date: Mon, 8 Oct 2018 22:05:09 -0600 Subject: [PATCH 2/9] Update console logging with all parsed values --- examples/inbound-parse-docker/README.md | 2 +- examples/inbound-parse-docker/app.js | 35 ++++++++++++++++++- .../docker-compose.debug.yml | 2 +- .../inbound-parse-docker/docker-compose.yml | 4 ++- 4 files changed, 39 insertions(+), 4 deletions(-) diff --git a/examples/inbound-parse-docker/README.md b/examples/inbound-parse-docker/README.md index ba2cd2019..09083b57c 100644 --- a/examples/inbound-parse-docker/README.md +++ b/examples/inbound-parse-docker/README.md @@ -5,7 +5,7 @@ This is an example project for using a Docker container as a webhook receiver. # Prerequisites * Docker CE 18 -* SendGrid Account with inbound parse enabled +* SendGrid Account with inbound parse enabled to send form data (default) * Ngrok or Internet accessible URL # Usage diff --git a/examples/inbound-parse-docker/app.js b/examples/inbound-parse-docker/app.js index 5a1d46367..d3c12afb5 100644 --- a/examples/inbound-parse-docker/app.js +++ b/examples/inbound-parse-docker/app.js @@ -10,16 +10,49 @@ app.use(formidable()) app.get('/', (req, res) => res.send('Hello World!')) app.post('/parse_webhook', (req, res) => { - console.log('---------- RECEIVED WEBHOOK DATA ----------') + console.log('---------- START RECEIVED WEBHOOK DATA ----------') // Email data comes in as a form. Using express-formidable to // handle the form data. Form fields are available in req.fields // Below, extracting the from and text. // You can take this data and do something more interesting with it // such as sending it to a database. + // Attachments are stored in /tmp based on the default configuration + // of the formidable middleware. + // console.log(req.fields) + console.log() + console.log('HEADERS: ' + req.fields.headers) + console.log() + console.log('DKIM: ' + req.fields.dkim) + // TODO: The server is sending a non JS compliant key name + // console.log() + // console.log('CONTENT-IDS: ' + req.fields.content-ids) + console.log() + console.log('TO: ' + req.fields.to) + console.log() + console.log('HTML: ' + req.fields.html) console.log() console.log('FROM: ' + req.fields.from) console.log() + console.log('SENDER-IP: ' + req.fields.sender_ip) + console.log() + console.log('SPAM-REPORT: ' + req.fields.spam_report) + console.log() + console.log('ENVELOPE: ' + req.fields.envelope) + console.log() + console.log('ATTACHMENTS: ' + req.fields.attachments) + console.log() + console.log('SPAM-SCORE: ' + req.fields.spam_score) + // TODO: The server is sending a non JS compliant key name + // console.log() + // console.log('ATTACHMENT-INFO: ' + req.fields.attachment-info) + console.log() + console.log('CHARSETS: ' + req.fields.charsets) + console.log() + console.log('SPF: ' + req.fields.SPF) + console.log() console.log('MESSAGE TEXT: ' + req.fields.text) + console.log() + console.log('---------- END RECEIVED WEBHOOK DATA ----------') res.sendStatus(200) }) diff --git a/examples/inbound-parse-docker/docker-compose.debug.yml b/examples/inbound-parse-docker/docker-compose.debug.yml index 4e56ea477..02eebea8f 100644 --- a/examples/inbound-parse-docker/docker-compose.debug.yml +++ b/examples/inbound-parse-docker/docker-compose.debug.yml @@ -10,4 +10,4 @@ services: - 3000:3000 - 9229:9229 ## set your startup file here - command: node --inspect index.js \ No newline at end of file + command: node --inspect app.js \ No newline at end of file diff --git a/examples/inbound-parse-docker/docker-compose.yml b/examples/inbound-parse-docker/docker-compose.yml index c0a1eb970..e823b72be 100644 --- a/examples/inbound-parse-docker/docker-compose.yml +++ b/examples/inbound-parse-docker/docker-compose.yml @@ -7,4 +7,6 @@ services: environment: NODE_ENV: production ports: - - 3000:3000 \ No newline at end of file + - 3000:3000 + # volumes: + # - /tmp:/tmp \ No newline at end of file From 0cb87a6e45d04f6b18134eaea59da9d870accbb0 Mon Sep 17 00:00:00 2001 From: Ashley Roach Date: Mon, 8 Oct 2018 22:05:17 -0600 Subject: [PATCH 3/9] Add example payload --- .../example-webhook-payload.txt | 104 ++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 examples/inbound-parse-docker/example-webhook-payload.txt diff --git a/examples/inbound-parse-docker/example-webhook-payload.txt b/examples/inbound-parse-docker/example-webhook-payload.txt new file mode 100644 index 000000000..697296a8f --- /dev/null +++ b/examples/inbound-parse-docker/example-webhook-payload.txt @@ -0,0 +1,104 @@ +--xYzZY +Content-Disposition: form-data; name="headers" + +Received: by mx0032p1las1.sendgrid.net with SMTP id NRnKey2i6C Tue, 09 Oct 2018 03:16:04 +0000 (UTC) +Received: from mail-yw1-f46.google.com (mail-yw1-f46.google.com [209.85.161.46]) by mx0032p1las1.sendgrid.net (Postfix) with ESMTPS id 1D3BB625D2C for ; Tue, 9 Oct 2018 03:16:04 +0000 (UTC) +Received: by mail-yw1-f46.google.com with SMTP id j75-v6so51762ywj.10 for ; Mon, 08 Oct 2018 20:16:04 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=gaGIowPBiLdos6k2wdsmJawftIRJniyY+QsmcvDWWCA=; b=PnAWpmAy0rbKREK7IOCzDQQeUepqTbv01k8bC72VtSVjRMpYpPsZhR/EUDMVaB+i4/ mng6byw7E62qQykPETggRRaEE8Kh3HDfs6lvQruX+SQZQeNOYiKR1NtrVHw2EL8XQj9C K/64HtprGtCgxA38HFBugUxjNHY0okdaK8tIuxKPC6fyW7YKP8Z50FC9oJO1ZcrZF3y0 nxewJEeyO8dxARX9TABZ5EbAvLqRic3Rjd/5T8j0uMfO+RegC2VUUdhEzQT/98mVzlL/ J7Iibp0PD7vFtLeb3Vm2RhczhrE9cP9SjceuifHbMhBpmBc9AHSqHIcrIGwgwE/Fqqci sn2Q== +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=gaGIowPBiLdos6k2wdsmJawftIRJniyY+QsmcvDWWCA=; b=gNDjNwNhOGTB1wXZ7ygXrStMGBVNpK6FrbVq9R06SR2bRPPHoC+lRK1JcMOs5Jz5RK +CQf8PyEIqe3SjqQQl7ThgrwrwM8S3ibQTUzpVnHDcxjmkgrOz8VTFC5TaikM8PhwHUy mw+vYe3Ukc3YclGX1QTrc2VjDJlGED9CYoACDFBjmgG7vn6Qi31n8ZZyR8fi6nvDB6k9 GyQjParwVM565eTX05Gw5SVd97MJ/8Sbpwu2IVM3Xn6CABcCeeMlyER4m5UnfJJAic/5 Ck11za013McUgSp1vPlpRFdFFHTv+z1++C6rYB1qH0Nnml9yxB9bSwE//csBAmMjimR+ fpvg== +X-Gm-Message-State: ABuFfogvHoUAi4aTQkKUQ3MTlDdwo/mh62FXdjJigY5caYWAbiMEYXLz e1OGhMqAJjkCeSW2MeihTbZe70+uShTCQ2DRi+LcyA== +X-Google-Smtp-Source: ACcGV634pgBc3Hl7xI6l2dh2DRIZ89uXLaYuE0zBoFCo3YI9n5F2EyfcVhonIDBVJUwTN+pJ94AhD2zd7++B4tde87w= +X-Received: by 2002:a0d:e445:: with SMTP id n66-v6mr15080932ywe.120.1539054962910; Mon, 08 Oct 2018 20:16:02 -0700 (PDT) +MIME-Version: 1.0 +References: +In-Reply-To: +From: Ashley Roach +Date: Tue, 9 Oct 2018 03:15:50 +0000 +Message-ID: +Subject: Fwd: testing webhook parse +To: test@parse.ashleyroach.com +Content-Type: multipart/alternative; boundary="0000000000005aac2a0577c32893" + +--xYzZY +Content-Disposition: form-data; name="dkim" + +{@gmail.com : pass} +--xYzZY +Content-Disposition: form-data; name="to" + +test@parse.ashleyroach.com +--xYzZY +Content-Disposition: form-data; name="html" + +


---------- Forwarded message ---------
From: Ashley Roach <aroach@gmail.com>
Date: Mon, Oct 8, 2018 at 7:33 PM
Subject: testing webhook parse
To: <test@parse.ashleyroach.com>


this is the body for webhook parse
+
+ +--xYzZY +Content-Disposition: form-data; name="from" + +Ashley Roach +--xYzZY +Content-Disposition: form-data; name="text" + +---------- Forwarded message --------- +From: Ashley Roach +Date: Mon, Oct 8, 2018 at 7:33 PM +Subject: testing webhook parse +To: + + +this is the body for webhook parse + +--xYzZY +Content-Disposition: form-data; name="sender_ip" + +209.85.161.46 +--xYzZY +Content-Disposition: form-data; name="spam_report" + +Spam detection software, running on the system "mx0032p1las1.sendgrid.net", has +identified this incoming email as possible spam. The original message +has been attached to this so you can view it (if it isn't spam) or label +similar future email. If you have any questions, see +@@CONTACT_ADDRESS@@ for details. + +Content preview: Forwarded message --------- From: Ashley Roach Date: Mon, + Oct 8, 2018 at 7:33 PM Subject: testing webhook parse To: this is the body + for webhook parse Forwarded message --------- From: Ashley Roach Date: Mon, + Oct 8, 2018 at 7:33 PM Subject: testing webhook parse To: [...] + +Content analysis details: (0.0 points, 5.0 required) + + pts rule name description +---- ---------------------- -------------------------------------------------- + 0.0 FREEMAIL_FROM Sender email is freemail (aroach[at]gmail.com) + 0.0 HTML_MESSAGE BODY: HTML included in message + 0.0 T_MIME_NO_TEXT No text body parts + 0.0 T_TO_NO_BRKTS_FREEMAIL T_TO_NO_BRKTS_FREEMAIL + + +--xYzZY +Content-Disposition: form-data; name="envelope" + +{"to":["test@parse.ashleyroach.com"],"from":"aroach@gmail.com"} +--xYzZY +Content-Disposition: form-data; name="attachments" + +0 +--xYzZY +Content-Disposition: form-data; name="subject" + +Fwd: testing webhook parse +--xYzZY +Content-Disposition: form-data; name="spam_score" + +0.022 +--xYzZY +Content-Disposition: form-data; name="charsets" + +{"to":"UTF-8","html":"UTF-8","subject":"UTF-8","from":"UTF-8","text":"UTF-8"} +--xYzZY +Content-Disposition: form-data; name="SPF" + +pass +--xYzZY-- \ No newline at end of file From 37555ba49ddfb1f6b771a6b3d41c76aa09b545fd Mon Sep 17 00:00:00 2001 From: Ashley Roach Date: Tue, 9 Oct 2018 13:37:44 -0600 Subject: [PATCH 4/9] Add Kubernetes manifests --- examples/inbound-parse-docker/README.md | 5 +++ .../k8s/inbound-parse.yml | 33 +++++++++++++++++++ 2 files changed, 38 insertions(+) create mode 100644 examples/inbound-parse-docker/k8s/inbound-parse.yml diff --git a/examples/inbound-parse-docker/README.md b/examples/inbound-parse-docker/README.md index 09083b57c..3a66b32bf 100644 --- a/examples/inbound-parse-docker/README.md +++ b/examples/inbound-parse-docker/README.md @@ -15,6 +15,11 @@ This is an example project for using a Docker container as a webhook receiver. At the moment, the `app.js` only prints data to the console. You can extend this project by adding more business logic to the `/parse_webhook` route. +# Deploy to Kubernetes + +* The container must be built and stored in a container registry. In this example, the Kubernetes (k8s) manifest uses the `imagePullPolicy: IfNotPresent` to pull from a local registry on the dev machine running docker-for-desktop. +* Once your k8s installation is running, from the root of the project execute: `kubectl apply -f k8s/inbound-parse.yml` + # License See [LICENSE](LICENSE) diff --git a/examples/inbound-parse-docker/k8s/inbound-parse.yml b/examples/inbound-parse-docker/k8s/inbound-parse.yml new file mode 100644 index 000000000..b33f2d94b --- /dev/null +++ b/examples/inbound-parse-docker/k8s/inbound-parse.yml @@ -0,0 +1,33 @@ +apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2 +kind: Deployment +metadata: + name: inbound-parse-deployment +spec: + selector: + matchLabels: + app: inbound-parse + replicas: 2 # tells deployment to run 2 pods matching the template + template: + metadata: + labels: + app: inbound-parse + spec: + containers: + - name: inbound-parse + image: inbound-parse-docker:latest + imagePullPolicy: IfNotPresent # enables pull from local system + ports: + - containerPort: 3000 +--- +apiVersion: v1 +kind: Service +metadata: + name: inbound-parse-svc +spec: + ports: + - protocol: TCP + port: 3000 + targetPort: 3000 + selector: + app: inbound-parse + type: LoadBalancer From ce695f3ba4a896d2c57845772e3391efb2efd8cc Mon Sep 17 00:00:00 2001 From: Ashley Roach Date: Tue, 9 Oct 2018 14:53:19 -0600 Subject: [PATCH 5/9] Send back a better status message --- examples/inbound-parse-docker/app.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/inbound-parse-docker/app.js b/examples/inbound-parse-docker/app.js index d3c12afb5..459a55f88 100644 --- a/examples/inbound-parse-docker/app.js +++ b/examples/inbound-parse-docker/app.js @@ -7,7 +7,7 @@ app.use(formidable()) -app.get('/', (req, res) => res.send('Hello World!')) +app.get('/', (req, res) => res.status(200).json({status: 'ok'})) app.post('/parse_webhook', (req, res) => { console.log('---------- START RECEIVED WEBHOOK DATA ----------') From 89fb8db3da6cca4b5a0dc27bd117b56c6e58a1b0 Mon Sep 17 00:00:00 2001 From: Ashley Roach Date: Tue, 9 Oct 2018 14:53:32 -0600 Subject: [PATCH 6/9] Add kubernetes documentation --- examples/inbound-parse-docker/README.md | 33 ++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/examples/inbound-parse-docker/README.md b/examples/inbound-parse-docker/README.md index 3a66b32bf..e5b5c33d7 100644 --- a/examples/inbound-parse-docker/README.md +++ b/examples/inbound-parse-docker/README.md @@ -4,21 +4,46 @@ This is an example project for using a Docker container as a webhook receiver. # Prerequisites -* Docker CE 18 +* [Docker CE 18](https://www.docker.com/get-started) * SendGrid Account with inbound parse enabled to send form data (default) * Ngrok or Internet accessible URL # Usage * `cd ` -* `docker-compose build && docker-compose up` +* Build the container: `docker-compose build` +* Run the container: `docker-compose up` +* Build & Run the container: `docker-compose build && docker-compose up` At the moment, the `app.js` only prints data to the console. You can extend this project by adding more business logic to the `/parse_webhook` route. +# A note on attachments + +This project uses the [express-formidable](https://github.com/utatti/express-formidable) middleware to process the form data sent by SendGrid's Inbound Parse Webhook. As a result, `express-formidable` automatically decodes and stores the images to the `/tmp` directory in the container. This is configurable by passing a configuration object to the middleware: + +```js +app.use(formidable({ + encoding: 'utf-8', + uploadDir: '/my/dir', + multiples: true, // req.files to be arrays of files +}); +``` + +Additionally, the [docker-compose](docker-compose.yml) has a volume mapping commented out if you'd prefer to store them in persistent storage outside of the container. + # Deploy to Kubernetes -* The container must be built and stored in a container registry. In this example, the Kubernetes (k8s) manifest uses the `imagePullPolicy: IfNotPresent` to pull from a local registry on the dev machine running docker-for-desktop. -* Once your k8s installation is running, from the root of the project execute: `kubectl apply -f k8s/inbound-parse.yml` +* In order for Kubernetes to use the container described in this project, the container must be built and stored in a container registry. You can choose to use a private registry in your cloud provider or a public registry (e.g., Docker Hub). You can also run a development environment of Kubernetes via [Docker for Mac or Windows](https://www.docker.com/get-started) +* In this project, the [Kubernetes (k8s) manifest](k8s/inbound-parse.yml) uses the `imagePullPolicy: IfNotPresent` to pull from a local registry on the dev machine running Kubernetes as part of Docker. If you were deploying to Google Cloud, for example, you should disable that option. +* `kubectl` is used to deploy. You should already have a working `kubectl context`. From the root of the project execute: `kubectl apply -f k8s/inbound-parse.yml` + +# Resources + +* [Kubernetes Developer Docs](https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/) +* [Install Kubernetes UI in Docker for Windows/Mac](https://www.ntweekly.com/2018/05/25/deploy-kubernetes-web-ui-dashboard-docker-windows/) +* [Deploy and Expose Apps in Kubernetes](https://www.ntweekly.com/2018/06/10/deploy-expose-applications-kubernetes-docker-windows/) +* [Download Docker](https://www.docker.com/get-started) + # License From a846cd8ec82c5f3deb8b4bd050dc27c496ddcff5 Mon Sep 17 00:00:00 2001 From: Ashley Roach Date: Tue, 9 Oct 2018 15:00:29 -0600 Subject: [PATCH 7/9] Clarify processing events in readme --- examples/inbound-parse-docker/README.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/examples/inbound-parse-docker/README.md b/examples/inbound-parse-docker/README.md index e5b5c33d7..7adeb2351 100644 --- a/examples/inbound-parse-docker/README.md +++ b/examples/inbound-parse-docker/README.md @@ -17,9 +17,13 @@ This is an example project for using a Docker container as a webhook receiver. At the moment, the `app.js` only prints data to the console. You can extend this project by adding more business logic to the `/parse_webhook` route. -# A note on attachments +# A note on processing events -This project uses the [express-formidable](https://github.com/utatti/express-formidable) middleware to process the form data sent by SendGrid's Inbound Parse Webhook. As a result, `express-formidable` automatically decodes and stores the images to the `/tmp` directory in the container. This is configurable by passing a configuration object to the middleware: +This project uses the [express-formidable](https://github.com/utatti/express-formidable) middleware to process the form data sent by SendGrid's Inbound Parse Webhook. + +The events are available in the `/parse_webhook` route in the `req.fields` object. The [app.js](app.js) contains logging statements for the elements that are available to you. It may be useful to review the [example-webhook-payload.txt](example-webhook-payload.txt) for what the form data looks like when extending this application. + +Attachments: `express-formidable` automatically decodes and stores the images to the `/tmp` directory in the container. This is configurable by passing a configuration object to the middleware: ```js app.use(formidable({ From 9a66772a924ce587dec06986fd51d578a1844b40 Mon Sep 17 00:00:00 2001 From: Ashley Roach Date: Tue, 9 Oct 2018 18:09:36 -0600 Subject: [PATCH 8/9] Update readme --- examples/inbound-parse-docker/README.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/examples/inbound-parse-docker/README.md b/examples/inbound-parse-docker/README.md index 7adeb2351..1bb5a921d 100644 --- a/examples/inbound-parse-docker/README.md +++ b/examples/inbound-parse-docker/README.md @@ -1,23 +1,32 @@ # Introduction -This is an example project for using a Docker container as a webhook receiver. +This is an example project for using a Docker container as a [SendGrid](https://sendgrid.com) [inbound parse email](https://sendgrid.com/docs/for-developers/parsing-email/setting-up-the-inbound-parse-webhook/) webhooks. # Prerequisites * [Docker CE 18](https://www.docker.com/get-started) +* (Optional for development) A local Kubernetes installation * SendGrid Account with inbound parse enabled to send form data (default) -* Ngrok or Internet accessible URL + * [Setup Inbound Parse Webhook](https://sendgrid.com/docs/for-developers/parsing-email/setting-up-the-inbound-parse-webhook/) +* [Ngrok](https://ngrok.com/) or Internet accessible URL -# Usage +# Usage Locally (aka for Development) * `cd ` * Build the container: `docker-compose build` * Run the container: `docker-compose up` -* Build & Run the container: `docker-compose build && docker-compose up` +* (Optional to save your self both commands above) Build & Run the container: `docker-compose build && docker-compose up` +* Run ngrok: `ngrok http 3000` +* Create an entry in the [Settings > Inbound Parse](https://app.sendgrid.com/settings/parse) with the ngrok URL. Use the `https` ngrok entry. +* Send an email to your inbound parse email address. + +NOTE: ngrok has a "replay" feature so you don't have to keep sending emails to yourself. You can access that when ngrok is running at [http://127.0.0.1:4040/inspect/http](http://127.0.0.1:4040/inspect/http) + +# Modifying the application At the moment, the `app.js` only prints data to the console. You can extend this project by adding more business logic to the `/parse_webhook` route. -# A note on processing events +## A note on processing events This project uses the [express-formidable](https://github.com/utatti/express-formidable) middleware to process the form data sent by SendGrid's Inbound Parse Webhook. @@ -35,7 +44,7 @@ app.use(formidable({ Additionally, the [docker-compose](docker-compose.yml) has a volume mapping commented out if you'd prefer to store them in persistent storage outside of the container. -# Deploy to Kubernetes +# Deployment to Kubernetes * In order for Kubernetes to use the container described in this project, the container must be built and stored in a container registry. You can choose to use a private registry in your cloud provider or a public registry (e.g., Docker Hub). You can also run a development environment of Kubernetes via [Docker for Mac or Windows](https://www.docker.com/get-started) * In this project, the [Kubernetes (k8s) manifest](k8s/inbound-parse.yml) uses the `imagePullPolicy: IfNotPresent` to pull from a local registry on the dev machine running Kubernetes as part of Docker. If you were deploying to Google Cloud, for example, you should disable that option. From 08e5b864d8fd80a4fb337134f76271e066ca030a Mon Sep 17 00:00:00 2001 From: Elmer Thomas Date: Tue, 9 Oct 2018 19:51:31 -0700 Subject: [PATCH 9/9] Just a few more installation steps --- examples/inbound-parse-docker/README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/examples/inbound-parse-docker/README.md b/examples/inbound-parse-docker/README.md index 1bb5a921d..218951b03 100644 --- a/examples/inbound-parse-docker/README.md +++ b/examples/inbound-parse-docker/README.md @@ -12,7 +12,11 @@ This is an example project for using a Docker container as a [SendGrid](https:// # Usage Locally (aka for Development) +* `mkdir ` * `cd ` +* `git clone https://github.com/sendgrid/sendgrid-nodejs.git` +* `cp -R sendgrid-nodejs/examples/inbound-parse-docker/* .` +* `rm -rf sendgrid-nodejs/` * Build the container: `docker-compose build` * Run the container: `docker-compose up` * (Optional to save your self both commands above) Build & Run the container: `docker-compose build && docker-compose up`