Skip to content

Commit

Permalink
README fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
alexleonescalera committed Apr 19, 2021
1 parent db1feaa commit 53161c5
Showing 1 changed file with 95 additions and 77 deletions.
172 changes: 95 additions & 77 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,35 +8,38 @@
Serverless Ephemeral (or Serephem) is a [Serverless Framework plugin](https://serverless.com/framework/docs/providers/aws/guide/plugins/) that helps bundle any stateless library into a Lambda deployment artifact.

## Pre-requirements
* Node >= 6.9
* Serverless Framework >= 1.12.0
* Docker (with docker compose)

- Node >= 6.9
- Serverless Framework >= 1.12.0
- Docker (with docker compose)

## Examples
* [TensorFlow Lambda (CPU only, Python 2.7)](examples/tensorflow-lambda): Pulls in or builds (via Docker) a TensorFlow package. For reference on how to manually build a TensorFlow package for Lambdas, see [docs/build-tensorflow-package.md](docs/build-tensorflow-package.md).

* [Image Processor](examples/image-processor): Builds and adds a Python [Pillow](https://python-pillow.org/) package in order to resize an image uploaded to an S3 bucket.
- [TensorFlow Lambda (CPU only, Python 2.7)](examples/tensorflow-lambda): Pulls in or builds (via Docker) a TensorFlow package. For reference on how to manually build a TensorFlow package for Lambdas, see [docs/build-tensorflow-package.md](docs/build-tensorflow-package.md).

- [Image Processor](examples/image-processor): Builds and adds a Python [Pillow](https://python-pillow.org/) package in order to resize an image uploaded to an S3 bucket.

## Add the plugin

1. Install it

```bash
npm i --save-dev serverless-ephemeral
```
```bash
npm i --save-dev serverless-ephemeral
```

1. Add it to your `serverless.yml` file and exclude the `.ephemeral` directory

```yaml
plugins:
- serverless-ephemeral
```yaml
plugins:
- serverless-ephemeral

package:
exclude:
- package.json
- package-lock.json
- node_modules/**
- .ephemeral/**
```
package:
exclude:
- package.json
- package-lock.json
- node_modules/**
- .ephemeral/**
```
1. Add the `.ephemeral` directory to `.gitignore`

Expand All @@ -46,13 +49,14 @@ Serverless Ephemeral (or Serephem) is a [Serverless Framework plugin](https://se
.ephemeral
```


## Configuration

The configuration for the Ephemeral plugin is set inside the `custom` section of the `serverless.yml` file. In it, you can define the list of stateless libraries you wish to pull into the final Lambda artifact.

There are two types of configuration:
* [Build a library during runtime](#build-a-library)
* [Download a library](#download-a-library)

- [Build a library during runtime](#build-a-library)
- [Download a library](#download-a-library)

Both can be enhanced with [global configuration options](#global-options).

Expand Down Expand Up @@ -86,75 +90,77 @@ You can create your own packager via Docker. To do so:

1. Create a directory where you will store all your Docker files:

```bash
mkdir my-packager
cd my-packager
```
```bash
mkdir my-packager
cd my-packager
```

1. Create a `docker-compose.yml` file. For example:

```yaml
version: '3'
services:
packager:
build: .
```
```yaml
version: "3"
services:
packager:
build: .
```

Keep note of the name of your packager service, in this case `packager`.
Keep note of the name of your packager service, in this case `packager`.

1. Create a `Dockerfile` and any other support files. For example:

**`Dockerfile`**
```yaml
FROM amazonlinux
**`Dockerfile`**

```yaml
FROM amazonlinux
COPY scripts/build.sh scripts/build.sh
RUN yum -y install zip && \
chmod +x scripts/build.sh
COPY scripts/build.sh scripts/build.sh
RUN yum -y install zip && \
chmod +x scripts/build.sh
CMD [ "scripts/build.sh" ]
```
CMD [ "scripts/build.sh" ]
```

**`scripts/build.sh`**
```bash
# create zip destination directory
mkdir -p /tmp/lambda-libraries
**`scripts/build.sh`**

# download library files
mkdir /tmp/files
cd /tmp/files
curl http://example.com/file-1.py --output file-1.py
curl http://example.com/file-2.py --output file-2.py
zip -9rq /tmp/lambda-libraries/library-a.zip *
```
```bash
# create zip destination directory
mkdir -p /tmp/lambda-libraries
**IMPORTANT**: the container must generate a zip file containing the stateless library files. Thus:
# download library files
mkdir /tmp/files
cd /tmp/files
curl http://example.com/file-1.py --output file-1.py
curl http://example.com/file-2.py --output file-2.py
zip -9rq /tmp/lambda-libraries/library-a.zip *
```

* Your container must zip the stateless library files.
**IMPORTANT**: the container must generate a zip file containing the stateless library files. Thus:

* You must create a directory where the final zip(s) will be stored. This directory will be mounted to the Serephem's libraries directory, so add only the necessary zip files.
- Your container must zip the stateless library files.

* It is recommended that your Docker container extends from `amazonlinux` image to maximize compatibility with the Lambda environment.
- You must create a directory where the final zip(s) will be stored. This directory will be mounted to the Serephem's libraries directory, so add only the necessary zip files.

- It is recommended that your Docker container extends from `amazonlinux` image to maximize compatibility with the Lambda environment.

1. Add this configuration to your `serverless.yml`:

```yaml
custom:
ephemeral:
libraries:
- packager:
compose: my-packager/docker-compose.yml
service: packager
output: /tmp/lambda-libraries/library-a.zip
```
```yaml
custom:
ephemeral:
libraries:
- packager:
compose: my-packager/docker-compose.yml
service: packager
output: /tmp/lambda-libraries/library-a.zip
```

Notice how each of the values correspond to a setting previously created:
Notice how each of the values correspond to a setting previously created:

* `compose`: points to your Docker compose file, inside the directory you created
- `compose`: points to your Docker compose file, inside the directory you created

* `service`: the name of the service inside the `docker-compose.yml` file
- `service`: the name of the service inside the `docker-compose.yml` file

* `output`: the output path for the zip file in the Docker container
- `output`: the output path for the zip file in the Docker container

### Download a library

Expand Down Expand Up @@ -187,36 +193,48 @@ custom:

- **directory** is optional. When ommitted, the package contents will be unzipped at service root level. If entered, a new folder will be created at that level using the specified name and everything will be unzipped there. The folder can only be named using alphanumeric characters and the symbols `. _ -`

- **nocache** is optional. When ommitted or set to *false*, it will use the locally cached copy of the library. Otherwise, if set to *true*, it will re-fetch (download or build) the library every time the service is packaged.
- **nocache** is optional. When ommitted or set to _false_, it will use the locally cached copy of the library. Otherwise, if set to _true_, it will re-fetch (download or build) the library every time the service is packaged.

> Note: the **forceDownload** option has been deprecated in favor of **nocache**
> Note: the **forceDownload** option has been deprecated in favor of **nocache**

## Deploy
5. Deploy your service normally with the `serverless deploy` (or `sls deploy`) command. If you use the `-v` option, Ephemeral will show more information about the process.

```bash
sls deploy -v
```
Deploy your service normally with the `serverless deploy` (or `sls deploy`) command. If you use the `-v` option, Ephemeral will show more information about the process.

```bash
sls deploy -v
```

> If the Serverless deployment is timing out, use the `AWS_CLIENT_TIMEOUT` environment variable: https://github.com/serverless/serverless/issues/490#issuecomment-204976134
> If the Serverless deployment is timing out, use the `AWS_CLIENT_TIMEOUT` environment variable: <https://github.com/serverless/serverless/issues/490#issuecomment-204976134>

### The .ephemeral directory

During the deployment process, the `.ephemeral` directory will be created. The purpose of this directory is:
* Saving the libraries' zip files inside the `.ephemeral/lib` folder
* Bundling the libraries and the Serverless Lambda function file(s) inside the `.ephemeral/pkg` folder

- Saving the libraries' zip files inside the `.ephemeral/lib` folder
- Bundling the libraries and the Serverless Lambda function file(s) inside the `.ephemeral/pkg` folder

---

## Development

This plugin is created with Node and uses the Serverless Framework hooks to execute the necessary actions.

### Getting started

1. Clone this repo
2. Install dependencies via `npm i`

### Running Lint

The plugin code uses the AirBnB ESLint rule set with some enhancements (see `.eslintrc` file). To run the linter:

```bash
npm run lint
```

### Tests

The unit tests are coded with [Ava](https://github.com/avajs/ava) and [SinonJS](http://sinonjs.org/docs/). They can be found inside the `spec` folder.

To run the tests:
Expand Down

0 comments on commit 53161c5

Please sign in to comment.