Skip to content

Elastic Discovery - Help applications that don't quite work in the cloud better handle autoscaling and other cloud events.

License

Notifications You must be signed in to change notification settings

gallagherrchris/Elasticd

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

78 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ElasticD for direct communication between frotends and backends

##Table Of Contents

Objective

The goal is to have frontends communicate directly with backends by using IP addresses instead of DNS in cloud environments. This will remove the unnecessary components in-between the two and associated overhead.

Why ElasticD?

  • AWS is very dynamic in nature. A lot of their services start with ‘E’ for elastic.
  • IP addresses change all the time, allowing more flexibility.
  • The number of server performing a task may increase or decrease at any time.
  • So far, standard industry technologies don’t operate optimally in this dynamic environment.
  • Many cache DNS lookups for performance. This is great in a static environment but a burden in AWS.
  • A few technologies used at FINRA required significant additional effort and workarounds included:
  • Varnish
  • Apache
  • Nginx (special configuration required)
  • It was difficult to automatically configure frontends and know about the backends without using DNS.
  • This required additional components between the front ends and back ends.
  • More components meant more overhead in maintenance, testing, and development.
  • More components meant more failure points.

Solution

Create a light weight application that informs the frontends of the backends, allowing them to communicate directly with IP addresses.

Features

  • Pluggable
  • It's able to extend components to suit your needs.
  • Frontend agnostic
  • Additional implementations are available for different frontend technologies (Varnish, Nginx, etc).
  • Backend agnostic
  • ElasticD only needs a way to find the backend's IP address.
  • Cloud agnostic
  • At FINRA, we use AWS but the resource locator can be extended for other cloud offerings.

Alt text

##Setup This section will walk you through how to start up the ElasticD services You will need to install the following

  • Pip
  • Flask
  • Jinja2
  • boto
  • apscheduler
  • pyyaml

After that you can run python bin/run.py or python test/test_driver.py

About

Elastic Discovery - Help applications that don't quite work in the cloud better handle autoscaling and other cloud events.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 97.5%
  • Perl 2.5%