INSTALL

0) Clone a copy of the skeleton:

   git clone git@github.com:sean-/flask-skeleton.git


0a) I use tcsh(1) for day-to-day use and then sh(1) for scripting. Most
    bash(1) enthusiasts don't have enough perspective to have an opinion
    worth listening to. New tcsh users, enjoy the up-arrow history,
    tab-completion for ssh hosts, `time long_command`, etc., ...

    See https://github.com/sean-/cshrc for details, or just:

    $ curl -o ~/.cshrc https://github.com/sean-/cshrc/raw/master/.cshrc
    $ less ~/.cshrc     # _Always_ review changes to your shell config
    $ exec /bin/tcsh    # Start using tcsh(1)
    % chsh /bin/tcsh    # Change your default shell once you feel comfortable
    % mv .enter.tcsh.dist .enter.tcsh; mv .enter.local.tcsh.dist .enter.local.tcsh
    % ${EDITOR} .enter.tcsh .enter.local.tcsh
    % cd $PWD           # Run this after you finish installing PostgreSQL


0b) Download and install python 2.7.1 (NOTE: python 3.X is a no-go with
   Flask - don't get clever)

   % sudo port install python27


0c) Download and install libmemcached >= 0.4.0. I'm going to leave this as a
   an exercise to the reader as to how to best do this in your environment,
   but if you're on a mac with MacPorts:

   % sudo port install libmemcached


0d) Download and install PostgreSQL. Similar to the above, but since this is
    a database and not a random library, you're on your own for not being a
    nub:

    % sudo port install postgresql91-server
    % sudo mkdir -p /opt/local/var/db/postgresql91/defaultdb
    % sudo chown postgres:postgres /opt/local/var/db/postgresql91/defaultdb

    # Run this in a different Terminal window
    % sudo su postgres -c '/opt/local/lib/postgresql91/bin/initdb -D /opt/local/var/db/postgresql91/defaultdb'

    # Not required, but man pages are handy!
    % sudo port install postgresql91-doc


1) Rename from 'skeleton' to 'myapp':

   # At some point I'll change this step to be scripted (hell, I might even
   # actually test this step at some point)
   find . -type f -print0 | xargs -0 -n 1 perl -p -i -e '#skeleton#myapp#go'
   mv skeleton myapp
   # Search for and replace the remaining references to 'skeleton'
   egrep -ri skeleton *
   find . | grep -i skeleton


2) Start the database:

   # This should be set by your .enter.local.tcsh file.
   #setenv PGUSER_ROOT pgsql

   # MacPorts installed version of PostgreSQL 9.1, I run this in a different
   # terminal when developing so I can see what's going on.
   sudo su - ${PGUSER_ROOT} -c '/opt/local/lib/postgresql91/bin/postgres -D /opt/local/var/db/postgresql91/defaultdb'


3) Create a virualenv for the skeleton:

   # MacPorts installed version of Python 2.7
   /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/virtualenv --no-site-packages .


4) Activate the virtualenv:

   # If you installed the .cshrc file mentioned above and use tcsh(1), just do this:
   chmod 0600 .enter.tcsh
   cd $PWD

   # To manually activate the virtualenv, run the following:
   source bin/activate.csh


5) Pull in the required packages.

   # Tweak the build environment slightly for python-libmemcached and
   # M2Crypto.
   setenv CPPFLAGS '-I/opt/local/include -I/opt/local/include/openssl'

   # Break the installation down in to two steps: download and install (if
   # something fails, you don't want to start over again from scratch). ~/tmp
   # is created by .cshrc.
   mkdir ~/tmp/pip
   pip install -U --download-cache=~/tmp/pip -I -r requirements.txt --no-install
   pip install -U --download-cache=~/tmp/pip -I -r requirements.txt --no-download

   # Make sure everything is there (or newer):
   pip freeze | sort > requirements.txt
   git diff requirements.txt

5.1) Patch(es).

     # For now, Werkzeug doesn't support python-libmemcached, but it can by
     # applying the following patch to your install (versions 0.6.2 & 0.7.0):
     % patch -d lib/python2.7/site-packages/ -p0 < patches/werkzeug::contrib::cache_452e8402d056.patch-0.6.2


6) Setup debugging:

   echo 'DEBUG = True' >> local_settings.py
   echo 'TESTING = False' >> local_settings.py

   When you go in to production, be sure to remove these two lines!!!


7) Initialize the database (these steps should be moved in to a script for
   easy unit testing):

   # l2dba 101.

   # Environment variables that should be set by your .enter.local.tcsh:
   # PGUSER_RW, PGUSER and PGDATABASE.
   setenv PGUSER_DBA skeleton_root
   setenv PGUSER_ROOT pgsql

   # Create the base roles
   psql -d template1 -U ${PGUSER_ROOT} -1f sql/initialize/100_create_roles.sql

   # Create your user roles (use two, one for read-only and the other for
   # read-write activities)
   psql -d template1 -U ${PGUSER_ROOT} -c "CREATE ROLE ${PGUSER} CONNECTION LIMIT 2 LOGIN;"
   psql -d template1 -U ${PGUSER_ROOT} -c "CREATE ROLE ${PGUSER_RW} CONNECTION LIMIT 1 LOGIN;"

   # Add the ROLEs to their respective DBA ROLE/GROUPs
   psql -d template1 -U ${PGUSER_ROOT} -c "ALTER GROUP skeleton_dba ADD USER ${PGUSER};"
   psql -d template1 -U ${PGUSER_ROOT} -c "ALTER GROUP skeleton_root ADD USER ${PGUSER_RW};"

   # Create the rest of the DB objects
   psql -d template1 -U ${PGUSER_ROOT} -f sql/initialize/130_create_database.sql
   psql -U ${PGUSER_ROOT} -1f sql/initialize/140_alter_public_schema.sql

   # Load the pgcrypto and uuid functions in to skeleton
   cat /opt/local/share/postgresql91/extension/pgcrypto--1.0.sql | sed -e 's#MODULE_PATHNAME#pgcrypto#' > ~/tmp/pgcrypto.sql
   cat /opt/local/share/postgresql91/extension/uuid-ossp--1.0.sql | sed -e 's#MODULE_PATHNAME#uuid-ossp#' > ~/tmp/uuid-ossp.sql
   psql -U ${PGUSER_ROOT} -1f ~/tmp/pgcrypto.sql
   psql -U ${PGUSER_ROOT} -1f ~/tmp/uuid-ossp.sql

   # Load the schema
   psql -U ${PGUSER_DBA} -1f sql/initialize/200_schema.sql
   psql -U ${PGUSER_DBA} -1f sql/initialize/205_integrated_definitions.sql
   psql -U ${PGUSER_DBA} -1f sql/initialize/210_tables.sql
   psql -U ${PGUSER_DBA} -1f sql/initialize/280_views.sql

   # Load the functions. This loads the functions and changes their
   # permissions. Change DATABASE_PASSWORD_HASH to a random string of bytes
   # that's at least 32 bytes in length. Please use secure key handling
   # techniques and make sure that the DATABASE_PASSWORD_HASH key only exists
   # on your database servers and is not present (in any capacity) along side
   # with your webserver password keys.
   pwgen -acn 32 1 > sql/db_password.hash
   psql -At -U ${PGUSER_DBA} -c 'SELECT uuid_generate_v4();' > sql/db_email.uuid
   chmod 600 sql/db_password.hash sql/db_email.uuid
   sed -e "s#DATABASE_PASSWORD_HASH#`cat sql/db_password.hash`#" \
       -e "s#DATABASE_EMAIL_UUID#`cat sql/db_email.uuid`#" \
       sql/initialize/300_funcs.sql.in > sql/initialize/300_funcs.sql
   psql -U ${PGUSER_DBA} -1f sql/initialize/300_funcs.sql

   # CREATE TRIGGER commands are run outside of funcs.sql because there is no
   # "CREATE OR REPLACE TRIGGER" syntax.
   psql -U ${PGUSER_DBA} -1f sql/initialize/400_triggers.sql

   # ALTER FUNCTION ... OWNER TO ... commands must be run by the -U ${PGUSER_ROOT} user.
   psql -U ${PGUSER_ROOT} -1f sql/initialize/500_fixup_func_owner.sql

   # Add aaa and mod1 to the skeleton_www user's default search_path. This
   # only needs to be done when you create a new user.
   psql -U ${PGUSER_ROOT} -c "ALTER ROLE skeleton_www IN DATABASE ${PGDATABASE} SET search_path = aaa, mod1, public"

   # Populate some initial data
   psql -U ${PGUSER_DBA} -1f sql/initialize/800_initial_data.sql
   psql -U ${PGUSER_DBA} -1f sql/initialize/801_timezone_data.sql

   # Setup the GRANTS. Be sure to audit the database permissions after the
   # fact via the psql(1) commands: \dp, \ddp and \dn+
   psql -U ${PGUSER_DBA} -1f sql/maintenance/100_perms.sql

   # Neuter the DBA user and use your own personal DBA account.
   psql -U ${PGUSER_ROOT} -1f sql/initialize/900_cleanup_users.sql

   # Other misc PostgreSQL tips:
   #
   # 1) Look in to using the contrib'ed auto-explain module:
   #    http://www.postgresql.org/docs/current/static/auto-explain.html
   #
   # 2) A few suggested development tunables for postgresql.conf:
        # shared_preload_libraries = 'pgcrypto,uuid-ossp'
        # synchronous_commit = off
        # log_connections = on
        # log_disconnections = on
        # log_duration = on
        # log_statement = 'all'
        # timezone = 'UTC'
   #
   # 3) Learn how to use dblink (and get creative with its use in functions
   #    and views):
   #    http://www.postgresql.org/docs/current/static/dblink.html
   #
   # 4) Always use pgbouncer (http://wiki.postgresql.org/wiki/PgBouncer). I
   #    recommend using PostgreSQL on 127.0.0.1:6432 and then pgbouncer on
   #    *:5432. Be sure to check out the 'pgbouncer' database and run the
   #    'SHOW STATS' commands.
   #
   # 5) Small reading comprehension test: until you set the database or the
   #    connection's timezone to UTC, timezone to UTC, you will get
   #    '*check1'-like errors.


8) Run the app:

    python runserver.py

    # The first few times you run runserver.py, it will puke up errors and
    # have you add various bits to local_settings.py. It is critically
    # important that:
    #
    # a) PASSWORD_HASH: All webservers have the same PASSWORD_HASH. If you
    #    loose PASSWORD_HASH, users will not be able to log in. The contents
    #    of PASSWORD_HASH should never exist on your database servers.
    #
    # b) sql/db_password.hash: The contents of sql/db_password.hash should
    #    never be present on your webservers. Like PASSWORD_HASH, if you
    #    loose the contents of db_password.hash, users will not be able to
    #    log in without resetting their password (on the plus side, at least
    #    db_password.hash is stored in the contents of the pl functions -
    #    which are visible to any logged in user, regardless of the
    #    permissions set on the function's enclosing schema and the function
    #    itself).
    #
    # c) SECRET_KEY: SECRET_KEY must be synchronized across all webservers,
    #    too, but unlike PASSWORD_HASH, you can regenerate it at any time and
    #    there's no real loss (users will have to re-login).
    #
    # d) BROWSER_SECRET_KEY: Similar story to SECRET_KEY. All web servers
    #    must have the same BROWSER_SECRET_KEY. If you loose this key, every
    #    BrowserID cookie will be reset. Not the end of the world, but
    #    tedious as hell to recover from.
    #
    # e) SSL_CERT_FILENAME and SSL_PRIVATE_KEY_FILENAME: These are your SSL
    #    certs. Don't loose them. Don't use Werkzeug for handling SSL in
    #    production, but it's really convenient to test in an SSL environment
    #    from the beginning.

  a) Register:

     https://127.0.0.1:5000/register

     Then update shadow.aaa_email_confirmation_log:

     skeleton=> UPDATE shadow.aaa_email_confirmation_log SET confirmed = TRUE WHERE id = 1;

9) In production, you will need to combine all of your static assets
    together and probably have them served by nginx. Something like:

    mkdir static
    cd static
    find ../skeleton -name static
    # Make a symlink for everything in to the static dir. e.g.:
    # ln -s ../skeleton/modules/mod1/static mod1

    # Be sure to change your CANONICAL_NAME and CANONICAL_PORT settings once
    # you move in to production.


10) Feel free to re-run the sql/maintenance/100_perms.sql script as many times as you'd
    like. \dp, \ddp and \dn+ are your friends.


11) Debugging notes. With your environment activated:

    # Nearly every time I need to do any first time work with SQLAlchemy (or
    # any real database work), I fire up a terminal window and execute the
    # following commands:
    [flask-skeleton] % python
    >>> from skeleton import create_app
    >>> app = create_app()
    >>> ctx = app.test_request_context()
    >>> ctx.push()
    >>> from skeleton import db
    >>> ses = db.session
    # Which is why I dump all of that stuff in a shell.py file:
    % python -i shell.py

    # From here, I can do useful things and figure out how to map objects
    # together in the ORM without pulling my hair out.
    >>> from mod3.models import Tag, Page
    >>> p = Page.query.filter_by(id = 1).first()
    >>> p.__dict__
{'url': u'http://stackjet.com/mod3/page/submit', '_sa_instance_state': <sqlalchemy.orm.state.InstanceState object at 0x1023a9490>, 'id': 1}
    >>> dir(p)
    >>> t = Tag.query.filter_by(name = 'mytag').first()
    >>> t.pages.filter(Page.id==p.id).first() is None

    # I frequently dump the tedium of that kind of debugging in to debugging
    # scripts (pro tip: echo '/debug_*' >> .git/info/exclude)
    >>> from debug_mod3 import *