Skip to content
This repository has been archived by the owner on Feb 8, 2018. It is now read-only.

refactor accounts elsewhere #1369

Merged
merged 110 commits into from
Feb 26, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
6270daa
Participant pages work for Google users.
lyndsysimon Mar 16, 2013
783b56d
WIP - Abstracting into a more generic architecture
lyndsysimon Mar 18, 2013
5e878ca
Merge branch 'master' into elsewhere
chadwhitacre Oct 1, 2013
2b0a7c0
Post-merge get_accounts_elsewhere
chadwhitacre Oct 1, 2013
657cca2
Clean up wireup as well
chadwhitacre Oct 1, 2013
3393dd9
Move Google config out into wireup
chadwhitacre Oct 1, 2013
f2e93e0
Get old accounts elsewhere barely working TTW
chadwhitacre Oct 1, 2013
642ae09
Modernize a db API call
chadwhitacre Oct 1, 2013
06c4342
Rip out Google auth from elsewhere branch
chadwhitacre Oct 1, 2013
ffd7c13
Trim up some imports on elsewhere page
chadwhitacre Oct 1, 2013
6356e13
Clean up elsewhere page some more
chadwhitacre Oct 1, 2013
5ad02a9
Merge branch 'master' into elsewhere
chadwhitacre Oct 4, 2013
237d40c
Here's a failing test for an Aspen upgrade issue
chadwhitacre Oct 4, 2013
5190ee7
Fix an Aspen upgrade issue (UnicodeWithParams)
chadwhitacre Oct 4, 2013
efffad9
Merge branch 'master' into elsewhere
chadwhitacre Oct 4, 2013
6760eae
Standardize nomenclature on "platform"
chadwhitacre Oct 4, 2013
86c1f5a
Add a little whitespace
chadwhitacre Oct 4, 2013
808a7cc
Add .spt extension to a couple files
chadwhitacre Oct 4, 2013
ca9c7f4
Rough-in of an elsewhere refactor refactor; #1369
chadwhitacre Oct 4, 2013
1ec5522
Eager-load participant when querying elsewhere
chadwhitacre Oct 5, 2013
d1748ba
Fix the test for eager-loading participant
chadwhitacre Oct 5, 2013
b6f63cf
Note lack of type inheritance in Postgres
chadwhitacre Oct 5, 2013
d57bf6a
Tweak whitespace in branch.sql
chadwhitacre Oct 5, 2013
323c9d2
Upgrade Postgres.py to 2.1.0
chadwhitacre Oct 5, 2013
f3ddfaf
Upgrade to postgres 2.1.1
chadwhitacre Oct 5, 2013
319748d
Implement subclassing of AccountElsewhere
chadwhitacre Oct 5, 2013
7dbd70b
Get Twitter page barely working TTW
chadwhitacre Oct 5, 2013
a891e82
Reimplement nbackers
chadwhitacre Oct 5, 2013
4195442
Prune crufty nbackers computed property stub
chadwhitacre Oct 5, 2013
ed4dee6
Remove cruft from configure-aspen.py
chadwhitacre Oct 5, 2013
a7589c3
Rip out _resolve
chadwhitacre Oct 5, 2013
1a74332
Start in on oauth for Twitter
chadwhitacre Oct 5, 2013
6c8be36
Rename website.elsewhere to .platforms
chadwhitacre Oct 6, 2013
efbc39c
Start fixing up tests for Participant
chadwhitacre Oct 6, 2013
c45122e
test_participant passing
chadwhitacre Oct 6, 2013
a20bdd9
Fix test_utils
chadwhitacre Oct 6, 2013
c83c5af
Fix test_elsewhere_twitter
chadwhitacre Oct 6, 2013
f567c39
Bring fetch_* into line with get_account
chadwhitacre Oct 6, 2013
bdec63b
Fix naming regressions in elsewhere
chadwhitacre Oct 6, 2013
9b1f5c1
Fix test_anonymous_json
chadwhitacre Oct 6, 2013
0785af7
Toe the waters on test_associate
chadwhitacre Oct 6, 2013
65be9f2
Fix test_twitter_proxy.
sim6 Oct 19, 2013
f8881e1
Fix login links.
sim6 Oct 20, 2013
99aa1ef
Fix about/me.html login links.
sim6 Oct 20, 2013
0f2733d
Change from "Bad request" to "Forbidden" of the expected result of te…
sim6 Oct 20, 2013
acec955
Add PlatformOAuth1 class a Platform class with support for OAuth1.
sim6 Oct 20, 2013
8ac08ff
Twitter login with OAuth1.
sim6 Oct 20, 2013
b494400
Add PlatformOAuth2 class a Platform class with support for OAuth2.
sim6 Oct 20, 2013
3072a2c
Github login with OAuth2.
sim6 Oct 20, 2013
1fad280
Adapt OAuth1 to Bitbucket.
sim6 Oct 21, 2013
6ded2be
Bitbucket login with OAuth1.
sim6 Oct 21, 2013
9f06418
Fix links to connect accounts.
sim6 Oct 21, 2013
51597fd
Fix action to connect accounts.
sim6 Oct 21, 2013
ffb6043
Fix links to connect accounts, again :(
sim6 Oct 22, 2013
dd22ea8
Fix TestPages.test_homepage_with_anonymous_giver.
sim6 Oct 22, 2013
9b529bf
Fix return values of OAuth posts of mocks in test_associate.py.
sim6 Oct 24, 2013
bd0042e
Fix TestAccountElsewhere.test_bitbucket_oauth_url_percent_encodes_then.
sim6 Oct 24, 2013
dbc1e4c
Fix TestAccountElsewhere.test_github_oauth_url_not_susceptible_to_inj…
sim6 Oct 24, 2013
95562a0
Fix TestAccountElsewhere.test_opt_in_can_change_username.
sim6 Oct 24, 2013
9f0dda2
Fix TestAccountElsewhere.test_twitter_oauth_url_percent_encodes_then.
sim6 Oct 24, 2013
fcfe65e
Merge branch 'master' into 'elsewhere'
Changaco Jan 25, 2014
dab9933
WIP
Changaco Feb 7, 2014
0fecb5e
clean up imports
Changaco Feb 9, 2014
e0e2418
Merge branch 'master' into elsewhere
Changaco Feb 9, 2014
23433d1
Switch travis to postgres 9.3.
zbynekwinkler Feb 9, 2014
302bd0f
remove files mistakenly added in merge
Changaco Feb 10, 2014
80b7ff2
re-add mistakenly removed import
Changaco Feb 10, 2014
771d3a8
fix test
Changaco Feb 10, 2014
dbef80c
don't deal with duplicates
Changaco Feb 11, 2014
11337ab
don't propagate email to participant
Changaco Feb 11, 2014
58023fc
move avatar_url propagation from DB trigger to python code
Changaco Feb 11, 2014
27f6f10
Merge branch 'master' into elsewhere
Changaco Feb 11, 2014
5282ece
add tests and fix spotted bugs
Changaco Feb 12, 2014
82ec96f
fix problems detected by pyflakes
Changaco Feb 13, 2014
ae6bfbd
fix bug found by whit537 while testing bitbucket auth
Changaco Feb 13, 2014
c045cfa
fix typo
Changaco Feb 13, 2014
2ec047c
fix bitbucket auth
Changaco Feb 13, 2014
447a97e
small optimization
Changaco Feb 13, 2014
e3697e4
don't set oauth2 scope by default
Changaco Feb 13, 2014
50316da
don't show an opt-in button when platform isn't in signin_platforms
Changaco Feb 13, 2014
92713b3
use the default image when avatar_url is empty
Changaco Feb 13, 2014
6e11fe0
reorganize branch.sql
Changaco Feb 14, 2014
0cf40ea
Merge branch 'master' into elsewhere
chadwhitacre Feb 15, 2014
00046c9
set a default oauth scope for venmo
Changaco Feb 15, 2014
e532ea2
fix bad test condition that resulted in user_id being None
Changaco Feb 15, 2014
5d11e95
make sure user_id is neither None nor an empty string
Changaco Feb 15, 2014
675aba8
add some docstrings
Changaco Feb 16, 2014
80bde4c
small HTML fix (<p> can't contain <form>)
Changaco Feb 17, 2014
a9b4d0b
create UserInfo class
Changaco Feb 18, 2014
12ddf91
extract type (single or team) of elsewhere accounts
Changaco Feb 18, 2014
665567e
add var name alias to fix included template sign-in-using-to-give.html
Changaco Feb 19, 2014
64ed97f
replace parse_user_info() by api_parser()
Changaco Feb 19, 2014
d88302e
minor changes (move some code, add some comments)
Changaco Feb 19, 2014
211e780
add support for API response pagination
Changaco Feb 20, 2014
c11b420
small CSS fix
Changaco Feb 21, 2014
43bf21a
more informative exception
Changaco Feb 22, 2014
b4397af
create get_team_members() method
Changaco Feb 22, 2014
cf5feff
bring back display of elsewhere team members
Changaco Feb 22, 2014
5b229e9
Merge branch 'master' into elsewhere
Changaco Feb 22, 2014
f64c1db
fix the definition of the declines_gifts variable
Changaco Feb 23, 2014
9ff12a9
Squish regression on "Giving" page; #1369
chadwhitacre Feb 25, 2014
9739c23
Merge branch 'master' into elsewhere
Changaco Feb 26, 2014
627df0d
Start making buttons look like buttons again
chadwhitacre Feb 26, 2014
46d520a
Tweak whitespace; #1369
chadwhitacre Feb 26, 2014
f23bab6
Obey pyflakes
chadwhitacre Feb 26, 2014
ff83659
Add comment about DEBUG=1 flag
chadwhitacre Feb 26, 2014
6c0db7b
fix venmo's default scope and user info extraction
Changaco Feb 26, 2014
e80ff73
Fix bug in extraction of Venmo user_info; #1369
chadwhitacre Feb 26, 2014
54a25af
Not sure how this ended up back in; #1369
chadwhitacre Feb 26, 2014
5508641
fix bug in extraction of Venmo user_info, again
Changaco Feb 26, 2014
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
language: python
addons:
postgresql: 9.3
before_install:
- node --version
install: make env node_modules
Expand Down
142 changes: 142 additions & 0 deletions branch.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
-------------------------------------------------------------------------------
-- https://github.com/gittip/www.gittip.com/pull/1369

BEGIN;


-- Add new columns

-- Note: using "user_name" instead of "username" avoids having the same
-- column name in the participants and elsewhere tables.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suspect you're attempting to avoid confusion by not simply using username in the elsewhere table, yes? Is there a stronger reason? I'd be inclined to call them both username and trust the table name to differentiate them. Unless they actually conflict somehow?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After spending more time with this PR I actually don't care about this. We've got user_id and display_name and you're right that it can be nice to distinguish from participants.username. TBH when I'm working with the elsewhere table in psql I often forget that I want elsewhere.participants instead of elsewhere.username. Actually having an elsewhere.username would no doubt trip me up. :-)

ALTER TABLE elsewhere ADD COLUMN user_name text;
ALTER TABLE elsewhere ADD COLUMN display_name text;
ALTER TABLE elsewhere ADD COLUMN email text;
ALTER TABLE elsewhere ADD COLUMN avatar_url text;
ALTER TABLE participants ADD COLUMN avatar_url text;
ALTER TABLE elsewhere ADD COLUMN is_team boolean NOT NULL DEFAULT FALSE;



-- Extract info

-- Extract user_name from user_info
UPDATE elsewhere SET user_name = user_id WHERE platform = 'bitbucket';
UPDATE elsewhere SET user_name = user_info->'display_name' WHERE platform = 'bountysource';
UPDATE elsewhere SET user_name = user_info->'login' WHERE platform = 'github';
UPDATE elsewhere SET user_name = user_info->'username' WHERE platform = 'openstreetmap';
UPDATE elsewhere SET user_name = user_info->'screen_name' WHERE platform = 'twitter';
UPDATE elsewhere SET user_name = user_info->'username' WHERE platform = 'venmo';

-- Extract display_name from user_info
UPDATE elsewhere SET display_name = user_info->'display_name' WHERE platform = 'bitbucket';
UPDATE elsewhere SET display_name = user_info->'name' WHERE platform = 'github';
UPDATE elsewhere SET display_name = user_info->'username' WHERE platform = 'openstreetmap';
UPDATE elsewhere SET display_name = user_info->'name' WHERE platform = 'twitter';
UPDATE elsewhere SET display_name = user_info->'display_name' WHERE platform = 'venmo';
UPDATE elsewhere SET display_name = NULL WHERE display_name = 'None';

-- Extract available email addresses
UPDATE elsewhere SET email = user_info->'email' WHERE user_info->'email' LIKE '%@%';

-- Extract available avatar URLs
UPDATE elsewhere SET avatar_url = concat('https://www.gravatar.com/avatar/',
user_info->'gravatar_id')
WHERE platform = 'github'
AND user_info->'gravatar_id' != ''
AND user_info->'gravatar_id' != 'None';
UPDATE elsewhere SET avatar_url = concat('https://www.gravatar.com/avatar/',
md5(lower(trim(email))))
WHERE email IS NOT NULL AND avatar_url IS NULL;
UPDATE elsewhere SET avatar_url = user_info->'avatar' WHERE platform = 'bitbucket';
UPDATE elsewhere SET avatar_url = user_info->'avatar_url'
WHERE platform = 'bitbucket' AND avatar_url IS NULL;
UPDATE elsewhere SET avatar_url = substring(user_info->'links', $$u'avatar': {u'href': u'([^']+)$$)
WHERE platform = 'bitbucket' AND avatar_url IS NULL;
UPDATE elsewhere SET avatar_url = user_info->'image_url' WHERE platform = 'bountysource';
UPDATE elsewhere SET avatar_url = user_info->'avatar_url' WHERE platform = 'github' AND avatar_url IS NULL;
UPDATE elsewhere SET avatar_url = user_info->'img_src' WHERE platform = 'openstreetmap';
UPDATE elsewhere SET avatar_url = replace(user_info->'profile_image_url_https', '_normal.', '.')
WHERE platform = 'twitter';
UPDATE elsewhere SET avatar_url = user_info->'profile_picture_url' WHERE platform = 'venmo';
UPDATE elsewhere SET avatar_url = NULL WHERE avatar_url = 'None';
-- Propagate avatar_url to participants
UPDATE participants p
SET avatar_url = (
SELECT avatar_url
FROM elsewhere
WHERE participant = p.username
ORDER BY platform = 'github' DESC,
avatar_url LIKE '%gravatar.com%' DESC
LIMIT 1
);

-- Extract is_team from user_info
UPDATE elsewhere SET is_team = true WHERE platform = 'bitbucket' AND user_info->'is_team' = 'True';
UPDATE elsewhere SET is_team = true WHERE platform = 'github' AND lower(user_info->'type') = 'organization';



-- Drop old columns and add new ones

-- Update user_name constraints
ALTER TABLE elsewhere ALTER COLUMN user_name SET NOT NULL,
ALTER COLUMN user_name DROP DEFAULT;

-- Replace user_info by a new column of type json (instead of hstore)
ALTER TABLE elsewhere DROP COLUMN user_info,
ADD COLUMN extra_info json;
DROP EXTENSION hstore;

-- Simplify homepage_top_* tables
ALTER TABLE homepage_top_givers DROP COLUMN gravatar_id,
DROP COLUMN twitter_pic,
ADD COLUMN avatar_url text;
ALTER TABLE homepage_top_receivers DROP COLUMN claimed_time,
DROP COLUMN gravatar_id,
DROP COLUMN twitter_pic,
ADD COLUMN avatar_url text;

-- The following lets us cast queries to elsewhere_with_participant to get the
-- participant data dereferenced and returned in a composite type along with
-- the elsewhere data. Then we can register orm.Models in the application for
-- both participant and elsewhere_with_participant, and when we cast queries
-- elsewhere.*::elsewhere_with_participant, we'll get a hydrated Participant
-- object at .participant. Woo-hoo!

CREATE TYPE elsewhere_with_participant AS
( id integer
, platform text
, user_id text
, user_name text
, display_name text
, email text
, avatar_url text
, extra_info json
, is_locked boolean
, is_team boolean
, participant participants
); -- If Postgres had type inheritance this would be even awesomer.

CREATE OR REPLACE FUNCTION load_participant_for_elsewhere (elsewhere)
RETURNS elsewhere_with_participant
AS $$
SELECT $1.id
, $1.platform
, $1.user_id
, $1.user_name
, $1.display_name
, $1.email
, $1.avatar_url
, $1.extra_info
, $1.is_locked
, $1.is_team
, participants.*::participants
FROM participants
WHERE participants.username = $1.participant
;
$$ LANGUAGE SQL;

CREATE CAST (elsewhere AS elsewhere_with_participant)
WITH FUNCTION load_participant_for_elsewhere(elsewhere);

END;
9 changes: 0 additions & 9 deletions configure-aspen.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
from __future__ import division

from importlib import import_module
import os
import sys
import threading
Expand All @@ -12,7 +11,6 @@
from gittip import canonize, configure_payments
from gittip.security import authentication, csrf, x_frame_options
from gittip.utils import cache_static, timer
from gittip.elsewhere import platform_classes


from aspen import log_dammit
Expand Down Expand Up @@ -46,12 +44,6 @@
gittip.wireup.envvars(website)
tell_sentry = gittip.wireup.make_sentry_teller(website)

# this serves two purposes:
# 1) ensure all platform classes are created (and thus added to platform_classes)
# 2) keep the platform modules around to be added to the context below
platform_modules = {platform: import_module("gittip.elsewhere.%s" % platform)
for platform in platform_classes}

# The homepage wants expensive queries. Let's periodically select into an
# intermediate table.

Expand Down Expand Up @@ -124,7 +116,6 @@ def log_busy_threads():

def add_stuff_to_context(request):
request.context['username'] = None
request.context.update(platform_modules)

def scab_body_onto_response(response):

Expand Down
13 changes: 8 additions & 5 deletions defaults.env
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ STRIPE_PUBLISHABLE_API_KEY=1

BALANCED_API_SECRET=90bb3648ca0a11e1a977026ba7e239a9

DEBUG=1 # Used by oauthlib to bypass security checks. We need it because when
# running locally the OAuth callbacks are http:, not https:. Of course
# DEBUG shouldn't be set in production.

GITHUB_CLIENT_ID=3785a9ac30df99feeef5
GITHUB_CLIENT_SECRET=e69825fafa163a0b0b6d2424c107a49333d46985
GITHUB_CALLBACK=http://127.0.0.1:8537/on/github/associate
Expand All @@ -26,23 +30,22 @@ BITBUCKET_CALLBACK=http://127.0.0.1:8537/on/bitbucket/associate

TWITTER_CONSUMER_KEY=QBB9vEhxO4DFiieRF68zTA
TWITTER_CONSUMER_SECRET=mUymh1hVMiQdMQbduQFYRi79EYYVeOZGrhj27H59H78
TWITTER_ACCESS_TOKEN=34175404-G6W8Hh19GWuUhIMEXK0LyZsy7N9aCMcy1bYJ9rI
TWITTER_ACCESS_TOKEN_SECRET=K6wxV1OCsihZAkEPkWtoLYDiRJnWajBBWn4UgliTRQ
TWITTER_CALLBACK=http://127.0.0.1:8537/on/twitter/associate

BOUNTYSOURCE_API_SECRET=e2BbqjNY60kC7V-Uq1dv2oHgGavbWm9pUJmiRHCApFZHDiY9aZyAspInhZaZ94x9
BOUNTYSOURCE_API_HOST=https://staging-qa.bountysource.com
BOUNTYSOURCE_WWW_HOST=https://staging.bountysource.com
BOUNTYSOURCE_CALLBACK=http://127.0.0.1:8537/on/bountysource/associate
BOUNTYSOURCE_API_HOST=https://staging-api.bountysource.com
BOUNTYSOURCE_WWW_HOST=https://staging.bountysource.com

VENMO_CLIENT_ID=1534
VENMO_CLIENT_SECRET=55ckgsguYC3cj7xWW5c95PHvUzrwgZMA
VENMO_CALLBACK=http://127.0.0.1:8537/on/venmo/associate

OPENSTREETMAP_API=http://master.apis.dev.openstreetmap.org
OPENSTREETMAP_CONSUMER_KEY=J2SS5GM0A7tM1CIBjAHXUTMeCEkRBMYsTJzGONxe
OPENSTREETMAP_CONSUMER_SECRET=hgvZkbtWVOEoaJV5AzQPcBI9m8f7BylkpT0cP7wS
OPENSTREETMAP_CALLBACK=http://127.0.0.1:8537/on/openstreetmap/associate
OPENSTREETMAP_API_URL=http://master.apis.dev.openstreetmap.org/api/0.6
OPENSTREETMAP_AUTH_URL=http://master.apis.dev.openstreetmap.org

NANSWERS_THRESHOLD=2

Expand Down
1 change: 0 additions & 1 deletion gittip/billing/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
from __future__ import unicode_literals
from urllib import quote

import gittip
import balanced
import stripe
from aspen.utils import typecheck
Expand Down
4 changes: 2 additions & 2 deletions gittip/billing/steady_state.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,12 +42,12 @@ def converge(payouts, epsilon = 1e-10, max_rounds = 100):
Converges to the steady state.
"""
if not issparse(payouts):
raise ArgumentError("Please provide a sparse matrix")
raise ValueError("Please provide a sparse matrix")

(n_rows, n_cols) = payouts.shape

if n_rows != n_cols:
raise ArgumentError("The payout matrix must be square")
raise ValueError("The payout matrix must be square")

payouts_d = lil_matrix((n_rows, n_cols))
payouts_d.setdiag(payouts.diagonal())
Expand Down
Loading