Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Direct traffic at CRO sandbox versions of pages #7652

Closed
3 of 5 tasks
hoosteeno opened this issue Sep 4, 2019 · 20 comments
Closed
3 of 5 tasks

Direct traffic at CRO sandbox versions of pages #7652

hoosteeno opened this issue Sep 4, 2019 · 20 comments
Assignees
Labels
P2 Second level priority - Should have

Comments

@hoosteeno
Copy link
Contributor

hoosteeno commented Sep 4, 2019

Description

After we have copies of some content in the CRO sandbox, we should build any tooling that we will require to direct some traffic to those copies. This may mean using TrafficCop, perhaps with some standardized settings, or it may mean something else.


💛 Success Criteria 💛

  • A percentage of visitors to pages we've duplicated in the CRO sandbox can be redirected to the CRO sandbox duplicates.
  • Starting a new CRO experiment using the sandbox is a simple configuration change away.
  • URL parameters will be preserved between redirects.

Tasks

  • This is an example of a task. Use a checklist!
  • This is an example of a task.
@alexgibson
Copy link
Member

@alexgibson
Copy link
Member

Depends on #7508

@amychurchwell
Copy link
Contributor

Sandbox pages with experimental tool behind /ex/ example: /ex/firefox/accounts

  • URL parameters preserved on redirection.
  • Adjust traffic percentage without a production push.
  • Adjusting locale without a production push may need be a need in the future.
  • Should be able to target any URL.

Server-side redirect solution may be preferable? Maybe convo with @jgmize ?

@jgmize
Copy link
Contributor

jgmize commented Sep 6, 2019

Please see mozmeao/infra#1050 for some links and an example of how Cloudflare workers might help with this.

@alexgibson
Copy link
Member

Another requirement for this solution should be that it does not interfere with our functional test runs.

Either it should only execute on www.mozilla.org / www.allizom.org domains (where it already won't interfere), or alternatively we could build a mechanism whereby our test runs can opt-out on incurring redirects.

@ejregithub ejregithub added this to the S5/Q3: 20190923-20191011 milestone Sep 17, 2019
@hoosteeno hoosteeno added the P2 Second level priority - Should have label Sep 18, 2019
@pmac
Copy link
Member

pmac commented Sep 19, 2019

Initial architecture doc for reference: https://docs.google.com/document/d/1RXzBVUk3LSbyBeJOesPrKcekvA5bOUBiUIiQy7frHgQ/edit#heading=h.ui1wp7dez36o

FYI for anyone without access to the above doc, "CRO" stands for "Conversion Rate Optimization".

The more you know 🌈

@alexgibson
Copy link
Member

alexgibson commented Sep 19, 2019

FYI for anyone without access to the above doc

Anyone at Mozilla should have access to this doc ^

@pmac
Copy link
Member

pmac commented Sep 19, 2019

This page is fully public, and that doc is not. That's why I was trying to keep everyone up to speed. I had to login to look at the doc to learn what "CRO" meant, so I was just trying to help others.

@alexgibson
Copy link
Member

@pmac the term CRO not being explained in the title of this issue, and the reason this doc is not public are, I feel, two separate issues. I’d be happy to discuss this with you in person, but I don’t feel async here is likely the appropriate place.

@pmac
Copy link
Member

pmac commented Sep 19, 2019

Fair enough. I just assumed since the issue was public and this repo has 115 watchers and 851 stars as of this comment that people would appreciate some explanation, like I wanted since this whole thing made no sense w/o context. If things need to be private that's fine, but it's also a capability we have with confidential bugzilla flags.

@alexgibson
Copy link
Member

alexgibson commented Sep 20, 2019

Whether we use confidential bugs in bugzilla, or an employee only gdoc - it will likely turn up here in GitHub, linked in some way, as that’s currently the nature of how we track progress of our work. But as I say, this feels like a different discussion as to why the meaning of CRO was not originally explained. Again, i’d be more than happy to discuss this with you in person, rather than here where we can concentrate on discussing the issue at hand.

@alexgibson
Copy link
Member

@jgmize just talked to @hoosteeno and we'd like to test out using cloud workers on staging, providing a) you have some time & availability to work on it, and b) it's not a high level of effort / from yourself (it didn't sound like it from our meeting, but just to double check).

Happy to discus further in person if it helps.

@jgmize
Copy link
Contributor

jgmize commented Sep 26, 2019

@alexgibson and I just had a short pairing session where we experimented with temporarily deploying some code generated with http://sloth.cloud to www.allizom.org. Unfortunately sloth couldn't generate exactly what we wanted for this, so we will need to write a bit of custom code to conditionally return a 302 redirect from https://www.allizom.org/en-US/firefox/new/ to https://www.allizom.org/en-US/exp/firefox/new/ a small percentage of the time while preserving query params. We plan on working on this together again on Monday.

@alexgibson
Copy link
Member

I wrote some pseudo-ish code that could serve as a redirector here: https://gist.github.com/alexgibson/3425f57088d6fde02b6f0797eb738e08

This is aimed at the browser, but hopefully something we can adapt easily enough for a worker environment.

@alexgibson
Copy link
Member

Both myself and @jgmize paired on another session today and we we're able to modify the code linked above in the gist to run in a cloudflare worker. We spent some time testing the redirector on stage and it seemed to be performing as expected.

Next steps:

  • I'm going to tidy the code up and perform some more testing on stage.
  • Once I'm happy that it's doing the right things, we're going to commit this boilerplate code to version control (in bedrock).

Once the above is complete, I think we're good to do a full end-to-end experiment test on stage.

@pmac
Copy link
Member

pmac commented Sep 30, 2019

My apologies for this tangent, but this talk of cloudflare workers has me curious: could such a worker be a better solution for our GEO redirects for whatsnew and others? Can a worker use the country of request origin as a data point in its logic?

@jgmize
Copy link
Contributor

jgmize commented Sep 30, 2019

@pmac yes, there's an example of doing exactly that here: https://developers.cloudflare.com/workers/templates/?q=count

@pmac
Copy link
Member

pmac commented Sep 30, 2019

Nice. I'm interested to see how the development and deployment automation for CF workers goes then. It'd be a nice way to accomplish these kinds of things, and much more compatible with moving to a more static deployment w/o sacrificing features.

@alexgibson
Copy link
Member

We set up a new repo called www-workers, and I've submitted a PR for the redirector code: mozmeao/www-workers#1

@alexgibson
Copy link
Member

The code for this now lives at https://github.com/mozmeao/www-workers and is running on stage. @jgmize still has some infra work to do with automation in that repo, which is detailed here: mozmeao/www-workers#2

On that basis, I'm going to close this particular issue and followups can be discussed in the www-workers repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 Second level priority - Should have
Projects
None yet
Development

No branches or pull requests

7 participants
@hoosteeno @jgmize @alexgibson @pmac @ejregithub @amychurchwell and others