Skip to content
This repository has been archived by the owner on May 6, 2024. It is now read-only.

feat: add playbooks for enterprise-access and enterprise-subsidy #7107

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions playbooks/enterprise_access.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
- name: Deploy enterprise-access
hosts: all
become: True
gather_facts: True
vars:
ENABLE_NEWRELIC: True
CLUSTER_NAME: 'enterprise_access'
# Note: enterprise-access uses memcache by default. If a different cache is used,
# set this to false and set ENTERPRISE_ACCESS_MEMCACHE and ENTERPRISE_ACCESS_MEMCACHE_BACKEND
# to appropriate values.
ENTERPRISE_ACCESS_MEMCACHE_ENABLED: true
ENTERPRISE_ACCESS_HERMES_ENABLED: false
roles:
- role: aws
when: COMMON_ENABLE_AWS_ROLE
- role: nginx
nginx_default_sites:
- enterprise_access
- role: memcache
when: ENTERPRISE_ACCESS_MEMCACHE_ENABLED
- enterprise_access
- role: splunkforwarder
when: COMMON_ENABLE_SPLUNKFORWARDER
- role: newrelic_infrastructure
when: COMMON_ENABLE_NEWRELIC_INFRASTRUCTURE
- role: hermes
HERMES_TARGET_SERVICE: 'enterprise_access'
when: ENTERPRISE_ACCESS_HERMES_ENABLED
2 changes: 1 addition & 1 deletion playbooks/enterprise_catalog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
- name: Deploy edX designer
- name: Deploy enterprise-catalog
hosts: all
become: True
gather_facts: True
Expand Down
28 changes: 28 additions & 0 deletions playbooks/enterprise_subsidy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
- name: Deploy enterprise-subsidy
hosts: all
become: True
gather_facts: True
vars:
ENABLE_NEWRELIC: True
CLUSTER_NAME: 'enterprise_subsidy'
# enterprise-subsidy by default uses Memcache for caching. If a different cache is used, set
# this to false and set ENTERPRISE_SUBSIDY_MEMCACHE and ENTERPRISE_SUBSIDY_MEMCACHE_BACKEND
# to appropriate values.
ENTERPRISE_SUBSIDY_MEMCACHE_ENABLED: true
ENTERPRISE_SUBSIDY_HERMES_ENABLED: false
roles:
- role: aws
when: COMMON_ENABLE_AWS_ROLE
- role: nginx
nginx_default_sites:
- enterprise_subsidy
- role: memcache
when: ENTERPRISE_SUBSIDY_MEMCACHE_ENABLED
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be undefined. Besides, do we need to deploy Memcached here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docker-compose.yml file of enterprise-subsidy does run memcache as a service. So, I added this flag to make it available as an option.

You are right, I missed properly mapping the CACHE config to this flag.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since both the services use memcache in their setup, I have added it to both the playbooks and used the default localhost:11211 in the config. Also included a comment that if the flag is turned off, then the other related settings need to be changed as well.

So, this should allow for disabling memcache installation and setting the config to a totally different backend, or pointing to a remote memcahce.

It makes sense to me, but I am not sure if this is the best way to do it. Kindly change it, if I am doing something wrong here.

- enterprise_subsidy
- role: splunkforwarder
when: COMMON_ENABLE_SPLUNKFORWARDER
- role: newrelic_infrastructure
when: COMMON_ENABLE_NEWRELIC_INFRASTRUCTURE
- role: hermes
HERMES_TARGET_SERVICE: 'enterprise_subsidy'
when: ENTERPRISE_SUBSIDY_HERMES_ENABLED
174 changes: 174 additions & 0 deletions playbooks/roles/enterprise_access/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
---
#
# edX Configuration
#
# github: https://github.com/openedx/configuration
# wiki: https://openedx.atlassian.net/wiki/display/OpenOPS
# code style: https://openedx.atlassian.net/wiki/spaces/OpenOPS/pages/26837527/Ansible+Code+Conventions
# license: https://github.com/openedx/configuration/blob/master/LICENSE.TXT
#
##
# Defaults for role enterprise_access
#


#
# vars are namespace with the module name.
#
enterprise_access_service_name: 'enterprise_access'

enterprise_access_user: "{{ enterprise_access_service_name }}"
enterprise_access_home: "{{ COMMON_APP_DIR }}/{{ enterprise_access_service_name }}"
enterprise_access_app_dir: "{{ COMMON_APP_DIR }}/{{ enterprise_access_service_name }}"
enterprise_access_code_dir: "{{ enterprise_access_app_dir }}/{{ enterprise_access_service_name }}"
enterprise_access_venvs_dir: "{{ enterprise_access_app_dir }}/venvs"
enterprise_access_venv_dir: "{{ enterprise_access_venvs_dir }}/enterprise_access"

enterprise_access_celery_default_queue: 'enterprise_access.default'

enterprise_access_hostname: 'enterprise-access'

ENTERPRISE_ACCESS_CELERY_ALWAYS_EAGER: false
ENTERPRISE_ACCESS_CELERY_BROKER_TRANSPORT: ''
ENTERPRISE_ACCESS_CELERY_BROKER_USER: ''
ENTERPRISE_ACCESS_CELERY_BROKER_PASSWORD: ''
ENTERPRISE_ACCESS_CELERY_BROKER_HOSTNAME: ''
ENTERPRISE_ACCESS_CELERY_BROKER_VHOST: ''

ENTERPRISE_ACCESS_USE_PYTHON38: True

ENTERPRISE_ACCESS_ENVIRONMENT_EXTRA: {}

enterprise_access_environment:
ENTERPRISE_ACCESS_CFG: '{{ COMMON_CFG_DIR }}/{{ enterprise_access_service_name }}.yml'
Comment on lines +42 to +43
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This must be overridden to specify more env variables. Why don't we add ENTERPRISE_ACCESS_ENVIRONMENT_EXTRA: {} and pass it like edx_django_service_environment_extra: '{{ enterprise_access_environment | combine(ENTERPRISE_ACCESS_ENVIRONMENT_EXTRA) }}'

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


enterprise_access_gunicorn_port: 8270

enterprise_access_debian_pkgs: []

ENTERPRISE_ACCESS_ENABLE_EXPERIMENTAL_DOCKER_SHIM: false

ENTERPRISE_ACCESS_NGINX_PORT: '1{{ enterprise_access_gunicorn_port }}'
ENTERPRISE_ACCESS_SSL_NGINX_PORT: '4{{ enterprise_access_gunicorn_port }}'

ENTERPRISE_ACCESS_DEFAULT_DB_NAME: 'enterprise_access'
ENTERPRISE_ACCESS_MYSQL_HOST: 'localhost'
# MySQL usernames are limited to 16 characters
ENTERPRISE_ACCESS_MYSQL_USER: 'entaccess001'
ENTERPRISE_ACCESS_MYSQL_PASSWORD: 'password'

ENTERPRISE_ACCESS_MEMCACHE: [ 'localhost:11211' ]
# The memcache config of edx_django_service is overriden in defaults/main.yml to use PyMemcacheCache
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# The memcache config of edx_django_service is overriden in defaults/main.yml to use PyMemcacheCache
# The memcache config of edx_django_service is overridden in defaults/main.yml to use PyMemcacheCache

# as enterprise-access runs on Django 4.2 which doesn't support MemcachedCache backend.
ENTERPRISE_ACCESS_MEMCACHE_BACKEND: 'django.core.cache.backends.memcached.PyMemcacheCache'

ENTERPRISE_ACCESS_DJANGO_SETTINGS_MODULE: 'enterprise_access.settings.production'
ENTERPRISE_ACCESS_DOMAIN: 'localhost'
ENTERPRISE_ACCESS_URL_ROOT: 'http://{{ ENTERPRISE_ACCESS_DOMAIN }}:{{ ENTERPRISE_ACCESS_NGINX_PORT }}'
ENTERPRISE_ACCESS_LOGOUT_URL: '{{ ENTERPRISE_ACCESS_URL_ROOT }}/logout/'

ENTERPRISE_ACCESS_LANGUAGE_CODE: 'en'
ENTERPRISE_ACCESS_LANGUAGE_COOKIE_NAME: 'openedx-language-preference'

ENTERPRISE_ACCESS_SERVICE_USER: 'enterprise_access_service_user'

ENTERPRISE_ACCESS_DATA_DIR: '{{ COMMON_DATA_DIR }}/{{ enterprise_access_service_name }}'
ENTERPRISE_ACCESS_MEDIA_ROOT: '{{ ENTERPRISE_ACCESS_DATA_DIR }}/media'
ENTERPRISE_ACCESS_MEDIA_URL: '/media/'

ENTERPRISE_ACCESS_MEDIA_STORAGE_BACKEND:
DEFAULT_FILE_STORAGE: 'django.core.files.storage.FileSystemStorage'
MEDIA_ROOT: '{{ ENTERPRISE_ACCESS_MEDIA_ROOT }}'
MEDIA_URL: '{{ ENTERPRISE_ACCESS_MEDIA_URL }}'

ENTERPRISE_ACCESS_STATIC_ROOT: '{{ ENTERPRISE_ACCESS_DATA_DIR }}/staticfiles'
ENTERPRISE_ACCESS_STATIC_URL: '/static/'

ENTERPRISE_ACCESS_STATICFILES_STORAGE: 'django.contrib.staticfiles.storage.StaticFilesStorage'

ENTERPRISE_ACCESS_CORS_ORIGIN_ALLOW_ALL: false
ENTERPRISE_ACCESS_CORS_ORIGIN_WHITELIST_DEFAULT:
- '{{ ENTERPRISE_ACCESS_URL_ROOT }}'

ENTERPRISE_ACCESS_CORS_ORIGIN_WHITELIST_EXTRA: []
ENTERPRISE_ACCESS_CORS_ORIGIN_WHITELIST: '{{ ENTERPRISE_ACCESS_CORS_ORIGIN_WHITELIST_DEFAULT + ENTERPRISE_ACCESS_CORS_ORIGIN_WHITELIST_EXTRA }}'
ENTERPRISE_ACCESS_CSRF_TRUSTED_ORIGINS: []

ENTERPRISE_ACCESS_VERSION: 'main'

ENTERPRISE_ACCESS_GUNICORN_EXTRA: ''

ENTERPRISE_ACCESS_EXTRA_APPS: []

ENTERPRISE_ACCESS_SESSION_EXPIRE_AT_BROWSER_CLOSE: false

enterprise_access_service_config_overrides:
LANGUAGE_COOKIE_NAME: '{{ ENTERPRISE_ACCESS_LANGUAGE_COOKIE_NAME }}'
CSRF_COOKIE_SECURE: '{{ ENTERPRISE_ACCESS_CSRF_COOKIE_SECURE }}'
CSRF_TRUSTED_ORIGINS: '{{ ENTERPRISE_ACCESS_CSRF_TRUSTED_ORIGINS }}'
CELERY_ALWAYS_EAGER: '{{ ENTERPRISE_ACCESS_CELERY_ALWAYS_EAGER }}'
CELERY_BROKER_TRANSPORT: '{{ ENTERPRISE_ACCESS_CELERY_BROKER_TRANSPORT }}'
CELERY_BROKER_USER: '{{ ENTERPRISE_ACCESS_CELERY_BROKER_USER }}'
CELERY_BROKER_PASSWORD: '{{ ENTERPRISE_ACCESS_CELERY_BROKER_PASSWORD }}'
CELERY_BROKER_HOSTNAME: '{{ ENTERPRISE_ACCESS_CELERY_BROKER_HOSTNAME }}'
CELERY_BROKER_VHOST: '{{ ENTERPRISE_ACCESS_CELERY_BROKER_VHOST }}'
CELERY_DEFAULT_EXCHANGE: 'enterprise_access'
CELERY_DEFAULT_ROUTING_KEY: 'enterprise_access'
CELERY_DEFAULT_QUEUE: '{{ enterprise_access_celery_default_queue }}'
CORS_ORIGIN_WHITELIST: "{{ ENTERPRISE_ACCESS_CORS_ORIGIN_WHITELIST }}"
EDX_DRF_EXTENSIONS:
OAUTH2_USER_INFO_URL: '{{ COMMON_LMS_BASE_URL }}/user_info'
JWT_PAYLOAD_USER_ATTRIBUTE_MAPPING: {
"administrator": "is_staff",
"email": "email",
"full_name": "full_name",
"user_id": "lms_user_id",
}

# Enterprise Access doesn't read the URLs for other services like LMS, License Manager..etc.,
# from the environment variables. They should be passed as extra configs
ENTERPRISE_ACCESS_SERVICE_CONFIG_EXTRA: {}

# See edx_django_service_automated_users for an example of what this should be
ENTERPRISE_ACCESS_AUTOMATED_USERS: {}

ENTERPRISE_ACCESS_CSRF_COOKIE_SECURE: false

ENTERPRISE_ACCESS_ENABLE_NEWRELIC_DISTRIBUTED_TRACING: false

enterprise_access_post_migrate_commands: []

ENTERPRISE_ACCESS_SOCIAL_AUTH_EDX_OAUTH2_KEY: 'enterprise-access-sso-key'
ENTERPRISE_ACCESS_SOCIAL_AUTH_EDX_OAUTH2_SECRET: 'enterprise-access-sso-secret'
ENTERPRISE_ACCESS_BACKEND_SERVICE_EDX_OAUTH2_KEY: 'enterprise-access-backend-service-key'
ENTERPRISE_ACCESS_BACKEND_SERVICE_EDX_OAUTH2_SECRET: 'enterprise-access-backend-service-secret'

ENTERPRISE_ACCESS_GIT_IDENTITY: !!null

ENTERPRISE_ACCESS_REPOS:
- PROTOCOL: '{{ COMMON_GIT_PROTOCOL }}'
DOMAIN: '{{ COMMON_GIT_MIRROR }}'
PATH: '{{ COMMON_GIT_PATH }}'
REPO: 'enterprise-access.git'
VERSION: '{{ ENTERPRISE_ACCESS_VERSION }}'
DESTINATION: "{{ enterprise_access_code_dir }}"
SSH_KEY: '{{ ENTERPRISE_ACCESS_GIT_IDENTITY }}'

ENTERPRISE_ACCESS_SECRET_KEY: 'SET-ME-PLEASE'

# Remote config
ENTERPRISE_ACCESS_HERMES_ENABLED: "{{ COMMON_HERMES_ENABLED }}"

ENTERPRISE_ACCESS_DECRYPT_CONFIG_ENABLED: "{{ COMMON_DECRYPT_CONFIG_ENABLED }}"
ENTERPRISE_ACCESS_COPY_CONFIG_ENABLED: "{{ COMMON_COPY_CONFIG_ENABLED }}"

ENTERPRISE_ACCESS_ENABLE_ADMIN_URLS_RESTRICTION: false
ENTERPRISE_ACCESS_ADMIN_URLS:
- admin

# Worker settings
worker_django_settings_module: "{{ ENTERPRISE_ACCESS_DJANGO_SETTINGS_MODULE }}"
ENTERPRISE_ACCESS_CELERY_WORKERS:
- queue: '{{ enterprise_access_celery_default_queue }}'
concurrency: 4
monitor: True
63 changes: 63 additions & 0 deletions playbooks/roles/enterprise_access/meta/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
#
# edX Configuration
#
# github: https://github.com/openedx/configuration
# wiki: https://openedx.atlassian.net/wiki/display/OpenOPS
# code style: https://openedx.atlassian.net/wiki/display/OpenOPS/Ansible+Code+Conventions
# license: https://github.com/openedx/configuration/blob/master/LICENSE.TXT
#
##
# Role includes for role enterprise_access

dependencies:
- role: edx_django_service
edx_django_service_use_python38: '{{ ENTERPRISE_ACCESS_USE_PYTHON38 }}'
edx_django_service_enable_experimental_docker_shim: '{{ ENTERPRISE_ACCESS_ENABLE_EXPERIMENTAL_DOCKER_SHIM }}'
edx_django_service_version: '{{ ENTERPRISE_ACCESS_VERSION }}'
edx_django_service_name: '{{ enterprise_access_service_name }}'
edx_django_service_config_overrides: '{{ enterprise_access_service_config_overrides | combine(ENTERPRISE_ACCESS_SERVICE_CONFIG_EXTRA) }}'
edx_django_service_debian_pkgs_extra: '{{ enterprise_access_debian_pkgs }}'
edx_django_service_gunicorn_port: '{{ enterprise_access_gunicorn_port }}'
edx_django_service_repos: '{{ ENTERPRISE_ACCESS_REPOS }}'
edx_django_service_django_settings_module: '{{ ENTERPRISE_ACCESS_DJANGO_SETTINGS_MODULE }}'
edx_django_service_environment_extra: '{{ enterprise_access_environment | combine(ENTERPRISE_ACCESS_ENVIRONMENT_EXTRA) }}'
edx_django_service_gunicorn_extra: '{{ ENTERPRISE_ACCESS_GUNICORN_EXTRA }}'
edx_django_service_nginx_port: '{{ ENTERPRISE_ACCESS_NGINX_PORT }}'
edx_django_service_ssl_nginx_port: '{{ ENTERPRISE_ACCESS_SSL_NGINX_PORT }}'
edx_django_service_language_code: '{{ ENTERPRISE_ACCESS_LANGUAGE_CODE }}'
edx_django_service_secret_key: '{{ ENTERPRISE_ACCESS_SECRET_KEY }}'
edx_django_service_media_storage_backend: '{{ ENTERPRISE_ACCESS_MEDIA_STORAGE_BACKEND }}'
edx_django_service_staticfiles_storage: '{{ ENTERPRISE_ACCESS_STATICFILES_STORAGE }}'
edx_django_service_memcache: '{{ ENTERPRISE_ACCESS_MEMCACHE }}'
edx_django_service_caches:
default:
BACKEND: '{{ ENTERPRISE_ACCESS_MEMCACHE_BACKEND }}'
KEY_PREFIX: '{{ edx_django_service_name }}'
LOCATION: '{{ edx_django_service_memcache }}'
edx_django_service_default_db_host: '{{ ENTERPRISE_ACCESS_MYSQL_HOST }}'
edx_django_service_default_db_name: '{{ ENTERPRISE_ACCESS_DEFAULT_DB_NAME }}'
edx_django_service_default_db_atomic_requests: false
edx_django_service_db_user: '{{ ENTERPRISE_ACCESS_MYSQL_USER }}'
edx_django_service_db_password: '{{ ENTERPRISE_ACCESS_MYSQL_PASSWORD }}'
edx_django_service_extra_apps: '{{ ENTERPRISE_ACCESS_EXTRA_APPS }}'
edx_django_service_session_expire_at_browser_close: '{{ ENTERPRISE_ACCESS_SESSION_EXPIRE_AT_BROWSER_CLOSE }}'
edx_django_service_social_auth_edx_oauth2_key: '{{ ENTERPRISE_ACCESS_SOCIAL_AUTH_EDX_OAUTH2_KEY }}'
edx_django_service_social_auth_edx_oauth2_secret: '{{ ENTERPRISE_ACCESS_SOCIAL_AUTH_EDX_OAUTH2_SECRET }}'
edx_django_service_backend_service_edx_oauth2_key: '{{ ENTERPRISE_ACCESS_BACKEND_SERVICE_EDX_OAUTH2_KEY }}'
edx_django_service_backend_service_edx_oauth2_secret: '{{ ENTERPRISE_ACCESS_BACKEND_SERVICE_EDX_OAUTH2_SECRET }}'
edx_django_service_automated_users: '{{ ENTERPRISE_ACCESS_AUTOMATED_USERS }}'
edx_django_service_cors_whitelist: '{{ ENTERPRISE_ACCESS_CORS_ORIGIN_WHITELIST }}'
edx_django_service_post_migrate_commands: '{{ enterprise_access_post_migrate_commands }}'
edx_django_service_enable_newrelic_distributed_tracing: '{{ ENTERPRISE_ACCESS_ENABLE_NEWRELIC_DISTRIBUTED_TRACING }}'
edx_django_service_decrypt_config_enabled: '{{ ENTERPRISE_ACCESS_DECRYPT_CONFIG_ENABLED }}'
edx_django_service_copy_config_enabled: '{{ ENTERPRISE_ACCESS_COPY_CONFIG_ENABLED }}'
edx_django_service_migration_check_services: '{{ enterprise_access_service_name }},{{ enterprise_access_service_name }}_workers'
edx_django_service_enable_celery_workers: true
edx_django_service_workers: '{{ ENTERPRISE_ACCESS_CELERY_WORKERS }}'
# Need this override because the existing AWS ASGs have a services tag with a name that doesn't match the convention
edx_django_service_workers_supervisor_conf: 'enterprise_access_worker.conf'
edx_django_service_docker_image_name: 'openedx/enterprise-access'
edx_django_service_hostname: '~^((stage|prod)-)?{{ enterprise_access_hostname }}.*'
EDX_DJANGO_SERVICE_ENABLE_ADMIN_URLS_RESTRICTION: '{{ ENTERPRISE_ACCESS_ENABLE_ADMIN_URLS_RESTRICTION }}'
EDX_DJANGO_SERVICE_ADMIN_URLS: '{{ ENTERPRISE_ACCESS_ADMIN_URLS }}'
22 changes: 22 additions & 0 deletions playbooks/roles/enterprise_access/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
#
# edX Configuration
#
# github: https://github.com/openedx/configuration
# wiki: https://openedx.atlassian.net/wiki/display/OpenOPS
# code style: https://openedx.atlassian.net/wiki/display/OpenOPS/Ansible+Code+Conventions
# license: https://github.com/openedx/configuration/blob/master/LICENSE.TXT
#
#
#
# Tasks for role enterprise_access
#
# Overview: This role's tasks come from edx_django_service.
#
#
# Dependencies:
#
#
# Example play:
#
#
3 changes: 2 additions & 1 deletion playbooks/roles/enterprise_catalog/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,8 @@ DEFAULT_FILE_STORAGE: 'django.core.files.storage.FileSystemStorage'
MEDIA_ROOT: '{{ ENTERPRISE_CATALOG_MEDIA_ROOT }}'
MEDIA_URL: '{{ ENTERPRISE_CATALOG_MEDIA_URL }}'

# TODO: Let edx_django_service manage ENTERPRISE_CATALOG_STATIC_ROOT in phase 2.
# TODO: Let edx_django_service manage ENTERPRISE_CATALOG_STATIC_ROOT in phase 2. Also, update enterprise_access and
# enterprise_subsidy.
ENTERPRISE_CATALOG_STATIC_ROOT: '{{ ENTERPRISE_CATALOG_DATA_DIR }}/staticfiles'
ENTERPRISE_CATALOG_STATIC_URL: '/static/'

Expand Down
Loading
Loading