Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix deploy job; use absolute path for HOME #175

Merged
merged 3 commits into from
Jul 10, 2023
Merged

Fix deploy job; use absolute path for HOME #175

merged 3 commits into from
Jul 10, 2023

Conversation

glin
Copy link
Contributor

@glin glin commented Jul 7, 2023

Pulled out the smaller changes from #171 to fix the deploy job with the existing docker inside docker packager:

  • Set HOME to an absolute path, env.WORKSPACE, rather than a relative ., which is needed for both the docker bind mounting to work correctly, and the serverless-python-requirements plugin correctly include Python packages in the zip file.
  • Add docs for how the deploy job works
  • Add verbose logging since the plugin hides the actual error on failures. Now you should be able to see the docker run command:
[2023-07-06T23:37:22.224Z] ./node_modules/serverless/bin/serverless.js deploy --stage staging --verbose
[2023-07-06T23:37:26.380Z] 
[2023-07-06T23:37:26.380Z] Deploying r-builds to stage staging (us-east-1)
[2023-07-06T23:37:26.380Z] 
[2023-07-06T23:37:26.380Z] Generated requirements from /var/lib/jenkins/workspace/r-builds/deploy-r-builds/requirements.txt in /var/lib/jenkins/workspace/r-builds/deploy-r-builds/.serverless/requirements.txt
[2023-07-06T23:37:26.380Z] Installing requirements from "/var/lib/jenkins/workspace/r-builds/deploy-r-builds/.cache/serverless-python-requirements/2b10c0467f75bfbcf12e9b77be65ec7563b90c5995f0eaa96faefbd42c337bdd_x86_64_slspyc/requirements.txt"
[2023-07-06T23:37:26.380Z] Docker Image: lambci/lambda:build-python3.8
[2023-07-06T23:37:26.380Z] Using download cache directory /var/lib/jenkins/workspace/r-builds/deploy-r-builds/.cache/serverless-python-requirements/downloadCacheslspyc
[2023-07-06T23:37:26.380Z] Running docker run --rm -v /var/lib/jenkins/workspace/r-builds/deploy-r-builds/.cache/serverless-python-requirements/2b10c0467f75bfbcf12e9b77be65ec7563b90c5995f0eaa96faefbd42c337bdd_x86_64_slspyc\:/var/task\:z -v /var/lib/jenkins/workspace/r-builds/deploy-r-builds/.cache/serverless-python-requirements/downloadCacheslspyc\:/var/useDownloadCache\:z lambci/lambda\:build-python3.8 /bin/sh -c 'chown -R 0\\:0 /var/useDownloadCache && python3.8 -m pip install -t /var/task/ -r /var/task/requirements.txt --cache-dir /var/useDownloadCache && chown -R 386\\:386 /var/task && chown -R 386\\:386 /var/useDownloadCache'...
  • Exclude other .cache/.npm generated files that didn't need to be in the serverless package
$ unzip r-builds-staging-queueBuilds-fda4c5bf-5e39-4c62-9eef-e358b4135ceb.zip 
$ du -sh $(ls -A) | sort -h | tail -n5
3.8M	docutils
28M	.npm
38M	botocore
46M	r-builds-staging-queueBuilds-fda4c5bf-5e39-4c62-9eef-e358b4135ceb.zip
56M	.cache

I just tested this again to be sure at https://build.posit.it/blue/organizations/jenkins/r-builds%2Fdeploy-r-builds/detail/deploy-r-builds/202/pipeline/, and confirmed that staging builds still work.

@glin glin requested review from jforest, jspiewak and stevenolen July 7, 2023 00:08
@glin glin merged commit 05d4d38 into main Jul 10, 2023
@glin glin deleted the glin-fix-deploy branch July 10, 2023 19:07
@glin glin mentioned this pull request Jul 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants