python error during build phase

[stemons]

Hello team,

I noticed the following error during the build phase. This error is common to several pipelines.

[Container] 2022/02/18 12:58:32 Running command pip install -r adf-build/requirements.txt -q

24 | Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-lv6q_hbc/MarkupSafe/
25 | You are using pip version 18.1, however version 22.0.3 is available.
26 | You should consider upgrading via the 'pip install --upgrade pip' command.

This has been resolved with a pip upgrade

[jcfdeb]

Also we've a Python pip error during the build phase:

============================== 21 passed in 0.31s ==============================
--
61 |  
62 | [Container] 2022/02/18 15:18:10 Phase complete: PRE_BUILD State: SUCCEEDED
63 | [Container] 2022/02/18 15:18:10 Phase context status code:  Message:
64 | [Container] 2022/02/18 15:18:10 Entering phase BUILD
65 | [Container] 2022/02/18 15:18:10 Running command sam build -t adf-bootstrap/deployment/global.yml
66 |  
67 | SAM CLI now collects telemetry to better understand customer needs.
68 |  
69 | You can OPT OUT and disable telemetry collection by setting the
70 | environment variable SAM_CLI_TELEMETRY=0 in your shell.
71 | Thanks for your help!
72 |  
73 | Learn More: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-telemetry.html
74 |  
75 | Building codeuri: lambda_codebase/ runtime: python3.8 metadata: {} functions: ['SendSlackNotification', 'EnableCrossAccountAccess', 'CheckPipelineStatus']
76 | Running PythonPipBuilder:ResolveDependencies
77 | Running PythonPipBuilder:CopySource
78 | Building codeuri: lambda_codebase/initial_commit runtime: python3.8 metadata: {} functions: ['InitialCommitHandler']
79 | Running PythonPipBuilder:ResolveDependencies
80 |  
81 | Build Failed
82 | Error: PythonPipBuilder:ResolveDependencies - {markupsafe==2.1.0(wheel)}
83 |  
84 | [Container] 2022/02/18 15:18:15 Command did not exit successfully sam build -t adf-bootstrap/deployment/global.yml exit status 1
85 | [Container] 2022/02/18 15:18:15 Phase complete: BUILD State: FAILED
86 | [Container] 2022/02/18 15:18:15 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: sam build -t adf-bootstrap/deployment/global.yml. Reason: exit status 1
87 | [Container] 2022/02/18 15:18:15 Entering phase POST_BUILD
88 | [Container] 2022/02/18 15:18:15 Phase complete: POST_BUILD State: SUCCEEDED
89 | [Container] 2022/02/18 15:18:15 Phase context status code:  Message:

The maifest used was:

accounts:
  - account_full_name: account-cloud-name
    organizational_unit_path: /
    email: [email protected]
    allow_billing: True
    delete_default_vpc: True
    alias: 
    tags:
      - created_by: adf
      - JiraTicket: CLOUDHD-627

Thanks

[StewartW]

If you update the requirements.txt file in adf-build (Located in your bootstrap repository) to mirror the current version in master (https://github.com/awslabs/aws-deployment-framework/blob/master/src/lambda_codebase/initial_commit/bootstrap_repository/adf-build/requirements.txt)

That appears to resolve this issue.

[stemons]

It worked. Thanks!

[hynynen]

Same here, had this problem and fixed it.

It would be good to have a bit faster release cycle, or some other mechanism to stay up-to-date with ADF without making full update via SAM.

[jcfdeb]

Also has worked for us in a pipeline but still failing in another one.

Using the 'requirements.txt' posted by @StewartW

[Container] 2022/02/21 13:30:36 Waiting for agent ping
--
2 | [Container] 2022/02/21 13:30:38 Waiting for DOWNLOAD_SOURCE
3 | [Container] 2022/02/21 13:30:38 Phase is DOWNLOAD_SOURCE
4 | [Container] 2022/02/21 13:30:38 CODEBUILD_SRC_DIR=/codebuild/output/src311577082/src
5 | [Container] 2022/02/21 13:30:38 YAML location is /codebuild/output/src311577082/src/buildspec.yml
6 | [Container] 2022/02/21 13:30:38 No commands found for phase name: install
7 | [Container] 2022/02/21 13:30:38 Processing environment variables
8 | [Container] 2022/02/21 13:30:38 [WARN] Skipping install of runtimes. Runtime version selection is not supported by this build image.
9 | [Container] 2022/02/21 13:30:40 Moving to directory /codebuild/output/src311577082/src
10 | [Container] 2022/02/21 13:30:40 Registering with agent
11 | [Container] 2022/02/21 13:30:40 Phases found in YAML: 3
12 | [Container] 2022/02/21 13:30:40  INSTALL: 0 commands
13 | [Container] 2022/02/21 13:30:40  PRE_BUILD: 2 commands
14 | [Container] 2022/02/21 13:30:40  BUILD: 2 commands
15 | [Container] 2022/02/21 13:30:40 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED
16 | [Container] 2022/02/21 13:30:40 Phase context status code:  Message:
17 | [Container] 2022/02/21 13:30:40 Entering phase INSTALL
18 | [Container] 2022/02/21 13:30:40 Phase complete: INSTALL State: SUCCEEDED
19 | [Container] 2022/02/21 13:30:40 Phase context status code:  Message:
20 | [Container] 2022/02/21 13:30:40 Entering phase PRE_BUILD
21 | [Container] 2022/02/21 13:30:40 Running command aws s3 cp s3://$S3_BUCKET_NAME/adf-build/ adf-build/ --recursive --quiet
22 |  
23 | [Container] 2022/02/21 13:30:41 Running command pip install -r adf-build/requirements.txt -q
24 | Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-f7dve6km/MarkupSafe/
25 | You are using pip version 18.1, however version 22.0.3 is available.
26 | You should consider upgrading via the 'pip install --upgrade pip' command.
27 |  
28 | [Container] 2022/02/21 13:31:00 Command did not exit successfully pip install -r adf-build/requirements.txt -q exit status 1
29 | [Container] 2022/02/21 13:31:00 Phase complete: PRE_BUILD State: FAILED
30 | [Container] 2022/02/21 13:31:00 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: pip install -r adf-build/requirements.txt -q. Reason: exit status 1

It was in the 'build' phase of our 'adf-pipeline-config' pipeline

Best regards

[stemons]

A quick update. I solved the issue on bootstrap pipeline adding the correct version of the libraries in requirements.txt, but we have the same issue on other pipelines that try to install the requirements:

[Container] 2022/02/21 17:29:17 Entering phase INSTALL
18 | [Container] 2022/02/21 17:29:17 Phase complete: INSTALL State: SUCCEEDED
19 | [Container] 2022/02/21 17:29:17 Phase context status code:  Message:
20 | [Container] 2022/02/21 17:29:17 Entering phase PRE_BUILD
21 | [Container] 2022/02/21 17:29:17 Running command aws s3 cp s3://$S3_BUCKET_NAME/adf-build/ adf-build/ --recursive --quiet
22 |  
23 | [Container] 2022/02/21 17:29:18 Running command cat adf-build/requirements.txt
24 | # Install libs here that you might want in AWS CodeBuild
25 | aws-cdk-assets==1.137
26 | aws-cdk.aws-apigateway==1.137
27 | aws-cdk.aws-applicationautoscaling==1.137
28 | aws-cdk.aws-autoscaling==1.137
29 | aws-cdk.aws-autoscaling_common==1.137
30 | aws-cdk.aws-autoscaling_hooktargets==1.137
31 | aws-cdk.aws-certificatemanager==1.137
32 | aws-cdk.aws-chatbot==1.137
33 | aws-cdk.aws-cloudformation==1.137
34 | aws-cdk.aws-cloudfront==1.137
35 | aws-cdk.aws-cloudwatch==1.137
36 | aws-cdk.aws-codebuild==1.137
37 | aws-cdk.aws-codecommit==1.137
38 | aws-cdk.aws-codedeploy==1.137
39 | aws-cdk.aws-codepipeline-actions==1.137
40 | aws-cdk.aws-codepipeline==1.137
41 | aws-cdk.aws-dynamodb==1.137
42 | aws-cdk.aws-ec2==1.137
43 | aws-cdk.aws-ecr==1.137
44 | aws-cdk.aws-ecr==1.137
45 | aws-cdk.aws-ecr_assets==1.137
46 | aws-cdk.aws-ecs==1.137
47 | aws-cdk.aws-elasticloadbalancing==1.137
48 | aws-cdk.aws-elasticloadbalancingv2==1.137
49 | aws-cdk.aws-events==1.137
50 | aws-cdk.aws-events_targets==1.137
51 | aws-cdk.aws-iam==1.137
52 | aws-cdk.aws-kinesis==1.137
53 | aws-cdk.aws-kms==1.137
54 | aws-cdk.aws-lambda==1.137
55 | aws-cdk.aws-lambda_event_sources==1.137
56 | aws-cdk.aws-logs==1.137
57 | aws-cdk.aws-route53==1.137
58 | aws-cdk.aws-route53_targets==1.137
59 | aws-cdk.aws-s3==1.137
60 | aws-cdk.aws-s3_assets==1.137
61 | aws-cdk.aws-s3_assets==1.137
62 | aws-cdk.aws-s3_notifications==1.137
63 | aws-cdk.aws-sam==1.137
64 | aws-cdk.aws-secretsmanager==1.137
65 | aws-cdk.aws-servicediscovery==1.137
66 | aws-cdk.aws-sns==1.137
67 | aws-cdk.aws-sns==1.137
68 | aws-cdk.aws-sns_subscriptions==1.137
69 | aws-cdk.aws-sqs==1.137
70 | aws-cdk.aws-ssm==1.137
71 | aws-cdk.aws-stepfunctions==1.137
72 | aws-cdk.core==1.137
73 | aws-cdk.cx-api==1.137
74 | aws-cdk.region-info==1.137
75 | awscli==1.22.26
76 | aws-sam-cli==1.36.0
77 | botocore==1.23.26
78 | boto3==1.20.26
79 | mock~=4.0.3
80 | pytest~=6.2.5
81 | pyyaml>=5.4.1
82 | schema~=0.7.5
83 | urllib3~=1.26.7
84 | [Container] 2022/02/21 17:29:18 Running command pip install -r adf-build/requirements.txt -q
85 | Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-nrpay2fb/MarkupSafe/
86 | You are using pip version 18.1, however version 22.0.3 is available.
87 | You should consider upgrading via the 'pip install --upgrade pip' command.
88 |  
89 | [Container] 2022/02/21 17:29:36 Command did not exit successfully pip install -r adf-build/requirements.txt -q exit status 1
90 | [Container] 2022/02/21 17:29:36 Phase complete: PRE_BUILD State: FAILED
91 | [Container] 2022/02/21 17:29:36 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: pip install -r adf-build/requirements.txt -q. Reason: exit status 1

[stemons]

I solved adding

- pip install --upgrade pip in the buildspec.yml of each pipeline before

pip install -r adf-build/requirements.txt -q

[AndreasAugustin]

Hi, we solved it with using another image then the default one:

build:
  provider: codebuild
  properties:
    image: STANDARD_5_0

the pip version is newer then on the standard one.