Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cherry-pick]Add notation accessory copy test case #19606

Merged
merged 1 commit into from
Nov 22, 2023
Merged

[cherry-pick]Add notation accessory copy test case #19606

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions tests/resources/Harbor-Pages/Project.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -375,13 +375,29 @@ Back Project Home
[Arguments] ${project_name}
Retry Link Click //a[contains(.,'${project_name}')]

Should Not Be Signed By Cosign
Should Be Signed
[Arguments] ${tag}
Retry Wait Element Visible //clr-dg-row[contains(.,'${tag}')]//clr-icon[contains(@class,'signed')]

Should Not Be Signed
[Arguments] ${tag}
Retry Wait Element Visible //clr-dg-row[contains(.,'${tag}')]//clr-icon[contains(@class,'color-red')]

Should Be Signed By Cosign
[Arguments] ${tag}
Retry Wait Element Visible //clr-dg-row[contains(.,'${tag}')]//clr-icon[contains(@class,'signed')]
[Arguments] ${tag}=${null} ${digest}=${null}
IF '${tag}' != '${null}'
Retry Wait Element Visible //clr-dg-row[./clr-expandable-animation/div/div/div/clr-dg-cell/div/clr-tooltip/div/div/span[contains(.,'${tag}')] and .//clr-dg-row[.//img[@title='signature.cosign']]]
ELSE
Retry Wait Element Visible //clr-dg-row[./clr-expandable-animation/div/div/div/clr-dg-cell/div/a[contains(.,'${digest}')] and .//clr-dg-row[.//img[@title='signature.cosign']]]
END

Should Be Signed By Notation
[Arguments] ${tag}=${null} ${digest}=${null}
IF '${tag}' != '${null}'
Retry Wait Element Visible //clr-dg-row[./clr-expandable-animation/div/div/div/clr-dg-cell/div/clr-tooltip/div/div/span[contains(.,'${tag}')] and .//clr-dg-row[.//img[@title='signature.notation']]]
ELSE
Retry Wait Element Visible //clr-dg-row[./clr-expandable-animation/div/div/div/clr-dg-cell/div/a[contains(.,'${digest}')] and .//clr-dg-row[.//img[@title='signature.notation']]]
END

Delete Accessory
[Arguments] ${tag}
Expand Down
26 changes: 26 additions & 0 deletions tests/resources/Notation_Util.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Copyright Project Harbor Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License

*** Settings ***
Documentation This resource provides helper functions for docker operations
Library OperatingSystem
Library Process

*** Keywords ***
Notation Generate Cert
Run And Return Rc And Output notation cert generate-test --default wabbit-networks.io

Notation Sign
[Arguments] ${artifact}
Wait Unitl Command Success notation sign -d --allow-referrers-api ${artifact}
1 change: 1 addition & 0 deletions tests/resources/Util.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ Resource SeleniumUtil.robot
Resource Nightly-Util.robot
Resource APITest-Util.robot
Resource Cosign_Util.robot
Resource Notation_Util.robot
Resource Imgpkg-Util.robot
Resource Webhook-Util.robot
Resource TestCaseBody.robot
Expand Down
17 changes: 13 additions & 4 deletions tests/robot-cases/Group1-Nightly/Common.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -474,19 +474,28 @@ Test Case - Copy A Image And Accessory
Create An New Project And Go Into Project ${source_project}

Push Image With Tag ${ip} ${user} ${pwd} ${source_project} ${image} ${tag}
Cosign Generate Key Pair
Docker Login ${ip} ${user} ${pwd}
Cosign Generate Key Pair
Cosign Sign ${ip}/${source_project}/${image}:${tag}
Docker Logout ${ip}
Notation Generate Cert
Notation Sign ${ip}/${source_project}/${image}:${tag}

Go Into Repo ${source_project} ${image}
Should Be Signed ${tag}
Retry Button Click ${artifact_list_accessory_btn}
Should Be Signed By Cosign ${tag}
Should Be Signed By Notation ${tag}

Copy Image ${tag} ${target_project} ${image}

Retry Double Keywords When Error Go Into Project ${target_project} Retry Wait Until Page Contains ${image}
Go Into Repo ${target_project} ${image}
Retry Wait Until Page Contains Element //clr-dg-row[contains(.,${tag})]
Should Be Signed ${tag}
Retry Button Click ${artifact_list_accessory_btn}
Should Be Signed By Cosign ${tag}
Should Be Signed By Notation ${tag}
Docker Logout ${ip}
Close Browser

Test Case - Create An New Project With Quotas Set
Expand Down Expand Up @@ -772,14 +781,14 @@ Test Case - Cosign And Cosign Deployment Security Policy
Push Image With Tag ${ip} ${user} ${pwd} project${d} ${image} ${tag}
Go Into Project project${d}
Go Into Repo project${d} ${image}
Should Not Be Signed By Cosign ${tag}
Should Not Be Signed ${tag}
Cannot Pull Image ${ip} ${user} ${pwd} project${d} ${image}:${tag} err_msg=The image is not signed by cosign.
Cosign Generate Key Pair
Cosign Verify ${ip}/project${d}/${image}:${tag} ${false}

Cosign Sign ${ip}/project${d}/${image}:${tag}
Cosign Verify ${ip}/project${d}/${image}:${tag} ${true}
Retry Double Keywords When Error Retry Element Click ${artifact_list_refresh_btn} Should Be Signed By Cosign ${tag}
Retry Double Keywords When Error Retry Element Click ${artifact_list_refresh_btn} Should Be Signed ${tag}
Pull image ${ip} ${user} ${pwd} project${d} ${image}:${tag}

Retry Double Keywords When Error Delete Accessory ${tag} Should be Accessory deleted ${tag}
Expand Down
72 changes: 36 additions & 36 deletions tests/robot-cases/Group1-Nightly/Replication.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -389,16 +389,16 @@ Test Case - Robot Account Do Replication
Logout Harbor
Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Image Should Be Replicated To Project project_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Should Be Signed ${tag1}
Image Should Be Replicated To Project project_dest${d} ${image2}
Should Be Signed By Cosign ${tag2}
Should Be Signed ${tag2}
Back Project Home project_dest${d}
Go Into Repo project_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Should Be Signed ${index_tag}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image1_short_sha256}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed By Cosign ${image2_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed ${image2_short_sha256}
# pull mode
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Expand All @@ -409,16 +409,16 @@ Test Case - Robot Account Do Replication
Check Latest Replication Job Status Succeeded
Check Latest Replication Enabled Copy By Chunk
Image Should Be Replicated To Project project_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Should Be Signed ${tag1}
Image Should Be Replicated To Project project_dest${d} ${image2}
Should Be Signed By Cosign ${tag2}
Should Be Signed ${tag2}
Back Project Home project_dest${d}
Go Into Repo project_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Should Be Signed ${index_tag}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image1_short_sha256}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed By Cosign ${image2_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed ${image2_short_sha256}
Close Browser

Test Case - Replication Triggered By Events
Expand Down Expand Up @@ -468,28 +468,28 @@ Test Case - Replication Triggered By Events
Logout Harbor
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Repo project${d} ${image1}
Should Be Signed By Cosign ${tag1}
Should Be Signed ${tag1}
Go Into Repo project${d} ${index}
Should Be Signed By Cosign ${index_tag}
Should Be Signed ${index_tag}
Go Into Repo project${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image1_short_sha256}
Go Into Repo project${d} ${image2}
Should Not Be Signed By Cosign ${tag2}
Should Not Be Signed ${tag2}
Go Into Repo project${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed By Cosign ${image2_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed ${image2_short_sha256}
Logout Harbor

Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Repo project_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Should Be Signed ${tag1}
Go Into Repo project_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Should Be Signed ${index_tag}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image1_short_sha256}
Go Into Repo project_dest${d} ${image2}
Should Not Be Signed By Cosign ${tag2}
Should Not Be Signed ${tag2}
Go Into Repo project_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed By Cosign ${image2_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Not Be Signed ${image2_short_sha256}
Logout Harbor
# delete
Sign In Harbor ${HARBOR_URL} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Expand All @@ -498,26 +498,26 @@ Test Case - Replication Triggered By Events
Repo Not Exist project${d} ${image2}
Go Into Repo project${d} ${image1}
Retry Double Keywords When Error Delete Accessory ${tag1} Should be Accessory deleted ${tag1}
Should Not Be Signed By Cosign ${tag1}
Should Not Be Signed ${tag1}
Go Into Repo project${d} ${index}
Retry Double Keywords When Error Delete Accessory ${index_tag} Should be Accessory deleted ${index_tag}
Should Not Be Signed By Cosign ${index_tag}
Should Not Be Signed ${index_tag}
Click Index Achieve ${index_tag}
Retry Double Keywords When Error Delete Accessory ${image1_short_sha256} Should be Accessory deleted ${image1_short_sha256}
Should Not Be Signed By Cosign ${image1_short_sha256}
Should Not Be Signed ${image1_short_sha256}
Logout Harbor

Sign In Harbor https://${ip1} ${HARBOR_ADMIN} ${HARBOR_PASSWORD}
Go Into Repo project_dest${d} ${image2}
Wait Until Page Contains We couldn't find any artifacts!
Go Into Repo project_dest${d} ${image1}
Should be Accessory deleted ${tag1}
Should Not Be Signed By Cosign ${tag1}
Should Not Be Signed ${tag1}
Go Into Repo project_dest${d} ${index}
Should be Accessory deleted ${index_tag}
Should Not Be Signed By Cosign ${index_tag}
Should Not Be Signed ${index_tag}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should be Accessory deleted ${image1_short_sha256}
Should Not Be Signed By Cosign ${image1_short_sha256}
Should Not Be Signed ${image1_short_sha256}
Close Browser

Test Case - Enable Replication Of Cosign Deployment Security Policy
Expand Down Expand Up @@ -595,31 +595,31 @@ Test Case - Enable Replication Of Cosign Deployment Security Policy
Repo Exist project_pull_dest${d} ${image2}
Repo Exist project_pull_dest${d} ${index}
Go Into Repo project_pull_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Should Be Signed ${tag1}
Go Into Repo project_pull_dest${d} ${image2}
Should Be Signed By Cosign ${tag2}
Should Be Signed ${tag2}
Go Into Repo project_pull_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Should Be Signed ${index_tag}
Go Into Repo project_pull_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image1_short_sha256}
Go Into Repo project_pull_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image2_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image2_short_sha256}
# check project_push_dest
Go Into Project project_push_dest${d}
Switch To Project Repo
Repo Exist project_push_dest${d} ${image1}
Repo Exist project_push_dest${d} ${image2}
Repo Exist project_push_dest${d} ${index}
Go Into Repo project_push_dest${d} ${image1}
Should Be Signed By Cosign ${tag1}
Should Be Signed ${tag1}
Go Into Repo project_push_dest${d} ${image2}
Should Be Signed By Cosign ${tag2}
Should Be Signed ${tag2}
Go Into Repo project_push_dest${d} ${index}
Should Be Signed By Cosign ${index_tag}
Should Be Signed ${index_tag}
Go Into Repo project_push_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image1_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image1_short_sha256}
Go Into Repo project_push_dest${d} ${index}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed By Cosign ${image2_short_sha256}
Retry Double Keywords When Error Click Index Achieve ${index_tag} Should Be Signed ${image2_short_sha256}
Close Browser

Test Case - Carvel Imgpkg Copy To Harbor
Expand Down
Loading