New Application Connectivity tests - Test suite for Application Gateway

[mvshao]

Description
This task aims to implement separate Application Gateway tests into a separate minimal suite, written in Go.
The general idea is described here.

Acceptance criteria

• New project created under the following directory: https://github.com/kyma-project/kyma/tree/main/tests/components
• All authentication methods listed here covered.
• All test-cases listed here covered.
• Test is deployable on Kubernetes.
• Makefile compatible with pipeline

Related issue
Test's build pipeline: test-infra#5563
#13977

[VOID404]

Workplan

• Endpoints + AppCR entries (docs):
  • no authorisation
  • Basic Auth
  • OAuth protected with mTLS (please see Implement mtls oauth token fetch #14401)
    • (04.07 - problem with certificates)
  • OAuth (grant client credentials flow)
  • mTLS (a.k.a CertGen)
  • CSRF with:
    • no authorisation - Gateway doesn't support it
    • Basic Auth - Add the CSRF test to the App Gateway tests #14746
    • OAuth (grant client credentials flow)
    • OAuth protected with mTLS
    • mTLS
  • Expired scenarios
    • design
    • implement
• Test scenarios
  • Simple Optimistic
  • Simple bad auth
  • Path related:
    • Application name and service name not provided in the path
    • Service name missing in the path
    • Path segments added to the target path. Example: there is an API under some URL (e.g. 'my-cool-api.com') registered in Application CRD. The user calls App Gateway URL and specifies a specific endpoint in the target API (e.g. central-application-gateway.kyma-integration:8080///some/endpoint). As a result the following URL is called: my-cool-api.com/some/endpoint
  • Kubernetes resources related:
    • Application name specified in the path doesn't exist
    • Service name specified in the path doesn't exist
    • Secret containing credentials doesn't exist
  • Proxying related:
    • Response code from target service returned
    • Response body from target service returned
    • Call to target API timeouts
• Logging
  • in test-app
  • in tests
• Log formatting?
• Deployments
  • Dockerfile
  • Helm Charts
  • Makefile

[akgalwas]

Plan

• Prepare and merge initial PR with project structure. It should test BasicAuth with positive authorisation only. The goal is to deliver MVP and allow parallel work. Implement minimal Application Connector test #14743 Done @VOID404 @koala7659 @majakurcius
• Prepare pipeline for building Gateway tests @koala7659
  Adding new optional build job for application-connector-tests test-infra#5686
• Prepare pipeline for executing Gateway tests @koala7659
  New pipelines for testing application-connector components test-infra#5563
• Enhance test with path-related scenarios, Kubernetes resources tests and basic cases (e.g. call to unprotected endpoints) Add the test cases to the Application Gateway test suite #14791 @mvshao
• Implement Basic Auth with CSRF scenario - @franpog859 - PR
• Investigate problem with invoking mTLS protected endpoints @akgalwas
• Enhance test with proxying related tests @VOID404 Implement proxying test cases #14831
• Enhance test with the following authentication methods:
  • No auth
  • OAuth - @franpog859 - PR
  • OAuth with mTLS in review @akgalwas - PR
  • Fix problem with ignoring skipVerify setting PR in Application Gateway in review @akgalwas
  • mTLS @akgalwas - PR
• Consider adding different authentication methods with CSRF - @franpog859 - PR
• Enhance test with expired OAuth token scenarios - @VOID404 Test Gateway OAuth token renewal #14845
• Enhance tests with additional headers & query parameters - @franpog859 - PR
• Cover additional http verbs (POST, DELETE, PUT) other than GET. Check if body is properly forwarded. - @VOID404 Implement verbs tests #14867
• Readme In review @VOID404 @majakurcius Update README for Gateway tests #14907
• Add mTLS related info to the readme In progress @akgalwas
• Cleanup the tests: @mvshao
  • Implement minimal Application Connector test #14743 (comment)
  • Implement minimal Application Connector test #14743 (comment)
  • Implement minimal Application Connector test #14743 (comment)
  • Implement minimal Application Connector test #14743 (comment)
  • Implement minimal Application Connector test #14743 (comment) - first check if the wait command works with the Job resource
  • Implement minimal Application Connector test #14743 (comment)
  • Implement minimal Application Connector test #14743 (comment)
  • verify job creating multiple pods on test failures
• Improve error logs in Application Connector integration tests #14994 @mvshao
  • Improve the error logging in Application Connector integration tests #15022 - done

[Disper]

Gateway OAuth cache issue has been extracted to another issue.

[franpog859]

Everything is merged. Please, reopen it if something is missing