Make runlocal-rp with Container Image #3593

shubhadapaithankar · 2024-05-22T23:36:28Z

Which issue this PR addresses:

We created new Makefile targets to streamline the process of launching containers. This allowed developers to test locally more thoroughly, reducing development cycle time and decreasing failures rate of cluster creation process.

Fixes ARO-7435

What this PR does / why we need it:

This PR adds new Makefile targets for building, running, and testing the RP container image locally using Podman.
This enables local development and testing workflows

Test plan for issue:

Tested the new Makefile targets by running make runlocal-rp to ensure the container image builds and runs correctly.
Verified the RP container is accessible locally.

Steps for validating on Mac (need to validate on Linux):

Mount your local MacOS filesystem into the podman machine:
podman machine init --now --cpus=4 --memory=4096 -v $HOME:$HOME
Use the openvpn config file (which is now mounted inside the podman machine) to start the VPN connection:
podman machine ssh
sudo rpm-ostree install openvpn
sudo systemctl reboot
podman machine ssh
sudo openvpn --config /Users/<user_name>/go/src/github.com/Azure/ARO-RP/secrets/vpn-aks-westeurope.ovpn --daemon --writepid vpnpid
ps aux | grep openvpn

Source the environment file before creating the cluster using the setup_resources.sh script(Added the updated env in the PR)
. ./env
cd /hack
./setup_resources.sh
Once the cluster create verify connectivity with the ARO cluster:
Download the admin kubeconfig file
az aro get-admin-kubeconfig --name <cluster_name> --resource-group v4-westeurope --file ~/.kube/aro-admin-kubeconfig
Set the KUBECONFIG environment variable
export KUBECONFIG=~/.kube/aro-admin-kubeconfig
Verify connectivity with the ARO cluster
kubectl get nodes

Additional Changes:

Change in pkg/env/dev.go from return net.Listen("tcp", "localhost:8443") to return net.Listen("tcp", ":8443").
Adding the loopback IP for the host machine before the port export:
podman run --rm -p 127.0.0.1:8443:8443

Is there any documentation that needs to be updated for this PR?

N/A

How do you know this will function as expected in production?

Currently not a production change - this will be a parallel CI/CD effort that for now does not impact prod.

docs/deploy-full-rp-service-in-dev.md

setup_resources.sh

Makefile

github-actions · 2024-06-04T11:53:03Z

Please rebase pull request.

shubhadapaithankar · 2024-08-07T15:52:50Z

/azp run ci

azure-pipelines · 2024-08-07T15:53:00Z

Azure Pipelines successfully started running 1 pipeline(s).

env.example

Makefile

kimorris27 · 2024-08-08T20:32:13Z

/azp run ci

azure-pipelines · 2024-08-08T20:32:23Z

Azure Pipelines successfully started running 1 pipeline(s).

…image locally with Podman fix the code setup_resources.sh has been fixed makefile has been fixed fix the code removed unwanted files vnet create and delete code has been added Makefile has been fixed makefile modified added code to add dedicated vnet unwanted file deleted added some imp files fix the issue code has been added code has been fixed fix the code hive cluster creation process has been done modified the file Updated the Dockerfile to install and configure openssh-server and openssl, and modified the Makefile to build and run the container with services. Modified the Makefile to build and run the container with services. Modified the Makefile to build and run the container with services. setup file has been fixed env file updated fix makefile fix makefile fix makefile added ci-rp in target file has been fixed file has been fixed veriable requirment has been updated Modify the environment variable CLUSTER_RESOURCEGROUP to include the user's distinct identifier, making it unique to each user veriable requirment has been updated veriable requirment has been updated fix the separate variable to handle all the images removed the unwanted stuff from the files added the loopback address before the port export fix the openshift version fix makefile target removed the unwanted target from Makefile env.example

…ullspecs, and update instructions fix: Use RP_IMAGE_LOCAL for Linux compatibility in runlocal-rp target remove the changes veriable file modified move the set file to the hack directory and updated the doc move the set file to the hack directory and updated the doc Improve Podman Compatibility for Local RP with Secrets Handling Improve Podman compatibility by switching from 72626 to syntax for environment variables in Makefile env file updated Added logic to print the VNet/Subnets env file updated Update OpenShift version to 4.13.40 Update OpenShift version to 4.13.40 file added file added modified the makefile file has been added file has been added Makefile has been modifies Added the dynamic variable to fetch the default openshift version Update setup_resources.sh to dynamically fetch OpenShift version and pull specs from const.go

…ion to ensure accurate deployment configurations fixed the env

…mming minor version from OpenShift version file has been added

shubhadapaithankar · 2024-08-14T16:33:22Z

/azp run ci

azure-pipelines · 2024-08-14T16:33:31Z

Azure Pipelines successfully started running 1 pipeline(s).

SudoBrendan

changes LGTM - I trust others creating clusters successfully on it - thank you SO MUCH Shubhada! I think as a final item, we should send out a quick notice of this change to the team.

SudoBrendan

...Even better - I was able to successfully create a cluster and pass local E2E on it. Sweet!

bitoku · 2024-08-21T19:00:38Z

~~Because ci-rp image has some lint and test commands, I don't think it's suitable for local dev RP.~~
~~Everytime we make changes in RP, it always runs e2e tests, which is not something we want.~~

ARO-RP/Dockerfile.ci-rp

Lines 52 to 56 in 2ce4ec6

    
           # Lint, generate, build, and test 
        
           RUN golangci-lint run --verbose 
        
           RUN go generate ./... 
        
           RUN go build -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=${ARO_VERSION}" ./cmd/aro 
        
           RUN go test ./test/e2e/... -tags e2e,codec.safe -c -ldflags "-X github.com/Azure/ARO-RP/pkg/util/version.GitCommit=${ARO_VERSION}" -o e2e.test

~~I don't know the context of this PR, but why do we want this?~~

bitoku · 2024-08-21T19:23:30Z

sorry I misunderstood. never mind.

Runlocal-RP is Containerized - Modified Makefile to execute a local `podman run` for the RP on 127.0.0.1 - Local RPs now by default interact with Hive due to MacOS limitations - Updated RP dev config to serve on all IPs due to MacOS limitations - Doc updates

shubhadapaithankar added size-small Size small loki labels May 22, 2024

shubhadapaithankar requested review from jewzaam, bennerv, hawkowl, rogbas, petrkotas, jharrington22, cblecker, cadenmarchese, UlrichSchlueter, SudoBrendan, Shivkumar13, yjst2012, jaitaiwan, anshulvermapatel, hlipsig and tiguelu as code owners May 22, 2024 23:36

shubhadapaithankar marked this pull request as draft May 22, 2024 23:36

razo7 reviewed Jun 2, 2024

View reviewed changes

docs/deploy-full-rp-service-in-dev.md Outdated Show resolved Hide resolved

razo7 reviewed Jun 2, 2024

View reviewed changes

setup_resources.sh Outdated Show resolved Hide resolved

razo7 reviewed Jun 2, 2024

View reviewed changes

setup_resources.sh Outdated Show resolved Hide resolved

razo7 reviewed Jun 2, 2024

View reviewed changes

Makefile Outdated Show resolved Hide resolved

github-actions bot added the needs-rebase branch needs a rebase label Jun 4, 2024

shubhadapaithankar removed the needs-rebase branch needs a rebase label Jun 4, 2024

shubhadapaithankar marked this pull request as ready for review June 4, 2024 21:07

shubhadapaithankar marked this pull request as draft June 4, 2024 21:28

shubhadapaithankar force-pushed the shubhadapaithankar/ARO-7435 branch 2 times, most recently from 6078f54 to 91c82f1 Compare June 4, 2024 23:10

slawande2 previously approved these changes Aug 7, 2024

View reviewed changes

kimorris27 reviewed Aug 8, 2024

View reviewed changes

env.example Outdated Show resolved Hide resolved

Makefile Outdated Show resolved Hide resolved

shubhadapaithankar dismissed slawande2’s stale review via 36a0393 August 8, 2024 19:29

kimorris27 approved these changes Aug 8, 2024

View reviewed changes

shubhadapaithankar requested a review from kevinobriendotca August 8, 2024 21:03

slawande2 approved these changes Aug 8, 2024

View reviewed changes

kevinobriendotca approved these changes Aug 9, 2024

View reviewed changes

shubhadapaithankar added 5 commits August 14, 2024 08:48

Set INSTALLER_PULLSPEC dynamically using the extracted OpenShift vers…

1b12497

…ion to ensure accurate deployment configurations fixed the env

Fixed INSTALLER_PULLSPEC to use correct release version format by tri…

1046a8d

…mming minor version from OpenShift version file has been added

fixed the env

d948f49

shubhadapaithankar force-pushed the shubhadapaithankar/ARO-7435 branch from 36a0393 to d948f49 Compare August 14, 2024 15:48

kimorris27 approved these changes Aug 14, 2024

View reviewed changes

rajdeepc2792 approved these changes Aug 14, 2024

View reviewed changes

SudoBrendan approved these changes Aug 15, 2024

View reviewed changes

SudoBrendan approved these changes Aug 16, 2024

View reviewed changes

SudoBrendan merged commit a387947 into master Aug 16, 2024
33 checks passed

shubhadapaithankar deleted the shubhadapaithankar/ARO-7435 branch August 22, 2024 05:50

razo7 mentioned this pull request Sep 16, 2024

Fix env.example Default Values #3827

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make runlocal-rp with Container Image #3593

Make runlocal-rp with Container Image #3593

shubhadapaithankar commented May 22, 2024 •

edited

Loading

github-actions bot commented Jun 4, 2024

shubhadapaithankar commented Aug 7, 2024

azure-pipelines bot commented Aug 7, 2024

kimorris27 commented Aug 8, 2024

azure-pipelines bot commented Aug 8, 2024

shubhadapaithankar commented Aug 14, 2024

azure-pipelines bot commented Aug 14, 2024

SudoBrendan left a comment

SudoBrendan left a comment

bitoku commented Aug 21, 2024 •

edited

Loading

bitoku commented Aug 21, 2024

Make runlocal-rp with Container Image #3593

Make runlocal-rp with Container Image #3593

Conversation

shubhadapaithankar commented May 22, 2024 • edited Loading

Which issue this PR addresses:

What this PR does / why we need it:

Test plan for issue:

Additional Changes:

Is there any documentation that needs to be updated for this PR?

How do you know this will function as expected in production?

github-actions bot commented Jun 4, 2024

shubhadapaithankar commented Aug 7, 2024

azure-pipelines bot commented Aug 7, 2024

kimorris27 commented Aug 8, 2024

azure-pipelines bot commented Aug 8, 2024

shubhadapaithankar commented Aug 14, 2024

azure-pipelines bot commented Aug 14, 2024

SudoBrendan left a comment

Choose a reason for hiding this comment

SudoBrendan left a comment

Choose a reason for hiding this comment

bitoku commented Aug 21, 2024 • edited Loading

bitoku commented Aug 21, 2024

shubhadapaithankar commented May 22, 2024 •

edited

Loading

bitoku commented Aug 21, 2024 •

edited

Loading