[Feature] Azure automated deployment for OPEA applications - Infosys #629

kkrishTa · 2024-12-06T12:47:04Z

Description

Infosys contribution to OPEA.
Azure AKS terraforms and steps to execute to deploy OPEA applications.

Issues

NA

Type of change

List the type of change like below. Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds new functionality)
Breaking change (fix or feature that would break existing design and interface)

Dependencies

None.

Tests

Tested terraforms on Azure platform.

arun-gupta

Thanks for the PR.

I did not try out the instructions but leaving comments based upon reading.

arun-gupta · 2024-12-07T04:52:24Z

cloud-service-provider/azure/aks/terraform/README.md

@@ -0,0 +1,85 @@
+# OPEA applications Azure ASK deployment guide
+
+This guide shows how to deploy OPEA applications on Azure Kubernetes Service (ASK) using Terraform.


ASK -> AKS

pre-commit bot has changed "AKS" -> "ASK" amd "aks" -> "ask".

I wondered about that consistent error!!

arun-gupta · 2024-12-07T04:52:36Z

cloud-service-provider/azure/aks/terraform/README.md

+
+## Prerequisites
+
+- Access to Azure ASK


arun-gupta · 2024-12-07T04:52:55Z

cloud-service-provider/azure/aks/terraform/README.md

+
+## Setup
+
+The setup uses Terraform to create ASK cluster with the following properties:


AKS - this is a global search and replace through out this doc

arun-gupta · 2024-12-07T04:53:29Z

cloud-service-provider/azure/aks/terraform/README.md

+The setup uses Terraform to create ASK cluster with the following properties:
+
+- 1-node ASK cluster with 50 GB disk and `Standard_D32d_v5` SPOT instance (16 vCPU and 32 GB memory)
+- Cluster autoscaling up to 10 nodes


Do we need this autoscaling to 10 nodes?

I think it is reasonable number.

arun-gupta · 2024-12-07T04:53:44Z

cloud-service-provider/azure/aks/terraform/README.md

+
+The setup uses Terraform to create ASK cluster with the following properties:
+
+- 1-node ASK cluster with 50 GB disk and `Standard_D32d_v5` SPOT instance (16 vCPU and 32 GB memory)


Do we need a SPOT instance? Can this be a standard virtual machine?

To support cost optimization, we have mentioned SPOT. Since instance is a variable, we will mention standard/SPOT in the readme file and mention about cost optimization in the variable file.

arun-gupta · 2024-12-07T04:56:00Z

cloud-service-provider/azure/aks/terraform/README.md

+- Cluster autoscaling up to 10 nodes
+- Storage Class (SC) `azfs-sc` and Persistent Volume Claim (PVC) `model-volume` for storing the model data
+- `LoadBalancer` address type for the service for external consumption
+- Updates the kubeconfig file for `kubectl` access


Not sure what needs to be done here?

arun-gupta · 2024-12-07T04:56:42Z

cloud-service-provider/azure/aks/terraform/README.md

+
+```bash
+kubectl get pod -n chatqna
+```


It would be useful to state that "ensure all pods are running or show 1/1 or something similar"

Sure Will add the statement.

poussa

LGTM. Just change the cluster name from opea-chatqna to opea.

poussa · 2024-12-09T09:29:03Z

cloud-service-provider/azure/aks/terraform/opea-chatqna.tfvars

@@ -0,0 +1,6 @@
+cluster_name = "opea-chatqna"


let's use cluster name opea. The AWS/EKS cluster will change to use that name as well.

Updated cluster name to opea

kkrishTa · 2024-12-10T06:05:20Z

Hello @poussa ,
Review comments are fixed. Could you please approve and merge the code?

poussa · 2024-12-10T07:20:56Z

Hello @poussa , Review comments are fixed. Could you please approve and merge the code?

I have already approved. We need a 2nd approver. I'll work on that now.

poussa · 2024-12-10T07:22:26Z

@kkrishTa can you also fix the DCO. That is, add signed-off-by line to the commit (-s).

kkrishTa · 2024-12-10T07:34:17Z

@kkrishTa can you also fix the DCO. That is, add signed-off-by line to the commit (-s).

@poussa Sure. Fixed this one.

kkrishTa · 2024-12-10T09:00:53Z

Hello @poussa / @lianhao ,
The pipeline got failed with an error:
/home/runner/work/GenAIInfra/GenAIInfra/docs/_build/rst/GenAIInfra/cloud-service-provider/azure/aks/terraform/README.md: WARNING: document isn't included in any toctree

But I see that ".md files not in toc tree" is ignored as per https://github.com/opea-project/docs/blob/main/.known-issues/sphinx.conf. Though this is a warning, the pipeline step has failed.

Could you please advise fix on this?

lianhao · 2024-12-10T09:09:32Z

Could you please advise fix on this?

@poussa will create another PR to add the doc link to those exiting md files under cloud-service-provider directory, then you can add the link follow his convention.

It's a newly added doc CI check complaining.

poussa · 2024-12-10T10:00:18Z

@kkrishTa once this PR#633 is merged, you can update yours to add link to Azure/AKS readme. Then the ci/cd is happy and we can merge.

Signed-off-by: Lianhao Lu <[email protected]> Signed-off-by: kkrishTa <[email protected]>

Signed-off-by: kkrishTa <[email protected]>

for more information, see https://pre-commit.ci Signed-off-by: kkrishTa <[email protected]>

Signed-off-by: kkrishTa <[email protected]>

for more information, see https://pre-commit.ci Signed-off-by: kkrishTa <[email protected]>

Signed-off-by: kkrishTa <[email protected]>

* helm: Add service account support in common services 1. Add service account creation support, disabled by default. 2. Add support of sharing the same service account by setting global.sharedSAName, disabled by default. Signed-off-by: Lianhao Lu <[email protected]> * helm: Add service account support in e2e charts 1. Add service account creation support, enabled by default. 2. Add support of sharing the same service account by setting global.sharedSAName, enabled by default. Signed-off-by: Lianhao Lu <[email protected]> --------- Signed-off-by: Lianhao Lu <[email protected]> Signed-off-by: kkrishTa <[email protected]>

* README: add links to terraform docs Signed-off-by: Sakari Poussa <[email protected]> * README: fix broken links Signed-off-by: Sakari Poussa <[email protected]> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Sakari Poussa <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: kkrishTa <[email protected]>

Signed-off-by: kkrishTa <[email protected]>

poussa · 2024-12-10T15:59:37Z

@kkrishTa can you resolve the conflict, please?

poussa · 2024-12-10T17:13:02Z

The readme filename is README.md not README.MD as it is in the link now. That's why ci/cd is still failing.

One more fix and we are there...

Signed-off-by: kkrishTa <[email protected]>

lianhao · 2024-12-11T02:19:17Z

The doc team is investigating why the Check Online Document Building CI fails now. Let's hold on this until they figure out the solution.

mkbhanda

LGTM

mkbhanda · 2024-12-10T12:25:35Z

cloud-service-provider/azure/aks/terraform/README.md

@@ -0,0 +1,85 @@
+# OPEA applications Azure ASK deployment guide
+
+This guide shows how to deploy OPEA applications on Azure Kubernetes Service (ASK) using Terraform.


I wondered about that consistent error!!

mkbhanda · 2024-12-11T05:34:46Z

Let me find someone on the docs team to help on this @kkrishTa. You have been most patient with our CICD tests.

lianhao · 2024-12-11T07:22:44Z

@kkrishTa The doc teams has resolved the doc build failure. For the Check Paths and Hyperlinks / check-the-validity-of-hyperlinks-in-README issue, could you please use relative link instead, just like PR #634

kkrishTa · 2024-12-11T07:38:30Z

@kkrishTa The doc teams has revolved the doc build failure. For the Check Paths and Hyperlinks / check-the-validity-of-hyperlinks-in-README issue, could you please use relative link instead, just like PR #634

sure @lianhao . Once #634 is merged, I will pull the latest and update the link apporpriately.

lianhao · 2024-12-11T07:39:49Z

@kkrishTa The doc teams has revolved the doc build failure. For the Check Paths and Hyperlinks / check-the-validity-of-hyperlinks-in-README issue, could you please use relative link instead, just like PR #634

sure @lianhao . Once this is merged, I will pull the latest and update the link apporpriately.

No need to wait for PR #634, just go ahead to fix your PR by using relative links to newly added .md files. Let's get your PR to land-in first.

Signed-off-by: kkrishTa <[email protected]>

kkrishTa · 2024-12-11T07:50:37Z

@kkrishTa The doc teams has revolved the doc build failure. For the Check Paths and Hyperlinks / check-the-validity-of-hyperlinks-in-README issue, could you please use relative link instead, just like PR #634

sure @lianhao . Once this is merged, I will pull the latest and update the link apporpriately.

No need to wait for PR #634, just go ahead to fix your PR by using relative links to newly added .md files. Let's get your PR to land-in first.

Updated readme. Since it has common read me, upadted other relative links as in #634 without breaking AWS readme link.

kkrishTa requested review from daisy-ycguo and mkbhanda as code owners December 6, 2024 12:47

kkrishTa changed the title ~~AKS Deployment for OPEA applications - Infosys~~ AKS automated deployment for OPEA applications - Infosys Dec 6, 2024

kkrishTa changed the title ~~AKS automated deployment for OPEA applications - Infosys~~ Azure automated deployment for OPEA applications - Infosys Dec 6, 2024

arun-gupta suggested changes Dec 7, 2024

View reviewed changes

lianhao requested review from poussa and removed request for daisy-ycguo December 9, 2024 00:57

poussa approved these changes Dec 9, 2024

View reviewed changes

poussa reviewed Dec 9, 2024

View reviewed changes

poussa requested a review from lianhao December 10, 2024 07:19

kkrishTa force-pushed the main branch from be2f386 to 88c68e3 Compare December 10, 2024 07:30

lianhao approved these changes Dec 10, 2024

View reviewed changes

lianhao and others added 11 commits December 10, 2024 21:23

CI: dump all containers' log for chart test (#627)

400b36e

Signed-off-by: Lianhao Lu <[email protected]> Signed-off-by: kkrishTa <[email protected]>

AKS Deployment for OPEA applications

501aeb2

Signed-off-by: kkrishTa <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

749f532

for more information, see https://pre-commit.ci Signed-off-by: kkrishTa <[email protected]>

Reverted changes by pre commit hook. Updated readme as per suggestions

511a3c5

Signed-off-by: kkrishTa <[email protected]>

Adding code spell ignore for Azure Kubernetes Service

85f0393

Signed-off-by: kkrishTa <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

cbebbc4

for more information, see https://pre-commit.ci Signed-off-by: kkrishTa <[email protected]>

Adding code spell ignore for AKS and reverting pre commit changes

12452a5

Signed-off-by: kkrishTa <[email protected]>

Updating cluster name to opea as a generic name

25a8763

Signed-off-by: kkrishTa <[email protected]>

Adding link to the Azure terraform readme to opea documentation

56cab37

Signed-off-by: kkrishTa <[email protected]>

kkrishTa force-pushed the main branch from 7b5254e to 56cab37 Compare December 10, 2024 15:54

kkrishTa requested a review from yongfengdu as a code owner December 10, 2024 15:54

Merge branch 'main' into main

4f9ae01

kkrishTa added 2 commits December 11, 2024 05:55

Corrected md file link

5312c56

Signed-off-by: kkrishTa <[email protected]>

Merge branch 'main' of https://github.com/kkrishTa/GenAIInfra

14a7dec

mkbhanda approved these changes Dec 11, 2024

View reviewed changes

Merge branch 'opea-project:main' into main

8f4dbc9

Updated relative links

69ca8d0

Signed-off-by: kkrishTa <[email protected]>

NeoZhangJianyu approved these changes Dec 11, 2024

View reviewed changes

mkbhanda merged commit e9dc58a into opea-project:main Dec 11, 2024
7 checks passed

lianhao mentioned this pull request Dec 11, 2024

Use relative link path for READMEs within the repo #634

Closed

1 task

joshuayao added this to the v1.2 milestone Dec 11, 2024

joshuayao added the feature New feature or request label Dec 15, 2024

joshuayao changed the title ~~Azure automated deployment for OPEA applications - Infosys~~ [Feature] Azure automated deployment for OPEA applications - Infosys Dec 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Azure automated deployment for OPEA applications - Infosys #629

[Feature] Azure automated deployment for OPEA applications - Infosys #629

kkrishTa commented Dec 6, 2024

arun-gupta left a comment

arun-gupta Dec 7, 2024

kkrishTa Dec 7, 2024

mkbhanda Dec 10, 2024

arun-gupta Dec 7, 2024

arun-gupta Dec 7, 2024

arun-gupta Dec 7, 2024

poussa Dec 9, 2024

arun-gupta Dec 7, 2024

kkrishTa Dec 7, 2024

arun-gupta Dec 7, 2024

arun-gupta Dec 7, 2024

kkrishTa Dec 7, 2024

poussa left a comment

poussa Dec 9, 2024

kkrishTa Dec 9, 2024

kkrishTa commented Dec 10, 2024

poussa commented Dec 10, 2024

poussa commented Dec 10, 2024

kkrishTa commented Dec 10, 2024

kkrishTa commented Dec 10, 2024

lianhao commented Dec 10, 2024 •

edited

Loading

poussa commented Dec 10, 2024

poussa commented Dec 10, 2024

poussa commented Dec 10, 2024

lianhao commented Dec 11, 2024

mkbhanda left a comment

mkbhanda Dec 10, 2024

mkbhanda commented Dec 11, 2024

lianhao commented Dec 11, 2024 •

edited

Loading

kkrishTa commented Dec 11, 2024 •

edited

Loading

lianhao commented Dec 11, 2024 •

edited

Loading

kkrishTa commented Dec 11, 2024

		@@ -0,0 +1,85 @@
		# OPEA applications Azure ASK deployment guide

		This guide shows how to deploy OPEA applications on Azure Kubernetes Service (ASK) using Terraform.


		## Setup

		The setup uses Terraform to create ASK cluster with the following properties:


		The setup uses Terraform to create ASK cluster with the following properties:

		- 1-node ASK cluster with 50 GB disk and `Standard_D32d_v5` SPOT instance (16 vCPU and 32 GB memory)

[Feature] Azure automated deployment for OPEA applications - Infosys #629

[Feature] Azure automated deployment for OPEA applications - Infosys #629

Conversation

kkrishTa commented Dec 6, 2024

Description

Issues

Type of change

Dependencies

Tests

arun-gupta left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

poussa left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kkrishTa commented Dec 10, 2024

poussa commented Dec 10, 2024

poussa commented Dec 10, 2024

kkrishTa commented Dec 10, 2024

kkrishTa commented Dec 10, 2024

lianhao commented Dec 10, 2024 • edited Loading

poussa commented Dec 10, 2024

poussa commented Dec 10, 2024

poussa commented Dec 10, 2024

lianhao commented Dec 11, 2024

mkbhanda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mkbhanda commented Dec 11, 2024

lianhao commented Dec 11, 2024 • edited Loading

kkrishTa commented Dec 11, 2024 • edited Loading

lianhao commented Dec 11, 2024 • edited Loading

kkrishTa commented Dec 11, 2024

lianhao commented Dec 10, 2024 •

edited

Loading

lianhao commented Dec 11, 2024 •

edited

Loading

kkrishTa commented Dec 11, 2024 •

edited

Loading

lianhao commented Dec 11, 2024 •

edited

Loading