Observability AI Assistant Tests Deployment Agnostic #205194

arturoliduena · 2024-12-27T11:29:53Z

Closes #192718

Summary

This PR add a deployment-agnostic testing environment for Observability AI Assistant tests by unifying the duplicated tests for stateful and serverless environments. It create the ObservabilityAIAssistantApiClient to work seamlessly in both environments, enabling a single test to run across stateful, CI, and MKI.

Initial efforts focus on deduplicating the conversations.spec.ts and connectors.spec.ts files, as these already run in all environments.

Move / dedup the tests that exist in stateful and serverless. They run in serverless CI but not MKI and add the skipMki tag.
chat.spec.ts
complete.spec.ts
elasticsearch.spec.ts
public_complete.spec.ts
alerts.spec.ts
knowledge_base_setup.spec.ts
knowledge_base_status.spec.ts
knowledge_base.spec.ts
summarize.ts
knowledge_base_user_instructions.spec.ts

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

elasticmachine · 2024-12-27T11:29:55Z

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

github-actions · 2024-12-27T12:05:44Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

sorenlouv · 2025-01-02T14:43:14Z

...ntegration/deployment_agnostic/apis/observability/ai_assistant/connectors/connectors.spec.ts

+
+    it('Returns a 2xx for enterprise license', async () => {
+      const { status } = await observabilityAIAssistantAPIClient.editor({
+        endpoint: `GET ${CONNECTOR_API_URL}`,


nit: I suggest not using variables here. It's harder to read and search for. Also, I'm not sure the typescript intellisense for parameters and typed responses works when using variables

Suggested change

endpoint: `GET ${CONNECTOR_API_URL}`,

endpoint: `GET /internal/observability_ai_assistant/connectors`,

If you really want to make it DRY I suggest adding a helper and re-use that instead:

const getConnectors = () => observabilityAIAssistantAPIClient.editor({ endpoint: `GET /internal/observability_ai_assistant/connectors`, });

Agree. Now that I’m moving many of these tests, I won’t use a variable and will ensure they follow a consistent pattern since we currently have a mix.

sorenlouv · 2025-01-02T14:52:39Z

...ntegration/deployment_agnostic/apis/observability/ai_assistant/connectors/connectors.spec.ts

+  }
+}
+
+async function createProxyActionConnector({


Should we have a shared helper for creating the proxy action connector? It's already defined at least 3 times:

kibana/x-pack/test/observability_ai_assistant_api_integration/common/action_connectors.ts

Line 31 in c4cf9fe

export async function createProxyActionConnector({

kibana/x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/common/action_connectors.ts

Line 40 in c4cf9fe

export async function createProxyActionConnector({

kibana/x-pack/test/observability_ai_assistant_functional/common/connectors.ts

Line 10 in c4cf9fe

export async function createConnector(proxy: LlmProxy, supertest: SuperTestAgent) {

I will keep only one, I will get rid of the duplicate functions.

sorenlouv · 2025-01-02T14:56:14Z

...tion/deployment_agnostic/apis/observability/ai_assistant/conversations/conversations.spec.ts

-        await observabilityAIAssistantAPIClient
-          .slsEditor({
+        try {
+          await observabilityAIAssistantAPIClient.editor({


Feels good getting rid of the serverless-specific client. Is the plan to get rid of x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/common/observability_ai_assistant_api_client.ts entirely?

We can remove it entirely if we get all the tests running in deployment agnostic, unless there is some case where we want a serverless only test because of some serverless only feature. Probably wouldn't hurt to leave it.

...tion/deployment_agnostic/apis/observability/ai_assistant/conversations/conversations.spec.ts

neptunian · 2025-01-02T19:00:24Z

...ck/test/observability_ai_assistant_api_integration/tests/conversations/conversations.spec.ts

@@ -46,212 +46,6 @@ export default function ApiTest({ getService }: FtrProviderContext) {
  });

  describe('Conversations', () => {
-    describe('without conversations', () => {


This whole file should be safe to move over to deployment agnostic and not exist in stateful tests. I'm assuming you decided not to move it was because of the security roles and access privileges test suite which doesn't exist in serverless, but it soon will: #205210

Exactly. I only moved the common tests between stateful and serverless. I’ll move the security roles and access privaleges to deployment-agnostic and remove it from stateful and serverless.

neptunian · 2025-01-02T19:01:29Z

x-pack/test/observability_ai_assistant_api_integration/tests/connectors/connectors.spec.ts

@@ -8,13 +8,11 @@
 import expect from '@kbn/expect';
 import type { Agent as SuperTestAgent } from 'supertest';


Same as https://github.com/elastic/kibana/pull/205194/files#r1901174829

sorenlouv · 2025-01-03T10:23:14Z

.github/CODEOWNERS

@@ -1264,7 +1264,7 @@ packages/kbn-monaco/src/esql @elastic/kibana-esql
 /x-pack/test/observability_ai_assistant_functional @elastic/obs-ai-assistant
 /x-pack/test_serverless/**/test_suites/observability/ai_assistant @elastic/obs-ai-assistant
 /x-pack/test/functional/es_archives/observability/ai_assistant @elastic/obs-ai-assistant
-
+/x-pack/test/api_integration/deployment_agnostic/apis/observability/ai_assistant @elastic/obs-ai-assistant


Doesn't look related to your change but Github complains about errors in this file

It's valid on main which is odd

Thanks for pointing that out, @sorenlouv. I checked, and the file is now valid in this branch as well. It might have been a temporary issue or something resolved during recent changes(I did a rebase recently).

sorenlouv

Just one comment about CODEOWNERS file

...tion/deployment_agnostic/apis/observability/ai_assistant/conversations/conversations.spec.ts

sorenlouv · 2025-01-06T12:31:31Z

x-pack/test/api_integration/deployment_agnostic/services/observability_ai_assistant_api.ts

@@ -201,6 +201,7 @@ export function ObservabilityAIAssistantApiProvider(context: DeploymentAgnosticF
    admin: observabilityAIAssistantApiClient.makeInternalApiRequest('admin'),
    viewer: observabilityAIAssistantApiClient.makeInternalApiRequest('viewer'),
    editor: observabilityAIAssistantApiClient.makeInternalApiRequest('editor'),
+    unauthorizedUser: observabilityAIAssistantApiClient.makeInternalApiRequest('viewer'),


Can't we simply use the existing viewer? Also, what do you expect to happen if unauthorizedUser is used to call an endpoint that viewer is allowed to access?

Hey @arturoliduena
You might not be able to use the viewer role for unauthorizedUser in this manner for DA tests. It was done for serverless this way, because we force the user to be unauthorized by not passing the cookie header.

https://github.com/elastic/kibana/pull/205210/files#diff-febc7c89d3de714505db725fb9c264974eea6a5b29542b8782ebb0bab98a92d7R219

However, since the cookie is always included in makeInternalApiRequest, this user will be a viewer with authorization.

Hey @sorenlouv and @viduni94,
We can simulate an unauthorized user by omitting credentials entirely with a function like makeInternalApiRequestWithoutCredentials:

function makeInternalApiRequest(role: string) { return async <TEndpoint extends InternalEndpoint<APIEndpoint>>( options: Options<TEndpoint> ): Promise<SupertestReturnType<TEndpoint>> => { const headers: Record<string, string> = { ...samlAuth.getInternalRequestHeader(), ...(await samlAuth.getM2MApiCookieCredentialsWithRoleScope(role)), }; return makeApiRequest({ options, headers, }); }; } function makeInternalApiRequestWithoutCredentials() { return async <TEndpoint extends InternalEndpoint<APIEndpoint>>( options: Options<TEndpoint> ): Promise<SupertestReturnType<TEndpoint>> => { const headers: Record<string, string> = { ...samlAuth.getInternalRequestHeader(), }; return makeApiRequest({ options, headers, }); }; }

However, the problem with this approach is that omitting credentials will result in a 401 Unauthorized response instead of a 403 Forbidden. This doesn’t align with the intention of the test, which is to verify whether a user with insufficient permissions can access specific endpoints (DELETE, POST, PUT).

To properly simulate this, the user needs to be authenticated (avoiding 401 Unauthorized) but lack the necessary permissions, resulting in a 403 Forbidden. This is why I think it makes sense to use the viewer role here.

I agree with Søren that adding an additional unauthorizedUser role like this:

unauthorizedUser: observabilityAIAssistantApiClient.makeInternalApiRequest('viewer')

is unnecessary. Instead, we can simply use the existing viewer role:

viewer: observabilityAIAssistantApiClient.makeInternalApiRequest('viewer')

This keeps the implementation clean and ensures that we’re testing for insufficient permissions correctly. Let me know if you agree

If I read that correctly, you'll authenticate as the viewer user. That sounds good to me.

What I meant is not to have a user without credentials, but to have a user with a role that has insufficient privileges to access the AI Assistant APIs.

If the viewer role fits that criteria, I'm good with this.

It just seems odd to have a user called unauthorizedUser. Because even with just the viewer role there are APIs they are authorised to access.

sorenlouv · 2025-01-07T09:46:50Z

x-pack/test/api_integration/deployment_agnostic/services/observability_ai_assistant_api.ts

+
+type PublicEndpoint<T extends APIEndpoint> = T extends `${string} /api/${string}` ? T : never;
+
+function createObservabilityAIAssistantApiClient({


fyi: @dgieselaar is extracting the API client in https://github.com/elastic/kibana/pull/204309/files#diff-cb59199a241f60514e9ba52c57b587929f6be6bcedb24c5a138d5ba72184fe85.

I don't think this is a blocker but eventually we should use that implementation so we don't have multiple. Afaict we now have three API clients:

https://github.com/elastic/kibana/blob/main/x-pack/test/observability_ai_assistant_api_integration/common/observability_ai_assistant_api_client.ts

https://github.com/elastic/kibana/blob/main/x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/common/observability_ai_assistant_api_client.ts

neptunian · 2025-01-07T14:32:01Z

...k/test/api_integration/deployment_agnostic/apis/observability/ai_assistant/chat/chat.spec.ts

-  describe('/internal/observability_ai_assistant/chat', () => {
+  describe('/internal/observability_ai_assistant/chat', function () {
+    // TODO: https://github.com/elastic/kibana/issues/192751
+    this.tags(['skipMKI']);


We can be replace skipMKI with failsOnMKI if we want to be more descriptive that we aren't wanting to skip but that its failing so we have to skip.

…s test suite (#205631) Closes #205537 ## Summary The knowledge base migration test suite is missing in serverless. This PR adds it to the serverless test suite. - This has a dependancy to #205194 since we are removing all serverless tests and adding them to DA tests. - If the DA tests PR gets merged first, I'll refactor this PR to add it there. ### Checklist - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

…s test suite (elastic#205631) Closes elastic#205537 ## Summary The knowledge base migration test suite is missing in serverless. This PR adds it to the serverless test suite. - This has a dependancy to elastic#205194 since we are removing all serverless tests and adding them to DA tests. - If the DA tests PR gets merged first, I'll refactor this PR to add it there. ### Checklist - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) (cherry picked from commit e32ff8e)

arturoliduena · 2025-01-08T13:50:26Z

@neptunian, @sorenlouv and @viduni94
Here’s an update:
After enabling the ml test service, I continued with the migration of the knowledge_base tests.

kibana/x-pack/test/api_integration/deployment_agnostic/services/deployment_agnostic_services.ts

Line 15 in e04b200

export const deploymentAgnosticServices = _.pick(apiIntegrationServices, [

Migrated to Deployment-Agnostic:

conversations.spec.ts
connectors.spec.ts
chat.spec.ts
complete.spec.ts
elasticsearch.spec.ts
public_complete.spec.ts
alerts.spec.ts
knowledge_base_setup.spec.ts
knowledge_base_status.spec.ts
knowledge_base.spec.ts
summarize.ts

Pending:

Migrate knowledge_base_user_instructions.spec.ts
Clean up all duplicated code

Issue with `knowledge_base_user_instructions.spec.ts`:

there is a difference between stateful and serverless environments.

In the Stateful Environment:

We create users (e.g., editor and secondary_editor) and their respective user_instructions.
The test ensures:
- User A can access their own user_instructions and the public ones.
- User B can do the same for their user_instructions.

Code reference:

kibana/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base_user_instructions.spec.ts

Lines 59 to 90 in 6d49348

    
           describe('when creating private and public user instructions', () => { 
        
             before(async () => { 
        
               await clearKnowledgeBase(es); 
        
               const promises = [ 
        
                 { 
        
                   username: 'editor' as const, 
        
                   isPublic: true, 
        
                 }, 
        
                 { 
        
                   username: 'editor' as const, 
        
                   isPublic: false, 
        
                 }, 
        
                 { 
        
                   username: 'secondary_editor' as const, 
        
                   isPublic: true, 
        
                 }, 
        
                 { 
        
                   username: 'secondary_editor' as const, 
        
                   isPublic: false, 
        
                 }, 
        
               ].map(async ({ username, isPublic }) => { 
        
                 const visibility = isPublic ? 'Public' : 'Private'; 
        
                 await getScopedApiClientForUsername(username)({ 
        
                   endpoint: 'PUT /internal/observability_ai_assistant/kb/user_instructions', 
        
                   params: { 
        
                     body: { 
        
                       id: `${visibility.toLowerCase()}-doc-from-${username}`, 
        
                       text: `${visibility} user instruction from "${username}"`, 
        
                       public: isPublic, 
        
                     },

In the Serverless Environment:

We rely on roles (editor and admin) rather than user creation.
The same functionality is tested, but achieved differently due to the lack of specific user creation.

Code reference:

kibana/x-pack/test_serverless/api_integration/test_suites/observability/ai_assistant/tests/knowledge_base/knowledge_base_user_instructions.spec.ts

Lines 72 to 98 in 6d49348

    
           describe('when creating private and public user instructions', () => { 
        
             before(async () => { 
        
               await clearKnowledgeBase(es); 
        
               const promises = [ 
        
                 { 
        
                   username: 'editor' as const, 
        
                   isPublic: true, 
        
                 }, 
        
                 { 
        
                   username: 'editor' as const, 
        
                   isPublic: false, 
        
                 }, 
        
                 { 
        
                   username: 'secondary_editor' as const, 
        
                   isPublic: true, 
        
                 }, 
        
                 { 
        
                   username: 'secondary_editor' as const, 
        
                   isPublic: false, 
        
                 }, 
        
               ].map(async ({ username, isPublic }) => { 
        
                 const visibility = isPublic ? 'Public' : 'Private'; 
        
                 const user = username === 'editor' ? 'slsEditor' : 'slsAdmin'; 
        
                 await observabilityAIAssistantAPIClient[user]({ 
        
                   endpoint: 'PUT /internal/observability_ai_assistant/kb/user_instructions',

Question:

Should we migrate these tests to deployment-agnostic in a similar manner to how it is implemented in the serverless environment, or should we maintain them as separate implementations between stateful and serverless environments?

…s test suite (elastic#205631) Closes elastic#205537 ## Summary The knowledge base migration test suite is missing in serverless. This PR adds it to the serverless test suite. - This has a dependancy to elastic#205194 since we are removing all serverless tests and adding them to DA tests. - If the DA tests PR gets merged first, I'll refactor this PR to add it there. ### Checklist - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

neptunian · 2025-01-08T19:24:47Z

Should we migrate these tests to deployment-agnostic in a similar manner to how it is implemented in the serverless environment, or should we maintain them as separate implementations between stateful and serverless environments?

@arturoliduena Thanks for bringing that question up. From what I've read here, my understanding is we should use the deployment agnostic samlAuth in the client (which you've done) and not create these users using basic auth as we currently are in stateful. Since the roles exist in stateful, they should be able to adopt the serverless approach and use the roles instead which should be tied to separate users. So yes, assuming this is a simple implementation using serverless-like roles, I would say we can make them deployment agnostic.

…test

… test

…c test

…dUser

…nt-agnostic

… due to failures, not because skipping is intended behavior

…fix'

…ses for conversation interception

…lity/ai_assistant/knowledge_base/knowledge_base_migration.spec.ts Co-authored-by: Viduni Wickramarachchi <[email protected]>

elasticmachine · 2025-01-13T14:16:32Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: d2028d2
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-205194-d2028d2a6987

Failed CI Steps

FTR Configs #63

Test Failures

[job] [logs] FTR Configs #63 / Actions and Triggers app Maintenance Windows Maintenance windows table paginates maintenance windows correctly

Metrics [docs]

✅ unchanged

History

💚 Build #266441 succeeded 4c57434
💚 Build #265961 succeeded abd18f0
💚 Build #265903 succeeded 9e37305
💔 Build #265722 failed fa9e3cc
💛 Build #265515 was flaky 0321c30

Closes elastic#192718 ## Summary This PR add a deployment-agnostic testing environment for Observability AI Assistant tests by unifying the duplicated tests for stateful and serverless environments. It create the ObservabilityAIAssistantApiClient to work seamlessly in both environments, enabling a single test to run across stateful, CI, and MKI. Initial efforts focus on deduplicating the `conversations.spec.ts` and `connectors.spec.ts` files, as these already run in all environments. Move / dedup the tests that exist in stateful and serverless. They run in serverless CI but not MKI and add the skipMki tag. `chat.spec.ts` `complete.spec.ts` `elasticsearch.spec.ts` `public_complete.spec.ts` `alerts.spec.ts` `knowledge_base_setup.spec.ts` `knowledge_base_status.spec.ts` `knowledge_base.spec.ts` `summarize.ts` `knowledge_base_user_instructions.spec.ts`

kibanamachine · 2025-01-13T20:45:19Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12755763835

kibanamachine · 2025-01-13T20:45:19Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12755763869

kibanamachine · 2025-01-13T20:50:33Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 205194

Questions ?

Please refer to the Backport tool documentation

kibanamachine · 2025-01-13T20:50:33Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 205194

Questions ?

Please refer to the Backport tool documentation

arturoliduena · 2025-01-13T21:23:02Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…206516) # Backport This will backport the following commits from `main` to `8.x`: - [Observability AI Assistant Tests Deployment Agnostic (#205194)](#205194)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  --------- Co-authored-by: kibanamachine <[email protected]>

arturoliduena added release_note:skip Skip the PR/issue when compiling release notes Team:Obs AI Assistant Observability AI Assistant v8.18.0 labels Dec 27, 2024

arturoliduena requested a review from a team as a code owner December 27, 2024 11:29

arturoliduena requested a review from a team as a code owner December 27, 2024 12:05

botelastic bot added the ci:project-deploy-observability Create an Observability project label Dec 27, 2024

sorenlouv reviewed Jan 2, 2025

View reviewed changes

...tion/deployment_agnostic/apis/observability/ai_assistant/conversations/conversations.spec.ts Outdated Show resolved Hide resolved

neptunian reviewed Jan 2, 2025

View reviewed changes

sorenlouv reviewed Jan 3, 2025

View reviewed changes

sorenlouv approved these changes Jan 3, 2025

View reviewed changes

This was referenced Jan 3, 2025

[Obs AI Assistant] Add alerts.spec.ts to serverless tests #205524

Closed

[Obs AI Assistant] move kb migration test suite to serverless #205537

Closed

arturoliduena force-pushed the obs-ai-assistant-192718-deployment-agnostic-api-integration-tests branch from c1c4339 to 1dc1987 Compare January 3, 2025 20:38

arturoliduena added the backport:skip This commit does not require backporting label Jan 4, 2025

sorenlouv reviewed Jan 6, 2025

View reviewed changes

...tion/deployment_agnostic/apis/observability/ai_assistant/conversations/conversations.spec.ts Outdated Show resolved Hide resolved

sorenlouv reviewed Jan 6, 2025

View reviewed changes

viduni94 mentioned this pull request Jan 6, 2025

[Obs AI Assistant] Add knowledge base migration test to the serverless test suite #205631

Merged

2 tasks

sorenlouv reviewed Jan 7, 2025

View reviewed changes

neptunian reviewed Jan 7, 2025

View reviewed changes

arturoliduena and others added 14 commits January 13, 2025 14:04

Migrating existing alerts.spec.ts test to deployment-agnostic test

6514edf

Add skipMKI tag to alerts function deployment-agnostic test

cd3ecf1

Migrating existing elasticsearch.spec.ts test to deployment-agnostic …

c105e67

…test

Add skipMKI tag to elasticsearch.spec.ts function deployment-agnostic…

3a0f3dc

… test

Migrating existing public_complete.spec.ts test to deployment-agnosti…

6026828

…c test

Refactor ai_assistant tests to use viewer role instead of unauthorize…

c672e60

…dUser

Migrating existing knowledge_base tests to deployment-agnostic

1a11519

Migrating existing knowledge_base_user_instructions tests to deployme…

459165b

…nt-agnostic

Updated skipMKI to failsOnMKI to make it clear that tests are skipped…

008da3c

… due to failures, not because skipping is intended behavior

Remove duplicated helper functions and clean up test files

c70aca7

[CI] Auto-commit changed files from 'node scripts/lint_ts_projects --…

0009746

…fix'

Refactor knowledge_base_user_instructions tests to use interceptPromi…

5906748

…ses for conversation interception

delete publicApi from ObservabilityAIAssistantApiProvider

d6b0484

Update x-pack/test/api_integration/deployment_agnostic/apis/observabi…

d2028d2

…lity/ai_assistant/knowledge_base/knowledge_base_migration.spec.ts Co-authored-by: Viduni Wickramarachchi <[email protected]>

arturoliduena force-pushed the obs-ai-assistant-192718-deployment-agnostic-api-integration-tests branch from 4c57434 to d2028d2 Compare January 13, 2025 13:04

arturoliduena merged commit ee6c5bd into elastic:main Jan 13, 2025
8 checks passed

kibanamachine added the v9.0.0 label Jan 13, 2025

kibanamachine mentioned this pull request Jan 13, 2025

[Obs AI Assistant] Attempt to fix flaky KB migration test #206324

Merged

2 tasks

arturoliduena added backport:version Backport to applied version labels and removed backport:skip This commit does not require backporting labels Jan 13, 2025

arturoliduena mentioned this pull request Jan 13, 2025

[8.x] Observability AI Assistant Tests Deployment Agnostic (#205194) #206516

Merged

pheyos mentioned this pull request Jan 14, 2025

[FailsOnMKI] Serverless Observability - Deployment-agnostic API Integration Tests.x-pack/test/api_integration/deployment_agnostic/apis/observability/ai_assistant/conversations/conversations.spec.ts #206549

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Observability AI Assistant Tests Deployment Agnostic #205194

Observability AI Assistant Tests Deployment Agnostic #205194

arturoliduena commented Dec 27, 2024 •

edited by kibanamachine

Loading

elasticmachine commented Dec 27, 2024

github-actions bot commented Dec 27, 2024

sorenlouv Jan 2, 2025 •

edited

Loading

sorenlouv Jan 2, 2025

arturoliduena Jan 3, 2025

sorenlouv Jan 2, 2025

arturoliduena Jan 3, 2025

sorenlouv Jan 2, 2025

neptunian Jan 2, 2025

neptunian Jan 2, 2025

arturoliduena Jan 3, 2025

neptunian Jan 2, 2025

sorenlouv Jan 3, 2025

sorenlouv Jan 3, 2025

arturoliduena Jan 6, 2025

sorenlouv left a comment

sorenlouv Jan 6, 2025 •

edited

Loading

viduni94 Jan 6, 2025 •

edited

Loading

arturoliduena Jan 7, 2025

sorenlouv Jan 7, 2025 •

edited

Loading

viduni94 Jan 7, 2025 •

edited

Loading

sorenlouv Jan 7, 2025

sorenlouv Jan 7, 2025 •

edited

Loading

neptunian Jan 7, 2025

arturoliduena Jan 9, 2025

arturoliduena commented Jan 8, 2025

neptunian commented Jan 8, 2025 •

edited

Loading

elasticmachine commented Jan 13, 2025 •

edited

Loading

kibanamachine commented Jan 13, 2025

kibanamachine commented Jan 13, 2025

kibanamachine commented Jan 13, 2025

kibanamachine commented Jan 13, 2025

arturoliduena commented Jan 13, 2025

	endpoint: `GET ${CONNECTOR_API_URL}`,
	endpoint: `GET /internal/observability_ai_assistant/connectors`,

		@@ -8,13 +8,11 @@
		import expect from '@kbn/expect';
		import type { Agent as SuperTestAgent } from 'supertest';


		type PublicEndpoint<T extends APIEndpoint> = T extends `${string} /api/${string}` ? T : never;

		function createObservabilityAIAssistantApiClient({

Observability AI Assistant Tests Deployment Agnostic #205194

Observability AI Assistant Tests Deployment Agnostic #205194

Conversation

arturoliduena commented Dec 27, 2024 • edited by kibanamachine Loading

Summary

Checklist

elasticmachine commented Dec 27, 2024

github-actions bot commented Dec 27, 2024

🤖 GitHub comments

sorenlouv Jan 2, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sorenlouv left a comment

Choose a reason for hiding this comment

sorenlouv Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

viduni94 Jan 6, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sorenlouv Jan 7, 2025 • edited Loading

Choose a reason for hiding this comment

viduni94 Jan 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sorenlouv Jan 7, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arturoliduena commented Jan 8, 2025

Migrated to Deployment-Agnostic:

Pending:

Issue with knowledge_base_user_instructions.spec.ts:

In the Stateful Environment:

In the Serverless Environment:

Question:

neptunian commented Jan 8, 2025 • edited Loading

elasticmachine commented Jan 13, 2025 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

kibanamachine commented Jan 13, 2025

kibanamachine commented Jan 13, 2025

kibanamachine commented Jan 13, 2025

💔 All backports failed

Manual backport

Questions ?

kibanamachine commented Jan 13, 2025

💔 All backports failed

Manual backport

Questions ?

arturoliduena commented Jan 13, 2025

💚 All backports created successfully

Questions ?

arturoliduena commented Dec 27, 2024 •

edited by kibanamachine

Loading

sorenlouv Jan 2, 2025 •

edited

Loading

sorenlouv Jan 6, 2025 •

edited

Loading

viduni94 Jan 6, 2025 •

edited

Loading

sorenlouv Jan 7, 2025 •

edited

Loading

viduni94 Jan 7, 2025 •

edited

Loading

sorenlouv Jan 7, 2025 •

edited

Loading

Issue with `knowledge_base_user_instructions.spec.ts`:

neptunian commented Jan 8, 2025 •

edited

Loading

elasticmachine commented Jan 13, 2025 •

edited

Loading