Add Action to post message to slack #1242

dbutenhof · 2024-02-02T17:15:27Z

Notes:

Horreum formatters: Slack markdown differs from GitHub markdown, so not all the existing formatters will be pretty on both. (Specifically the link syntax.) I considered working out a way to customize common markdown templates, but didn't do that.
To enable a simple test I created a new Test object formatter with Slack markdown syntax. I have no idea whether a formatter for Test objects might actually be useful.
The change in the pom.xml goes way back to our first debugging session when I was having Maven build problems...

The behavior of the action is similar to the existing GitHub actions, and largely a cookie-cutter clone. (Some common code could be extracted into a base class, but I resisted making unnecessary changes.)

Essentially, it accepts a Slack service token, a Slack channel ID, and formatter, and posts the formatted message to the designated Slack channel.

On the Slack side, the setup required includes:

Define and install a Slack "app" in your Slack workspace, using the slack CLI or https://api.slack.com/apps, then generate an app token with at least scopes "channel:join" and "chat:write". (Note that a workspace may require administrative review and approval to install an app.) The token should be a long alphanumeric string starting with
"xoxb-".
Use the Slack UI (or API) to determine the ID of the channel you want to use. (Which looks something like "C00A000A0A".)
Use the https://api.slack.com/methods/conversations.join/test API tester with your token and the channel ID to "join" your app to the channel so it can post. (There are of course other ways, but this is probably the easiest, and provides a quick verification that the token and channel ID are good.)

Creating a slack-channel-message action requires two "config" parameters and one "secrets" parameter:

The "channel" conf value
The Horreum "formatter" name
The "token" secret

The Action will always post a "mrkdwn" Slack message using the markdown document generated by the Horreum formatter template. It will post using the name of the Slack app (e.g., "Horreum Server").

Note that there's some incompatibility between GitHub and Slack markdown syntax, specifically for links. Existing Horreum markdown templates for GitHub use the [text](url) syntax whereas Slack uses <url|text>. This suggests that Horreum may need to extend the templating classes to conditionalize link formatting based on the action context.

For testing (and as an example), this PR proposes a new formatter template for the Test class, using Slack link syntax.

Resolves issue #838

Fixes Issue

Changes proposed

Check List (Check all the applicable boxes)

My code follows the code style of this project.
My change requires changes to the documentation.
I have updated the documentation accordingly.
All new and existing tests passed.

johnaohara

Please see comments

horreum-backend/src/main/java/io/hyperfoil/tools/horreum/action/SlackPluginBase.java

johnaohara · 2024-02-21T07:01:05Z

horreum-backend/src/main/java/io/hyperfoil/tools/horreum/action/SlackChannelMessageAction.java

+
+   @Override
+   public void validate(JsonNode config, JsonNode secrets) {
+      log.infof("Validating config %s, secrets %s", config, secrets);


Please can this be a TRACE message

I must have missed this when I was cleaning up: I'd meant to remove it, although it was really useful in initial testing to confirm that it had been called.

johnaohara · 2024-02-21T07:06:47Z

horreum-backend/src/main/java/io/hyperfoil/tools/horreum/action/SlackChannelMessageAction.java

+      ObjectMapper mapper = new ObjectMapper();
+      ObjectNode body = mapper.createObjectNode();


We should re-use Util.OBJECT_MAPPER here

Ah, thanks: I'd expected there must be an object mapper already instantiated somewhere, but I failed to find it.

johnaohara · 2024-02-21T07:13:42Z

horreum-backend/src/main/java/io/hyperfoil/tools/horreum/action/SlackPluginBase.java

+            .setHost("slack.com")
+            .setPort(443)
+            .setURI("/api/chat.postMessage")
+            .setSsl(true);


These values should be configurable (I see it is not in the github example, but I think that is an oversight), with these specific values as the default. This will allow us to make changes to service config if either a) anything changes in the future b) if we want to mock a slack service in testing

Please see https://github.com/Hyperfoil/Horreum/blob/master/horreum-backend/src/main/java/io/hyperfoil/tools/horreum/action/HttpAction.java#L36 for configuration examples with a default values

We could then define difference profiles in application.properties;

%prod.horreum.action.slack.host=slack.com %prod.horreum.action.slack.port=443 %prod.horreum.action.slack.ssl=true %test.horreum.action.slack.host=localhost %test.horreum.action.slack.port=8081 %test.horreum.action.slack.ssl=false

I'd initially toyed with various levels of URI configurability. For example, until I got the Slack auth token mechanism figured out I wasn't entirely sure that I could ignore Slack's subdomains for workspaces (e.g., redhat-internal.slack.com). In the end I determined there wasn't anything that needed to be user-configurable: but your suggestion of making it application configurable makes sense.

I think I'll follow the reasonable HTTP action model, accept a full URL and break it down using the URL class rather than configuring each part separately, so it'd just be more like %test.horreum.action.slack.url=http://localhost:8081/api/slack

Sounds like a good idea

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

johnaohara · 2024-02-21T07:21:09Z

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

+      // If only ...
+      // Assert.assertTrue("No slack action errors found", errors > 0);


If we mock the slack service and use the mocked endpoint for testing, we can control the reponse, and therefore the errors we see in testing. A mocked service could be something as simple as ;

package io.hyperfoil.tools.horreum; import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.node.ObjectNode; import jakarta.enterprise.context.ApplicationScoped; import jakarta.inject.Inject; import jakarta.ws.rs.Consumes; import jakarta.ws.rs.POST; import jakarta.ws.rs.Path; import jakarta.ws.rs.Produces; import jakarta.ws.rs.core.MediaType; import org.eclipse.microprofile.openapi.annotations.tags.Tag; import org.jboss.logging.Logger; @ApplicationScoped @Path("/api/chat.postMessage") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.APPLICATION_JSON) @Tag(name = "SlackService", description = "Mock endpoint for Slack service.") public class SlackDummyService { private static final Logger log = Logger.getLogger(SlackDummyService.class); @Inject ObjectMapper mapper; @POST public ObjectNode mockSlackEndpoint(String payload) { log.infof("Received payload: %s", payload); return mapper.createObjectNode().put("ok", true); } }

And if we implement the config options as above, we can use that service (instead of the live slack service) with a config change in application.properties;

%test.horreum.action.slack.host=localhost %test.horreum.action.slack.port=8081 %test.horreum.action.slack.ssl=false

Thanks for the sample. I'd considered mocking but while I knew I'd run across some API mocking somewhere, I couldn't find it again and at that point I decided I was better off posting something to review than spending more time searching.

pom.xml

johnaohara · 2024-02-21T07:27:06Z

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

+      ObjectMapper mapper = new ObjectMapper();
+      ObjectNode config = mapper.createObjectNode()


We should re-use Util.OBJECT_MAPPER here

johnaohara · 2024-02-21T07:30:02Z

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

+      log.infof("Waiting for action to run");
+      int errors = 0;
+      for (int i = 0; i < 10; i++) {
+         try {
+            Thread.sleep(1000);
+         } catch (InterruptedException e) {
+         }
+
+         // Should these action errors be logged under dummyTest.id? The
+         // ActionServiceImpl handler seems to always log under its argument
+         // value, -1: but I still can't find them.
+         List<ActionLog> logs = jsonRequest().auth().oauth2(getTesterToken())
+               .queryParam("level", PersistentLogDAO.ERROR).get("/api/log/action/-1")
+               .then().statusCode(200)
+               .contentType(ContentType.JSON).extract().body().jsonPath().getList(".", ActionLog.class);
+         log.infof("[%d] Retrieved %d log entries", i, logs.size());
+         for (ActionLog l : logs) {
+            if (l.type == SlackChannelMessageAction.TYPE_SLACK_MESSAGE) {
+               log.infof("Found log for %d: %d, %s, %s: %s", l.testId, l.level, l.event, l.type, l.message);
+               errors++;
+            }
+         }
+      }
+      log.infof("Found %d errors for %s on test %d", errors, SlackChannelMessageAction.TYPE_SLACK_MESSAGE, -1);


If we resolve the RLS issue, this will be a single check. and we can drop the loop.

However, we should test multiple scenarios with a mocked slack service, to test that valid and invalid requests to the slack API are handled appropriately

johnaohara · 2024-02-21T07:57:57Z

Draft notes:

1. I tried to write a test for a failing action, by querying the action log service, but I'm not seeing anything even though the journal log shows that the action is running and generating the expected authentication failure;]

There is an issue with Row Level Security in the database not returning the log messages, I have opened an issue to tack this: #1353

2. Maybe I should just remove the test case anyway as it's generating failing REST calls to Slack on each run!

I think we should mock the slack service in the tests. I included a simple example of how to do this in the review

3. Horreum formatters: Slack markdown differs from GitHub markdown, so not all the existing formatters will be pretty on both. (Specifically the link syntax.) I considered working out a way to customize common markdown templates, but didn't do that.

I think this is fine for now. There is a much bigger exercise about making templates customizable, both for users and for integrations, I think we can/should handle the templating customizations in a different PR.

The simplest way to handle this atm is within the BodyFormatters. Instead of creating a templates for;

git; Please [check it out]({publicUrl}/changes?test={testNameEncoded}&fingerprint={fingerprint}) and provide an...
slack; Please <{publicUrl}/changes?test={testNameEncoded}|check it out> and provide an...

The generic template could be; Please {markdownLink} and provide an...

The git formatter could provide the correct link with;

template.data("markdownLink", "Please [check it out](".concat(publicUrl).concat("/changes?test=").concat(URLEncoder.encode(event.testName, StandardCharsets.UTF_8)).concat("&fingerprint=").concat(URLEncoder.encode(fingerprint, StandardCharsets.UTF_8)).concat(")") )

The slack formatter could provide the correct link with;
template.data("markdownLink", "Please <".concat(publicUrl).concat("/changes?test=").concat(URLEncoder.encode(event.testName, StandardCharsets.UTF_8)).concat("&fingerprint=").concat(URLEncoder.encode(fingerprint, StandardCharsets.UTF_8)).concat(" | check it out>") )

4. To enable a simple test I created a new `Test` object formatter with Slack markdown syntax. I have no idea whether a formatter for `Test` objects might actually be useful.

5. The change in the `pom.xml` goes way back to our first debugging session when I was having Maven build problems...

Thanks for figuring this out, I have pinged @whitingjr to take a look as it was related to an issue opening the project with eclipse, but I am unsure why the ant plugin resolved this issue

The behavior of the action is similar to the existing GitHub actions, and largely a cookie-cutter clone. (Some common code could be extracted into a base class, but I resisted making unnecessary changes.)

Essentially, it accepts a Slack service token, a Slack channel ID, and formatter, and posts the formatted message to the designated Slack channel.

This will need to be configured within the UI, please see https://github.com/Hyperfoil/Horreum/blob/master/horreum-web/src/domain/actions/ActionComponentForm.tsx#L111 on how this is done for the github actions

Atm the UI is not pluggable for actions, so we need to add the UI logic directly into the component

On the Slack side, the setup required includes:

1. Define and install a Slack "app" in your Slack workspace, using the slack CLI or https://api.slack.com/apps, then generate an app token with at least scopes "channel:join" and "chat:write". (Note that a workspace may require administrative review and approval to install an app.) The token should be a long alphanumeric string starting with
   "xoxb-".

2. Use the Slack UI (or API) to determine the ID of the channel you want to use. (Which looks something like "C00A000A0A".)

3. Use the https://api.slack.com/methods/conversations.join/test API tester with your token and the channel ID to "join" your app to the channel so it can post. (There are of course other ways, but this is probably the easiest, and provides a quick verification that the token and channel ID are good.)

Creating a slack-channel-message action requires two "config" parameters and one "secrets" parameter:

1. The "channel" conf value

2. The Horreum "formatter" name

3. The "token" secret

Please can we have this information in a doc for slack integration in docs/site/content/en/docs/Integrations?

The Action will always post a "mrkdwn" Slack message using the markdown document generated by the Horreum formatter template. It will post using the name of the Slack app (e.g., "Horreum Server").

Note that there's some incompatibility between GitHub and Slack markdown syntax, specifically for links. Existing Horreum markdown templates for GitHub use the [text](url) syntax whereas Slack uses <url|text>. This suggests that Horreum may need to extend the templating classes to conditionalize link formatting based on the action context.

For testing (and as an example), this PR proposes a new formatter template for the Test class, using Slack link syntax.

Finally, I tried to add testing for this using the framework in the ActionServiceTest.java module. The new test attempts to create a Slack action, force it to execute, and validate failure (due to a "bad" fake channel and token) by querying the action log. The code fails to find any log entries either for the actual test ID or for test "-1", which appears to be what ActionServiceImpl.java actually uses when logging an error. Errors are however visible in the log output.

Resolves issue #838

Fixes Issue

Changes proposed

Check List (Check all the applicable boxes)

* [ ]  My code follows the code style of this project.

* [ ]  My change requires changes to the documentation.

* [ ]  I have updated the documentation accordingly.

* [ ]  All new and existing tests passed.

johnaohara · 2024-02-21T07:58:30Z

@dbutenhof thank you for working on this :)

dbutenhof

This will need to be configured within the UI,

Yes, and that was an issue I neglected to list. I'd started hacking around in the UI code but gave up when I was having trouble getting the (existing) action sheets working in my test environment. I'm not really a UI coder, but I do know just enough Javascript and React to be slightly dangerous (mostly to myself), so I can give it another try.

Please can we have this information in a doc for slack integration in docs/site/content/en/docs/Integrations?

Sorry, yes: another missing question, since at the time I'd looked I hadn't found much information on actions at all aside from a somewhat incomplete bit on the HTTP action.

dbutenhof · 2024-02-21T12:28:49Z

horreum-backend/src/main/java/io/hyperfoil/tools/horreum/action/SlackChannelMessageAction.java

+
+   @Override
+   public void validate(JsonNode config, JsonNode secrets) {
+      log.infof("Validating config %s, secrets %s", config, secrets);


I must have missed this when I was cleaning up: I'd meant to remove it, although it was really useful in initial testing to confirm that it had been called.

dbutenhof · 2024-02-21T12:29:29Z

horreum-backend/src/main/java/io/hyperfoil/tools/horreum/action/SlackChannelMessageAction.java

+      ObjectMapper mapper = new ObjectMapper();
+      ObjectNode body = mapper.createObjectNode();


Ah, thanks: I'd expected there must be an object mapper already instantiated somewhere, but I failed to find it.

dbutenhof · 2024-02-21T12:34:10Z

horreum-backend/src/main/java/io/hyperfoil/tools/horreum/action/SlackPluginBase.java

+            .setHost("slack.com")
+            .setPort(443)
+            .setURI("/api/chat.postMessage")
+            .setSsl(true);


I'd initially toyed with various levels of URI configurability. For example, until I got the Slack auth token mechanism figured out I wasn't entirely sure that I could ignore Slack's subdomains for workspaces (e.g., redhat-internal.slack.com). In the end I determined there wasn't anything that needed to be user-configurable: but your suggestion of making it application configurable makes sense.

I think I'll follow the reasonable HTTP action model, accept a full URL and break it down using the URL class rather than configuring each part separately, so it'd just be more like %test.horreum.action.slack.url=http://localhost:8081/api/slack

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

dbutenhof · 2024-02-21T12:41:54Z

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

+      // If only ...
+      // Assert.assertTrue("No slack action errors found", errors > 0);


Thanks for the sample. I'd considered mocking but while I knew I'd run across some API mocking somewhere, I couldn't find it again and at that point I decided I was better off posting something to review than spending more time searching.

pom.xml

johnaohara · 2024-02-22T06:58:17Z

Yes, and that was an issue I neglected to list. I'd started hacking around in the UI code but gave up when I was having trouble getting the (existing) action sheets working in my test environment. I'm not really a UI coder, but I do know just enough Javascript and React to be slightly dangerous (mostly to myself), so I can give it another try.

If you would like any help with React+Typescript 😰 let me know and we can help with that part

johnaohara · 2024-02-22T06:59:38Z

Sorry, yes: another missing question, since at the time I'd looked I hadn't found much information on actions at all aside from a somewhat incomplete bit on the HTTP action.

yes, unfortunately it is still on the backlog: #1177

dbutenhof · 2024-03-20T14:53:20Z

There is an issue with Row Level Security in the database not returning the log messages, I have opened an issue to tack this: #1353

@johnaohara ... Any status here? I'd feel better if my test could validate that the fake Slack post worked through standard APIs. (Although I did consider adding a Slack mock GET API that would tell me what it had seen, and that might still be a viable alternative...)

Anyway, the new commit has the UI working. (Or at least I can create global Actions.)

johnaohara · 2024-04-16T09:45:57Z

@dbutenhof I have added @barreiro as a reviewer, he has volunteered to help get this PR merged. We are targeting the 0.13 release for this PR

barreiro

@dbutenhof I added a few comments, but overall looks good.

just one last thing, after you address those please squash your commits into a single one

thanks for your work !!

barreiro · 2024-04-17T01:14:08Z

docs/site/content/en/openapi/openapi.yaml

-          - $ref: '#/components/schemas/GithubIssueCommentAction'
-          - $ref: '#/components/schemas/GithubIssueCreateAction'
+          - $ref: '#/components/schemas/GithubIssueCommentActionConfig'
+          - $ref: '#/components/schemas/GithubIssueCommentActionConfig'


Generated because of the duplicate annotation in the DTO; but, thanks, good catch.

barreiro · 2024-04-17T01:14:50Z

horreum-api/src/main/java/io/hyperfoil/tools/horreum/api/data/Action.java

+            @DiscriminatorMapping(schema = SlackChannelMessageActionConfig.class, value = "slack-channel-message"),
+    }, oneOf = {
+            GithubIssueCommentActionConfig.class,
+            GithubIssueCommentActionConfig.class,


barreiro · 2024-04-17T01:16:00Z

horreum-web/src/domain/actions/ActionComponentForm.tsx

+                                      ? { experimentResultToMarkdown: "Experiment result to Markdown" }
+                                      : props.action.event === TEST_NEW
+                                        ? { testToSlack: "Test to Slack Markdown" }
+                                        : {}


RUN_NEW is missing

There's no formatter for RUN_NEW. Perhaps there should be. I decided that creating it was outside the scope of "implementing the slack action". (It affects GitHub issue comment, too, which allows selecting "run/new" while there's no applicable formatter, and the action can't actually be created.)

I find the implementation of the action form a bit awkward, and one of my complaints is this formatter popup when selecting the event really defines which formatter can apply anyway and it seems odd to require also selecting the formatter. I wrote up an issue about this, #1532

I see two issues with the UI ... the formatters are one and another is the token that is sent as ****** to the server. since both are common to all the actions I say that we fix them in separate issues.

if you can get all the formatters for slack to be present in this PR that would be appreciated

It would be easy enough to construct a new formatter, although we'd need to decide exactly what should be formatted for a new run and how to present it.

And if I do create a new formatter and use it here, it'll also be available for the GitHub issue comment action: if it includes a URL link, that'll be "badly formatted" for GitHub Markdown without

I decided it didn't make sense to get into all of that with this PR as it's not a new problem.

barreiro · 2024-04-17T01:22:39Z

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

+          * retried asynchronously and we won't necessarily catch an error, but
+          * it's sufficient for the other test cases.
+          */
+         Assert.assertEquals("Test case " + test_case.getKey() + " failed", test_case.getValue() ? 0 : 1, errors);


Assertions.assertEquals()

Sure: any particular reason you prefer Assertions over Assert? There are so many different assertion packages floating around, and existing code seems to use several.

no reason, just consistency with other assertions in the test

barreiro · 2024-04-17T01:24:48Z

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

 import jakarta.ws.rs.core.HttpHeaders;
 import org.eclipse.microprofile.config.inject.ConfigProperty;
+
+import org.jboss.logging.Logger;


I'd removed all the log output and apparently forgot to remove the import.

barreiro · 2024-04-17T01:40:47Z

horreum-integration-tests/src/test/resources/application.properties

@@ -1 +1,2 @@
 quarkus.test.enable-callbacks-for-integration-tests=true
+quarkus.http.test-port=8081


this is not needed

Maybe not; I'll try again, but this solved one of the failures I'd run across during testing.

Yes, I was able to remove it. Great. Whatever I ran into must have gone away. 😆

dbutenhof

just one last thing, after you address those please squash your commits into a single one

Sure. I usually don't like to squash during reviews because it complicates re-reviewing (and I'd already exposed this as a draft to John and Ståle). I'm used to routinely doing squash-and-merge anyway, so the extra commits don't generally matter.

thanks for your work !!

You're welcome. In a "previous life" I worked with Java EE exclusively for years (although Spring instead of Quarkus), but it's amazing how much my Java had gotten "out of shape" after working in Python for 3 years now. It's been an interesting experience.

dbutenhof · 2024-04-17T11:33:04Z

docs/site/content/en/openapi/openapi.yaml

-          - $ref: '#/components/schemas/GithubIssueCommentAction'
-          - $ref: '#/components/schemas/GithubIssueCreateAction'
+          - $ref: '#/components/schemas/GithubIssueCommentActionConfig'
+          - $ref: '#/components/schemas/GithubIssueCommentActionConfig'


Generated because of the duplicate annotation in the DTO; but, thanks, good catch.

dbutenhof · 2024-04-17T11:34:10Z

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

 import jakarta.ws.rs.core.HttpHeaders;
 import org.eclipse.microprofile.config.inject.ConfigProperty;
+
+import org.jboss.logging.Logger;


I'd removed all the log output and apparently forgot to remove the import.

dbutenhof · 2024-04-17T11:36:51Z

horreum-backend/src/test/java/io/hyperfoil/tools/horreum/svc/ActionServiceTest.java

+          * retried asynchronously and we won't necessarily catch an error, but
+          * it's sufficient for the other test cases.
+          */
+         Assert.assertEquals("Test case " + test_case.getKey() + " failed", test_case.getValue() ? 0 : 1, errors);


Sure: any particular reason you prefer Assertions over Assert? There are so many different assertion packages floating around, and existing code seems to use several.

dbutenhof · 2024-04-17T11:37:54Z

horreum-integration-tests/src/test/resources/application.properties

@@ -1 +1,2 @@
 quarkus.test.enable-callbacks-for-integration-tests=true
+quarkus.http.test-port=8081


Maybe not; I'll try again, but this solved one of the failures I'd run across during testing.

dbutenhof · 2024-04-17T11:43:30Z

horreum-web/src/domain/actions/ActionComponentForm.tsx

+                                      ? { experimentResultToMarkdown: "Experiment result to Markdown" }
+                                      : props.action.event === TEST_NEW
+                                        ? { testToSlack: "Test to Slack Markdown" }
+                                        : {}


There's no formatter for RUN_NEW. Perhaps there should be. I decided that creating it was outside the scope of "implementing the slack action". (It affects GitHub issue comment, too, which allows selecting "run/new" while there's no applicable formatter, and the action can't actually be created.)

I find the implementation of the action form a bit awkward, and one of my complaints is this formatter popup when selecting the event really defines which formatter can apply anyway and it seems odd to require also selecting the formatter. I wrote up an issue about this, #1532

The behavior of the action is similar to the existing GitHub actions, and largely a cookie-cutter clone. (Some common code could be extracted into a base class, but I resisted making unnecessary changes.) Essentially, it accepts a Slack service token, a Slack channel ID, and formatter, and posts the formatted message to the designated Slack channel. Creating a `slack-channel-message` action requires two "config" parameters and one "secrets" parameter: 1 The "channel" conf value 2 The Horreum "formatter" name 3 The "token" secret The Action will always post a "mrkdwn" Slack message using the markdown document generated by the Horreum formatter template. It will post using the name of the Slack app (e.g., "Horreum Server"). Note that there's some incompatibility between GitHub and Slack markdown syntax, specifically for links. Existing Horreum markdown templates for GitHub use the `[text](url)` syntax whereas Slack uses `<url|text>`. This suggests that Horreum may need to extend the templating classes to conditionalize link formatting based on the action context. Resolves issue Hyperfoil#838

dbutenhof · 2024-04-17T16:04:30Z

Is this new? I've never seen it in local test runs. This seems to happen immediately after my new action test is torn down, so there could be a connection, but it's murky...

2024-04-17 15:25:43,739 ERROR [io.sma.rea.mes.provider] (horreum.dataset.pool-1) SRMSG00200: The method io.hyperfoil.tools.horreum.svc.ServiceMediator#processDatasetEvents has thrown an exception: java.lang.NullPointerException: Cannot load from object array because "values" is null
at org.hibernate.persister.entity.AbstractEntityPersister.initializeLazyPropertiesFromDatastore(AbstractEntityPersister.java:1573)
at org.hibernate.persister.entity.AbstractEntityPersister.initializeLazyProperty(AbstractEntityPersister.java:1497)
at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.lambda$loadAttribute$0(LazyAttributeLoadingInterceptor.java:112)
at org.hibernate.bytecode.enhance.spi.interceptor.EnhancementHelper.performWork(EnhancementHelper.java:206)
at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.loadAttribute(LazyAttributeLoadingInterceptor.java:82)
at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.fetchAttribute(LazyAttributeLoadingInterceptor.java:78)
at org.hibernate.bytecode.enhance.spi.interceptor.LazyAttributeLoadingInterceptor.handleRead(LazyAttributeLoadingInterceptor.java:59)
at org.hibernate.bytecode.enhance.spi.interceptor.AbstractInterceptor.readObject(AbstractInterceptor.java:152)
at io.hyperfoil.tools.horreum.entity.data.DatasetDAO.$$_hibernate_read_data(DatasetDAO.java)
at io.hyperfoil.tools.horreum.entity.data.DatasetDAO.getData(DatasetDAO.java)
at io.hyperfoil.tools.horreum.svc.SchemaServiceImpl.validateDatasetData(SchemaServiceImpl.java:346)
at io.hyperfoil.tools.horreum.svc.SchemaServiceImpl_Subclass.validateDatasetData$$superforward(Unknown Source)
at io.hyperfoil.tools.horreum.svc.SchemaServiceImpl_Subclass$$function$$21.apply(Unknown Source)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
at io.quarkus.arc.impl.AroundInvokeInvocationContext$NextAroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:97)
at io.hyperfoil.tools.horreum.server.RolesInterceptor.intercept(RolesInterceptor.java:70)
at io.hyperfoil.tools.horreum.server.RolesInterceptor_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:70)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:136)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.invokeInOurTx(TransactionalInterceptorBase.java:107)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.doIntercept(TransactionalInterceptorRequired.java:38)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorBase.intercept(TransactionalInterceptorBase.java:61)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired.intercept(TransactionalInterceptorRequired.java:32)
at io.quarkus.narayana.jta.runtime.interceptor.TransactionalInterceptorRequired_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
at io.hyperfoil.tools.horreum.svc.SchemaServiceImpl_Subclass.validateDatasetData(Unknown Source)
at io.hyperfoil.tools.horreum.svc.ServiceMediator.validateDataset(ServiceMediator.java:242)
at io.hyperfoil.tools.horreum.svc.ServiceMediator.processDatasetEvents(ServiceMediator.java:155)
at io.hyperfoil.tools.horreum.svc.ServiceMediator_Subclass.processDatasetEvents$$superforward(Unknown Source)
at io.hyperfoil.tools.horreum.svc.ServiceMediator_Subclass$$function$$6.apply(Unknown Source)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:73)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:62)
at io.quarkus.arc.impl.ActivateRequestContextInterceptor.invoke(ActivateRequestContextInterceptor.java:129)
at io.quarkus.arc.impl.ActivateRequestContextInterceptor.aroundInvoke(ActivateRequestContextInterceptor.java:33)
at io.quarkus.arc.impl.ActivateRequestContextInterceptor_Bean.intercept(Unknown Source)
at io.quarkus.arc.impl.InterceptorInvocation.invoke(InterceptorInvocation.java:42)
at io.quarkus.arc.impl.AroundInvokeInvocationContext.perform(AroundInvokeInvocationContext.java:30)
at io.quarkus.arc.impl.InvocationContexts.performAroundInvoke(InvocationContexts.java:27)
at io.hyperfoil.tools.horreum.svc.ServiceMediator_Subclass.processDatasetEvents(Unknown Source)
at io.hyperfoil.tools.horreum.svc.ServiceMediator_ClientProxy.processDatasetEvents(Unknown Source)
at io.hyperfoil.tools.horreum.svc.ServiceMediator_SmallRyeMessagingInvoker_processDatasetEvents_ebe511ee548f91c634de3d182c814a6da94b30b1.invoke(Unknown Source)
at io.smallrye.reactive.messaging.providers.AbstractMediator.lambda$invokeBlocking$10(AbstractMediator.java:171)
at io.smallrye.context.impl.wrappers.SlowContextualSupplier.get(SlowContextualSupplier.java:21)
at io.smallrye.mutiny.operators.uni.builders.UniCreateFromDeferredSupplier.subscribe(UniCreateFromDeferredSupplier.java:25)
at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
at io.smallrye.mutiny.groups.UniSubscribe.withSubscriber(UniSubscribe.java:51)
at io.smallrye.mutiny.groups.UniSubscribe.with(UniSubscribe.java:110)
at io.smallrye.mutiny.groups.UniSubscribe.with(UniSubscribe.java:88)
at io.vertx.mutiny.core.WorkerExecutor$1.handle(WorkerExecutor.java:107)
at io.vertx.mutiny.core.WorkerExecutor$1.handle(WorkerExecutor.java:105)
at io.vertx.core.impl.ContextBase.lambda$executeBlocking$1(ContextBase.java:180)
at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:277)
at io.vertx.core.impl.ContextBase.lambda$internalExecuteBlocking$2(ContextBase.java:199)
at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1512)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:840)

johnaohara · 2024-04-18T12:48:22Z

Is this new? I've never seen it in local test runs. This seems to happen immediately after my new action test is torn down, so there could be a connection, but it's murky...

@dbutenhof this is a transient test failure that we see in the CI occasionally. We need to investigate the root cause, but it appears to be a race condition updating the ValidationErrors for Datasets in the database.

Usually the issue resolves when the test is re-run.

It is likely a bug in the test case that we need to fix

dbutenhof · 2024-04-18T13:10:40Z

Usually the issue resolves when the test is re-run.

It is likely a bug in the test case that we need to fix

Thanks. Transient errors are annoying, but in this case I'm mostly just happy to hear that it's not my transient error. 😆

johnaohara · 2024-04-22T12:01:21Z

@barreiro Where are we with a review of this PR?

barreiro · 2024-04-23T11:00:10Z

@dbutenhof thanks a lot for your contribution!

johnaohara · 2024-04-23T11:24:09Z

@dbutenhof thank you very much for the contribution!

johnaohara requested changes Feb 21, 2024

View reviewed changes

johnaohara assigned dbutenhof Feb 21, 2024

dbutenhof commented Feb 21, 2024

View reviewed changes

dbutenhof force-pushed the slack branch from 6c839b4 to 5b0c77d Compare February 28, 2024 21:08

dbutenhof force-pushed the slack branch from 5b0c77d to e83d514 Compare March 20, 2024 14:40

dbutenhof force-pushed the slack branch from 7a5394e to aeab399 Compare April 15, 2024 15:41

dbutenhof marked this pull request as ready for review April 15, 2024 15:57

johnaohara requested a review from barreiro April 16, 2024 09:45

barreiro requested changes Apr 17, 2024

View reviewed changes

dbutenhof commented Apr 17, 2024

View reviewed changes

dbutenhof force-pushed the slack branch from aeab399 to 47391c9 Compare April 17, 2024 12:36

johnaohara linked an issue Apr 22, 2024 that may be closed by this pull request

Slack integration #838

Closed

barreiro merged commit ef8cfe1 into Hyperfoil:master Apr 23, 2024
2 checks passed

dbutenhof deleted the slack branch April 25, 2024 19:02

		ObjectMapper mapper = new ObjectMapper();
		ObjectNode body = mapper.createObjectNode();

		// If only ...
		// Assert.assertTrue("No slack action errors found", errors > 0);

		ObjectMapper mapper = new ObjectMapper();
		ObjectNode config = mapper.createObjectNode()

		@@ -1 +1,2 @@
		quarkus.test.enable-callbacks-for-integration-tests=true
		quarkus.http.test-port=8081

Add Action to post message to slack #1242

Add Action to post message to slack #1242

Conversation

dbutenhof commented Feb 2, 2024 • edited Loading

Fixes Issue

Changes proposed

Check List (Check all the applicable boxes)

johnaohara left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johnaohara commented Feb 21, 2024

Fixes Issue

Changes proposed

Check List (Check all the applicable boxes)

johnaohara commented Feb 21, 2024

dbutenhof left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

johnaohara commented Feb 22, 2024

johnaohara commented Feb 22, 2024

dbutenhof commented Mar 20, 2024

johnaohara commented Apr 16, 2024

barreiro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dbutenhof Apr 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dbutenhof left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dbutenhof Apr 17, 2024 • edited Loading

Choose a reason for hiding this comment

dbutenhof commented Apr 17, 2024

johnaohara commented Apr 18, 2024

dbutenhof commented Apr 18, 2024

johnaohara commented Apr 22, 2024

barreiro commented Apr 23, 2024

johnaohara commented Apr 23, 2024

dbutenhof commented Feb 2, 2024 •

edited

Loading

dbutenhof Apr 17, 2024 •

edited

Loading

dbutenhof Apr 17, 2024 •

edited

Loading