initial commit - integrate lighthouse #1350

chaudharydeepak · 2020-10-27T04:22:18Z

Draft PR for lighthouse integration

Relevant Issues : (compulsory)
Relevant PRs : (optional)
Type of change :
- New feature

chaudharydeepak · 2020-10-27T04:30:59Z

@ptrthomas - this is draft work - something on the lines of progress so far -

Summary -

This solution relies on lighthouse module installed on the local machine - which possibly could be dockerized for docker based images.
axe-core has a selenium driver integration library. From my understanding, Karate implements its own drivers - hence some work will be required to implement axe-core integration libraries. Otherwise axe provide node based api's - required to be installed locally - more here -> https://www.deque.com/axe/core-documentation/integrations/
Lighthouse exposes cli or node module based api's for programmatic integration - nothing possibly in devtools protocol that we can use out-of-the-box. I believe, lighthouse starts its own devtools protocol sessions and has rules builtin to create reports / numbers out of the raw data fetched from devtools session.
Google also provides lighthouse API - via PageSpeed Insights API: https://developers.google.com/speed/docs/insights/v5/about - which requires google credentials and is bit less responsive.

This is just a sample - wanted to pick your brain before going too further with this. Please let me know, Thank you for the opportunity.

chaudharydeepak · 2020-10-27T04:32:40Z

karate-core/src/main/java/com/intuit/karate/Actions.java

@@ -138,4 +139,6 @@

    void robot(String expression);

+    void lightHouseWrapper(String URL, List<String> options) throws IOException, InterruptedException;


wrapper around node installed lighthouse module. This takes the URL to analyze and is configurable [ options ].

for all driver stuff I prefer this as a driver.lighthouse() API

I know, it is a shift from how karate started, but something I think about. some thoughts here: #398

@ptrthomas - actually, this capability isn't linked to driver, hence I didn't placed it there. Lighthouse is cli based, and creates its own cdp session.

chaudharydeepak · 2020-10-27T04:33:10Z

karate-core/src/main/java/com/intuit/karate/StringUtils.java

@@ -222,4 +226,24 @@ public static String fixJavaScriptFunction(String text) {
        }
    }

+    public static class KStreamGobbler implements Runnable {


utility method to invoke lighthouse on underlying OS.

not sure I understand. is this a CLI ? do be aware of the utilities that already exist: https://stackoverflow.com/a/62911366/143475

yeah - its pretty much same thing as Karate CLI support -

chaudharydeepak · 2020-10-27T04:33:53Z

karate-core/src/main/java/com/intuit/karate/core/ScenarioContext.java

@@ -1149,4 +1151,28 @@ public void stop(StepResult lastStepResult) {
        }
    }

+    public void lightHouseWrapperAction(String URL, List<String> options) throws IOException, InterruptedException {


actual lighthouse wrapper implementation - this can be configured further for many different options.

so it IS a CLI - ! see my comment above. I think you should first get that working as a custom feature and experiment - that may be a good enough first version / experiment

yeah! actually, in that case we might not need any of this at all - since Karate CLI support is good enough to run this all -

* def proc = karate.fork('lighthouse https://github.com/login --only-categories=accessibility,performance -- output=json --emulated-form-factor=none --throttling-method=provided') * proc.waitSync() * match proc.exitCode == 0

and I hope there is support to parse the response from command to fetch scores / details for categories like accessibility / performance etc.

ok, so let us start with this simple approach then. this is a good starting point to invite more contributions - at least it helps others including me to get up and running with axe / lighthouse

ok, so anything further needed for this work? since its all pre-existing with CLI support...

yes I think for now this PR can be parked. instead I think if you create a Git example project maybe with a Docker script included that shows how you can call axe / lighthouse and integrate into a karate test via CLI - that would be a great reference that we can build on. does that make sense ?

yeah sure, will do. Thank you.

chaudharydeepak · 2020-10-27T04:35:05Z

karate-demo/src/test/java/demo/lighthouse/demo-07.feature

+    * print LHR.categories.accessibility.score
+    * print LHR.categories.performance.score
+    #* match LHR.categories.accessibility.score == 0.96
+    #* match LHR.categories.performance.score == 1


possible further work to check for > , < threshold values on accessibility / performance etc scores. Additionally, we can generate complete report.

chaudharydeepak · 2020-10-27T06:33:02Z

karate-core/src/main/java/com/intuit/karate/driver/DevToolsDriver.java

@@ -968,4 +969,29 @@ public void inputFile(String locator, String... relativePaths) {
        method("DOM.setFileInputFiles").param("files", files).param("nodeId", nodeId).send();
    }

+    public void enableAccessibility()


attempted devtools protocols around -

enable accessibility -> getFullAXTree ( this tree by itself isnt useful). Need some standard implementations to parse it.

enable audits -> this generates events around issues primarily CORS / mixed content type issues.

enable performance -> getMetrics gives current page load metrics.

All above not much useful ( except performance, which is already provided by lighthouse ).

Just wanted to demonstrate the attempt :(

hmm can you point to a sample of getFullAXTree result if really needed, we can write a custom parser. I'm confident of doing these things if it is JSON. the new rewrite code can even handle scraping HTML BTW

ptrthomas · 2020-10-27T13:15:05Z

karate-core/src/main/java/com/intuit/karate/Actions.java

@@ -138,4 +139,6 @@

    void robot(String expression);

+    void lightHouseWrapper(String URL, List<String> options) throws IOException, InterruptedException;


for all driver stuff I prefer this as a driver.lighthouse() API

I know, it is a shift from how karate started, but something I think about. some thoughts here: #398

ptrthomas · 2020-10-27T13:17:10Z

karate-core/src/main/java/com/intuit/karate/StringUtils.java

@@ -222,4 +226,24 @@ public static String fixJavaScriptFunction(String text) {
        }
    }

+    public static class KStreamGobbler implements Runnable {


not sure I understand. is this a CLI ? do be aware of the utilities that already exist: https://stackoverflow.com/a/62911366/143475

ptrthomas · 2020-10-27T13:18:03Z

karate-core/src/main/java/com/intuit/karate/core/ScenarioContext.java

@@ -1149,4 +1151,28 @@ public void stop(StepResult lastStepResult) {
        }
    }

+    public void lightHouseWrapperAction(String URL, List<String> options) throws IOException, InterruptedException {


so it IS a CLI - ! see my comment above. I think you should first get that working as a custom feature and experiment - that may be a good enough first version / experiment

ptrthomas · 2020-10-27T13:19:38Z

karate-core/src/main/java/com/intuit/karate/driver/DevToolsDriver.java

@@ -968,4 +969,29 @@ public void inputFile(String locator, String... relativePaths) {
        method("DOM.setFileInputFiles").param("files", files).param("nodeId", nodeId).send();
    }

+    public void enableAccessibility()


hmm can you point to a sample of getFullAXTree result if really needed, we can write a custom parser. I'm confident of doing these things if it is JSON. the new rewrite code can even handle scraping HTML BTW

initial commit - integrate lighthouse

d4bcda6

chaudharydeepak commented Oct 27, 2020

View reviewed changes

attempted devtools protocols - not enough though

f3b5536

chaudharydeepak commented Oct 27, 2020

View reviewed changes

ptrthomas requested changes Oct 27, 2020

View reviewed changes

chaudharydeepak closed this Oct 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

initial commit - integrate lighthouse #1350

initial commit - integrate lighthouse #1350

chaudharydeepak commented Oct 27, 2020

chaudharydeepak commented Oct 27, 2020 •

edited

Loading

chaudharydeepak Oct 27, 2020

ptrthomas Oct 27, 2020

chaudharydeepak Oct 27, 2020

chaudharydeepak Oct 27, 2020

ptrthomas Oct 27, 2020

chaudharydeepak Oct 27, 2020

chaudharydeepak Oct 27, 2020

ptrthomas Oct 27, 2020

chaudharydeepak Oct 27, 2020 •

edited

Loading

ptrthomas Oct 28, 2020

chaudharydeepak Oct 28, 2020

ptrthomas Oct 28, 2020

chaudharydeepak Oct 28, 2020

chaudharydeepak Oct 27, 2020

chaudharydeepak Oct 27, 2020

ptrthomas Oct 27, 2020

ptrthomas Oct 27, 2020

ptrthomas Oct 27, 2020

ptrthomas Oct 27, 2020

ptrthomas Oct 27, 2020

		@@ -138,4 +139,6 @@

		void robot(String expression);

		void lightHouseWrapper(String URL, List<String> options) throws IOException, InterruptedException;

initial commit - integrate lighthouse #1350

initial commit - integrate lighthouse #1350

Conversation

chaudharydeepak commented Oct 27, 2020

chaudharydeepak commented Oct 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chaudharydeepak Oct 27, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chaudharydeepak commented Oct 27, 2020 •

edited

Loading

chaudharydeepak Oct 27, 2020 •

edited

Loading