Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote Runner / PostMessge api #456

Open
wants to merge 58 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
614d2f5
add initial experiment
flashdesignory Aug 21, 2024
c1450fb
install & build
flashdesignory Aug 21, 2024
4bddce7
checkin
flashdesignory Aug 21, 2024
a478795
initial working poc
flashdesignory Aug 21, 2024
a7c6fb0
cleanup
flashdesignory Aug 21, 2024
28354b4
feedback
flashdesignory Aug 22, 2024
d1cff87
more feedback
flashdesignory Aug 22, 2024
ee81ce3
simplify
flashdesignory Oct 7, 2024
0da54a0
format
flashdesignory Oct 7, 2024
627376c
simplify
flashdesignory Oct 8, 2024
6558c87
format
flashdesignory Oct 8, 2024
a5cf90b
undo change
flashdesignory Oct 8, 2024
09cf75d
Merge branch 'main' into feature/postMessage
flashdesignory Oct 8, 2024
25a8071
feedback
flashdesignory Oct 8, 2024
33c3f52
pass along params
flashdesignory Oct 8, 2024
7d520e3
cleanup
flashdesignory Oct 9, 2024
c10ac0c
simplify tests
flashdesignory Oct 9, 2024
482feec
update subscribeOnce
flashdesignory Oct 9, 2024
8a91e44
small refactor
flashdesignory Oct 9, 2024
7c2febb
cleanup handler
flashdesignory Oct 9, 2024
71368c4
Merge branch 'main' into feature/postMessage
flashdesignory Oct 21, 2024
76db526
remote runner class
flashdesignory Oct 21, 2024
5eadca4
remove empty line
flashdesignory Oct 21, 2024
26f3e04
remove empty line
flashdesignory Oct 21, 2024
a8bc24e
add extra file
flashdesignory Oct 21, 2024
36b4dd5
cleanup
flashdesignory Oct 21, 2024
47beb33
more explanation
flashdesignory Oct 21, 2024
2e1e00f
testinvoker params
flashdesignory Oct 22, 2024
a9e386e
add suite.type
flashdesignory Oct 22, 2024
75f9c71
adjust naming
flashdesignory Oct 22, 2024
2c49e18
add invoker lookup
flashdesignory Oct 22, 2024
1186f4b
Merge branch 'main' into feature/postMessage
flashdesignory Nov 5, 2024
de2c065
merged main
flashdesignory Nov 5, 2024
f7caf32
simplify sendmessage
flashdesignory Nov 7, 2024
35beb33
adjust sendMessage
flashdesignory Nov 7, 2024
ae76076
adjust workload
flashdesignory Nov 7, 2024
9c8178f
Merge branch 'main' into feature/postMessage
flashdesignory Nov 13, 2024
b8d2ec7
Merge branch 'main' into feature/postMessage
flashdesignory Nov 13, 2024
b8ec788
use test-invoker
flashdesignory Nov 13, 2024
671998e
rename files
flashdesignory Nov 14, 2024
4cbb74c
Merge branch 'main' into feature/postMessage
flashdesignory Nov 15, 2024
baaebd8
cleanup
flashdesignory Nov 15, 2024
3aaa5e0
use test-runner in remote workload
flashdesignory Nov 15, 2024
a698688
remove stub for async runner
flashdesignory Nov 15, 2024
eddd017
remove old code block
flashdesignory Nov 17, 2024
ee3ceb4
install single files
flashdesignory Nov 19, 2024
849fddd
refactor
flashdesignory Nov 21, 2024
96353b4
cleanup
flashdesignory Nov 21, 2024
f7038ca
update path
flashdesignory Nov 21, 2024
3310491
share params
flashdesignory Nov 22, 2024
34991de
update another path in tests
flashdesignory Nov 22, 2024
99c148a
remove dynamic import
flashdesignory Nov 22, 2024
67820e8
pr feedback
flashdesignory Dec 2, 2024
28da4e9
Update resources/shared/workload-testing-utils.mjs
flashdesignory Dec 2, 2024
147ec2c
cleanup
flashdesignory Dec 2, 2024
4bfff88
rebuild workload
flashdesignory Dec 2, 2024
7ec1fe7
switch to useEffect
flashdesignory Dec 2, 2024
2ced0b1
run format
flashdesignory Dec 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion resources/benchmark-runner.mjs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { Metric } from "./metric.mjs";
import { params } from "./params.mjs";
import { params } from "./shared/params.mjs";
import { SUITE_RUNNER_LOOKUP } from "./suite-runner.mjs";

const performance = globalThis.performance;
Expand Down
2 changes: 1 addition & 1 deletion resources/developer-mode.mjs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { Suites, Tags } from "./tests.mjs";
import { params, defaultParams } from "./params.mjs";
import { params, defaultParams } from "./shared/params.mjs";

export function createDeveloperModeContainer() {
const container = document.createElement("div");
Expand Down
4 changes: 2 additions & 2 deletions resources/main.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import { BenchmarkRunner } from "./benchmark-runner.mjs";
import * as Statistics from "./statistics.mjs";
import { Suites } from "./tests.mjs";
import { renderMetricView } from "./metric-ui.mjs";
import { params } from "./params.mjs";
import { params } from "./shared/params.mjs";
import { createDeveloperModeContainer } from "./developer-mode.mjs";

// FIXME(camillobruni): Add base class
Expand Down Expand Up @@ -94,7 +94,7 @@ class MainBenchmarkClient {

const enabledSuites = Suites.filter((suite) => !suite.disabled);
const totalSubtestsCount = enabledSuites.reduce((testsCount, suite) => {
return testsCount + suite.tests.length;
return testsCount + (suite.tests?.length ?? suite.config?.steps);
}, 0);
this.stepCount = params.iterationCount * totalSubtestsCount;
this._progressCompleted.max = this.stepCount;
Expand Down
2 changes: 1 addition & 1 deletion resources/newssite/news-next/dist/404.html
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="./_next/static/css/a0dca1379a01e5cf.css" as="style"/><link rel="stylesheet" href="./_next/static/css/a0dca1379a01e5cf.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="./_next/static/chunks/polyfills-c67a75d1b6f99dc8.js"></script><script src="./_next/static/chunks/webpack-e50e9853db18b759.js" defer=""></script><script src="./_next/static/chunks/framework-2c79e2a64abdb08b.js" defer=""></script><script src="./_next/static/chunks/main-2ba37e62325cc71b.js" defer=""></script><script src="./_next/static/chunks/pages/_app-77983e68be50f72a.js" defer=""></script><script src="./_next/static/chunks/pages/_error-54de1933a164a1ff.js" defer=""></script><script src="./_next/static/ofW8d8vHz4HS9u6cDJv4X/_buildManifest.js" defer=""></script><script src="./_next/static/ofW8d8vHz4HS9u6cDJv4X/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><div id="settings-container"></div><div id="notifications-container"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"statusCode":404}},"page":"/_error","query":{},"buildId":"ofW8d8vHz4HS9u6cDJv4X","assetPrefix":".","nextExport":true,"isFallback":false,"gip":true,"scriptLoader":[]}</script></body></html>
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="./_next/static/css/a0dca1379a01e5cf.css" as="style"/><link rel="stylesheet" href="./_next/static/css/a0dca1379a01e5cf.css" data-n-g=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="./_next/static/chunks/polyfills-c67a75d1b6f99dc8.js"></script><script src="./_next/static/chunks/webpack-e50e9853db18b759.js" defer=""></script><script src="./_next/static/chunks/framework-2c79e2a64abdb08b.js" defer=""></script><script src="./_next/static/chunks/main-2ba37e62325cc71b.js" defer=""></script><script src="./_next/static/chunks/pages/_app-77983e68be50f72a.js" defer=""></script><script src="./_next/static/chunks/pages/_error-54de1933a164a1ff.js" defer=""></script><script src="./_next/static/XRziDoLlUPkrw2MQfjPik/_buildManifest.js" defer=""></script><script src="./_next/static/XRziDoLlUPkrw2MQfjPik/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><div id="settings-container"></div><div id="notifications-container"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{"statusCode":404}},"page":"/_error","query":{},"buildId":"XRziDoLlUPkrw2MQfjPik","assetPrefix":".","nextExport":true,"isFallback":false,"gip":true,"scriptLoader":[]}</script></body></html>

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Large diffs are not rendered by default.

This file was deleted.

2 changes: 1 addition & 1 deletion resources/newssite/news-next/dist/index.html
Original file line number Diff line number Diff line change
@@ -1 +1 @@
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="./_next/static/css/a0dca1379a01e5cf.css" as="style"/><link rel="stylesheet" href="./_next/static/css/a0dca1379a01e5cf.css" data-n-g=""/><link rel="preload" href="./_next/static/css/2cf5163b53bb0adb.css" as="style"/><link rel="stylesheet" href="./_next/static/css/2cf5163b53bb0adb.css" data-n-p=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="./_next/static/chunks/polyfills-c67a75d1b6f99dc8.js"></script><script src="./_next/static/chunks/webpack-e50e9853db18b759.js" defer=""></script><script src="./_next/static/chunks/framework-2c79e2a64abdb08b.js" defer=""></script><script src="./_next/static/chunks/main-2ba37e62325cc71b.js" defer=""></script><script src="./_next/static/chunks/pages/_app-77983e68be50f72a.js" defer=""></script><script src="./_next/static/chunks/743-fd706aeabb7828e3.js" defer=""></script><script src="./_next/static/chunks/pages/index-7fd4a7c8e35958df.js" defer=""></script><script src="./_next/static/ofW8d8vHz4HS9u6cDJv4X/_buildManifest.js" defer=""></script><script src="./_next/static/ofW8d8vHz4HS9u6cDJv4X/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><div id="settings-container"></div><div id="notifications-container"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/","query":{},"buildId":"ofW8d8vHz4HS9u6cDJv4X","assetPrefix":".","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>
<!DOCTYPE html><html lang="en"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width"/><meta name="next-head-count" content="2"/><link rel="preload" href="./_next/static/css/a0dca1379a01e5cf.css" as="style"/><link rel="stylesheet" href="./_next/static/css/a0dca1379a01e5cf.css" data-n-g=""/><link rel="preload" href="./_next/static/css/2cf5163b53bb0adb.css" as="style"/><link rel="stylesheet" href="./_next/static/css/2cf5163b53bb0adb.css" data-n-p=""/><noscript data-n-css=""></noscript><script defer="" nomodule="" src="./_next/static/chunks/polyfills-c67a75d1b6f99dc8.js"></script><script src="./_next/static/chunks/webpack-e50e9853db18b759.js" defer=""></script><script src="./_next/static/chunks/framework-2c79e2a64abdb08b.js" defer=""></script><script src="./_next/static/chunks/main-2ba37e62325cc71b.js" defer=""></script><script src="./_next/static/chunks/pages/_app-77983e68be50f72a.js" defer=""></script><script src="./_next/static/chunks/743-fd706aeabb7828e3.js" defer=""></script><script src="./_next/static/chunks/pages/index-2c799f1d9400a895.js" defer=""></script><script src="./_next/static/XRziDoLlUPkrw2MQfjPik/_buildManifest.js" defer=""></script><script src="./_next/static/XRziDoLlUPkrw2MQfjPik/_ssgManifest.js" defer=""></script></head><body><div id="__next"></div><div id="settings-container"></div><div id="notifications-container"></div><script id="__NEXT_DATA__" type="application/json">{"props":{"pageProps":{}},"page":"/","query":{},"buildId":"XRziDoLlUPkrw2MQfjPik","assetPrefix":".","nextExport":true,"autoExport":true,"isFallback":false,"scriptLoader":[]}</script></body></html>
18 changes: 18 additions & 0 deletions resources/newssite/news-next/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions resources/newssite/news-next/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
"react-dom": "18.2.0",
"react-router-dom": "^6.11.1",
"react-router-hash-link": "^2.4.3",
"speedometer-utils": "../../shared",
"uuid": "^9.0.0"
}
}
11 changes: 10 additions & 1 deletion resources/newssite/news-next/src/pages/index.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,19 @@
import React from "react";
import { useEffect } from "react";
import { HashRouter as Router, Routes, Route } from "react-router-dom";
import Page from "@/partials/page/page";
import Head from "next/head";
import { DataContextProvider } from "@/context/data-context";
import { connectToBenchmark } from "speedometer-utils/workload-testing-utils.mjs";
import { getBenchmarkSuitesManager } from "@/workload-test.mjs";

export default function App() {
// Using 'useLayoutEffect' here, since this will connect the workload after all DOM mutations happened.
// This ensures that all elemetns are in the DOM, prior to signaling to the Benchmark that the workload is ready to run a test suite.
useEffect(() => {
const disconnect = connectToBenchmark(getBenchmarkSuitesManager(), "news-next", 1);
return disconnect;
}, []);

return (
<>
<Head>
Expand Down
41 changes: 41 additions & 0 deletions resources/newssite/news-next/src/workload-test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
import { BenchmarkTestStep, BenchmarkTestSuite, BenchmarkSuitesManager, forceLayout, getElement } from "speedometer-utils/workload-testing-utils.mjs";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh I so want #371 to be merged finally


export function getBenchmarkSuitesManager() {
return new BenchmarkSuitesManager(window.name, [
new BenchmarkTestSuite("default", [
new BenchmarkTestStep("Navigate-to-US-page", () => {
for (let i = 0; i < 25; i++) {
getElement("#navbar-dropdown-toggle").click();
forceLayout();
getElement("#navbar-dropdown-toggle").click();
forceLayout();
}

getElement("#navbar-navlist-us-link").click();
forceLayout();
}),
new BenchmarkTestStep("Navigate-to-World-page", () => {
for (let i = 0; i < 25; i++) {
getElement("#navbar-dropdown-toggle").click();
forceLayout();
getElement("#navbar-dropdown-toggle").click();
forceLayout();
}

getElement("#navbar-navlist-world-link").click();
forceLayout();
}),
new BenchmarkTestStep("Navigate-to-Politics-page", () => {
for (let i = 0; i < 25; i++) {
getElement("#navbar-dropdown-toggle").click();
forceLayout();
getElement("#navbar-dropdown-toggle").click();
forceLayout();
}

getElement("#navbar-navlist-politics-link").click();
forceLayout();
}),
]),
]);
}
5 changes: 5 additions & 0 deletions resources/shared/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"name": "speedometer-utils",
"version": "1.0.0",
"description": "Utility files for Speedometer & Workloads"
}
15 changes: 11 additions & 4 deletions resources/params.mjs → resources/shared/params.mjs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
class Params {
export class Params {
viewport = {
width: 800,
height: 600,
Expand Down Expand Up @@ -146,21 +146,28 @@ class Params {
return shuffleSeed;
}

toSearchParams() {
toSearchParams(forRemote = false) {
const rawParams = { ...this };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we have an option object parameter instead of a boolean? it's really not clear what this does from the caller site.

(and actually from here neither, a comment would be nice)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some comments in that function.
We don't use option params in our current code base and it feels too much, just for readability.
We can rename "forRemote" or I can also add a comment on the caller site, if that helps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a comment on the caller site would be good, thanks

rawParams["viewport"] = `${this.viewport.width}x${this.viewport.height}`;

// Only returning params that are useful for the workload.
// Both, 'suites' and 'tags', are specific to the debug menu.
if (forRemote) {
delete rawParams["suites"];
delete rawParams["tags"];
}

return new URLSearchParams(rawParams).toString();
}
}

export const defaultParams = new Params();

const searchParams = new URLSearchParams(window.location.search);
const searchParams = new URLSearchParams(typeof window !== "undefined" ? window.location.search : undefined);
let maybeCustomParams = new Params();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when is it happening that window isn't present?

optional nit: use ?. operator

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next.js throws an error during the build process.
I did try the ?. operator, but the current implementation was the only way to make Next.js stop complaining.

try {
maybeCustomParams = new Params(searchParams);
} catch (e) {
console.error("Invalid URL Param", e, "\nUsing defaults as fallback:", maybeCustomParams);
alert(`Invalid URL Param: ${e}`);
}
export const params = maybeCustomParams;
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ export class TimerTestInvoker extends TestInvoker {
setTimeout(() => {
this._asyncCallback();
requestAnimationFrame(async () => {
await this._reportCallback();
resolve();
const result = await this._reportCallback();
resolve(result);
});
}, 0);
}, this._params.waitBeforeSync);
Expand All @@ -40,8 +40,8 @@ export class RAFTestInvoker extends TestInvoker {
setTimeout(() => {
this._asyncCallback();
setTimeout(async () => {
await this._reportCallback();
resolve();
const result = await this._reportCallback();
resolve(result);
}, 0);
}, 0);
});
Expand Down
File renamed without changes.
Loading
Loading