[Schema Registry Avro] Add a serialize performance test (#21057)

### Packages impacted by this PR @azure/schema-registry-avro ### Issues associated with this PR Related to #16983 ### Describe the problem that is addressed by this PR Lack of performance tests of the avro serializer ### What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen? There're many potential perf tests but I went with one that uses a single schema and serializes an array. Here is a sample output: ``` > @azure-tests/[email protected] perf-test:node > ts-node test/index.spec.ts "SerializeTest" "--warmup" "1" "--iterations" "10" "--parallel" "100" "--duration" "15" === Starting the perf test === === Creating 100 instance(s) of SerializeTest === === Versions === @azure-tests/[email protected] /home/dealmaha/schemaregistry/sdk/schemaregistry/perf-tests/schema-registry-avro ├── @azure/[email protected] -> ./../../../../common/temp/node_modules/.pnpm/@Azure [email protected]/node_modules/@azure/identity ├── @azure/[email protected] -> ./../../schema-registry-avro ├── @azure/[email protected] -> ./../../schema-registry ├── @azure/[email protected] -> ./../../../test-utils/perf === Parsed options === ┌──────────────────────────────┬──────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────┬────────────────────────────────┬──────────────┬───────────┐ │ (index) │ required │ description │ shortName │ longName │ defaultValue │ value │ ├──────────────────────────────┼──────────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┼────────────────────────────────┼──────────────┼───────────┤ │ items-count │ false │ 'Number of array items' │ 'n' │ 'items-count' │ 1000 │ 1000 │ │ help │ │ 'Shows all of the available options' │ 'h' │ 'help' │ │ undefined │ │ parallel │ │ 'How many of the same test to call at the same time' │ 'p' │ 'parallel' │ 1 │ 100 │ │ duration │ │ 'When to stop calling tests at all' │ 'd' │ 'duration' │ 10 │ 15 │ │ warmup │ │ 'Duration of warmup in seconds' │ 'w' │ 'warmup' │ 5 │ 1 │ │ iterations │ │ 'Times to repeat the whole process, after warmup' │ 'i' │ 'iterations' │ 1 │ 10 │ │ no-cleanup │ │ 'Disables test cleanup' │ │ 'no-cleanup' │ │ undefined │ │ test-proxies │ │ "URIs of TestProxy servers (separated by ';')" │ │ 'test-proxies' │ undefined │ undefined │ │ insecure │ │ 'Applied when test-proxies option is defined, connects with https(insecurely by disabling SSL validation)' │ 'ins' │ 'insecure' │ false │ false │ │ milliseconds-to-log │ │ 'Log frequency in milliseconds' │ 'mtl' │ 'milliseconds-to-log' │ 1000 │ 1000 │ │ list-transitive-dependencies │ │ 'List all dependencies, instead of only direct ones, before test run' │ 'ltd' │ 'list-transitive-dependencies' │ false │ false │ └──────────────────────────────┴──────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────┴────────────────────────────────┴──────────────┴───────────┘ === Calling globalSetup() once for (all) the instance(s) of SerializeTest === === warmup mode, iteration 1. Logs every 1s === ElapsedTime Current Total Average 00:01 1554 1554 1553.98 00:02 2 1556 1555.19 00:03 63 1619 1584.07 === warmup mode, results of iteration 1 === Completed 1,653 operations in a weighted-average of 1.04s (1,595.11 ops/s, 0.001 s/op) === test mode, iteration 1. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 1 === Completed 675,200 operations in a weighted-average of 15.00s (45,017.53 ops/s, 0.000 s/op) === test mode, iteration 2. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 2 === Completed 795,700 operations in a weighted-average of 15.00s (53,045.42 ops/s, 0.000 s/op) === test mode, iteration 3. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 3 === Completed 634,300 operations in a weighted-average of 15.01s (42,272.47 ops/s, 0.000 s/op) === test mode, iteration 4. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 4 === Completed 566,500 operations in a weighted-average of 15.00s (37,763.46 ops/s, 0.000 s/op) === test mode, iteration 5. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 5 === Completed 321,300 operations in a weighted-average of 15.00s (21,419.22 ops/s, 0.000 s/op) === test mode, iteration 6. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 6 === Completed 387,400 operations in a weighted-average of 15.01s (25,813.16 ops/s, 0.000 s/op) === test mode, iteration 7. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 7 === Completed 389,300 operations in a weighted-average of 15.00s (25,951.76 ops/s, 0.000 s/op) === test mode, iteration 8. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 8 === Completed 597,700 operations in a weighted-average of 15.00s (39,846.31 ops/s, 0.000 s/op) === test mode, iteration 9. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 9 === Completed 608,800 operations in a weighted-average of 15.00s (40,584.65 ops/s, 0.000 s/op) === test mode, iteration 10. Logs every 1s === ElapsedTime Current Total Average === test mode, results of iteration 10 === Completed 580,200 operations in a weighted-average of 15.00s (38,677.77 ops/s, 0.000 s/op) ``` ### Are there test cases added in this PR? _(If not, why?)_ Yes ### Provide a list of related PRs _(if any)_ ### Command used to generate this PR:**_(Applicable only to SDK release request PRs)_ ### Checklists - [x] Added impacted package name to the issue description - [ ] Does this PR needs any fixes in the SDK Generator?** _(If so, create an Issue in the [Autorest/typescript](https://github.com/Azure/autorest.typescript) repository and link it here)_ - [ ] Added a changelog (if necessary)
Azure · Mar 28, 2022 · 9a046e7 · 9a046e7
1 parent 2ecb6d6
commit 9a046e7
Show file tree

Hide file tree

Showing 9 changed files with 222 additions and 0 deletions.
diff --git a/common/config/rush/pnpm-lock.yaml b/common/config/rush/pnpm-lock.yaml
diff --git a/rush.json b/rush.json
@@ -851,6 +851,11 @@
       "projectFolder": "sdk/template/perf-tests/template",
       "versionPolicyName": "test"
     },
+    {
+      "packageName": "@azure-tests/perf-schema-registry-avro",
+      "projectFolder": "sdk/schemaregistry/perf-tests/schema-registry-avro",
+      "versionPolicyName": "test"
+    },
     {
       "packageName": "@azure/mixed-reality-remote-rendering",
       "projectFolder": "sdk/remoterendering/mixed-reality-remote-rendering",

diff --git a/sdk/schemaregistry/perf-tests/schema-registry-avro/README.md b/sdk/schemaregistry/perf-tests/schema-registry-avro/README.md
@@ -0,0 +1,11 @@
+# Performance Testing for Avro Serializer
+
+## Instructions
+
+1. Build the schema-registry perf tests package `rush build -t perf-schema-registry-avro`.
+2. Copy the `sample.env` file and name it as `.env`.
+3. Create an Event Hubs account and populate the `.env` file with the relevant credentials.
+4. Run the tests as follows
+
+   - detect language
+     - `npm run perf-test:node -- SerializeTest --warmup 1 --iterations 10 --parallel 100 --duration 15 --items-count 1000`
diff --git a/sdk/schemaregistry/perf-tests/schema-registry-avro/package.json b/sdk/schemaregistry/perf-tests/schema-registry-avro/package.json
@@ -0,0 +1,49 @@
+{
+  "name": "@azure-tests/perf-schema-registry-avro",
+  "sdk-type": "perf-test",
+  "version": "1.0.0",
+  "description": "",
+  "main": "",
+  "keywords": [],
+  "author": "",
+  "license": "ISC",
+  "dependencies": {
+    "@azure/schema-registry": "^1.0.0",
+    "@azure/schema-registry-avro": "^1.0.0-beta.7",
+    "@azure/identity": "^2.0.1",
+    "@azure/test-utils-perf": "^1.0.0",
+    "dotenv": "^8.2.0"
+  },
+  "devDependencies": {
+    "@types/node": "^12.0.0",
+    "eslint": "^7.15.0",
+    "prettier": "^2.5.1",
+    "rimraf": "^3.0.0",
+    "tslib": "^2.2.0",
+    "ts-node": "^10.0.0",
+    "typescript": "~4.2.0"
+  },
+  "private": true,
+  "scripts": {
+    "perf-test:node": "ts-node test/index.spec.ts",
+    "audit": "node ../../../common/scripts/rush-audit.js && rimraf node_modules package-lock.json && npm i --package-lock-only 2>&1 && npm audit",
+    "build": "npm run clean && tsc -p .",
+    "build:samples": "echo skipped",
+    "build:test": "echo skipped",
+    "check-format": "prettier --list-different --config ../../../../.prettierrc.json --ignore-path ../../../../.prettierignore \"test/**/*.ts\" \"*.{js,json}\"",
+    "clean": "rimraf dist dist-* types *.tgz *.log",
+    "format": "prettier --write --config ../../../../.prettierrc.json --ignore-path ../../../../.prettierignore \"test/**/*.ts\" \"*.{js,json}\"",
+    "integration-test:browser": "echo skipped",
+    "integration-test:node": "echo skipped",
+    "integration-test": "echo skipped",
+    "lint:fix": "eslint --no-eslintrc -c ../../../.eslintrc.internal.json package.json test --ext .ts --fix --fix-type [problem,suggestion]",
+    "lint": "eslint --no-eslintrc -c ../../../.eslintrc.internal.json package.json test --ext .ts",
+    "pack": "npm pack 2>&1",
+    "unit-test:browser": "echo skipped",
+    "unit-test:node": "echo skipped",
+    "unit-test": "echo skipped",
+    "test:browser": "echo skipped",
+    "test:node": "echo skipped",
+    "test": "echo skipped"
+  }
+}
diff --git a/sdk/schemaregistry/perf-tests/schema-registry-avro/sample.env b/sdk/schemaregistry/perf-tests/schema-registry-avro/sample.env
@@ -0,0 +1,11 @@
+SCHEMA_REGISTRY_ENDPOINT=<Endpoint URL>
+SCHEMA_REGISTRY_GROUP=<Group name for schemas in registry>
+
+# Used to authenticate using Azure AD as a service principal for role-based authentication
+# in the tokenAuth sample.
+#
+# See the documentation for `EnvironmentCredential` at the following link:
+# https://docs.microsoft.com/javascript/api/@azure/identity/environmentcredential
+AZURE_TENANT_ID=<AD tenant id or name>
+AZURE_CLIENT_ID=<ID of the user/service principal to authenticate as>
+AZURE_CLIENT_SECRET=<client secret used to authenticate to Azure AD>
diff --git a/sdk/schemaregistry/perf-tests/schema-registry-avro/test/avroSerializerTest.spec.ts b/sdk/schemaregistry/perf-tests/schema-registry-avro/test/avroSerializerTest.spec.ts
@@ -0,0 +1,60 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT license.
+
+import { PerfTest, getEnvVar } from "@azure/test-utils-perf";
+import { AvroSerializer } from "@azure/schema-registry-avro";
+import { SchemaRegistryClient } from "@azure/schema-registry";
+
+// Expects the .env file at the same level
+import * as dotenv from "dotenv";
+import { DefaultAzureCredential } from "@azure/identity";
+dotenv.config();
+
+export abstract class AvroSerializerTest<TOptions> extends PerfTest<TOptions> {
+  static schema = JSON.stringify({
+    type: "record",
+    name: "AvroPerf",
+    namespace: "performance",
+    fields: [
+      {
+        name: "name",
+        type: "string",
+      },
+      {
+        name: "favoriteNumbers",
+        type: {
+          type: "array",
+          items: "int",
+        },
+      },
+    ],
+  });
+
+  client: SchemaRegistryClient;
+  serializer: AvroSerializer;
+  groupName: string;
+
+  constructor() {
+    super();
+
+    const credential = new DefaultAzureCredential();
+
+    const fullyQualifiedNamespace = getEnvVar("SCHEMA_REGISTRY_ENDPOINT");
+    this.groupName = getEnvVar("SCHEMA_REGISTRY_GROUP");
+
+    this.client = new SchemaRegistryClient(fullyQualifiedNamespace, credential);
+    this.serializer = new AvroSerializer(this.client, {
+      groupName: this.groupName,
+    });
+  }
+
+  public async globalSetup(): Promise<void> {
+    const schemaObject = JSON.parse(AvroSerializerTest.schema);
+    await this.client.registerSchema({
+      definition: AvroSerializerTest.schema,
+      format: "Avro",
+      groupName: this.groupName,
+      name: `${schemaObject.namespace}.${schemaObject.name}`,
+    });
+  }
+}
diff --git a/sdk/schemaregistry/perf-tests/schema-registry-avro/test/index.spec.ts b/sdk/schemaregistry/perf-tests/schema-registry-avro/test/index.spec.ts
@@ -0,0 +1,14 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT license.
+
+import { PerfProgram, selectPerfTest } from "@azure/test-utils-perf";
+import { SerializeTest } from "./serialize.spec";
+
+import dotenv from "dotenv";
+dotenv.config();
+
+console.log("=== Starting the perf test ===");
+
+const perfProgram = new PerfProgram(selectPerfTest([SerializeTest]));
+
+perfProgram.run();
diff --git a/sdk/schemaregistry/perf-tests/schema-registry-avro/test/serialize.spec.ts b/sdk/schemaregistry/perf-tests/schema-registry-avro/test/serialize.spec.ts
@@ -0,0 +1,38 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT license.
+
+import { AvroSerializerTest } from "./avroSerializerTest.spec";
+import { PerfOptionDictionary } from "@azure/test-utils-perf";
+
+interface SerializePerfTestOptions {
+  "items-count": number;
+}
+
+export class SerializeTest extends AvroSerializerTest<SerializePerfTestOptions> {
+  options: PerfOptionDictionary<SerializePerfTestOptions> = {
+    "items-count": {
+      required: false,
+      description: "Number of array items",
+      shortName: "n",
+      longName: "items-count",
+      defaultValue: 1000,
+    },
+  };
+  array: number[];
+
+  constructor() {
+    super();
+    this.options = this.parsedOptions;
+    this.array = [...Array(this.options["items-count"].value).keys()];
+  }
+
+  async run(): Promise<void> {
+    await this.serializer.serializeMessageData(
+      {
+        name: "test",
+        favoriteNumbers: this.array,
+      },
+      AvroSerializerTest.schema
+    );
+  }
+}
diff --git a/sdk/schemaregistry/perf-tests/schema-registry-avro/tsconfig.json b/sdk/schemaregistry/perf-tests/schema-registry-avro/tsconfig.json
@@ -0,0 +1,11 @@
+{
+  "extends": "../../../../tsconfig.package",
+  "compilerOptions": {
+    "module": "CommonJS",
+    "declarationDir": "./typings/latest",
+    "lib": ["ES6", "ESNext.AsyncIterable"],
+    "noEmit": true
+  },
+  "compileOnSave": true,
+  "include": ["./test/**/*.ts"]
+}