Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate subtract from if-plugins to builtins #723

Merged
merged 13 commits into from
May 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
102 changes: 102 additions & 0 deletions src/__tests__/unit/builtins/subtract.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
import {Subtract} from '../../../builtins/subtract';

import {ERRORS} from '../../../util/errors';

const {InputValidationError} = ERRORS;

describe('lib/subtract: ', () => {
describe('Subtract: ', () => {
const globalConfig = {
'input-parameters': ['cpu/energy', 'network/energy', 'memory/energy'],
'output-parameter': 'energy/diff',
};
const subtract = Subtract(globalConfig);

describe('init: ', () => {
it('successfully initalized.', () => {
expect(subtract).toHaveProperty('metadata');
expect(subtract).toHaveProperty('execute');
});
});

describe('execute(): ', () => {
it('successfully applies Subtract strategy to given input.', async () => {
expect.assertions(1);

const expectedResult = [
{
duration: 3600,
'cpu/energy': 4,
'network/energy': 2,
'memory/energy': 1,
'energy/diff': 1,
timestamp: '2021-01-01T00:00:00Z',
},
];

const result = await subtract.execute([
{
duration: 3600,
'cpu/energy': 4,
'network/energy': 2,
'memory/energy': 1,
timestamp: '2021-01-01T00:00:00Z',
},
]);

expect(result).toStrictEqual(expectedResult);
});

it('throws an error on missing params in input.', async () => {
const expectedMessage =
'Subtract: cpu/energy is missing from the input array.';

expect.assertions(1);

try {
await subtract.execute([
{
duration: 3600,
timestamp: '2021-01-01T00:00:00Z',
},
]);
} catch (error) {
expect(error).toStrictEqual(
new InputValidationError(expectedMessage)
);
}
});

it('returns a result with input params not related to energy.', async () => {
expect.assertions(1);
const newConfig = {
'input-parameters': ['carbon', 'other-carbon'],
'output-parameter': 'carbon-diff',
};
const subtract = Subtract(newConfig);

const data = [
{
duration: 3600,
timestamp: '2021-01-01T00:00:00Z',
carbon: 3,
'other-carbon': 2,
},
];
const response = await subtract.execute(data);

const expectedResult = [
{
duration: 3600,
carbon: 3,
'other-carbon': 2,
'carbon-diff': 1,
timestamp: '2021-01-01T00:00:00Z',
},
];

expect(response).toEqual(expectedResult);
});
});
});
});
1 change: 1 addition & 0 deletions src/builtins/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
export {GroupBy} from './group-by';
export {TimeSync} from './time-sync';
export {Subtract} from './subtract';
export {Coefficient} from './coefficient';
export {Multiply} from './multiply';
export {Sum} from './sum';
Expand Down
94 changes: 94 additions & 0 deletions src/builtins/subtract/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# Subtract

`subtract` is a generic plugin for doing arithmetic subtractions of two or more values in an `input` array.

You provide the names of the values you want to subtract, and a name to use to add the subtraction to the output array.

For example, you could subtract `cpu/energy` and `network/energy` and name the result `offset/energy`. `offset/energy` would then be added to every observation in your input array as the diff of `cpu/energy` and `network/energy`.

## Parameters

### Plugin config

Two parameters are required in global config: `input-parameters` and `output-parameter`.

`input-parameters`: an array of strings. Each string should match an existing key in the `inputs` array
`output-parameter`: a string defining the name to use to add the result of the diff to the output array.

### Inputs

All of `input-parameters` must be available in the input array.

## Returns

- `output-parameter`: the subtraction of all `input-parameters` with the parameter name defined by `output-parameter` in global config.

## Calculation

```pseudocode
output = input0 - input1 - input2 ... - inputN
```

## Implementation

To run the plugin, you must first create an instance of `Subtract`. Then, you can call `execute()`.

```typescript
import {Subtract} from 'builtins';

const config = {
inputParameters: ['cpu/energy', 'network/energy'],
outputParameter: 'offset/energy',
};

const subtract = Subtract(config);
const result = subtract subtract.execute([
{
duration: 3600,
timestamp: '2021-01-01T00:00:00Z',
'cpu/energy': 0.005,
'memory/energy': 0.0001,
},
]);
```

## Example manifest

IF users will typically call the plugin as part of a pipeline defined in a manifest file. In this case, instantiating the plugin is handled by and does not have to be done explicitly by the user. The following is an example manifest that calls `subtract`:

```yaml
name: subtract demo
description:
tags:
initialize:
outputs:
- yaml
plugins:
subtract:
method: Subtract
path: 'builtin'
global-config:
input-parameters: ['cpu/energy', 'network/energy']
output-parameter: 'energy/diff'
tree:
children:
child:
pipeline:
- subtract
config:
subtract:
inputs:
- timestamp: 2023-08-06T00:00
duration: 3600
cpu/energy: 0.003
network/energy: 0.001
```

You can run this example by saving it as `./examples/manifests/test/subrtact.yml` and executing the following command from the project root:

```sh
npm i -g @grnsft/if
ie --manifest /manifests/plugins/subtract.yml --output manifests/outputs/subtract.yml
```

The results will be saved to a new `yaml` file in `manifests/outputs`.
101 changes: 101 additions & 0 deletions src/builtins/subtract/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
import {z} from 'zod';

import {ERRORS} from '../../util/errors';
import {buildErrorMessage} from '../../util/helpers';
import {validate} from '../../util/validations';

import {ExecutePlugin, PluginParams} from '../../types/interface';
import {SubtractConfig} from './types';

const {InputValidationError} = ERRORS;

export const Subtract = (globalConfig: SubtractConfig): ExecutePlugin => {
const errorBuilder = buildErrorMessage(Subtract.name);
const metadata = {
kind: 'execute',
};

/**
* Checks global config value are valid.
*/
const validateGlobalConfig = () => {
const globalConfigSchema = z.object({
'input-parameters': z.array(z.string()),
'output-parameter': z.string().min(1),
});

return validate<z.infer<typeof globalConfigSchema>>(
globalConfigSchema,
globalConfig
);
};

/**
* Checks for required fields in input.
*/
const validateSingleInput = (
input: PluginParams,
inputParameters: string[]
) => {
inputParameters.forEach(metricToSubtract => {
validateParamExists(input, metricToSubtract);
validateNumericString(input[metricToSubtract]);
});

return input;
};

const validateParamExists = (input: PluginParams, param: string) => {
if (input[param] === undefined) {
throw new InputValidationError(
errorBuilder({
message: `${param} is missing from the input array`,
})
);
}
};

const validateNumericString = (str: string) => {
if (isNaN(+Number(str))) {
throw new InputValidationError(
errorBuilder({
message: `${str} is not numberic`,
})
);
}
};

/**
* Subtract items from inputParams[1..n] from inputParams[0] and write the result in a new param outputParam.
*/
const execute = (inputs: PluginParams[]): PluginParams[] => {
const {
'input-parameters': inputParameters,
'output-parameter': outputParameter,
} = validateGlobalConfig();
return inputs.map(input => {
validateSingleInput(input, inputParameters);

return {
...input,
[outputParameter]: calculateDiff(input, inputParameters),
};
});
};

/**
* Calculates the diff between the 1st item in the inputs nad the rest of the items
*/
const calculateDiff = (input: PluginParams, inputParameters: string[]) => {
const [firstItem, ...restItems] = inputParameters;
return restItems.reduce(
(accumulator, metricToSubtract) => accumulator - input[metricToSubtract],
input[firstItem] // Starting accumulator with the value of the first item
);
};

return {
metadata,
execute,
};
};
4 changes: 4 additions & 0 deletions src/builtins/subtract/types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export type SubtractConfig = {
'input-parameters': string[];
'output-parameter': string;
};