Skip to content

Commit

Permalink
Merge pull request #14 from testable/TES-572_test_steps
Browse files Browse the repository at this point in the history 
Tes 572 test steps
  • Loading branch information
@avistramer
avistramer authored Jun 23, 2021
2 parents c417a66 + 24f6281 commit 694d975
Show file tree
Hide file tree
Showing 8 changed files with 360 additions and 59 deletions.
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,4 @@ jspm_packages
.node_repl_history

test/
.idea
71 changes: 52 additions & 19 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ A set of utility APIs for use while running [Testable](https://testable.io) scen
* [Iterate CSV](#iterate-csv)
* [Async code](#async-code)
* [Manual live event](#manual-live-event)
* [Test Framework Syntax](#test-framework-syntax)
* [Wait for finish](#wait-for-finish)
* [Webdriver.io custom commands](#webdriverio-commands)
* [Screenshots](#screenshots)
Expand Down Expand Up @@ -289,7 +290,7 @@ dataTable

#### Iterate CSV

Iterate over the CSV file, retrieving 1 or more rows. The iterator is global across the entire test execution.
Iterate over the CSV file, retrieving 1 or more rows. The iterator is global across the entire test execution.

The `next()` function takes an optional `options` object that supports the following properties:

Expand All @@ -311,7 +312,7 @@ dataTable

**Node.js Only**

When running a Node.js script on Testable and use a 3rd party module that performs async actions you might need to tell Testable when the action is finished. Testable automatically instruments many modules so you don't need to do this including async, http, https, request, net, ws, socketio, engineio, tls, setTimeout, setInterval.
When running a Node.js script on Testable and use a 3rd party module that performs async actions you might need to tell Testable when the action is finished. Testable automatically instruments many modules so you don't need to do this including async, http, https, request, net, ws, socketio, engineio, tls, setTimeout, setInterval.

For other async code use the below.

Expand Down Expand Up @@ -342,13 +343,15 @@ const execute = testableUtils.execute;
execute(function(finished) {
events.on('my-event', function(symbol) {
request.get('http://sample.testable.io/stocks/' + symbol);
events.finish()
finished();
});
});

if (fireNow) {
// trigger the event when smoke testing or run locally for testing
events.emit('my-event', 'MSFT');
// trigger the event when smoke testing or run locally for testing
events.emit('my-event', 'MSFT');
events.finish()
}
```

Expand All @@ -369,6 +372,36 @@ describe('Load Url Requested in Event', function() {
});
```

### Test Framework Syntax

Structure your tests using the familiar Mocha.js test case syntax. When run on the Testable platform you will be able to see a test case report of all test steps including pass/fail status, duration, and error details on failure.


For Example:

```javascript
const testableUtils = require('testable-utils');
const describe = testableUtils.describe;
const it = testableUtils.it;
// await if there is any async code in your tests
await describe('My example test suite', function() {
it('First test step', async function() {
await driver.get('http://www.google.com'); // Selenium javascript example code
});
});
```

Full API:

```javascript
describe(suite, fn); // fn should be a function that contains a sequence of it() blocks
it(test, fn); // fn executes a single test step
beforeEach(fn); // runs before each test step (it block)
afterEach(fn); // runs after each test step (it block)
before(fn); // runs before each suite (describe block)
after(fn); // runs after each suite (describe block)
```

### Wait For Finish

Use this function to wait for the remainder of the test duration before the script finishes. Returns a Promise so that you can run some cleanup code before the script exits.
Expand Down Expand Up @@ -406,58 +439,58 @@ One command that has no `testable-utils` equivalent is `browser.testableScreensh
<td><a href="#execution-info"><pre>info</pre></a></td>
</tr>
<tr>
<td><pre>var result =
<td><pre>var result =
browser.testableCsvGet(csvFile, index);</pre></td>
<td><a href="#get-row-by-index"><pre>dataTable
.open(csvFile)
.get(index)
.then(function(result) { ... }</pre></a></td>
</tr>
<tr>
<td><pre>var result =
<td><pre>var result =
browser.testableCsvRandom(csvFile);</pre></td>
<td><a href="#get-random-row"><pre>dataTable
.open(csvFile)
.random()
.then(function(result) { ... }</pre></a></td>
</tr>
<tr>
<td><pre>var results =
<td><pre>var results =
browser.testableCsvNext(csvFile[, options]);</pre></td>
<td><a href="#get-random-row"><pre>dataTable
.open(csvFile)
.random([options])
.then(function(results) { ... }</pre></a></td>
</tr>
<tr>
<td><pre>var result =
<td><pre>var result =
browser.testableResult([resource], [url]);
browser.testableCounter(
result,
name,
[increment],
result,
name,
[increment],
[units]);</pre></td>
<td><a href="#counter"><pre>results([resource], [url])
.counter(name, [increment], [units]);</pre></a></td>
</tr>
<tr>
<td><pre>var result =
<td><pre>var result =
browser.testableResult([resource], [url]);
browser.testableTiming(
result,
name,
timing,
result,
name,
timing,
[units]);</pre></td>
<td><a href="#timing"><pre>results([resource], [url])
.timing(name, timing, [units]);</pre></a></td>
</tr>
<tr>
<td><pre>var result =
<td><pre>var result =
browser.testableResult([resource], [url]);
browser.testableHistogram(
result,
name,
bucket,
result,
name,
bucket,
[increment]);</pre></td>
<td><a href="#histogram"><pre>results([resource], [url])
.histogram(name, bucket, [increment]);</pre></a></td>
Expand Down
22 changes: 17 additions & 5 deletions index.d.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,18 @@ declare namespace testableUtils {

function waitForFinish(): Promise<void>

function describe(suiteName: string, fn: Function): Promise<void>;

function it(testName: string, fn: Function): Promise<void>;

function before(fn: Function): void;

function after(fn: Function): void;

function beforeEach(fn: Function): void;

function afterEach(fn: Function): void;


interface Log {
debug(message?: any);
Expand All @@ -37,7 +49,7 @@ declare namespace testableUtils {
};


interface Chunk{
interface Chunk {
id: number,
executionType: string,
agent: string,
Expand All @@ -46,7 +58,7 @@ declare namespace testableUtils {
startedAt: Date,
concurrentClients: number
}
interface Execution{
interface Execution {
id: number,
createdAt: Date,
updatedAt: Date,
Expand Down Expand Up @@ -101,17 +113,17 @@ declare namespace testableUtils {
markAsSuccess(): void;
markAsFailure(): void;

data:{resource: string, url: string},
data: { resource: string, url: string },
}
interface ResultOptions {
namespace?: string;
namespace?: string;
name: string;
key?: string;
val?: number;
units?: string;
}
interface GetOptions {
namespace?: string;
namespace?: string;
name: string;
key?: string;
}
Expand Down
41 changes: 21 additions & 20 deletions lib/events-watcher.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,25 +4,26 @@ const TailIntervalMs = 100;
const TailBlockSize = 10485760;

function watchForEvents(fileName, events, log) {
const tail = new Tail(fileName, '\n', { start: 0, interval: TailIntervalMs, blockSize: TailBlockSize });
tail.on('line', function(data) {
if (!data)
return;
try {
const event = JSON.parse(data);
if (event.name) {
log.debug(`Received event ${event.name} with contents ${event.contents || ''}`);
events.emit(event.name, event.contents);
}
} catch(err) {
log.error(`Error processing live event ${data}`, helper.toErrorMessage(null, err));
}
});
tail.on('error', function(data) {
log.error('Error tailing live event', data);
});
tail.watch();
return tail;
const tail = new Tail(fileName, '\n', { start: 0, interval: TailIntervalMs, blockSize: TailBlockSize });
tail.on('line', function (data) {
if ( !data )
return;
try {
const event = JSON.parse(data);
if ( event.name ) {
log.debug(`Received event ${event.name} with contents ${event.contents || ''}`);
events.emit(event.name, event.contents);
}
} catch (err) {
log.error(`Error processing live event ${data}`, helper.toErrorMessage(null, err));
}
});
tail.on('error', function (data) {
log.error('Error tailing live event', data);
});
tail.watch();
events.finish = () => { tail.unwatch(); }
return tail;
}

module.exports = watchForEvents;
module.exports = watchForEvents;
33 changes: 22 additions & 11 deletions lib/main.js
View file
Original file line number Diff line number Diff line change
@@ -1,38 +1,38 @@

const fs = require('fs');
const _ = require('lodash');
const createLog = require('./log.js');
const createResults = require('./results.js');
const csvRemote = require('./csv-remote.js');
const csvLocal = require('./csv-local.js');
const createLog = require('./log');
const createResults = require('./results');
const csvRemote = require('./csv-remote');
const csvLocal = require('./csv-local');
const wdio = require('./wdio-commands');
const eventsWatcher = require('./events-watcher');
const EventEmitter = require('events');
const TestSteps = require('./testable-test');

const LocalInfo = {
expectedFinishTimestamp: -1,
iteration: 0,
client: 0,
globalClientIndex: 0,
regionalClientIndex: 0,
chunk: {
chunk: {
id: -1,
executionType: 'Main',
agent: 'local',
createdAt: Date.now(),
updatedAt: Date.now(),
startedAt: Date.now(),
concurrentClients: 1
concurrentClients: 1
},
agent: 'local',
execution: {
execution: {
id: -1,
createdAt: Date.now(),
updatedAt: Date.now(),
startedAt: Date.now(),
concurrentClients: 1
},
region: {
region: {
id: 1,
createdAt: Date.now(),
updatedAt: Date.now(),
Expand All @@ -41,7 +41,7 @@ const LocalInfo = {
latitude: 39.0436,
longitude: -77.4878,
description: 'Local',
active: true
active: true
},
outputDir: process.cwd()
};
Expand Down Expand Up @@ -112,7 +112,9 @@ const registerCommands = typeof browser !== 'undefined' && _.isUndefined(browser

const events = new EventEmitter();
if (process.env.TESTABLE_EVENTS_FILE) {
events.once('newListener', () => { // i.e. run this init code one time upon a new listener being registered
eventsWatcher(process.env.TESTABLE_EVENTS_FILE, events, log);
})
}

if (registerCommands) {
Expand All @@ -133,6 +135,8 @@ if (registerCommands) {
});
}

const testSteps = new TestSteps();

module.exports.isLocal = isLocal;
module.exports.isSmokeTest = isSmokeTest;
module.exports.info = info;
Expand All @@ -142,4 +146,11 @@ module.exports.stopwatch = stopwatch;
module.exports.execute = execute;
module.exports.events = events;
module.exports.dataTable = csv;
module.exports.waitForFinish = waitForFinish;
module.exports.waitForFinish = waitForFinish;

module.exports.describe = testSteps.describe.bind(testSteps);
module.exports.it = testSteps.it.bind(testSteps);
module.exports.before = testSteps.before.bind(testSteps);
module.exports.after = testSteps.after.bind(testSteps);
module.exports.beforeEach = testSteps.beforeEach.bind(testSteps);
module.exports.afterEach = testSteps.afterEach.bind(testSteps);
Loading

0 comments on commit 694d975

Please sign in to comment.