Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VBLOCKS-2030 | API stubs and error checking #202

Merged
merged 5 commits into from
Oct 2, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion lib/twilio.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@
* @packageDocumentation
* @internalapi
*/
import AudioProcessor from './twilio/audioprocessor';
import Call from './twilio/call';
import Device from './twilio/device';
import * as TwilioError from './twilio/errors';
import { Logger } from './twilio/log';
import { PreflightTest } from './twilio/preflight/preflight';

export { Call, Device, PreflightTest, Logger, TwilioError };
export { AudioProcessor, Call, Device, PreflightTest, Logger, TwilioError };
47 changes: 47 additions & 0 deletions lib/twilio/audiohelper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
* @module Voice
*/
import { EventEmitter } from 'events';
import AudioProcessor from './audioprocessor';
import Device from './device';
import { InvalidArgumentError, NotSupportedError } from './errors';
import Log from './log';
Expand Down Expand Up @@ -147,6 +148,11 @@ class AudioHelper extends EventEmitter {
*/
private _onActiveInputChanged: (stream: MediaStream | null) => Promise<void>;

/**
* Internal reference to the added AudioProcessor
*/
private _processor: AudioProcessor | null;

/**
* A record of unknown devices (Devices without labels)
*/
Expand Down Expand Up @@ -342,6 +348,35 @@ class AudioHelper extends EventEmitter {
});
}

/**
* Adds an {@link AudioProcessor}. If an {@link AudioProcessor} exists in the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion, the wording here seems to imply "If an AudioProcess (already) exists in the..." due to grammatical structure. My gut instinct reading this is that the functionality only works on the second or further invocation of the function, which I am assuming is not true. I'm trying to think of how I would word this... maybe something like

Adds an AudioProcessor.

Once the AudioProcessor contains at least one AudioProcessor, the AudioHelper routes the input audio stream to the AudioProcessor.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that currently we do not support multiple AudioProcessors, so maybe you'd want to reword that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what you mean by "the functionality only works on the second or further invocation of the function". This was reviewed with the whole team. And another one with you in the IDL. I'm just wondering why the confusion now? Can you elaborate?

* {@link AudioHelper}, the {@link AudioHelper} routes the input audio stream
* to the {@link AudioProcessor}.
*
* The {@link AudioHelper} guarantees that this audio stream is always
* the active input audio stream and will automatically
* update whenever the user's preference changes.
*
* Only one {@link AudioProcessor} can be added at this time.
* @param processor
*/
addProcessor(processor: AudioProcessor): void {
if (this._processor) {
throw new NotSupportedError('Adding multiple AudioProcessors is not supported at this time.');
}

if (!processor) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to do something a little stricter here like typeof processor != 'object'?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. But @manjeshbhargav 's suggestion seems better. It accounts for null.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing it more, your suggestion is better but with a check on null. Will update.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change will make sure a value of null will not trigger this error:

Suggested change
if (!processor) {
if (typeof processor === 'undefined') {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want the user to set this to null using this method. I'll check for object and null instead.

throw new InvalidArgumentError('Missing AudioProcessor argument.');
}

if (typeof processor.createProcessedStream !== 'function' ||
typeof processor.destroyProcessedStream !== 'function') {
throw new InvalidArgumentError('Missing createProcessedStream or destroyProcessedStream.');
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (typeof processor.createProcessedStream !== 'function' ||
typeof processor.destroyProcessedStream !== 'function') {
throw new InvalidArgumentError('Missing createProcessedStream or destroyProcessedStream.');
if (typeof processor.createProcessedStream !== 'function') {
throw new InvalidArgumentError('Missing createProcessedStream() method.');
}
if (typeof processor.destroyProcessedStream !== 'function') {
throw new InvalidArgumentError('Missing destroyProcessedStream() method.');

}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (typeof processor.createProcessedStream !== 'function' ||
typeof processor.destroyProcessedStream !== 'function') {
throw new InvalidArgumentError('Missing createProcessedStream or destroyProcessedStream.');
}
if (
typeof processor.createProcessedStream !== 'function' ||
typeof processor.destroyProcessedStream !== 'function'
) {
throw new InvalidArgumentError('Missing createProcessedStream or destroyProcessedStream.');
}

Nitpick


this._processor = processor;
}

/**
* Enable or disable the disconnect sound.
* @param doEnable Passing `true` will enable the sound and `false` will disable the sound.
Expand Down Expand Up @@ -372,6 +407,18 @@ class AudioHelper extends EventEmitter {
return this._maybeEnableSound(Device.SoundName.Outgoing, doEnable);
}

/**
* Removes an AudioProcessor.
* @param processor
*/
removeProcessor(processor: AudioProcessor): void {
if (this._processor !== processor) {
throw new InvalidArgumentError('Cannot remove an AudioProcessor that has not been previously added.');
}

this._processor = null;
}

/**
* Set the MediaTrackConstraints to be applied on every getUserMedia call for new input
* device audio. Any deviceId specified here will be ignored. Instead, device IDs should
Expand Down
34 changes: 34 additions & 0 deletions lib/twilio/audioprocessor.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/**
* @packageDocumentation
* @module Voice
*/

/**
* Represents an AudioProcessor object that receives an audio stream for processing.
* @publicapi
*/
interface AudioProcessor {
/**
* Called whenever the active input audio stream is updated
* and is ready for processing such as adding audio filters
* or removing background noise.
* Use this method to initiate your audio processing pipeline.
*
* @param stream The input audio stream.
* @returns The modified input audio stream after applying filters.
*/
createProcessedStream(stream: MediaStream): Promise<MediaStream>;

/**
* Called after the processed stream has been destroyed.
* This happens whenever the current input stream is updated.
* Use this method to run any necessary teardown routines
* needed by your audio processing pipeline.
*
* @param stream The torn down processed audio stream.
* @returns
*/
destroyProcessedStream(stream: MediaStream): Promise<void>;
}

export default AudioProcessor;