Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ring buffers to AudioWorklet processors to support variable buffer sizes #376

Merged
merged 6 commits into from
Aug 26, 2019
Merged

Add ring buffers to AudioWorklet processors to support variable buffer sizes #376

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
49ada2a
Use a ring buffer to allow for a buffer size of 1024 in p5.SoundRecor…
oshoham Aug 7, 2019
515b366
Add ring buffer to p5.SoundFile AudioWorklet processor
oshoham Aug 7, 2019
fb5e96f
Add ring buffer to p5.Amplitude AudioWorklet processor
oshoham Aug 7, 2019
226a237
Update build with new ring buffer functionality
oshoham Aug 7, 2019
d423aec
Make sure that polyfill-ed worketl nodes never have a smaller buffer …
oshoham Aug 21, 2019
2014fbc
extract buffer size calculation into shared method
oshoham Aug 23, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 44 additions & 9 deletions lib/p5.sound.js
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lib/p5.sound.js.map
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lib/p5.sound.min.js
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion lib/p5.sound.min.js.map
View file
Open in desktop

Large diffs are not rendered by default.

11 changes: 11 additions & 0 deletions src/amplitude.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,17 @@ define(function (require) {
}
}.bind(this);

// if the AudioWorkletNode is actually a ScriptProcessorNode created via polyfill,
// make sure that our chosen buffer size isn't smaller than the buffer size automatically
// selected by the polyfill
// reference: https://github.com/GoogleChromeLabs/audioworklet-polyfill/issues/13#issuecomment-425014930
if (this._workletNode instanceof ScriptProcessorNode) {
this._workletNode.port.postMessage({
name: 'bufferSize',
bufferSize: Math.max(this.bufferSize, this._workletNode.bufferSize)
});
}

// for connections
this.input = this._workletNode;

Expand Down
14 changes: 10 additions & 4 deletions src/audioWorklet/amplitudeProcessor.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,8 @@ class AmplitudeProcessor extends AudioWorkletProcessor {
this.numInputChannels = processorOptions.numInputChannels || 2;
this.normalize = processorOptions.normalize || false;
this.smoothing = processorOptions.smoothing || 0;
this.bufferSize = processorOptions.bufferSize || 2048;

this.inputRingBuffer = new RingBuffer(this.bufferSize, this.numInputChannels);
this.outputRingBuffer = new RingBuffer(this.bufferSize, this.numOutputChannels);
this.inputRingBufferArraySequence = new Array(this.numInputChannels).fill(null).map(() => new Float32Array(this.bufferSize));
this.setBufferSize(processorOptions.bufferSize || 2048);

this.stereoVol = [0, 0];
this.stereoVolNorm = [0, 0];
Expand All @@ -28,10 +25,19 @@ class AmplitudeProcessor extends AudioWorkletProcessor {
this.normalize = data.normalize;
} else if (data.name === 'smoothing') {
this.smoothing = Math.max(0, Math.min(1, data.smoothing));
} else if (data.name === 'bufferSize') {
this.setBufferSize(data.bufferSize);
}
};
}

setBufferSize(bufferSize) {
this.bufferSize = bufferSize;
this.inputRingBuffer = new RingBuffer(this.bufferSize, this.numInputChannels);
this.outputRingBuffer = new RingBuffer(this.bufferSize, this.numOutputChannels);
this.inputRingBufferArraySequence = new Array(this.numInputChannels).fill(null).map(() => new Float32Array(this.bufferSize));
}

// TO DO make this stereo / dependent on # of audio channels
process(inputs, outputs) {
const input = inputs[0];
Expand Down
7 changes: 7 additions & 0 deletions src/audioWorklet/recorderProcessor.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,17 @@ class RecorderProcessor extends AudioWorkletProcessor {
this.record(data.duration);
} else if (data.name === 'stop') {
this.stop();
} else if (data.name === 'bufferSize') {
this.setBufferSize(data.bufferSize);
}
};
}

setBufferSize(bufferSize) {
this.bufferSize = bufferSize;
this.clear();
}

process(inputs) {
if (!this.recording) {
return true;
Expand Down
13 changes: 12 additions & 1 deletion src/audioWorklet/soundFileProcessor.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,18 @@ class SoundFileProcessor extends AudioWorkletProcessor {
super();

const processorOptions = options.processorOptions || {};
this.bufferSize = processorOptions.bufferSize || 256;
this.setBufferSize(processorOptions.bufferSize || 256);

this.port.onmessage = (event) => {
const data = event.data;
if (data.name === 'bufferSize') {
this.setBufferSize(data.bufferSize);
}
};
}

setBufferSize(bufferSize) {
this.bufferSize = bufferSize;
this.inputRingBuffer = new RingBuffer(this.bufferSize, 1);
this.inputRingBufferArraySequence = [new Float32Array(this.bufferSize)];
}
Expand Down
Empty file removed src/customAudioNode.js
View file
Open in desktop
Empty file.
15 changes: 14 additions & 1 deletion src/soundRecorder.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,13 @@ define(function (require) {
this._inputChannels = 2;
this._outputChannels = 2; // stereo output, even if input is mono

const workletBufferSize = 1024;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could determine the proper buffer size here, and abstract the code to do it into a shared method. Maybe something like

safeBufferSize(idealBufferSize) {
  if (<AudioWorklet is not supported>) return idealBufferSize
  else createScriptProcessorNode --> return its bufferSize
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@therewasaguy I've extracted this logic into a shared method in helpers.js: 2014fbc#diff-b38f7254be1a9f817fd2e33c294481a1R305


this._workletNode = new AudioWorkletNode(ac, processorNames.recorderProcessor, {
outputChannelCount: [this._outputChannels],
processorOptions: {
numInputChannels: this._inputChannels,
bufferSize: 1024
bufferSize: workletBufferSize
}
});

Expand All @@ -101,6 +103,17 @@ define(function (require) {
}
}.bind(this);

// if the AudioWorkletNode is actually a ScriptProcessorNode created via polyfill,
// make sure that our chosen buffer size isn't smaller than the buffer size automatically
// selected by the polyfill
// reference: https://github.com/GoogleChromeLabs/audioworklet-polyfill/issues/13#issuecomment-425014930
if (this._workletNode instanceof ScriptProcessorNode) {
this._workletNode.port.postMessage({
name: 'bufferSize',
bufferSize: Math.max(workletBufferSize, this._workletNode.bufferSize)
});
}

/**
* callback invoked when the recording is over
* @private
Expand Down
15 changes: 14 additions & 1 deletion src/soundfile.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1238,13 +1238,15 @@ define(function (require) {
var now = ac.currentTime;
var cNode = ac.createBufferSource();

const workletBufferSize = 256;

// dispose of worklet node if it already exists
if (self._workletNode) {
self._workletNode.disconnect();
delete self._workletNode;
}
self._workletNode = new AudioWorkletNode(ac, processorNames.soundFileProcessor, {
processorOptions: { bufferSize: 256 }
processorOptions: { bufferSize: workletBufferSize }
});
self._workletNode.port.onmessage = event => {
if (event.data.name === 'position') {
Expand All @@ -1259,6 +1261,17 @@ define(function (require) {
}
};

// if the AudioWorkletNode is actually a ScriptProcessorNode created via polyfill,
// make sure that our chosen buffer size isn't smaller than the buffer size automatically
// selected by the polyfill
// reference: https://github.com/GoogleChromeLabs/audioworklet-polyfill/issues/13#issuecomment-425014930
if (self._workletNode instanceof ScriptProcessorNode) {
self._workletNode.port.postMessage({
name: 'bufferSize',
bufferSize: Math.max(workletBufferSize, self._workletNode.bufferSize)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it better to pick the max, or—in cases where AudioWorklet is not supported—to always go with the script processor node buffer size?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I agree that it's probably better to always go with the script processor node buffer size - I've used that approach in my latest commit

});
}

// create counter buffer of the same length as self.buffer
cNode.buffer = _createCounterBuffer( self.buffer );

Expand Down