Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update dependency torchaudio to v0.13.1 #45

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Update dependency torchaudio to v0.13.1 #45

wants to merge 1 commit into from

Conversation

renovate[bot]
Copy link
Contributor

@renovate renovate bot commented Aug 5, 2022 •
edited
Loading

This PR contains the following updates:

Package Change Age Adoption Passing Confidence
torchaudio ==0.11.0 -> ==0.13.1 age adoption passing confidence

Release Notes

pytorch/audio (torchaudio)

v0.13.1: TorchAudio 0.13.1 Release Note

Compare Source

This is a minor release, which is compatible with PyTorch 1.13.1 and includes bug fixes, improvements and documentation updates. There is no new feature added.

Bug Fix

IO

  • Make buffer size configurable in ffmpeg file object operations and set size in backend (#​2810)
  • Fix issue with the missing video frame in StreamWriter (#​2789)
  • Fix decimal FPS handling StreamWriter (#​2831)
  • Fix wrong frame allocation in StreamWriter (#​2905)
  • Fix duplicated memory allocation in StreamWriter (#​2906)

Model

Recipe

  • Fix issues in HuBERT fine-tuning recipe (#​2851)
  • Fix automatic mixed precision in HuBERT pre-training recipe (#​2854)

v0.13.0: torchaudio 0.13.0 Release Note

Compare Source

Highlights

TorchAudio 0.13.0 release includes:

  • Source separation models and pre-trained bundles (Hybrid Demucs, ConvTasNet)
  • New datasets and metadata mode for the SUPERB benchmark
  • Custom language model support for CTC beam search decoding
  • StreamWriter for audio and video encoding
[Beta] Source Separation Models and Bundles

Hybrid Demucs is a music source separation model that uses both spectrogram and time domain features. It has demonstrated state-of-the-art performance in the Sony Music DeMixing Challenge. (citation: https://arxiv.org/abs/2111.03600)

The TorchAudio v0.13 release includes the following features

  • MUSDB_HQ Dataset, which is used in Hybrid Demucs training (docs)
  • Hybrid Demucs model architecture (docs)
  • Three factory functions suitable for different sample rate ranges
  • Pre-trained pipelines (docs) and tutorial

SDR Results of pre-trained pipelines on MUSDB-HQ test set

Pipeline All Drums Bass Other Vocals
HDEMUCS_HIGH_MUSDB* 6.42 7.76 6.51 4.47 6.93
HDEMUCS_HIGH_MUSDB_PLUS** 9.37 11.38 10.53 7.24 8.32

* Trained on the training data of MUSDB-HQ dataset.
** Trained on both training and test sets of MUSDB-HQ and 150 extra songs from an internal database that were specifically produced for Meta.

Special thanks to @​adefossez for the guidance.

ConvTasNet model architecture was added in TorchAudio 0.7.0. It is the first source separation model that outperforms the oracle ideal ratio mask. In this release, TorchAudio adds the pre-trained pipeline that is trained within TorchAudio on the Libri2Mix dataset. The pipeline achieves 15.6dB SDR improvement and 15.3dB Si-SNR improvement on the Libri2Mix test set.

[Beta] Datasets and Metadata Mode for SUPERB Benchmarks

With the addition of four new audio-related datasets, there is now support for all downstream tasks in version 1 of the SUPERB benchmark. Furthermore, these datasets support metadata mode through a get_metadata function, which enables faster dataset iteration or preprocessing without the need to load or store waveforms.

Datasets with metadata functionality:

[Beta] Custom Language Model support in CTC Beam Search Decoding

In release 0.12, TorchAudio released a CTC beam search decoder with KenLM language model support. This release, there is added functionality for creating custom Python language models that are compatible with the decoder, using the torchaudio.models.decoder.CTCDecoderLM wrapper.

[Beta] StreamWriter

torchaudio.io.StreamWriter is a class for encoding media including audio and video. This can handle a wide variety of codecs, chunk-by-chunk encoding and GPU encoding.

Backward-incompatible changes

  • [BC-breaking] Fix momentum in transforms.GriffinLim (#​2568)
    The GriffinLim implementations in transforms and functional used the momentum parameter differently, resulting in inconsistent results between the two implementations. The transforms.GriffinLim usage of momentum is updated to resolve this discrepancy.
  • Make torchaudio.info decode audio to compute num_frames if it is not found in metadata (#​2740).
    In such cases, torchaudio.info may now return non-zero values for num_frames.

Bug Fixes

  • Fix random Gaussian generation (#​2639)
    torchaudio.compliance.kaldi.fbank with dither option produced a different output from kaldi because it used a skewed, rather than gaussian, distribution for dither. This is updated in this release to correctly use a random gaussian instead.
  • Update download link for speech commands (#​2777)
    The previous download link for SpeechCommands v2 did not include data for the valid and test sets, resulting in errors when trying to use those subsets. Update the download link to correctly download the whole dataset.

New Features

IO
Ops
  • Add ITU-R BS.1770-4 loudness recommendation (#​2472)
  • Add convolution operator (#​2602)
  • Add additive noise function (#​2608)
Models
Pipelines
  • Add SourceSeparationBundle to prototype (#​2440, #​2559)
  • Adding pipeline changes, factory functions to HDemucs (#​2547, #​2565)
  • Create tutorial for HDemucs (#​2572)
  • Add HDEMUCS_HIGH_MUSDB (#​2601)
  • Move SourceSeparationBundle and pre-trained ConvTasNet pipeline into Beta (#​2669)
  • Move Hybrid Demucs pipeline to beta (#​2673)
  • Update description of HDemucs pipelines
Datasets

Improvements

IO
Ops
  • Speed up resample with kernel generation modification (#​2553, #​2561)
    The kernel generation for resampling is optimized in this release. The following table illustrates the performance improvements from the previous release for the torchaudio.functional.resample function using the sinc resampling method, on float32 tensor with two channels and one second duration.

CPU

torchaudio version 8k → 16k [Hz] 16k → 8k 16k → 44.1k 44.1k → 16k
0.13 0.256 0.549 0.769 0.820
0.12 0.386 0.534 31.8 12.1

CUDA

torchaudio version 8k → 16k [Hz] 16k → 8k 16k → 44.1k 44.1k → 16k
0.13 0.332 0.336 0.345 0.381
0.12 0.524 0.334 64.4 22.8
  • Add normalization parameter on spectrogram and inverse spectrogram (#​2554)
  • Replace assert with raise for ops (#​2579, #​2599)
  • Replace CHECK_ by TORCH_CHECK_ (#​2582)
  • Fix argument validation in TorchAudio filtering (#​2609)
Models
Datasets
  • Replace assert with raise in datasets (#​2571)
  • Add unit test for LibriMix dataset (#​2659)
  • Add gtzan download note (#​2763)
Tutorials
Recipes
  • Use lazy import for joblib (#​2498)
  • Revise LibriSpeech Conformer RNN-T recipe (#​2535)
  • Fix bug in Conformer RNN-T recipe (#​2611)
  • Replace bg_iterator in examples (#​2645)
  • Remove obsolete examples (#​2655)
  • Fix LibriSpeech Conforner RNN-T eval script (#​2666)
  • Replace IValue::toString()->string() with IValue::toStringRef() (#​2700)
  • Improve wav2vec2/hubert model for pre-training (#​2716)
  • Improve hubert recipe for pre-training and fine-tuning (#​2744)

WER improvement on LibriSpeech dev and test sets

Viterbi (v0.12) Viterbi (v0.13) KenLM (v0.12) KenLM (v0.13)
dev-clean 10.7 10.9 4.4 4.2
dev-other 18.3 17.5 9.7 9.4
test-clean 10.8 10.9 4.4 4.4
test-other 18.5 17.8 10.1 9.5

Documentation

Examples
  • Add example for Vol transform (#​2597)
  • Add example for Vad transform (#​2598)
  • Add example for SlidingWindowCmn transform (#​2600)
  • Add example for MelScale transform (#​2616)
  • Add example for AmplitudeToDB transform (#​2615)
  • Add example for InverseMelScale transform (#​2635)
  • Add example for MFCC transform (#​2637)
  • Add example for LFCC transform (#​2640)
  • Add example for Loudness transform (#​2641)
Other

Build/CI

v0.12.1: torchaudio 0.12.1 Release Note

Compare Source

This is a minor release, which is compatible with PyTorch 1.12.1 and include small bug fixes, improvements and documentation update. There is no new feature added.

Bug Fix

  • #​2560 Fix fall back failure in sox_io backend
  • #​2588 Fix hubert fine-tuning recipe bugs

Improvement

  • #​2552 Remove unused boost source code
  • #​2527 Improve speech enhancement tutorial
  • #​2544 Update forced alignment tutorial
  • #​2595 Update data augmentation tutorial

For the full feature of v0.12, please refer to the v0.12.0 release note.

v0.12.0

Compare Source

TorchAudio 0.12.0 Release Notes

Highlights

TorchAudio 0.12.0 includes the following:

  • CTC beam search decoder
  • New beamforming modules and methods
  • Streaming API
[Beta] CTC beam search decoder

To support inference-time decoding, the release adds the wav2letter CTC beam search decoder, ported over from Flashlight (GitHub). Both lexicon and lexicon-free decoding are supported, and decoding can be done without a language model or with a KenLM n-gram language model. Compatible token, lexicon, and certain pretrained KenLM files for the LibriSpeech dataset are also available for download.

For usage details, please check out the documentation and ASR inference tutorial.

[Beta] New beamforming modules and methods

To improve flexibility in usage, the release adds two new beamforming modules under torchaudio.transforms: SoudenMVDR and RTFMVDR. They differ from MVDR mainly in that they:

  • Use power spectral density (PSD) and relative transfer function (RTF) matrices as inputs instead of time-frequency masks. The module can be integrated with neural networks that directly predict complex-valued STFT coefficients of speech and noise.
  • Add reference_channel as an input argument in the forward method to allow users to select the reference channel in model training or dynamically change the reference channel in inference.

Besides the two modules, the release adds new function-level beamforming methods under torchaudio.functional. These include

For usage details, please check out the documentation at torchaudio.transforms and torchaudio.functional and the Speech Enhancement with MVDR Beamforming tutorial.

[Beta] Streaming API

StreamReader is TorchAudio’s new I/O API. It is backed by FFmpeg† and allows users to

  • Decode various audio and video formats, including MP4 and AAC.
  • Handle various input forms, such as local files, network protocols, microphones, webcams, screen captures and file-like objects.
  • Iterate over and decode media chunk-by-chunk, while changing the sample rate or frame rate.
  • Apply various audio and video filters, such as low-pass filter and image scaling.
  • Decode video with Nvidia's hardware-based decoder (NVDEC).

For usage details, please check out the documentation and tutorials:

† To use StreamReader, FFmpeg libraries are required. Please install FFmpeg. The coverage of codecs depends on how these libraries are configured. TorchAudio official binaries are compiled to work with FFmpeg 4 libraries; FFmpeg 5 can be used if TorchAudio is built from source.

Backwards-incompatible changes

I/O
  • MP3 decoding is now handled by FFmpeg in sox_io backend. (#​2419, #​2428)
    • FFmpeg is now used as fallback in sox_io backend, and now MP3 decoding is handled by FFmpeg. To load MP3 audio with torchaudio.load, please install a compatible version of FFmpeg (Version 4 when using an official binary distribution).
    • Note that, whereas the previous MP3 decoding scheme pads the output audio, the new scheme does not. As a consequence, the new version returns shorter audio tensors.
    • torchaudio.info now returns num_frames=0 for MP3.
Models
  • Change underlying implementation of RNN-T hypothesis to tuple (#​2339)
    • In release 0.11, Hypothesis subclassed namedtuple. Containers of namedtuple instances, however, are incompatible with the PyTorch Lite Interpreter. To achieve compatibility, Hypothesis has been modified in release 0.12 to instead alias tuple. This affects RNNTBeamSearch as it accepts and returns a list of Hypothesis instances.

Bug Fixes

Ops
  • Fix return dtype in MVDR module (#​2376)
    • In release 0.11, the MVDR module converts the dtype of input spectrum to complex128 to improve the precision and robustness of downstream matrix computations. The output dtype, however, is not correctly converted back to the original dtype. In release 0.12, we fix the output dtype to be consistent with the original input dtype.
Build
  • Fix Kaldi submodule integration (#​2269)
  • Pin jinja2 version for build_docs (#​2292)
  • Use sourceforge url to fetch zlib (#​2297)

New Features

I/O
Ops
Datasets

Improvements

I/O
Ops
  • Raise error for resampling int waveform (#​2318)
  • Move multi-channel modules to a separate file (#​2382)
  • Refactor MVDR module (#​2383)
Models
  • Add an option to use Tanh instead of ReLU in RNNT joiner (#​2319)
  • Support GroupNorm and re-ordering Convolution/MHA in Conformer (#​2320)
  • Add extra arguments to hubert pretrain factory functions (#​2345)
  • Add feature_grad_mult argument to HuBERTPretrainModel (#​2335)
Datasets
  • Refactor LibriSpeech dataset (#​2387)
  • Raising RuntimeErrors when datasets missing (#​2430)
Performance
  • Make Pitchshift for faster by caching resampling kernel (#​2441)
    The following table illustrates the performance improvement over the previous release by comparing the time in msecs it takes torchaudio.transforms.PitchShift, after its first call, to perform the operation on float32 Tensor with two channels and 8000 frames, resampled to 44.1 kHz across various shifted steps.
TorchAudio Version 2 3 4 5
0.12 2.76 5 1860 223
0.11 6.71 161 8680 1450
Tests
  • Add complex dtype support in functional autograd test (#​2244)
  • Refactor torchscript consistency test in functional (#​2246)
  • Add unit tests for PyTorch Lightning modules of emformer_rnnt recipes (#​2240)
  • Refactor batch consistency test in functional (#​2245)
  • Run smoke tests on regular PRs (#​2364)
  • Refactor smoke test executions (#​2365)
  • Move seed to setup (#​2425)
  • Remove possible manual seeds from test files (#​2436)
Build
  • Revise the parameterization of third party libraries (#​2282)
  • Use zlib v1.2.12 with GitHub source (#​2300)
  • Fix ffmpeg integration for ffmpeg 5.0 (#​2326)
  • Use custom FFmpeg libraries for torchaudio binary distributions (#​2355)
  • Adding m1 builds to torchaudio (#​2421)
Other
  • Add download utility specialized for torchaudio (#​2283)
  • Use module-level __getattr__ to implement delayed initialization (#​2377)
  • Update build_doc job to use Conda CUDA package (#​2395)
  • Update I/O initialization (#​2417)
  • Add Python 3.10 (build and test) (#​2224)
  • Retrieve version from version.txt (#​2434)
  • Disable OpenMP on mac (#​2431)

Examples

Ops
Pipelines
  • Refactor pipeline_demo.py to support variant EMFORMER_RNNT bundles (#​2203)
  • Refactor eval and pipeline_demo scripts in emformer_rnnt (#​2238)
  • Refactor pipeline_demo script in emformer_rnnt recipes (#​2239)
  • Add EMFORMER_RNNT_BASE_MUSTC into pipeline demo script (#​2248)
Tests
  • Add unit tests for Emformer RNN-T LibriSpeech recipe (#​2216)
  • Add fixed random seed for Emformer RNN-T recipe test (#​2220)
Training recipes
  • Add recipe for HuBERT model pre-training (#​2143, #​2198, #​2296, #​2310, #​2311, #​2412)
  • Add HuBERT fine-tuning recipe (#​2352)
  • Refactor Emformer RNNT recipes (#​2212)
  • Fix bugs from Emformer RNN-T recipes merge (#​2217)
  • Add SentencePiece model training script for LibriSpeech Emformer RNN-T (#​2218)
  • Add training recipe for Emformer RNNT trained on MuST-C release v2.0 dataset (#​2219)
  • Refactor ArgumentParser arguments in emformer_rnnt recipes (#​2236)
  • Add shebang lines to scripts in emformer_rnnt recipes (#​2237)
  • Introduce DistributedBatchSampler (#​2299)
  • Add Conformer RNN-T LibriSpeech training recipe (#​2329)
  • Refactor LibriSpeech Conformer RNN-T recipe (#​2366)
  • Refactor LibriSpeech Lightning datamodule to accommodate different dataset implementations (#​2437)

Prototypes

Models
  • Add Conformer RNN-T model prototype (#​2322)
  • Add ConvEmformer module (streaming-capable Conformer) (#​2324, #​2358)
  • Add conv_tasnet_base factory function to prototype (#​2411)
Pipelines
  • Add EMFORMER_RNNT_BASE_MUSTC bundle to torchaudio.prototype (#​2241)

Documentation

  • Add ASR CTC decoding inference tutorial (#​2106)
  • Update context building to not delay the inference (#​2213)
  • Update online ASR tutorial (#​2226)
  • Update CTC decoder docs and add citation (#​2278)
  • [Doc] fix typo and backlink (#​2281)
  • Fix calculation of SNR value in tutorial (#​2285)
  • Add notes about prototype features in tutorials (#​2288)
  • Update README around version compatibility matrix (#​2293)
  • Update decoder pretrained lm docs (#​2291)
  • Add devices/properties badges (#​2321)
  • Fix LibriMix documentation (#​2351)
  • Update wavernn.py (#​2347)
  • Add citations for datasets (#​2371)
  • Update audio I/O tutorials (#​2385)
  • Update MVDR beamforming tutorial (#​2398)
  • Update audio feature extraction tutorial (#​2391)
  • Update audio resampling tutorial (#​2386)
  • Update audio data augmentation tutorial (#​2388)
  • Add tutorial to use NVDEC with Stream API (#​2393)
  • Expand subsections in tutorials by default (#​2397)
  • Fix documentation (#​2407)
  • Fix documentation (#​2409)
  • Dataset doc fixes (#​2426)
  • Update CTC decoder docs (#​2443)
  • Split Streaming API tutorials into two (#​2446)
  • Update HW decoding tutorial and add notes about unseekable object (#​2408)

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate renovate bot force-pushed the renovate/torchaudio-0.x branch from 32ad890 to 06b78ec Compare November 20, 2022 17:06
@renovate renovate bot changed the title Update dependency torchaudio to v0.12.1 Update dependency torchaudio to v0.13.0 Nov 20, 2022
@renovate renovate bot force-pushed the renovate/torchaudio-0.x branch from 06b78ec to 3795378 Compare December 3, 2022 03:29
@renovate renovate bot force-pushed the renovate/torchaudio-0.x branch from 3795378 to 4be71e4 Compare December 15, 2022 20:54
@renovate renovate bot changed the title Update dependency torchaudio to v0.13.0 Update dependency torchaudio to v0.13.1 Dec 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
0 participants