Add WaveRNN Model #735

jimchen90 · 2020-06-22T13:08:25Z

This is the WaveRNN model.
Related to #446

Stack:

~~Add MelResNet Block #705, #751~~
~~Add Upsampling Block #724~~
~~Add WaveRNN Model #735~~
Add example pipeline with WaveRNN #749

jimchen90 · 2020-06-22T18:35:09Z

Three failed tests look irrelevant to this PR.

mthrok · 2020-06-23T17:28:09Z

torchaudio/models/_wavernn.py

-    It is a block used in WaveRNN. WaveRNN is based on the paper "Efficient Neural Audio Synthesis".
-    Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart,
-    Florian Stimberg, Aaron van den Oord, Sander Dieleman, Koray Kavukcuoglu. arXiv:1802.08435, 2018.
+    It is a block used in WaveRNN.


In Python, the first line has to be complete and give the summary of what it is.
See https://www.python.org/dev/peps/pep-0257/#multi-line-docstrings

Starting with "This is ..." is lengthy but not adding a value, instead, try to come up with some useful information for first time reader. In this case something like ResNet block based on "Deep Residual Learning for Image Recognition" would do.

The description has been updated. Thanks for the comment and link information.

I updated this change to each stack PR (#751, #724, #735 ). #751 and #724 are melresnet (updated) and upsampling blocks of #735.

mthrok · 2020-06-23T17:31:59Z

torchaudio/models/_wavernn.py

-        for i in range(res_blocks):
-            ResBlocks.append(_ResBlock(hidden_dims))
+        for i in range(n_res_block):
+            ResBlocks.append(_ResBlock(n_hidden))


You can use one-liner here for better readability.

ResBlocks = [_ResBlock(n_hidden) for _ in range(n_res_block)]

Use underscore as variable name when you do not use the variable.

Nice suggestion! fixed.

This part is in the the melresnet part PR #751.

mthrok · 2020-06-23T17:36:14Z

torchaudio/models/_wavernn.py

+        up_layers = []
+        for scale in upsample_scales:
+            k_size = (1, scale * 2 + 1)
+            padding = (0, scale)


defining variables like this looks uncommon yet it does not improve readability.

just put variables in keyword argument and it is readable enough.

conv = nn.Conv2d( in_channels=1, out_channels=1, kernel_size= (1, scale * 2 + 1), padding=(0, scale), bias=False)

Fixed, thanks.

This part is in the upsampling part PR #724 .

mthrok · 2020-06-23T17:38:42Z

torchaudio/models/_wavernn.py

+            x: the input sequence to the _UpsampleNetwork layer
+
+        Shape:
+            - x: :math:`(batch, freq, time)`.


In Sphinx docstring, code block is double back-ticks.

I used the math block here inspired by the format in transformer. But I agree that the code block is a good idea used here which is clear and matches other places. I will update this part.

I updated this change to each stack PR (#751, #724, #735 )

mthrok · 2020-06-23T17:40:45Z

torchaudio/models/_wavernn.py

+                                       mode='RAW')
+        >>> x = torch.rand(10, 24800, 512)
+        >>> mels = torch.rand(10, 128, 512)
+        >>> output = upsamplenetwork(x, mels)


Are these examples ran by documentation generator?
I do not see anything special about this usage, so unless it is here to be tested at documentation generation time, it is not adding much value to the documentation.

I agree that the example doesn't the relationship between x and mels. How about the following?

>>> waveform = torchaudio.load(file) # shape: batch x channel x time >>> specgram = MelSpectrogram(waveform) # shape: batch x freq x time` >>> output = upsamplenetwork(waveform, specgram) # shape: ...

Nice suggestions.

I updated this change to each stack PR (#751, #724, #735 )

mthrok · 2020-06-23T17:41:37Z

torchaudio/models/_wavernn.py

+        n_freq: the number of bins in a spectrogram (default=128)
+        n_hidden: the number of hidden dimensions (default=128)
+        n_output: the number of output dimensions (default=128)
+        mode: the type of input waveform (default='RAW')


What are the other available options for mode? Can you list all of them in docstring?

Updated in this PR.

mthrok · 2020-06-23T17:41:58Z

torchaudio/models/_wavernn.py

+        if self.mode == 'RAW':
+            self.n_classes = 2 ** n_bits
+        elif self.mode == 'MOL':
+            self.n_classes = 30


Can you throw error when mode is invalid?

Updated in this PR.

mthrok · 2020-06-23T17:44:17Z

torchaudio/models/_wavernn.py

+
+        batch_size = x.size(0)
+        h1 = torch.zeros(1, batch_size, self.n_rnn, device=x.device)
+        h2 = torch.zeros(1, batch_size, self.n_rnn, device=x.device)


dtype is missing from h1 and h2, so if float64, is passed, this will fail.

dtype is added in this PR.

mthrok · 2020-06-23T17:45:58Z

test/test_models.py

+class TestUpsampleNetwork(common_utils.TorchaudioTestCase):
+
+    def test_waveform(self):
+


For each test, can you add comment on what is tested?
Test docstring can be one line or multiple lines but it should express what you are trying to test here.
It is often common that the written test is not what was intended to, and such wrong test is very difficult to detect and fix.

Use pytest test --collect-only -v and see examples of the current tests.

Test comments added.

I updated this change to each stack PR (#751, #724, #735 )

mthrok · 2020-06-23T17:46:51Z

test/test_models.py

+class TestWaveRNN(common_utils.TorchaudioTestCase):
+
+    def test_waveform(self):
+


same here, explanation of test is required. otherwise it will be difficult to make proper changes to this test later.

Test comments added.

I updated this change to each stack PR (#751, #724, #735 )

codecov · 2020-06-25T13:48:39Z

Codecov Report

Merging #735 into master will increase coverage by 0.17%.
The diff coverage is 96.77%.

@@            Coverage Diff             @@
##           master     #735      +/-   ##
==========================================
+ Coverage   89.16%   89.34%   +0.17%     
==========================================
  Files          32       32              
  Lines        2566     2627      +61     
==========================================
+ Hits         2288     2347      +59     
- Misses        278      280       +2

Impacted Files	Coverage Δ
torchaudio/models/_wavernn.py	`98.18% <96.77%> (-1.82%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ad7f43f...1cb02fd. Read the comment docs.

mthrok · 2020-06-25T14:01:57Z

torchaudio/models/_wavernn.py

-    Nal Kalchbrenner, Erich Elsen, Karen Simonyan, Seb Noury, Norman Casagrande, Edward Lockhart,
-    Florian Stimberg, Aaron van den Oord, Sander Dieleman, Koray Kavukcuoglu. arXiv:1802.08435, 2018.
+    r"""ResNet block layer based on
+    `"Deep Residual Learning for Image Recognition" <https://arxiv.org/pdf/1512.03385.pdf>`_


short description has to fit in one line. https://www.python.org/dev/peps/pep-0257/#one-line-docstrings

"""ResNet block based on "Deep Residual Learning for Image Recognition" more detailed explanation here. """

mthrok · 2020-06-25T14:03:48Z

test/test_models.py

+        """
+        Create a tensor as the input of _MelResNet layer
+        and test if the output dimensions are correct.
+        """


Please use the same docstring notation.

"""Short description on the first line as the opening of docstring Then the more detailed explanation only if necessary. Every test should be simple so if you need to add more explanation, you should also consider the design of test and simplify it. """

Updated.

This change is updated in the melresnet PR #751 .

Thanks for also opening #751 to update this changes

mthrok · 2020-06-25T14:05:19Z

torchaudio/models/_wavernn.py

        )

    def forward(self, x: Tensor) -> Tensor:
-        r"""Pass the input through the _MelResNet layer.
-
+        r"""


What's your rational behind removing the original short description, Pass the input through the _MelResNet layer.??

When I removed it, I was thinking that this description repeats because the forward function itself means passing the input through this function. I am okay with it and this sentence has been added in each function.

I updated this change to each stack PR (#751, #724, #735 )

mthrok · 2020-06-25T14:06:30Z

torchaudio/models/_wavernn.py

+
+
+class _Stretch2d(nn.Module):
+    r"""Two-dimensional stretch layer.


I do not think stretch layer is common enough and reading the description does not help understand what it does. Can you mention the outcome of the layer?

I agree, the description of outcome has been added.

I updated this change in PR #724 .

mthrok · 2020-06-25T14:08:03Z

torchaudio/models/_wavernn.py

+
+
+class _UpsampleNetwork(nn.Module):
+    r"""Upsample block based on a stack of Conv2d and Strech2d layers.


based on a stack of Conv2d and Strech2d layers

Instead of mentioning how it's implemented internally, explain what it does.

I updated this change in PR #724 .

mthrok · 2020-06-25T14:08:34Z

torchaudio/models/_wavernn.py

+                                               n_output=128,
+                                               kernel_size=5)
+        >>> input = torch.rand(10, 128, 512)
+        >>> output = upsamplenetwork(input)


This example is missing the most important information of what the output shape looks like

The output shape is added.

I updated this change to each stack PRs (#751, #724, #735 )

mthrok · 2020-06-25T14:10:09Z

torchaudio/models/_wavernn.py

+
+class _WaveRNN(nn.Module):
+    r"""WaveRNN model based on
+    `"Efficient Neural Audio Synthesis" <https://arxiv.org/pdf/1802.08435.pdf>`_


Remove the link from the summary of docstring and put it in the detail section.

mthrok · 2020-06-25T14:12:00Z

torchaudio/models/_wavernn.py

+        elif self.mode == 'MOL':
+            self.n_classes = 30
+        else:
+            raise ValueError("Unknown input mode - {}".format(self.mode))


Adding what is expected values would improve user experience.

ValueError(f"Expected mode: `RAW` or `MOL`, but found {self.mode}")

also instead of .format, using f-string will improve the readability of code.

Updated in this PR.

vincentqb · 2020-06-25T15:26:17Z

torchaudio/models/_wavernn.py

+        >>> upsamplenetwork = _waveRNN(upsample_scales=[5,5,8],
+                                       n_bits=9,
+                                       sample_rate=24000,
+                                       hop_length=200,
+                                       n_res_block=10,
+                                       n_rnn=512,
+                                       n_fc=512,
+                                       kernel_size=5,
+                                       n_freq=128,
+                                       n_hidden=128,
+                                       n_output=128,
+                                       mode='RAW')


nit: This whole block needs to be a code block. Would it be nicer to write the following?

>>> ... = _waveRNN( >>> n_bits=9, >>> ... >>> )

In fact, can you only specify the parameters that are required and omitted optional ones in this list?

>>> upsamplenetwork = _waveRNN( >>> upsample_scales=[5,5,8], >>> n_bits=9, >>> sample_rate=24000, >>> hop_length=200, >>> )

I have added the whole block and omitted optional variables in the list.

I updated this change to each stack PR (#751, #724, #735 ).

vincentqb · 2020-06-25T15:28:23Z

torchaudio/models/_wavernn.py

+                                       mode='RAW')
+        >>> x = torch.rand(10, 24800, 512)
+        >>> mels = torch.rand(10, 128, 512)
+        >>> output = upsamplenetwork(x, mels)


I agree that the example doesn't the relationship between x and mels. How about the following?

>>> waveform = torchaudio.load(file) # shape: batch x channel x time >>> specgram = MelSpectrogram(waveform) # shape: batch x freq x time` >>> output = upsamplenetwork(waveform, specgram) # shape: ...

vincentqb · 2020-06-25T15:31:44Z

torchaudio/models/_wavernn.py

+        n_output: the number of output dimensions (default=128)
+        mode: the type of input waveform in ['RAW', 'MOL'] (default='RAW')
+
+    Examples::


nit: no needs for "::", see example

I updated this change to each stack PR (#751, #724, #735 ).

jimchen90 · 2020-07-01T18:47:35Z

torchaudio/models/_wavernn.py

+        batch_size = waveform.size(0)
+        h1 = torch.zeros(1, batch_size, self.n_rnn, dtype=waveform.dtype, device=waveform.device)
+        h2 = torch.zeros(1, batch_size, self.n_rnn, dtype=waveform.dtype, device=waveform.device)
+        mels, aux = self.upsample(specgram)


Need to add transpose here because the transpose operation in upsampling part is removed.

Since this may not be "standard" compared to other implementations, could you add a comment like this?

# output of upsample is batch x ...

vincentqb

Listed a few nits, but overall looks good :)

vincentqb · 2020-07-02T18:50:03Z

torchaudio/models/_wavernn.py

+    Examples
+        >>> wavernn = _waveRNN(upsample_scales=[5,5,8], n_bits=9, sample_rate=24000, hop_length=200)
+        >>> waveform, sample_rate = torchaudio.load(file)  # waveform shape:
+        >>> (n_batch, n_channel, (n_time - kernel_size + 1) * hop_length)


nit: presented like this, the shape looks like a new command. how about the following if the line is too long?

>>> waveform, sample_rate = torchaudio.load(file) >>> # waveform shape: (n_batch, n_channel, (n_time - kernel_size + 1) * hop_length)

Good suggestion. Updated.

vincentqb · 2020-07-02T18:50:23Z

torchaudio/models/_wavernn.py

+        >>> (n_batch, n_channel, (n_time - kernel_size + 1) * hop_length)
+        >>> specgram = MelSpectrogram(sample_rate)(waveform)  # shape: (n_batch, n_channel, n_freq, n_time)
+        >>> output = wavernn(waveform.squeeze(1), specgram.squeeze(1))  # shape:
+        >>> (n_batch, (n_time - kernel_size + 1) * hop_length, 2 ** n_bits)


nit: same

>>> output = wavernn(waveform.squeeze(1), specgram.squeeze(1)) >>> # output shape: (n_batch, (n_time - kernel_size + 1) * hop_length, 2 ** n_bits)

vincentqb · 2020-07-02T18:51:30Z

torchaudio/models/_wavernn.py

+        n_res_block: the number of ResBlock in stack (default=10)
+        n_rnn: the dimension of RNN layer (default=512)
+        n_fc: the dimension of fully connected layer (default=512)
+        kernel_size: the number of kernel size in the first Conv1d layer (default=5)


nit: see here

(Default: 5)

(similar below)

Since other blocks #751 #724 have the same format (use =). I plan to open a separate pull request to change all of them later. Now I will keep them same here.

vincentqb · 2020-07-02T18:54:54Z

torchaudio/models/_wavernn.py

+        batch_size = waveform.size(0)
+        h1 = torch.zeros(1, batch_size, self.n_rnn, dtype=waveform.dtype, device=waveform.device)
+        h2 = torch.zeros(1, batch_size, self.n_rnn, dtype=waveform.dtype, device=waveform.device)
+        mels, aux = self.upsample(specgram)


Since this may not be "standard" compared to other implementations, could you add a comment like this?

# output of upsample is batch x ...

torchaudio/models/_wavernn.py

jimchen90 · 2020-07-06T18:40:15Z

Failed tests relate to #766

vincentqb · 2020-07-06T20:40:41Z

torchaudio/models/_wavernn.py

+class _WaveRNN(nn.Module):
+    r"""WaveRNN model based on "Efficient Neural Audio Synthesis".
+
+    The paper link is `<https://arxiv.org/pdf/1802.08435.pdf>`_. The input channels of waveform


nit: For more info see the paper Efficient Neural Audio Synthesis <https://arxiv.org/pdf/1802.08435.pdf>_

Given the feedback in comment, let's make it more explicit that this version is not the original.

""" WaveRNN model based on the implementation from `fatchord <https://github.com/fatchord/WaveRNN>`_ . The original implementation was introduced in `"Efficient Neural Audio Synthesis" <https://arxiv.org/pdf/1802.08435.pdf>`_. The input channels of waveform and spectrogram have to be 1. The product of `upsample_scales` must equal `hop_length`. """

If it doesn't make the first line too long, I would merge the first and second paragraph.

Another option is to add a flag in WaveRNN init, say version that can only take the value fatchord for now, but could eventually also take the value deepmind later if/when implemented. Thoughts?

(Thanks for the feedback in comment @PetrochukM :))

torchaudio/models/_wavernn.py

vincentqb · 2020-07-07T21:24:53Z

torchaudio/models/_wavernn.py

@@ -320,5 +324,6 @@ def forward(self, waveform: Tensor, specgram: Tensor) -> Tensor:
        x = torch.cat([x, a4], dim=-1)
        x = self.fc2(x)
        x = self.relu2(x)
+        x = self.fc3(x).unsqueeze(1)


nit: let's make that a new paragraph :)

x = torch.cat([x, a4], dim=-1) x = self.fc2(x) x = self.relu2(x) x = self.fc3(x) # bring back channel dimension return x.unsqueeze(1)

The comment has been added.

vincentqb · 2020-07-07T21:25:52Z

torchaudio/models/_wavernn.py

        """

+        assert waveform.size(1) == 1, 'Require the input channel of waveform is 1'
+        assert specgram.size(1) == 1, 'Require the input channel of specgram is 1'
+        waveform, specgram = waveform.squeeze(1), specgram.squeeze(1)


nit:

# remove channel dimension until the end waveform, specgram = waveform.squeeze(1), specgram.squeeze(1)

The comment has been added.

vincentqb

Let's add the two minor comments I suggested, but otherwise LGTM!

vincentqb · 2020-07-17T14:27:10Z

torchaudio/models/_wavernn.py

+        if self.mode == 'waveform':
+            self.n_classes = 2 ** n_bits
+        elif self.mode == 'mol':
+            self.n_classes = 30


Let's replace mode and n_bits parameters simply by n_classes.

cc comment

Yes, I changed the mode to loss and replaced n_bits by n_classes in #797 .

vincentqb · 2020-07-17T14:33:43Z

torchaudio/models/_wavernn.py

+        if total_scale != self.hop_length:
+            raise ValueError(f"Expected: total_scale == hop_length, but found {total_scale} != {hop_length}")
+
+        self.upsample = _UpsampleNetwork(upsample_scales, n_res_block, n_freq, n_hidden, n_output, kernel_size)


Let's change to n_hidden_resblock and n_output_upsample. Thoughts?

I updated the name as n_hidden_resblock and n_output_melresnet in #797 .
because the n_output is used in melresenet block and melresnet block is one part of upsample block, so I use n_output_melresnet. Any suggestion?

* Add WaveRNN example This is the pipeline example based on [WaveRNN model](#735) in torchaudio. The design of this pipeline is inspired by [#632](#632). It offers a standardized implementation of WaveRNN vocoder in torchaudio. * Add utils and readme The metric logger is added based on the Wav2letter pipeline [#632](#632). It offers the way to parse the standard output as described in readme. * Add channel dimension The channel dimension of waveform in datasets is added to match the input dimensions of WaveRNN model because the channel dimensions of waveform and spectrogram are added in [this part] (https://github.com/pytorch/audio/blob/master/torchaudio/models/_wavernn.py#L281) of WaveRNN model. * Update date split and transform The design of dataset structure is discussed in [this comment](#749 (comment)). Now the dataset file has a clearer workflow after using the random-split function instead of walking through all the files. All transform functions are put together inside the transforms block. Co-authored-by: Ji Chen <[email protected]>

jimchen90 requested a review from vincentqb June 22, 2020 13:08

jimchen90 changed the title ~~WaveRNN model~~ Add WaveRNN Model Jun 22, 2020

jimchen90 requested a review from mthrok June 23, 2020 16:37

jimchen90 force-pushed the wavernn branch from cd6ebaa to bbf3003 Compare June 23, 2020 16:46

jimchen90 mentioned this pull request Jun 23, 2020

Test error in unittest_windows #741

Closed

mthrok reviewed Jun 23, 2020

View reviewed changes

jimchen90 mentioned this pull request Jun 24, 2020

Add wavernn example pipeline #749

Merged

2 tasks

jimchen90 force-pushed the wavernn branch from bbf3003 to 490081c Compare June 25, 2020 13:36

mthrok reviewed Jun 25, 2020

View reviewed changes

vincentqb reviewed Jun 25, 2020

View reviewed changes

This was referenced Jun 25, 2020

Add MelResNet Block #705

Merged

UpsampleNetwork #724

Merged

jimchen90 requested a review from mthrok June 26, 2020 13:04

jimchen90 mentioned this pull request Jun 26, 2020

Update MelResNet #751

Merged

jimchen90 commented Jul 1, 2020

View reviewed changes

jimchen90 force-pushed the wavernn branch from ce7e9d2 to a6b5b10 Compare July 1, 2020 20:04

jimchen90 marked this pull request as ready for review July 1, 2020 21:02

vincentqb reviewed Jul 2, 2020

View reviewed changes

torchaudio/models/_wavernn.py Outdated Show resolved Hide resolved

vincentqb reviewed Jul 2, 2020

View reviewed changes

torchaudio/models/_wavernn.py Outdated Show resolved Hide resolved

jimchen90 requested a review from vincentqb July 6, 2020 15:45

vincentqb reviewed Jul 6, 2020

View reviewed changes

Ji Chen added 12 commits July 6, 2020 18:16

update variable names

2f91a9b

update variable name

63391d6

add wavernn model

0346f23

update test

27e26aa

update format

6981d1c

update format

c41ac8f

update format

c343166

fix conflicts and add transpose

17455a3

import update

2c44bc7

update transpose

634bc7f

update format

6a2b8a7

update docstring

b547482

jimchen90 force-pushed the wavernn branch from 7f53c53 to b547482 Compare July 7, 2020 02:08

vincentqb reviewed Jul 7, 2020

View reviewed changes

torchaudio/models/_wavernn.py Outdated Show resolved Hide resolved

vincentqb reviewed Jul 7, 2020

View reviewed changes

torchaudio/models/_wavernn.py Show resolved Hide resolved

add n_channel in input

978e101

vincentqb reviewed Jul 7, 2020

View reviewed changes

vincentqb approved these changes Jul 7, 2020

View reviewed changes

Ji Chen added 3 commits July 7, 2020 14:32

add comment

01fbbda

update docstring

0ed6da8

update docstring

1cb02fd

jimchen90 merged commit 68cc72d into pytorch:master Jul 8, 2020

jimchen90 mentioned this pull request Jul 13, 2020

Offer pretrained wavernn model #776

Closed

vincentqb reviewed Jul 17, 2020

View reviewed changes

This was referenced Jul 17, 2020

Update variable names in wavernn model #797

Merged

Update form of default value in docstring #802

Merged

jimchen90 mentioned this pull request Jul 22, 2020

Remove underscore of wavernn model #810

Merged

		class TestUpsampleNetwork(common_utils.TorchaudioTestCase):

		def test_waveform(self):

		class TestWaveRNN(common_utils.TorchaudioTestCase):

		def test_waveform(self):



		class _Stretch2d(nn.Module):
		r"""Two-dimensional stretch layer.



		class _UpsampleNetwork(nn.Module):
		r"""Upsample block based on a stack of Conv2d and Strech2d layers.

Add WaveRNN Model #735

Add WaveRNN Model #735

Conversation

jimchen90 commented Jun 22, 2020 • edited Loading

jimchen90 commented Jun 22, 2020

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jun 25, 2020 • edited Loading

Codecov Report

mthrok Jun 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

mthrok Jun 25, 2020 • edited by jimchen90 Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

vincentqb Jun 25, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 Jun 26, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vincentqb left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimchen90 commented Jun 22, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

jimchen90 Jun 25, 2020 •

edited

Loading

jimchen90 Jun 25, 2020 •

edited

Loading

jimchen90 Jun 25, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

jimchen90 Jun 25, 2020 •

edited

Loading

jimchen90 Jun 25, 2020 •

edited

Loading

jimchen90 Jun 25, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

codecov bot commented Jun 25, 2020 •

edited

Loading

mthrok Jun 25, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

mthrok Jun 25, 2020 •

edited by jimchen90

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

vincentqb Jun 25, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

jimchen90 Jun 26, 2020 •

edited

Loading

vincentqb Jul 6, 2020 •

edited

Loading

jimchen90 Jul 17, 2020 •

edited

Loading