feat: code for whisper-large-v3 #548

stillmatic · 2023-11-07T14:59:02Z

I think these are the necessary changes

make feature_size/num_mels configurable - loaded from preprocessor_config.json, which comes from the HF repos
adds HF large-v3 conversion from @bungerr
add yue cantonese token
bumps version to 0.10.0

stillmatic · 2023-11-07T15:06:32Z

fyi @gradjitta -- using your translated model here

Purfview · 2023-11-07T15:18:01Z

Is this enough without CTranslate2 update?

stillmatic · 2023-11-07T15:25:08Z

Is this enough without OpenNMT/CTranslate2#1530?

I think that PR updates some docstrings and adds the n_mels() function, which indicates how many mels the model expects, which is helpful, but none of the logic changes inside encode. in the faster-whisper code, the feature extraction is done in Python (and already has an n_mels arg) and just passes the extracted array to CTranslate2, which works.

What would be helpful to add in CTranslate2 is the corresponding num_languages function, instead of relying on the hack here.

funboarder13920 · 2023-11-07T15:43:01Z

I don't think that propagating num_languages is necessary as it only matters for one language. We can assume that people forcing the inference to yue know that they should not use large-v2 or large-v1.
large-v3 will be set to the new default large model

yue needs to be added to _LANGUAGE_CODES

n_mels can be infered from the model params, which is why I am updating ctranslate2 to propagate this param.

stillmatic · 2023-11-07T15:51:46Z

I don't think that propagating num_languages is necessary as it only matters for one language. We can assume that people forcing the inference to yue know that they should not use large-v2 or large-v1.

v3 supports code-switching https://twitter.com/jeremyphoward/status/1721696652506100175 so multi-language support does matter

ThomasKluiters · 2023-11-07T16:09:06Z

When trying to use .transcribe(language='de') an error is returned, as the model is "not multilingual".

funboarder13920 · 2023-11-07T16:11:16Z

Can you give me precisely where in your PR the num_languages is necessary ?

I don't think your PR is complete I can only see a change that is more a warning for a user sending a not supported language into the params which would not make the inference fail.
You didn't add the new language to _LANGUAGE_CODES so your change has no effect.

A change might be required when loading the hugging face tokenizer, is the v3 compatible huggingface tokenizer available yet ?

stillmatic · 2023-11-07T16:16:43Z

When trying to use .transcribe(language='de') an error is returned, as the model is "not multilingual".

reproduced, also with another conversion: https://github.com/guillaumekln/faster-whisper/pull/549/files#diff-dff5046df32208d1eaf3b13702e9c9a4c5b44ac2451aa59a6f31aebf8b4d66e3R24

seems like the conversion is tricky to get the is_multilingual support (this logic hasn't changed in the PR)

You didn't add the new language to _LANGUAGE_CODES so your change has no effect.

https://github.com/guillaumekln/faster-whisper/blob/9c378b681c5c73795e01f7d6aaec368ece6499cf/faster_whisper/tokenizer.py#L278

funboarder13920 · 2023-11-07T16:22:24Z

In ctranslate2 is_multilingual is inferred from the size of the vocabulary, the condition is not met anymore

funboarder13920 · 2023-11-07T16:44:03Z

Can you give me precisely where in your PR the num_languages is necessary ?

stillmatic · 2023-11-07T17:57:41Z

you're right - after thinking about it more, I think it has no effect on the current release. however -- if the next version adds another language, then we will be in the same boat. I think it's useful to keep this logic here, I took it from the upstream Python repo and feel like it's more future proof. the is_multilingual logic could be similarly handled, using the same logic as upstream

funboarder13920 · 2023-11-07T18:58:17Z

In the openai repo they have the language logic because they wrapped the tiktoken tokenizer.
Faster-whisper is using the huggingface tokenizer and does not really handle the language encoding, everything is done in the hugginface tokenizer.
I think there won't be any change in the hugginface tokenizer, it won't require any parameter change either, we just need the new vocabulary file in the download folder and the tokenizer should be v3 compatible.
In the future, if openai releases a new model with different languages, just having the new vocab file and the weights in the ctranslate2 format will be enough.

I agree that is_multilingual is not very robust, especially if a model is release with less languages or with a different amount of tokens. I think that the logic should remain in ctranslate2 but could be improved if another issue comes up in the future.

I will submit a PR when the HF model will be released and after having run some tests.

Purfview · 2023-11-07T19:01:23Z

when the HF model will be released

It's now released: https://huggingface.co/openai/whisper-large-v3

bungerr · 2023-11-07T22:55:56Z

i have a working conversion uploaded to bababababooey/faster-whisper-large-v3
(see: #544 )

yeah the is_multilingual check never hits so you'll get something like The current model is English-only but the language parameter is set to 'yue'; using 'en' instead. if you try to use another lang. i just kind of lazily commented out all the multilingual checks in transcribe.py to test and it didnt break anything

i added a manual is_multilingual flag to WhisperModel() init as a temporary hack in the meantime

bungerr@8bafc9f

bungerr · 2023-11-08T03:04:54Z

okay i noticed quality with the english only model/tokenizer was significantly worse (missing punctuation, not segmenting sentences, etc) than when forcing the multilingual tokenizer, so i'm using multilingual for everything. seems like im getting the same quality as the openai whisper package now

i think the only thing missing now is getting is_multilingual properly set in the ct2 model? it breaks language detection (currently defaults to 'en'). if you try to force it, it throws a runtimeerror

  File "C:\git\faster-whisper-3\faster_whisper\transcribe.py", line 335, in transcribe
    results = self.model.detect_language(encoder_output)[0]
RuntimeError: detect_language can only be called on multilingual models

@stillmatic @funboarder13920

flyingleafe · 2023-11-08T06:09:04Z

@stillmatic Opened basically the same PR yesterday lol
I think that it is best to determine is_multilingual and num_languages, as well as mel parameters, systematically from the configs provided by HF, if those are available (first two are derived from vocab_size in config.json, the latter are given in preprocessor_config.json). The latter part of this is done in my PR, feel free to merge that in here and I will close my one, since yours was the first.

funboarder13920 · 2023-11-08T09:19:09Z

The HF model was released but the tokenizer is not in the required format and I am not able to convert it to the fast tokenizer format tokenizer.json
The pytorch_model.bin is not available either but I was able to convert the model.safetensors to the bin format.

I will wait for the official files. If people are in a rush to use large-v3 they can use any of the proposed PR

RafaRed · 2023-11-08T10:16:04Z

As I mentioned on CTranslate2/pull/1530

The current check for multilingual support seems to be hardcoded with a specific vocabulary size:
_is_multilingual = vocabulary.size() == 51865;
Link to code

For instance, I believe the whisper-latest-v3 model has a vocabulary size of 51866, which is one more than the hardcoded value. This discrepancy could lead to the multilingual feature being incorrectly disabled for this model.

Probably a more dynamic check need to be implemented to ensure compatibility with future models.

I could make the model work doing the changes suggested by @flyingleafe and removing the last line for the vocabulary.json "<|30.02|>" but, of course it will affect the model and it's not a solution.

edit: oh sorry, did not notice its already fixed on the @funboarder13920 PR.

stillmatic · 2023-11-08T14:59:42Z

cool - let's wait for @funboarder13920 PR to be merged, the multilingual fix might be the only thing necessary there?

the other question is how to handle the num_languages and related tokenizer things. I would like to get that from CTranslate2, as that would be consistent with other operations that rely on vocab size (ie the is_multilingual check).

question on the preprocessor_config.json -- can we get it from the config.json on the hub that already exists? we're already downloading it here: https://github.com/guillaumekln/faster-whisper/blob/master/faster_whisper/utils.py#L78C10-L78C16 - has num_mel_bins. let me try and update to use this file, so that we do not need to download an additional new one. -- makes sense to just use preprocessor_config, I've updated here, ty for the pointer

fyi - pytorch weights are also available on hub: https://huggingface.co/openai/whisper-large-v3/blob/main/pytorch_model.bin

funboarder13920 · 2023-11-08T15:34:15Z

I will make a num_languages available in the ctranslate2 model. I don't see when we will need the num_languages though.

stillmatic · 2023-11-08T18:19:11Z

I think we should be able to succeed now just on this ... testing on CTranslate2 head commit now

funboarder13920 · 2023-11-08T18:30:37Z

faster_whisper/utils.py

@@ -21,6 +21,7 @@
    "large-v1": "guillaumekln/faster-whisper-large-v1",
    "large-v2": "guillaumekln/faster-whisper-large-v2",
    "large": "guillaumekln/faster-whisper-large-v2",
+    "large-v3": "gradjitta/ct2-whisper-large-v3",


This repo is not official, was not generated from the official HF openai/whisper-large-v3 repo and the tokenizer.json is likely wrong, there is no yue in the languages.

which would you suggest using? @bungerr has one at bababababooey/faster-whisper-large-v3 but didn't copy the preprocessor_config.json over, which is useful to get the feature extractor setup correctly with.

oop i just added it in, had to copy the alignment heads over from there into config.json & forgot

also accidentally wiped my local openai/whisper-large-v3 repo, pulling again & will reconvert when the new wheels finish building https://github.com/OpenNMT/CTranslate2/actions/runs/6801730648

note: tokenizer.json is not provided in the official repo so if you want to run the ct2 translation script, you'll have to either use hf's openai_to_hf converter script (which may have been updated since i last checked(?)), or just do what i did and load+save from AutoTokenizer

from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("openai/whisper-large-v3") tokenizer.save_pretrained("./tokenizerjson")

funboarder13920 · 2023-11-13T09:36:48Z

I am still waiting for the large-v3 HF hub to be aligned with all the other models, specifically the tokenizer.json config before trying anything

flyingleafe · 2023-11-14T06:27:48Z

@funboarder13920 There is a very simple way to obtain the tokenizer.json file from the HF model.

from transformers import WhisperTokenizer, WhisperTokenizerFast

slow = WhisperTokenizer.from_pretrained("openai/whisper-large-v3")
fast = WhisperTokenizerFast(__slow_tokenizer=slow)
fast.save_pretrained("/path/to/dir")

This yields all the necessary files for the tokenizer including tokenizer.json

flyingleafe · 2023-11-14T06:44:55Z

@funboarder13920 alternatively you can use conversion script from my PR to transformers heh (huggingface/transformers#27336)

I pinged Sanchit there though regarding tokenizer.json in the official checkpoint to save everybody some hassle, so I think the issue should be resolved soon.

Purfview · 2023-11-14T13:50:02Z

@flyingleafe what about config.json issue mentioned there -> #548 (comment)

flyingleafe · 2023-11-15T12:02:18Z

@Purfview I figured the issue out, so basically the difference between large-v2 and large-v3 tokenizer checkpoints is that <|endoftext|> token is included in the additional special tokens list for previous checkpoints versions, but excluded from it for large-v3.

This token is BOS token so by the logic of tokenizers library it should not be in this list in the first place, so the fact it was there before is some kind of bug apparently. But CTranslate2 converter relied on this token being present in the list.

I opened the PR in CTranslate2 fixing this: OpenNMT/CTranslate2#1546

The other issue which was present in the converter is that they did not get the suppressed token values from HF model's generation_config. Those are important for proper transcription (all the faster-whisper checkpoints by @guillaumekln so far included those, and they were equivalent to HF ones). I fixed that too.
https://huggingface.co/flyingleafe/faster-whisper-large-v3 <- this checkpoint is made using those fixes, together with this PR seems to be working well. @ThomasKluiters please feel free to test your punctuation issues on this checkpoint, I suspect those could be due to nonexistent suppress_tokens set in your previous attempts.

Purfview · 2023-11-15T12:53:08Z

@flyingleafe I'm not an expert, but your fixes make sense. Are you sure that there are no other quirks with tokens?
Going to test it, I'll report later.

ThomasKluiters · 2023-11-15T13:04:25Z

@flyingleafe Thanks! I'm gonna have a look later today!

Purfview · 2023-11-16T02:52:58Z

@flyingleafe I didn't noticed any difference so far.
Btw, there is new commit: https://huggingface.co/openai/whisper-large-v3/commit/99ef777c8f3c81bade67c14cfa359a3d6303b788

flyingleafe · 2023-11-16T06:34:00Z

@Purfview can you send me some of your test samples maybe? maybe my samples are too easy, so far the results I see are virtually identical to large-v2.

the half of the new commit to the HF checkpoint makes sense (bos/eos/pad token ids are wrong indeed), but the reduction of the vocabulary size actually does not make sense, will test HF version today with and without this change to figure out why they did that.

Purfview · 2023-11-16T11:27:25Z

send me some of your test samples

Check this.

funboarder13920 · 2023-11-16T17:14:23Z

I ran some benchmarks to compare large-v2 with large-v3. large-v2 is currently way better than large-v3.
Some possible explanations:

large-v3 is not accurate
there is an issue with the conversion (openai to hf or hf to ct2)
there is an issue with the code from HF we are using
there is an issue with this code
the default inference params are different from large-v2 ?

Next week I will benchmark openai/large-v3 against hf/large-v3 against ct2/large-v3 and openai/large-v2 against openai/large-v3 to narrow down the possibilities.

flyingleafe · 2023-11-17T09:36:18Z

@funboarder13920 can confirm, on the example provided by the @Purfview large-v3 hallucinates a lot of BS in the end and in-between phrases, while large-v2 does not; no problems with punctuation though

iorilu · 2023-11-18T01:54:58Z

after this long discussion

any update on the final result

as I want to be if it's ok to update to v3, at least with no negative impact copmare to v2

iorilu · 2023-11-18T01:56:11Z

I ran some benchmarks to compare large-v2 with large-v3. large-v2 is currently way better than large-v3. Some possible explanations:

large-v3 is not accurate

there is an issue with the conversion (openai to hf or hf to ct2)

there is an issue with the code from HF we are using

there is an issue with this code

the default inference params are different from large-v2 ?

Next week I will benchmark openai/large-v3 against hf/large-v3 against ct2/large-v3 and openai/large-v2 against openai/large-v3 to narrow down the possibilities.

what language do you use to test ? test on mulitiple language or just English?

blackpolarz · 2023-11-21T07:11:05Z

From my own testing, the same problem occurs with whisper v3 translation (JP to EN) task. For most parts, whisper v3 has better grammar, but it tends to hallucinate and fill in words by itself, which can cause the context to change at times. This occurs even with the Hugging face's implementation, so I am not sure if the problem lies with the conversion from open-ai's model to hf or the model itself. Whisper v2 is slightly better in terms of "word for word" so context don't change as much but hallucination is still a problem.
I also noticed that "avg_logprob" in whisper v3 tends to be lower when testing the same audio on whisper v2.

doublex · 2023-11-22T12:15:38Z

@flyingleafe
Did you use this model in your test?
https://huggingface.co/flyingleafe/faster-whisper-large-v3

funboarder13920 · 2023-11-22T13:35:40Z

faster_whisper/transcribe.py

@@ -113,6 +114,9 @@ def __init__(
            are saved in the standard Hugging Face cache directory.
          local_files_only:  If True, avoid downloading the file and return the path to the
            local cached file if it exists.
+          feature_size: Number of mel filters to use for feature extraction. If not set,


Not used anymore

funboarder13920 · 2023-11-22T13:36:36Z

faster_whisper/transcribe.py

@@ -142,7 +146,25 @@ def __init__(
                "openai/whisper-tiny" + ("" if self.model.is_multilingual else ".en")
            )

-        self.feature_extractor = FeatureExtractor()
+        feature_extractor_file = os.path.join(model_path, "preprocessor_config.json")


Maybe move that into a specific method ?
And use the n_mels from ct2 as a fallback ?

I'll try to get to this tomorrow. extremely limited bandwidth with American holidays currently

funboarder13920 · 2023-11-22T13:38:33Z

requirements.txt

@@ -1,5 +1,5 @@
 av==10.*
-ctranslate2>=3.17,<4
+ctranslate2>=3.21,<4
 huggingface_hub>=0.13
 tokenizers>=0.13,<0.15


Add support to tokenizer 0.15 version, otherwise it is inconvenient to install faster-whisper and transformers/tokenizers

do you know if any compatibility fixes are necessary? or will just tokenizers>=0.13 suffice?

I guess Guillaume did that because ctranslate2 relies heavily on the tokenizers, to prevent a new tokenizers version to introduce breaking change into faster-whisper. We can go for tokenizers>=0.13,<0.16

We made some tests on ourside, the tokenizers 0.15 works normally, so: tokenizers>=0.13,<0.16 looks good for me.

funboarder13920 · 2023-11-22T13:39:26Z

faster_whisper/version.py

@@ -1,3 +1,3 @@
 """Version information."""

-__version__ = "0.9.0"


this commit is for the release

funboarder13920 · 2023-11-22T13:41:12Z

faster_whisper/utils.py

@@ -21,6 +21,7 @@
    "large-v1": "guillaumekln/faster-whisper-large-v1",
    "large-v2": "guillaumekln/faster-whisper-large-v2",
    "large": "guillaumekln/faster-whisper-large-v2",
+    "large-v3": "bababababooey/faster-whisper-large-v3",


who is owning that ?
Can systran create its hub to keep ownership of the models and easily update old models and upload new models ?

who is owning that ?

Some user posted that link on issues.

Best would be to keep model under official acc like "systran", maybe move all models there.

Good point, we are waiting for the release on CTranslate2, then push the new converted model tomorrow (with the fix by OpenNMT/CTranslate2#1546) to Systran organization

FYI, several models are now available on Systran organization: https://huggingface.co/Systran (including the large-v3 converted by the last CTranslate2 3.22.0)

@nguyendc-systran thank you! So essentially just waiting for @stillmatic to do the last fixes you mentioned before merge?

@Purfview I think you accidentally pasted the same URL.

@AvivSham Fixed it.

Not sure if I missed out any but in tokenizer.json, there is a difference in token 50363, nospeech vs nocaptions.
The same difference is in vocabulary.json.
ctranslate-large v2 uses nocaptions which is the same as what flyingleaf is using.
However, in hf-large-v3, it uses nospeech which is the same as what systran is using.

@nguyendc-systran thank you! So essentially just waiting for @stillmatic to do the last fixes you mentioned before merge?

IMHO, yes.
Another point may be interesting / relevant is about this benchmark: #548 (comment).
Not sure if @funboarder13920 had a chance for that?

Yep, the inference between openai/hf/faster_whisper are not exactly the same but very similar. I guess I was witnessing the differences between v2 and v3.

nguyendc-systran · 2023-11-22T17:05:06Z

requirements.txt

@@ -1,5 +1,5 @@
 av==10.*
-ctranslate2>=3.17,<4
+ctranslate2>=3.21,<4


New release 3.22.0 on Ctranslate2 is in progress: OpenNMT/CTranslate2#1561
Please update this accordingly to have this fix when converting HF to CT2

will do, please ping when it's on pypi

@stillmatic https://pypi.org/project/ctranslate2/3.22.0

nguyendc-systran · 2023-11-23T10:08:10Z

faster_whisper/transcribe.py

+                for k in [
+                    "n_fft",
+                    "hop_length",
+                    "feature_size",
+                    "sampling_rate",
+                    "chunk_length",
+                ]


Minor remark: in this new method could you make these parameters less "hard-code" way because they come from this class: https://github.com/SYSTRAN/faster-whisper/blob/master/faster_whisper/feature_extractor.py#L8-L12

Oscaarjs · 2023-11-23T13:49:01Z

@nguyendc-systran in https://huggingface.co/Systran/faster-whisper-large-v3/tree/main vocabulary is a .json but for the other models e.g. https://huggingface.co/Systran/faster-whisper-large-v2/tree/main its a .txt file. Is this intended?

minhthuc2502 · 2023-11-23T15:15:44Z

@nguyendc-systran in https://huggingface.co/Systran/faster-whisper-large-v3/tree/main vocabulary is a .json but for the other models e.g. https://huggingface.co/Systran/faster-whisper-large-v2/tree/main its a .txt file. Is this intended?

Hello, Ct2 support vocabulary in both format json and txt. Recently, after this PR all the conversions will generate vocabulary.json instead of txt. That's why in https://huggingface.co/Systran/faster-whisper-large-v3/tree/main, we have vocabulary.json

Oscaarjs · 2023-11-24T10:55:11Z

Added some changes and reflections from this thread in a new PR #578

nguyendc-systran · 2023-11-24T22:44:54Z

Hi @stillmatic, @Oscaarjs et al.
Thank you very much for the contribution and all discussion in this thread.
I take the liberty of closing this discussion as the remaining works has been done and merged by #578
Thanks both of you ;)

Purfview · 2023-11-24T22:57:26Z

Still, nospeech vs nocaptions difference in token 50363 wasn't answered.

@flyingleafe

feat: code for whisper-large-v3

9c378b6

stillmatic force-pushed the master branch from 155703f to 9c378b6 Compare November 7, 2023 15:02

stillmatic mentioned this pull request Nov 7, 2023

Will it be possible to use the large-v3 model? #544

Closed

alexivdv mentioned this pull request Nov 7, 2023

Whisper v3 m-bain/whisperX#560

Closed

stillmatic added 3 commits November 8, 2023 10:16

feat: read num_mels from config.json, revert num_lang

0b9ab4d

fix: use preprocessor_config.json instead

bb052ea

fix: lint

a477d84

fix: isort

00594e6

funboarder13920 reviewed Nov 8, 2023

View reviewed changes

Jeronymous mentioned this pull request Nov 13, 2023

Support of new Whisper model large-v3 linto-ai/faster-whisper#1

Merged

flyingleafe mentioned this pull request Nov 14, 2023

[Whisper] Add large-v3 version support huggingface/transformers#27336

Merged

5 tasks

nguyendc-systran self-assigned this Nov 21, 2023

funboarder13920 reviewed Nov 22, 2023

View reviewed changes

nguyendc-systran reviewed Nov 22, 2023

View reviewed changes

zh-plus mentioned this pull request Nov 23, 2023

TypeError: transcribe() got an unexpected keyword argument 'repetition_penalty' zh-plus/openlrc#21

Closed

nguyendc-systran reviewed Nov 23, 2023

View reviewed changes

Oscaarjs mentioned this pull request Nov 24, 2023

Add V3 Support #578

Merged

nguyendc-systran closed this Nov 24, 2023

		@@ -1,3 +1,3 @@
		"""Version information."""

		__version__ = "0.9.0"

feat: code for whisper-large-v3 #548

feat: code for whisper-large-v3 #548

Conversation

stillmatic commented Nov 7, 2023 • edited Loading

stillmatic commented Nov 7, 2023

Purfview commented Nov 7, 2023

stillmatic commented Nov 7, 2023 • edited Loading

funboarder13920 commented Nov 7, 2023

stillmatic commented Nov 7, 2023

ThomasKluiters commented Nov 7, 2023

funboarder13920 commented Nov 7, 2023

stillmatic commented Nov 7, 2023

funboarder13920 commented Nov 7, 2023 • edited Loading

funboarder13920 commented Nov 7, 2023

stillmatic commented Nov 7, 2023

funboarder13920 commented Nov 7, 2023

Purfview commented Nov 7, 2023

bungerr commented Nov 7, 2023 • edited Loading

bungerr commented Nov 8, 2023

flyingleafe commented Nov 8, 2023

funboarder13920 commented Nov 8, 2023

RafaRed commented Nov 8, 2023 • edited Loading

stillmatic commented Nov 8, 2023 • edited Loading

funboarder13920 commented Nov 8, 2023

stillmatic commented Nov 8, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bungerr Nov 8, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

funboarder13920 commented Nov 13, 2023

flyingleafe commented Nov 14, 2023

flyingleafe commented Nov 14, 2023

Purfview commented Nov 14, 2023

flyingleafe commented Nov 15, 2023

Purfview commented Nov 15, 2023

ThomasKluiters commented Nov 15, 2023

Purfview commented Nov 16, 2023

flyingleafe commented Nov 16, 2023

Purfview commented Nov 16, 2023

funboarder13920 commented Nov 16, 2023 • edited Loading

flyingleafe commented Nov 17, 2023

iorilu commented Nov 18, 2023

iorilu commented Nov 18, 2023

blackpolarz commented Nov 21, 2023

doublex commented Nov 22, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nguyendc-systran Nov 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

blackpolarz Nov 23, 2023 • edited Loading

Choose a reason for hiding this comment

nguyendc-systran Nov 23, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nguyendc-systran Nov 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hoonlight Nov 23, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Oscaarjs commented Nov 23, 2023

minhthuc2502 commented Nov 23, 2023 • edited Loading

Oscaarjs commented Nov 24, 2023

nguyendc-systran commented Nov 24, 2023

Purfview commented Nov 24, 2023 • edited Loading

stillmatic commented Nov 7, 2023 •

edited

Loading

stillmatic commented Nov 7, 2023 •

edited

Loading

funboarder13920 commented Nov 7, 2023 •

edited

Loading

bungerr commented Nov 7, 2023 •

edited

Loading

RafaRed commented Nov 8, 2023 •

edited

Loading

stillmatic commented Nov 8, 2023 •

edited

Loading

bungerr Nov 8, 2023 •

edited

Loading

funboarder13920 commented Nov 16, 2023 •

edited

Loading

nguyendc-systran Nov 22, 2023 •

edited

Loading

blackpolarz Nov 23, 2023 •

edited

Loading

nguyendc-systran Nov 23, 2023 •

edited

Loading

nguyendc-systran Nov 22, 2023 •

edited

Loading

hoonlight Nov 23, 2023 •

edited

Loading

minhthuc2502 commented Nov 23, 2023 •

edited

Loading

Purfview commented Nov 24, 2023 •

edited

Loading