add normalizer, tokenizer to rouge

[hookSSi]

What does this PR do?

Fixes #825
I wrote some code but I don't know how to test it sorry.
plz review me hard

Before submitting

• Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure to update the docs?
• Did you write any new necessary tests?

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[Borda]

@stancld mind help with adding relevant tests? :)

[codecov]

Codecov Report

Merging #838 (a1116cb) into master (76646b6) will decrease coverage by 0%.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #838   +/-   ##
=====================================
- Coverage      95%    95%   -0%     
=====================================
  Files         167    167           
  Lines        6904   6906    +2     
=====================================
- Hits         6562   6553    -9     
- Misses        342    353   +11     

[Borda]

@stancld how is it going here? :)

[stancld]

@hookSSi Thanks a lot for your contribution. I left a few minor comments but, in general, I think the PR looks quite good.
Regarding the tests.. I think we can just test default behaviour against the scenario we input the default tokenizer and normalizer manually into the module/functional metric and just verify everything goes through without problems. :]

[Borda]

@hookSSi how is it going here? have you had a change to check recent review/suggestion from @stancld 🐰

[hookSSi]

@hookSSi how is it going here? have you had a change to check recent review/suggestion from @stancld 🐰

Sorry, I'm checking on it now

[stancld]

@hookSSi how is it going here? have you had a change to check recent review/suggestion from @stancld 🐰

Sorry, I'm checking on it now

Thanks for the update (I'll go through the code once again today). May you just add some tests suggested in the code review comment?

[hookSSi]

Thanks for the update (I'll go through the code once again today). May you just add some tests suggested in the code review comment?

I added test that just checking value. Have a look and review me please.

[Borda]

@hookSSi could you please mark addressed comments as resolved so we easily see what has been left? 🐰

[hookSSi]

@hookSSi could you please mark addressed comments as resolved so we easily see what has been left? 🐰

Oh I didn't know that I can mark these sorry about that.

[stancld]

Nitpicks