[ENH]Extend default text vectorizer pre-processing to remove tokens of length 1.

[VibhuJawa]

Is your feature request related to a problem? Please describe.

We should extend default text vectorizer pre-processing to remove tokens of length 1 following scikit-learn.

Currently, we don't remove them but now that rapidsai/cudf#5658 is in we should able to handle this too.

Scikit-learn docs on default token extractor:

The default regexp select tokens of 2 or more alphanumeric characters (
punctuation is completely ignored and always treated as a token separator).

Divergence Example: https://gist.github.com/VibhuJawa/5f8e2666da9ecbb63d0398277fc93ac5

Token Level Preprocessing:

Another good thing to benchmark will be switching all of the pre-processing at a token level which the rapidsai/cudf#5739 enables. I expect to see improvements especially for datasets with divergent length characteristics.

[VibhuJawa]

I plan to start a pr benchmarking and addressing this soon. 😊

[VibhuJawa]

Just waiting for rapidsai/cudf#5975 to land.

Will update this and #2590 together once rapidsai/cudf#5975 lands.