Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix issue "Add quality check and cleanup for problematic unicode characters" #10817

Closed
wants to merge 14 commits into from
Closed

Fix issue "Add quality check and cleanup for problematic unicode characters" #10817

wants to merge 14 commits into from

Conversation

harsh1898
Copy link
Contributor

@harsh1898 harsh1898 commented Jan 23, 2024 •
edited by ThiloteE
Loading

fixes #10506

  • Added a new integrity check for problematic unicode fields that is not in NFC format. Now, it will show a message in the Quality/Integrity dialog box for non NFC values.
  • Added a new cleanup feature in Quality/Cleanup Entries to format the problematic Unicode characters with Unicode Normalize formatter.
    image
    image
    image
    Here, we can see value is in NFC format after clean up.

Mandatory checks

  • Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
  • Tests created for changes (if applicable)
  • Manually tested changed features in running JabRef (always required)
  • Screenshots added in PR description (for UI changes)
  • Checked developer's documentation: Is the information available and up to date? If not, I outlined it in this pull request.
  • Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

@harsh1898 harsh1898 mentioned this pull request Jan 23, 2024
Closed
calixtus
calixtus reviewed Jan 23, 2024
View reviewed changes
src/main/java/org/jabref/logic/integrity/FieldCheckers.java Outdated
Comment on lines 56 to 57
// fieldCheckers.put(StandardField.TITLE, new UnicodeNormalFormCCheck(databaseContext));
// fieldCheckers.put(StandardField.AUTHOR, new UnicodeNormalFormCCheck(databaseContext));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you remove the feld checkers? Are they now included in your new Unicode Normal Form Checker?
If so, please remove them completly, otherwise we have a lot of dead code fragments.

Copy link
Contributor Author

@harsh1898 harsh1898 Jan 23, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @calixtus
No, these particular field checker that I removed , is not included in new Unicode Normal Form Checker.
Initially, I was trying here but it was not correct. So, I removed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But now they will remain commented. Will you uncomment these again or will you remove them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, It will not uncomment again.
We can completely remove them from here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, then go ahead, remove them and all the dead code related to them. 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @calixtus

I have removed that unnecessary comment from particular FieldCheckers file. Now, It is good

@koppor
Copy link
Member

koppor commented Feb 28, 2024

@harsh1898 We did not see any comments or updates the last week. Are you still working on this?

@koppor koppor marked this pull request as draft March 4, 2024 11:19
@koppor koppor added the status: changes required Pull requests that are not yet complete label Mar 4, 2024
@koppor
Copy link
Member

koppor commented Mar 19, 2024

@harsh1898 I think, you forgot to commit NormalizeUnicodeFormatterTest

@koppor
Copy link
Member

koppor commented Mar 19, 2024

I have no push permission. Closing and creating a new PR.

@koppor koppor closed this Mar 19, 2024
@koppor koppor mentioned this pull request Mar 19, 2024
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@calixtus calixtus calixtus left review comments

@koppor koppor koppor requested changes

Assignees
No one assigned
Labels
status: changes required Pull requests that are not yet complete
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add quality check and cleanup for problematic unicode characters
3 participants
@harsh1898 @koppor @calixtus