Upgrade note field if containing url

[koppor]

note         = {\url{http://www.arcway.com/fileadmin/arcwaydateien/pdf/BPRE_Whitepaper.pdf}, visited 2007-09-15},

should become

url = {http://www.arcway.com/fileadmin/arcwaydateien/pdf/BPRE_Whitepaper.pdf},
note = {visited 2007-09-15},

The conversion should be named "Move URLs in note field to url field".

Should be placed as third the upper part of the cleanup dialog: https://docs.jabref.org/finding-sorting-and-cleaning-entries/cleanupentries

One can look at jabRef/jabref/src/main/java/org/jabref/logic/cleanup/EprintCleanup.java for an impression, how the implementation should like. There is also a test case: jjabRef/jabref/src/test/java/org/jabref/logic/cleanup/EprintCleanupTest.java

Please use @ParameterizedTest if possible (and doing more than one test, which is recommended).

[dkokkotas]

hello @koppor, can I get assigned?

[Alexandra-Stath]

Hello there,
For our semester course @demetres12 and I are working as a team to fulfill our contribution on an open source software project. We are collaborating on this issue and I would like to be assigned as well for the necessities of our lesson.
Please assign me @koppor
Thank you in advance.

[Alexandra-Stath]

Hello there @koppor,
In order to keep you updated on my progress, I would like to inform you that within the next days I will open a draft pull request for this issue. Currently, I am making the necessary adjustments to several classes in order to achieve the program's functionality. I need to add some parameterized tests, as well as, to verify the expected behavior of my code.

Alexandra Stathopoulou

[Alexandra-Stath]

Hello @koppor,
I have almost finished with the requirements of the issue.
I have a question to ask you regarding the fuctionality of my code.
In case there is already a url in the url field do I need to replace the existing one or just add the new one in the url field?
Please inform me,
Thank you in advance,
Alexandra Stathopoulou

[k3KAW8Pnf7mkmdSMPHz27]

@Alexandra-Stath
As far as I know, you cannot/shouldn't add an additional URL to the URL field.

I'd propose,

1. changing the formulation of "Move URLs in note field to URL field" -> "Replace URL field with URL found in the note field"
2. if there are multiple URLs in the note field, do nothing

I apologize in advance if someone else overrides this later. From my point-of-view, this makes the most sense.

[Alexandra-Stath]

Hello @k3KAW8Pnf7mkmdSMPHz27

if there are multiple URLs in the note field, do nothing

Shall we consider that case?
As you mentioned, we cannot store an additional URL to the url field, so why it would be possible two URLs to exist in the note field?

Thank you for the feedback.

[k3KAW8Pnf7mkmdSMPHz27]

As you mentioned, we cannot store an additional URL to the url field, so why it would be possible two URLs to exist in the note field?

Because many fields in bibtex aren't well defined, the note field is one of them, e.g., https://www.bibtex.com/f/note-field/ essentially argues it is a field to store any relevant information that did not fit in any other field.
In a nutshell, you'll have to anticipate the user doing really weird things. In this case, I suppose zero, one, or more than one URL in the note field are the relevant cases.

Additionally, If you search around a bit, you'll find posts like https://tex.stackexchange.com/a/57990 proposing it as an option for multiple URLs, which means someone is going to use it in that way for sure.

[Alexandra-Stath]

Thank you for the quick response.
Wait till Monday, in case Oliver need to note/add something.

[koppor]

@Alexandra-Stath As far as I know, you cannot/shouldn't add an additional URL to the URL field.

In case the URL field contains something, no action should be taken. -- JabRef should not destroy data.

1. changing the formulation of "Move URLs in note field to URL field" -> "Replace URL field with URL found in the note field"

In my opptionen, The whole operation is intended for mass cleanup - without requiring manual inspection after the action.

The text "Move ..." is also used by the other actions above. They are (hopefully) not destroying anything either... I opted for "Move" to indicate that the URL is disappearing from the note field after the action.

Thus, I ask for keeping the original text.

(And in the devcall or other sessions, we should discuss the whole dialog and the wording)

2. if there are multiple URLs in the note field, do nothing

With one exeption: In case the URL field is empty, move the first found url - and keep the second. -- This one is optional - if implemented, please also add test cases.

The test cases should cover

note = {\url{http://example.org}}

note = {\url{https://example.org}}

note = {http://example.org}

Thus, the latex command \url{} should also be recognized - and removed at the move operation.

[dkokkotas]

@koppor

In case the URL field contains something, no action should be taken. -- JabRef should not destroy data.

In case url field contains the same URL with the one in note, then wouldn't it be proper at least to remove the url from the note?

[koppor]

Agree! Thank you for thinking about that edge case! -- It is similar to the Move DOI behavior!

[Alexandra-Stath]

I have just opened the pull request regarding the issue.
@koppor @k3KAW8Pnf7mkmdSMPHz27

[koppor]

Discussion with @demetres12 - ´visited 2007-09-15should be put inurldate` (org.jabref.model.entry.field.StandardField#URLDATE.

Excerpt from biblatex manual:

Will be a follow-up PR!

[dkokkotas]

So we are looking for visited / retrieved / accessed 2007-09-15 in both
URL and Note field to move it to urldate, or just in the Note field?

[koppor]

Interesting question. Handling note only would lead to include it at the new functionality URLCleanup. Making it available for both url and note field, it would be an additional CleanUpJob.

I like that JabRef tries to be precise about the things it does. There is no "magic cleanup" doing "something useful". This "visited" cleanup is something very special. I didn't encounter any bibtex file containging , visited at 28. December 2022 or similar in the URL field. Did you? Did you do an extensive research on that?

Without any empirical data, I vote for handling the note field only and "magically" also updating "visited".

Please use org.jabref.model.entry.Date#parse(java.lang.String) for date parsing.

[dkokkotas]

Handling note only would lead to include it at the new functionality URLCleanup

Agree. On the one hand JabRef needs to be precise and cover several unexpected cases a user may perform, but on the other hand this could lead to overkill (in our case of the cleanup activities).

Did you? Did you do an extensive research on that?

Actually, I'm worried about the chance a user for unknown reasons include in the URL field info about date, but as you mentioned, there is no need for an additional check on URL field.

[koppor]

Actually, I'm worried about the chance a user for unknown reasons include in the URL field info about date, but as you mentioned, there is no need for an additional check on URL field.

You can add an integrity check.

UI: https://docs.jabref.org/finding-sorting-and-cleaning-entries/checkintegrity

While reading that code, I found org.jabref.logic.integrity.UrlChecker. You can modify it to return more warnings (not a URL, containing more than an URL, containing \url{...).

[dkokkotas]

@koppor
When date is given from the user in the following format January 15, 2021, should it be converted to 2021-01-15 before set in urldate field, or JabRef takes care of that conversion afterwards?

[koppor]

What is easier to implement? 😅 I think, keeping would be better

Does JabRef have a date cleanup? Converting all dates? NormalizeDateFormat. Then a user can do that a separate step...