Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Craft round-trip test for CFF importer/exporter #10993

Closed
4 tasks done
jeanprbt opened this issue Mar 7, 2024 · 2 comments · Fixed by #10995
Closed
4 tasks done

Craft round-trip test for CFF importer/exporter #10993

jeanprbt opened this issue Mar 7, 2024 · 2 comments · Fixed by #10995

Comments

@jeanprbt
Copy link
Contributor

jeanprbt commented Mar 7, 2024 •
edited
Loading

As mentioned in #10917 (comment), it would be great to craft a round-trip test for the cff format, since we now have both an importer and an exporter.

The round-trip test should work importing https://github.com/JabRef/jabref/blob/main/CITATION.cff.

The current problem is that the importer only parses imports to BibEntry software type, as it was primarily intended by the Citation-File Format. On the other hand, the exporter uses the field preferred-citation to parse entries to their actual type, even if it is different that software. Also, the exporter uses the preferred-citation field even when the type is Software, which is not the expected behavior according to the CFF format.

There are four main things to implement in order for this round-trip test to work.

  • make the importer parse the preferred-citation field, and in this case create another BibEntry for what is contained.
  • make the exporter parse the BibEntry Software type, and in this case output a cff file without any preferred-citation field.
  • parse the author's first name and last name separately, so that the exporter could use the fields given-names and family-names instead of the generic field name in the authors section.
  • (more simple) parse all the fields from https://github.com/JabRef/jabref/blob/main/CITATION.cff in both importer and exporter

The exporter works using org.jabref.logic.exporter.TemplateExporter and the src/main/resources/resource/layout/cff.layout file. Importer is located in org.jabref.logic.importer.fileformat.CffImporter.

Some work was done in #10957 but then abandoned, it may help.
@jeanprbt jeanprbt mentioned this issue Mar 8, 2024
Merged
16 tasks
@koppor
Copy link
Member

koppor commented Mar 13, 2024

@jeanprbt Some GitHub knowledge: Please do not reference PRs in commits. The commits are assigned to a branch and the branch is connected to thr PR. Thus, they will appear hear. Otherwise, we have a long list of non-useful information here on #10993

@jeanprbt
Copy link
Contributor Author

Sorry about that, thanks for noticing ! I won't reference PRs nor issues in commits anymore.

github-merge-queue bot pushed a commit that referenced this issue Mar 21, 2024
@jeanprbt @koppor
@calixtus @Frequinzy @annamaartensson @Tian0602 @LottaJohnsson @filippanilsson @Siedlerchr @andersblomqvist @JXNCTED @hanstig @real-darth @dependabot @ror3d
Improve CFF import/export and craft a round-trip test (#10995)
a1b6d38 
* issue #10993 - feat: added ability to parse preferred-citation field to CffImporter

* issue #10993 - feat: added all fields of JabRef/CITATION.cff to CffImporter

* issue #10993 - feat: rewrote CffExporter to parse Software, Dataset types and authors names correctly

* issue #10993 - feat: added keywords and unknown fields support

* issue #10993 - feat: added round-trip test

* issue #10993 - doc: updated CHANGELOG.md

* Convert RemoveBracesFormatterTest to @ParameterizedTest (#11033)

* Convert to @ParameterizedTest

* Convert to csvsource

---------

Co-authored-by: Carl Christian Snethlage <[email protected]>

* Importing of BibDesk Groups and Linked Files (#10968)

* Add test to check parsing of BibDesk Static Groups

* Add test to check parsing of BibDesk Static Groups

* Change isExpanded attribute to false in expected groups

* remove extra blank line

* Add tests to check parsing of BibDesk Smart and mixed groups

* Add parsing of BibDesk Files

* Attempts at plist

* Now parses bdsk-file and shows it as a file in JabRef

* Add test for parsing a bdsk-file field

* Fix formatting

* Add dd-plist library to documentation

---------

Co-authored-by: Tian0602 <[email protected]>

* Add creation of static JabRef group from a BibDesk file

* Creates an empty ExplicitGroup from BibDesk comment

* Adds citations to new groups
modifies group creations to support multiple groups in the same BibDeskFile

* Fix requested changes
Refactor imports since they did not match with main
Add safety check in addBibDeskGroupEntriesToJabRefGroups

---------

Co-authored-by: Filippa Nilsson <[email protected]>

* Refactor newline to match main branch

Co-authored-by: Filippa Nilsson <[email protected]>

* Add changes to CHANGELOG.md

* Reformat indentation to match previous

* Revert external libraries

Adjust groups serializing

* checkstyle and optional magic

* fix

* fix tests

* fix

* fix dangling do

* better group tree metadata setting

* merge group trees, prevent duplicate group assignment in entry
Add new BibDesk group

Fix IOB for change listeing

* fix tests, and extract constant

* return early

* fixtest and checkstyle

---------

Co-authored-by: Anna Maartensson <[email protected]>
Co-authored-by: Tian0602 <[email protected]>
Co-authored-by: LottaJohnsson <[email protected]>
Co-authored-by: Filippa Nilsson <[email protected]>
Co-authored-by: Filippa Nilsson <[email protected]>
Co-authored-by: Oliver Kopp <[email protected]>
Co-authored-by: Siedlerchr <[email protected]>

* Speed up failure reporting (#11030)

* Fixes Zotero file handling for absolute paths (#11038)

* Fixes Zotero file handling for absolute paths

Fixes #10959

* checkstyle mimiimm

* fix changelog

* cannot fix

* Change copy-paste function to handle string constants (follow up PR) (#11037)

* [Copy] Include string constants in copy (#11)

Signed-off-by: Anders Blomqvist <[email protected]>

* [Copy] New method for serializing string constants (#12)

Signed-off-by: Anders Blomqvist <[email protected]>

* Add a sanity check for null for clipboard content

Currenlty, the clipboard content can be null since the database
does not seem to be updating. This is a sanity check to prevent
the program from adding null to the clipboard.

Link to DD2480-Group1#13

* [Fix] Add parsed serilization when save settings

When loading from existing files or libraries, the parser will set
the serilization of the string constant to the correct value. However,
when editing via the GUI, the serilization was not set and a new
string constant list will be created without the serilization.
This result in the serilization being null and when copying with
the clipboard.

Link to DD2480-Group1#13

* feat: import string constants when pasting #9

Add functionality to import string constants in the paste function

Should add functionality to handle colliding string constants.
Should also check that the constants are valid using the
ConstantsItemModel class.

* feat: Add string constant validity checker and dialog messages #9

Check that a pasted string constant is valid using the
ConstantsItemModel class.

Add diagnostic messages notifying users when adding a string constant
fails while pasting.

* [Copy] Copy referenced constant strings to clipboard  (#16)

* feat: Add parsed serialized string when cloning
* feat: Add sanity check for null in ClipBoardManager
* closes #15

* feat: new unit tests

Add 4 new unit tests, testing the new features added for issue-10872. Specifically the tests are for the `storeSettings` method in the ConstantsPropertiesViewModel.java, and `setContent` in the ClipBaordManager.java.

Closes #6

* Update CHANGELOG with copy and paste function

* Fix Checkstyle failing by reformat the code

* Fix OpenRewrite failing by running rewriteRun

* Refactor by extract methods in setContent

* collet failures

* changelog and use os.newline

* checkstyle

* use real bibentrytypes manager

* Fix CHANGELOG.md

* Swap if branches

* Code cleanup

* Use List for getUsedStringValues

* Fix submodule

* Collection is better

* Fix csl-styles

* Remove empty line

* Group BibTeX string l10n together

---------

Signed-off-by: Anders Blomqvist <[email protected]>
Co-authored-by: Anders Blomqvist <[email protected]>
Co-authored-by: ZOU Hetai <[email protected]>
Co-authored-by: Hannes Stig <[email protected]>
Co-authored-by: Elliot <[email protected]>
Co-authored-by: Oliver Kopp <[email protected]>

* Bump gittools/actions from 0.13.4 to 1.1.1 (#11039)

Bumps [gittools/actions](https://github.com/gittools/actions) from 0.13.4 to 1.1.1.
- [Release notes](https://github.com/gittools/actions/releases)
- [Commits](GitTools/actions@v0.13.4...v1.1.1)

---
updated-dependencies:
- dependency-name: gittools/actions
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump com.googlecode.plist:dd-plist from 1.23 to 1.28 (#11040)

Bumps [com.googlecode.plist:dd-plist](https://github.com/3breadt/dd-plist) from 1.23 to 1.28.
- [Release notes](https://github.com/3breadt/dd-plist/releases)
- [Commits](3breadt/dd-plist@dd-plist-1.23...v1.28.0)

---
updated-dependencies:
- dependency-name: com.googlecode.plist:dd-plist
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump org.apache.pdfbox:xmpbox from 3.0.1 to 3.0.2 (#11041)

Bumps org.apache.pdfbox:xmpbox from 3.0.1 to 3.0.2.

---
updated-dependencies:
- dependency-name: org.apache.pdfbox:xmpbox
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump com.dlsc.gemsfx:gemsfx from 2.2.0 to 2.4.0 (#11044)

Bumps [com.dlsc.gemsfx:gemsfx](https://github.com/dlsc-software-consulting-gmbh/GemsFX) from 2.2.0 to 2.4.0.
- [Release notes](https://github.com/dlsc-software-consulting-gmbh/GemsFX/releases)
- [Changelog](https://github.com/dlsc-software-consulting-gmbh/GemsFX/blob/master/CHANGELOG.md)
- [Commits](dlsc-software-consulting-gmbh/GemsFX@v2.2.0...v2.4.0)

---
updated-dependencies:
- dependency-name: com.dlsc.gemsfx:gemsfx
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump org.apache.pdfbox:fontbox from 3.0.1 to 3.0.2 (#11042)

Bumps org.apache.pdfbox:fontbox from 3.0.1 to 3.0.2.

---
updated-dependencies:
- dependency-name: org.apache.pdfbox:fontbox
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Keep enclosing braces of authors (#11034)

* Add test cases

* Add test cases

* Keep braces for last part

* Refine method description

* Adapt test to new braces keeping

* Add CHANGELOG.md entry

* Adapt tests

* More edge cases

* Minor code beautification

* Simplify code

* Fix braces removing

* Extract static fields, refactor code

* Fix removal of {} for export

* Re-add Objects.requireNonNull

* Fix typo

* Re-add NPE throwing

* Rename to modern terms

* Consistent initialization

* Improve citation relations (#11016)

* Collect DOI and publication type from semantich scholar to be able to expand the information of the new entries later by search through DOI

* Include abstract in the request. This lets the GUI show the abstract since that was implemented already.
Refactor api request string since most of it is shared

* Add button to open the relation paper's DOI URL.
Fix DOI for some ArXiv entries.

* Don't show the open link button if there is no link to open.

* Make field value null error a bit more useful

* Include SemanticScholar url in the request and use it as the URL field.

* Add changes to changelog

* Change tooltip text to an existing, more informative one

* Run rewriter to fix pull request

* improve url optional handling

---------

Co-authored-by: Siedlerchr <[email protected]>

* issue #10993 - doc: updated CHANGELOG.md

* fix: fixed unit tests not passing due to name changes in Author interface (#10995)

* feat: changed CFFExporter to use YAML library snakeyaml instead (#10995)

* feat: added support for references and ALL possible CFF fields in importer (#10995)

* fix: added requested changes (#10995)

+ updated CHANGELOG.md
+ removed useless comments
+ refactored both CffImporter and CffExporter to use more specific methods
+ used a BiMap to avoid repeating mappings between CffImporter and CffExporter
+ copied entryMap in exporter to avoid side-effects

* fix: task rewriteDryRun fixed to pass by removing test in BibEntryTest

* refactor: deleted useless methods in CffImporter (#10995)

* doc: added decision MADR document for cff export (#10995)

* feat: add a cites or related relationship between imported entries in CffImporter (#10995)

* doc: updated MADR decision document for cff export to pass markdownlint (#10995)

* fix: fixed round-trip test to use mock citatioKeyPatternPreferences correctly (#10995)

* fix: fixed MADR document for CFF export decision to pass Jekyll CI check (#10995)

* fix: fixed requested changes (#10995)

+ fixed typo in CHANGELOG.md
+ tested multiline abstract in CFFImporter

* feat: finished CFFExporter logic and crafted working round-trip test (#10995)

* fix: fixed typos in MADR decision doc for CFF export and refactore ImportFormatReader signature (#10995)

* Some code beautification

* Use existing method getEntryLinkList

* Use getEntryLinkList

* Use JabRef's Date class for parsing

---------

Signed-off-by: Anders Blomqvist <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Oliver Kopp <[email protected]>
Co-authored-by: Carl Christian Snethlage <[email protected]>
Co-authored-by: Emil Hultcrantz <[email protected]>
Co-authored-by: Anna Maartensson <[email protected]>
Co-authored-by: Tian0602 <[email protected]>
Co-authored-by: LottaJohnsson <[email protected]>
Co-authored-by: Filippa Nilsson <[email protected]>
Co-authored-by: Filippa Nilsson <[email protected]>
Co-authored-by: Siedlerchr <[email protected]>
Co-authored-by: Anders Blomqvist <[email protected]>
Co-authored-by: ZOU Hetai <[email protected]>
Co-authored-by: Hannes Stig <[email protected]>
Co-authored-by: Elliot <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Roc <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Improve CFF import/export and craft a round-trip test
2 participants
@koppor @jeanprbt