Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement an interface to import PDF metadata from multiple sources (XMP, Grobid, ...) #7929

Merged
merged 128 commits into from
Aug 21, 2021
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
128 commits
Select commit Hold shift + click to select a range
22f0241
GrobidPdfMetadataImporter implemented
btut Jul 20, 2021
8effaa9
Fixed class when accessing resources
btut Jul 20, 2021
5d487d2
Draft of merge dialog
btut Jul 20, 2021
96cd5cf
Default to first available entry
btut Jul 21, 2021
8b5510e
Changed layout
btut Jul 21, 2021
3a4a01a
Checkstyle
btut Jul 21, 2021
8314855
Bind buttons with equal content together
btut Jul 21, 2021
05964bc
Use TextArea only for multiline fields
btut Jul 23, 2021
0f64b1c
Use SplitPane
btut Jul 23, 2021
1260cf9
Fixed scaling of labels
btut Jul 23, 2021
97fb43d
Add tooltip for toggle buttons
btut Jul 23, 2021
733415f
Implemented loading BibEntries in background
btut Jul 23, 2021
620424c
Implemented DOI Lookup button
btut Jul 23, 2021
46bf75a
Changed Button content to TextFlow
btut Jul 24, 2021
f036112
Change DOI button to icon
btut Jul 28, 2021
a5d216c
Use FileHelper method to get extension
btut Jul 28, 2021
d9dc84e
Use ellipsing text flow
btut Jul 28, 2021
6e6b5bc
Ignore empty fields
btut Jul 28, 2021
3546715
Use jsoup to issue POST request
btut Jul 28, 2021
6906d11
Removed unnecessary field
btut Jul 28, 2021
c9e4c06
Reverted URLDownload
btut Jul 28, 2021
f319fc7
Enable VGrow
btut Jul 28, 2021
993ad84
Insets and DiffHighlighting
btut Jul 30, 2021
8a7b80f
GrobidPdfMetadataImporter implemented
btut Jul 20, 2021
6fe2a23
Fixed class when accessing resources
btut Jul 20, 2021
f99bc52
Use FileHelper method to get extension
btut Jul 28, 2021
1d64d80
Use jsoup to issue POST request
btut Jul 28, 2021
f591bfc
Removed unnecessary field
btut Jul 28, 2021
b2bd365
Reverted URLDownload
btut Jul 28, 2021
e458c77
Changelog entry
btut Jul 30, 2021
5478585
Add pdf link to imported entry
btut Jul 30, 2021
d0cc663
Remove citationkey from Grobid
btut Jul 30, 2021
2cd78fc
FirstPageImporter
btut Jul 30, 2021
eb22157
Fixed grammar mistake in CHANGELOG.md
btut Jul 30, 2021
3ac0094
Fixed Grobid tests
btut Jul 30, 2021
c87ed4e
Fixed Grobid URL
btut Jul 30, 2021
3d8c4da
Checkstyle
btut Jul 30, 2021
168b866
Fixed doc
btut Jul 30, 2021
42adea9
Checkstyle
btut Jul 30, 2021
980af83
MVVM split
btut Aug 1, 2021
73dc505
Use JSoup for plaintext citations as well
btut Aug 1, 2021
33cbc95
Merge branch 'improvement/morePdfImporters' into improvement/pdfMetad…
btut Aug 1, 2021
d841207
Actual MVVM
btut Aug 2, 2021
616e73d
Fixes and styling
btut Aug 2, 2021
e2a215e
Cleanup Diff-highlighting
btut Aug 2, 2021
44dfebd
Checkstyle
btut Aug 2, 2021
1841cdf
Prettier loading indicator
btut Aug 2, 2021
7ce7105
Renamed FirstPageImporter to PdfVerbatimBibTextImporter
btut Aug 4, 2021
53d8e9a
Fixed getName (no importer)
btut Aug 4, 2021
9080f14
Renamed Grobid importer to match convention
btut Aug 4, 2021
4c74d51
Fixed loading issue
btut Aug 5, 2021
2757be6
PdfEmbeddedBibTeXImporter
btut Aug 5, 2021
8a05c3e
Renamed PdfEmbeddedBibTeXImporter to PdfEmbeddedBibFileImporter
btut Aug 5, 2021
0c488ec
Checkstyle
btut Aug 5, 2021
02057f0
Remove debug output
btut Aug 5, 2021
3d66855
Checkstyle
btut Aug 5, 2021
fd8918b
PdfMergeMetadataImporter
btut Aug 5, 2021
56868f5
Add DOI and ISBN fetching in PdfMergeMetadataImporter
btut Aug 5, 2021
479a0bc
Fixed concurrent list access
btut Aug 5, 2021
cb6a910
Adapted tests to contain fetchable ID's
btut Aug 5, 2021
0b64ebd
Configurable diff-modes and styling
btut Aug 10, 2021
649049c
Localization
btut Aug 10, 2021
b6e3aaa
Refactor
btut Aug 10, 2021
7f78c9e
Merge branch 'main' of github.com:JabRef/jabref into improvement/pdfM…
btut Aug 10, 2021
e18eabd
Merge branch 'main' of github.com:JabRef/jabref into improvement/more…
btut Aug 10, 2021
1bf6409
Derive XMP preferences from importFormatPreferences
btut Aug 10, 2021
787e040
Localization
btut Aug 10, 2021
a3cdff9
Use Importers in JabRef
btut Aug 10, 2021
564988a
Remove unnecessary test documents
btut Aug 10, 2021
4db14f4
Fixed error introduced by refactor
btut Aug 10, 2021
ba13971
Fit field-editor-column to width
btut Aug 10, 2021
e3d279a
Checkstyle
btut Aug 11, 2021
25e7b2e
Localization in diff-mode
btut Aug 14, 2021
04eecaf
Grobid Timeout
btut Aug 14, 2021
b7e5b62
Null-check
btut Aug 14, 2021
5cbf919
Use MergeImporter as WebFetcher
btut Aug 14, 2021
1cb4dfc
Only force BibTeX import if everything else fails
btut Aug 16, 2021
3ab8ebb
Prioritize non-bruteforce importers that
btut Aug 16, 2021
7ba8b40
Checkstyle
btut Aug 16, 2021
eadbf67
Added explanaition on need for runInJavaFXThread
btut Aug 16, 2021
2b00f47
Styling for dark theme
btut Aug 16, 2021
18dbb67
Fixed WebFetchersTest
btut Aug 16, 2021
9a138b6
Added parse pdf metadata button to GUI
btut Aug 16, 2021
41de0d0
Changelog
btut Aug 16, 2021
4fdd850
Merge branch 'main' of github.com:JabRef/jabref into improvement/pdfM…
btut Aug 16, 2021
6cd9544
Fixed moving-text glitch
btut Aug 16, 2021
1f4bf84
Follow up on glitch-fix
btut Aug 16, 2021
0468e67
Checkstyle and localization
btut Aug 16, 2021
3d46df4
Grobid does not need localization
btut Aug 16, 2021
40b2759
Followup on removed Grobid localization
btut Aug 16, 2021
6324cf2
Fixed tests
btut Aug 16, 2021
54274be
Merge branch 'improvement/morePdfImporters' of github.com:btut/jabref…
btut Aug 16, 2021
089b025
Enable all importers
btut Aug 16, 2021
5cf2af7
Merge branch 'main' of github.com:JabRef/jabref into improvement/more…
btut Aug 16, 2021
b555ada
Checkstyle
btut Aug 16, 2021
a956a37
Merge branch 'improvement/morePdfImporters' of github.com:btut/jabref…
btut Aug 17, 2021
5994a5d
Merge branch 'main' of github.com:JabRef/jabref into improvement/pdfM…
btut Aug 17, 2021
44fee74
Improved display
btut Aug 18, 2021
7a98c8a
Modern switch statements
btut Aug 18, 2021
fb186e3
Fixed position of buttons in LinkedFilesEditor
calixtus Aug 18, 2021
63272bb
Merge remote-tracking branch 'btut/improvement/pdfMetadataImport' int…
calixtus Aug 18, 2021
9db1045
Merge branch 'main' of github.com:JabRef/jabref into improvement/pdfM…
btut Aug 18, 2021
8b1974c
Merge branch 'improvement/pdfMetadataImport' of github.com:btut/jabre…
btut Aug 18, 2021
9f69569
Collapse importers that yield no result
btut Aug 19, 2021
31b62a0
Settings for grobid
btut Aug 19, 2021
cdc9fb1
Merge branch 'main' of github.com:JabRef/jabref into useGrobidPreference
btut Aug 19, 2021
fcdb5a4
Use settings
btut Aug 19, 2021
7e918a3
Updated PdfImporter priorization
btut Aug 19, 2021
3440b32
Store opt-out preference
btut Aug 19, 2021
e5222ce
Partial implementation of opt-in/out dialogue
btut Aug 19, 2021
ef5444a
Show dialog before all Grobid actions
btut Aug 20, 2021
ffaeec5
Static code checks
btut Aug 20, 2021
4f4f398
Merge branch 'main' of github.com:JabRef/jabref into useGrobidPreference
btut Aug 20, 2021
e9ef2e3
Merge branch 'useGrobidPreference' into improvement/pdfMetadataImport
btut Aug 20, 2021
ae32a40
Use Grobid Settings and Opt-In dialog
btut Aug 20, 2021
bd6c8e4
Fix l10n issue (introduced in merge)
btut Aug 20, 2021
70a2e3e
Merge branch 'main' of github.com:JabRef/jabref into useGrobidPreference
btut Aug 20, 2021
51eb15d
Merge branch 'useGrobidPreference' into improvement/pdfMetadataImport
btut Aug 20, 2021
61b3b5b
Fixed missing import (introduced by merge)
btut Aug 20, 2021
3dbafbe
Merge branch 'useGrobidPreference' into improvement/pdfMetadataImport
btut Aug 20, 2021
69af125
Extract given-clause in test
btut Aug 21, 2021
f676003
Improved readability
btut Aug 21, 2021
1e97104
Changelog
btut Aug 21, 2021
43aaa05
Merge branch 'useGrobidPreference' into improvement/pdfMetadataImport
btut Aug 21, 2021
55a4653
Changelog update
btut Aug 21, 2021
f0afe0c
Merge branch 'useGrobidPreference' into improvement/pdfMetadataImport
btut Aug 21, 2021
87eded9
Renamed Entry to EntrySource
btut Aug 21, 2021
32c0a3d
Merge branch 'main' of github.com:JabRef/jabref into improvement/pdfM…
btut Aug 21, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Note that this project **does not** adhere to [Semantic Versioning](http://semve
- We added unprotect_terms to the list of bracketed pattern modifiers [#7826](https://github.com/JabRef/jabref/pull/7960)
- We added a dialog that allows to parse metadata from linked pdfs. [#7929](https://github.com/JabRef/jabref/pull/7929)
- We added an icon picker in group edit dialog. [#6142](https://github.com/JabRef/jabref/issues/6142)
- We added a preference to Opt-In to Grobid usage. [8002](https://github.com/JabRef/jabref/pull/8002)
btut marked this conversation as resolved.
Show resolved Hide resolved

### Changed

Expand Down
17 changes: 15 additions & 2 deletions src/main/java/org/jabref/gui/entryeditor/EntryEditor.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.SortedSet;
import java.util.stream.Collectors;

import javax.inject.Inject;
Expand Down Expand Up @@ -357,13 +358,25 @@ private void setupToolBar() {

// Add menu for fetching bibliographic information
ContextMenu fetcherMenu = new ContextMenu();
for (EntryBasedFetcher fetcher : WebFetchers.getEntryBasedFetchers(preferencesService.getImportSettingsPreferences(), preferencesService.getImportFormatPreferences(), preferencesService.getFilePreferences(), databaseContext, preferencesService.getDefaultEncoding())) {
SortedSet<EntryBasedFetcher> entryBasedFetchers = WebFetchers.getEntryBasedFetchers(
preferencesService.getImportSettingsPreferences(),
preferencesService.getImportFormatPreferences(),
preferencesService.getFilePreferences(),
databaseContext,
preferencesService.getDefaultEncoding());
for (EntryBasedFetcher fetcher : entryBasedFetchers) {
MenuItem fetcherMenuItem = new MenuItem(fetcher.getName());
if (fetcher instanceof PdfMergeMetadataImporter.EntryBasedFetcherWrapper) {
// Handle Grobid Opt-In in case of the PdfMergeMetadataImporter
fetcherMenuItem.setOnAction(event -> {
GrobidOptInDialogHelper.showAndWaitIfUserIsUndecided(dialogService);
PdfMergeMetadataImporter.EntryBasedFetcherWrapper pdfMergeMetadataImporter = new PdfMergeMetadataImporter.EntryBasedFetcherWrapper(preferencesService.getImportSettingsPreferences(), preferencesService.getImportFormatPreferences(), preferencesService.getFilePreferences(), databaseContext, preferencesService.getDefaultEncoding());
PdfMergeMetadataImporter.EntryBasedFetcherWrapper pdfMergeMetadataImporter =
new PdfMergeMetadataImporter.EntryBasedFetcherWrapper(
preferencesService.getImportSettingsPreferences(),
preferencesService.getImportFormatPreferences(),
preferencesService.getFilePreferences(),
databaseContext,
preferencesService.getDefaultEncoding());
fetchAndMerge(pdfMergeMetadataImporter);
});
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,10 @@ public ImportHandler(BibDatabaseContext database,
this.stateManager = stateManager;

this.linker = new ExternalFilesEntryLinker(externalFileTypes, preferencesService.getFilePreferences(), database);
this.contentImporter = new ExternalFilesContentImporter(preferencesService.getImportSettingsPreferences(), preferencesService.getImportFormatPreferences(), preferencesService.getTimestampPreferences());
this.contentImporter = new ExternalFilesContentImporter(
preferencesService.getImportSettingsPreferences(),
preferencesService.getImportFormatPreferences(),
preferencesService.getTimestampPreferences());
this.undoManager = undoManager;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ public void processInvalidCitationTest() {

@Test
public void failsWhenGrobidDisabled() {
assertThrows(UnsupportedOperationException.class, () -> new GrobidService(new ImportSettingsPreferences(false, false, false, "http://grobid.jabref.org:8070")));
ImportSettingsPreferences importSettingsWithGrobidDisabled = importSettingsPreferences.withGrobidEnabled(false);
assertThrows(UnsupportedOperationException.class, () -> new GrobidService(importSettingsWithGrobidDisabled));
}

@Test
Expand Down