Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduces test http server #9748

Merged
merged 27 commits into from
Sep 4, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
7c1df88
Introduces test http server
koppor Apr 8, 2023
37dd6b9
Remove temporary code
koppor Apr 8, 2023
8f05677
Fix checkstyle
koppor Apr 8, 2023
057e982
Create http-server.md with a how-to for the SSL certificate generation
koppor Apr 10, 2023
0d5a695
Merge remote-tracking branch 'upstream/main' into min-http-api
koppor Apr 20, 2023
be12664
Enables passing files to a test Server
koppor Apr 20, 2023
817d77d
Merge branch 'main' into min-http-api
koppor Apr 22, 2023
9299917
Merge remote-tracking branch 'upstream/main' into min-http-api
Siedlerchr Sep 3, 2023
9cde0a6
fix some gradle tasks
Siedlerchr Sep 3, 2023
7fb5100
fix test build
Siedlerchr Sep 3, 2023
75e7de1
Fix broken JavaDoc
koppor Sep 3, 2023
ef1c34a
upgrade jersey to 3.1.3
Siedlerchr Sep 3, 2023
f003363
Merge branch 'min-http-api' of github.com:JabRef/jabref into min-http…
Siedlerchr Sep 3, 2023
94a05ca
Fix checkstyle
koppor Sep 3, 2023
e09ead0
Update http-server.md
koppor Sep 3, 2023
29072fc
rewrite run
Siedlerchr Sep 3, 2023
7f0172a
Merge branch 'min-http-api' of github.com:JabRef/jabref into min-http…
Siedlerchr Sep 3, 2023
59788d0
rename to avoid clash
Siedlerchr Sep 3, 2023
018277e
Hints on http server
koppor Sep 3, 2023
1c15c60
Update http-server.md
koppor Sep 3, 2023
2e9ae7d
Better parameter name
koppor Sep 3, 2023
2d9e8db
Add missing "requires"
koppor Sep 3, 2023
bccf22d
Create resource path only if required
koppor Sep 3, 2023
72d0593
Refine howto
koppor Sep 3, 2023
19e73e6
Add IntelliJ hint
koppor Sep 3, 2023
6f51b07
Add AllowedToUseStandardStreams
koppor Sep 4, 2023
8deec24
Also exclude TestBibFile
koppor Sep 4, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ gradlew text eol=lf

# .bib files have to be written using OS specific line endings to enable our tests working
*.bib text !eol
# Exception: The files used for the http server test - they should have linux line endings
src/test/resources/org/jabref/http/server/*.bib text eol=lf

# Citavi needs to be LF line ending
# This overwrites the setting of "*.bib"
Expand Down
20 changes: 20 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,8 @@ dependencies {
implementation "org.tinylog:tinylog-api:2.6.1"
implementation "org.tinylog:slf4j-tinylog:2.6.1"
implementation "org.tinylog:tinylog-impl:2.6.1"
// route all requests to java.util.logging to SLF4J (which in turn routes to tinylog)
implementation 'org.slf4j:jul-to-slf4j:2.0.7'

implementation 'de.undercouch:citeproc-java:3.0.0-beta.2'

Expand All @@ -199,6 +201,24 @@ dependencies {

implementation group: 'net.harawata', name: 'appdirs', version: '1.2.1'

// JAX-RS implemented by Jersey
// API
implementation 'jakarta.ws.rs:jakarta.ws.rs-api:3.1.0'
// Implementation of the API
implementation 'org.glassfish.jersey.core:jersey-server:3.1.1'
// injection framework
implementation 'org.glassfish.jersey.inject:jersey-hk2:3.1.1'
implementation 'org.glassfish.hk2:hk2-api:2.6.1'
// testImplementation 'org.glassfish.hk2:hk2-testing:3.0.4'
// implementation 'org.glassfish.hk2:hk2-testing-jersey:3.0.4'
// testImplementation 'org.glassfish.hk2:hk2-junitrunner:3.0.4'
// HTTP server
// implementation 'org.glassfish.jersey.containers:jersey-container-netty-http:3.1.1'
implementation 'org.glassfish.jersey.containers:jersey-container-grizzly2-http:3.1.1'
testImplementation 'org.glassfish.jersey.test-framework.providers:jersey-test-framework-provider-grizzly2:3.1.1'
// Allow objects "magically" to be mapped to JSON using GSON
// implementation 'org.glassfish.jersey.media:jersey-media-json-gson:3.1.1'

testImplementation 'io.github.classgraph:classgraph:4.8.157'
testImplementation 'org.junit.jupiter:junit-jupiter:5.9.2'
testImplementation 'org.junit.platform:junit-platform-launcher:1.9.2'
Expand Down
53 changes: 53 additions & 0 deletions docs/decisions/0027-http-return-bibtex-string.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
nav_order: 27
parent: Decision Records
---
<!-- we need to disable MD025, because we use the different heading "ADR Template" in the homepage (see above) than it is foreseen in the template -->
<!-- markdownlint-disable-next-line MD025 -->
# Return BibTeX string and CSL Item JSON in the API

## Context and Problem Statement

In the context of an http server, when a http client `GETs` a JSON data structure containing BibTeX data, which format should that have?

## Considered Options

* Offer both, BibTeX string and CSL JSON
* Return BibTeX as is as string
* Convert BibTeX to JSON

## Decision Outcome

Chosen option: "Offer both, BibTeX string and CSL JSON", because there are many browser libraries out there being able to parse BibTeX. Thus, we don't need to convert it.

## Pros and Cons of the Options

### Offer both, BibTeX string and CSL JSON

- Good, because this follows "Backend for Frontend"
- Good, because Word Addin works seamless with the data provided (and does not need another dependency)
- Good, because other clients can work with BibTeX data
- Bad, because two serializations have to be kept

### Return BibTeX as is as string

- Good, because we don't need to think about any conversion
- Bad, because it is unclear how to ship BibTeX data where the entry is dependent on
- Bad, because client needs add additional parsing logic

### Convert BibTeX to JSON

More thought has to be done when converting to JSON.
There seems to be a JSON format from [@citation-js/plugin-bibtex](https://www.npmjs.com/package/@citation-js/plugin-bibtex).
We could do an additional self-made JSON format, but this increases the number of available JSON serializations for BibTeX.

- Good, because it could flatten BibTeX data (example: `author = first # " and " # second`)
- Bad, because conversion is difficult in BibTeX special cases. For instance, if Strings are used (example: `author = first # " and " # second`) and one doesn't want to flatten ("normalize") this.

## More Information

Existing JavaScript BibTeX libraries:

* [bibtex-js](https://github.com/digitalheir/bibtex-js)
* [bibtexParseJS](https://github.com/ORCID/bibtexParseJs)
* [@citation-js/plugin-bibtex](https://www.npmjs.com/package/@citation-js/plugin-bibtex)
72 changes: 44 additions & 28 deletions src/main/java/module-info.java
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
requires afterburner.fx;
requires com.jfoenix;
requires de.saxsys.mvvmfx;
requires reactfx;
requires de.saxsys.mvvmfx.validation;
requires org.fxmisc.flowless;

requires org.kordamp.ikonli.core;
requires org.kordamp.ikonli.javafx;
Expand All @@ -38,6 +41,7 @@

// Logging
requires org.slf4j;
requires jul.to.slf4j;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this jul adapter?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. The Grizzly http server uses java-util-logging. To be able to handle the log output with our logging framework tinylog, we need jul-slf4j-tinylog.

More detailed:

  • We opted for SLF4J as the JabRef logging framework. We decided against direct calls to tinylog.
  • For log output of an application, there should be one logging framework
  • When using the http server, JabRef internal methods are used. They might log. Thus, we have logs using of SLF4J
  • Grizzly uses Java-Util-Logging - and outputs are produced.

The alternative can be to disable logging of Grizzly completely and bring it back (with the bridge) if needed.

requires org.tinylog.api;
requires org.tinylog.api.slf4j;
requires org.tinylog.impl;
Expand All @@ -46,47 +50,58 @@
with org.jabref.gui.logging.GuiWriter,
org.jabref.gui.logging.ApplicationInsightsWriter;

// Preferences and XML
requires java.prefs;

// Annotations (@PostConstruct)
requires jakarta.annotation;
requires jakarta.inject;

// http server and client exchange
requires java.net.http;
requires jakarta.ws.rs;
requires grizzly.framework;

// data mapping
requires jakarta.xml.bind;
requires jdk.xml.dom;
requires com.google.gson;
requires com.fasterxml.jackson.databind;
requires com.fasterxml.jackson.dataformat.yaml;
requires com.fasterxml.jackson.datatype.jsr310;
// needs to be loaded here as it's otherwise not found at runtime
requires org.glassfish.jaxb.runtime;
requires jdk.xml.dom;

// Annotations (@PostConstruct)
requires jakarta.annotation;
// dependency injection using HK2
requires org.glassfish.hk2.api;

// Microsoft application insights
requires applicationinsights.core;
requires applicationinsights.logging.log4j2;

// Libre Office
requires org.libreoffice.uno;

// Other modules
requires com.google.common;
requires jakarta.inject;
requires reactfx;
requires commons.cli;
requires com.github.tomtung.latex2unicode;
requires fastparse;
requires jbibtex;
requires citeproc.java;
requires de.saxsys.mvvmfx.validation;
requires com.google.gson;
// http clients
requires unirest.java;
requires org.apache.httpcomponents.httpclient;
requires org.jsoup;
requires org.apache.commons.csv;
requires io.github.javadiffutils;
requires java.string.similarity;

// SQL databases
requires ojdbc10;
requires org.postgresql.jdbc;
requires org.mariadb.jdbc;
uses org.mariadb.jdbc.credential.CredentialPlugin;

// Apache Commons and other (similar) helper libraries
requires commons.cli;
requires org.apache.commons.csv;
requires org.apache.commons.lang3;
requires org.antlr.antlr4.runtime;
requires org.fxmisc.flowless;
requires com.google.common;
requires io.github.javadiffutils;
requires java.string.similarity;

requires com.github.tomtung.latex2unicode;
requires fastparse;

requires jbibtex;
requires citeproc.java;

requires pdfbox;
requires xmpbox;
Expand All @@ -95,26 +110,27 @@
requires flexmark;
requires flexmark.util.ast;
requires flexmark.util.data;
requires com.h2database.mvstore;

// fulltext search
requires org.apache.lucene.core;
// In case the version is updated, please also adapt SearchFieldConstants#VERSION to the newly used version
uses org.apache.lucene.codecs.lucene94.Lucene94Codec;

requires org.apache.lucene.queryparser;
uses org.apache.lucene.queryparser.classic.MultiFieldQueryParser;
requires org.apache.lucene.analysis.common;
requires org.apache.lucene.highlighter;

requires com.fasterxml.jackson.databind;
requires com.fasterxml.jackson.dataformat.yaml;
requires com.fasterxml.jackson.datatype.jsr310;
requires net.harawata.appdirs;
requires com.sun.jna;
requires com.sun.jna.platform;

requires org.eclipse.jgit;
uses org.eclipse.jgit.transport.SshSessionFactory;
uses org.eclipse.jgit.lib.GpgSigner;

// other libraries
requires com.h2database.mvstore;
requires org.antlr.antlr4.runtime;
requires org.libreoffice.uno;

}
21 changes: 14 additions & 7 deletions src/main/java/org/jabref/cli/Launcher.java
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
import org.apache.commons.cli.ParseException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.bridge.SLF4JBridgeHandler;
import org.tinylog.configuration.Configuration;

/**
Expand All @@ -47,6 +48,7 @@ public class Launcher {
private static String[] ARGUMENTS;

public static void main(String[] args) {
routeLoggingToSlf4J();
ARGUMENTS = args;
addLogToDisk();
try {
Expand Down Expand Up @@ -85,6 +87,11 @@ public static void main(String[] args) {
}
}

private static void routeLoggingToSlf4J() {
SLF4JBridgeHandler.removeHandlersForRootLogger();
SLF4JBridgeHandler.install();
}

/**
* This needs to be called as early as possible. After the first log write, it
* is not possible to alter
Expand All @@ -93,10 +100,10 @@ public static void main(String[] args) {
private static void addLogToDisk() {
Path directory = Path.of(AppDirsFactory.getInstance()
.getUserDataDir(
OS.APP_DIR_APP_NAME,
"logs",
OS.APP_DIR_APP_AUTHOR))
.resolve(new BuildInfo().version.toString());
OS.APP_DIR_APP_NAME,
"logs",
OS.APP_DIR_APP_AUTHOR))
.resolve(new BuildInfo().version.toString());
try {
Files.createDirectories(directory);
} catch (IOException e) {
Expand Down Expand Up @@ -187,9 +194,9 @@ private static void clearOldSearchIndices() {
&& !path.equals(currentIndexPath)) {
LOGGER.info("Deleting out-of-date fulltext search index at {}.", path);
Files.walk(path)
.sorted(Comparator.reverseOrder())
.map(Path::toFile)
.forEach(File::delete);
.sorted(Comparator.reverseOrder())
.map(Path::toFile)
.forEach(File::delete);
}
}
} catch (IOException e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,7 @@ public void addCertificateFile() {
.build();

dialogService.showFileOpenDialog(fileDialogConfiguration).ifPresent(certPath -> SSLCertificate.fromPath(certPath).ifPresent(sslCertificate -> {
if (!trustStoreManager.isCertificateExist(formatCustomAlias(sslCertificate.getSHA256Thumbprint()))) {
if (!trustStoreManager.certificateExists(formatCustomAlias(sslCertificate.getSHA256Thumbprint()))) {
customCertificateListProperty.add(CustomCertificateViewModel.fromSSLCertificate(sslCertificate)
.setPath(certPath.toAbsolutePath().toString()));
} else {
Expand Down
6 changes: 6 additions & 0 deletions src/main/java/org/jabref/http/MediaType.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package org.jabref.http;

public class MediaType {
public static final String BIBTEX = "application/x-bibtex";
public static final String JSON_CSL_ITEM = "application/x-bibtex-library-csl+json";
}
71 changes: 71 additions & 0 deletions src/main/java/org/jabref/http/dto/BibEntryDTO.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
package org.jabref.http.dto;

import java.io.IOException;
import java.io.StringWriter;

import org.jabref.logic.bibtex.BibEntryWriter;
import org.jabref.logic.bibtex.FieldWriter;
import org.jabref.logic.bibtex.FieldWriterPreferences;
import org.jabref.logic.exporter.BibWriter;
import org.jabref.model.database.BibDatabaseMode;
import org.jabref.model.entry.BibEntry;
import org.jabref.model.entry.BibEntryTypesManager;
import org.jabref.model.entry.SharedBibEntryData;

import com.google.common.base.MoreObjects;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
* The data transfer object (DTO) for an BibEntry
*
* @param sharingMetadata the data used for sharing
* @param userComments the comments before the BibTeX entry
* @param citationKey the citation key (duplicated from BibEntry to ease processing by the client)
* @param bibtex the BibEntry as BibTeX string (see ADR-0027 for more information, why we don't use a HashMap / JSON)
*/
public record BibEntryDTO(SharedBibEntryData sharingMetadata, String userComments, String citationKey, String bibtex) implements Comparable<BibEntryDTO> {

public static final Logger LOGGER = LoggerFactory.getLogger(BibEntryDTO.class);

public BibEntryDTO(BibEntry bibEntry, BibDatabaseMode bibDatabaseMode, FieldWriterPreferences fieldWriterPreferences, BibEntryTypesManager bibEntryTypesManager) {
this(bibEntry.getSharedBibEntryData(),
bibEntry.getUserComments(),
bibEntry.getCitationKey().orElse(""),
convertToString(bibEntry, bibDatabaseMode, fieldWriterPreferences, bibEntryTypesManager)
);
}

private static String convertToString(BibEntry entry, BibDatabaseMode bibDatabaseMode, FieldWriterPreferences fieldWriterPreferences, BibEntryTypesManager bibEntryTypesManager) {
StringWriter rawEntry = new StringWriter();
BibWriter bibWriter = new BibWriter(rawEntry, "\n");
BibEntryWriter bibtexEntryWriter = new BibEntryWriter(new FieldWriter(fieldWriterPreferences), bibEntryTypesManager);
try {
bibtexEntryWriter.write(entry, bibWriter, bibDatabaseMode);
} catch (IOException e) {
LOGGER.warn("Problem creating BibTeX entry.", e);
return "error";
}
return rawEntry.toString();
}

@Override
public int compareTo(BibEntryDTO o) {
int sharingComparison = sharingMetadata.compareTo(o.sharingMetadata);
if (sharingComparison != 0) {
return sharingComparison;
}
LOGGER.error("Comparing equal DTOs");
return 0;
}

@Override
public String toString() {
return MoreObjects.toStringHelper(this)
.add("sharingMetadata", sharingMetadata)
.add("userComments", userComments)
.add("citationkey", citationKey)
.add("bibtex", bibtex)
.toString();
}
}
18 changes: 18 additions & 0 deletions src/main/java/org/jabref/http/dto/GsonFactory.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
package org.jabref.http.dto;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import org.glassfish.hk2.api.Factory;

public class GsonFactory implements Factory<Gson> {
@Override
public Gson provide() {
return new GsonBuilder()
.setPrettyPrinting()
.create();
}

@Override
public void dispose(Gson instance) {
}
}
Loading