Entry Editor extremely slow with shared database on remote MySQL server #3736

memering · 2018-02-17T16:30:48Z

I have tested the latest master version from http://builds.jabref.org/master/ and the problem persists

JabRef 4.1 or
JabRef 4.2-dev--snapshot--2018-02-16--master--bb2b07833
Linux 3.13.0-141-generic amd64
Java 1.8.0_162

Steps to reproduce:

Connect to shared database on remote MySQL server with somewhat less-than-ideal connection speed/latency (SSH tunnel in this particular case).
Open an entry in the Entry Editor.
Typing is extremely slow, practically unusable.

Notes:

No error messages on console/event log.
Size of database is irrelevant (problem is the same even with new entry in empty database).
Connection to database seems reasonably fast/responsive. No perceptible lag on the mysql command line.

koppor · 2018-02-20T06:29:55Z

Could you please try with JabRef 3.8.2 and report back whether it is equally slow?

lenhard · 2018-02-21T14:53:56Z

Regarding the frequency of the synchronization with the database, the following should be the case:

Open a connection to the shared database
Type a first character in a field in the entry editor, e.g. title. JabRef will try to sync that, possibly triggering the lag
Continue typing in the same field, e.g. title. Subsequent characters will not be immediately transfered. Hence, there should be no lag
Start typing in a new field, e.g. author. JabRef will synchronize the data again, and there is probably a lag.

Is this, roughly, your user experience? The key point is step 3.

memering · 2018-02-21T18:58:05Z

@koppor I tried 3.8.2 and with this version, the problem does not occur. The Entry Editor takes a little bit more time to load, and changing fields (with tab key or mouse) is somewhat laggy, but typing is OK, even with autocompletion. All in all, usable.

@lenhard No, that is not my experience with version 4.x. At step 3 the situation does not change. Every character lags for about 1,5 seconds. Curiously enough, with 4.x, the Editor pops up faster, and changing fields is responsive as well.

lenhard · 2018-02-22T09:34:51Z

@memering Thank you very much for investigating this! Unfortunately that is not the behavior I expect from looking at the code. But it seems that something somewhere is triggering an entire save operation for every character change that (my guess) does a complete overwrite of the database schema.

Before 4.0, there were much less updates in JabRef and probably the database code did not become aware of every character change in the database. That's why the situation is different with 3.8.2.

It is really unfortunate that the people who wrote all this seem to have vanished...

Siedlerchr · 2018-03-21T08:15:13Z

What about putting the changes in a queue and let a different thread consume that and send it to the database.
For me the problem looks like the typical producer/consumer pattern with multiple threads.
For share latex I do use a queue as I have to wait for a server response first until I can continue sending changes

koppor · 2018-03-21T08:19:41Z

This is NOT the case here. The MySQL server does not support pub/sub. So, the changes are sent if the SYNC button is pushed (if I recall correctly). It is not properly documented at http://help.jabref.org/en/SQLDatabase, but it was implemented like that by @obraliar for JabRef 3.6. I don't know what was changed until then.

Only PostgreSQL and Oracle have pub/sub, where changes are pushed and pulled without the need of the sync button.

obraliar · 2018-03-22T15:54:21Z

The changes are pushed on every FieldChangedEvent. In Version 3.6 such an event was postet when a textfield was left (e.g. by the mouse). Now a FieldChangedEvent is posted even if a character is typed in. That is the reason why typing may be slow. This problem could be solved with a background thread, while offering the SYNC button for pulling the changes from the MySQL server, which does not support any notifier.

Unfortunately I'm too busy right now to fix this problem. I think this will get changed in a month or two, where I'm going to rework this module.

koppor · 2018-03-22T15:57:51Z

@obraliar Thank you for the insights.

Maybe, we can quickfix by introducing another event with FieldLeftEvent or even better ChangedFieldLeftEvent?

Siedlerchr · 2018-03-22T16:19:10Z

Didn't @lenhard introduce the Coarse Change Event Listener for the change scanner?

obraliar · 2018-03-22T16:39:00Z

Does anybody know where UI listener of the entryeditor are organized?

tobiasdiez · 2018-03-22T17:30:59Z

There are no longer event listeners on the text fields in the entry editor. The contents of the text field is automatically synced to the entry, which then posts the change event. All listeners are thus registered on the entry (or on the database).

For the autosave/backup, the CoarseChangeFilter filters out minor changes so that a save is not triggered upon every key press. It seems to me, however, this filter is not installed in front of the SQL database sync.

obraliar · 2018-03-22T18:11:28Z

It actually is:

jabref/src/main/java/org/jabref/model/database/BibDatabaseContext.java

Line 262 in 98f5544

this.dbmsListener = new CoarseChangeFilter(this);

But for some reason

jabref/src/main/java/org/jabref/model/database/event/CoarseChangeFilter.java

Line 36 in 98f5544

eventBus.post(event);

is executed three times (!) on every key press. I think this not the desired behaviour, since it leads DBMSSynchronizer to synchronize the same change multiple times.

lenhard · 2018-03-22T18:58:01Z

The code of the CoarseChangeFilter is really very simple, it may or may not forward a single event. If it is triggered three times for a single key press, then there is something wrong deeper down that triggers three change events for that key press. This might be several UI components in the entry editor that trigger changes in a BibEntry.

Something that comes to my mind: Is the source tab somehow playing ping pong with other components?

Siedlerchr · 2018-03-22T19:17:01Z

Its not only related to the sharedatabase, it happens even if you simply type one char
I added a new Throwable().printStackTrace(); to get the call hierachy stack trace.
It's three times the same:

java.lang.Throwable
	at org.jabref.model.database.event.CoarseChangeFilter.listen(CoarseChangeFilter.java:27)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
	at com.google.common.eventbus.Subscriber$SynchronizedSubscriber.invokeSubscriberMethod(Subscriber.java:144)
	at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
	at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398)
	at com.google.common.eventbus.Subscriber.dispatchEvent(Subscriber.java:67)
	at com.google.common.eventbus.Dispatcher$PerThreadQueuedDispatcher.dispatch(Dispatcher.java:108)
	at com.google.common.eventbus.EventBus.post(EventBus.java:212)
	at org.jabref.model.database.BibDatabase.relayEntryChangeEvent(BibDatabase.java:582)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:87)
	at com.google.common.eventbus.Subscriber$SynchronizedSubscriber.invokeSubscriberMethod(Subscriber.java:144)
	at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:72)
	at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398)
	at com.google.common.eventbus.Subscriber.dispatchEvent(Subscriber.java:67)
	at com.google.common.eventbus.Dispatcher$PerThreadQueuedDispatcher.dispatch(Dispatcher.java:108)
	at com.google.common.eventbus.EventBus.post(EventBus.java:212)
	at org.jabref.model.entry.BibEntry.setField(BibEntry.java:428)
	at org.jabref.model.entry.BibEntry.setField(BibEntry.java:447)
	at org.jabref.gui.fieldeditors.AbstractEditorViewModel.lambda$2(AbstractEditorViewModel.java:66)
	at org.jabref.gui.util.BindingsHelper$BidirectionalBinding.updateLocked(BindingsHelper.java:186)
	at org.jabref.gui.util.BindingsHelper$BidirectionalBinding.changedA(BindingsHelper.java:175)
	at com.sun.javafx.binding.ExpressionHelper$Generic.fireValueChangedEvent(ExpressionHelper.java:361)
	at com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(ExpressionHelper.java:81)
	at javafx.beans.property.StringPropertyBase.fireValueChangedEvent(StringPropertyBase.java:103)
	at javafx.beans.property.StringPropertyBase.markInvalid(StringPropertyBase.java:110)
	at javafx.beans.property.StringPropertyBase.set(StringPropertyBase.java:144)
	at javafx.beans.property.StringPropertyBase.set(StringPropertyBase.java:49)
	at javafx.beans.property.StringProperty.setValue(StringProperty.java:65)
	at javafx.beans.property.StringProperty.setValue(StringProperty.java:57)
	at com.sun.javafx.binding.BidirectionalBinding$TypedGenericBidirectionalBinding.changed(BidirectionalBinding.java:599)
	at com.sun.javafx.binding.ExpressionHelper$Generic.fireValueChangedEvent(ExpressionHelper.java:361)
	at com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(ExpressionHelper.java:81)
	at javafx.scene.control.TextInputControl$TextProperty.fireValueChangedEvent(TextInputControl.java:1389)
	at javafx.scene.control.TextInputControl$TextProperty.markInvalid(TextInputControl.java:1393)
	at javafx.scene.control.TextInputControl$TextProperty.controlContentHasChanged(TextInputControl.java:1332)
	at javafx.scene.control.TextInputControl$TextProperty.access$1600(TextInputControl.java:1300)
	at javafx.scene.control.TextInputControl.lambda$new$162(TextInputControl.java:139)
	at com.sun.javafx.binding.ExpressionHelper$SingleInvalidation.fireValueChangedEvent(ExpressionHelper.java:137)
	at com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(ExpressionHelper.java:81)
	at javafx.scene.control.TextArea$TextAreaContent.insert(TextArea.java:207)
	at javafx.scene.control.TextInputControl.replaceText(TextInputControl.java:1204)
	at javafx.scene.control.TextInputControl.updateContent(TextInputControl.java:556)
	at javafx.scene.control.TextInputControl.replaceText(TextInputControl.java:548)
	at com.sun.javafx.scene.control.behavior.TextAreaBehavior.replaceText(TextAreaBehavior.java:305)
	at com.sun.javafx.scene.control.behavior.TextInputControlBehavior.defaultKeyTyped(TextInputControlBehavior.java:238)
	at com.sun.javafx.scene.control.behavior.TextInputControlBehavior.callAction(TextInputControlBehavior.java:139)
	at com.sun.javafx.scene.control.behavior.TextAreaBehavior.callAction(TextAreaBehavior.java:259)
	at com.sun.javafx.scene.control.behavior.BehaviorBase.callActionForEvent(BehaviorBase.java:218)
	at com.sun.javafx.scene.control.behavior.TextInputControlBehavior.callActionForEvent(TextInputControlBehavior.java:127)
	at com.sun.javafx.scene.control.behavior.BehaviorBase.lambda$new$74(BehaviorBase.java:135)
	at com.sun.javafx.event.CompositeEventHandler$NormalEventHandlerRecord.handleBubblingEvent(CompositeEventHandler.java:218)
	at com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:80)
	at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:238)
	at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191)
	at com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(CompositeEventDispatcher.java:59)
	at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58)
	at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
	at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
	at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
	at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
	at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
	at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
	at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
	at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
	at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
	at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
	at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
	at com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74)
	at com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:54)
	at javafx.event.Event.fireEvent(Event.java:198)
	at javafx.scene.Scene$KeyHandler.process(Scene.java:3964)
	at javafx.scene.Scene$KeyHandler.access$1800(Scene.java:3910)
	at javafx.scene.Scene.impl_processKeyEvent(Scene.java:2040)
	at javafx.scene.Scene$ScenePeerListener.keyEvent(Scene.java:2501)
	at com.sun.javafx.tk.quantum.EmbeddedScene.lambda$null$299(EmbeddedScene.java:310)
	at java.security.AccessController.doPrivileged(Native Method)
	at com.sun.javafx.tk.quantum.EmbeddedScene.lambda$keyEvent$300(EmbeddedScene.java:296)
	at com.sun.javafx.application.PlatformImpl.lambda$null$172(PlatformImpl.java:295)
	at java.security.AccessController.doPrivileged(Native Method)
	at com.sun.javafx.application.PlatformImpl.lambda$runLater$173(PlatformImpl.java:294)
	at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
	at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
	at com.sun.glass.ui.win.WinApplication.lambda$null$147(WinApplication.java:177)
	at java.lang.Thread.run(Thread.java:748)

Siedlerchr · 2018-03-22T20:17:14Z

I debugged this whole stuff and still can not figure out where exactly this change is coming from. It's definitely coming from the UI. So there must be some bindings which fire each other again.
Just type one char in the author field to test it.
Maybe @tobiasdiez has an idea what this could trigger

tobiasdiez · 2018-03-22T20:58:22Z

I just tried it in the maintable-beta branch and there the CoarseChangeFilter is invoked twice for each key pressed. But it is important to note that two different instance are invoked: one for the auto-completion and one in the BackupManager. So this seems to work perfectly fine for me.

bruot · 2019-07-14T18:14:28Z

I have just posted here what DB commands are executed on a PostgreSQL server in various situations.

See in particular the test cases when typing 1 and 10 characters in the Title field. It appears that for each character, the field is updated only once, but many other extra statements are executed, and some of these appear several times.

Siedlerchr · 2020-12-13T18:32:11Z

Thank you for reporting this issue. We think, that is already fixed in our development version and consequently the change will be included in the next release.

We would like to ask you to use a development build from https://builds.jabref.org/master and report back if it works for you. Please remember to make a backup of your library before trying-out this version.

tobiasdiez added component: shared-database [outdated] type: bug Confirmed bugs or reports that are very likely to be bugs labels Feb 17, 2018

lenhard removed the [outdated] type: bug Confirmed bugs or reports that are very likely to be bugs label Feb 17, 2018

koppor mentioned this issue Feb 20, 2018

Connection error to a host other than localhost #3118

Closed

lenhard added the help wanted label Feb 22, 2018

koppor mentioned this issue Feb 25, 2020

Remove MySQL support #6014

Closed

Siedlerchr closed this as completed Dec 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Entry Editor extremely slow with shared database on remote MySQL server #3736

Entry Editor extremely slow with shared database on remote MySQL server #3736

memering commented Feb 17, 2018

koppor commented Feb 20, 2018

lenhard commented Feb 21, 2018

memering commented Feb 21, 2018

lenhard commented Feb 22, 2018 •

edited

Loading

Siedlerchr commented Mar 21, 2018

koppor commented Mar 21, 2018

obraliar commented Mar 22, 2018

koppor commented Mar 22, 2018

Siedlerchr commented Mar 22, 2018

obraliar commented Mar 22, 2018

tobiasdiez commented Mar 22, 2018

obraliar commented Mar 22, 2018

lenhard commented Mar 22, 2018

Siedlerchr commented Mar 22, 2018

Siedlerchr commented Mar 22, 2018

tobiasdiez commented Mar 22, 2018 •

edited

Loading

bruot commented Jul 14, 2019

Siedlerchr commented Dec 13, 2020

Entry Editor extremely slow with shared database on remote MySQL server #3736

Entry Editor extremely slow with shared database on remote MySQL server #3736

Comments

memering commented Feb 17, 2018

koppor commented Feb 20, 2018

lenhard commented Feb 21, 2018

memering commented Feb 21, 2018

lenhard commented Feb 22, 2018 • edited Loading

Siedlerchr commented Mar 21, 2018

koppor commented Mar 21, 2018

obraliar commented Mar 22, 2018

koppor commented Mar 22, 2018

Siedlerchr commented Mar 22, 2018

obraliar commented Mar 22, 2018

tobiasdiez commented Mar 22, 2018

obraliar commented Mar 22, 2018

lenhard commented Mar 22, 2018

Siedlerchr commented Mar 22, 2018

Siedlerchr commented Mar 22, 2018

tobiasdiez commented Mar 22, 2018 • edited Loading

bruot commented Jul 14, 2019

Siedlerchr commented Dec 13, 2020

lenhard commented Feb 22, 2018 •

edited

Loading

tobiasdiez commented Mar 22, 2018 •

edited

Loading