Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lucene Spatial Index is closed without any reason #7585

Closed
EricSchreiner opened this issue Jul 25, 2017 · 21 comments
Closed

Lucene Spatial Index is closed without any reason #7585

EricSchreiner opened this issue Jul 25, 2017 · 21 comments
Assignees
@robfrank
Milestone
2.2.26

Comments

@EricSchreiner
Copy link

OrientDB Version: 2.2.24 (Embedded API)

Java Version: 1.8

OS: Windows 10

Expected behavior

Lucene Spatial Index is closed without any reason. ( org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed) This does not happen in 2.2.22 or 2.2.23

Actual behavior

ERROR@ 15:18:08.644 Exception Error PhotoCrawler.DirectoryConsumer.handlePhotoUpdate:D:\Arbeit\Photos\1988\Portugal\Portugal-Spanien-DEV-PIC_0104.jpg: com.orientechnologies.orient.core.exception.ODatabaseException: Error during saving of record with rid #25:1755 DB name="db.2.2.22"
EXCEP@ ============================================================
EXCEP@ Exception at: 2017-07-25 15:18:08
EXCEP@ Msg:
EXCEP@ Error during saving of record with rid #25:1755
DB name="db.2.2.22"
EXCEP@ ------------------------------------------------------------
EXCEP@ com.orientechnologies.orient.core.exception.ODatabaseException: Error during saving of record with rid #25:1755
EXCEP@ DB name="db.2.2.22"
EXCEP@ at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:203)
EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:2758)
EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.save(ODatabaseDocumentTx.java:2638)
EXCEP@ at de.contecon.picapport.db.Photo.saveToDocument(Photo.java:226)
EXCEP@ at de.contecon.picapport.db.Photo.insert(Photo.java:169)
EXCEP@ at de.contecon.picapport.db.DbWrapper.insertPhoto(DbWrapper.java:417)
EXCEP@ at de.contecon.picapport.db.DbWrapper.updatePhoto(DbWrapper.java:352)
EXCEP@ at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.handlePhotoUpdate(PhotoCrawler.java:403)
EXCEP@ at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.consume(PhotoCrawler.java:278)
EXCEP@ at de.contecon.picapport.directoryservices.PhotoCrawler$DirectoryConsumer.run(PhotoCrawler.java:190)
EXCEP@ at java.lang.Thread.run(Unknown Source)
EXCEP@ Caused by: com.orientechnologies.orient.core.exception.OStorageException: Cannot put key value entry in index
EXCEP@ DB name="db.2.2.22"
EXCEP@ at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateIndexEntry(OAbstractPaginatedStorage.java:2256)
EXCEP@ at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.updateIndexEntry(OAbstractPaginatedStorage.java:2153)
EXCEP@ at com.orientechnologies.lucene.index.OLuceneIndexNotUnique.put(OLuceneIndexNotUnique.java:317)
EXCEP@ at com.orientechnologies.spatial.index.OLuceneSpatialIndex.put(OLuceneSpatialIndex.java:50)
EXCEP@ at com.orientechnologies.spatial.index.OLuceneSpatialIndex.put(OLuceneSpatialIndex.java:35)
EXCEP@ at com.orientechnologies.orient.core.index.OClassIndexManager.putInIndex(OClassIndexManager.java:659)
EXCEP@ at com.orientechnologies.orient.core.index.OClassIndexManager.addIndexEntry(OClassIndexManager.java:554)
EXCEP@ at com.orientechnologies.orient.core.index.OClassIndexManager.addIndexesEntries(OClassIndexManager.java:541)
EXCEP@ at com.orientechnologies.orient.core.index.OClassIndexManager.onRecordAfterCreate(OClassIndexManager.java:419)
EXCEP@ at com.orientechnologies.orient.core.hook.ODocumentHookAbstract.onTrigger(ODocumentHookAbstract.java:221)
EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.callbackHooks(ODatabaseDocumentTx.java:1144)
EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.callbackHookSuccess(ODatabaseDocumentTx.java:3273)
EXCEP@ at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.executeSaveRecord(ODatabaseDocumentTx.java:2237)
EXCEP@ at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveNew(OTransactionNoTx.java:241)
EXCEP@ at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTransactionNoTx.java:171)
EXCEP@ ... 10 more
EXCEP@ Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed
EXCEP@ at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:719)
EXCEP@ at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:733)
EXCEP@ at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1471)
EXCEP@ at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
EXCEP@ at org.apache.lucene.index.TrackingIndexWriter.addDocument(TrackingIndexWriter.java:112)
EXCEP@ at com.orientechnologies.lucene.engine.OLuceneIndexEngineAbstract.addDocument(OLuceneIndexEngineAbstract.java:155)
EXCEP@ at com.orientechnologies.spatial.engine.OLuceneLegacySpatialIndexEngine.put(OLuceneLegacySpatialIndexEngine.java:167)
EXCEP@ at com.orientechnologies.spatial.engine.OLuceneSpatialIndexEngineDelegate.put(OLuceneSpatialIndexEngineDelegate.java:144)
EXCEP@ at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doUpdateIndexEntry(OAbstractPaginatedStorage.java:2242)
EXCEP@ ... 24 more

Steps to reproduce
@robfrank
Copy link
Contributor

I'm fixing. BTW, the legacy spatial index is deprecated and will be removed in the future.
My suggestion is to migrate to Spatial Module: http://orientdb.com/docs/last/Spatial-Index.html

@EricSchreiner
Copy link
Author

??? We are using the Spatial Module (http://central.maven.org/maven2/com/orientechnologies/orientdb-spatial/2.2.23/orientdb-spatial-2.2.23-dist.jar)

@robfrank
Copy link
Contributor

ok, but you are using the legacy part: http://orientdb.com/docs/last/Spatial-Index.html#legacy

not the one with functions

@robfrank
Copy link
Contributor

just pushed the fix: orientechnologies/orientdb-spatial@0fec0b9
Can you test with the latest snapshot?
https://oss.sonatype.org/content/repositories/snapshots/com/orientechnologies/orientdb-spatial/2.2.25-SNAPSHOT/orientdb-spatial-2.2.25-20170725.155011-12-dist.jar

just put this jar instead of the 2.2.24

@EricSchreiner
Copy link
Author

..this is what we do:DbSchema.createSchema: Photo.longitude (type=DOUBLE) may be null (CREATE INDEX idxlocation ON Photo(latitude,longitude) SPATIAL ENGINE LUCENE METADATA {ignoreNullValues: true})
Will this be removed? What should we do instead?

@lvca
Copy link
Member

lvca commented Jul 25, 2017 •
edited
Loading

@EricSchreiner did you try latest fix from @robfrank ?

@robfrank
Copy link
Contributor

Starting from 2.2, the new spatial module allow indexing of points and other spatial structures, such as lines and polygons:

http://orientdb.com/docs/last/Spatial-Index.html#geometry-data

It needs that spatial data are stored in specilized classes:

http://orientdb.com/docs/last/Spatial-Index.html#geometry-data-example

and offers a lot of functions (ST_: Spatial type from The Open Geospatial Consortium) to handle spatial queries:

http://orientdb.com/docs/last/Spatial-Index.html#functions

The legacy api can index only points and offers only near and within operators.

So, don't worry, the legacy API will be there on 2.2.x. But if you can migrate to the new api, you will gain a lot of benefits.

@EricSchreiner
Copy link
Author

EricSchreiner commented Jul 26, 2017 •
edited
Loading

Hi @robfrank, @lvca your Fix seems to work :-) when can we expect an official 2.2.25?

@lvca
Copy link
Member

lvca commented Jul 26, 2017

Cool, closing it.

@lvca lvca closed this as completed Jul 26, 2017
@lvca lvca added this to the 2.2.x (next hotfix) milestone Jul 26, 2017
@EricSchreiner
Copy link
Author

@lvca any idea when the next hotfix will come?

@robfrank
Copy link
Contributor

we have just released 2.2.24, I think at least 2 weeks.

@EricSchreiner
Copy link
Author

... 2 weeks for a serious Bug seems to be a long time.....

@lvca
Copy link
Member

lvca commented Jul 26, 2017

Is this issue fixed in 2.2.x? Confirmed?

@luigidellaquila
Copy link
Member

@lvca yes, it's fixed, maybe we can release sooner...?

Luigi

@robfrank robfrank modified the milestones: 2.2.x (next hotfix), 2.2.25 Aug 2, 2017
@EricSchreiner EricSchreiner mentioned this issue Aug 10, 2017
Closed
@andrii0lomakin
Copy link
Member

Reproduced on latest snapshot

@robfrank
Copy link
Contributor

@Laa he uses a 2.2.23 with lucene jar from 2.2.26-SNAPSHOT. But this was fixed on 2.2.25.

@andrii0lomakin
Copy link
Member

@robfrank sorry issue still persist in 2.2.26

@andrii0lomakin
Copy link
Member

You see this from the latest log published in #7586

@robfrank
Copy link
Contributor

Ok, there was still a sync problem on autoclose of indexes. Testing

robfrank added a commit that referenced this issue Aug 11, 2017
@robfrank
fixes sync on open/close cycle
6df15ce 
refs #7585
robfrank added a commit to orientechnologies/orientdb-spatial that referenced this issue Aug 11, 2017
@robfrank
fixes sync on open/close cycle
d09d20c 
refs orientechnologies/orientdb#7585
@santo-it santo-it modified the milestones: 2.2.26, 2.2.x (next hotfix) Aug 16, 2017
@robfrank
Copy link
Contributor

this problem should be fixed in 2.2.26. Can confirm?

@EricSchreiner
Copy link
Author

@robfrank confirmed

@robfrank robfrank modified the milestones: 2.2.26, 2.2.x (next hotfix) Aug 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@robfrank robfrank

Labels
None yet
Milestone
2.2.26
Development

No branches or pull requests
7 participants
@andrii0lomakin @lvca @robfrank @luigidellaquila @EricSchreiner @prjhub @santo-it