WIP: Tables for labels metadata, mirroring CALL db.schema.(nodeTypeProperties|relTypeProperties) #1318
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 4905f19.
|
@moxious are you good with this? |
|
Feature complete at this point, this PR could use a detailed code feedback. I haven't sorted out by Travis which test is failing though. When Shashi joins I'm going to have him engage here. |
|
I take back previous comment -- not yet feature complete. We're investigating adding relationship domain and range information to improve performance for rel table generation. Shashi will be assisting with this PR as well. Pause for a bit on code review. |
|
This PR is superceded by Shash's here: #1389 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement in APOC a version of
db.schema.nodeTypeProperties()anddb.schema.relTypeProperties()that permit flexible probabilistic sampling, rather than the full DB scan that those others do.These procedures are key because they provide metadata output in a very specific format that permits the "Tables for Labels" mapping. T4L can be thought of as a basic/naive Graph->Tables mapping. This has lots of uses, the most particular one I have in mind is for use with the BI driver that's in progress. Tools can call these functions to build a virtual RDBMS schema over top of Neo4j which can be queried, basically.
The trouble with the product built-in procedures is that they don't use sampling, they scan the entire DB, so for large DBs this will churn the page cache and perform very badly.
This approach has been discussed with Mats & the internal Morpheus team who wrote the product versions of the stored procedures, and has generally been agreed on as the best available approach.
This is just a code spike right now for visibility.
A design document describing how this is supposed to work and what behavior should be is here: https://docs.google.com/document/d/1-U-L5anu50CGHXYipjdj5AwdeFprd7lvggD97fbqHeA/edit?usp=sharing