-
Notifications
You must be signed in to change notification settings - Fork 494
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
* Fixes #4253: Missing apache arrow/jsonParams documentation * Added missing apache arrow/jsonParams documentation * fix typos * changes review
- Loading branch information
Showing
24 changed files
with
876 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
[[export-arrow]] | ||
= Export to Apache Arrow | ||
:description: This section describes procedures that can be used to export data in Apache Arrow format. | ||
|
||
These procedures export data into a format that's used by many Apache and non-Apache tools. | ||
|
||
|
||
[[export-arrow-available-procedures]] | ||
== Available Procedures | ||
|
||
The table below describes the available procedures: | ||
|
||
[separator=¦,opts=header,cols="5,1m"] | ||
|=== | ||
¦Qualified Name¦Type | ||
¦xref::overview/apoc.export/apoc.export.arrow.all.adoc[apoc.export.arrow.all icon:book[]] + | ||
`apoc.export.arrow.all(file STRING, config MAP<STRING, ANY>)` - exports the full database as an arrow file. | ||
¦label:procedure[] | ||
|
||
¦xref::overview/apoc.export/apoc.export.arrow.graph.adoc[apoc.export.arrow.graph.adoc icon:book[]] + | ||
`apoc.export.arrow.graph(file STRING, graph ANY, config MAP<STRING, ANY>)` - exports the given graph as an arrow file. | ||
¦label:procedure[] | ||
|
||
¦xref::overview/apoc.export/apoc.export.arrow.query.adoc[apoc.export.arrow.query.adoc icon:book[]] + | ||
`apoc.export.arrow.stream.all(config MAP<STRING, ANY>)` - exports the full database as an arrow byte array. | ||
¦label:procedure[] | ||
|
||
¦xref::overview/apoc.export/apoc.export.arrow.stream.all.adoc[apoc.export.arrow.stream.all icon:book[]] + | ||
`apoc.export.arrow.all(file STRING, config MAP<STRING, ANY>)` - exports the full database as an arrow file. | ||
¦label:procedure[] | ||
|
||
¦xref::overview/apoc.export/apoc.export.arrow.stream.graph.adoc[apoc.export.arrow.stream.graph.adoc icon:book[]] + | ||
`apoc.export.arrow.stream.graph(graph ANY, config MAP<STRING, ANY>)` - exports the given graph as an arrow byte array. | ||
¦label:procedure[] | ||
|
||
¦xref::overview/apoc.export/apoc.export.arrow.stream.query.adoc[apoc.export.arrow.stream.query.adoc icon:book[]] + | ||
`apoc.export.arrow.stream.query(query ANY, config MAP<STRING, ANY>)` - exports the given Cypher query as an arrow byte array. | ||
¦label:procedure[] | ||
|=== | ||
|
||
|
||
[[export-arrow-file-export]] | ||
== Exporting to a file | ||
|
||
include::partial$enableFileExport.adoc[] | ||
|
||
|
||
[[export-arrow-examples]] | ||
== Examples | ||
|
||
|
||
[[export-cypher-query-arrow]] | ||
=== Export results of Cypher query to Apache Arrow file | ||
|
||
[source,cypher] | ||
---- | ||
CALL apoc.export.arrow.query('query_test.arrow', | ||
"RETURN 1 AS intData, 'a' AS stringData, | ||
true AS boolData, | ||
[1, 2, 3] AS intArray, | ||
[1.1, 2.2, 3.3] AS doubleArray, | ||
[true, false, true] AS boolArray, | ||
[1, '2', true, null] AS mixedArray, | ||
{foo: 'bar'} AS mapData, | ||
localdatetime('2015-05-18T19:32:24') as dateData, | ||
[[0]] AS arrayArray, | ||
1.1 AS doubleData" | ||
) YIELD file | ||
---- | ||
|
||
.Results | ||
[opts="header"] | ||
|=== | ||
| file | source | format | nodes | relationships | properties | time | rows | batchSize | batches | done | data | ||
| "query_test.arrow" | "statement: cols(11)" | "arrow" | 0 | 0 | 11 | 468 | 11 | 2000 | 1 | true | <null> | ||
|=== | ||
|
||
|
||
|
||
[[export-cypher-query-arrow-stream]] | ||
=== Export results of Cypher query to Apache Arrow binary output | ||
|
||
[source,cypher] | ||
---- | ||
CALL apoc.export.arrow.stream.query('query_test.arrow', | ||
"RETURN 1 AS intData, 'a' AS stringData, | ||
true AS boolData, | ||
[1, 2, 3] AS intArray, | ||
[1.1, 2.2, 3.3] AS doubleArray, | ||
[true, false, true] AS boolArray, | ||
[1, '2', true, null] AS mixedArray, | ||
{foo: 'bar'} AS mapData, | ||
localdatetime('2015-05-18T19:32:24') as dateData, | ||
[[0]] AS arrayArray, | ||
1.1 AS doubleData" | ||
) YIELD value | ||
---- | ||
|
||
.Results | ||
[opts="header"] | ||
|=== | ||
| value | ||
| <binary Apache Arrow output> | ||
|=== |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,121 @@ | ||
[[load-arrow]] | ||
= Load and Import Arrow | ||
:description: This section describes procedures that can be used to import Apache Arrow data from web APIs or files. | ||
|
||
|
||
|
||
The following procedures allow you to read an Apache Arrow file exported via xref::export/arrow.adoc[apoc.export.arrow.* procedures]. | ||
It could also potentially read other Apache Arrow files that have not been created via the export procedures. | ||
|
||
[[load-arrow-available-procedures]] | ||
== Procedure and Function Overview | ||
|
||
The table below describes the available procedures and functions: | ||
|
||
[separator=¦,opts=header,cols="5,1m"] | ||
|=== | ||
¦Qualified Name¦Type | ||
¦xref::overview/apoc.load/apoc.load.arrow.adoc[apoc.load.arrow icon:book[]] + | ||
`apoc.load.arrow(file STRING, config MAP<STRING, ANY>)` - loads values from the provided Arrow file. | ||
¦label:procedure[] | ||
|
||
¦xref::overview/apoc.load/apoc.load.arrow.stream.adoc[apoc.load.arrow.stream.adoc icon:book[]] + | ||
`apoc.load.arrow.stream(source LIST<INTEGER>, config MAP<STRING, ANY>)` - loads values from the provided Arrow byte array. | ||
¦label:procedure[] | ||
|
||
¦xref::overview/apoc.import/apoc.import.arrow.adoc[apoc.import.arrow icon:book[]] + | ||
`apoc.import.arrow(urlOrBinaryFile ANY, config MAP<STRING, ANY>)` - imports entities from the provided Arrow file or byte array | ||
¦label:procedure[] | ||
|=== | ||
|
||
|
||
|
||
[[load-arrow-available-procedures-apoc.load.arrow]] | ||
=== `apoc.load.arrow` | ||
|
||
|
||
This procedure takes a file or HTTP URL and parses the Apache Arrow into a map data structure. | ||
|
||
[separator=¦,opts=header,cols="1m"] | ||
|=== | ||
¦signature | ||
¦apoc.load.arrow(file :: STRING, config = {} :: MAP) :: (value :: MAP) | ||
|=== | ||
|
||
|
||
Currently, this procedure does not support any config parameters. | ||
|
||
include::includes/enableFileImport.adoc[] | ||
|
||
[[load-arrow-available-procedures-apoc.load.arrow.stream]] | ||
=== `apoc.load.arrow.stream` | ||
|
||
|
||
This procedure takes a byte[] source and parses the Apache Arrow into a map data structure. | ||
|
||
[separator=¦,opts=header,cols="1m"] | ||
|=== | ||
¦signature | ||
¦apoc.load.arrow.stream(source :: LIST<INTEGER>, config = {} :: MAP) :: (value :: MAP) | ||
|=== | ||
|
||
Currently, this procedure does not support any config parameters. | ||
|
||
|
||
[[load-arrow-examples]] | ||
== Examples | ||
|
||
The following section contains examples showing how to import data from various Apache Arrow sources. | ||
|
||
[[load-arrow-examples-local-file]] | ||
=== Import from local file | ||
|
||
Taking the output xref::export/arrow.adoc#export-cypher-query-arrow[of this case]: | ||
|
||
.The following query processes a `test.arrow` file and returns the content as Cypher data structures | ||
[source,cypher] | ||
---- | ||
CALL apoc.load.arrow('test.arrow') | ||
YIELD value | ||
RETURN value | ||
---- | ||
|
||
.Results | ||
[options="header"] | ||
|=== | ||
| value | ||
| {arrayArray -> ["[0]"], dateData -> 2015-05-18T19:32:24Z, boolArray -> [true,false,true], intArray -> [1,2,3], mapData -> "{"foo":"bar"}", boolData -> true, intData -> 1, mixedArray -> ["1","2","true",<null>], doubleArray -> [1.1,2.2,3.3], doubleData -> 1.1, stringData -> "a"} | ||
|=== | ||
|
||
|
||
[[load-arrow-examples-binary-source]] | ||
=== Import from binary source | ||
|
||
Taking the output xref::export/arrow.adoc#export-cypher-query-arrow-stream[of this case]: | ||
|
||
.The following query processes a `test.arrow` file and returns the content as Cypher data structures | ||
[source,cypher] | ||
---- | ||
CALL apoc.load.arrow.stream('<binary arrow file>') | ||
YIELD value | ||
RETURN value | ||
---- | ||
|
||
.Results | ||
[options="header"] | ||
|=== | ||
| value | ||
| {arrayArray -> ["[0]"], dateData -> 2015-05-18T19:32:24Z, boolArray -> [true,false,true], intArray -> [1,2,3], mapData -> "{"foo":"bar"}", boolData -> true, intData -> 1, mixedArray -> ["1","2","true",<null>], doubleArray -> [1.1,2.2,3.3], doubleData -> 1.1, stringData -> "a"} | ||
|=== | ||
|
||
|
||
[[import-arrow]] | ||
=== Import Arrow file created by Export Arrow procedures | ||
|
||
The `apoc.import.arrow` procedure can be used to import Apache Arrow files created by the xref::export/arrow.adoc[apoc.export.arrow.*] procedures. | ||
|
||
This procedure should not be confused with the `apoc.load.arrow*` procedures, | ||
which just loads the values of the Arrow file, and does not create entities in the database. | ||
|
||
See xref::overview/apoc.import/apoc.import.arrow.adoc[this page] for more info. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
[[apoc.load.jsonParams]] | ||
= Load JSON | ||
:description: This section contains reference documentation for the apoc.load.jsonParams procedure. | ||
|
||
label:procedure[] label:apoc-core[] | ||
|
||
[.emphasis] | ||
==== | ||
apoc.load.jsonParams('urlOrBinary',\{headers\}, \{payload\}, $config) YIELD value | ||
It loads from JSON URL, a RestAPI endpoint URL, or from a binary source. | ||
The headers and the payload parameters can be used to execute a RestAPI. | ||
The procedure will return multiple values if the starting JSON is a list of JSON or a single result in case it is a single JSON. | ||
==== | ||
|
||
== Signature | ||
|
||
[source] | ||
---- | ||
apoc.load.jsonParams(urlOrKeyOrBinary :: ANY?, headers :: MAP?, payload :: STRING?, path = :: STRING?, config = {} :: MAP?) :: (value :: MAP?) | ||
---- | ||
|
||
== Input parameters | ||
[.procedures, opts=header] | ||
|=== | ||
| Name | Type | Default | ||
|urlOrKeyOrBinary|ANY?|null | ||
|headers|MAP?|null | ||
|payload|STRING?|null | ||
|path|STRING?| | ||
|config|MAP?|{} | ||
|=== | ||
|
||
== Output parameters | ||
[.procedures, opts=header] | ||
|=== | ||
| Name | Type | ||
|value|MAP? | ||
|=== | ||
|
||
== Reading from a file | ||
include::includes/enableFileImport.adoc[] | ||
|
||
[[usage-apoc.load.jsonParams]] | ||
== Usage Examples | ||
include::partial$usage/apoc.load.jsonParams.adoc[] | ||
|
56 changes: 56 additions & 0 deletions
56
docs/asciidoc/modules/ROOT/pages/overview/apoc.export/apoc.export.arrow.all.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
= apoc.export.arrow.all | ||
:description: This section contains reference documentation for the apoc.export.arrow.all procedure. | ||
|
||
label:procedure[] | ||
|
||
[.emphasis] | ||
`apoc.export.arrow.all(file STRING, config MAP<STRING, ANY>)` - exports the full database as an arrow file. | ||
|
||
[NOTE] | ||
==== | ||
It is considered unsafe to run this procedure using multiple threads. | ||
It is therefore not supported by the parallel runtime (introduced in Neo4j 5.13). | ||
For more information, see the link:{neo4j-docs-base-uri}/cypher-manual/{page-version}/planning-and-tuning/runtimes/concepts#runtimes-parallel-runtime[Cypher Manual -> Parallel runtime]. | ||
==== | ||
|
||
== Signature | ||
|
||
[source] | ||
---- | ||
apoc.export.arrow.all(file :: STRING, config = {} :: MAP) :: (file :: STRING, source :: STRING, format :: STRING, nodes :: INTEGER, relationships :: INTEGER, properties :: INTEGER, time :: INTEGER, rows :: INTEGER, batchSize :: INTEGER, batches :: INTEGER, done :: BOOLEAN, data :: ANY) | ||
---- | ||
|
||
== Input parameters | ||
[.procedures, opts=header] | ||
|=== | ||
| Name | Type | Default | ||
|file|STRING|null | ||
|config|MAP|null | ||
|=== | ||
|
||
== Output parameters | ||
[.procedures, opts=header] | ||
|=== | ||
| Name | Type | ||
|file|STRING | ||
|source|STRING | ||
|format|STRING | ||
|nodes|INTEGER | ||
|relationships|INTEGER | ||
|properties|INTEGER | ||
|time|INTEGER | ||
|rows|INTEGER | ||
|batchSize|INTEGER | ||
|batches|INTEGER | ||
|done|BOOLEAN | ||
|data|STRING | ||
|=== | ||
|
||
== Config parameters | ||
include::partial$usage/config/apoc.export.arrow.all.adoc[] | ||
|
||
[[usage-apoc.export.arrow.all]] | ||
== Usage Examples | ||
include::partial$usage/apoc.export.arrow.all.adoc[] | ||
|
||
xref::export/arrow.adoc[More documentation of apoc.export.arrow.all,role=more information] |
Oops, something went wrong.