Improve integration testing docs

arrow-flight/Cargo.toml

@@ -43,7 +43,3 @@ tonic-build = "0.6"
 # Pin specific version of the tonic-build dependencies to avoid auto-generated
 # (and checked in) arrow.flight.protocol.rs from changing
 proc-macro2 = ">1.0.30"
-
-#[lib]
-#name = "flight"
-#path = "src/lib.rs"

arrow-pyarrow-integration-testing/README.md

@@ -17,7 +17,7 @@
   under the License.
 -->
 
-# Arrow c integration
+# Arrow + PyArrow integration testing
 
 This is a Rust crate that tests compatibility between Rust's Arrow implementation and PyArrow.
 

integration-testing/README.md

@@ -19,12 +19,76 @@
 
 # Apache Arrow Rust Integration Testing
 
-See [Integration.rst](../../docs/source/format/Integration.rst) for an overview of integration testing.
+See [Integration Testing](https://arrow.apache.org/docs/format/Integration.html) for an overview of integration testing.
 
 This crate contains the following binaries, which are invoked by Archery during integration testing with other Arrow implementations.
 
-| Binary                      | Purpose                                   |
-| --------------------------- | ----------------------------------------- |
-| arrow-file-to-stream        | Converts an Arrow file to an Arrow stream |
-| arrow-stream-to-file        | Converts an Arrow stream to an Arrow file |
-| arrow-json-integration-test | Converts between Arrow and JSON formats   |
+| Binary                         | Purpose                                   |
+| ------------------------------ | ----------------------------------------- |
+| arrow-file-to-stream           | Converts an Arrow file to an Arrow stream |
+| arrow-stream-to-file           | Converts an Arrow stream to an Arrow file |
+| arrow-json-integration-test    | Converts between Arrow and JSON formats   |
+| flight-test-integration-server | Flight integration test: Server           |
+| flight-test-integration-client | Flight integration test: Client           |
+
+# Notes on how to run Rust Integration Test against C/C++
+
+The code for running the integration tests is in the [arrow](https://github.com/apache/arrow) repository
+
+### Check out code:
+
+```shell
+# check out arrow
+git clone [email protected]:apache/arrow.git
+# link rust source code into arrow
+ln -s <path_to_arrow_rs> arrow/rust
+```
+
+### Install the tools:
+
+```shell
+cd arrow
+pip install -e dev/archery[docker]
+```
+
+### Build the C++ binaries:
+
+Follow the [C++ Direction](https://github.com/apache/arrow/tree/master/docs/source/developers/cpp) and build the integration test binaries with a command like this:
+
+```
+# build cpp binaries
+cd arrow/cpp
+mkdir build
+cd  build
+cmake  -DARROW_BUILD_INTEGRATION=ON -DARROW_FLIGHT=ON --preset ninja-debug-minimal ..
+cd ..
+ninja
+```
+
+### Build the Rust binaries
+
+Then
+
+```
+# build rust:
+cd ../arrow-rs
+cargo build --all
+```
+
+### Run archery
+
+You can run the Archery tool using a command such as the following:
+
+```shell
+archery integration --with-cpp=true --with-rust=true
+```
+
+To debug an individual test scenario, it is also possible to run the binaries directly:
+
+```shell
+# Run cpp server
+$ arrow/cpp/build/debug/flight-test-integration-server -port 49153
+
+# run rust client (you can see file names if you run archery --debug
+$ arrow/rust/target/debug/flight-test-integration-client --host localhost --port=49153 --path /tmp/generated_dictionary_unsigned.json
+```

integration-testing/src/bin/flight-test-integration-client.rs

@@ -37,7 +37,7 @@ struct Args {
     #[clap(
         short,
         long,
-        help = "path to the descriptor file, only used when scenario is not provided"
+        help = "path to the descriptor file, only used when scenario is not provided. See https://arrow.apache.org/docs/format/Integration.html#json-test-data-format"
     )]
     path: Option<String>,
     #[clap(long, arg_enum)]