Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated user-guide library docs with optimized config #976

Merged
merged 2 commits into from
Sep 7, 2021

Conversation

matthewmturner
Copy link
Contributor

Which issue does this PR close?

Closes #882 .

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@matthewmturner
Copy link
Contributor Author

@houqp @alamb @andygrove FYI took a first cut at this by adding to the library section of the user-guide. Let me know your thoughts.


```toml
[dependencies]
datafusion = { git = "https://github.com/apache/arrow-datafusion.git", features = ["simd"]}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
datafusion = { git = "https://github.com/apache/arrow-datafusion.git", features = ["simd"]}
datafusion = "5.0"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we dont need to keep the features = ["simd"]?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah you're right 👍 But probably we don't want people building from git, if not needed.

snmalloc-rs = {version = "0.2", features= ["cache-friendly"]}
num_cpus = "1.0"

[profile.release]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe mention that using this will increase the build time considerably

static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
```

Finally, in order to build with these optimizations `cargo nightly` is required. Based on the instruction
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nightly is only required for using simd

@houqp houqp merged commit 50cce1a into apache:master Sep 7, 2021
@houqp houqp added datafusion Changes in the datafusion crate documentation Improvements or additions to documentation labels Nov 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datafusion Changes in the datafusion crate documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Documenting building with simd feature
3 participants