Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: adding blocking in a table convert function #2625

Conversation

Alexandr-Solovev
Copy link
Contributor

@Alexandr-Solovev Alexandr-Solovev commented Jan 5, 2024

Description

Its feature for adding support of extremely big datasets in all algorithms

@Alexandr-Solovev
Copy link
Contributor Author

/intelci: run

@Alexandr-Solovev
Copy link
Contributor Author

/intelci: run

@Alexandr-Solovev
Copy link
Contributor Author

/intelci: run

@Alexandr-Solovev
Copy link
Contributor Author

/intelci: run

@Alexandr-Solovev Alexandr-Solovev changed the title init debug prints for table converter feature: adding blocking in table convert functions Jan 8, 2024
@Alexandr-Solovev
Copy link
Contributor Author

/intelci: run

@Alexandr-Solovev Alexandr-Solovev marked this pull request as ready for review January 8, 2024 11:42
@Alexandr-Solovev Alexandr-Solovev changed the title feature: adding blocking in table convert functions feature: adding blocking in a table convert function Jan 8, 2024
@Alexandr-Solovev
Copy link
Contributor Author

/intelci: run

@Alexandr-Solovev
Copy link
Contributor Author

/intelci: run

cpp/oneapi/dal/backend/transfer_dpc.cpp Outdated Show resolved Hide resolved
cpp/oneapi/dal/backend/transfer_dpc.cpp Outdated Show resolved Hide resolved
cpp/oneapi/dal/backend/transfer_dpc.cpp Outdated Show resolved Hide resolved
cpp/oneapi/dal/backend/transfer_dpc.cpp Outdated Show resolved Hide resolved
cpp/oneapi/dal/backend/transfer_dpc.cpp Outdated Show resolved Hide resolved
Comment on lines +23 to +26
std::int64_t propose_block_size(const sycl::queue& q, const std::int64_t r) {
constexpr std::int64_t fsize = sizeof(Float);
return 0x10000l * (8 / fsize);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it be exposed to an optimization library?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its better to do it in another pr, due to we have a lot of local propose_block_size functions, and some of them are not aligned

cpp/oneapi/dal/backend/transfer_dpc.cpp Outdated Show resolved Hide resolved
@Alexandr-Solovev
Copy link
Contributor Author

/intelci: run

Comment on lines +110 to +112
const byte_t* const gathered_byte =
reinterpret_cast<const byte_t*>(gathered_device_unique.get());
byte_t* const dst_byte = reinterpret_cast<byte_t*>(dst_device);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is 110 const byte_t* and 112 byte_t* const?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One line is source, and other one is a consumer aka destination)

@ethanglaser
Copy link
Contributor

/intelci: run

@ethanglaser
Copy link
Contributor

Have these been changes been tested with the original spark issue?

@Alexandr-Solovev
Copy link
Contributor Author

Have these been changes been tested with the original spark issue?

it was exactly tested with the spark issue

Copy link
Contributor

@inteldimitrius inteldimitrius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Alexandr-Solovev Alexandr-Solovev merged commit 138ddd6 into uxlfoundation:main Jan 10, 2024
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants