Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] Serializer for mdspans #1173

Merged
merged 40 commits into from
Feb 2, 2023
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
7b6e4ba
Implement mdspan serializer
hcho3 Jan 24, 2023
8f012d5
Implement deserializer
hcho3 Jan 24, 2023
4e00e4a
Merge branch 'branch-23.02' into mdspan_serializer
cjnolet Jan 25, 2023
9d4b332
Update ANN to use the new mdspan serializer
hcho3 Jan 25, 2023
71d1b0a
Fix copyright year
hcho3 Jan 25, 2023
7391313
Solve link error due to duplicated symbols
hcho3 Jan 25, 2023
03efe03
Add gtest to test serializing device_mdspan
hcho3 Jan 26, 2023
1dc2ce6
Merge remote-tracking branch 'origin/branch-23.02' into mdspan_serial…
hcho3 Jan 26, 2023
1b22df4
Rename header to serialize.hpp
hcho3 Jan 26, 2023
cf13fd5
Implement scalar serializer
hcho3 Jan 26, 2023
6be4c0b
Merge remote-tracking branch 'origin/branch-23.02' into mdspan_serial…
hcho3 Jan 26, 2023
d9062b2
Use device_resources instead of handle_t
hcho3 Jan 26, 2023
b3477d0
Use 64-bit alignment
hcho3 Jan 26, 2023
268e55f
Move static_assert
hcho3 Jan 26, 2023
1770f12
Ensure data have finished copying
hcho3 Jan 26, 2023
f53957a
Serialize scalars as 0D NumPy array
hcho3 Jan 26, 2023
2cf24e5
Check endianness when deserializing
hcho3 Jan 26, 2023
3f129d4
Add missing return
hcho3 Jan 26, 2023
b1d0a75
Fix gtest
hcho3 Jan 26, 2023
f2f75c0
Rename gtest file to numpy_serializer.cu
hcho3 Jan 26, 2023
63cd19e
Bump up serialization version for IVF
hcho3 Jan 26, 2023
b4ff38e
Fix endian check, to account for 1-byte types
hcho3 Jan 26, 2023
79a7f0a
Merge remote-tracking branch 'origin/branch-23.02' into mdspan_serial…
hcho3 Jan 26, 2023
145bdde
Test RoundTrip for multiple data types
hcho3 Jan 27, 2023
b64146c
Add gtest for scalars
hcho3 Jan 27, 2023
3a94fef
Slim down gtest NumPySerializerMDSpan.E2ERoundTrip
hcho3 Jan 27, 2023
65fdf53
Set up Pytest for mdspan serialization
hcho3 Jan 27, 2023
30b4238
Merge branch 'branch-23.02' into mdspan_serializer
cjnolet Jan 28, 2023
3f7481d
Merge remote-tracking branch 'origin/branch-23.02' into mdspan_serial…
hcho3 Jan 31, 2023
30733f3
Rename load/save to (de)serialize
hcho3 Jan 31, 2023
4c025de
Don't export dextents from raft
hcho3 Jan 31, 2023
0196f00
Make serialization_version a constexpr
hcho3 Jan 31, 2023
1cd63a8
Implement serializer for managed_mdspan
hcho3 Jan 31, 2023
a91c64a
Fix build error
hcho3 Jan 31, 2023
42eccbc
Merge remote-tracking branch 'origin/branch-23.02' into mdspan_serial…
hcho3 Jan 31, 2023
cf83274
Revert spurious copyright year updates
hcho3 Jan 31, 2023
3cf5a79
Revert spurious copyright year updates
hcho3 Jan 31, 2023
3bd6f13
Add static_assert about size
hcho3 Jan 31, 2023
dca29a9
Merge branch 'branch-23.02' into mdspan_serializer
cjnolet Feb 1, 2023
9bb0043
Setting parallel_level-8
cjnolet Feb 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# =============================================================================
# Copyright (c) 2020-2022, NVIDIA CORPORATION.
# Copyright (c) 2020-2023, NVIDIA CORPORATION.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except
# in compliance with the License. You may obtain a copy of the License at
Expand Down Expand Up @@ -218,6 +218,15 @@ target_link_libraries(

target_compile_features(raft INTERFACE cxx_std_17 $<BUILD_INTERFACE:cuda_std_17>)

# Endian detection
include(TestBigEndian)
test_big_endian(BIG_ENDIAN)
if(BIG_ENDIAN)
target_compile_definitions(raft INTERFACE RAFT_SYSTEM_LITTLE_ENDIAN=0)
else()
target_compile_definitions(raft INTERFACE RAFT_SYSTEM_LITTLE_ENDIAN=1)
endif()

if(RAFT_COMPILE_DIST_LIBRARY OR RAFT_COMPILE_NN_LIBRARY)
file(
WRITE "${CMAKE_CURRENT_BINARY_DIR}/fatbin.ld"
Expand Down
Loading