Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify linking, update dependencies #2069

Merged
merged 7 commits into from
Jun 14, 2023
Merged

Simplify linking, update dependencies #2069

merged 7 commits into from
Jun 14, 2023

Conversation

julianoes
Copy link
Collaborator

@julianoes julianoes commented Jun 1, 2023

On Windows, the mavsdk_server unit tests don't link properly when building with shared libs. For whatever reason the transitive dependencies didn't get tracked from the protobu/grpc components all the way into the mavsdk_server unit tests.

I suspect that either some symbols get stripped too early, or it has to do with the export visibility in MSVC.

I couldn't quite get to the bottom of it but I realized that we could just simplify it by avoiding the little libraries and just adding all sources to the mavsdk_server library in the first place.

Let's see if it works...

Edit: same problem...

@julianoes
Copy link
Collaborator Author

The problem persists, this is what the linking error looks like:

telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::VelocityBody::`vftable'" (??_7VelocityBody@telemetry@rpc@mavsdk@@6B@) ref
erenced in function "public: __cdecl mavsdk::rpc::telemetry::VelocityBody::VelocityBody(void)" (??0VelocityBody@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MAVSDK\build
\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::PositionBody::`vftable'" (??_7PositionBody@telemetry@rpc@mavsdk@@6B@) ref
erenced in function "public: __cdecl mavsdk::rpc::telemetry::PositionBody::PositionBody(void)" (??0PositionBody@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MAVSDK\build
\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::Odometry::`vftable'" (??_7Odometry@telemetry@rpc@mavsdk@@6B@) referenced
in function "public: __cdecl mavsdk::rpc::telemetry::Odometry::Odometry(void)" (??0Odometry@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MAVSDK\build\default\src\mavsdk_
server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::DistanceSensor::`vftable'" (??_7DistanceSensor@telemetry@rpc@mavsdk@@6B@)
 referenced in function "public: __cdecl mavsdk::rpc::telemetry::DistanceSensor::DistanceSensor(void)" (??0DistanceSensor@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MA
VSDK\build\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::ScaledPressure::`vftable'" (??_7ScaledPressure@telemetry@rpc@mavsdk@@6B@)
 referenced in function "public: __cdecl mavsdk::rpc::telemetry::ScaledPressure::ScaledPressure(void)" (??0ScaledPressure@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MA
VSDK\build\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::PositionNed::`vftable'" (??_7PositionNed@telemetry@rpc@mavsdk@@6B@) refer
enced in function "public: __cdecl mavsdk::rpc::telemetry::PositionNed::PositionNed(void)" (??0PositionNed@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MAVSDK\build\defa
ult\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::VelocityNed::`vftable'" (??_7VelocityNed@telemetry@rpc@mavsdk@@6B@) refer
enced in function "public: __cdecl mavsdk::rpc::telemetry::VelocityNed::VelocityNed(void)" (??0VelocityNed@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MAVSDK\build\defa
ult\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::PositionVelocityNed::`vftable'" (??_7PositionVelocityNed@telemetry@rpc@ma
vsdk@@6B@) referenced in function "public: __cdecl mavsdk::rpc::telemetry::PositionVelocityNed::PositionVelocityNed(void)" (??0PositionVelocityNed@telemetry@rpc@mavsdk@@QEAA@XZ) [C
:\Users\Julian Oes\src\MAVSDK\build\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::GroundTruth::`vftable'" (??_7GroundTruth@telemetry@rpc@mavsdk@@6B@) refer
enced in function "public: __cdecl mavsdk::rpc::telemetry::GroundTruth::GroundTruth(void)" (??0GroundTruth@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MAVSDK\build\defa
ult\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::FixedwingMetrics::`vftable'" (??_7FixedwingMetrics@telemetry@rpc@mavsdk@@
6B@) referenced in function "public: __cdecl mavsdk::rpc::telemetry::FixedwingMetrics::FixedwingMetrics(void)" (??0FixedwingMetrics@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian
Oes\src\MAVSDK\build\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::AccelerationFrd::`vftable'" (??_7AccelerationFrd@telemetry@rpc@mavsdk@@6B
@) referenced in function "public: __cdecl mavsdk::rpc::telemetry::AccelerationFrd::AccelerationFrd(void)" (??0AccelerationFrd@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\s
rc\MAVSDK\build\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::AngularVelocityFrd::`vftable'" (??_7AngularVelocityFrd@telemetry@rpc@mavs
dk@@6B@) referenced in function "public: __cdecl mavsdk::rpc::telemetry::AngularVelocityFrd::AngularVelocityFrd(void)" (??0AngularVelocityFrd@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Use
rs\Julian Oes\src\MAVSDK\build\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::MagneticFieldFrd::`vftable'" (??_7MagneticFieldFrd@telemetry@rpc@mavsdk@@
6B@) referenced in function "public: __cdecl mavsdk::rpc::telemetry::MagneticFieldFrd::MagneticFieldFrd(void)" (??0MagneticFieldFrd@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian
Oes\src\MAVSDK\build\default\src\mavsdk_server\test\unit_tests_mavsdk_server.vcxproj]
telemetry_service_impl_test.obj : error LNK2019: unresolved external symbol "const mavsdk::rpc::telemetry::Imu::`vftable'" (??_7Imu@telemetry@rpc@mavsdk@@6B@) referenced in functio
n "public: __cdecl mavsdk::rpc::telemetry::Imu::Imu(void)" (??0Imu@telemetry@rpc@mavsdk@@QEAA@XZ) [C:\Users\Julian Oes\src\MAVSDK\build\default\src\mavsdk_server\test\unit_tests_ma

JonasVautherin
JonasVautherin previously approved these changes Jun 1, 2023
Copy link
Collaborator

@JonasVautherin JonasVautherin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I think it was made like this such that it would trigger the build when the proto files changed or something like this. But then we moved the generation out of CMake and versioned the generated files.

So this is simpler indeed!

@JonasVautherin
Copy link
Collaborator

Not completely sure what the latest changes imply. Do they solve the Windows problem?

@julianoes
Copy link
Collaborator Author

Nope, nothing solves the Windows problem 😞

@julianoes julianoes force-pushed the pr-simplify-linking branch 3 times, most recently from d0c74e0 to ef5e3cf Compare June 11, 2023 03:48
@julianoes julianoes changed the title mavsdk_server: simplify linking Simplify linking, update dependencies Jun 11, 2023
@julianoes julianoes marked this pull request as ready for review June 11, 2023 05:36
CMakeLists.txt Outdated
@@ -1,4 +1,4 @@
cmake_minimum_required(VERSION 3.13)
cmake_minimum_required(VERSION 3.13...3.22)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm what's that? 🤔

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's just a max version that it has been tested with and policies set. I added it because there was a warning about it but it actually came from the zlib CMakeLists.txt where the minimum version was 2.4.

@@ -36,7 +36,7 @@ if(ANDROID)
PREFIX openssl
PATCH_COMMAND patch -p 0 < ${PROJECT_SOURCE_DIR}/dockcross-android.patch
CONFIGURE_COMMAND ${CMAKE_COMMAND} -E env ANDROID_NDK=$ENV{CROSS_ROOT} <SOURCE_DIR>/Configure --sysroot=$ENV{CROSS_ROOT}/sysroot ${OPENSSL_PLATFORM} --prefix=${CMAKE_INSTALL_PREFIX} ${OPENSSL_BUILD_TYPE} no-shared
BUILD_COMMAND make
BUILD_COMMAND make install_sw
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like an INSTALL_COMMAND?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, we could write ./configure && make && make install in the CONFIGURE_COMMAND, and CMake wouldn't know about it. But we would 😁.

It's nice if the log output says "starting install step" before running the installation step, I think 😇

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me try.

PREFIX zlib
PATCH_COMMAND git checkout . && git apply ${PROJECT_SOURCE_DIR}/build_shared_libs.patch
PATCH_COMMAND git checkout . && git apply ${PROJECT_SOURCE_DIR}/build_shared_libs_and_cmake_version.patch
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we split that in two PATCH_COMMAND lines? They seem unrelated 🤔

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I just thought putting it into one patch is easier.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, as you want. I see patches like commits. It's easier for me to read a commit that does one thing than to try to understand which line corresponds to what when reading the commit.

But if you like one patch better, technically it works 👍.

- DESTINATION ${gRPC_INSTALL_CMAKEDIR}
- NAMESPACE gRPC::
- )
+ if (gRPCPluginTargets AND gRPC_BUILD_CODEGEN)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a bug in gRPC upstream?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so and it has supposedly been fixed but either the fix didn't go into v1.55.1 or it doesn't fully fix it for us for some reason:
grpc/grpc#32885

PREFIX zlib
PATCH_COMMAND git checkout . && git apply ${PROJECT_SOURCE_DIR}/build_shared_libs.patch
PATCH_COMMAND git checkout . && git apply ${PROJECT_SOURCE_DIR}/build_shared_libs.patch && git apply ${PROJECT_SOURCE_DIR}/bump_cmake_min_version.patch
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note (but it does not matter much). You can add as many COMMAND as you want between the "key" commands. So this should work:

PATCH_COMMAND git checkout . && git apply ${PROJECT_SOURCE_DIR}/build_shared_libs.patch
COMMAND git apply ${PROJECT_SOURCE_DIR}/bump_cmake_min_version.patch

See the Miscellaneous Options in the docs:

Any of the other ..._COMMAND options can have additional commands appended to them by following them with as many COMMAND ... options as needed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yer, on the other hand I like having all of the 3 commands together wrapped as "the" patch command.

@julianoes
Copy link
Collaborator Author

I think I want to clean up the commits a bit before merging.

On Windows, the mavsdk_server unit tests don't link properly when
building with shared libs. For whatever reason the transitive
dependencies didn't get tracked from the protobu/grpc components all the
way into the mavsdk_server unit tests.

I suspect that either some symbols get stripped too early, or it has to
do with the export visibility in MSVC.

I couldn't quite get to the bottom of it but I realized that we could
just simplify it by avoiding the little libraries and just adding all
sources to the mavsdk_server library in the first place.

Let's see if it works...

Signed-off-by: Julian Oes <[email protected]>
This updates protobuf, gRPC, absl, and c-ares, and then regenerates the
code using that.

Signed-off-by: Julian Oes <[email protected]>
We now should have XCode 14.2 by default which should no longer require
bitcode.

Signed-off-by: Julian Oes <[email protected]>
@julianoes julianoes mentioned this pull request Dec 27, 2023
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants