Skip to content
This repository has been archived by the owner on Nov 1, 2020. It is now read-only.

Generate version script automatically unix #7929

Merged
merged 8 commits into from
Jan 1, 2020
Merged

Generate version script automatically unix #7929

merged 8 commits into from
Jan 1, 2020

Conversation

josephmoresena
Copy link
Contributor

As we discussed in #s, these changes will allow at shared libraries compilation on unix-like platforms to automatically generate a version-script and remove all local symbols and unused code.
As is said on #7340 (comment) the .so file contains all debug information so we need still remove all this information from file with strip in order to reduce significatly the size of .so file.

@jkotas
Copy link
Member

jkotas commented Dec 29, 2019

For reference, could you please share some numbers about the impact of this change on the final binary sizes?

<CustomLinkerArg Include="-Wl,--version-script=$(ExportsFile)" />
<CustomLinkerArg Include="-Wl,--discard-all" />
<CustomLinkerArg Include="-Wl,--gc-sections" />
<CustomLinkerArg Include="-Wunused-command-line-argument" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need -Wunused-command-line-argument ? A comment may be nice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake, I thought to long term use all these commands with --exported-symbols. The new version is incompatible with that.

@@ -59,7 +59,8 @@ See the LICENSE file in the project root for more information.
<NativeBinaryExt Condition="'$(NativeCodeGen)' == 'wasm'">.html</NativeBinaryExt>

<ExportsFileExt Condition="'$(OutputType)' != 'Exe' and '$(TargetOS)' == 'Windows_NT' and '$(NativeLib)' == 'Shared'">.def</ExportsFileExt>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The conditions on ExportsFileExt can be simplified:

 <ExportsFileExt Condition="'$(TargetOS)' == 'Windows_NT'">.def</ExportsFileExt>
 <ExportsFileExt Condition="'$(TargetOS)' == 'OSX'">.exports</ExportsFileExt>
...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ExportsFileExt should only be generated when final binary is a shared library, shouldn't?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, the exports file should be generated only when final binary is shared library.

But it is simpler to set ExportsFileExt property all the time, even when the exports file is not actually generated. It does not hurt for it to be set even when the value is not actually used.

@@ -59,7 +59,8 @@ See the LICENSE file in the project root for more information.
<NativeBinaryExt Condition="'$(NativeCodeGen)' == 'wasm'">.html</NativeBinaryExt>

<ExportsFileExt Condition="'$(OutputType)' != 'Exe' and '$(TargetOS)' == 'Windows_NT' and '$(NativeLib)' == 'Shared'">.def</ExportsFileExt>
<ExportsFileExt Condition="'$(OutputType)' != 'Exe' and '$(TargetOS)' != 'Windows_NT' and '$(NativeLib)' == 'Shared'">.exports</ExportsFileExt>
<ExportsFileExt Condition="'$(OutputType)' != 'Exe' and '$(TargetOS)' == 'OSX' and '$(NativeLib)' == 'Shared'">.exports</ExportsFileExt>
<ExportsFileExt Condition="'$(OutputType)' != 'Exe' and '$(TargetOS)' != 'Windows_NT' and '$(TargetOS)' != 'OSX' and '$(NativeLib)' == 'Shared'">.map</ExportsFileExt>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I am not sure how standard is the .map suffix for version scripts. I have done a quick search https://github.com/search?q=--version-script&type=Code and it is all over the place.

It may be ok to use .exports for them to keep things simpler. It is what the CoreCLR build does.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.map is used at https://www.gnu.org/software/gnulib/manual/html_node/LD-Version-Scripts.html
I think that there is no problem with use .exports, really it's just a flat file.

@josephmoresena
Copy link
Contributor Author

For reference, could you please share some numbers about the impact of this change on the final binary sizes?

22.396.480 Full.so (Untouched)
16.789.048 Only1.so (--discard-all)
22.394.632 Only2.so (--gc-sections)
16.787.752 Reduced.so (--discard-all --gc-sections)
22.375.048 VersionedFull.so (--version-script)
16.767.616 VersionedOnly1.so (--version-script --discard-all)
22.369.056 VersionedOnly2.so (--version-script --gc-sections)
16.762.176 VersionedReduced.so (--version-script --discard-all --gc-sections)

@jkotas
Copy link
Member

jkotas commented Dec 31, 2019

16.762.176 VersionedReduced.so (--version-script --discard-all --gc-sections)

Nice!

Does the --discard-all and --gc-sections combination have similar effect on size of the executable binaries? If yes, it should be set unconditionally on Linux, for both shared libraries and executables.

@josephmoresena
Copy link
Contributor Author

josephmoresena commented Dec 31, 2019

16.762.176 VersionedReduced.so (--version-script --discard-all --gc-sections)

Nice!

Does the --discard-all and --gc-sections combination have similar effect on size of the executable binaries? If yes, it should be set unconditionally on Linux, for both shared libraries and executables.

Yes, I'm agree with you. The effect it's usesfull with RyuJITCodeGen because debug information is smaller than it's CppCodeGen version.
Here is a table with the tests sizes:

Test Full No CPP --discard-all No CPP --gc-sections No Cpp --discard-all --gc-sections No CPP
BasicThreading 22.864.512   17.177.600   22.863.576   17.177.208  
Delegates 71.692.864 21.379.296 67.208.064 15.720.160 71.691.872 21.378.344 67.207.632 15.719.760
DynamicGenerics 29.319.608   20.203.392   29.314.568   20.198.928  
Exceptions 21.373.864   15.716.240   21.372.904   15.715.848  
Generics 73.855.656 21.906.952 69.155.048 16.013.048 73.854.664 21.905.992 69.154.616 16.012.680
Hello 71.495.640 21.321.600 67.023.632 15.681.568 71.490.552 21.320.648 67.019.104 15.681.128
Interfaces 72.038.512 21.479.376 67.518.424 15.776.904 72.037.520 21.478.408 67.517.992 15.776.488
PInvoke 71.760.208 21.669.208 67.281.368 15.915.912 71.759.216 21.668.288 67.280.936 15.915.520
Reflection 72.308.400 21.678.608 67.775.400 15.917.696 72.307.408 21.677.688 67.774.968 15.917.280
StaticLibrary.a 28.140.306   28.140.306   28.140.306   28.140.314  
StaticLibraryTest 21.247.504   21.247.504   21.247.504   21.247.512  
Threading 21.727.888   15.934.192   21.726.952   15.933.808  

It should be clarified that when using strip the resulting file size is the same in all cases by test.

jgmoreno and others added 2 commits December 31, 2019 17:36
* Simplification of ExportsFileExt condition.
* Same extension for version-script (Linux) and Exported-Symbols-List (OSX).
* Separation of version-script fixes and size file reduction changes.
* Remove useless -Wunused-command-line-argument.
Copy link
Member

@jkotas jkotas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Thank you!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants