Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shrink the size of the service core runtime nuget #15295

Merged
merged 10 commits into from
Jul 1, 2024

Conversation

mjkkirschner
Copy link
Member

@mjkkirschner mjkkirschner commented Jun 7, 2024

Purpose

This my first attempt to shrink the size of the service core runtime nuget. I plan on sending a few more PRs, each removing functionality that might be less obvious / more contentious or require larger changes.

This is done by:

  • removing some projects from the DynamoCore.sln
  • introducing a constant #MINIMAL_SERVICE to control compiling some code that references the removed projects.I think long term it would be good to remove this constant and instead split the assemblies and use interfaces or some other decoupling method. For example, it would be nice to move all the lucene code to another assembly and reference it with an interface in DynamoServices, like ISearchBackend - so we can totally avoid needing to ship all the lucene dependencies where we are never going to use them.
  • excluding some files from the nuspec that produces the nupkg.
  • I also used IsServiceMode to control a few places that took a bit of time in profiling - compared to ASM loading they are mostly inconsequential though they do avoid some file system access.
  • I also aligned isHeadless and CLIMode since lucene indexing was still happening in the CLIs. (still TODO)

FYI - it would be great if we could find some time to align all these wacky overlapping config settings...

  • IsHeadless
  • IsServiceMode
  • CLIMode
  • TestMode

Any others?

In the end the nupkg goes 275 megs to 104 megs.
Starting the CLI in service mode is now 400 ms faster (measured with diagnostics.stopwatch).

dont attempt to write with lucene
dont copy docs for openxml
dont copy samples,licenses,or node docs
Copy link

github-actions bot commented Jun 10, 2024

UI Smoke Tests

Test: success. 3 passed, 0 failed.
TestComplete Test Result
Workflow Run: UI Smoke Tests
Check: UI Smoke Tests - net8.0

@@ -31,7 +31,7 @@
<Isolated>False</Isolated>
<EmbedInteropTypes>True</EmbedInteropTypes>
</COMReference>
<PackageReference Include="DocumentFormat.OpenXml" Version="2.12.3" CopyXML="true" />
<PackageReference Include="DocumentFormat.OpenXml" Version="2.12.3" CopyXML="false" />
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this change affect?

Copy link
Member Author

@mjkkirschner mjkkirschner Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think it affects dynamo users at all- but it keeps a 12 megabyte xml file out of the bin folder.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And what XML file is this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this not needed for Dynamo.All anymore ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so, but I will double check.

@mjkkirschner mjkkirschner removed the WIP label Jun 11, 2024
<file src="**" target="lib\$TargetFramework$\" exclude="en-US\fallback_docs\*"/>
<file src="**"
target="lib\$TargetFramework$\"
exclude="en-US\fallback_docs\*;NodeHelpSharedDocs\*;Open Source Licenses\*;samples\** "/>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aparajit-pratap @pinzart90 do you think it makes sense to invert this and manually include every file we want to deploy so that changes to Dynamo bin folder don't automatically make it to the service?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so this right now will deploy all of the dll's in the bin folder into the service, even those that are not part of the service runtime core?

Copy link
Member Author

@mjkkirschner mjkkirschner Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have been more clear.
Right now, the way the job is setup for this nuget and build - it will deploy just the DynamoCore.Sln - I was just thinking of someone in the future added something to that .sln it will then make into the nuget package automatically. (it's a separate build job from our normal ones)

My worry is that it's not clear what else this DynamoCore.sln might be used for in the future besides the service.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying. I think this is good enough for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is easier to control what dlls and content in general within the csproj (more tools at our disposal)
I think we should have a solution (or configs in general) that produce on disk exactly what the service is. It will be valuable to build exactly what will be inside the docker container.

Maybe we should rename the sln to DynamoService.sln (if that is makes it clearer to anyone)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nugget config might help with filtering out test assemblies.

Or maybe that can be done earlier...(dynamo.All needs that too)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have a solution (or configs in general) that produce on disk exactly what the service is. It will be valuable to build exactly what will be inside the docker container. I agree. I will think about if we should change the name, add a config, or add a new .sln.

Copy link
Contributor

@pinzart90 pinzart90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment

@pinzart90
Copy link
Contributor

I am fine with the changes in this PR for now if you want to get another PR for other stuff

@mjkkirschner
Copy link
Member Author

I am fine with the changes in this PR for now if you want to get another PR for other stuff

unfortunately there appear to be a few test failures I need to look at first.

@pinzart90
Copy link
Contributor

Merging for now. We can follow up on the remaining issues

@pinzart90 pinzart90 merged commit eb750cc into DynamoDS:master Jul 1, 2024
22 of 25 checks passed
@QilongTang QilongTang added this to the 3.3 milestone Jul 1, 2024
@mjkkirschner mjkkirschner deleted the shrinkservice branch October 3, 2024 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants