Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Productize LibraryImportGenerator for use outside of runtime #60595

Closed
24 of 25 tasks
AaronRobinsonMSFT opened this issue Oct 18, 2021 · 17 comments
Closed
24 of 25 tasks

Productize LibraryImportGenerator for use outside of runtime #60595

AaronRobinsonMSFT opened this issue Oct 18, 2021 · 17 comments

Comments

@AaronRobinsonMSFT
Copy link
Member

AaronRobinsonMSFT commented Oct 18, 2021

Issues to address/consider after ship-ready work (#60212):

APIs:

Reconciliation of blittable/unmanaged:

Custom marshallers:

Release:

Fixer:

A code fixer should be provided to help with the conversion to using generated p/invokes via the new attribute. We may not support all options based on added complexity versus usage.

  • Basic conversion to new attribute where all options specified on DllImport are supported by source generator
  • PreserveSig: change managed signature to return int and have an out param if necessary, add custom marshaller for return value to throw if necessary, update callers,
  • ExactSpelling: different fix option to append suffix on EntryPoint property
  • Add stubbed out implementation for custom marshallers

Issues:

@kant2002
Copy link
Contributor

Does attempt to add DllImportGenerator to WinForms require all outstanding tasks here to be completed?

@AaronRobinsonMSFT
Copy link
Member Author

AaronRobinsonMSFT commented Feb 18, 2022

Does attempt to add DllImportGenerator to WinForms require all outstanding tasks here to be completed?

Not necessarily but we reserve the right to change any of the generated code or APIs that we are using. Additionally, we haven't put the source generator on a feed that would make consumption simple. From the Interop team's perspective, all products not in the dotnet/runtime repo are the same.

@kant2002
Copy link
Contributor

There some version on the feed 7.0.0-preview.2.22110.4 (in addiional to experimental feed), but that package blocked by internal GeneratedDllImportAttribute.

I do understand that WinForms is external product for you. I would like to throw some boulder on WinForms by applying BuiltInComSupprted=false and DisableBuiltInMarshalling=true. I expect this to be problematic and prefer not to build custom runtime for that experiment. Is this somehow possible?

If not, can you tell me what's blocking the team from going preview mode?

@AaronRobinsonMSFT
Copy link
Member Author

There some version on the feed 7.0.0-preview.2.22110.4

I didn't realize that. Perhaps something can be done here.

If not, can you tell me what's blocking the team from going preview mode?

The API reviews Reconciliation of blittable/unmanaged, and Release sections above are all mostly required. We then need to update the tooling with the new name and provide APIs that support the support for the generated, right now everything is internal since the API surface area and design are all in flux.

I should also note that the current plan is to productize this for .NET 7, but that is not something we're committed to. The only commitment is to ship the entire dotnet/runtime repo with generated source for all DllImports. Based on other factors this could slip and become productized in the next release. I don't think that is going to happen, but it is possible. The general point is there are a lot of moving parts to productizing this source generator because of expectations on how it works and support for assemblies containing the generated code. Even acknowledging that previews are not supported doesn't change much. We would like to ensure what we do release in a preview is something we can feasibly deliver so we don't need to remove it prior to an RC. The majority of the work here is around software support infrastructure and tenets that must be followed due to what the product is doing.

Other than hacking together a custom runtime, I'm not sure how easy this is going to be at present.

@kkukshtel
Copy link

Just as a question - is an intended use case of this to be something that can replace the ClangSharp P/Invoke generator? https://www.nuget.org/packages/ClangSharpPInvokeGenerator

@AraHaan
Copy link
Member

AraHaan commented Mar 7, 2022

Not at all, ChangSharp is meant for converting native header files into .NET PInvokes (like the Windows SDK files for TerraFX.Interop.Windows for example).

@DualBrain
Copy link

Is the source generator being built leveraging Roslyn SyntaxGenerators so that it will be consumable by C# and VB?

@AaronRobinsonMSFT
Copy link
Member Author

Is the source generator being built leveraging Roslyn SyntaxGenerators so that it will be consumable by C# and VB?

That is a potential future option, however it is not going to work for VB.Net presently. The reason is lack of support for function pointers and other low-level .NET features in VB.Net.

The reason it was chosen in our case was for nicely formatting the source and it is simply easier to express/debug the intent if we aligned with how Roslyn expresses the tree.

@elinor-fung elinor-fung changed the title Productize DllImportGenerator for use outside of runtime Productize LibraryImportGenerator for use outside of runtime Mar 24, 2022
@kant2002
Copy link
Contributor

I would like to comeback and ask again what's happening with this feature. LibraryImportGenerator is missing from feeds right now and there no easy way to consume source generator. From what I see it was disabled after discussion here #65495 (comment) but that still does not explain why it is needed.

Which is sad, since that limit ability to play with feature in the preview mode without expecting it to be shipped. There couple other enthusiasts from the Discord who probably willing to put their dirty hands into this feature.

Is there something which I can do to unblock packaging (because from my understanding it was some build failures)?

@AaronRobinsonMSFT
Copy link
Member Author

This is now consumable in preview 5. This source generator is in the ref pack for when targeting the preview 5 TFM - this is by design and that is the minimum TFM requirement.

See #68247.

@shaggygi
Copy link
Contributor

@AaronRobinsonMSFT Any plans to add docs/examples on scenarios and how to use? I'm sure there already is and I'm overlooking. Just trying to understand feature and some of the basics to get started. Thx

@kant2002
Copy link
Contributor

Thank you! This new made my day. So if I understand correctly LibraryImportGenerator is part of .NET SDK and would be shipped together with it. Correct?

@AaronRobinsonMSFT
Copy link
Member Author

Any plans to add docs/examples on scenarios and how to use? I'm sure there already is and I'm overlooking. Just trying to understand feature and some of the basics to get started. Thx

There are plans and as with any product there will be docs and samples once we get out of preview. The repo does contain a sample - see https://github.com/dotnet/runtime/tree/main/src/samples/LibraryImportGeneratorSample.

Low-level design docs and how it works can be found under https://github.com/dotnet/runtime/tree/main/docs/design/features and in https://github.com/dotnet/runtime/tree/main/docs/design/libraries/LibraryImportGenerator.

This is a non-trivial feature that is still technically in flux since we are in preview. Please be patient with the low-level nature of the documentation and how the source generator is used from an end-user perspective. Proper higher-level walkthroughs are forth coming but will likely be closer to the RC. For now, the team is focused on ensuring correctness for non-netcoreapp scenarios and comprehensive testing. Also, the work to make anything an enterprise quality product is extensive - see this issue and all the non-functional work items - which is why the walkthroughs and samples are not yet ready.

@shaggygi
Copy link
Contributor

@AaronRobinsonMSFT completely understand. thanks for the info and look forward to feature. keep up the good work 👍

@AraHaan
Copy link
Member

AraHaan commented Apr 27, 2022

This is now consumable in preview 5. This source generator is in the ref pack for when targeting the preview 5 TFM - this is by design and that is the minimum TFM requirement.

See #68247.

sad it cant be in a nuget package as well for those who want to use the generator downlevel (in .NET 6 projects).

@AaronRobinsonMSFT
Copy link
Member Author

@AraHaan Unfortunately the source generator has strict requirements on .NET 7 features. However, the Roslyn team's source generator API permits writing the source to disk, which could then be modified as needed to target a .NET 6 project. This would likely require substantial care use down level as there are runtime semantics that aren't known to the compiler. See #60639.

Enabling source being written to disk can be done by adding the following MSBuild property to the project.

<EmitCompilerGeneratedFiles>true</EmitCompilerGeneratedFiles>

@elinor-fung
Copy link
Member

Closing out this issue as the work to the LibraryImportGenerator exposed and consumable outside of the runtime repo is done. It is in the daily main builds and will be in .NET 7.0 Preview 5.

Please do file issues to let us know about problems you hit or questions you have when trying to consume this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants