[Questions] Several questions about source generators #60256

QuantumDeveloper · 2022-03-18T15:43:26Z

Sorry, but I didnt find any other suitable place for asking this questions.
I have several question, which I cannot find answer for this time:

Is there a way to subscribe to AdditionalFiles changes? I mean can I detect that text in any of additional file was changed and automatically invoke generator to update sources?
Is there a way to access AdditionalFiles from IncrementSourceGenerator? I didnt find such property there.
Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

The text was updated successfully, but these errors were encountered:

Eli-Black-Work · 2022-03-25T04:11:33Z

I suspect that you may have already found the answers to these questions, but in case not:

If your generator implements ISourceGenerator, I believe that ISourceGenerator::Execute() will be called every time the contents of one of the files in AdditionalFiles changes.
Access AdditionalFiles from IIncrementalGenerator like so:

public void Initialize(IncrementalGeneratorInitializationContext initializationContent)
{
   // Some of this code is from https://github.com/dotnet/roslyn/blob/main/docs/features/incremental-generators.md
    var files = initializationContent.AdditionalTextsProvider.Where(/*static*/ file => file.Path.EndsWith(".json"));

    var namesAndContents = files.Select((file, cancellationToken) => (Name: Path.GetFileNameWithoutExtension(file.Path), Content: file.GetText(cancellationToken).ToString(), Path: file.Path));

    initializationContent.RegisterSourceOutput(namesAndContents, AddSource);
}

private void AddSource(SourceProductionContext context, (string Name, string Content, string Path) file)
{
    string fileName = $"{file.Name}.g.cs";

    try
    {
        if (file.Content == null)
            throw new Exception("Failed to read file \"" + file.Path + "\"");

        string sourceCode = .....

        context.AddSource(fileName, sourceCode);
    }
    catch (Exception e)
    {
       string errorMessage = $"Error: {e.Message}\n\nStrack trace: {e.StackTrace}";

       context.AddSource(fileName, sourceCode);
    }
}

Let me know if you still need answer to this question 🙂

QuantumDeveloper · 2022-03-25T08:03:53Z

@Bosch-Eli-Black Thanks for your answers.
I suspect, that answer for my 3rd question is 'Yes', but just to be sure would like to know your answer :)

And one more question appears when I start working with incremental generator:

Does updates in Additional files will be immediately reflected in AdditionalTextsProvider or I need to do some extra job for this? I mean if I will add/remove some files or update files content.

What I want to achieve is when content one of additional files is changed or if file was added, I want my generator to run and update sources correspondinly. Currently I can update source only after the whole build, which is not what I want, unfortunately

Eli-Black-Work · 2022-03-29T07:19:52Z

@QuantumDeveloper No problem! 🙂

We're actually trying to accomplish the exact same thing as you, I think 🙂 We have some .json files that we generate C# from, and we'd like for that to happen whenever one of the .json files changes.

In our project, we specify AdditionalFiles like so:

<AdditionalFiles Include="$(ProjectDir)examplesubfolder\*.json" />

If I change one of the examplesubfolder\*.json files, the source generator automatically runs and generates new C# code. I don't need to rebuild the project.
Similarly, if I put a new .json file in examplesubfolder\, the source generator will automatically run and generate C# code for the new file. I don't need to rebuild the project. I do need to wait about 10 seconds while all of the analyzers and the generator in my project run.
If I delete a .json file in examplesubfolder\, I have to restart Visual Studio before the generated C# code will go away.

Note that we're using ISourceGenerator instead of incremental generators, so this may be different for your project 🙂

In case it helps, here's what our project files look like:

MyProject.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">

	<PropertyGroup>
		<TargetFramework>net6.0</TargetFramework>
	</PropertyGroup>

	<ItemGroup>
		<ProjectReference Include="..\MyGenerator\MyGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" SetTargetFramework="TargetFramework=netstandard2.0" />
	</ItemGroup>

	<ItemGroup>
		<AdditionalFiles Include="$(ProjectDir)examplesubfolder\*.json" />
	</ItemGroup>

	<PropertyGroup>
		<Example_Variable>in_case_you_need_to_pass_variables_to_your_generator</Example_Variable>
	</PropertyGroup>
	<ItemGroup>
		<CompilerVisibleProperty Include="Example_Variable" />
	</ItemGroup>
</Project>

MyGenerator.csproj

<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
		<TargetFramework>netstandard2.0</TargetFramework>
		<LangVersion>8.0</LangVersion>
		<!-- Setting `IsRoslynComponent` to `true` allows us to debug the source generator.
		     See https://dominikjeske.github.io/source-generators/#comment-5804484397 for debugging instructions. -->
		<IsRoslynComponent>true</IsRoslynComponent>
	</PropertyGroup>
</Project>

Eli-Black-Work · 2022-03-29T08:31:02Z

I've filed an issue for the bug where deleting an input file doesn't cause the analyzer to be run again: #60455

Eli-Black-Work · 2022-03-29T08:36:54Z

Ah, I forgot to reply to your original question! 🙂

Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

QuantumDeveloper · 2022-03-29T08:47:42Z

Ah, I forgot to reply to your original question! 🙂

Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

No, I mean that I have ProjectA that references ProjectB. I attach Generator only to ProjectA and generate sources for ProjectB and ProjectA as ProjectB is referenced from ProjectA

QuantumDeveloper · 2022-03-29T08:54:18Z

@QuantumDeveloper No problem! 🙂

We're actually trying to accomplish the exact same thing as you, I think 🙂 We have some .json files that we generate C# from, and we'd like for that to happen whenever one of the .json files changes.

In our project, we specify AdditionalFiles like so:
<AdditionalFiles Include="$(ProjectDir)examplesubfolder\*.json" />
If I change one of the examplesubfolder\*.json files, the source generator automatically runs and generates new C# code. I don't need to rebuild the project.

Similarly, if I put a new .json file in examplesubfolder\, the source generator will automatically run and generate C# code for the new file. I don't need to rebuild the project. I do need to wait about 10 seconds while all of the analyzers and the generator in my project run.

If I delete a .json file in examplesubfolder\, I have to restart Visual Studio before the generated C# code will go away.

Note that we're using ISourceGenerator instead of incremental generators, so this may be different for your project 🙂

In case it helps, here's what our project files look like:

MyProject.csproj
<Project Sdk="Microsoft.NET.Sdk.Web">

	<PropertyGroup>
		<TargetFramework>net6.0</TargetFramework>
	</PropertyGroup>

	<ItemGroup>
		<ProjectReference Include="..\MyGenerator\MyGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" SetTargetFramework="TargetFramework=netstandard2.0" />
	</ItemGroup>

	<ItemGroup>
		<AdditionalFiles Include="$(ProjectDir)examplesubfolder\*.json" />
	</ItemGroup>

	<PropertyGroup>
		<Example_Variable>in_case_you_need_to_pass_variables_to_your_generator</Example_Variable>
	</PropertyGroup>
	<ItemGroup>
		<CompilerVisibleProperty Include="Example_Variable" />
	</ItemGroup>
</Project>
MyGenerator.csproj
<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
		<TargetFramework>netstandard2.0</TargetFramework>
		<LangVersion>8.0</LangVersion>
		
		<IsRoslynComponent>true</IsRoslynComponent>
	</PropertyGroup>
</Project>

I tried to replicate described behavior in my demo project. I even use SourceGenerator instead Incremental and it still does not work as I expect. It creates sources only if I rebuild main project and not when I change Additional files. I will attach my small project. Maybe you can find want I am doing wrong here.
CodeGenerationDemo.zip

Eli-Black-Work · 2022-04-11T09:10:28Z

@QuantumDeveloper Sorry, forgot to reply!

I downloaded your project and gave it a try, and it seems to work on my computer 🙂

Here's what I did:

Opened MainWindow.xml, changed the contents from <Window59></Window59> to <Window59>asdfasdf</Window59>, and saved the file. The auto-generated code immediately changed to be this:

//Auto-generated code
namespace CodeGenerationDemo.Window;
public partial class MainWindow
{
public string text = "<Window59></Window59>";
}

Copied and pasted the file MainWindow.xml, so that I got a file named MainWindow - Copy.xml. I immediately saw a new file named MainWindow - Copy.g.cs be created by the generator.

Is that the same as what you're seeing? If not, I'm running Visual Studio 17.2.0 Preview 2.1; maybe something got fixed in the newer preview versions 🙂

Eli-Black-Work · 2022-04-11T09:12:06Z

Ah, I forgot to reply to your original question! 🙂

Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

No, I mean that I have ProjectA that references ProjectB. I attach Generator only to ProjectA and generate sources for ProjectB and ProjectA as ProjectB is referenced from ProjectA

If ProjectA references ProjectB, then I believe you'll want to generate your sources in ProjectB. That way, they're visible to both ProjectA and ProjectB 🙂

QuantumDeveloper · 2022-04-11T13:00:50Z

Ah, I forgot to reply to your original question! 🙂

Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

No, I mean that I have ProjectA that references ProjectB. I attach Generator only to ProjectA and generate sources for ProjectB and ProjectA as ProjectB is referenced from ProjectA

If ProjectA references ProjectB, then I believe you'll want to generate your sources in ProjectB. That way, they're visible to both ProjectA and ProjectB 🙂

Ok, I was talking just about that :)

QuantumDeveloper · 2022-04-11T13:02:42Z

@QuantumDeveloper Sorry, forgot to reply!

I downloaded your project and gave it a try, and it seems to work on my computer 🙂

Here's what I did:

Opened MainWindow.xml, changed the contents from <Window59></Window59> to <Window59>asdfasdf</Window59>, and saved the file. The auto-generated code immediately changed to be this:
//Auto-generated code
namespace CodeGenerationDemo.Window;
public partial class MainWindow
{
public string text = "<Window59></Window59>";
}
Copied and pasted the file MainWindow.xml, so that I got a file named MainWindow - Copy.xml. I immediately saw a new file named MainWindow - Copy.g.cs be created by the generator.

Is that the same as what you're seeing? If not, I'm running Visual Studio 17.2.0 Preview 2.1; maybe something got fixed in the newer preview versions 🙂

No, thats the issue. You must see <Window59>asdfasdf</Window59> instead of <Window59></Window59>. And I am sure you will see it after rebuild, but not in runtime unfortunately. This issue both in Visual Studio and in Rider.

Eli-Black-Work · 2022-04-12T10:17:37Z

@QuantumDeveloper Whoops, sorry, I mistyped!

Here's what I did:

Opened MainWindow.xml in Visual Studio.
Changed the contents to <Window59>testing</Window59>
MainWindow.g.cs immediately changed to be this:

//Auto-generated code
namespace CodeGenerationDemo.Window;
public partial class MainWindow
{
public string text = "<Window59>testing</Window59>";
}

I didn't need to rebuild; as soon as I changed the .xml file, it updated the .g.cs file 🙂

Which version of Visual Studio are you using?

QuantumDeveloper · 2022-04-12T12:39:32Z

@Bosch-Eli-Black I am using version 17.1.3

Eli-Black-Work · 2022-04-13T08:49:17Z

@QuantumDeveloper If you have a chance, you could try with 17.2.0 Preview 3.0, which is what I'm using 🙂

QuantumDeveloper · 2022-04-13T18:28:11Z

@Bosch-Eli-Black I installed latest preview and I dont see much difference. Still no auto generation after I change file content and I need to build project each time I want to generate updated sources.
First screen show no update when I changed file:

Second scrren is after I press "build" option:

I have no idea why your version of VS is working fine and mine is not :(

Eli-Black-Work · 2022-05-09T07:51:59Z

@QuantumDeveloper Sorry for the late reply! Just got back from vacation 🙂

Here's a GIF of what I'm seeing:

Is the behaviour that you're seeing different, or are we doing different things?

Eli-Black-Work · 2022-05-09T07:53:23Z

P.S.

I used Screen to Gif for this 🙂

QuantumDeveloper · 2022-05-19T07:55:52Z

@Bosch-Eli-Black Sorry for long delay. When you showed me where to find generated code, I realized that my generator also regenerates code just after I change my xml, but it does not update cs file on disk until I build assembly.
This is good news.

Bad news is that seems generators have issues with referencing another projects.
In my real project I have local reference inside my generator via <ProjectReference>. I found that I cannot simply reference it, because in that case generator does not generate anything.
After some googling, I found https://stackoverflow.com/questions/69764185/c-sharp-source-generator-not-including-results-from-project-reference and when I add OutputItemType="Analyzer" to my refenece project, It starts working, BUT after some testing found out that at some random point it just stopps working.
After 10-20 builds it just stop emitting output. but at the same time it continue updating cs file in runtime (via Solution Explorer).

Then I decided to attach all code from the refeneced project to generator and seems that it works perfectly. At least I didnt faced with described issue.
But I would like to make it work with <ProjectReference> because it`s quite stupid to copy all code from other project to make generator work as expected.

Do you know how to workaround such issue?

Eli-Black-Work · 2022-05-19T08:35:01Z

@QuantumDeveloper Np! 🙂

Does this help with the dependency issue: #47517 (comment) ?

QuantumDeveloper · 2022-05-19T08:40:00Z

@Bosch-Eli-Black didnt try that. I am wondering does local project can be referenced via <PackageReference> instead of <ProjectReference>?
<PackageReference> its about nugets, not local projects

Eli-Black-Work · 2022-05-19T08:44:44Z

@QuantumDeveloper I would read through the replies to Sharwell's comment; there's some good information there about how to link to NuGet packages from an analyzer, how to link to transient NuGet packages from an analyzer, and how to link to other projects from within an analyzer.

I think that specifically what you're looking for is this comment: #47517 (reply in thread) 🙂

QuantumDeveloper · 2022-05-19T16:34:56Z

@Bosch-Eli-Black After whole day of testing seems one of comments helped. My generator now working quite stable.
I am talking about this comment: #47517 (reply in thread)

But anyway I am still thinking that such tricky and not obvious way of linking local project to generator is a bad design and should be improved and simplified.

Eli-Black-Work · 2022-05-20T03:10:48Z

@QuantumDeveloper Ya, I've been wishing for Microsoft to simplify this, too 🙂

QuantumDeveloper · 2022-06-24T16:18:08Z

@Bosch-Eli-Black Hi.
Working recently on new generator I found that it does not want to work with multitarget assemblies. If I remove multitargeting and leave only netstandard2.0, it starts working correctly, but I need multitargeting for my assemblies.

Is there a way to make generator work with such assemblies?
Did you faced with such issue?

Eli-Black-Work · 2022-06-28T09:25:13Z

@QuantumDeveloper I'm afraid I don't have any experience there 🙂 I'd recommend filing a new bug.

ThomasHeijtink · 2024-10-31T12:55:59Z

@Eli-Black-Work

Ah, I forgot to reply to your original question! 🙂

Is it true that if I have several projects, which have references from top to bottom (in hierarchy level), I cannot use only generator, which is placed inside main project and should add my generator to each of referenced projects instead?

Do you mean that you have ProjectA that depends on ProjectB, and you want both projects to use MyGenerator?

No, I mean that I have ProjectA that references ProjectB. I attach Generator only to ProjectA and generate sources for ProjectB and ProjectA as ProjectB is referenced from ProjectA

If ProjectA references ProjectB, then I believe you'll want to generate your sources in ProjectB. That way, they're visible to both ProjectA and ProjectB 🙂

Sorry to revive this old question. But since this issue is still open, I just took the liberty. What if I want to generate code in Project A based on code in Project B or any project referenced by Project A how can I generate code based on that? In my first trials it doesn't seem to be able to find the code in the referenced projects. While, if I copy the code to Project A it works fine. But this is not ideal, as the code actually belongs in Project B. I also don't want to add the source generator to Project B as this would generate the code there while I want the generated code to be in Project A.

CyrusNajmabadi · 2024-10-31T15:55:31Z

You will have to base it purely on the types and members exported from that library. You will not be able to see the code in it.

This is how compilation works. Project B is compiled into a dll, and the dll is referenced from Project A. So you won't see the code in it @ThomasHeijtink

ThomasHeijtink · 2024-10-31T16:03:38Z

@CyrusNajmabadi
Thanks for clarifying this. Is there currently a best practice to solve this? I can imagine I will have to build two generators and somehow pass one analysis done in 'Project B' to the generator in 'Project A'.

Also, I can't imagine I'm the only one with this challenge. Are there talks in the community or issues referring to a feature request to have better support for this scenario?

CyrusNajmabadi · 2024-10-31T16:07:42Z

@CyrusNajmabadi
Thanks for clarifying this. Is there currently a best practice to solve this?

Yes. Include whatever data the generator needs to generate from into all projects that need it.

I can imagine I will have to build two generators and somehow pass one analysis done in 'Project B' to the generator in 'Project A'.

You cannot pass any data between generators. Any attempt to do so is considered undefined behavior. We are also likely to break any attempts to do that as we work on sandboxing extensions.

Also, I can't imagine I'm the only one with this challenge. Are there talks in the community or issues referring to a feature request to have better support for this scenario?

There are no talks on this. The advice above is the way to do it.

Alternatively, since the generator doesn't need the data from the project it is in, but data from another project, don't use a Roslyn generator. Just write your own tool that runs before that project and emits what is necessary. Roslyn generators are specifically about solving the problem where the generator needs to read and change the specific project it is compiling against.

maryamariyan added the Area-Compilers label Mar 18, 2022

Eli-Black-Work mentioned this issue Jun 16, 2022

VS doesn't support MSBuild <CopyLocalLockFileAssemblies> property #61940

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Questions] Several questions about source generators #60256

[Questions] Several questions about source generators #60256

QuantumDeveloper commented Mar 18, 2022

Eli-Black-Work commented Mar 25, 2022 •

edited

Loading

QuantumDeveloper commented Mar 25, 2022 •

edited

Loading

Eli-Black-Work commented Mar 29, 2022

Eli-Black-Work commented Mar 29, 2022 •

edited

Loading

Eli-Black-Work commented Mar 29, 2022

QuantumDeveloper commented Mar 29, 2022

QuantumDeveloper commented Mar 29, 2022

Eli-Black-Work commented Apr 11, 2022

Eli-Black-Work commented Apr 11, 2022

QuantumDeveloper commented Apr 11, 2022

QuantumDeveloper commented Apr 11, 2022

Eli-Black-Work commented Apr 12, 2022

QuantumDeveloper commented Apr 12, 2022

Eli-Black-Work commented Apr 13, 2022

QuantumDeveloper commented Apr 13, 2022

Eli-Black-Work commented May 9, 2022

Eli-Black-Work commented May 9, 2022

QuantumDeveloper commented May 19, 2022

Eli-Black-Work commented May 19, 2022 •

edited

Loading

QuantumDeveloper commented May 19, 2022

Eli-Black-Work commented May 19, 2022 •

edited

Loading

QuantumDeveloper commented May 19, 2022

Eli-Black-Work commented May 20, 2022

QuantumDeveloper commented Jun 24, 2022

Eli-Black-Work commented Jun 28, 2022

ThomasHeijtink commented Oct 31, 2024

CyrusNajmabadi commented Oct 31, 2024

ThomasHeijtink commented Oct 31, 2024

CyrusNajmabadi commented Oct 31, 2024

[Questions] Several questions about source generators #60256

[Questions] Several questions about source generators #60256

Comments

QuantumDeveloper commented Mar 18, 2022

Eli-Black-Work commented Mar 25, 2022 • edited Loading

QuantumDeveloper commented Mar 25, 2022 • edited Loading

Eli-Black-Work commented Mar 29, 2022

Eli-Black-Work commented Mar 29, 2022 • edited Loading

Eli-Black-Work commented Mar 29, 2022

QuantumDeveloper commented Mar 29, 2022

QuantumDeveloper commented Mar 29, 2022

Eli-Black-Work commented Apr 11, 2022

Eli-Black-Work commented Apr 11, 2022

QuantumDeveloper commented Apr 11, 2022

QuantumDeveloper commented Apr 11, 2022

Eli-Black-Work commented Apr 12, 2022

QuantumDeveloper commented Apr 12, 2022

Eli-Black-Work commented Apr 13, 2022

QuantumDeveloper commented Apr 13, 2022

Eli-Black-Work commented May 9, 2022

Eli-Black-Work commented May 9, 2022

QuantumDeveloper commented May 19, 2022

Eli-Black-Work commented May 19, 2022 • edited Loading

QuantumDeveloper commented May 19, 2022

Eli-Black-Work commented May 19, 2022 • edited Loading

QuantumDeveloper commented May 19, 2022

Eli-Black-Work commented May 20, 2022

QuantumDeveloper commented Jun 24, 2022

Eli-Black-Work commented Jun 28, 2022

ThomasHeijtink commented Oct 31, 2024

CyrusNajmabadi commented Oct 31, 2024

ThomasHeijtink commented Oct 31, 2024

CyrusNajmabadi commented Oct 31, 2024

Eli-Black-Work commented Mar 25, 2022 •

edited

Loading

QuantumDeveloper commented Mar 25, 2022 •

edited

Loading

Eli-Black-Work commented Mar 29, 2022 •

edited

Loading

Eli-Black-Work commented May 19, 2022 •

edited

Loading

Eli-Black-Work commented May 19, 2022 •

edited

Loading