-
Notifications
You must be signed in to change notification settings - Fork 206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Green Thread Experiment Results #2398
Comments
It would be super interesting to hear why and why the overhead is so significant. |
What kind of improvements of the existing (async/await) model do you have in mind? |
Sounds great, and I believe this to be the right decision based on the outcomes and results described. Maybe solving the problem some other way would be more efficient, by not making drastic changes to how the current coding model is and turning everything upside down. If there was a way to correctly, safely and performantly "await" async calls in sync code (if you could use |
I have seen numerous existing, LOB applications (being developed for over a longer period), that do still do many-man sync IO - that nobody can reasonably update to async/await as the whole stack needs to become async/await too. Typically, there are thousands of sync EF/EFCore DB requests implemented. Having a switch to enable green threads to get a significant perf gain sounds very appealing for these scenarios. |
What about other scenarios? I understand that for a web server the conclusion might be spot on but if you are coding UI apps (like WinForms) and heavily rely on consuming event handlers, green threads might be a better solution. Async/await is great for specific use cases but horrible for other use cases. Having an alternative (even when a bit slower) would be very welcome. |
@StefanKoell can you give an example where green threads are better for GUI apps? From my experience, event-like nature of async/await works pretty good in GUI apps. And current sync context based infrastructure also works well. The only possibly disadvantage is sync event handlers. But "async void" there behaves as expected, where any exception is not lost, but raised on the dispatcher (UI thread jobs queue). |
@maxkatz6 maybe I'm missing something but having async void methods (event handlers) will swallow exceptions and are hard to handle in general.
Not sure I understand that. Can you elaborate? Another really pain is to work on brown-field projects and try to implement new functionality using async/await. It's so painful because it's a rabbit hole where you have to rewrite every single funtion in the call chain to make it work. And sometimes, you just can't do it properly because of some 3rd party dependency and you are forced to use GetAwaiter().GetResult - which could cause deadlocks. The same is true for events which have CancelEventArgs. There's no elegant solution. You have to implement ManualResetEvents, so the code gets really complicated. If I need functionality which runs sync and async I basically have to write it twice. There's no easy way to just tell a sync method to run async or vice versa (without the hurdles of GetAwaiter().GetResult0. I was hoping that Green Threads are universal and could simply decide whether to run a method sync or async without all the pain and gotchas mentioned above. That would have been nice and suitable for many scenarios - especially in large brown-field projects. |
Affirmative on that one, and so...
...this sounds really good! I'm currently wanting to implement an interface which will work with either the native (i.e. .NET) filesystem, or use a 3rd party (e.g. Dropbox). Currently for creating a folder .NET ONLY provides a sync method, and Dropbox ONLY provides async methods (sigh). It would be great to not have to worry about that. Anyone reading this who knows, I'm currently looking for the best pattern to use to implement this. i.e. the method may or may not be async depending on which provider is currently in use. |
I cannot express how massive this work looks. Being given a lot of effort put into asyncification of projects out there, are you planning to lay out some migration map as well? Or it much to early to have this discussions? There is a lot of projects that spent a lot of time on making their code bases async-friendly when Tasks arrived. Having some good guidance would be helpful. |
I'd expect that desktop apps are much more likely to have use cases where they do native interop and therefore green threads are likely more problematic for them than for ASP.NET apps |
@Eirenarch maybe I'm missing something but why exactly would that be a problem? I haven't really seen any sample code how green threads are used exactly and how the programming model looks like. What would be problematic? |
IIRC (don't quote me on that) when using green threads the runtime constantly plays with the (green)thread stack which means that calling native code which is not aware of these swaps will fail. To work around this the runtimes which use green threads do a bunch of marshalling which kills the performance of native code calls. |
Great that you did this experiment and mapped a path forward. Too bad it’s on hold however. Because Line of business apps and domain logic code greatly benefit from having a lower tier concern such as IO hidden from the code. Its now as if c# is moving towards C and away from its VB roots. Asynch is very verbose in the code and gets most attention, where as the coder/user wants the OOP model to be the focus and tied to business entities, not IO concepts or infrastructure Eg “total = order.Lines().Sum(line => line.Amount), and not “total = await (order.Lines()).Sum(l => l.Amount). it’s not for all cases, like a web server module, but for classic VB style readability in business workflow apps green threads would as much a win as hot update support in C# was in continuation of VB6 debug-edit-and-continue. So I hope you follow the edit and continue roots. |
I never really understood why public string DoSomethingAsync(){
string x = await getSomeValueFromTheDB();
// Do more work
return x;
} Why can't the compiler infer that the method is actually returning And for that matter, why can't it infer, when I am assigning a Task return value to a non Task variable, that I want to use await in that case? You would have a compiler error anyway so you're forced to await and in a forced situation, can't the compiler make an inference? For the less common case where you need to do Make the behavior a compiler switch for those that prefer their code to be explicit. |
It could be. The question is: is that good thing? For that reason, we prefer the abi to be explicit. We could have gone a different direction, but we believe it's better this way. |
Because that might be very wrong and undesirable. Many other task-like types might be more appropriate there. It's important to understand that |
For the same reason as above. This would actually be hugely bad for at least one large group of customers depending on how we did this. Either we would do the equivalent of a naked-await, and be bad for libraries. Or we'd do the equivalent of ConfigureAwait(false) and be terrible for things that depend on sync contexts. Different domains need different patterns, which is why this is an intentionally explicit system. |
We are strongly opposed to dialects of the language on principle. We've only added one in the last 25 years, and only because the benefit was so overwhelming, and the costs too high for the ecosystem otherwise for the success of that feature. That doesn't apply here, so it would be highly unlikely for us to go that route and bifurcate the ecosystem. |
From my perspective, things like minimal APIs and top level statements bifurcate the system, but it seems like there was a justification for doing so, ease of adoption I believe being one of them, and ease of use is something myself and others find relevant. There are people/systems that have different goals. Writing a highly engineered performant library that is going to be used for video processing, or at a scale like Netflix, Amazon and others, those warrant utilization of very explicit Task types (and Spans, Vectors, etc.), and it's fantastic that .NET provides such great performance in these scenarios. For a large number of people, there is a reduced level of performance scrutiny required, as they balance out their workload with the need to produce more business logic. "Wrong/Right" is a matter of perspective and situation. Like others, I've found myself in a situation where something that I never would have thought would be async (I don't know, setting a message header or JSON serialization) suddenly became an async operation and next thing you know you're in a rabbit hole of updates. Having the compiler handle that, when its able to, is a win, at least for some of us, even if it's not perfect from an engineering perspective, because it gets you to a better place and you can always go back and be more explicit if need be. Premature optimization can be a fault. |
If performance isn't a concern then just keep everything synchronous. For any async-only apis, just FYI sync-over-async. It's not good, but it's not going to matter if you're on a reduced perf scrutiny environment. :-)
In this case, you didn't have to go full async. Esp if you are not in a high perf scenario (like you mentioned). Just do the simple thing here and you'll get a solution that works with minimal fuss or perf impact. Likely far less impact than if you had to switch to Green threads. :-) |
Sure. I'm just explaining the perspective as one of the language designers. That there are many of these groups, and we don't want to bifurcate over them, led to the decisions I was commenting on. |
They don't bifurcate the language (which is what I was responding to when I was discussing compiler switches). Top level statements are just c#. They're not an alternative set of semantics that you need to opt into with a compiler switch that then changes them meaning of existing code. Having different meanings for the same code is what we mean by 'dialects' and 'bifurcation'. Having the language support more, while preserving samantic-compat with the last 25 years of c# is fine :-) |
How about a new block type to auto-await code? It essentially a matter of preprocessing for the purpose of domain clarity over IO/concurrency clarity.
|
Sure. Feel free to propose and design over at dotnet/csharplang. If the proposal picks up steam, it could definitely happen. |
What I realize from the discussion is that the existing async/await pattern that has been the recommended approach for i/o intensive workloads has become too much of a burden to coexist with green threads. I wish that some brand new language would come to dotnet runtime with no backward compatibility concerns. |
I would like to express "dissatisfaction" with report. I honestly expect it to be more technical in nature, and not "informational" only. I understand that a lot of time passed after experiment was done, and maybe some details is faded in memory. But if possible that it was cut due to lack of time for preparing more technical explanation, I really would like to read more about what was broken/complicated. And probably what was surprisingly easy to do. |
.NET is itself a 20yo ecosystem, it is not just the languages, but also the runtime, the libraries, the tooling, etc. You will always have back-compat concerns and the only way to get rid of some of those concerns would be to start a new ecosystem from the ground up. That being said, starting a new ecosystem would be a massive undertaking for honestly little benefit. Sure there's things we wish were different, but none so much that they cause unmanageable or unreasonable to handle issues; and none that are so fundamental that it necessitates starting over. I'd also point out that the developers that would most benefit from green threads are the ones that can't or won't rewrite their app to use proper async/await. Such codebases would be unlikely to get approval to move to a new ecosystem/language for similar reasons as to why the async/await rewrite is rejected. Often this is that the short term cost is "too high", even if the long term benefit justifies it. Finally, green threads are not some magic feature that simply make everything better. They are a feature that can make some types of existing code better and which come with their own negatives/drawbacks. Namely it can improve existing purely synchronous code by making it perform closer to how properly written async/await code would, but it gives the devs less control and may not work well with scenarios that leave the "green thread aware" domain (the primary example of which is interop with native or other languages like Java/ObjC) or with scenarios that are trying to do more explicit control via explicit tasks or even explicit threading. So while it might help code that can't migrate; it could inversely hurt code that has already migrated. -- Views/thoughts here are my own and may not be shared by everyone |
@tannergooding your points are well reasoned, but it doesn't address the can't make it async scenario. In a scenario where you can't make things async all the way(dependency you are not in control of) and you have to 'hack' it (sync over async) there is no good answer. That's always bothered me and green threads seemed an elegant solution. The issue around native interop and interoperability with other languages is a significant issue but I don't know. It's trade offs, maintain backwards compatibility, performance of interoperability, finer control over async code vs drastically simpler developer mental model of async code. The older I get and the more code I write the more I lean towards simpler mental models. I care so much about it I can leave some perf on the table to keep things simpler. But that's just my limited experience. Having said that, it's your guys call which way the trade off goes and the team has presented a rational decision. I don't have to like it, but I understand and respect it. |
For anyone looking for more technical details, a report has been checked in with what we found: https://github.com/dotnet/runtimelab/blob/bec51070f1071d83f686be347d160ea864828ef8/docs/design/features/greenthreads.md |
Correct me if I'm wrong, but iirc, currently .NET generates a state machine for every I am not sure why the decision was made to develop a whole new approach, but was the option to replace the generated state machine with green threads even considered? This would not break existing APIs or language paradigms and would improve the existing applications' performance, as I see it. |
The C# compiler emits a state machine for every
From the technical details posted it sounds like the teams did explore this avenue, by having existing blocking I/O methods detect whether they were on a green thread and wiring up notification before yielding. They did rely on |
@rogeralsing If I understand correctly, the main cost of increased foreign function call overhead comes from stack switching. Green thread is initialized with a small stack and grow by-demand, to reduce memory overhead of having many green threads. The called native code does not have stack growing functionality, so not switching stack could cause stack overflow. Golang does stack switching when calling FFI which is also slow. |
I would like to remind that async/awit serve not just for performance, but more importantly to describe the behaviour of the program, so that programmer himself could later understand what program is doing I, personally, dont want the runtime to implicitly create and run some "green threads" I am afraid that the productivity of a develper would only decline with sutch feature because programs may become more bug prone |
Hi, I'm junior system dev who want C# be a better System Programming Language. C# current Async state machine model is really good to handle I/O bound which running in user space(not kernel). Where the recent evolation of system langs like Rust/Zig much care about high performance, control and most imp Memery safety. More over io_uring in Linux space and IOCP in Wids space way to go for thus langs to async work more fater. And I'm hearing some news about Microsoft integrating and investing more in Rustlang which is good alternative for C++ where the CORECLR is writen in am I wrong here ? After the Green Thread result it's clear that async programming is more faster better to work on async model to beat Golang. Pointing out zig issue: ziglang/zig#8224 |
I think the caller should decide whether to be asynchronous or not, which means that the API should all be synchronous. When the caller wishes to make asynchronous calls, the compiler wraps them as asynchronous. So as far as I can see either .net style : |
I'm not sure what you're suggesting there. Without callbacks (facilitated by The closest you get to allowing the caller to determine whether it's async or not is via green threads, by virtue of the caller having to run within a green thread in order for the asynchronous method to be able to park the thread at all. But you still run into the problem that you need the entire ecosystem under that method to be written in a manner that supports notifications and unparking the green threads. That requires splitting the ecosystem, whether that be through relying on existing asynchronous APIs, or having the methods manage in internally. Otherwise, you're back to blocking kernel threads, even if they happen to be executing a green thread. Either way, everything under the hood has to be written to be async-friendly. It has to call specific APIs that support notifications and handle all of the plumbing to resume operation. Both approaches are necessarily viral (tasks or green threads all the way down) otherwise you still block kernel threads. |
When will have it, i dont want use async, await method and normal method, which need 2 method |
I think the go language handles this aspect very well. It has no magic at the usage level and looks simple. The difference between synchronization and asynchronous is just a go keyword. And the goroutine+chan+select method can be applied to most situations. If possible, may be able to use the @ symbol to represent asynchronous calls (currently the @ symbol should only have a keyword escaping function when calling a function prefix? But we might as well add this function), or maybe can directly implement the go keyword of the go language. @CallMathod();//Use the @ symbol to tell the compiler that a synchronous method needs to be called asynchronously
go CallMathod();//Use the go keyword to tell the compiler that the synchronous method needs to be called asynchronously. We already have BlockingCollection and Channel Of course, my understanding of the Go language is still a few years ago. |
When we do something asynchronous, we don't necessarily need a callback. Often we just need a result, which is a concurrent structure. rlt is an AsyncResult<BlockingCollection<T>> or AsyncResult<Channel>
var rlt= go CallMethod;//Asynchronous execution
var rlt=@CallMethod;//Asynchronous execution
CallOtherMethod(); //executes other synchronously
if(rlt.OK/Error/Other) is similar to await, here you can wait for the result synchronously. |
The difference with Go is that everything is a green thread. The process starts on a green thread, and
I suspect that ASP.NET is used much more widely than Go for server-side applications. |
My 2 Euros: I think we should start thinking about CPU compute threads and PCI signals as the un-parking mechanism (flags). Especially curious to see how one motherboard would differ to another one performance-wise. Edit: then we could use the GPU as a memory bank when not handling heavy graphics. Proof: Ideally cubical computer. Asymptotic n-linear space. Threads are being spawned out through (managed?) IRQs. Regular logic. QED. Controversy: It's not a geometric logic, yet. Codename: #KernelNT2 |
I am disappointed that green threads were abandoned. I think the comfort of writing code is much more important than losing small performance (or giving people a choice). Java is looking interesting again with new technology. |
This comment has been minimized.
This comment has been minimized.
@sgf that message is not acceptable. Everyone is free to state their thoughts and opinions here. We don't gatekeep here. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
I would like to see a way to do things without having to resort to special syntactic choices like I have written |
What we really need is for a way for the caller to decide what the implementation of Task should be. Async/Await should work much more like the FSharp does it. |
You could keep the "async" and "await" concept intact but simply change what happens after their compilation. For example, currently, a lot of IL code is generated and executed. Instead of that IL code, everything could be run in green threads. |
This issue is still useful as a summary of the experiment, but current efforts are now focused on Runtime-Async, so I'm going to close this issue and people can look at dotnet/runtime#109632 instead. |
Update
For the runtime-async feature currently in development, see dotnet/runtime#109632
Our goal with the green thread experiment was to understand the basic costs and benefits of introducing green threads to the .NET Runtime environment.
Why green threads
The .NET asynchronous programming model makes it a breeze to write application asynchronous code, which is crucial to achieve scalability of I/O bound scenarios.
I/O bound code spends most of its time waiting, for example waiting for data to be returned from another service over the network. The scalability benefits of asynchronous code come from reducing the cost of requests waiting for I/O by several orders of magnitude . For reference, the baseline cost of a request that is waiting for I/O operation to complete is in the 100 bytes range for async C# code. The cost of the same request is in the 10 kilobytes range with synchronous code since the entire operating system thread is blocked. Async C# code allows a server of the given size to handle several orders of magnitude more requests in parallel.
The downside of async C# code is in that developers must decide which methods need to be async. It is not viable to simply make all methods in the program async. Async methods have lower performance, limitations on the type of operations that they can perform and async methods can only be called from other async methods . It makes the programming model complicated. What color is your function is a great description of this problem.
The key benefit of green threads is that it makes function colors disappear and simplifies the programming model. The green threads should be cheap enough to allow all code to be written as synchronous, without giving up on scalability and performance. Green threads have been proven to be a viable model in other programming environments. We wanted to see if it is viable with C# given the existence of async/await and the need to coexist with that model.
What we have done
As part of this experiment, we prototyped green threads implementation within the .NET runtime exposed by new APIs to schedule/yield green thread based tasks. We also updated sockets and ASP.NET Core to use the new API to validate a basic webapi scenario end-to-end.
The prototype proved that implementing green threads in .NET and ASP.NET Core would be viable.
Async style:
Green thread style:
The performance of the green threads prototype was competitive with the current async/await.
The exact performance found in the prototype was not as fast as with async, but it is considered likely that optimization work can make the gap smaller. The microbenchmark suite created as part of the prototype highlighted the areas with performance issues that future optimizations need to focus on. In particular, the microbenchmarks showed that deep green thread stacks have worse performance compared to deep async await chains.
A clear path towards debugging/diagnostics experience was seen, but not implemented.
Technical details can be found in https://github.com/dotnet/runtimelab/blob/feature/green-threads/docs/design/features/greenthreads.md
Key Challenges
Green threads introduce a completely new async programming model. The interaction between green threads and the existing async model is quite complex for .NET developers. For example, invoking async methods from green thread code requires a sync-over-async code pattern that is a very poor choice if the code is executed on a regular thread.
Interop with native code in green threads model is complex and comparatively slow . With a benchmark of a minimal P/Invoke, the cost of making 100,000,000 P/Invoke calls changed from 300ms to about 1800ms when running on a green thread. This was expected as similar issues impact other languages implementing green threads. We found that there are surprising functional issues in interactions with code which uses thread-local static variables or exposes native thread state.
Interactions with security mitigations such as shadow stacks intended to protect against return-oriented programming would be quite challenging.
It is possible or even likely that we could make the green threads model (a bit) faster than async in important scenarios. The key challenge is that this capability would come with a cost of it being significantly slower in other scenarios and having to give up compatibility and other characteristics.
It is less clear that we could make green threads faster than async if we put significant effort into improving async.
Conclusions and next steps
We have chosen to place the green threads experiment on hold and instead keep improving the existing (async/await) model for developing asynchronous code in .NET. This decision is primarily due to concerns about introducing a new programming model. We can likely provide more value to our users by improving the async model we already have. We will continue to monitor industry trends in this field.
The text was updated successfully, but these errors were encountered: