-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Example Lua module for coroutining #2851
Merged
Merged
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
# cohelper Module | ||
| Since | Origin / Contributor | Maintainer | Source | | ||
| :----- | :-------------------- | :---------- | :------ | | ||
| 2019-07-24 | [TerryE](https://github.com/TerryE) | [TerryE](https://github.com/TerryE) | [cohelper.lua](../../lua_modules/cohelper/cohelper.lua) | | ||
|
||
This module provides a simple wrapper around long running functions to allow | ||
these to execute within the SDK and its advised limit of 15 mSec per individual | ||
task execution. It does this by exploiting the standard Lua coroutine | ||
functionality as described in the [Lua RM §2.11](https://www.lua.org/manual/5.1/manual.html#2.11) and [PiL Chapter 9](https://www.lua.org/pil/9.html). | ||
|
||
The NodeMCU Lua VM fully supports the standard coroutine functionality. Any | ||
interactive or callback tasks are executed in the default thread, and the coroutine | ||
itself runs in a second Lua thread. The coroutine can call any library functions, | ||
but any subsequent callbacks will, of course, execute in the default thread. | ||
TerryE marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Interaction between the coroutine and the parent is through yield and resume | ||
statements, and since the order of SDK tasks is indeterminate, the application | ||
must take care to handle any ordering issues. This particular example uses | ||
the `node.task.post()` API withi the `taskYield()`function to resume itself, | ||
TerryE marked this conversation as resolved.
Show resolved
Hide resolved
|
||
so the running code can simple call `taskYield()` at regular points in the | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure about my English but should it be"simply" |
||
processing to spilt the work into separate SDK tasks. | ||
|
||
TerryE marked this conversation as resolved.
Show resolved
Hide resolved
|
||
A similar approach could be based on timer or on a socket or pipe CB. If you | ||
want to develop such a variant then start by reviewing the source and understanding | ||
what it does. | ||
|
||
## Use | ||
TerryE marked this conversation as resolved.
Show resolved
Hide resolved
|
||
```Lua | ||
do | ||
local function longRunningFunc(yield, ...) | ||
-- <content of function to be split into tasks> | ||
end | ||
require "cohelper".exec(longRunningFunc, params) | ||
end | ||
``` | ||
|
||
## Full Example | ||
|
||
Here is a function which recursively walks the globals environment, the ROM table | ||
and the Registry. Without coroutining, this walk terminate with a PANIC following | ||
a watchdog timout. I don't want to sprinkle the code with `tmr.wdclr(`) that could | ||
in turn cause the network stack to fail. Here is how to do it using coroutining: | ||
|
||
```Lua | ||
require "cohelper".exec( | ||
function(taskYield, list) | ||
local s, n, nCBs = {}, 0, 0 | ||
|
||
local function list_entry (name, v) -- upval: taskYield, nCBs | ||
print(name, v) | ||
n = n + 1 | ||
if n % 20 == 0 then nCBs = taskYield(nCBs) end | ||
TerryE marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if type(v):sub(-5) ~= 'table' or s[v] or name == 'Reg.stdout' then return end | ||
s[v]=true | ||
for k,tv in pairs(v) do | ||
list_entry(name..'.'..k, tv) | ||
end | ||
s[v] = nil | ||
end | ||
|
||
for k,v in pairs(list) do | ||
list_entry(k, v) | ||
end | ||
print ('Total lines, print batches = ', n, nCBs) | ||
end, | ||
{_G = _G, Reg = debug.getregistry(), ROM = ROM} | ||
) | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
# Coroutine Helper Module | ||
|
||
Documentation for this Lua module is available in the [ftpserver.md](../../docs/lua-modules/cohelper.md) file and in the [Official NodeMCU Documentation](https://nodemcu.readthedocs.io/) in `Lua Modules` section. | ||
TerryE marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
--[[ A coroutine Helper T. Ellison, June 2019 | ||
|
||
This version of couroutine helper demonstrates the use of corouting within | ||
NodeMCU execution to split structured Lua code into smaller tasks | ||
|
||
]] | ||
--luacheck: read globals node | ||
TerryE marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
local modname = ... | ||
|
||
local function taskYieldFactory(co) | ||
local post = node.task.post | ||
return function(nCBs) -- upval: co,post | ||
post(function () -- upval: co, nCBs | ||
coroutine.resume(co, nCBs or 0) | ||
end) | ||
return coroutine.yield() + 1 | ||
TerryE marked this conversation as resolved.
Show resolved
Hide resolved
|
||
end | ||
end | ||
|
||
return { exec = function(func, ...) -- upval: modname | ||
package.loaded[modname] = nil | ||
local co = coroutine.create(func) | ||
coroutine.resume(co, taskYieldFactory(co), ... ) | ||
end } | ||
|
||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it is no good calling this a thread. As the PiL also points out it is only similar. A thread is executed concurrently. So taffeta will get confused. Maybe use "execution context" or whatever.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gregor, I must disagree on this one. Until we had multi-processor CPUs. threads would never be executed concurrently, since there was only one processing unit. The WP thread article has a nice definition "a thread of execution is the smallest sequence of programmed instructions that can be managed independently by a scheduler". Features such as concurrence and pre-emption are more emergent benefits of this independence. In our Lua world the machine is the Lua VM and this must, like node.js, run in a single OS thread.
What I am trying to do is to explain how this works to the average IoT developer. If you can suggest better wording then I'll adopt this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The term to use, I think, is "cooperative multi-threading", assuming I am correct that, on our C substrate, a Lua task will run to completion, without preemption and without concurrent access to the Lua VM and its heap, even if tasks are being dispatched by multiple threads. I'm not sure that this module's documentation is the right place to spell it out; perhaps
node.task.post()
is the right place and this should simply cite that.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've used stack instead of thread.