From cd9569c89c2263b135a4ef3450413520dc48a7ea Mon Sep 17 00:00:00 2001 From: Terry Ellison Date: Wed, 24 Jul 2019 11:23:04 +0100 Subject: [PATCH 1/3] Example Lua module for coroutining --- docs/lua-modules/cohelper.md | 69 +++++++++++++++++++++++++++++++ lua_modules/cohelper/README.md | 3 ++ lua_modules/cohelper/cohelper.lua | 27 ++++++++++++ 3 files changed, 99 insertions(+) create mode 100644 docs/lua-modules/cohelper.md create mode 100644 lua_modules/cohelper/README.md create mode 100644 lua_modules/cohelper/cohelper.lua diff --git a/docs/lua-modules/cohelper.md b/docs/lua-modules/cohelper.md new file mode 100644 index 0000000000..6b56ade034 --- /dev/null +++ b/docs/lua-modules/cohelper.md @@ -0,0 +1,69 @@ +# cohelper Module +| Since | Origin / Contributor | Maintainer | Source | +| :----- | :-------------------- | :---------- | :------ | +| 2019-07-24 | [TerryE](https://github.com/TerryE) | [TerryE](https://github.com/TerryE) | [cohelper.lua](../../lua_modules/cohelper/cohelper.lua) | + +This module provides a simple wrapper around long running functions to allow +these to execute within the SDK and its advised limit of 15 mSec per individual +task execution. It does this by exploiting the standard Lua coroutine +functionality as described in the [Lua RM §2.11](https://www.lua.org/manual/5.1/manual.html#2.11) and [PiL Chapter 9](https://www.lua.org/pil/9.html). + +The NodeMCU Lua VM fully supports the standard coroutine functionality. Any +interactive or callback tasks are executed in the default thread, and the coroutine +itself runs in a second Lua thread. The coroutine can call any library functions, +but any subsequent callbacks will, of course, execute in the default thread. + +Interaction between the coroutine and the parent is through yield and resume +statements, and since the order of SDK tasks is indeterminate, the application +must take care to handle any ordering issues. This particular example uses +the `node.task.post()` API withi the `taskYield()`function to resume itself, +so the running code can simple call `taskYield()` at regular points in the +processing to spilt the work into separate SDK tasks. + +A similar approach could be based on timer or on a socket or pipe CB. If you +want to develop such a variant then start by reviewing the source and understanding +what it does. + +## Use +```Lua +do + local function longRunningFunc(yield, ...) + -- + end + require "cohelper".exec(longRunningFunc, params) +end +``` + +## Full Example + +Here is a function which recursively walks the globals environment, the ROM table +and the Registry. Without coroutining, this walk terminate with a PANIC following +a watchdog timout. I don't want to sprinkle the code with `tmr.wdclr(`) that could +in turn cause the network stack to fail. Here is how to do it using coroutining: + +```Lua +require "cohelper".exec( + function(taskYield, list) + local s, n, nCBs = {}, 0, 0 + + local function list_entry (name, v) -- upval: taskYield, nCBs + print(name, v) + n = n + 1 + if n % 20 == 0 then nCBs = taskYield(nCBs) end + if type(v):sub(-5) ~= 'table' or s[v] or name == 'Reg.stdout' then return end + s[v]=true + for k,tv in pairs(v) do + list_entry(name..'.'..k, tv) + end + s[v] = nil + end + + for k,v in pairs(list) do + list_entry(k, v) + end + print ('Total lines, print batches = ', n, nCBs) + end, + {_G = _G, Reg = debug.getregistry(), ROM = ROM} +) +``` + diff --git a/lua_modules/cohelper/README.md b/lua_modules/cohelper/README.md new file mode 100644 index 0000000000..48164b4cf0 --- /dev/null +++ b/lua_modules/cohelper/README.md @@ -0,0 +1,3 @@ +# Coroutine Helper Module + +Documentation for this Lua module is available in the [ftpserver.md](../../docs/lua-modules/cohelper.md) file and in the [Official NodeMCU Documentation](https://nodemcu.readthedocs.io/) in `Lua Modules` section. diff --git a/lua_modules/cohelper/cohelper.lua b/lua_modules/cohelper/cohelper.lua new file mode 100644 index 0000000000..77dcc39c79 --- /dev/null +++ b/lua_modules/cohelper/cohelper.lua @@ -0,0 +1,27 @@ +--[[ A coroutine Helper T. Ellison, June 2019 + +This version of couroutine helper demonstrates the use of corouting within +NodeMCU execution to split structured Lua code into smaller tasks + +]] +--luacheck: read globals node + +local modname = ... + +local function taskYieldFactory(co) + local post = node.task.post + return function(nCBs) -- upval: co,post + post(function () -- upval: co, nCBs + coroutine.resume(co, nCBs or 0) + end) + return coroutine.yield() + 1 + end +end + +return { exec = function(func, ...) -- upval: modname + package.loaded[modname] = nil + local co = coroutine.create(func) + coroutine.resume(co, taskYieldFactory(co), ... ) +end } + + From d2e3cd8d47167df94c166be3c2dda0b6b192a892 Mon Sep 17 00:00:00 2001 From: Terry Ellison Date: Wed, 24 Jul 2019 17:43:29 +0100 Subject: [PATCH 2/3] Updats in light of review --- docs/lua-modules/cohelper.md | 38 +++++++++++++++++++++++-------- lua_modules/cohelper/README.md | 2 +- lua_modules/cohelper/cohelper.lua | 2 +- 3 files changed, 30 insertions(+), 12 deletions(-) diff --git a/docs/lua-modules/cohelper.md b/docs/lua-modules/cohelper.md index 6b56ade034..37e0832564 100644 --- a/docs/lua-modules/cohelper.md +++ b/docs/lua-modules/cohelper.md @@ -16,7 +16,7 @@ but any subsequent callbacks will, of course, execute in the default thread. Interaction between the coroutine and the parent is through yield and resume statements, and since the order of SDK tasks is indeterminate, the application must take care to handle any ordering issues. This particular example uses -the `node.task.post()` API withi the `taskYield()`function to resume itself, +the `node.task.post()` API with the `taskYield()`function to resume itself, so the running code can simple call `taskYield()` at regular points in the processing to spilt the work into separate SDK tasks. @@ -24,17 +24,35 @@ A similar approach could be based on timer or on a socket or pipe CB. If you want to develop such a variant then start by reviewing the source and understanding what it does. -## Use -```Lua -do - local function longRunningFunc(yield, ...) - -- - end - require "cohelper".exec(longRunningFunc, params) -end +### Require +```lua +local cohelper = require("cohelper") +-- or linked directly with the `exec()` method +require("cohelper").exec(func, ) ``` -## Full Example +### Release + +Not required. All resoruces are release on completion of the `exec()` method + +## `cohelper.exec()` +Execute a function which is wrapper by a coroutine handler. + +#### Syntax +`require("cohelper").exec(func, )` + +#### Parameters +- `func`: Port number for HTTP server. Most HTTP servers listen at port 80. +- ``: list of 0 or more parameters used to initialise func. the number and types must be matched to the funct declaration + +#### Returns +Return result of first yield. + +#### Notes +1. The coroutine function `func()` has 1+_n_ arguments The first is the supplied task yield function. Calling this yield function within `func()` will temporarily break execution and cause an SDK reschedule which migh allow other executinng tasks to be executed before is resumed. The remaining arguments are passed to the `func()` on first call. +2. The current implementation passes a single integer parameter across `resume()` / `yield()` interface. This acts to count the number of yields that occur. Depending on your appplication requirements, you might wish to amend this. + +### Full Example Here is a function which recursively walks the globals environment, the ROM table and the Registry. Without coroutining, this walk terminate with a PANIC following diff --git a/lua_modules/cohelper/README.md b/lua_modules/cohelper/README.md index 48164b4cf0..7706aa2c3c 100644 --- a/lua_modules/cohelper/README.md +++ b/lua_modules/cohelper/README.md @@ -1,3 +1,3 @@ # Coroutine Helper Module -Documentation for this Lua module is available in the [ftpserver.md](../../docs/lua-modules/cohelper.md) file and in the [Official NodeMCU Documentation](https://nodemcu.readthedocs.io/) in `Lua Modules` section. +Documentation for this Lua module is available in the [Lua Modules->cohelper](../../docs/lua-modules/cohelper.md) MD file and in the [Official NodeMCU Documentation](https://nodemcu.readthedocs.io/) in `Lua Modules` section. diff --git a/lua_modules/cohelper/cohelper.lua b/lua_modules/cohelper/cohelper.lua index 77dcc39c79..f463f9b4c1 100644 --- a/lua_modules/cohelper/cohelper.lua +++ b/lua_modules/cohelper/cohelper.lua @@ -21,7 +21,7 @@ end return { exec = function(func, ...) -- upval: modname package.loaded[modname] = nil local co = coroutine.create(func) - coroutine.resume(co, taskYieldFactory(co), ... ) + return coroutine.resume(co, taskYieldFactory(co), ... ) end } From 0a4af035354d45d3a2eb2a6713d6812baded5a30 Mon Sep 17 00:00:00 2001 From: Terry Ellison Date: Thu, 25 Jul 2019 13:53:03 +0100 Subject: [PATCH 3/3] Updates following PR review --- docs/lua-modules/cohelper.md | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/docs/lua-modules/cohelper.md b/docs/lua-modules/cohelper.md index 37e0832564..7909ab123f 100644 --- a/docs/lua-modules/cohelper.md +++ b/docs/lua-modules/cohelper.md @@ -3,26 +3,27 @@ | :----- | :-------------------- | :---------- | :------ | | 2019-07-24 | [TerryE](https://github.com/TerryE) | [TerryE](https://github.com/TerryE) | [cohelper.lua](../../lua_modules/cohelper/cohelper.lua) | -This module provides a simple wrapper around long running functions to allow +This module provides a simple wrapper around long running functions to allow these to execute within the SDK and its advised limit of 15 mSec per individual -task execution. It does this by exploiting the standard Lua coroutine +task execution. It does this by exploiting the standard Lua coroutine functionality as described in the [Lua RM §2.11](https://www.lua.org/manual/5.1/manual.html#2.11) and [PiL Chapter 9](https://www.lua.org/pil/9.html). The NodeMCU Lua VM fully supports the standard coroutine functionality. Any interactive or callback tasks are executed in the default thread, and the coroutine -itself runs in a second Lua thread. The coroutine can call any library functions, -but any subsequent callbacks will, of course, execute in the default thread. +itself runs in a second separate Lua stack. The coroutine can call any library +functions, but any subsequent callbacks will, of course, execute in the default +stack. -Interaction between the coroutine and the parent is through yield and resume -statements, and since the order of SDK tasks is indeterminate, the application +Interaction between the coroutine and the parent is through yield and resume +statements, and since the order of SDK tasks is indeterminate, the application must take care to handle any ordering issues. This particular example uses -the `node.task.post()` API with the `taskYield()`function to resume itself, -so the running code can simple call `taskYield()` at regular points in the -processing to spilt the work into separate SDK tasks. +the `node.task.post()` API with the `taskYield()`function to resume itself, +so the running code can call `taskYield()` at regular points in the processing +to spilt the work into separate SDK tasks. -A similar approach could be based on timer or on a socket or pipe CB. If you +A similar approach could be based on timer or on a socket or pipe CB. If you want to develop such a variant then start by reviewing the source and understanding -what it does. +what it does. ### Require ```lua @@ -33,17 +34,17 @@ require("cohelper").exec(func, ) ### Release -Not required. All resoruces are release on completion of the `exec()` method +Not required. All resources are released on completion of the `exec()` method. ## `cohelper.exec()` -Execute a function which is wrapper by a coroutine handler. +Execute a function which is wrapped by a coroutine handler. #### Syntax `require("cohelper").exec(func, )` #### Parameters -- `func`: Port number for HTTP server. Most HTTP servers listen at port 80. -- ``: list of 0 or more parameters used to initialise func. the number and types must be matched to the funct declaration +- `func`: Lua function to be executed as a coroutine. +- ``: list of 0 or more parameters used to initialise func. the number and types must be matched to the funct declaration #### Returns Return result of first yield.