GraalVM JavaScript replaces Nashorn, breaking Blockly and existing Nashorn rules #2433
Comments
I don't think so currently. But it would be nice if the core would support multiple versions of script engine languages. That way you could slowly migrate your rules from one major version to another. But it would also mean that script engine versions need to be stored for rules created in the UI and there needs to be a way to determine the script engine version for scripts in files. Because Nashorn has been removed from Java 15, there won't be any ECMAScript 5.1 script engine available in newer Java versions (unless an Automation add-on is created that provides Nashorn (openhab/openhab-distro#1316 (comment))). Eventually the Blockly integration will also need to support generating code for newer ECMAScript versions. So it wouldn't hurt if the UI enables Blockly depending on the available ECMAScript engine version. It would be even nicer if the Blockly integration supports generating code for multiple ECMAScript versions. 🙂 |
|
Given how different the GraalVM add-on implements the interactions between rules and openHAB itself compared to Rules DSL, Nashorn, and Jython, this might be a chance to step back and think about how we want it to work. To a large extent, it's going to be way more than just migration from ECMAScript 5.1 to ECMAScript whatever GraalVM supports. It's going to be reimplementation in how everyone needs to write rules. In the traditional languages all sorts of things are imported and available to the rules coder by default:
There's more but those are the most used. In GraalVM JavaScript a design decision was made to import nothing. And it's not clear in the current set of documentation how to access that stuff. But to access that stuff means each and every rule/Script Action/Script Condition is going to require a whole bunch of import statements/load statements just to do simple stuff like logging or sending a command to an Item. It might be a good idea to bring this up as part of any conversation when moving to Java 15+. GraalVM isn't just an updated JavaScript support. |
|
I downloaded the newest distribution, installed the GraalVM JS add-on, and tried a new Blockly script and indeed it doesn't work but this is the actual error: So from my point of view, maybe we could focus on this error and try to fix it before bailing out and saying installing the GraalVM add-on breaks Blockly because of compatibility issues? In terms of the ECMAScript language, if Blockly generates code targeting a certain language version, subsequent versions should be able to interpret it still. It seems that the JSScripting add-on expects some directories and will fail if they are not present, this seems easy enough to fix? |
|
It's significantly more involved than just an upgrade to the ECMAScript issue. In addition to using a different folder structure (which is the root of the error above), the GraalVM JavaScript add-on does not import anything from openHAB. In Nashorn we are used to having:
GraalVM JavaScript makes none of that available. You have to go and manually import all of those things manually if you want to use them in your script. So even when that error gets fixed, Blockly would still need to be significantly changed to produce code that will work for the GraalVM JavaScript and that code will not be compatible with Nashorn. For all intents and purposes Nashorn JavaScript and GraalVM JavaScript may as well be two completely different languages. |
|
I see, thanks for the explanation. I would have thought these were provided by DefaultScopeProvider and available regardless of the script engine. Still, it's not that far of a stretch to adapt Blockly code to make these imports when needed - it's the case already when you use the "log" block (https://github.com/openhab/openhab-webui/blob/d8a16db016b149d007ef31a320fcdc000b8c51dc/bundles/org.openhab.ui/web/src/assets/definitions/blockly/ohblocks.js#L124-L132). My point is that with Nashorn deprecated, Jython unmaintained (and a EOL 2.7 version of Python), DSL Rules being legacy, and Groovy being kind of niche, GraalVM might be the only way viable option left (with JRuby?) as a embedded scripting engine going forward, so we should try to make it work. |
|
The developer went out of his way to avoid polluting the namespaces so rules could be pure JavaScript. There are legitimate reasons for doing so but it's not clear to me how the DefaultScopeProvider is used but it's not in the same way as it happens in Nashorn. In Nashorn the variables are just there thanks to the provider but in GraalVM JS rules they are not. I don't disagree that we should try to make it work with something that will have support in the long run. But in the mean time GraalVM JavaScript will continue to break anything that is written for Nashorn which will trip up any user who wants to use Blockly or existing Nashorn code. If it's going to be months or years before Blockly can be Given that GraalVM JavaScript has almost no documentation yet, pretty much only those who have the skills and willingness to study @jpg0's library are able to really build anything with it which makes moving Blockly over to support it somewhat of a challenge. I also don't know if anyone is trying to build rules in the UI with it so there may be gotchas we are unaware of. I know the jRuby folks too are ignoring UI rules. I haven't tried to keep up with their documentation though. They might be a bit more well documented but I don't know if they've even submitted jRuby to be considered as an official add-on yet. I filed an issue on openhab-core to see if we could get Nashorn and GraalVM to exist side-by-side which would be the best way to bridge between what we have now to where we need to go but that doesn't seem to be possible. It's either or, not both. I'm worried about months and months or even years between now and when Blockly can support something more future proof where users can't figure out why stuff is broken and not working because they installed an add-on. I'd love to hear @jpg0's opinion on how to best to proceed. |
|
Firstly, I'd like to apologize for the slow progress on the add-on and docs; I've recently moved house (well, moved the family from one side of the world to the other) so have had other priorities. As for work on the GraalVM add-on, the plan is to update the folder structure (to make it compatible with the JS ecosystem), then write the docs. @rkoshak is correct that the add-on diverges from the Nashorn by not importing things by default. Default things can be imported explicitly if required - they are at Anyway, regarding side-by-side installs - when you install the GraalJS add-on, in fact it doesn't replace the Nashorn engine (that engine is actually still used for JS transformations), however when openHAB attempts to find an engine for a script it looks it up by file extension, and only one engine can be returned. To address your concern, it seems that the challenge is that neither Nashorn nor Graal can accept each others' scripts. Ideally we would have a backwards-compatible upgrade. The options that I can see are:
Of course there is also the option to never move to this 'everything explicit' model, but that forever shuts out the ability to rely on and integrate with existing 3rd party JS. |
|
I don't think you need to apologize. We all have life commitments. It is certainly not the only place where openHAB docs are lacking.
Deferring the issue though could be a good thing if it keeps people from being confused and breaking their OH in the time between now and a final solution is implemented. It really is causing problems for all sorts of users, even OH users who have been using OH for years.
That only addresses text based rules, not UI rules. There's no file in that case. The engine is chosen, presumably, based on the "type" property of the action configuration. However, the version of the language doesn't seem to be a part of that. It's just This does seem like it might be an OK approach long term perhaps. I can imagine that there might be similar incompatible versions of languages that might gain support sometime in the future. But that's all stuff that has to happen over on openhab-core I think. |
|
So in the Blockly case, after reading openhab/openhab-addons#11221 and creating the missing directory (something that should IMHO be needed but probably easy to fix), given that the code generated by Blockly only uses a couple objects for now (namely
var runtime = (typeof(require) === "function") ? require("@runtime") : undefined;
if (runtime) {
var itemRegistry = runtime.itemRegistry, events = runtime.events; // etc.
}
var runtime = (typeof(require) === "function") ? require("@runtime") : {
itemRegistry: itemRegistry,
events: events,
// etc.
};The trick is that Then I confirmed that (in the 1st alternative): print(itemRegistry);
print(events);resp. in the 2nd alternative: print(runtime.itemRegistry);
print(runtime.events);yields the same result in both Nashorn & GraalVM: With that simple addition to Blockly-generated code we could then still offer it when the JSScripting add-on is installed (the rest of the generated ECMAScript code being simple enough should hopefully work in both ECMAScript versions). Maybe ithis add-on could even become installed as part of the standard package in the future, like RRD4j persistence is today. |
|
@ghys you may also want to merge everything from the runtime into the current context (with something like @rkoshak Another thing I was thinking about was the option to explicitly version, possibly via adding to the mime types in this case (I realised that if we do change the root directories for JS with GraalVM, then we can use this to indicate the version for files). I believe that the mime-types are simply strings, so it would be perfectly possible for the new implementation to register with something like |
|
@jpg0 I like the idea of being able to define the version number somehow. It seems to me that this is only the first time that we will run into this problem but indeed it will come up again and again. For example, what happens when ECMAScript 2022 or what ever comes out? Having the version that a given rule/script was written for can also help with your third proposal with error messages. "I see you used foo. In the new JavaScript XXXX you should now use bar." @ghys, this is fantastic news. I would really really like to be able to move towards selecting a "default" language that isn't already deprecated and will disappear as soon as OH needs to move to a new JVM and it makes sense that the default would be the same thing backing Blockly. GraalVM JavaScript is a good fit for this, I've just not had the time to figure out how to use it yet. But I agree, having it installed as a default like rrd4j seems to be a good idea once we can ensure that doing so doesn't break Nashorn. Doing so in the near term will give users time to migrate to GraalVM JavaScript (or maybe other languages too, Python would be greatly appreciated by a lot of users). Heck, I'll even write some tutorials for how to do it and contribute to some docs if I can. The trouble for me right now is time but I've got some off work time coming up so maybe I can look into it then. I've been relatively unhappy with the thought of writing rule templates for the marketplace using languages that are basically deprecated and will need to be rewritten. |
Yes, thanks, it makes sense! For now as I've mentioned earlier only 2 objects are needed from the openHAB API in Blockly so it's also reasonable to expose only those two but as we eventually add functionality it may become the go-to option. The good thing is that the actual JS code generated by Blockly can be changed on a whim because it's regenerated in full whenever the source blocks change.
I obviously meant not be needed, at least manually.
Completely agree with this and all of your points. I'm almost ready to add rule template support in the UI and eventually open the marketplace to the public, but I'm actually worried about the stability of the script languages that might be used in them - not only languages but also API differences to address OH objects since we have 2 JS engines and they are incompatibilities between them. So before we do that we might have to settle on a common API and stick to it - or even maybe mandate GraalVM for rule templates that have JS scripts in them so every script would have to |
|
Sorry if this is has been discussed before (I'm sure it has) , but is there a reason we are not injecting the runtime objects by default into the GraalVM instance? Or at least making it an option (turned on by default)? My assumption is that nearly every user (90%+ maybe?) switching to use GraalVM would need to add those to their scripts , seems like a lot of forced boiler plate code to make users copy and paste. I'm sure there's a reason not too, hence maybe making it configurable. I for one really struggled with this when i was trying it out a few months ago, it made what is a super awesome feature feel not production ready. |
There's a balancing act that has to be taken between injecting stuff and potentially polluting the namespace with the ability to support third party libraries which might use the same variables. My understanding is that @jpg0 choose the side that maintains the maximum ability to use any third party libraries (e.g. like libraries installed using npm, not just OH specific libraries) over making less work for rules developers who can rely on openHAB stuff just being there. I personally am not sure where the best balance is. I do agree that it does seem like a lot fo boiler plate that has to be added, especially when working in the UI where one could have more than one Script Action and Script Condition per rule and each one would have to re-include the boilerplate. I'd almost suggest that with UI scripts that stuff be included by default no matter what through some behind the scenes magic or make it configurable as you suggest. People writing rules in the UI are unlikely to ever install a third party library via npm or any other way and the boiler plate is most glaring there. But even with Nashorn we are stuck with a bunch of boiler plate already. I can't call So even though it's a lot of boiler plate, it's not really any more boiler plate than we already have to do. To anyone switching from Jython or Nashorn JavaScript it's just a different set of boiler plate, not really more boiler plate. And anyone switching from Rules DSL to any other language already has to add a lot of boiler plate to their rules. And really, this is what the Helper Libraries are all about. A way to abstract and consolidate all of that boiler plate and even more boilerplate once you look at what it takes to actually create a rule in a text file in any of these languages. Without them (or their equivalent in GraalVM JavaScript or jRuby) writing even simple rules requires a level of understanding for openHAB internal classes and structure that is beyond what should be needed for our users. It has always been my opinion that the automation add-on by itself has never been sufficient on it's own. A helper library for that language is also required to make the interactions between OH and the script seamless. To date, none of the helper libraries that have been written are even part a part of the OH project though. And while I think it's theoretically possible to package a helper library as separate add-on there are none who have done so yet. @jpg0, correct me if I'm wrong but your helper library is installed through NPM. The jRuby helper library is installed through ruby gems. The Nashorn/JavaScript/Groovy? helper libraries are installed by cloning the github repo and copying the files to the OH conf folder. Why do I need to resort to an external tool to install the library that makes the language usable to write rules? Were I king for a day, the helper libraries for these would be a part of the add-on so you get both. Then the docs become easier to write, the code gets easier for end users to write, and the overall experience is much nicer. But every time I bring this up I seem to get pushback which I don't understand. Anyway, I'm off my soapbox. Rant over, for now. ;-) |
|
I wonder if it would make sense to have a generic "Javascript" binding which is more like a "meta" package, so mostly empty, but depends on both the GraalVM binding and a "GraalVM Runtime Helpers" binding which simply injects the runtime objects, logging ,etc... Most user users would install this generic "Javascript" binding (and get these two as dependencies) and be happy. Advanced users could install the raw GraalVM binding and have more control over the namespace. |
|
@rkoshak spot on with your explanation. Helper libraries are essential, and the standard JS module loading ( @digitaldan My fear is that bifurcating script runtimes will mean that we have two 'flavours' of script which will lead to greater confusion in the future when users share their scripts etc. From many years of development experience I firmly believe that too much 'magic' always bites you in the end. As a pretty trivial example, there is nothing stopping there being new variables added to the default context - in fact any part of openHAB core or any add-on can do it (and it is designed this way) - what do they choose as the name for the variable? Because if it has already been used in a script somewhere, this can cause nasty problems (and with existing names like All you need at the top of your script is a statement like Other, better, standard options:
(Note that it is even possible to put your code in a |
|
To plays devil's advocate here a little though. While I agree that too much magic can become a problem, and in fact I think it is a problem in Rules DSL, too much boilerplate can also be a problem. When one is writing rules in text files where you might have 100+ lines of code with two or three rules defined adding one or two lines to the top of the file to import stuff you will almost always need is no big deal. It's just the cost of writing code that will work with openHAB. However, when it comes to UI rules it becomes much more onerous. In my experience a typical Script Condition in Nashorn right now is about three lines of code that does real work plus another four to five lines of code to import the stuff I need even with all the magic that Nashorn provides with items and event and whatnot. It's not quite so bad in Script Actions as those, again in my experience, average around 20 lines of code that do real work again with another four to five lines of code to import stuff. But that still means roughly 1/5th of the code I need to write for each and every Script Action (and there can be more than one Script Action in a single rule) is boilerplate. I'm always gonna want to log. I'm always gonna want to interact with Items. I frequently need to interact with the ItemRegistry and openHAB core Actions (which are harder to use than binding provided Actions today). So I'll have to import these for each and every Script Action I write. If you assume one Script Condition and one Script Action per rule and a modest 15 rules, that's a lot of boiler plate. On the low end that's 120 lines of code just to get started. We haven't even begun to actually implement anything useful, we've just imported the stuff we need to interact with OH. At least in the context of UI written rules, anything but the "dangerous path" described above is placing a pretty big burden on our least technically adept users. Even with the dangerous path we are looking at 30 lines of code out of 375 lines of production code being boilerplate. That's almost 10%. This sort of thing is also why I always bring up "have you thought about how it will work in UI rules?" and why I get so frustrated when the answer is invariably "don't know. haven't thought about it. probably but that's not my focus." These kind of choices impose a pretty big burden on the OH users who are least able to bear it. If one is writing rules in the UI one is most likely not an advanced coder. What one is doing is relatively simple and the likelihood they'll ever need to import a third party library is pretty small. So why impose a 10-20% burden on these users if we can avoid it? |
|
An area which is maybe an extreme version of your example is JS transformations (defined inline, not in a distinct file). My belief is that for many of these, users just want a simple line of JS, such as One thing that I would note is that with ES6 you can import everything you need on a single line: Maybe one option to help this would be to allow users to define a custom header for all their scripts in the UI? It could default with a load of things imported. They could add things to the list if they want, for all their scripts. I would just want to ensure that openHAB never updates the list itself (which it does today) as that can break existing scripts. |
|
I would expect them to shrink quite a bit, though it would still be irksome. We will find out soon I suspect, especially if Yannick's proposal to limit rule templates to GraalVM JS comes to pass. Even though I focused on lines of code, that's only part of the problem though. The users also need to know what to import in the first place. That too is a burden, especially for the less skilled users. It can be greatly alleviated with a Helper Library but without it one needs to know of and how to use the Java and OH Javadocs just to know what to import and from where and how to use them.
One question out of ignorance. That imports the OH Java classes we might need, JS libraries, or both? Could I use that same line to import say I like any approach that makes creating rules in the UI better for the less technical users. |
|
To answer your last question, it allows importing any symbols that have been exported by the JS library that is being referenced. It does not allow importing Java classes directly (that could not be part of standard JS), however there is nothing preventing the JS library exporting Java classes for the script to import. It would probably even be possible to create a library allow importation of arbitrary java classes, which could be used like: Note that this style of import requires the name of the thing being imported, and in this case a |
|
When making openhab/openhab-webui#1170 I found that i.e.
if (typeof(require) === "function") {
ctx = this;
var runtime = require("@runtime");
itemRegistry = runtime.itemRegistry;
events = runtime.events;
}
function mathRandomInt(a, b) {
if (a > b) {
// Swap a and b to ensure a is smaller.
var c = a;
a = b;
b = c;
}
return Math.floor(Math.random() * (b - a + 1) + a);
}
var logger = Java.type('org.slf4j.LoggerFactory').getLogger('org.openhab.rule.' + ctx.ruleUID);
events.sendCommand('NewItem', (String(mathRandomInt(1, 100))));
logger.info(itemRegistry.getItem('NewItem').getState());...and the logging works. |
|
So it's looking like my original assertion that an automation add-on without corresponding helper libraries is going to be pretty unusable for many OH users still. The average OH rules developer shouldn't have to know that ZonedDateTime needs to be imported one way and personal library handled a completely different way. They shouldn't have to spend time looking through the JavaSE Javadocs and the openHAB Javadocs to know what needs to be imported from where and then figure out how. I know this complaint is bigger than this particular issue but my biggest concern with Jython, Nashorn JavaScript, GraalVM JavaScript and all the rest is that we require the end users to know and understand far too much about differences in types (just today I had to explain why someone can't just multiple a Number Item's state by 1000 because the state is a java.lang.Number but 1000 is a Python primitive int). They have to know far too much about openHAB internals and they have to pay close attention to know when they are working with a Java Object or a language native entity. Helper libraries can greatly help with these but, as I said, those are all outside of the OH project. Take creating a Timer for example. First I have to find where That's what's missing and what I think is absolutely required to make these languages usable. Without them it's just too hard. You have to be a pretty experienced programmer to get anywhere or be a pretty lucky cargo-cult-programmer. So I wonder if the discussion about imports and such is heading down the wrong path. Maybe we need to focus on making these helper libraries a part of the OH project itself and making them as easy to install as an add-on, or even better make them a part of the automation add-ons themselves. A whole lot of these problems can be handled in the helper libraries. |
I completely agree with this, our focus should be "What's the best experience for the majority of users?" Right now, this is not an ideal solution for the vast majority, which is a shame, as i think this could be one of our best features!
So i would agree more choices is not desirable here, my suggestion was under the assumption that there is a use case for having a runtime without global namespace pollution . But i believe this only benefits a minority of users, so to quote Rich, if i were king for a day, there would be one javascript binding, and that would have sane global objects that would allow me to access runtime features like logging, items, etc... Without this, the GraalVM binding is not useable, even to experienced users. Could this conversation not pivot to what that global injection namespace looks like? Could we not restrict this to a few common ones that would stay constant, but whose internal functions may grow over time (like a I am fully aware many here have vast previous experience discussing this, and i am not suggesting anything new or revolutionary, but i'm also feeling like we have been stuck for awhile in this situation, and I for one would really, really like to start using this binding. |
Interestingly, when I first started looking at using JS with openHAB, this was a point of contention between myself and Scott (at the time). I did agree with him that we should push most functionality into Java, however there are too many rough edges in the Java-to-other-language conversions that makes using Java objects directly fraught with challenges (often due to type issues). I do agree that we should have helper libraries for each language, and that these should be the primary interface to openHAB for script writers. I couldn't agree more that script writers should not have to know Java! Personally I have no problem with helper libraries diverging between languages, because this allows idiomatic usage of each language. As for whether these are bundled in the add-on, or installed via the package manager for the language in question, I don't really mind. I'm aware that this isn't a new discussion. (Oh, and I'm not sure about libraries being part of automation itself, as this would make updating them really hard and they would likely fall behind.)
This is certainly something which has been raised before. However we could not use anything else that could clash with existing names, and I'm sure that My view is that we can use a combination of the all the discussed approaches. What I would love to see is:
My hope is that this should be able to serve both beginner and advanced users. I do believe that advanced users are worth supporting because I expect them to generate community libraries. For example, I have my system to calculate and adjust light temperatures throughout the day, for which I use chroma.js, and it would be great to make this available to the community. (And maybe a supporting point is the lack of libraries for ES5 because the dev experience is so poor.)
I agree with this! Although maybe not for the same reasons - there are bugs, a broken directory layout and no docs, all of which need fixing before I would declare it ready. (I am confident in it's stability and capability due to having using it extensively over more than a year.) |
|
I think it's a sensible approach if we do the transition in phases and make proper announcements - something like:
Also I think there could be a case for the new marketplace to distribute scripting libraries as add-ons, and even custom Blockly blocks made by the community (described declaratively, like UI widgets or rule templates - defining that could prove challenging but is not impossible). A dedicated developer tool in the UI would allow defining the block's structure and the code they generate, in YAML, like widgets. |
|
The suggested timeline ties openHAB to Java 11 for nearly 1.5 years, since Nashorn was removed in Java 15. |
|
I'm essentially still scratching around with my 3 year old very rudimentary scripts that I handed over to the openhab-helper-libraries back in the day. Since then I'm waiting for the firm integration of the really incredible great work you've done with GraalVM in openHab. A few times I set up a test system with it and I had tears in my eyes, when I finally will be able to use these things in production systems. Disappointed I am back to my old systems from many years ago. Unfortunately I and many people can't use this ingenious GraalVM work of yours, because the basic architecture of the integration in openHab is still in flux after now three years and even the contradictory documentation drives me crazy - and I'm tough. For an own clean implementation my dusty Java architecture knowledge is not enough. Get rid of the nashorn-engine and agree on fixed rules for the connection and include the GraalVM as standard. The resentment of some users will surely come, be limited and quickly subside. The gratitude of the newcomers should outweigh by orders of magnitude. The once rewrite of no matter how many scripts is not the problem! I think it's not just me. The horror is the constant adjustment with several systems because constantly something changes or something is missing or there it is missing, then it is again outdated in the next version, because nashorn this or that etc.. Shorten the changeover. Finally make a cut with the old nashorn-engine, please. Preferably today and here. Yes, I know, that is much too heretical... ;-) Please do not be angry at my so clear words.... I hope I can dedicate myself as soon as possible again much more with the further development of openHab and make a contribution. But life is sometimes curious. |
Agreed , this sounds very reasonable and what i was thinking as well.
I don't think we have to go this far, i would rather us keep it there until we decide Nashorn is no longer included , I know there will be some migration woes as people need to change things like I also want to clarify something , there was a mention of only injecting when using the openHAB UI? I'm not clear if thats actually the intention, but If that is the case, i think that would be a huge mistake. For one, it means one could not freely copy scripts to and from the UI to files, leading to all sorts of confusion. Also, while i don't think injecting a global object to interact with openHAb as "magic" by any means, doing so when only using the OH UI would be magic, as i would have a hard time explaining why that choice was made . I think we need our engine to behave one way, consistently, no matter how you configure openHAB (UI, files, or both). If this was not the intent, then please disregard this whole paragraph :-) Not to jump the gun here, but if we can agree on using |
My suggestion is to never inject, rather to allow the UI to provide an explicit header which declares all the things that a script writer is likely to need. When viewing the script in the UI, this should be present. Something that I'd also like to point out regarding a 'global namespace object': When I was porting much existing JS to the new engine (I think much written by @lewie - thanks very much), I found it much easier to not use a global object. The porting (or maybe: updating) process goes as follows:
This did not require going to each reference of an openHAB object and modifying it to reference via some namespace. This could absolutely be done, but it's simpler to not do it (and leaves more concise code) because you don't need to find each reference. |
|
Strange, i'm playing with it now to see if i can reproduce
Hmm, you have a start level of 20 as a trigger, so i would assume it should have started earlier and complained? I agree both errors smell like Nashorn is handling the scripts. I'm not entirely clear how or when JSScripting becomes the default provider for JS scripts, i'm assuming its when its loaded (in osgi) and registers itself as a handler for the js mime type, but i guess that means Nashorn could have a window of opportunity to load scripts first on bootup? I cannot yet reproduce on my system (i also added a start level 20 rule), not sure if its a race condition ? |
|
I have seen this too, however only when installing the new JS plugin, not once it's already installed and the system is started. However I have no rules at startlevel 20, although I did at startlevel 25. |
|
Hi, First of all, I would like to express my big compliments and thank you for your work here. Some background to not blow up this hereI've been there since OH2, but haven't had any rules so far, the first rules I wanted to make I kind of didn't like it. To cut a long story short, with ES2021 I wanted to start with rules, because I liked that best and what I know best from my dotnet world (or my Java times), for example. I have update now to this addon version #2433 (comment) and also updated My questions
# like @rkoshak I am using the logger, but i see differences: var logger = log("org.openhab.ruleNew." + this.ruleUID);
var loggerOld = Java.type("org.slf4j.LoggerFactory").getLogger("org.openhab.rule." + this.ruleUID);logger.warn("Hello world! {}", item.state);
loggerOld.warn("Hello world Old! {}", item.state);Outputs in logger.info("Hello world! {}", item.state);
loggerOld.info("Hello world Old! {}", item.state);Outputs: so only the old logger output, not the new one. Some remarks to the cache (#2433 (comment)) let counter = cache.get(this.ruleUID + "counter");
if(counter == null){
counter = {times: 0};
cache.put(this.ruleUID + "counter", counter);
}
console.log("Count",counter.times++);to prefix the |
Install the version from a couple posts later and you don't need to install the helper library using NPM. You also don't need to import it in UI rules.
You can remove it if you install that later verison.
All rules will be displayed in the UI. But if they are defined in .js files you will only be able to modify them in the .js file.
I believe personal libraries will go in automation/lib/js/personal. Libraries installed from npm will be installed to automation/lib/js/personal/node_modules.
That one is probably going to take some time to fix. It requires a significant change in how UI rules are executed. For now you cannot use
I think
I think I mentioned this above in this thread.
One may not always want to make it unique to the rule. The cache will also be a way to share data (e.g. timers) between rules or between script actions and script conditions of the same rule. I don't have a problem with showing this as a way to make a key unique but I don't want to show it as the only way to define a key. Also, I suspect that, as with event, data passed from a calling rule and others, |
If it's done right and the types differ AFAIK there is nothing to do in the UI.
Apparently using AFAIK parameters don't need to be registered so it might make sense to use those to differ between GraalVM JS and Nashorn JS (as in, |
|
I found another difference in behavior. If a timer expires and inside the lambda executed by the timer that timer is cancelled an error is generated with a It's not clear if this is JSScripting specific or not but given it's a PolyglotExcepion I'd guess it is. |
|
To provide a bit more info I changed the start level for the rule I posted above to 100 and restarted openHAB. The following is what I see in the logs during startup (redacted to just show rules related stuff. From this point onward I get an error every time a rule runs. The rule I posted above that complained about Once I navigate to each rule and just click "Save" the errors go away. The two things I notice are that it takes a bit before the errors start to list the rule's UID. At first it just errors without the reference. The second is that there are a couple of rules that work just fine. |
Out of curiosity, whats the use case from canceling a timer from inside its callback? Since timers are one shot deals, canceling after it's executed its callback would not seem to do anything? I saw this error as well when implementing the setInterval support, which turns the timer into a recurring timer, but was I trying to reuse the same timer object and cancel future runs from happening, you can see that code here https://github.com/openhab/openhab-addons/blob/d60e0715e2fac5a2765835db860236a34b2c2af2/bundles/org.openhab.automation.jsscripting/src/main/resources/node_modules/%40jsscripting-globals.js#L173.
This is so odd, i'm not sure why your system is behaving so differently. Can you check to make sure there are not multiple versions of the bundle loaded ? |
Thanks! I was thinking that noone ever saw it so didn't put much thought into it, but then realised that it will show up in the UI. I think that the best option would be to use: |
@rkoshak the current proposal is that all modules will go into |
|
@digitaldan you may want to take a look
|
In this case it's a bit of an M. C. Escher self referential situation. In the time of day rule above I'm using TimerMgr which handles all the book keeping associated with Timers based on the states of a bunch of DateTime Items. That rule also gets triggered when ever one of those DateTime Items change state. These DateTime Items might change state in mass around midnight causing the rule to be triggered a bunch of times around midnight. So I schedule a little debounce timer in that rule so it doesn't actually do anything until a minute after the last change to the DateTime Items. And since I have TimerMgr there, I use TimerMgr to schedule and manage that timer too. The problem comes in because the first thing I need to do is cancel any already existing Timers before creating new ones. So in the body of the debounce Timer I call Ultimately the use case was that I was lazy and should probably have implemented the debounce timer separate from the DateTime timers. But it's kind of the same situation you describe when you saw the error too. I don't think this will be a general error a lot of people will see but I have seen a large number of users (often beginners) try to cancel a Timer inside a Timer. Until now that's not caused a problem.
As long as it's documented I don't really care where we need to put stuff. I was just going on the instructions for how to install the helper library with npm prior to them being included in the bundle.. I've always thought the automation folder was kind of a mess the way it's organized and never understood why the scripts and modules were kept in widely separate folders under automation. It always made sense to me that each language would have it's own folder under automation (or maybe we don't even need automation at all) and how it's organized after that be language specific. But when I brought that up way back when I was told it was too late. I'm happy to see us getting back to that.
Just based solely on the start levels shown in the UI, the rules are loaded at level 40. But 40 is the earliest system trigger we can choose so one wonders what's executing them between runlevel 40 and 50 if the rule engine isn't started until level 50.
I only see the one JSScripting bundle installed and active. |
|
@jpg0 so i have your changes in core and the binding installed, script files work great (i left some comments on the PR about creating the script dirs on activate). The problem now is that the Graal Script engine does not seem to be registering as a script action module type which the UI pulls from The rest endpoint i mentioned contains this in regards to the current script engines: {
"inputs": [],
"outputs": [
{
"name": "result",
"type": "java.lang.Object",
"label": "result",
"description": "the script result"
}
],
"uid": "script.ScriptAction",
"visibility": "VISIBLE",
"tags": [],
"label": "execute a given script",
"description": "Allows the execution of a user-defined script.",
"configDescriptions": [
{
"description": "the scripting language used",
"label": "Script Type",
"name": "type",
"required": true,
"type": "TEXT",
"readOnly": false,
"multiple": false,
"advanced": false,
"verify": false,
"limitToOptions": true,
"options": [
{
"label": "ECMAScript (ECMAScript 262 Edition 11)",
"value": "application/javascript"
},
{
"label": "Rule DSL (v1)",
"value": "application/vnd.openhab.dsl.rule"
}
],
"filterCriteria": []
} |
|
I ran into a problem with interacting with Item metadata. We need to pull the MetadataRegistry from OSGI. I submitted a PR with the fix (it's a one liner). |
|
So I was not correct with my last comment, JSSCripting looks loaded in the UI, but nashorn is not, also Nashorn is still executing UI rules regardless. So will investigate later tonight . |
|
@jpg0 does this look like the core is using what the actual script engine reports on mime types it supports VS what we advertise in the GraalJSScriptEngineFactory ? So our custom mime type is not being used? |
|
Yep, if i change that to use the openHAB ScriptEngineFactory instead, it works!
|
|
See #2595 for a fix for 👆 |
|
@digitaldan great, thanks! |
|
@rkoshak i think the changes in
Will solve the startup problem with scripts that you are seeing as GraalVM scripts in the UI will be paired to the new mime type. |
|
This is just a little note to mention that I've made all of my rule templates posted to the Marketplace compatible with both Nashorn and JSScripting. Eventually I'll remove support for Nashorn so I can take advantage of the Helper Library in JSScripting but in the mean time it should help to have them work with both during the transition time. In general it's mostly just a case of adding However, checking types works differently between the two. In Nashorn I can use One other gotcha is Nashorn is a little nicer about being able to automatically convert States and Commands to Strings when calling sendCommand/postUpdate but JSScripting flat out refuses. So there were cases I was lazy that needed to be updated to explicitly use the String. Finally, there is the timer cancel issue that I brought up earlier that probably won't impact anyone else but me which is easy enough to fix by checking for both hasTerminated as well as isRunning. Once the three PRs are merged and the startup stuff seems to be fixed and the docs are beefed up a bit my plan is to mess around with the edges of the helper library looking for errors (e.g. creating Items, creating Rules from UI Rules (I have a use case for this one actually in a rule template), etc.). Then I think I'll propose adding some of my rules_tools libraries to the Helper Library. I agree it would be better to make them a part of core and plan on doing that eventually. But in the mean time I think they are useful enough to greatly aid users now and would rather get them to users right away instead of making them wait until I have the time to figure out how to make them part of core. In my rules at least, I hardly have a rule that doesn't use TimerMgr, RateLimit, Gatekeeper, or Delay and using them saves dozens of lines of code per rule, sometimes more. |
|
This issue has been mentioned on openHAB Community. There might be relevant details there: |
|
I'm inclined to close this issue now with the 3.2 release. However, I just want to mention that if you forget to remove the old add-on jar file from add-ons prior to upgrading to OH 3.2, some weirdness will occur, including that it will only show the new ECMA as an option in the UI. Clearing the cache seemed to do the trick. But with the PR that makes it so we can run Nashorn and JSScripting side-by-side I think we can call this issue fixed. Thanks a lot for all the wonderful work! The future looks bright! |
|
Yes, thanks everyone, it was awesome that we were able to move quickly and get this into 3.2! |
|
https://www.heise.de/news/Smart-Home-openHAB-3-2-erweitert-die-JavaScript-Integration-6301578.html I would like to thank you very much for this great work!!!! |
|
Special thanks to you @digitaldan; things moved very quickly with your help! |
|
This issue has been mentioned on openHAB Community. There might be relevant details there: https://community.openhab.org/t/ecmascript-edition-11-error-no-globals-like-items-populated/130532/1 |
Is there a way to have Nashorn and the GraalVM JavaScript add-on live side by side? Now it appears to completely replace Nashorn which breaks things.
I've had to deal with two problems in two days caused by the fact that GraalVM JavaScript is installed and then the users are trying to use a Nashorn rule or to use Blockly.
If it is not possible, I'll move this issue over to the webuis repo to have Blockly removed as an option to build rules when the add-on is installed. However, that's not a very satisfying approach. Installing an add-on shouldn't replace or break stuff that is built into openHAB be default in my opinion.
The text was updated successfully, but these errors were encountered: