Idea: Bundle Yarn - a dialogue editor for interactive stories

[blurymind]

Description

Stories,often told through text are a very common thing in games. Especially indie games- be it rpg, action rpg, visual novels or even simple platformers. This is so common that every time I start writing my own dialogue/story system- it feels like I am reinventing the wheel rather than making my game fun.

Some engines are built for telling stories- the rpg maker engines (http://www.rpgmakerweb.com/) for example are very popular for that reason. But so is renpy (https://www.renpy.org/) and others like twinery (http://twinery.org/)
They all have very vibrant communities, full of creative people with stories to tell

Looking at gdevelop- its a very good and user friendly engine- but it doesnt have a strong storytelling toolset like the others listed. The user has to do a lot of work to create and manage a dialogue tree system and the editor has no tools to write non linear stories

Solution suggested

A few years ago I discovered YARN - a story system and editor already used on some commercially successful games:
https://github.com/InfiniteAmmoInc/Yarn
The yarn editor is all web based, very tiny (500 kb) and super easy to use. Its written in javascript/nodejs and the story files it outputs are simple json format.
Now the editor hasnt seen much updates officially, but the indie game community has been quietly using it on more and more games-focusing more on developing more implementations of the story parser.

While the official yarn story parser was written for the Unity3d engine, recently I found out that somebody has rewritten it in js and released it as an npm package here:
https://www.npmjs.com/package/bondage

There is already an npm library that interprets yarn story files that we can use - bondage.js. I was able to succesfully integrate it with yarn itself and add the capability to test the story
https://github.com/jhayley/bondage.js

The library takes care of remembering user made choices (storing and checking yarn variables without leaving yarn or having to write event sheet logic in gdevelop)
I added text scrolling and support for bbcode ontop of it. My code is embeddable, so in theory should be easy to integrate into gdevelop as well. The only bulk of the job is exposing all of this to gdevelop as an object and event sheet actions/conditions to trigger it.

My suggestion is to bundle Yarn editor and the bondage.js yarn story parser with gdevelop. It will make creation of story based games much easier on gdevelop.

The reason I am suggesting it is that I already had looked into its source code and ported Yarn to electron for another personal project that integrates it with three.js :p

The question is - is there any interest in having Yarn bundled with gdevelop? I know it's kind of becoming my shtick to glue open source projects recently, but after playing some of the games made with Yarn editor and using it myself - I definitely see a lot of value in it. It might help us attract some twinery users to gdevelop imo

What would be hard and new for me in the case is implementing a yarn parser behavior for gdevelop - to print out text on the screen. I am not sure how this is done and would appreciate any examples or code that might help me understand this :)
Getting yarn editor itself inside gdevelop and have it communicate with gdevelop would be easy- as I already know how to do that.

In any case, are you interested in this? Without approval from @4ian I wont even attempt it

[4ian]

What would be hard and new for me in the case is implementing a yarn parser behavior for gdevelop - to print out text on the screen. I am not sure how this is done and would appreciate any examples or code that might help me understand this :)

I guess that if you can get the parsing done to JSON, it's easy then to manipulate this in Javascript and write a few conditions/actions to expose it to GDevelop :)

The main thing to consider when wanting to "bundle" a software or feature like this is:

• What is the number of users interested? Is it worth the time passed on it?
• How hard will it be later to migrate from this to a better solution if something new is created?
• How hard will it be later to fix bugs arising from using this? (because there will be bugs).
• If Yarn is not maintained, who will fix the bugs in it? If we or I have to do it, then it means that it's less time for GDevelop.
• Similar question: what is the cost of future maintenance? How many hours to make sure that GDevelop 6 will be able to run games using Yarn?
• Who will write the docs for using this and make tutorials for it? GDevelop is already missing tutorials :)

So enlarging the surface covered by a software like GDevelop is always tempting, but here I'm afraid that it will make things harder to maintain, especially if the other project is not evolving or used a lot.

[4ian]

Feel free to make me reconsider things though :) But I think other part of GDevelop would need some love first.

[ddabrahim]

I can see the potential in this for making none linear dialogues but in case it is no longer maintained I don't think it worth it. It might be easy to hook it up to GD and read the texts from it, but in my opinion each scene should have it own Yarn editor so we can keep the dialogues separated for each scene, then we definitely need some dialogue events to kick off and switch between dialogues using a node name. For example I could have a dialogue to talk with Mark about picking the carrots then an other set of dialogue to talk to Mary about to feed the cows. When I talk to Mark or Mary, I need to be able to set which dialogue to start by entering the node name or something. Then I also need to be able to pass information to Yarn, because Mark want me to pick 10 carrots, but what if I have only 6 and I want a different Node to be triggered if I never talked to Mark, if I have 10 carrots or in case I talked to Mark but I have less than 10. The dialogue system need to process that and switch to the corresponding node. Finally, it should also display the dialogue and options to choose from, using the Text object would make it even more work. I think it should have it own text object with this node system in mind to display the text and options to choose from.

Wondering if it would be the same effort to make one from scratch that integrates perfectly with the event system and create the dialogues directly in the events editor similar to how it works in RPG Maker.

[blurymind]

Hi, It is maintained, but not as often as Piskel is. The last update to the codebase was 2 months ago.

I am actually thinking of making my own fork, since it required some changes to get it to work with electron. The main version was designed for nwjs, so I amended some code to port it over to electron on my build - get it to load and save files. I will try to get them to migrate the app to electron and see how that goes.

I also added a new feature to make it easier to create links to other nodes:

I added a few new features to the js parser to help my project a bit (bondage.js):
hylyh/bondage.js#33
I got it to work in my personal project - yarn has a new button that saves and closes- sending the data to my app as an object to be parsed.

So I am at a point where I have some understanding of both and am genuinely interested in using them both beyond my personal project. If you are interested in allowing to bundle it, I am vouching to help maintain the electron fork of yarn- hopefuly geting the original developer to migrate to it as well.
I will release my fork on github in the weekend hopefully.

In the last 10 years there has been no alternative to Yarn that is as good and simple as Yarn, while also being web based, open source and battle tested on released games.

The goal with this is to make gdevelop a better tool for storytellers - on par with game maker and renpy

Alot of the work to get it to work with Gdevelop has been done and tested on my project that is also electron based, but getting it to be userfriendly on gdevelop consists of also creating a new "Story scroll" behavior that prints and advances text from the yarn object, using bondage.js

@ddabrahim you can check and set yarn story variables inside yarn- that makes it very easy to set what you require straight in it.
You can also register your own commands with bondage.js that are gdevelop specific - so getting the two to communicate both way is actually quite easy. Getting gdevelop to set yarn story variables and also yarn to check gdevelop game variables. I got all of that functionality on my electron project already.

[Lizard-13]

I'm not a dialogs guy, but I would really like to know how GD events would look using this feature, which kind of conditions / actions are we talking about?

[ddabrahim]

In RPG Maker we can make dialogues using simple if-else statements right in the event editor but in RPG Maker each object has it own event sheet which make it more simple to keep it organised. in GD we can use external events in a similar fashion for dialogues, at the moment I'm not convinced if Yarn on it own would make a huge difference to be honest, except in case each scene would have it own instance of yarn.

One way I could imagine the events with Yarn is to simply have an action to start a dialogue and pass a node name to it, For example StartDialogue("Mark") then it would get the instance of Yarn for the current scene and look for the Node "Mark". Once found the node, Yarn should compare the story variables then to see where are we in the story and kick off the corresponding node in the node tree of "Mark" and return the text. But, in my opinion it should not only return the text that we can display but it should also return the options and we should have a new text object dedicated for dialogues to do less condition checking manually and let the dialogue system do the heavy lifting. We would just write the dialogues in yarn, pick options, set values and the dialogue system should take care of the rest.

[blurymind]

@Lizard-13
The way I am thinking of approaching this from end user point of design is to have a "Yarn Story" game object that serves to render text - kind of like a smart resizable text label.
I would probably copy and extend the text object in gdevelop to have a text object that wraps the text, but it also has some extra functionality and variables ontop. With its contents dictated by bondage.js

The text contents - will be replaced by the yarn data resource's text during runtime. So they are updated by bondage.js technically- having the text scroll out of the box,

All of the below could be set/get during runtime (via event sheet logic) or within the properties editor:

• Yarn Data - (drop down list to select yarn data resource) - this will let you set an existing yarn data to be used, regardless of scene. Yarn data resources are treated like a global resource. You can delete or add new ones. Edit them with yarn editor. Load new ones via yarn json files- which yarn can load and save locally or pass as a data resource to gdevelop.
  Edit selected button below(opens Yarn editor and loads the selected yarn data resource) / remove / create new one
• Run chapter (string) - setting this string tells the object to load the node the yarn data resource you have selected above ("Start" or "myNpcStart")
• Text Scrolling speed - this controls how fast the text scrolling speed changes. You can set in the event sheet to scroll faster while holding a button - by increasing this value while a button is held
• visibility, font, color (would be cool if we can have a richTextLabel - capable of rendeing bbcode parsed to it later on)

Only for the event sheet (conditions):

• Yarn completed text scrolling (on which yarn data resource )
• Yarn reached end of Data (on which yarn data resource )
• User made a choice (on which yarn data resource)
• Get yarn variable (on which yarn data) - yarn variables are variables established in Yarn Data

Only for the event sheet (Actions):

• Advance dialogue - moves to the next line
• Pause dialogue (in case we need to pause the game)
• Set yarn variable (select which yarn data resource)
• save yarnData state (select which yarn data resource)

Basically we are exposing the functionality of bondage.js to gdevelop editor. Bondage.js is a library to communicate with and control yarn story objects.

Setting and starting it is actually very similar to what @ddabrahim is describing. Its super easy, but very powerful/flexible
Btw you can set and check variables without leaving yarn editor- without having to set them up in gdevelop.
Bondage.js does all the work of interpreting the flow for you- so no need to set up event sheet for the dialogue flow at all. Yarn editor can also parse commands to gdevelop and even call javascript functions if you want to from it like this <<myfunction(parameters)>>.

So it's quite sweet in the end - because with very little work you can get an npc talking and storing your choices in the yarn data resource itself during runtime. The event sheet in general is only needed to set up how text scrolling is handled by input, or for special cases to trigger some sprite animation

[blurymind]

Update on this:
I did some work on writing a rendering procedure for yarn stories. It uses bondage.js and a tiny bbcode parser to do it's job and is tiny. I added it to my fork of yarn itself, but it can also be very easily reused by gdevelop/other engines (tested on my personal project)

I updated the first post to add a gif of it in action with some description. It still needs some polish, but its almost there now

[ddabrahim]

It is looking promissing actually.
One thing, if we end up having this in GD, could you make it so that the yarn story is saved as a separated file for each scene? So in case we are working in a team, can one person work on the story and dialogues and others on other parts of the game and just grab the yarn story and import it in to our project?

I did actually requested recently (yesterday) to make GD projects modular, instead of having everything stored in a single json file, would be better if the content would be divided in to separated modules/files. I don't expect this to happen overnight for the existing features but would be nice if we would go in to this modular direction with any new features from now on if possible.

Thanks.

[4ian]

A few idea/notes:

• I'm not sure if making a new objects that is a copy of the text object is a good idea, as it would be extra work to make it both for Pixi.js and potentially Cocos2d-JS renderers.
• Instead, it could be interesting to see if we have a way to "plug" the flow into a text object. A bit like a behavior.
• Separate files are the way to go. It was an option in GD4 ("Folder project") that could be added back to GD5 in a similar way (each scene, external events and external layout would be saved in different json files, possibly other things too).
  (Technically, the idea is that the software still save ("serialize") in memory the whole project as usual, but then split the result in different files, and reconstruct the other way the project when opening).

[blurymind]

@4ian That is also an option :) We can just use the text object.
The only reason I suggested to extend it is that I figured how to do scrolling with bbcode on my renderer

But having a richtext label should be its own object imo- if that ever becomes available in gdevelop.

[blurymind]

If you guys are interested I can give it a go and make some pull requests - to integrate my electron ported yarn with new features and bondage.js into gdevelop - similar to how we did with piskel - but its actually much simpler this time.
As noted bondage.js already takes care of most of the parsing functionality and story conditions checking- no need to use event sheets to drive the story- only to drive what happens on the screen as the text scrolls (cut scenes/player controller pausing during chat).
I will need help to expose it to the editor though. Is it possible to use javascript to extend the gdevelop runtime in a plugin way? If so- is there an example to look at somewhere?

[blurymind]

@4ian on game saving - bondage.js can interpret story data from standard json files or straight from an array. It has an online demo here:
http://hayley.zone/bondage.js/

example files here
https://github.com/jhayley/bondage.js/tree/master/tests/yarn_files

It keeps track of user choices and that can also be exported/saved

[4ian]

But having a richtext label should be its own object imo- if that ever becomes available in gdevelop.

Yes the idea is to favor small reusable objects over a large "do-it-all" yarn specific objects that would be a pain to maintain and could not be reused for other creative things :)
A richtext label would there be a solution.

Before going any further, please remember that all of this is experimental and for now I don't plan to maintain it in the long term. This should still be considered as a third party extension should it be created (contrary to particles or Piskel, which I consider "officially" part of GD5 and maintained by me if necessary).
Just don't want to waste your time even if trying is always nice to see what you can do :)

I will need help to expose it to the editor though. Is it possible to use javascript to extend the gdevelop runtime in a plugin way? If so- is there an example to look at somewhere?

For now, you would need to add a folder with some C++ sources files in Extensions, expose it here and compile it (https://github.com/4ian/GDevelop.js).
It's not well documented. I suggest that you go and hack things into the runtime (you can edit GDJS/Runtime folder and launch npm run import-resources in newIDE/app folder to copy your changes into GDevelop 5 (then launch the preview).
(If you want to iterate faster, you can also create a script to copy to GDJS/Runtime into the temporary folder where previews are created).

By "hacking" things into the runtime, I mean use an existing objects and tweak/modify it (maybe the text object ;)), and same for conditions/actions (try to find some that are not used a lot to do your tests).

@4ian on game saving - bondage.js can interpret story data from standard json files or straight from an array. It has an online demo here:

use json - avoid specific format as much as possible

[blurymind]

@4ian another option is to use javascript straight in a gdevelop project and keep it separate- expose it that way.
I wish gdevelop had a pure js addon system that did not require dealing with any c++ or recompiling

If that is planned, I will wait for it, before trying anything

[4ian]

This is not planned for now as there was not much demand. I do agree that it would be great still :)
But the current way is the easiest it was to transition from GD4 to GD5 while still maintaining full compatibility and without rewriting all extensions (which would be a huge waste of time).

So you can start with Javascript straight into a GDevelop project. Who knows, maybe we can innovate and see later how to package custom Javascript as modular "lightweight" extensions providing conditions/actions or even objects/behaviors.
A long term idea that I'd like to explore is to create more modularity by allowing such extensions to be created using JS or even events themselves.

[blurymind]

@4ian perhaps we should check on the community forum if people would like to see it in gdevelop?
I personally think that it will give the engine an edge over the competition - taking more advantage of its open source nature, but it is perhaps out of scope for the overal design.

I will continue to develop it for my personal project and contribute to yarn's development anyways :)
The embedding and implementation is almost complete here. Seems to work ok , but will need more testing

[ddabrahim]

A dialogue system if done right would be definitely welcomed by the community, the trouble is, as long it doesn't cost money and got it for free I don't think anyone would say no for it but in case you want feedback I think you should try the forum, Not sure how many users are on GitHub. Considering GDevelop targeting people who don't know or don't want to code, my guess is that you find more users on the forum than here.

Would be nice to see indeed a plugin system that would allow us to use JSON and JS to add new events, objects and behaviours maybe even expressions and even modify existing ones without need to recompile anything. This way, if people would like to play around with things don't need to worry about that the feature they are working on never going to make it in to a public release. Also in case multiple people would like to make their own implementation of the same feature, can do and people can choose which one is better. People could also make money by selling their plugins which would definitely also help to bring more devs to GD.

An other note for the end, GD4 compatibility should be not a priority any more in my opinion. We have technically sacrificed years of work made by the community in forms of tutorials, documentations and examples in favour of a new IDE, new workflow, new design and better Linux and MacOS compatibility, Makes no sense in my opinion to try to keep compatibility between the two version under the hood.
I also don't really understand the point of maintaining two JavaScript codebase, Pixi.js and Cocos2D.js. Pixi.js become very popular, used by many companies in commercial projects, I would recommend to drop Cocos2D too. I see no benefit in Cocos.

Anyway, Good luck I hope for the best.

[blurymind]

I think that a big part of both construct2 and clickteam fusion's success is due to the fact they have addon systems that are easy to develop for. Hell, their community could integrate a whole 3d engine in both - and those are proprietary apps

[4ian]

Beware of the "feature creep", which is very easy to get as a developer or contributor: it means to always being added new features for the sake of "this would be nice if gdevelop could do xxx...".

Don't be wrong here: the only thing GDevelop really need is new tutorials.
With more tutorials, people would watch/read them, be more proficient with the software, and lots of tutorials on Youtube or 3rd party website would offer better visibility to the game engine.

(edit: I'm not even sure we can consider the MMF extension system as being "super easy to develop". They were successful because everybody know about it - while GDevelop is not advertised a lot).

Don't get me wrong: contributions are super useful and I do hope to see again things like Piskel or all the contributions from other contributors recently. That's super cool.
But don't confuse success with lots of feature: with lots of tutorials and advertising, GDevelop is 10 times more evolved than what is needed for most people to create super nice games and make the software successful ;)

[4ian]

Also note that I say this because I was very focused when I was younger to add new features because it was cool. It was also a mistake. People do want new features, but first people need tutorials.
Even if people don't say so, they prefer a software which does a few things, but a few things well, with lots of explanations and polished features, rather than a lots of things, but without tutorials and without being polished.

Edit: the data are very clear. More than 50% of the feedback I get about GDevelop is "cool but where can I find tutorials?". More. than. 50% 😱

That's the reason behind my question at the beginning of the thread:

Who will write the docs for using this and make tutorials for it? GDevelop is already missing tutorials :)

Even for Piskel, we almost have no mention of this on the wiki. We should at least have 4 or 5 different pages telling about this feature and how to use it!! :)
I added a link on the website, but this would need a brand new page only to tell about it and how to use it.
Sometime I think I should stop the development for 2 or 3 months and only do marketing about GDevelop until I observe a substantial amount of new users :)

(again that being said, I'm super open to new features like this one, just trying to find the right balance so that we're all successful in what we're doing for GDevelop :) I also don't want to discourage you or anything - just that we are all aligned about what is the current state of GDevelop and that we don't make false assumptions about success just because we do like something)

[blurymind]

@4ian One thing I was thinking about - if this gets more interest to integrate in the newIde - is to create a new example/template project for the newIde that demonstrates how to use Yarn in the context of a gd game. I was thinking of a small rpg game room where you walk around a village and talk to the npcs - demonstrating all the features of bondage.js

Since rpg maker users are the main target demographic I want to lure to gdevelop- I will also make a video where I explain how the project is structured and how to create a new npc in the newIde - demonstrating how easy it is and how much better the story editor is.
Then it would be cool to also publish the example project with the video as a web game on itch.io, where I have an account :)
In any case that was my plan with this. Here is an example:
https://ansimuz.itch.io/tiny-rpg-forest
btw he has some awesome free assets we can use in our templates
https://ansimuz.itch.io/

I didnt think that piskel in the context of gdevelop needs a tutorial, but it does need some coverage.
When I asked gamesfromscratch to cover it, he just retweeted my post on twitter. Since he has a big audience,it would be awesome if we can get him to make some tutorials/coverage for the newIde. He has already made some for construct, fusion and the old ide.

The reason I mention the plugin system in other engines is to note that construct2 exposes the engine's ide to javascript and their addon system makes it easy to extend the engine without recompilation- thus why it has so many community made addons.

But you are right, a big part of our problem is the lack of video tutorials on youtube for the newide. We need to come up with some sort of an initiative.

One thing I would suggest is to put all demo games we have so far on itch.io - and put gdevelop's logo on the title/splash screen with a link to gdevelop.
You can also embed the newIde on itch.io.
Btw, if you are interested- I can start doing that to try and increase our presence there

[4ian]

Btw, if you are interested- I can start doing that to try and increase our presence there

Yeah sure, that's a great idea! Let me know if you need anything.

a new example/template project for the newIde that demonstrates how to use Yarn in the context of a gd game

Yes, I mean that's not even optional, it's a must have! Most people that want to try something new just launch the examples to see how it works.

Having more examples would also be an easy way to improve GDevelop audience I think :)

The reason I mention the plugin system in other engines is to note that construct2 exposes the engine's ide to javascript and their addon system makes it easy to extend the engine without recompilation- thus why it has so many community made addons.

I do agree that it's a good way to have more community made addons as it's more approachable.

[blurymind]

On the topic of examples - somebody has uploaded gdevelop examples on itch.io:
https://gametemplates.itch.io/gdevelop-examples

But there are two problems with that :

• They are not a part of an official gdevelop group
• They are not running embedded in the browser- so people can't get a quick demonstration of gdevelop's capability.

We need to create an official presence on itch.io to help increase the user base imo
Would you like me to create a gdevelop group there? That is another way I can help

[zatsme]

I would be happy to help with examples, what do you think would attract new users?

…

On Sat, 9 Jun 2018, 10:34 am Todor Imreorov, ***@***.***> wrote: On the topic of examples - somebody has uploaded gdevelop examples on itch.io: https://gametemplates.itch.io/gdevelop-examples But there are two problems with that : - They are not a part of an official gdevelop group - They are not running embedded in the browser- so people can't get a quick demonstration of gdevelop's capability. We need to create an official presence on itch.io to help increase the user base imo Would you like me to create a gdevelop group? That is another way I can help — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#511 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AQ8mi9k2iJ2cBx16KKWQraAuwQvsmYkwks5t65algaJpZM4UX10x> .

[ddabrahim]

The only thing I don't understand what sort of tutorials would help?

@4ian When you see people asking for tutorials, what sort of tutorials they would like to see? In the platformer and the tank shooter tutorial we have covered a lot.
What is exactly missing, what information?
What I see is that people are keep asking about the same things over and over again that we have already covered.

@zatsme I have shared tons of examples, no one cared to download them
https://gametemplates.itch.io/gdevelop-examples
And when I asked what examples to do, no one had an idea which means people don't need examples.
I have also shared project files of more complete game . People have no use for them unless you are ready to document it but as I mentioned all the basics already there in the platformer and tank shooter tutorial

[blurymind]

@zatsme More variety of 2d games running in the browser.
It would help if you download a free tileset from here:
https://itch.io/game-assets/free

and create a small game room, then uploaded it as a web game to itch.io with a zip file of the project of the game and a link to gdevelop in the description

[blurymind]

@ddabrahim it will help if you create an individual itch page for each example you have - embedded like this:
https://ansimuz.itch.io/tiny-rpg-forest
So people can play the example in the web browser. We need the examples to look good- so if you dont have any cool graphics in them - use free pixel art packs from itch

[ddabrahim]

@blurymind this is what I did. I was dedicated the itch.io page to GDevelop, Each example had it own page and I have also embeded the examples in to the page so people can try before download.. No one cared :( So I moved all the GD examples on to a single page, and now I decided to dedicate the pages to complete games only. But when I asked the community what sort of game they would be interested in, I got no answer...

GD community simply has no use for examples the way I see it. Tutorials, step by step explanations maybe but there is all the basics explained in the platformer and tank shooter already. I have no idea what else to do...

[4ian]

Strange, the script should copy recursively all JS files that are in GDJS/Runtimeand Extensions/.
So I would say that if you add your yarn folder to GDJS/Runtime and launch it it should be copied to C:\Users\blury\AppData\Local\Temp\preview if you run CopyRuntimeToGD.bat C:\Users\blury\AppData\Local\Temp\preview.

[blurymind]

It says
Copying GDJS and extensions runtime files (*.js) to "..\..\Binaries\Output\Release_Windows\JsPlatform\Runtime
and it does indeed copy them to that folder

But when I platest the game, the yarn folder is gone - it doesnt exist in
C:\Users\blury\AppData\Local\Temp\preview

[4ian]

Normally if you run

CopyRuntimeToGD.bat C:\Users\blury\AppData\Local\Temp\preview

This should indicate that files are copied in C:\Users\blury\AppData\Local\Temp\preview
If that's not the case, then something is wrong inside the script.

But when I platest the game, the yarn folder is gone

Yes it's normal as only the files that are needed by extensions are copied. Can you double check if you can have CopyRuntimeToGD.bat make the copy directly inside C:\Users\blury\AppData\Local\Temp\preview?

[blurymind]

Running this
CopyRuntimeToGD.bat C:\Users\todor.imreorov\AppData\Local\Temp\preview
copies the missing files:

Copying GDJS and extensions runtime files (*.js) to C:\Users\todor.imreorov\AppData\Local\Temp\preview...
29 File(s) copied
45 File(s) copied
 Copied GDJS and extensions runtime files (*.js) to 'C:\Users\todor.imreorov\AppData\Local\Temp\preview'.

Unfortunatelly when I playtest, gdevelop wipes them out, as it decides it doesnt need them

So I guess the only way to include them is by writing an extension and recompiling gdjs?

[4ian]

Unfortunatelly when I playtest, gdevelop wipes them out, as it decides it doesnt need them
So I guess the only way to include them is by writing an extension and recompiling gdjs?

No need for this. Launch the preview, launch the script and reload the game (Open Developer tools in the preview and hit Ctrl+R).

[blurymind]

@4ian it works that way - but every time I launch the game I have to open developer tools,run the script and then update? :)
This is no solution if I want to actually export the game- I need to write an extension to make it usable at all

[4ian]

@4ian it works that way - but every time I launch the game I have to open developer tools,run the script and then update? :)

Yes. Note that if you're working on JS files, you'll probably just end up launch the script, hitting Ctrl+R to see if it works, then do it again.

If you want to export the game you have to manually copy yarn folder (so won't work for automatic online builds - in which as anyway you'll need to make an extension but wait for having something that is working).

[blurymind]

Would be nice if we had some way of including js script files in the project and calling their methods inside the event sheet :D It would make the event sheet so much more hackable without need to recompiling ..

[4ian]

There is something called SourceFile in GDevelop core allowing to bundle custom JS files but it's not exposed in GDevelop 5 as it wasn't being used a lot in GD4.
Might expose it in GDevelop 5 as part of "Developer tools" that you can enable in the preferences.

[blurymind]

I guess it would be a step towards having a more accessible js extension system that doesn't require any recompilation of gdjs :)

You can already kind of access the runtime via the JS event. Add ability to include external js files and we can kind of do it in the event sheet. But the event sheet can not modify the editor itself.

Godot engine does it in a very interesting way where anything inside project/addons/myaddon is automatically detected by the engine as an addon to interpret.
Inside that folder there is a bunch of script files that can alter the editor's interface (using 'tool' key word) and describe what the addon does, with instructions to add new game objects and all. It is quite powerful and easy to use - also doesnt require recompiling godot
http://docs.godotengine.org/en/3.0/tutorials/plugins/editor/making_plugins.html

But anyways- I am going offtopic with this :)
If gdevelop had this, you will see new community made functionality so much more. We will start getting new stuff on a monthly basis! I mean look how well it is working for godot:
https://godotengine.org/asset-library/asset

[bil-ash]

@4ian That would be a quite useful feature. Specially if there are 5-6 line js functions which need to be called multiple times in a game then writing the whole thing in events editor multiple times isn't a very convenient option( I am currently facing this). Instead if there is an external js file where we write the function and just call it in javascript section in the events editor,then that is a better option.

[blurymind]

@4ian I can move this to a sepparate issue if you like the idea :)

[4ian]

@blurymind Yes please do :) Please write something quite specific in this separate issue because it's still vague for me and I'm not sure we're all talking about the same thing. Also include precise examples of what it would help to achieve.

I ask all of this to avoid feature creep - don't want to spend hours and hours working on something that will end up being too complicated and used by no one (which has been the case in the past for GDevelop 4).
The newIDE is already open-source and written in JavaScript so virtually anyone could already start creating Pull Requests to improve it. I'm not sure if yet another extension system would improve things - so that's why I'm asking you to be as precise as possible and include examples.

But the event sheet can not modify the editor itself.

Not sure if it's a problem. Events sheet are part of the game. It's not their role to be used to modify the editor. I think you're mixing your need of creating some new feature for the game with another need which is to be able to change the editor.

[blurymind]

@4ian I created it here:
#540
Hope I explain it well. Will provide more examples if required. I have actually written some addons for godot in the past and thats why I am pointing this out.

In any case, I am going to try to create a yarn extension by forking the shopify one. Hacking the index.html file is unsustainable imo

[4ian]

Ok thanks :) I've answered on the thread. Hope I don't feel too harsh when reading me - it's not the case 😇 It's just that the issue is complex and we have to be VERY precise about what is really needed to avoid losing a huge amount of time into working on an extension system that will complexify the whole software.

[blurymind]

@4ian quick question - in my extension I want to point to a json file in the project folder path. Should I use a "string" parameter of the action? :)
Can json files be treated as a type of a project resource?

Also would you like me to submit the progress I make in the form of a pull request?
From my understanding the c++ script adds the actions/conditions to gdjs, it also tells it to include a bunch of js files. In the js script I add the functionality I want, then test it after recompiling gdjs. Whenever I make changes to the extension - I recompile gdjs, then run that batch script that copies gdjs to the newide and finaly restart the newIde, then test?

Can I skip recompiling and copying If my only changes are to the js files somehow? If so , which js files should I edit for testing? Seems like a lot of steps between itteration
I started writing a test extension file, but will need to try doing the compilation stuff back home - as it seems to require a lot of work

[Lizard-13]

Compiling GDevelop.js exposes the C++ extension code in the IDE, so the IDE knows about the actions, conditions and the .js files it has to export. If you'll modify the .js files only there is no need to recompile GDevelop.js (it just stores the filenames).
Then you can change the .js files at GD/Extensions (modifications kept through app "builds"), newIDE/app/resources/GDJS/Runtime/Extensions (modifications kept through electron-app builds), newIDE/electron-app/dist/your-build-system-unpacked/resources/GDJS/Runtime/Extensions (modifications kept through preview), or on the preview folder at preview/Extensions (faster/just reload the game, but modifications get lost on each preview). I can be wrong as I don't know the entire proces, though

[blurymind]

@Lizard-13 thank you for explaining. So really recompiling gdjs is only required to register the actions/conditions in the editor? That means that after I get the c++ part right once, I no longer need to recompile it :)
This is so strange- the editor is web based, but needs c++ recompilation to get access to functionality in custom js files in its event sheet. Is that designed to keep them available in gdevelop4?

I think that we need better documentation on writing js extensions with the newide in mind, but I will try to stop requesting that and get on with getting this to work 👍

[4ian]

So really recompiling gdjs is only required to register the actions/conditions in the editor?
That means that after I get the c++ part right once, I no longer need to recompile it :)

Yes and yes :)
Extensions, still written in C++ contains declarations only of actions/conditions and the JS function names to be called.
All the implementation of these functions are done in your extension JS files. So if you don't change the action/condition/extension interface, you don't need to recompile.

[blurymind]

@4ian thank you for clarifying!
In that case there really is no need to rewrite the extension system or dramatically refactor it, just to enable a way to bypass recompiling gdjs to expose the newIde to custom actions/conditions. That is the only thing needed to close #540

But anyways, with this knowledge now, developing this as an extension doesnt seem that intimidating to me :)

[blurymind]

Some progress with this - it looks likely the electron port I made on Yarn will get merged - the reviewer has some notes that I will address tonight.

I also got contacted by an Australian game developer interested in my work on porting yarn to electron. They said that they have also been working on a tool+parser similar to Yarn - using Reactjs. I think the intent was to steer me towards their project - as its overlapping with some of my goals:
https://github.com/StirfireStudios/Jacquard
@4ian if you are interested I can cc you in the email. I did mention that my interest is to integrate it in gdevelop, but it looks like they are working with Unity/Unreal - so their tool is primarily targetting those two.

After a quick look at the source code and comits - their tool seems more advanced but less polished. Unfortunately due to its architecture - it would be harder to integrate and use in other projects.
I think of staying with Yarn for this because:

• cleaner/more polished interface design - very artist friendly
• much simpler codebase, which I understand pretty well at this point
• battle tested on succesful game projects
• has seen more contributors in the past

The part of preparing yarn to bundle with other electron software is largely solved, now what is left is to do is write the extension to parse yarn files during runtime :)

I wonder wether to keep the yarn data inside sepparate json files - to be used as a resource. Or should the yarn data be kept inside game objects. I really want to make this competitive with the popular rpg game maker engines for the storytelling aspect of it. Keeping it in external files will make games easier to translate to other languages, also allow for cleaner project tracking on github for gdevelop game projects using it

[4ian]

Hey 👋 I've put this issue on the roadmap (so that anyone can vote for it!) there: https://trello.com/c/vcRqzoGk/74-bundle-yarn-a-dialogue-editor-for-interactive-stories

I'm closing this issue as open issues on Github is now be dedicated to bugs only :) (But it's still possible to continue the discussion on this issue even if it's closed)

[blurymind]

I am getting a bit closer to making this a reality- recently added the missing parts to communicate between the yarn data and gdevelop.js to bondage.js - the parser. See:
hylyh/bondage.js#41
hylyh/bondage.js#40
hylyh/bondage.js#39

Now I need to also update the debugger I implemented in yarn-electron to display them - get that merged.

Then see how to read/write to the runner state of bondage.js - for save/load game states

After that is all done the next step is to try to write a yarn data parser for gdevelop.js that will use bondage.js

[Alyks]

I would certainly need a tutorial or more features in GDevelop to make story driven game. @blurymind could you please tell me what was the consensus on this matter? Would it be possible to have a simple system that fit in the GDevelop roadmap ? Are you making something as a tiers features maker that you would maintain ? Is GDevelop working on new features that would make it possible to develop narrative game ? Because, at this time, people have tried and it makes the event sheet a mess and it doesn't accomplish completely the goal : be able to make something like Night in the Wood with a lot of text and actions according to text conditions (click, store variable, input, character sprite animation according to line like line happy = sprite animation happy etc...). Maybe as Florian says, it just that there's no tutorial for such a game or maybe GDevelop needs some new features. Let me know your thought on this @blurymind and @4ian please :)

[blurymind]

The main element required to do this is not Yarn itself. I can add yarn's editor to gdevelop in a weekend. It's the runtime parser that parses the yarn file during the game's execution. Unity has such a parser and it's called yarn spinner. It's what was used for Night in the Woods.

Now, there is a parser written in javascript. It's a parser that I have even contributed some code to - in order to make it more flexible. I have used that parser both on a personal project and inside Yarn itself - to preview stories in Yarn-electron. It's called Bondage.js.
This is the thing I need to include with a JS extension for gdevelop- in order to be able to parse the data that yarn creates in the game. Unfortunately this can not be done within gdevelop's new behaviour extensions api. It's not allowed to include external js files in the event sheet at all, even though you can write some js. The thing is writing any JS in the embedded monaco editor does not allow you to import classes from actual js files. I think construct3's implementation of writing js allows you to do that and even store any js you write to files that are attached to the game object- very similar to all the other traditional game engines (Unity, godot, etc - they are all designed in a way where you write your code to external files and they can import other classes)
As I see it, this fully prohibits me from using anything cool that makes it worth writing any js code in the first place. I can't import bondage.js or any other cool library, so what's the point of writing js to do what can already be achieve with the event sheet.

So the only option is really to write it as a pure js extension - not using the new behaviours and in it to import bondage.js and create a wrapper for it to communicate with GD.

This is the exact same blocking issue for mod files and bundling bassoon tracker with GD. I have no way of actually treating the mod files as a new resource and read them during gameplay.

When I figure out how to do that on some weekend and have the yarn json data parsed inside a GD game, then I will bundle Yarn with GD and it will be super easy to make story driven games in GD.

To add to all this- there is another problem to overcome. Gdevelop can not treat json files as a resource. This means that all the yarn data you write will have to be stored as metadata inside project.json.
This is vastly inconvenient because one has to literally open the entire game to create translations for it or to fix some typos and gramatical errors in a dialogue tree. So really, when we edit files with yarn bundled, it should save the files inside the project folder - not inside project.json. It should treat them as resources, so the user wouldnt have to make boilerplate event logic to load them every time into the yarn parsing behaviour. This is not something you have to do in rpg maker for example.

Until then, you have to do it with what's available. You can still make a story type game in GD, you will just have to do it from scratch with the event sheet.
There are some hurdles to overcome to do this nicely

[4ian]

@blurymind You can still write a classic JavaScript extension and include the library file right?

For JSON, it's true that it's hard to add them in the file resources, but if you work around this (by for example, pasting the JSON in a JS event to use it), I'm happy to help adding some capabilities to avoid this work around. :)
It's just that I have so much to do lately that I prefer to do things if I'm 100% sure about how they are going to be used, rather than a feature that turns out not to be what we exactly need and will bloat GDevelop for the rest of its days (GD4 really had too much of these :)).

[Alyks]

Hi. @4ian I aggree, a feature must be added if it used by many. As a user i love GDevelop for its approach on game development but when it comes to make dialogs for story driven narrative game with conditions, variables and actions based on user clicks to display lines that condition the display of sprites and events before, during and after a line, like some narrative game engine do, it's very difficult and the event sheet can become a mess for a complete narrative game. I think the work around for this is to produce an example of a powerful dialog system with new GD core features if necessary working along with an extension if needed to make something working for any dialog based game/scene like adventure game, rpg game, visual novel games etc... But i think a powerful dialog system will be a great attractive feature for people like me who want to add complex narrative scenes in their game.

Some one helped me on the french community forum with a dialog example. If @4ian could check here. It does basically a small percentage of what i need but it seems to me very complex to work with for a entire game. In the example the condition for the dialog to appear is when the player collide with a png, but sometime you just want the condition to be a click on space key, and actions should be fired line after line like following a series of line > actions > line > action > line etc.... So, that was just to say that if @blurymind solution can improve productivity for dialog based games on GD, it would be so cool !

[blurymind]

@Alyks thank you for sharing that with us. I will have a look :)

@4ian I was thinking of storing them as metadata- similar to piskel. I am not sure how exactly it will work yet. Been looking for inspiration from other engines that are focused on story based games