RFC 0008: Future of the Back-Office Front-End

[lssweatherhead]

See RFC document (in this PR): 0000-backoffice-future.md

Summary

With this RFC we are highlighting priorities for making new decisions for the future of the Umbraco CMS front-end project. This document outlines several areas that we believe should be part of the discussions which can then lead to decisions on technology, implementation and roll-out strategies.

[Migaroez]

Below are some points that were discussed during the open circle session led by pete at cg19. I have tried to link em as best I could to the existing points.

3.1 There doesn't seem to be a code map of where which dependencies/directives are used in the code base. If this existed and somebody updated a service, directive, controller,... they could see in which places in the back-office it is actually used and test all of these areas.
It would also help the project with the long term changes as it would show which pieces of code have a high impact due to it being used in many different places.

We proposed 2 ways this can be done.

• add a gulp task that adds identification ids so that when you look at the compiled output, you can quickly see which source files were used to build the view you are in. (Need more info from Pete about this)
• use the upcoming documentation tools to enhance the auto generated docs with custom documentation that holds this kind of data.
• This would also help with 5.1

7 (new) There doesn't seem to be a product owner

• Make a seperate RFC to ask the community what they expect of this role.
• After talking to (front-end) Niels, product owner might not be the right term. But it feels like we need a spoc (one or more people, like the PR team). That (amongst other things) takes final decisions and has a good understanding and overview on everything relating to the back office-front end.

2.1 Strong typing

• This seems to be a man hour problem. The code base is big and the immediate payoff seems small because it might be wasted time when in x amount of time the whole framework is going to be replaced anyway
• It is for the most part a low complexity issue, which can be further reduced to monkey work by tightening the scope and giving clear instructions on how to complete this task. Since the output with and without strong typing should in most cases be the same.
• It is also a good jumpboard for a framework change as it will be clear which data models are being used, making the conversion to new patterns easier.
• It can be concluded from the above points that this is a candidate for a high priority action that can be delegated to the community.
• For the more complex scenarios, it was suggested to put a test plan in place using Sellinium and Ghostinspector

[anthonydotnet]

I personally don't believe strong-typing spaghetti code is worth it. The problem is the architecture. Strong-typing a bad architecture can actually shackle future development.

IMO the best way to get the architecture in shape is to wrap the spaghetti in SOLID decoupled strongly typed classes, and add unit testing to those classes. So you are only strongly typing "good" code.

You may then deal with the spaghetti in a calmer way.

[dampee]

Accessibility is mentioned only as a part of the user experience. I think that is something worth to have a separate section. And we should also mention we need at least an AA level as discussed during open space by @MikeMasey

[nathanwoulfe]

Additional post-CG19 bit from the open spaces - the actual folder/file structure of the project should be reviewed and improved. Given the build task largely doesn't care where it finds the assets, this is a low-risk activity, but would reduce complexity (gulp task does reference paths, but these are high-level so moving within these would be fine, new folders would require task changes).

It's difficult to navigate the project when we have lovely paths like common/directives/components/overlays and views/common/overlays 😃

Any substantive changes to existing code should be moving as much as possible to being framework agnostic, to clean the update path (where ever that path might actually lead) => think consistent patterns, ES6, typed classes etc

[PeteDuncanson]

I'm in the middle of writing a longer reply to this but thought I'd best just pull some of the existing thoughts, content and issues together now for others to read in the meantime. Been thinking about this one a long while so there is quite a bit out there already:

Related issues from the old tracker (now over a year old) https://issues.umbraco.org/issue/U4-11042

• add a gulp task that adds identification ids so that when you look at the compiled output, you can quickly see which source files were used to build the view you are in. (Need more info from Pete about this)

See more details of this idea here: umbraco/Umbraco-CMS#5047

Converting the existing code base to use TypeScript (and reasons why we might want to): umbraco/Umbraco-CMS#5215

Getting "PrettierJS" in the build chain so we have a consistent way of formating our JS: umbraco/Umbraco-CMS#5211

Agree a coding style for AngularJS 1.7 moving forward: umbraco/Umbraco-CMS#4913

Tidy up the many free standing controllers by pulling them into directives: umbraco/Umbraco-CMS#4875

Plenty more to come. I'm trying to put together a "battle plan" thats pragmatic and achievable that will get us in a place where we could then enjoy the tidy code base for ease of bug fixing, development and have a clear view of what the code base looks like. I hope that code base can then "hold the fort" while future framework choices etc. can be made with a clear head and a clearer view of what it needs to replace. In the meantime, a better code base for all for as long as we need it. That is the short version anyway.

[Migaroez]

I'm trying to put together a "battle plan" thats pragmatic and achievable that will get us in a place where we could then enjoy the tidy code base for ease of bug fixing, development and have a clear view of what the code base looks like. I hope that code base can then "hold the fort" while future framework choices etc. can be made with a clear head and a clearer view of what it needs to replace. In the meantime, a better code base for all for as long as we need it. That is the short version anyway.

In my opinion this is the way to go forward, emphasis on the reasons why. How is up to people with more experience/knowledge than me.

[nielslyngsoe]

Hi everyone

Thanks for bringing your concerns to the table, we will be diving into specifics on how to move forward. But to keep the conversation on our goal and purpose of changing the front-end, I would like to ask you to consider the following questions:

• Are there other key priorities that should be taken into account when considering the future of the back-office?
• Have we covered the main key areas that will improve the front-end going forwards?
• Are there other action points that you believe is more important to focus on at this stage?
• Are there action points that need to be clarified or expanded on?

— Can you elaborate on what you think is important for this project?

Thanks

[PeteDuncanson]

Are there other key priorities that should be taken into account when considering the future of the back-office?

Yes, the current code base is and has been for a long time an "additive" code base. No one does any tidy up work for fear of breaking anything. This is down to a number of factors I think we can mitigate:

• API's are not being documented enough or not doing what is needed leading to people reaching deeper into core code than they should
• The project file structure and naming is not helpful
• no one really knows the whole code base or the knock on effects of what their changes to existing code might do so they add to rather than change/clean up
• changing the code base is full of "danger" currently as refactoring is more a game of chance rather than something you can do with confidence. Breaking a release would be a bad thing so people don't do it
• showing off nice UI is more feel good and easier to shout about than cleaning up code, there is not much glory in it or twitter retweets and it really doesn't make for a good keynote slide or marketing pitch.

I'm all for looking to the future and what amazing wonders the back office might be built with in the future but in the meantime I think we can gain a lot of goodness from focusing on repaying some maintenance debt, cleaning up the code base to ease bug fixes, feature developments and simply increase ease of contributions. To do that we need to invest some time in a sane, safe and managed code clean up.

My believe that a code clean up should be the first step is based on experience in this project and numerous others we've done ourselves at Offroadcode. I think discussing a rewrite now is too soon and we would end up with a V5 again if we took it on as is. I think trying to do a prototype sprint to see what could be done right now is also too early, the API's aren't up for it. We would end up with a pretty UI that does 10% of what it should (and only the easy stuff) but we would be no further on. A hybrid approach bloats the code base and doubles the knowledge needed and puts up barriers to contribution rather than reduces it.

Magically hoping we can fix everything with "unit tests everywhere" or "make everything SOLID" are noble statements but they aren't currently achievable with the code in the current state or require specialist knowledge and epic solo efforts or death by committee style coding. I'm all for tests but currently the code isn't testable easily. We can burn a lot of time trying to do that first and then rework many of those tests as we refactor or we can make the code easier to work with first which in turn will become easier to test (removing $scope leak for instance) and have the benefit that the code base just gets better along the way. Yes it means we will have to do some manual testing while we do it but that has got us quite far so far, lets lean on it some more one more time until its in a better place to then unit test it to the eye balls.

At some point we have to clean up the mess. The only way to untangle a slinky is to sit down and spend some time untangling it.

Have we covered the main key areas that will improve the front-end going forwards?

You wouldn't be able to remake a car from a car crash (don't worry I'm not saying the code is that bad, its just a visual) so we should pull the car back into a recognisable shape so we can see what we are facing and make it easier to work on it.

In short I'd suggest this as a plan of action (each step gets us to a better code base than we started):

• Assess the size of the problem/code base (ie report on all the different controllers, services, directives, etc.) and make some notes on what we could do with each. This has already been started with this spreadsheet (https://docs.google.com/spreadsheets/d/1sEH0Cz59MDsMT8Fv-sDhli8I1KK79iyMFNCKhgBqVog/edit#gid=0) which was created as part of V8: Agree on AngularJS coding style and standards moving forward Umbraco-CMS#4913.
• Get docs working again, without these we are running blind and can't report back on what we've changed. I believe this has been fixed but there still seemed to be some uncertainties about it and where the docs should live.
• Move and rename all the directives and controllers to group them by the area or role they do rather than the type of thing they are. Login components below in /ui/login for instance. This include namespaces, class names and file names/locations.
• Add "Prettier" to the tool chain so we have a consistent coding style, its a code formatter for javascript and very quickly has become the "defacto" tool for making code bases have a standard style. Install it and forget about formatting forever.
• Add type safety to the code base. I've suggest TypeScript from Microsoft as its "won the war" for the type languages for javascript and is a revelation to use. This should be added without changing any of the logic of the existing code. We are just making it Type safe for the next step.
• Start shifting to using AngularJS v1.7 features and best practises. A lot has changed since the codebase was started in v1. We should move towards the modern best practises by slowing upgrading what we can. This will reduce boiler plate noise, get the objects into classes, reduce $scope leaks, document the API's, remove magic strings, add some tests (shock horror), move directives to components (where possible) and make the code leaner and easier to reason about. There are a number of steps we can take for this which are expertly documented in this great ebook (https://www.codelord.net/modern-angularjs-migration-guide/).
• Now the code is in a more standard and modern state now we can go back and start safely refactoring some of the horrors we found while adding all of the above. The type safety and reduce code base will greatly help us clean, move, remove and fix the code that is left.
• Write lots of tests...

Are there other action points that you believe is more important to focus on at this stage?

The "tidy up the code base" plan needs some over-sight from HQ but I think a lot of the work can be done by the community if it is scoped out nicely and in small enough units (ie convert one file to TypeScript, Write one test to do X then do a PR on that one thing). Its going to need someone (or more) to manage the number of PR's coming in (which ideally will be small and concise so easy to accept). It really needs someone inside HQ who can champion it, dare I say the back office needs a product owner who has final say, authority and clarity of vision to get this to where it needs to go in a planned, managed and achievable way. Without this we might as well not start as there will be no guarantee of getting any of this work in. Once we have community momentum on this I think it can be turned around pretty swiftly but we can't have that momentum rocked or paused as its hard to get it going and harder to restart it. Plan it out and then ask for help, show that that help is having a positive effect. Simples.

We need a way to track issues we find while doing all the above (remember our plan isn't to do a huge refactor straight away, each step we will find things that need revisiting at the right time). I'd suggest a temp separate issue tracker (currently I'm doing this via the spreadsheet mentioned above but it might not scale) but the existing Umbraco one might handle it too. Issues that we find that we want to come back to and refactor for instance need logging somewhere. We could add a comment in to the code for those areas but again that needs to be documented and agreed so we have a standard way to do that and find them again.

We ideally need some leaders on this to engage the community too, its a big job but by us all doing a few small steps of it and if we have enough of us at it we can get a lot more done than just HQ on its own ever could.

Are there action points that need to be clarified or expanded on?

I'd be tempted to time box the development so we have a target to focus on, currently its so open ended that we might end up with nothing coming forward at all.

I'd reduce the scope of this RFC to focus on how to prepare the code base to get it to a point that we can see what the lay of the land is so we can make some good informed choices about back office 2.0 currently its so board I fear we will end up "bike shedding" for months and progress little. The future rebuild can come in another future RFC. The competition in the CMS market aren't hanging around, lets keep up to date and get this thing ship shape.

[nielslyngsoe]

Hi @PeteDuncanson

Thanks for your enthusiasm and extensive thoughts on how we can make this project succeed.

The intention of this RFC is to align(begin transparent) on the goal we aim to achieve. When we have that settled (when this RFC is closed), we will be opening new RFCs regarding a concrete strategy, or several strategies, on how we potentially could reach the goal.

Therefor I will encourage you to keep these thoughts a little longer. They are good inputs but you'r one step ahead. 😉

Until that please continue thinking about this strategy. Imagine everything was done as you suggest. The next and important step would then be: How would we move on with refactoring the code?, how much time would we set aside for that part?, how would we make sure to have an aligned procedure for contributors to follow when refactoring? I even have to ask you again: why would we refactor? 😉 — now that the code is in a better state, what arguments do we have to set the time aside and motivate community members to take part? What is it that we want to achieve with the refactoring?
Cause thats what we are trying to settle on for this RFC. We need those arguments/reasons to get the time and budget set aside to be able to be able to start moving forward and manage the process.

BUT please don't answer those specific questions in this thread, thats not the purpose of this RFC. Let's try to keep it mainly regarding the arguments/reasons for change/refactoring. We are trying to gather the arguments that we later can use to lead us when making decisions on strategies, implementations and technologies.

The conversation regarding your specific strategy could for now continue here umbraco/Umbraco-CMS#4913 or on Our? Until the RFCs for it is in place.

Thanks 😀

[filipbech]

Having talked to Niels about this already, I thought I would share my thoughts inhere as well...

1. To encourage users to actually switch to the new sections (Running alongside the old, if I understand the RFC action proposal), I think it is crucial that at least a baseline of existing packages will continue to work, or that an automated update-path is available. This will probably require a minimal mock of the current angularJS backoffice to live along-side the new implementation.

2. Make sure to decouple the extension-developer api from the implementation of the backoffice, so future framework-decisions theoretically could be made in a minor-update without any breaking changes to the public api.

[PeteDuncanson]

@nielslyngsoe thanks for the feedback. I've reread your reply a few times now and reread the original RFC and I'm confused as to what response or direction you want me to go in. Can you elaborate or give me some more focus? I "think" your asking me for some more blue sky thinking over an action plan but that sort of is the opposite of what you asked of me at Code Garden hence my confusion I think ;)

My ideas above satisfy all 5 of the main motivation points in the RFC, it meets all 6 of the Opportunities listed too. Its achievable and mapped out as to how to do it. I totally agree with this motivations and opportunities. I'd have thought these are the purpose of the RFC in which case they already list what is the desired outcome brilliantly.

I do have my doubts on the proposed solution though, I fear it will lead to a half finished mishmash as the sheer scale of the job as is shows itself and then we ship with half the back office converted and we actually end up breaking several of those motivations and missing several of the opportunities (how can we be making it easier to contribute when you now have to know two different frameworks, one in a poor state of health and the glue that we've home brewed to stitch it all together).

For me whenever I take on a project I have to feel it out and scope out what needs doing. That can sometimes take a long time and multiple rounds of "clean up" of the code base before we know the best route. I'm just applying the same battle tested route to the back office rebuild. We all want it to be amazing, the question is what does that look like and how do we get there? Without any constraints on what is or isn't doable we can end up just losing a lot of time dreaming about future frameworks and border styles. At some point we need some constraints to focus on what is or isn't important/needed/doable. Personally I think its too early to be thinking about future framework choices, we don't even know what the current code base is doing so how can we recreate it (and make it even better) than the current one? We should consolidate the current code base (ideally without breaking any packages if we can), have upgrade guides and compatibility poly fills (to make name changes etc still work on older code and throw some helpful console warnings telling you what to update) where we can and document it. That can be it for v8. Then from that point you can think about what vNext is going to be like.

Ideas for the back office regardless:

I'd love to see:

• A style guide/design guide that shows you all the various components in their various states and how to use them. Currently building a nice looking back office widget in Umbraco style is tricky and a lot harder than it should be
• Safe places that you can add UI to. If I want to inject a button into the top of the editor I should be able to that safely and not have to mess with the DOM or hope HQ don't move or rename something and end up breaking my code. There should be placeholder divs with constant names that we can inject stuff into and they will always be available. By all means HQ can move the whole div but not mess with the markup or names within it. Imagine an "action button panel" that I can assign additional buttons too or if I want to but another button next to the save button. That should be doable without needing to hack the DOM as we currently do
• Standardised CSS styles which are documented. Big job this one but needed. Again we could even use some javascript compatability layers to sniff out retired classes as we change them and console warn when they are found in the back office with instructions on what they should be changed to and a link to documentation on why and how. This is all part of improving the Developer Experience (DX) and giving people trust that the work is being done with some thought and care. For this to work we need to keep track of what changes and to what. Again we are probably back to spreadsheets to track this (can't really do this just with source control) and will need to be well managed but would lead to a complete solution. Rather than just focusing on the initial task of "make the css styles sane" we instead get the complete package of "make the css styles sane, document what we changed and how to update easily while trying to ensure nothing breaks if you don't have time via a compat layer we can retire later"...phew!
• Better performance, less javascript, less css getting more done quicker and easier.
• Documented best practises. You shouldn't have to go on training to be able to code in the back office, there should be docs for that. You shouldn't have to have the Umbraco source code open to code up something for the back office. You shouldn't have to guess what is a performance bottle neck there should be some guides.

All of the above is doable within the existing framework btw. I'm a ReactJS fanboy and should be shouting from the roof tops that we should write it in that (apparently) but I don't think we are at the framework choice stage as yet. I think the existing framework can keep us going for another 4 years with ease if we need it to and if we invest in some long over due house keeping and make the back office a first class citizen again.

If I've gone off track again let me know. I'm struggling with the mixed nature of this RFC being both very open ("blue sky thinking") yet restricted ("here is our solution") at the same time. It seems to be facing two ways at once. Mix in some 10 years of Umbraco history and 20+ years development experience and you get the ramblings above. Hope they make some sense and add to the discussion.

[tobbbe]

• don't tie the backoffice frontend code to any framework (they will all get old + easier to maintain). Offer vanilla javascript pub/sub, web-apis and maybe some helpers. Let the community build framework-specific code if they want it.
• documentation
• styleguide

[liamlaverty]

Would it be worth looking into Blazor?

https://dotnet.microsoft.com/apps/aspnet/web-apps/client

[PeteDuncanson]

• Safe places that you can add UI to. If I want to inject a button into the top of the editor I should be able to that safely and not have to mess with the DOM or hope HQ don't move or rename something and end up breaking my code.

There is an issue for this already for completeness umbraco/Umbraco-CMS#3910

[techmind-git]

It has been a while since the last comment on this issue. AngularJS has LTS support until 1st July 2021 (of course it will still work after that time) but moving to something new will also take time so I'm curious how this RFC and optionally the ones resulting from this RFC will be placed in time. Just curious if there's an update on it. Thanks.

[hemraker]

@techmind-git you're absolutely right, this has been open for a very long time. There's a lot of great stuff in here. We will add a final-comment-period so we can get this one closed and digested.

I've not been a part of this RFC so far, so forgive me for being vague right now, there's a lot here to take in :) The RFC itself seems to raise a lot of questions and my immmediate impression is they won't be answered in one go.

[hemraker]

We will not be accepting this RFC, so rather than putting it in a final comment period, I'm going to close this PR.

This does NOT mean that we will not be looking at improving the backoffice frontend or addressing the choice of framework for a future version of Umbraco. I'd also like to underline that the discussion that has been had here is not in vain. It will likely be the starting point for several RFCs based on the topics discussed but we want the RFCs to be more focussed, actionable and align with the current roadmap.

Comments will still be open, even though the RFC is rejected and you are more than welcome to continue the discussion and add additional ideas.

Thanks for the contributions so far. If you have any questions don't hesitate to reach here, on twitter @hemraker or send us an email on product[at]umbraco.com.

[PeteDuncanson]

Thanks for the update Rune, 10 weeks since a HQ response before your comment, welcome back.

I can't help but feel more that a little frustrated that 5 months on from initial issues that raises the need for some direction/action on the back office (umbraco/Umbraco-CMS#4875) and we are still none the closer to having an official idea or buy in of what the future of the back office looks like as far as HQ is concerned. As a result the community has no idea what to work on, so we work on other things (I've been doing a lot of robotics with and for my boy for instance). I guess more time is needed.

[hemraker]

@PeteDuncanson thanks for the welcome back 🙌

Thanks for voicing your concern. My reason for closing the PR in its current state speaks to your point about no responses on this PR for 10 weeks and no direction/action after 5 months. An open RFC has to indicate that we are actively working on it or it is road mapped for the near future.

This RFC was one of the first created after the Codegarden retreat and the idea was to capture what was discussed at the retreat and open up the discussion to the community at large. To some extent, I do think it has served this purpose and it is an improvement over previous years, where discussions from the retreat, especially blue sky thinking like this, did not reach a wider audience.

What we have learned is that for this to be a good tool we need to be better at focusing the RFCs. The TinyMCE one is a good example of this. It is based on two previous RFCs where the discussions, prototypes, user tests etc. has helped shape the one that finally got accepted and is now being worked on. While not perfect, I think there are a lot of good takeaways from that process.

We want to be as transparent as possible with what we do, are doing and intend to do. So while I totally understand your frustration, I hope you understand the reasoning. And just to re-iterate:

• This is not intended to shut down the conversation here, comments are still open.
• It is not a value judgement on any of the specific ideas or solutions.
• It is an honest indication of where we are currently focusing our efforts.

To quote my previous response "It will likely be the starting point for several RFCs based on the topics discussed", by this I mean we will use the knowledge captured here when we embark on a project that is related to one of the many topics covered here.