Refactoring Discussion 20171Q or Gohan2

[nati]

This is an issue for noting think-aloud refactoring discussion. Could you share your pain point in current design here?
I'm wondering if this may end up start new project such as Gohan2 or some small refactoring.

[1] IAM and Policy framework
Current IAM and Policy framework is really tightly coupled with OpenStack Keystone.
This would limit the possibility of this project.
We should support more flexible IAM and policy framework.

[2] Extension & Gode generation
Personally, I feel choosing Javascript for extension was the biggest mistake I made in this project.
I don't rant the Javascript language itself, but given slowness and difficulties for debugging server side logic using go-based javascript implementation, now I believe using Go is the best way in server side.
I tried to implement some MACRO language by myself, but I felt this isn't sustainable effort.
Current API logic code looks complicated to support many use-cases.

One of my goal of this project was to avoid code generation and compiling. This really works for client and WebUI, but it may not makes sense for server side codes which have custom logic on top of what Gohan provides.

so now, I believe we should build server-side code generation tool.

[3] OpenAPI / Swagger
I'm still thinking whether we should adopt OpenAPI / Swagger or not.
We may use more tool around this eco system, but the spec don't fully provides what we need..

[4] Replacing martini to Echo
Echo looks like more high performance,

[marcin-ptaszynski]

Hi @nati,

I'm happy to give my feedback.

[2] Extension & Gode generation

I feel that YAML schema and JS are good way to quickly get into prototype, but they've proven hard to maintain consistency and correctness for a larger system. It's very easy to make an error because we lack static checks when interacting with DB and Gohan also lacks abstractions in JS library to allow validation or skip unnecessary boilerplate.

I don't agree with statement that

Go is the best way in server side

At least not with dynamic and schema driven nature of Gohan. There are a couple of approaches to build generic API servers:

• Libraries. Most lightweight. If you want to build a server, gather useful libraries (API, DB, Auth) and build one for single purpose only and don't try to be generic. Often the easiest and most elastic approach.
• Typical MVC Framework (Models expressed as native structs, logic expressed in Model methods and Controller handlers). All is statically compiled. Sometimes combined with code generation tools (generate boilerplate from schema, etc).
• Completely dynamic schema driven approach with multiple ways to script (current Gohan)

In any of above cases, we may provide it with schema. Either translate our models to some schema output or use code-gen tool to generate initial Models.

If we were to retain dynamic schemas while still requiring user to recompile his extensions, I fear we'd be inheriting the worst from both worlds. Meaning, we'd get no elasticity coming from dynamic schema (restricted to Go extensions) while receiving no benefit from Go being statically typed language, which is map[property].(typeAssertion) hell and almost no compile-time static checks when interacting with resources.

[3] OpenAPI / Swagger

I feel that we could provide better contract for UI. Single schema driving DB, API and UI view is not always the best fit. Example: required field. Does it mean property is required in UI, API or it's not NULLable in DB? Currently, it's all of them, since every layer interprets it in it's own way. It's the same with default. Another thing is that it doesn't allow to customize UI widgets. For example, https://github.com/mozilla-services/react-jsonschema-form allows such customizations on top of API jsonschema.

OpenAPI is a good tool for documentation and it's good to provide support for it as output, but I don't think it should be put in the core, since as you've said, it would limit us.

[5]?, DB handling

I'd like to suggest another nagging issue. I feel that plugging into some existing ORM would be much easier to handle than Gohan providing it's own DB layer. Gohan DB is very limited. There is no ability to express joins (unless preset in relation_property), the only filters are = and WHERE IN. There is no OR, >, <, >=, <=. There is always fallback to raw SQL queries, which is too often required.

[nati]

@marcin-ptaszynski Thank you for your feedback!

[2] Extension & Gode generation

if we allow compiling for server side code (and I feel this isn't an issue from experience with Gohan developments), code generating Typical MVC Framework would work.
and we can use go struct such as go web application development without Gohan. so we can afford go's type system.
I'm doing some POC for this approach, and I'll ask your review on that.

[3]
I agree. let's keep as-is.
We may have additional UI schema for supporting your purpose.

[5]
I agree.
It is hard to change Gohan DB code for existing app, but I think we can apply ORM for this new version.

I'm thinking apply this one
https://github.com/vattle/sqlboiler

[j-maxi]

I would be nice if we can run some integration test for API functional tests using gohan framework.
Since in gohan test_extension, AFAIK, we test handler function, but because the policy is not applied, we cannot test the exact API behavior in gohan test framework.

[thayashi]

Better late than never for comment...
IAM
One of the reason to get difficult to start/try gohan framework is keystone. No one knows about keystone except OpenStack community. Supporting OAuth or something is important to get popularity and interest from users.

UI Schema
As Marcin mentioned, the current one to one mapping from schema to UI is sometimes not convenient in actual use cases. One of the reason to start gohan webui v2 is supporting flexible UI widgets for schema. Even I want to have gohan v1's UI schema support.