Retooling Build Tools

[jlukic]

I spent some extra time in the park this morning pondering things in #1350

I've come up with some thoughts of how I think we should move this forward to improve workflow for npm and bower.

For reference "pm folder" in this discussion is referring to the local package managed folder i.e. node_modules/semantic-ui or bower_components/semantic-ui.

Problems

Nothing that should be permanent should exist in a PM Folder

Maintainers of both package managers have made it explicit that these folders may be wiped during updates, and generally no one wants to cd into their components folder.

This means that an install should generally not change files inside of its own directory, because it can't expect them to exist on update. If it does has to modify a file in its own directory, than it must make sure that it is trivial to regenerate this file on update.

LESS import must be hard-coded inside definitions

Currently LESS theming and path variables are set in a theme.config file which is expected to exist in PM folder src/. This is because any import paths in a definitions must be hard-coded to something because definition files can't be modified after install or will risk losing changes (see issue above).

If LESS theming variables were added as run-time variables pulled in from semantic.json, then only our build tools would work with our less files. Which is a functional solution, but would not allow the less source to be compiled without our build tools. This might alienate developers.

Proposal Summary

• Gulp install will check if installed with npm or bower.
• If using PM it will suggest user paths in project root.
• Install will also create a gulpfile at project root or add tasks to an existing one.
• Install will create paths if not currently available
• Install will modify PM folder file src/theme.config to a single line file that imports a user's real theme.config located outside of pm folder.
• This file will be considered a temporary file (that could be lost on update/upgrade) and should be automatically regenerated if not found by a backup value stored in semantic.json

Proposal

1) Install tools should recursively search for a semantic.json across parent directories.

This would mean having to modify the project's gulpfile to walk recursively through each parent directory until it hit a semantic.json.

Having a semantic.json at project root would be similar to having a gulpfile gruntfile or package.json or karma.json which is a familiar idea for developers, and reduce the barrier for entry for new developers.

2) Theme folder should include two paths. One for default theme inside PM folder, and another for user themes in project root.

This is to deal with package managed folders being wiped on update or machine change. We will need to modify theme.less to use separate paths for default theme and user themes.

3) User folders should be created on install

Currently the install tool expects you to set paths that already exist. We will need to modify the install task to check the specified path then offer to create the directory if unavailable

4) Install should suggest sensible default paths based on install location

The install tools should check whether it is in a PM folder then suggest a default paths that exists at project root escaping the PM folder.

My suggested setup would be

./node_modules/semantic-ui/
./semantic.json
./semantic/
./semantic/theme.config
./semantic/themes/
./semantic/dist/

5) If a Gulpfile is found at project root during install. We should ask the user if we can modify it to import our own tasks. If no gulpfile exists we should offer to make it.

Developers do not want to cd into a PM folder to run gulp tasks. They want to run it from their project root.

If a gulpfile exists already this would have to be some magic, which imported our tasks as build ui, watch ui by modifying the existing gulp file.

If a gulpfile did not exist, this would just a copy and paste of our gulp file to project root, with build, watch also aliased as build ui watch ui

6) Remove theme config from PM managed src/theme.config, replace with single line @import of user specified theme.config path.

This would be the only pm managed file that would be modified, it would only include a single line which would specify user path from install

@import '../../semantic/theme.config'

Because this file exists in the pm folder, it may be lost on update upgrade or machine change, so I propose including a backup of this path in semantic.json.

Now here's where the magic comes: when a gulp task is run, it checks to see if the PMed src/theme.config exists, if it does not exist, it automatically regenerates it from the backed up path outside of the PM folder in semantic.json.

Thoughts / Involvement

If you would like to contribute with this feature, or have suggestions feel free to comment here. I'll be organizing a feature branch for this where we can begin work on retooling after we can come to some form of consensus.

[chanon]

+1

[kevinSuttle]

The last thing the FED community needs is another file to keep track of. Before you respond, please read these links.

bower/bower#1520
bower/bower#1626
http://blog.npmjs.org/post/101775448305/npm-and-front-end-packaging

https://docs.npmjs.com/files/package.json#git-urls-as-dependencies
http://blog.kewah.com/2014/npm-as-a-front-end-package-manager/
http://blog.keithcirkel.co.uk/how-to-use-npm-as-a-build-tool/

[kristoferjoseph]

Listen to @kevinSuttle please. I've gone down this path and it sucked. A better idea is to add all metadata into package.json with their own properties.
i.e.

{
   "main": "foo",
   "semantic-ui": {
       "custom-stuff": "things"
   }
}

[jlukic]

I've gone ahead and read everything you've linked to @kevinSuttle

The real issue I'm finding difficult with is trying to let users choose everything for themselves—a configuration that works in bower, as a downloadable zip, as an npm package, also when installing with Meteor's atmosphere, and with angular and ember bindings too boot, oh and dont forget each of these should also have an RTL package, and probably a SASS version because people want flexibility.

And all this without asking the developers to learn any new technologies...

[einkoro]

I would aim for a configuration that works with bower and as a downloadable zip (either precompiled or as less). Installing semantic with npm would be the same as using composer for installing semantic in my opinion and the php community has forsaken managing frontend packages with it in favour of front end specific tools. While npm hasn't ruled against it (yet) like composer did I wouldn't spend excess time worrying about it just yet particularly with such a unclear future and the myriad of competing front end package managers these days: spm, duo, component, browserify and normalize if I remember correctly.

Worth a read: http://blog.npmjs.org/post/101775448305/npm-and-front-end-packaging

In any case I'm working on a couple projects at the moment where I'd very much like to use Semantic rather than Foundation and I need to pull it in via bower working from my own project root with any dependencies excluded from our repository so I'm more than willing to lend a hand if need be.

Gulp install will check if installed with npm or bower.

Are you going to do this by looking for node_modules or bower_components? I've encountered plenty of projects where the directories are named differently and this wouldn't work unless you check for bower.json (and .bowerrc) or package.json instead. Much more common with bower than npm though as the only place I've seen it using npm was with a composer plugin to manage bower/npm dependencies: https://github.com/francoispluchino/composer-asset-plugin

[kristoferjoseph]

@brianleroux @kevinSuttle see what I mean?

@brianleroux Sounds pretty familiar. I would, but it'd be the same old story. People need to learn their own way. Otherwise I'm the jerk.

— Kristofer Joseph (@dam) December 10, 2014

<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>

[brianleroux]

this might be read as aggressive or harsh but it isn't. just one guy's opinion possibly shared by a few. there's many ways. you can support all the ways or you can pick a golden path for your project and support that. some people like .zip and globals. some people prefer modularity and composition. but you want consistent and cohesive path for your users. try to do everything and you'll do nothing well.

here's how I'm recommending front end stuff be composed in projects orbiting our work:

• package.json should be the only manifest file in the root of a project. its a standard cowpath now.
• npm i should be how I get my deps
• theme/layout/components/reset should be isolated, discreet and opt in
• favor npm scripts over build too fashion: they're more flexible, do not require the end user to install and run global binary executable or learn another manifest format

[aaronbhansen]

Taking a step back, instead of restructuring the build tools to better support theming and other changes (and still depending on Semantic build tools), what about going the direction of Zurb Foundation and creating an install for a project that is agnostic of the build tools and instead just depends on the css pre-compiler? This allows you to still theme and modify the base files, but use whatever framework you would like to build the semantic css. For Ember especially and the semantic ember tools, we haven't fully fleshed it out yet, but the Ember CLI (which is the recommended build tools for Ember) and gulp are causing some difficulties in combining the two. If Semantic was just an install with the less or sass files like Foundation, this would resolve any issues between different build systems.

[einkoro]

@brianleroux I don't think that reads as either aggressive or harsh. I think we are both of the same opinion that supporting every method is going to be a huge time sink and might not work out very well. It certainly wouldn't hurt to stick to KISS either rather than over complicating things.

I agree with aaron that focusing on providing agnostic install depending on the preprocessor like Foundation would easily allow us to automatically build/package semantic ui for other dependency managers and not have to worry so much about pleasing everyone in the semantic ui repo itself. Foundation's bower builds are a good example of this: https://github.com/zurb/bower-foundation

At that point anyone could pick their favourite task runner (or mandated one depending on your organization/project) to handle preprocessing and other tasks.

[kevinSuttle]

@dam

[nathanpalmer]

I think we should avoid the build tools as much as possible and like @aaronbhansen said rely on the css engine instead. For me it's not about forcing a choice of "my" tool versus what's included in semantic but rather a conflict between what's already included with the web framework and this. Otherwise it's going to increase friction and reduce adoption.

Ember, for example, in order to use Gulp we have to disable the default css pipeline and step outside of that in order to serve things for semantic. Rails is similar which makes me feel that a lot of people would pull in the compiled dist/ version and serve that up instead ignoring the theming altogether because it has the least amount of friction with their existing framework.

[jlukic]

The next release will allow for LESS files to be compiled together. There is also now a single include semantic.less that can be used for importing the library. Hopefully this will make everyone's life easier.

See merge commit

/* import component */
@import 'src/definitions/elements/button'; 
/* import everything */
@import 'src/semantic';

Once meteor branch merges the project will also have much simpler gulp tasks, each defined in their own file, which can be imported into external gulp files.

var
  watch = require('path/to/semantic/tasks/watch')
;
gulp.task('watch ui', 'Watch Semantic UI', watch));

Explore meteor branch

Related Issues
#1385 #1147 #675 #1780 #1498 #1350 #1441

[jlukic]

1.9.0 is now published I encourage anyone who had issues before with build tools to check it out. It should solve for the issues discussed specifically in this thread.

[idurotola]

Hey y'all what's up with this project and whats the 411 on it? Can I bower install now?

[Steveb-p]

Project has since moved to 2.1 version and there are installation instructions here:
http://semantic-ui.com/introduction/getting-started.html
and if you only need package, then here:
http://learnsemantic.com/guide/download.html

[DaSchTour]

I don't know why, but this is totally not working. I tried to work with bower and the semantic.json but it seams like the paths from semantic.json aren't really used because it only works if my theme.config is inside my root although I have specified that it should be in my conf/semantic-ui/
I also added my site overrides to conf/semantic-ui/site but it doesn't use it. I than have seen that there are also paths in the theme.config so I changed them. but it also doesn't work. The build process isn't done. It really frustrating that the build process work with a broken configuration but simply doesn't do anything.

[cmnstmntmn]

one year, and still the process of using semantic-ui with webpack is a pain in the ass

[DaSchTour]

It's really bad. I love Semantic UI because of the style, the components and so on. But the build process and because of the fact, that there is no SASS version, I need to consider to use other frameworks for my feature projects :(

[saeidscorp]

The components still rely on the hard-coded src/theme.config file. (and we're now on the 2.2.0 release!)

1.9.0 is now published I encourage anyone who had issues before with build tools to check it out. It should solve for the issues discussed specifically in this thread.

@jlukic

It solves issues with importing Semantic's LESS file directly, but with bower... no luck. I changed semantic.json residing outside bower_modules to point to the correct paths, but Semantic still relies on hard-coded paths.

It's a pity that such an awesome web framework has this build tool issues.

[Pithikos]

Why is this closed? Two years and still the same issues.

The framework is awesome but without easy customization it's a pain in the ass. I have spent countless hours just to change a single color and still I haven't managed to do so without hard-coding everything.

[cmnstmntmn]

@Pithikos anyway, i find these kind of frameworks bad by design; you can't customize anything without broke some other thing; you should consider tachyons.io that uses composition. this way you could to create unlimited components.

[atrauzzi]

SASS or Stylus. This project needs to migrate over to something more modern, sane and in particular, in such a way that doesn't require gulp as a dependency. I wish I could use all of this, but the whole thing is stuck in 2010.

#6012