From 48c032fd95db39de117737eb99c0231d213573f7 Mon Sep 17 00:00:00 2001 From: "david.bashford" Date: Fri, 12 Dec 2014 16:46:51 -0500 Subject: [PATCH] fixes #16, allowing external/configured transformers to modify files before they are combined --- README.md | 81 ++++++++++++++++++++++++++++++++++++++++++++++- lib/config.js | 31 +++++++++++++++--- lib/index.js | 73 ++++++++++++++++++++++++++++-------------- src/config.coffee | 19 ++++++++++- src/index.coffee | 61 +++++++++++++++++++++++------------ 5 files changed, 215 insertions(+), 50 deletions(-) diff --git a/README.md b/README.md index c10f51e..2ff48c0 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,7 @@ When `mimosa clean` or `mimosa watch` with the `--clean` flag is run, the `combi ```javascript combine: { folders: [], + transforms:[], removeCombined: { enabled:true, exclude:[] @@ -44,8 +45,20 @@ combine: { output:"stylesheets/vendor.css", exclude:null, include:null, - order:null + order:null, + transforms:[ + function(inputText,inputName,outputName) { + // transform text + return transformedText; + } + ] }], + transforms:[ + function(inputText,inputName,outputName) { + // transform text + return transformedText; + } + ], removeCombined: { enabled:true, exclude:[] @@ -60,6 +73,72 @@ combine: { * `combine.folders.exclude`: an array of strings and/or regexs, the list of files and file patterns to exclude from the combine. Paths should be relative to `folder` and should point at the compiled file. So foo.css, not foo.less. Regexes can also be used at the same time. ex: `ex: [/\.txt$/, "vendor/jqueryui.js"]`. Can be left off or made `null` if not needed. Note: `include` and `exclude` are exclusive. If you have configured both, mimosa will error out during startup with validation errors. * `combine.folders.include`: an array of strings and/or regexs, the list of files and file patterns to include in the combine. Paths should be relative to `folder` and should point at the compiled file. So foo.css, not foo.less. Regexes can also be used at the same time. ex: `ex: [/\.txt$/, "vendor/jqueryui.js"]`. Can be left off or made `null` if not needed. Note: `include` and `exclude` are exclusive. If you have configured both, mimosa will error out during startup with validation errors. * `combine.folders.order`: an array of strings, the list of files to include in the combined file first. Does not need to be all the files, just the files for which order is important. Paths should be relative to `folder` and should point at the compiled file. So foo.css, not foo.less. Can be left off or made null if not needed. +* `combine.folders.transforms`: See [Transform Functions]() below. Transform functions provided at the `folders` level are applied to only the files being merged for this folder. +* `combine.transforms`: See [Transform Functions]() below. Top level transform functions are applied to all `combine.folders` entries. * `combine.removeCombined`: configuration for cleaning up during a `mimosa build` * `combine.removeCombined.enabled`: Defaults to `true`, whether or not to clean up the files that went into making the combine files. * `combine.removeCombined.exclude`: Files to exclude from removal, can be regex or string, strings are relative to the `watch.compiledDir`. + +# Transform Functions + +Both with `combine.transforms` and `combine.folders.transforms` you are able to use functions provided via the config to transform the text of files being combined. + +Why would you want to do this? Lets say you have a folder structure that looks like this: + +``` +/assets + /stylesheets + /vendor + /leaflet.draw + leaflet.draw.css + /images + spritesheet.png +``` + +And lets say you want to combine all vendor stylesheets into a `/public/stylesheets/vendor.css` file. So your `leaflet.draw.css` ends up combined inside `vendor.css`. `spritesheet.png` is obviously not combined because it is binary. So your `/public` directory structure looks like this: + +``` +/public + /stylesheets + vendor.css + /vendor + /leaflet.draw + /images + spritesheet.png +``` + +Purposefully it was decided not to JUST COPY the spritesheet into the same location as the combined file. Leaflet plugins, for example, tend to always have spritesheets named `spritesheet.png`, which means they cannot all exist in the same directory. It is best to keep them apart. + +`leaflet.draw.css` has references to the `spritesheet.png` inside of it. References that are now broken because the paths no longer resolve. + +```css +.leaflet-draw-toolbar a { + background-image: url('images/spritesheet.png'); + background-repeat: no-repeat; +} +``` + +A transform function can take care of this. It can parse the text to find the paths and then alter them. + +## How to use/create a transform function + +Both `combine.transforms` and `combine.folders.transforms` can take an array of transform functions. These functions are passed three parameters and must returned the transformed text + +```javascript +var transformFunction = function( inputText, inputFileName, outputFileName) { + // transform the text + return transformedText; +} +``` + +* `inputText` is the text of the file being processed, it is the contents of a file that is going to be combined +* `inputFileName` is the name of the file being processed from inside the `watch.compiledDir` +* `outputFileName` is the name of the combined output file where the input file is going to end up. + +You can provide functions directly in-line, but it is recommended you keep transforms someplace outside the config and `require` them in. + +```javascript +transforms:[require('./scripts/transformX')] +``` + +Soon there will be transforms available in NPM that you can include in the `package.json` of your project and `require` in directly. diff --git a/lib/config.js b/lib/config.js index 3c009e9..1c9c4c0 100644 --- a/lib/config.js +++ b/lib/config.js @@ -2,6 +2,7 @@ exports.defaults = function() { return { combine: { + transforms: [], folders: [], removeCombined: { enabled: true, @@ -13,18 +14,27 @@ exports.defaults = function() { }; exports.placeholder = function() { - return "\t\n\n combine:\n folders: [] # Configuration for folder combining. See\n # https://github.com/dbashford/mimosa-combine for details on how to set up\n # entries in the folders array\n removeCombined: # configuration for removing combined files\n enabled:true # when set to true, during 'mimosa build' only, mimosa-combine will remove\n # the files that were merged into single files\n exclude:[] # mimosa-combine will not remove any of these files.\n"; + return "\t\n\n combine:\n transforms:[] # an array of transform functions to use to alter files\n # before they are combined\n folders: [] # Configuration for folder combining. See\n # https://github.com/dbashford/mimosa-combine for details on how to set up\n # entries in the folders array\n removeCombined: # configuration for removing combined files\n enabled:true # when set to true, during 'mimosa build' only, mimosa-combine will remove\n # the files that were merged into single files\n exclude:[] # mimosa-combine will not remove any of these files.\n"; }; exports.validate = function(config, validators) { - var combine, combines, errorStart, errors, _i, _len; + var combine, combines, errorStart, errors, transform, _i, _j, _k, _len, _len1, _len2, _ref, _ref1, _ref2, _ref3; errors = []; errorStart = "combine.folders"; if (validators.ifExistsIsObject(errors, "combine", config.combine)) { + if (validators.ifExistsIsArray(errors, "combine.transforms", config.combine.transforms)) { + _ref = config.combine.transforms; + for (_i = 0, _len = _ref.length; _i < _len; _i++) { + transform = _ref[_i]; + if (Object.prototype.toString.call(transform) !== '[object Function]') { + errors.push("combine.transforms entries must be of type Function"); + } + } + } combines = config.combine.folders; if (validators.ifExistsIsArray(errors, errorStart, combines)) { - for (_i = 0, _len = combines.length; _i < _len; _i++) { - combine = combines[_i]; + for (_j = 0, _len1 = combines.length; _j < _len1; _j++) { + combine = combines[_j]; if (typeof combine === "object" && !Array.isArray(combine)) { if (combine.folder) { combine.folder = validators.multiPathNeedNotExist(errors, "" + errorStart + ".folder", combine.folder, config.watch.compiledDir); @@ -40,6 +50,15 @@ exports.validate = function(config, validators) { continue; } validators.ifExistsArrayOfMultiPaths(errors, "" + errorStart + ".order", combine.order, combine.folder); + if (validators.ifExistsIsArray(errors, "" + errorStart + ".transforms", combine.transforms)) { + _ref1 = combine.transforms; + for (_k = 0, _len2 = _ref1.length; _k < _len2; _k++) { + transform = _ref1[_k]; + if (Object.prototype.toString.call(transform) !== '[object Function]') { + errors.push("" + errorStart + ".transforms entries must be of type Function"); + } + } + } if (combine.exclude && combine.include) { errors.push("Cannot have both combine.folders.include and combine.folders.exclude"); } else { @@ -53,6 +72,10 @@ exports.validate = function(config, validators) { errors.push("Installed version of Mimosa does not support combine.folders.include. Need Mimosa version 2.3.22 for this feature. You may want to use older version of mimosa-combine."); } } + if (errors.length > 0) { + continue; + } + combine.transforms = ((_ref3 = combine.transforms) != null ? _ref3 : []).concat((_ref2 = config.combine.transforms) != null ? _ref2 : []); } else { errors.push("" + errorStart + " must be an array of objects."); } diff --git a/lib/index.js b/lib/index.js index eee8564..335634f 100644 --- a/lib/index.js +++ b/lib/index.js @@ -1,5 +1,5 @@ "use strict"; -var config, fs, logger, path, registration, wrench, _, __cleanUpDirectories, __getEncoding, __getFileText, __mergeDirectory, __removeAllCombined, _checkForMerge, _cleanCombined, _mergeAll; +var config, fs, logger, path, registration, wrench, _, __cleanUpDirectories, __getEncoding, __getFileText, __mergeDirectory, __processIncludeExclude, __removeAllCombined, __transformText, _checkForMerge, _cleanCombined, _mergeAll; fs = require("fs"); @@ -90,26 +90,26 @@ _mergeAll = function(mimosaConfig, options, next) { return next(); }; -__mergeDirectory = function(combine) { - var addFileText, folderFiles, includedFiles, outputFileText, removeFiles; - if (!fs.existsSync(combine.folder)) { - return logger.warn("mimosa-combine: combine folder [[ " + combine.folder + " ]] does not exist"); - } - if (logger.isDebug()) { - logger.debug("Combining [[ " + combine.folder + " ]]"); - } - if (fs.existsSync(combine.output)) { - if (logger.isDebug()) { - logger.debug("Removing current combined file [[ " + combine.output + " ]]"); +__transformText = function(folderCombineConfig, inputFileName, inputFileText) { + var outputFileName, transform, transformedText, transforms, _i, _len; + transforms = folderCombineConfig.transforms; + if (transforms && transforms.length) { + outputFileName = folderCombineConfig.output; + for (_i = 0, _len = transforms.length; _i < _len; _i++) { + transform = transforms[_i]; + transformedText = transform(inputFileText, inputFileName, outputFileName); + if (transformedText === void 0) { + logger.error("mimosa-combine transform returned undefined"); + } else { + inputFileText = transformedText; + } } - fs.unlinkSync(combine.output); } - folderFiles = wrench.readdirSyncRecursive(combine.folder).map(function(f) { - return path.join(combine.folder, f); - }); - folderFiles = folderFiles.filter(function(f) { - return fs.statSync(f).isFile(); - }); + return inputFileText; +}; + +__processIncludeExclude = function(combine, folderFiles) { + var includedFiles; if (combine.isExclude) { folderFiles = _.difference(folderFiles, combine.exclude); if (combine.excludeRegex) { @@ -131,19 +131,44 @@ __mergeDirectory = function(combine) { folderFiles = includedFiles; } } + return folderFiles; +}; + +__mergeDirectory = function(combine) { + var addFileText, folderFiles, outputFileText, removeFiles; + if (!fs.existsSync(combine.folder)) { + return logger.warn("mimosa-combine: combine folder [[ " + combine.folder + " ]] does not exist"); + } + if (logger.isDebug()) { + logger.debug("Combining [[ " + combine.folder + " ]]"); + } + if (fs.existsSync(combine.output)) { + if (logger.isDebug()) { + logger.debug("Removing current combined file [[ " + combine.output + " ]]"); + } + fs.unlinkSync(combine.output); + } + folderFiles = wrench.readdirSyncRecursive(combine.folder).map(function(f) { + return path.join(combine.folder, f); + }); + folderFiles = folderFiles.filter(function(f) { + return fs.statSync(f).isFile(); + }); + folderFiles = __processIncludeExclude(combine, folderFiles); if (folderFiles.length === 0) { logger.info("mimosa-combine: there are no files to combine for configuration"); return []; } outputFileText = ""; removeFiles = []; - addFileText = function(f) { - var fileText; - fileText = __getFileText(f); + addFileText = function(fileName) { + var fileText, transformedText; + fileText = __getFileText(fileName); if (fileText) { - return outputFileText += fileText; + transformedText = __transformText(combine, fileName, fileText); + return outputFileText += transformedText != null ? transformedText : ""; } else { - return removeFiles.push(f); + return removeFiles.push(fileName); } }; if (combine.order != null) { diff --git a/src/config.coffee b/src/config.coffee index 91647d9..2ef83df 100644 --- a/src/config.coffee +++ b/src/config.coffee @@ -2,6 +2,7 @@ exports.defaults = -> combine: + transforms:[] folders: [] removeCombined: enabled:true @@ -13,6 +14,8 @@ exports.placeholder = -> \t combine: + transforms:[] # an array of transform functions to use to alter files + # before they are combined folders: [] # Configuration for folder combining. See # https://github.com/dbashford/mimosa-combine for details on how to set up # entries in the folders array @@ -29,8 +32,13 @@ exports.validate = (config, validators) -> errorStart = "combine.folders" if validators.ifExistsIsObject(errors, "combine", config.combine) - combines = config.combine.folders + if validators.ifExistsIsArray(errors, "combine.transforms", config.combine.transforms) + for transform in config.combine.transforms + if Object.prototype.toString.call(transform) isnt '[object Function]' + errors.push "combine.transforms entries must be of type Function" + + combines = config.combine.folders if validators.ifExistsIsArray(errors, errorStart, combines) for combine in combines if typeof combine is "object" and not Array.isArray(combine) @@ -49,6 +57,11 @@ exports.validate = (config, validators) -> validators.ifExistsArrayOfMultiPaths(errors, "#{errorStart}.order", combine.order, combine.folder) + if validators.ifExistsIsArray(errors, "#{errorStart}.transforms", combine.transforms) + for transform in combine.transforms + if Object.prototype.toString.call(transform) isnt '[object Function]' + errors.push "#{errorStart}.transforms entries must be of type Function" + if combine.exclude and combine.include errors.push "Cannot have both combine.folders.include and combine.folders.exclude" else @@ -61,6 +74,10 @@ exports.validate = (config, validators) -> else errors.push "Installed version of Mimosa does not support combine.folders.include. Need Mimosa version 2.3.22 for this feature. You may want to use older version of mimosa-combine." + continue if errors.length > 0 + + combine.transforms = (combine.transforms ? []).concat( config.combine.transforms ? [] ) + else errors.push "#{errorStart} must be an array of objects." diff --git a/src/index.coffee b/src/index.coffee index a867cba..49d3cd7 100644 --- a/src/index.coffee +++ b/src/index.coffee @@ -45,7 +45,8 @@ _checkForMerge = (mimosaConfig, options, next) -> if combine.includeRegex? and fileName.match(combine.includeRegex) doit = true - __mergeDirectory combine if doit + if doit + __mergeDirectory combine next() @@ -61,22 +62,20 @@ _mergeAll = (mimosaConfig, options, next) -> next() -__mergeDirectory = (combine) -> - unless fs.existsSync combine.folder - return logger.warn "mimosa-combine: combine folder [[ #{combine.folder} ]] does not exist" - - if logger.isDebug() - logger.debug "Combining [[ #{combine.folder} ]]" - - if fs.existsSync combine.output - if logger.isDebug() - logger.debug "Removing current combined file [[ #{combine.output} ]]" - fs.unlinkSync combine.output +__transformText = (folderCombineConfig, inputFileName, inputFileText) -> + transforms = folderCombineConfig.transforms + if transforms and transforms.length + outputFileName = folderCombineConfig.output + for transform in transforms + transformedText = transform( inputFileText, inputFileName, outputFileName) + if transformedText is undefined + logger.error("mimosa-combine transform returned undefined") + else + inputFileText = transformedText - folderFiles = wrench.readdirSyncRecursive(combine.folder).map (f) -> path.join combine.folder, f - folderFiles = folderFiles.filter (f) -> fs.statSync(f).isFile() + inputFileText - # deal with include/exclude of files from folder +__processIncludeExclude = (combine, folderFiles) -> if combine.isExclude # remove files mentioned by name folderFiles = _.difference(folderFiles, combine.exclude) @@ -98,6 +97,27 @@ __mergeDirectory = (combine) -> folderFiles = includedFiles + folderFiles + +__mergeDirectory = (combine) -> + unless fs.existsSync combine.folder + return logger.warn "mimosa-combine: combine folder [[ #{combine.folder} ]] does not exist" + + if logger.isDebug() + logger.debug "Combining [[ #{combine.folder} ]]" + + if fs.existsSync combine.output + if logger.isDebug() + logger.debug "Removing current combined file [[ #{combine.output} ]]" + fs.unlinkSync combine.output + + # read in all files in folder, remove directories + folderFiles = wrench.readdirSyncRecursive(combine.folder).map (f) -> path.join combine.folder, f + folderFiles = folderFiles.filter (f) -> fs.statSync(f).isFile() + + # deal with include/exclude of files from folder + folderFiles = __processIncludeExclude(combine, folderFiles) + if folderFiles.length is 0 logger.info "mimosa-combine: there are no files to combine for configuration" return [] @@ -105,12 +125,13 @@ __mergeDirectory = (combine) -> outputFileText = "" removeFiles = [] - addFileText = (f) -> - fileText = __getFileText f + addFileText = (fileName) -> + fileText = __getFileText fileName if fileText - outputFileText += fileText + transformedText = __transformText combine, fileName, fileText + outputFileText += transformedText ? "" else - removeFiles.push f + removeFiles.push fileName if combine.order? folderFiles = _.difference(folderFiles, combine.order) @@ -209,4 +230,4 @@ module.exports = registration: registration defaults: config.defaults placeholder: config.placeholder - validate: config.validate + validate: config.validate \ No newline at end of file