Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI bundler #2618

Closed
wants to merge 5 commits into from
Closed

CLI bundler #2618

wants to merge 5 commits into from

Conversation

RunDevelopment
Copy link
Member

@RunDevelopment RunDevelopment commented Nov 4, 2020

This resolves #2404.

I added a small CLI bundler that is functionally equivalent to the download page. It can be used like this:
npx prismjs bundle --languages=html,java,php --theme=coy --js-out=prism.js --css-out=prism.css

API of the CLI:

PS C:\Users\micha\Git\prism> node .\bin\cli.js bundle --help
prismjs bundle

Create JS and CSS bundles for a specific configuration of
languages/plugins/themes.

Options:
      --version    Show version number                                 [boolean]
      --help       Show help                                           [boolean]
  -c, --config     The path of a JS/JSON configuration file that specifies the
                   languages, plugins, and theme of the bundle. The file has to
                   have the following format:
                   | {
                   |   // a list of languages (e.g. ["html", "php", "java"])
                   |   languages?: "all" | string[],
                   |   // a list of plugins (e.g. ["line-number", "toolbar"])
                   |   plugins?: "all" | string[],
                   |   // a theme id (e.g. "default", "coy")
                   |   theme?: string,
                   |   minified?: boolean,
                   | }                                                  [string]
      --languages  A list of languages to includes. Languages are separated by
                   comma (e.g. html,php,java). Use "--languages=all" to include
                   all languages.                                       [string]
      --plugins    A list of plugins to includes. Plugins are separated by comma
                   (e.g. line-numbers,toolbar). Use "--plugins=all" to include
                   all plugins.                                         [string]
      --theme      The name of the theme. If no theme is specified, Prism's
                   default theme will be used.                          [string]
      --minified   Whether languages and plugins should be minified. Use
                   --minified=false for development files.
                                                       [boolean] [default: true]
      --js-out     The output JS file(s) of the generated bundle. Existing files
                   will be overwritten.                                  [array]
      --css-out    The output CSS file(s) of the generated bundle. Existing
                   files will be overwritten.                            [array]

Aside from Prism's files, it has 2 additional dependencies:

  1. yargs is used to parse arguments and handle documentation.
  2. gzip-size is used to determine the compressed bundle size.

This is a problem. I incorrectly installed them as dev deps for now so that Prism still has 0 hard dependencies. I don't know what the best course of action for this is.

There's also the question where we even want a CLI for Prism. (I think it would be nice.)

Some input would be nice.

@siavashs
Copy link

siavashs commented Nov 9, 2020

@RunDevelopment is it possible to support both development and minified versions as output?

@mAAdhaTTah
Copy link
Member

mAAdhaTTah commented Dec 22, 2020

This is a problem. I incorrectly installed them as dev deps for now so that Prism still has 0 hard dependencies. I don't know what the best course of action for this is.

We could bundle up the cli into a single file with something like ncc.

There's also the question where we even want a CLI for Prism. (I think it would be nice.)

I'm not opposed per se but it is code we'll have to maintain for questionable value. You think there are a lot of people who'd find this useful? It's not a huge savings over downloading it from the site, and if you're going to run this from the command line several times, you'd be better off just installing it from npm and bundling it, I think.

@RunDevelopment
Copy link
Member Author

We could bundle up the cli into a single file with something like ncc.

That looks REALLY good!

You think there are a lot of people who'd find this useful?

Hard to say. It does make some things easier (e.g. updating Prism bundles) but idk if people will use it.

It's not a huge savings over downloading it from the site, and if you're going to run this from the command line several times, you'd be better off just installing it from npm and bundling it, I think.

The "bundling it" part is the problem. Bundlers don't know in which order Prism's files have to be loaded because their dependencies are declared in components.{json,js} and not in the files themselves. We wouldn't need the download page and the babel plugin if it was that easy...

@github-actions
Copy link

github-actions bot commented Dec 22, 2020

No JS Changes

Generated by 🚫 dangerJS against d16c0ee

@RunDevelopment
Copy link
Member Author

I used ncc. We now have a bin/src folder that contains the CLI logic and a bin/dist folder that contains the bundled up CLI logic. bin/dist is ignored by git and bin/src is ignored by npm. The new prepublishOnly task will ensure that we won't accidentally publish a release without the CLI.

@mAAdhaTTah
Copy link
Member

We wouldn't need the download page and the babel plugin if it was that easy...

My thinking is, as a PrismJS consumer, if I'm going to download Prism as a one-off, I will use the download page, but if I'm going to need to keep it updated over time, I'm going to use a bundler + npm and can use the babel plugin. I feel like the CLI sits in a space between "spinning up a quick project to play around with" (use the download page) and "building a serious application" (use the babel plugin) that isn't going to get much use.

@RunDevelopment
Copy link
Member Author

How about "building a serious application without babel"? We already had issues from people that don't want to use babel. CLI is a nice solution for those people.

But it's not like I don't get what you mean. In the worst case, this CLI will be something that gets basically no usage but we still have to maintain it. The reason why I want to somewhat push CLI is that it could be a potential solution for issues like #2617.

@mAAdhaTTah
Copy link
Member

How about "building a serious application without babel"?

This could very well just be my bias, but... does anyone do this? Even looking at the "people aren't importing things correctly" issue, the user would be using ESModules, and I don't know how many people are using ESModules for serious applications without a bundler, at which point including babel is fairly straightforward.

@RunDevelopment
Copy link
Member Author

Well, with ES6+ features and ESModules coming close to being universally supported, there are fewer and fewer reasons to use Babel with each passing year. Projects with TypeScript don't need Babel. Projects with bundlers don't need Babel.

I think that the main reason Babel is still so popular is that there is a small framework called React with nice custom syntax. It's the only reason I use Babel but I mainly do things in TypeScript, so things might be different on the JS side.

@mAAdhaTTah
Copy link
Member

there are fewer and fewer reasons to use Babel with each passing year.

Given that the language will continue to evolve faster than the long tail of browsers will add support, I don't think this is true. My team could not easily figure out the exact set of syntax features we can use given our intended browser matrix. However, we can easily configure babel to either figure that out for us or transpile everything down to es5 lowest common denominator (which will be the case for any team that still supports IE11). This allows us to adopt new JS features as they get added to the language rather than when they're adopted by all our supported browsers. I expect this to be the case for a large number of teams for quite a while, React projects or otherwise, and I think this will be a feature of the ecosystem for the foreseeable future.

Projects with bundlers don't need Babel.

This could be true, but I think at the point where you've got a bundler set up, and you want to integrate Prism, is it easier to integrate babel + the plugin or to integrate the CLI? I don't think configuring babel is significantly more work than setting up most bundlers in the first place (e.g. parcel is zero-config for babel), and you get to maintain your version through a system you're already using (package.json) rather than adopting an external mechanism for keeping your Prism version up-to-date (CLI).


Another thought: there's also a bit of ecosystem complexity involved in providing a third mechanism for consuming Prism. Instead of comparing the download page + babel plugin, you also have to consider the CLI and decide between them.

@RunDevelopment
Copy link
Member Author

I find your "ecosystem complexity" argument convincing. While I wouldn't mind third-part tooling (is the babel plugin thrid-party? Are you a third party @mAAdhaTTah???????? ;) ), I also think that it would be a bit of a problem if Prism provided X many ways to do essentially the same thing.

The problem is that components.json, the file that contains all the information needed to create bundles, isn't part of Prism's public API. If it was public, somebody could just make a package, call it prismjs-cli, and have the CLI in there. There already are a few libraries out there that use components.{js,json} because they need dependency information or just because they want to list all available languages.

Maybe we should move towards stabilizing components.{js,json}? Right now, they contain dependency information but also stuff that is specific to the website. With stable and public components.{js,json}, there will be no need to have a CLI in this package.

@mAAdhaTTah
Copy link
Member

is the babel plugin thrid-party? Are you a third party @mAAdhaTTah???????? ;

lol but on a serious note, I've actually wondered if it makes more sense to move the babel plugin into the PrismJS org, considering it's officially recc'd in the docs.

There already are a few libraries out there that use components.{js,json} because they need dependency information or just because they want to list all available languages.

The babel plugin is one of those, and we created an API for that (getLoader). Right now, it's also using the component.js (even those it's technically private/not stable). Maybe it makes sense to expand dependencies.js to provide an API that allows us to implement both what the babel plugin + cli need?

I don't necessarily mind stabilizing components.json but we've seen some changes in that over the lifetime of the project and how we handle dependencies is a bit of a pain point so I kind of prefer having that flexibility in the internal representation.

@RunDevelopment
Copy link
Member Author

I've actually wondered if it makes more sense to move the babel plugin into the PrismJS org, considering it's officially recc'd in the docs.

I don't really have strong opinions on this. The babel plugin would appear more official in the organization, I guess, but that's about the only argument I can see for moving it.

Maybe it makes sense to expand dependencies.js to provide an API that allows us to implement both what the babel plugin + cli need?

That is probably the best solution but the name dependencies.js might be a bit misleading then. The minimum set of features I need to implement the CLI are:

  • List all languages and plugins. (API for supported languages/plugins/etc. #2146)
  • Get the id of the default theme (or have its name stabilized).
  • Get whether a component (lang/plugin/theme/etc.) has CSS and/or JS files.
  • Get the minified/dev file paths of the CSS/JS files of a component.

(Looking at third-party libraries, it would also be good to be able to get the title and aliases of components as well.)

IMO, implementing those features as free functions in dependencies.js would be suboptimal. I'd rather have a class that takes the components.json data in the constructor. Something like:

new Components(require('components.json')).listAll("languages") // => ["markup", "css", ...]

One problem with this class is that the getLoader functionality should be part of the class but deprecating getLoader after only how-many releases is a bit...

@mAAdhaTTah
Copy link
Member

I don't really have strong opinions on this. The babel plugin would appear more official in the organization, I guess, but that's about the only argument I can see for moving it.

Yeah, there's an "official blessing" aspect to it joining the org proper. It's not super important, so we can revisit it later.

IMO, implementing those features as free functions in dependencies.js would be suboptimal. I'd rather have a class that takes the components.json data in the constructor.

Could you elaborate on why? I think if we're requiring consumers to pass components.json in via the constructor, we're kind of treating components.json as "public" in that it's something we're expecting consumers to interact with. I don't have a strong opinion on "class" vs "free functions" (although I tend to prefer functional approaches in my work) but requiring passing in the data structure I'm less sure about.

@RunDevelopment
Copy link
Member Author

if we're requiring consumers to pass components.json in via the constructor, we're kind of treating components.json as "public"

Uhmm. We are currently doing exactly that with getLoader... The current state with getLoader is that the function needs components.json to work but the user has to pass that data in. We only guarantee that getLoader can work with the data from components.json and nothing more. components.json may contain whatever.

We could also use components.js to hold the functionality we need.

@mAAdhaTTah
Copy link
Member

lol well guess I can toss that principle out.

Can we hang the functionality we want off that getLoader implementation? I know its API is not great, and I don't mind considering whatever implementation as a source of tech debt we'd pay off in v2, but I would def prefer avoiding multiple implementations that do similar things.

@RunDevelopment
Copy link
Member Author

The idea was to move getLoader into the new class/set of functions. The current getLoader implementation already does a lot the other functions will also have to do, so having everything in one place seems like a good idea.


Also, I just thought of a little trick to implement all of this in a backward-compatible way. The public interfaces of components.js and dependencies.js looks like this:

// components.js
var components = {}; // something
module.exports = components; 
// dependencies.js
function getLoader(components, load, loaded) { ... }
module.exports = getLoader;

With an intended usage like this:

const components = require('components.js');
const getLoader = require('dependencies.js');

const loader = getLoader(components, load, loaded);

How about we change it to this:

// components.js
class Components {
  getLoader(load, loaded) { ... }
  // other functions (e.g. listAll, getTitle, etc.)
}
var components = new Components({ /* the JSON from `components.json` */ });
module.exports = components; 
// dependencies.js
function getLoader(components, load, loaded) { 
  console.warn('Deprecated! Use `components.getLoader` instead.');
  return components.getLoader(load, loaded);
}
module.exports = getLoader;

@RunDevelopment
Copy link
Member Author

An ESM-based module system for languages and plugins is planned for v2 completely eliminating the need for this.

I'll close this now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automatically downloaded customised files
3 participants