Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

npm and iojs with new es6 features #269

Closed
iarna opened this issue Jan 9, 2015 · 40 comments
Closed

npm and iojs with new es6 features #269

iarna opened this issue Jan 9, 2015 · 40 comments

Comments

@iarna
Copy link
Member

iarna commented Jan 9, 2015

Since io.js is going to be getting a new v8, and thus a slew of new es6 features not previously enabled by default, this is going to create a situation where some modules will only work with io.js, at least, until such time as node.js updates its own v8.

The feature in npm that currently handles this kind of thing is package.json engines field.

Now there’s an obvious problem if we add an “iojs” engine type: future versions of node will be likely be api & v8 compatible with iojs, and so depending on a specific iojs version would incorrectly block those node versions.

If adding an “iojs” engine type is out, another option would be to add a “v8” engine type. This would solve the immediate problem, but end users don’t ordinarily think or care about v8 versions, and telling them “your v8 is too old” would be a very crappy user experience (and wouldn’t give them any immediate information on how to fix the problem).

So a third option would be to create a series of “feature-xxx”* modules (eg feature-generators) that have preinstall scripts that fail with meaningful messages if you try to install them in an environment without the feature. These wouldn’t import functionality, obviously, they’d just assert a engine that can handle the feature. Further, if one desired to, even in-development features could easily be supported with major version bumps as their implementation changes.

* Perhaps not actually "feature-xxx", another prefix would do just as well, maybe even just "v8-xxx"?

What do you all think? Do you have other ideas on how we might handle this?

engines

You can specify the version of node that your stuff works on:

{ "engines" : { "node" : ">=0.10.3 <0.12" } }

And, like with dependencies, if you don't specify the version (or if you specify "*" as the version), then any version of node will do.

If you specify an "engines" field, then npm will require that "node" be somewhere on that list. If "engines" is omitted, then npm will just assume that it works on node.

You can also use the "engines" field to specify which versions of npm are capable of properly installing your program. For example:

{ "engines" : { "npm" : "~1.0.20" } }

Note that, unless the user has set the engine-strict config flag, this field is advisory only.

engineStrict

If you are sure that your module will definitely not run properly on versions of Node/npm other than those specified in the engines object, then you can set"engineStrict": true in your package.json file. This will override the user'sengine-strict config setting.

Please do not do this unless you are really very very sure. If your engines object is something overly restrictive, you can quite easily and inadvertently lock yourself into obscurity and prevent your users from updating to new versions of Node. Consider this choice carefully. If people abuse it, it will be removed in a future version of npm.

@jonathanong
Copy link
Contributor

how about a v8 engine option?

@iarna
Copy link
Member Author

iarna commented Jan 9, 2015

@jonathonong: Yeah, I discussed that in paragraph 4, but it does have user experience issues. Do you have thoughts on those? That also won't provide a solution to other kinds of iojs incompatibility that may not be coming this month, but could happen.

@ruimarinho
Copy link

I think that there is potential for io.js to introduce new apis not directly tied to V8 that may also lead to this situation. Ultimately, all new features introduces by io.js not available under node would likely require a feature-xxx declaration on the package. Correct me if I'm wrong, but I think generally npm ecosystems could help solve this problem in a simple way (e.g. limit search to "packages that leverage ES7 await"). In the end, it would be first, up to developer to decide whether to give that package a try and, secondly, to feature detect and polyfill when necessary. The same is true for node right now with packages like native-or-promise or async-listener for example.

There is some overlapse with the engines definition, but with time, versions will have less meaning, just like user agents, and become a bigger burden to developers to maintain.

@mikeal
Copy link
Contributor

mikeal commented Jan 10, 2015

I'm -1 on using engines for this.

Best practice is for module authors to feature detect when possible, this would encourage them not to do that.

There actually should be a "cost" to using new features prior to them being widely adopted but I wouldn't rule out the possibility of people creating tooling for older versions of node that check for modules using new features and compile them down to ES5.

@ruimarinho
Copy link

Agreed. Several packages targeted at node 0.11 (especially for generators stuff) are recommending users to rely on transpilers such as regenerator for node 0.10 compatibility. And then there's always Traceur and 6to5 runtimes for those who want to ES6 features in node and do not want make the switch to io.js just yet.

@rlidwka
Copy link
Contributor

rlidwka commented Jan 10, 2015

v8 engine option makes the most sense imho.

@jbergstroem
Copy link
Member

I agree that testing v8 versions doesn't really give you any more information since there now will be combinations of what flags io.js (or node.js) is started with.

@mikeal how about exposing those features through package.json somehow? That at least give package to package feature dependencies over doing it within the modules (I can see scenarios where it wouldn't make sense to test for certain features since you'd probably wouldn't even install it in the first place without the feature in place). These entries could probably even be generated.

@sunflowerdeath
Copy link

It is possible to automatically find out version of V8 (and any other parameters that affect compatibility) from specified engine and version, and use it, when later there will be another version compatible with specified.

For example, if specified { "engines": { "iojs": ">=1.0.0" } } and node 0.15+ will be compatible with iojs 1.0, when you will try to install on old node version, error message will say that you need iojs >= 1.0.0 OR node >= 0.15, which is compatible.

And when you will install this package on node >= 0.15 it will just warn, that current engine is not same, that was specified, but is compatible.

@iarna
Copy link
Member Author

iarna commented Jan 12, 2015

@sunflowerdeath npm could certainly keep a table of iojs and node versions and their associated v8 versions, and so translate iojs >= 1.0.0 into v8 >= x.x.x. I don't think we currently can see other parameters that effect compatibility though. But yeah, io.js could expose those, but for that to work node.js would also have to be convinced to expose them.

@ljharb
Copy link
Member

ljharb commented Jan 12, 2015

I'd love to see both - a consistent "engines" notation, which indicates what it's been /tested/ on, and a parallel "features" notation (I do not care about the name, bikeshed all you want), which indicates which JS features it depends on. Then it's left up to the npm cli, or the transpiler, or the saas platform, or whatever (or some npm module) to determine dynamically what is required to make the module work, or whether it's possible at all. Thoughts?

@edef1c
Copy link
Contributor

edef1c commented Jan 12, 2015

Currently, all my packages that use generators have a foo.es6.js with the actual source, a foo.es5.js with transpiled code that is generated by the prepublish script, and a foo.js that feature-detects and requires the appropriate file.
No need for magic in package.json, and works across all engines. It might be nice to have a more convenient wrapper around this kind of feature detection / transpilation.

@ljharb
Copy link
Member

ljharb commented Jan 12, 2015

I'm not thinking "magic" as much as "documentation" - your approach is great, but how am I to know that's the format you've used? As described, I'd have to duck type it, whereas with a package.json notation, you'd explicitly call out what you supported.

@caitp
Copy link
Contributor

caitp commented Jan 12, 2015

adding more metadata to package.json about language features used, or v8 versions tested against/supported:

  1. complicates package.json
  2. easily causes package.json to become out of date
  3. easily causes package.json to contain incorrect information

These are all fine, so long as you are vigilant about updating your package.json, but if we're honest, most people aren't.


automating detection of features used:

  1. makes assumptions about how you're going to use the module (transpiling? different version of node/iojs? loading a particular build shipped with the module?)
  2. causes headaches when using or developing modules

This seems like a really, really bad thing to bake into npm, and probably not a very good thing to do in an individual project either (although it's obviously up to you to make the call on how you ship your project)

@indexzero
Copy link

Currently, all my packages that use generators have a foo.es6.js with the actual source, a foo.es5.js with transpiled code that is generated by the prepublish script, and a foo.js that feature-detects and requires the appropriate file.

Yes, oh 1000 times yes. That being said it wouldn't be bad practice (although not required) for something in package.json for batch scripts doing analytics. Not everything can be polyfilled via a transpiler. For those things I wouldn't want npm to warn me, but it's also a like riding a Ferrari to the grocery store to need static analysis to figure this out from a machine perspective.

@brianleroux
Copy link

👍 for using package.json main point to compiled ES5 source. 🍻 🐴

@greim
Copy link

greim commented Jan 13, 2015

...all my packages that use generators have a foo.es6.js with the actual source, a foo.es5.js with transpiled code ... prepublish script ...

I'd prefer if index.js was the canonical, as-written es6 code, while es5/index.js gets built by a prepub script. Module builders who are still in es5-land can require('foo/es5') while everyone else can just require('foo').

@soareschen
Copy link

How about something similar to an npm preinstall script? That script will be written in ES5 and does ES6/V8 feature detection. If any error is thrown, then npm install fails and notify the user to upgrade?

That way feature detection can be as complicated as anyone likes, and npm packages can make sure certain features exist before being installed.

@quantizor
Copy link

I would prefer having the extra metadata in package.json for non-standard features. At least as a console warning so the consumer knows to fetch the appropriate polyfill or upgrade their platform version.

@ljharb
Copy link
Member

ljharb commented Jan 13, 2015

@greim Anything where the main require isn't commonJS is just not going to get adopted by the community, imo. The vast majority of module builders are going to be in ES5-land for a long time.

@greim
Copy link

greim commented Jan 13, 2015

@ljharb Is require('foo/es5') not CJS? It doesn't seem too cumbersome to me.

(BTW I'm not suggesting npm should auto-transpile things to es5 for everyone; only suggesting a convention module authors could follow.)

@ljharb
Copy link
Member

ljharb commented Jan 13, 2015

@greim It is, but the main require - the default if i require the module by name with no slashes.

@edef1c
Copy link
Contributor

edef1c commented Jan 14, 2015

@ljharb please don't do preinstall scripts, it's bad enough that we can't take them out of npm.
Transpilation should be done at prepublish, and feature tests should occur at runtime.
We can probably put together some npm packages to ease the pain of prepublish transpiling and feature-detecting.

@ljharb
Copy link
Member

ljharb commented Jan 14, 2015

I definitely don't advocate preinstall scripts. I fully agree with your comment.

@bajtos
Copy link
Contributor

bajtos commented Jan 14, 2015

Related: #253 (comment). It suggest that the solution is { "engines": { "node": ">=1.0.0" } }.

@rektide
Copy link

rektide commented Jan 19, 2015

I've mentioned a like idea in the tail of #90, and opened a quickly-shut follow-up ticket specifically on this issue npm/npm#90.

@nathan7 @indexzero @mikeal what kills me about the "be a smart author" route- such as the .es5.js/.es6.js variant- is that it leaves great responsibility on each package, and from that a whole bunch of decisions get packages into each app that may or may not conflict with a consumer's desired use-case.

As someone bundling es6+es5 apps and transpiling them myself, minifying them, and otherwise preparing them for deployment, I really don't want that extra baggage to be dealt with at all by the library- I'd rather have a library that just targets ES6, know that it targets ES6, and work that into my toolchain, rather than have a library "help" me by using transpiler X to do the job.

The preferred default state for a package is as simple as possible. Anything any author does to make their package fit for consumption is going to make it harder for me the consumer to overwrite the baked in assumptions & impose the choices I'm making in my distribution. I don't want to deal with your source maps, I don't want to deal with your transpiling- I just want javascript, and my build-chain will handle these things as I feel appropraite.

I see the "put more complexity on the package" versus "simple packages" as a AMD 2.0 debate: I'd way rather make authorship dumb/simple, and leave questions of processing to as late as possible, at the responsibility of the consumer & their chosen build-chain. Otherwise too many assumptions get baked into each package.

@edef1c
Copy link
Contributor

edef1c commented Jan 21, 2015

@rektide we can bundle all of this complexity into a package of its own that you simply include. the AMD approach isn't the npm approach.

@edef1c
Copy link
Contributor

edef1c commented Jan 21, 2015

Example feature-detection package: has-generators

// foo.js
module.exports = require('has-generators') ? require('./foo.es6') : require('./foo.es5')

@edef1c
Copy link
Contributor

edef1c commented Jan 21, 2015

If your stuff requires multiple ES6 things, (require('has-foo') && require('has-bar')) ? … : ….
We can create these tiny little packages with ease for all the ES6 features that are available in io.js today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests