I actually think this is a good idea. We used to ship some other gems like rspec and removed them because they were not universal. Bundler is nearly universal in usage on a project of any size. So +1 from me.

Of any substantial size...

We have actually moved toward shipping fewer libraries as built in. Currently, I don't think our normal distribution ships anything but jruby-launcher.

However, I could see adding more utilities to jruby-launcher. Rake and Bundler would both be good candidates, as would RSpec. But there's a lot of different possible combinations here and I don't want us to get mired down deciding what should be included...

jruby-bin-* does actually install rake already. This might be another error to be fixed perhaps with out Maven changes? The main reason for adding bundler (and I think rake, gem) is that jruby-complete still does not have a nice mechanism for adding specific gems to it past manually re-packing or having a ENV plan. It would make the above scenario very simple. To me it is a balance between which one will generate more complaints :)

well, reading about the bundler idea - it reminded me of
soveran/cuba#20 (comment)

about jruby-complete - how can bundler help there ? (sorry for not seeing
something maybe obvious)

@mkristian If jruby-complete contained Bundler, then Bundler would be immediately available in a JRuby downloaded via Maven (my build scripts can reference JRuby via Maven, and use Bundler from the classpath).

I removed the 'automatic install of a bundler version' with the
bundler-maven-plugin and force the pom.xml to reference the bundler version
which is needed. for me it feels less is more ;) i.e. less magic is more
control. guess personal preference.

@mkristian Perhaps I should have said 'from Maven' instead of 'via Maven'.

In my particular case, I am using Gradle, but getting JRuby from Maven. Gradle is also self-bootstrapping with gradlew. Or, in older versions of this script, using make and wget.

+1 Pretty much impossible to deploy a ruby app without bundler these days.

I would like to understand why jruby should include bundler.

if I install MRI on my OS I do not get bundler, I still need to
$ gem install bundler
before I can use bundler.

same holds true if I install any ruby version via RVM or rbenv

even on travis they install ruby and then they install bundler the usual
way, they even need rake to be part of the Gemfile !!!

so what is the rational why jruby needs bundler and all the other rubies do
NOT.

just try to understand that demand ;)

• christian

1 word reason for including bundler: bootstrapping. Specifically, using JRuby to get a working Ruby in an arbitrary/unknown environment with minimal effort.

My use case is bootstrapping projects that use Ruby dependencies. Typically nanoc sites. These sites are used, modified, and deployed in a managed computing environment where we may or may not have Ruby, we may or may not have a Ruby that has adequate facilities to let us build extensions, etc. Users also have crazy-limited home directory quotas, and who knows what gems installed. I don't think the standard Ruby our system administrators give us includes bundler, either.

We do have a working JVM reliably.

One of the beauties of JRuby is that it is effectively self-contained. I can download the JRuby binary zip file and it just works on any Linux, Mac, or even Windows system, so long as it can find a JVM. If my build tool (Gradle, in my case), can talk to Maven repositories, then I can just have it pull in jruby-all and not even have to write that download logic myself.

Once I have Bundler, it takes care of installing nanoc and all the other required gems locally into the project directory. Again, almost zero hassle.

The one hassle in this chain is getting Bundler installed. I have to install it somewhere; if I unpack the JRuby zip file, I can install it there, but it's still another piece. If I am launching JRuby from Maven via Gradle's JavaExec task, I don't really have anywhere to install it except a Bundler-only gem directory in the project or build directory. In which case I have to set up environment variables to tell JRuby where to find Bundler for all subsequent tasks.

If JRuby included Bundler, then boostrapping an app would be crazy-simple:

1. Download and unpack JRuby (or just require it to be on the classpath, fetched from Maven)
2. Run bundle install
3. Enjoy

thank you very much indeed for the detailed explanation. I have to admit
that I do not agree with you fully, if gradle can run

bundle install

then gradle can also run

gem install bundler
bundle install

and I think jruby knows where to install bundler so that the second call
finds it to install the remaining gems.

OK the is my personal option and let's see how the jruby core teams think
about this ;)

regards,
christian

In my opinion bundler should be a part of the core rails distribution in
the MRI too along with rake and gems. The fact is that it's just not
possible to write and deploy a ruby app without gem, rake and bundler.

In this case I am fooling around with jruby standalone because it allows me
to not have ruby installed on the server at all and I can "bundle" my app
with the ruby it needs.

On Tue, Dec 3, 2013 at 4:48 AM, Christian Meier notifications@github.comwrote:

I would like to understand why jruby should include bundler.

if I install MRI on my OS I do not get bundler, I still need to
$ gem install bundler
before I can use bundler.

same holds true if I install any ruby version via RVM or rbenv

even on travis they install ruby and then they install bundler the usual
way, they even need rake to be part of the Gemfile !!!

so what is the rational why jruby needs bundler and all the other rubies do
NOT.

just try to understand that demand ;)

• christian

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1146#issuecomment-29628300
.

+1

I'm currently "manually re-packing" (as @enebo mentioned) to include Bundler, like this: https://gist.github.com/qerub/7675016

so what is the rational why jruby needs bundler and all the other rubies do NOT.

All rubies should include Bundler, IMHO. :)

But I think it's more important for JRuby to include it because of jruby-complete where it is not installable out-of-the-box:

$ java -jar jruby-complete-1.7.8.jar -S gem install bundler
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the file:/Users/qerub/local/sw/jruby-complete/jruby-complete-1.7.8.jar!/META-INF/jruby.home/lib/ruby/gems/shared directory.

(One has to fiddle with GEM_HOME to make it work as @elehack already said.)

Bundler is easy, specially if you have it already installed. Most rubyists already use it daily, so having it as a default looks like a good idea. But on the other hand, you can solve the same problem with simpler tools. The lines of code count for Bundler yields a hard to explain number of 9800. I don't have Bundler installed, I use a combination of two gems (gs and dep) and I recommend people to do the same. People that go that route are happy with the result, but as I'm not very influential in the Ruby community, that group is rather small.

Tools like npm do a better job at managing dependencies (and fighting complexity) than what RubyGems and Bundler accomplish. In my opinion, we should learn from better tools and strive for a radical change. Instead, we have slowly but steadily perpetrated the idea that RubyGems and Bundler is all we deserve. I'm against including Bundler in any Ruby distribution, because I can't find a reasonable justification to add 9800 lines of code when I can obtain the same result with ~150.

will just nicely install it in default location of your home directory. no
need for GEM_HOME and GEM_PATH

On Mon, Dec 2, 2013 at 10:15 PM, Christoffer Sawicki <
notifications@github.com> wrote:

$ java -jar jruby-complete-1.7.8.jar -S gem install bundler
ERROR: While executing gem ... (Gem::FilePermissionError)
You don't have write permissions for the file:/Users/qerub/local/sw/jruby-complete/jruby-complete-1.7.8.jar!/META-INF/jruby.home/lib/ruby/gems/shared directory.

(One has to fiddle with GEM_HOME to make it work as @elehackhttps://github.com/elehackalready said.)

$ java -jar jruby-complete-1.7.8.jar -S gem install bundler --user-install

will just nicely install it in default location of your home directory. no
need for GEM_HOME and GEM_PATH.

I am the RubyGems maintainer.

The RubyGems team is working with the Bundler team to merge all of its features and integration tests into RubyGems. Currently RubyGems 2.2.0 is scheduled to ship along with Ruby 2.1.0 with the basic features of bundler and some compatibility. I expect to be largely complete around this time next year with bundler being a small shim around the imported features in RubyGems sometime in the following year.

Since Bundler is scheduled to die in under two years time, would it be prudent to include Bundler in JRuby?

@mkristian That assumes you are OK with stomping on whatever the user has in ~/.gem. I'm going for something more self-contained. JRuby and Bundler get me 95% there; my interest in this bug is making the last 5% easy to obtain.

@drbrain That is very good news. 1–2 years feels like a long way off, but it may well not make sense to put bundler in and then take it out again.

All we need is for Rubygems (or another tool) to gain the ability to turn a Gemfile into a Gemfile.lock. Given a Gemfile.lock, installing the locked dependencies is (as far as I've experienced) trivial. Distributing bundler with jruby, or any ruby distribution, would perpetuate needless complexity. Please, no.

@soveran RubyGems 2.1.x and earlier + dep + gs cannot do what Bundler does today as none of the three contain a proper dependency resolver.

@regularfry RubyGems 2.2 is shipping with Gemfile and Gemfile.lock support with Ruby 2.1.0 this Christmas, but full compatibility with Bundler will not (yet) be guaranteed.

@drbrain It's a different approach, where a dependency resolver in runtime is no longer needed. I've been using that approach for the past six years, and during all that time I have been the happiest rubyist in town while my friends struggle with Bundler. The idea is to trade space for complexity. Once the gems are installed in isolation, there's no need for dependency resolution in runtime. The node guys have been enjoying that for a long time, and they are very proud of npm, not only because it solves that problem, but also because it is modular: http://andrew.ghost.io/emulating-node-js-modules-in-ruby/

@drbrain As long as I can continue to avoid runtime dependency resolution, I'll be happy. If I can simplify my stack, I'll be happier still.

One reason bundler exists in the first place is because the MRI is not
Jruby. With jruby I know that every gem will work on every platform but
with the MRI any gem that requires a C lib has to be compiled on the target
platform.

In a way the real answer to what I am trying to do is warbler but I
recently watched a talk at a ruby conference where the presenter made the
point that rails doesn't want to live in a jar, it really needs to be in a
directory structure for it work properly.

Maybe what is really needed is a jruby specific framework which takes
advantage of all the strengths of Jruby.

@soveran Are gs + dep guarantee that only one gem version for every gem is installed? What happens if different gems have the same dependency but with different versions? As long as you have multiple versions of one gem installed - you have to have a runtime setup just like Bundler does to force specific gem versions for all the dependencies. /cc @drbrain

@fxposter for RubyGems 2.1.x and older dep + gs will fail to install a correct set of dependencies because RubyGems does not resolve correctly and neither tool provide a resolver.

@timuckun bundler does not solve the problem of platform specific gems. It solves the problem of having a consistent set of libraries when deploying your application. I do not see how that problem would be different between MRI and Jruby.

For the record, I don't think the plan to eventually merge Bundler into Rubygems should impact the decision to include it in JRuby. The version of Rubygems that eventually ships with full Bundler support integrated will fully support Bundler. There shouldn't be big costs associated with the transition from Bundler gem to Bundler in Rubygems — if there are, people will just keep using the gem, and the goals of the merge will have failed. Assuming that we're able to successfully merge everything, it will simply mean upgrading Rubygems versions and not needing the Bundler gem after that.

I'm not sure where the "boo runtime dependency resolution" stuff is coming from, but Bundler doesn't do that (by design) in deployment mode. One of the reasons Bundler exists is that runtime dependency resolution cannot solve some dependency graphs.

In the meantime, if JRuby wants to ship universally used gems anytime sooner than the next two years, I'm happy to assist with anything I can in order to add Bundler to the distribution.

@fxposter gs + dep don't guarantee that only one version for every gem is installed, but as you isolate the gemsets, having two versions of the same gem is not likely to happen. If it does happen, you only need to specify a strict version dependency in the .gems manifest. But I think it will be good for people dealing with this problem to try that approach, and even try npm if they want to investigate, as first hand experience is way better than having to trust my words.

OK, sounds good. I was just making a comment about the current state of the Bundler code and why it currently isn't suitable for JRuby. Nobody else had brought up this issue so I thought it was worth saying. I wasn't really aware that you are a Bundler developer and wanting to help. If I had known that, I might have softened the first sentence of my comment to be something like, "It seems to me that Bundler should not be included JRuby until we can get the test suite to run in JRuby."

On another note: I never claimed that generating the manual was important, so it is strange you were asking me about that.

Generating the manual is the only thing in the spec suite that doesn’t work on JRuby, as far as I know. That’s why I’m so confused about your insistence that Bundler isn’t suitable for inclusion.

Incidentally, the Bundler tests are run against JRuby as part of every commit to Bundler (albeit not passing at this moment, but that’s due to a git upgrade causing problems, I think).

On Dec 9, 2013, at 6:16 PM, David Grayson notifications@github.com wrote:

OK. I was just making a comment about the current state of the Bundler code and why it currently isn't suitable for JRuby. Nobody else had brought up this issue so I thought it was worth saying. I wasn't really aware that you are a Bundler developer and wanting to help. If I had known that, I might have softened the first sentence of my comment to be something like, "It seems to me that Bundler should not be included JRuby until we can get the test suite to run in JRuby."

On another note: I never claimed that generating the manual was important, so it is strange you were asking me about that.

—
Reply to this email directly or view it on GitHub.

My experience with Bundler's specs were different from yours. I figured I had to run them with "rake spec" so I was unable to run any specs in JRuby, but it sounds like you probably have another way of running them that gets around that, perhaps just running "rspec" in the top directory?

@indirect I am very very sorry that the link got transported the wrong message. the essence of the article was, if a (linux) distribution wants to use jruby as drop in replacements for ruby it want to be able to compile bundle and run their tests with it. as far I understood (I might mix up info from somewhere else) that is not possible with gentoo. and other distribution as might have similar issues - but I do not know. so my intention was not clear and I SHOULD have added some comment along with the link. again all I can say now that have will be more careful the next time.

about including bundler - you have to look at the jruby_1_7 branch and at the master branch. the master is ruby-2.1 only and will probably follow the respective rubygems version as well and that will have bundler built in. whether it really makes a lot of sense to include it into that branch - IMO there are lots of little pros and cons and no clear yes let's do it.

Remember, there is not yet a RubyGems version that contains all the Bundler features, and won't be for at least two years. I'm pretty sure JRuby will have a release long before RubyGems is a viable replacement for Bundler.

Related: rubygems/bundler#2895

If we knew bundler specs were green on JRuby it would go a long way toward considering including it as a default gem.

@indirect You say the specs are run on JRuby on every commit, but if you are referring to travis it appears that the specs just fail immediately trying to install the aforementioned C extensions: https://travis-ci.org/bundler/bundler/jobs/19072055

My PR would go a long way toward getting specs to run – and run green – on JRuby.

@headius When I wrote that, JRuby was newly added to the build, and all the specs were failing because of git. I didn't realize JRuby had additional issues. Thank you for the pull request, and we'll keep working on getting the specs green on JRuby!

Application Servers without access to the Internet would benefit from having bundler as part of any Ruby Distribution.

@Asmod4n except what would be the point if there's no internet access and thus now way to install gems :) ?

@kares you could have a local repository of gems in your intranet.

... including the bundler gem since its for the best using the very same version the bundle was installed with.

From @drbrain's assessment in late 2013:

Remember, there is not yet a RubyGems version that contains all the Bundler features, and won't be for at least two years.

Well, we're closing in on three years now and RG still does not include all of (or even most of) Bundler's features, right @indirect @drbrain?

I'd like to see what MRI folks think about including Bundler as a default gem. It has become nearly as indispensable as RubyGems itself, and yet it isn't shipped in the box.

even if MRI folks think its fine I still feel like its not that definite for JRuby - most of the times its fine but :

• even lately Bundler (1.13.1) was released with a JRuby specific bug (they only test against MRI?)
• JRuby does some RGs extensions, Bundler patches RGs heavily - potential conflict is always near
• there has been problems (on JRuby) working around default gems - believe there's still left overs

... if Bundler is included JRuby should strongly consider running their suite (at the given release tag)

@kares I think your points are true but regardless whether we ship bundler or not we have to deal with these issues now. We cannot live for too long without working bundler support and retain users. I definitely agree we need to do something about having CI for bundler to know when something breaks sooner than later.

I am still somewhat for this idea since RG still does not subsume the important bits of bundler yet and we will end up shipping a version of bundler we are confident in rather than letting people install the latest (and possibly have an immediate negative response).

I am all for it, just that I do have concerns with the current state of RG/Bundler. default gem issues might be most critical - recall there has been an issue where it prohibited using an explicitly installed gem version (of jossl) and always loaded the default - packed version.

@kares isn't there even a test for the case which you just mentioned about the default gem. maybe not in the main test suite but should rather easy to add, that an downgraded version of bundler is really picked.

@mkristian unsure but I know @etehtsea has been struggling lately with some default gem behavior.

@kares is there more than #4083 which is clearly a problem ?

@kares @mkristian to be honest I can't reproduce this anymore. Maybe there was some local issue.

@mkristian @etehtsea that was the one on my mind - thank you both for pointing me into directions...
guess my concerns are no longer relevant than ... 🚸 looking forward for anyone attempting the Bundler packaging setup ... maybe for 9.2? I will check (or add) 4 a gem default test case @mkristian mentioned.

I was going to resolve this based on not everyone agreeing we should include it but most people seem to support including it.

I guess our main issue is operationally knowing whether a particular version of bundler is working well enough for us to include it in the distribution? This obviously is an issue whether we include it or not but we have more pressure to make sure the particular version we include is working if we plan on endorsing it by putting in the distro...

still resisting a bit ... just hit a regression in their point release (1.14.4) that only seemingly broke in a .war
... to be fair its JRuby 1.7 but they claim support for 1.9.3 the work-around is not that straight-forward as you can not really gem manage bundler itself (chicken-egg prob). having it in would mean a small disaster 😜

Closing this as C Ruby 2.6 will actually include this...so this will be part of ordinary updates for 2.6 support. I could leave this issue open as that work but it has a lot of discussion about whether this should happen vs knowing we need to just do it.