Cannot build mathematical with native extensions

[samkellett]

When adding the mathematical package to a Gemfile, like so:

source 'https://rubygems.org'

gem 'mathematical', '~> 1.6', '>= 1.6.7'

And run the following command:

bazel run @ruby//:bundle --java_runtime_version=remotejdk_11 -- install

I get an error which I think is that it cannot find lxml2 (which is installed via apt install libxml2-dev on my system):

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

current directory:
/home/samkellett/.cache/bazel/_bazel_samkellett/ba57479d3fc58ccb58079217d732fa88/external/rules_ruby++ruby+ruby/dist/lib/ruby/gems/shared/gems/mathematical-1.6.20/ext/mathematical
/home/samkellett/.cache/bazel/_bazel_samkellett/ba57479d3fc58ccb58079217d732fa88/external/rules_ruby++ruby+ruby/dist/bin/jruby
-I
/home/samkellett/.cache/bazel/_bazel_samkellett/ba57479d3fc58ccb58079217d732fa88/external/rules_ruby++ruby+ruby/dist/lib/ruby/stdlib
extconf.rb
checking for cmake... yes
checking for main() in -lxml2... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
--ruby=/home/samkellett/.cache/bazel/_bazel_samkellett/ba57479d3fc58ccb58079217d732fa88/external/rules_ruby++ruby+ruby/dist/bin/jruby
	--with-xml2lib
	--without-xml2lib
RuntimeError: The compiler failed to generate an executable file.
You have to install development tools first

mkmf.log contains the following info:

find_executable: checking for cmake... -------------------- yes

--------------------

" -o conftest -I/include/universal-java11 -I/home/samkellett/.cache/bazel/_bazel_samkellett/ba57479d3fc58ccb58079217d732fa88/external/rules_ruby++ruby+ruby/dist/lib/ruby/include/ruby/backward -I/home/samkellett/.cache/bazel/_bazel_samkellett/ba57479d3fc58ccb58079217d732fa88/external/rules_ruby++ruby+ruby/dist/lib/ruby/include -I.     -fno-omit-frame-pointer -fno-strict-aliasing  -fexceptions  conftest.c  -L. -L/home/samkellett/.cache/bazel/_bazel_samkellett/ba57479d3fc58ccb58079217d732fa88/external/rules_ruby++ruby+ruby/dist/lib    -m64      "
checked program was:
/* begin */
1: #include "ruby.h"
2:
3: #include <ruby.h>
4: int main(int argc, char **argv)
5: {
6:   return 0;
7: }
/* end */

I've reproduced this on both Linux and macOS.

[p0deje]

Can you please create a repository I could use to reproduce the error?
When it comes to native extensions, rules_ruby would rely on system installed libraries. This applies both to JRuby and MRI. Which one do you use?

[samkellett]

Sure, I've created a PR here: #228

I just add mathematical to the Gemfile and then I run:

bazel run @ruby//:bundle --java_runtime_version=remotejdk_11 -- install

Inside rules_ruby/examples/gem.

It fails initially looking for cmake which is fixed by doing brew install cmake on macOS. But then it fails on the next check which I think is looking for libxml2 which is provided by the macOS SDK but also brew install libxml2 and neither of those work.

I have the same issue when trying it on Linux and doing apt install libxml2-dev as well.

Lemme know if that's not enough to reproduce the issue on your side.

Cheers,
Sam

[samkellett]

This applies both to JRuby and MRI. Which one do you use?

I'm using jruby-9.4.10.0 but am not tied to that if another on works instead.

[p0deje]

@samkellett You can't really use the mathematical gem on JRuby, it has native C extensions that are only compatible with MRI.

[samkellett]

Ah I see! Thanks for the heads up. What do I put as .ruby-version to use MRI?

[samkellett]

To answer my question, 3.4.1 worked and I was able to run bundle -- install with that matematical extension! Thanks for the help

[samkellett]

Hey sorry I have a follow-up question related to this, I hope you don't mind me reusing this ticket!

So now that bazel run @ruby//:bundle -- install works by finding the cmake I have installed on the command line and it updates the Gemfile.lock as expected.

However when I do bazel build @bundle//... it fails because I cannot find the cmake I have installed:

checking for cmake... no

Is there any way to run ruby.bundle_fetch and expose $PATH? Or, even better, to expose the tools provided by a toolchain like @rules_foreign_cc//toolchains:current_cmake_toolchain. Something like how rules_rust seems to support: https://github.com/bazel-contrib/rules_foreign_cc/blob/ed5f7c974f67d0201be8d87780dcb0fe9557ea6d/examples/MODULE.bazel#L46

[p0deje]

So now that bazel run @ruby//:bundle -- install works by finding the cmake I have installed on the command line and it updates the Gemfile.lock as expected.

That's probably because bazel run is not hermetic (#221), while bazel build is. You should avoid running bundler directly, and only use it to update the lockfile via bazel run @ruby//:bundle -- lock --gemfile=$(pwd)/rb/Gemfile. Frankly, whenever I had to use bazel run with bundler, everything got messy so I had to bazel clean --expunge in the end.

Is there any way to run ruby.bundle_fetch and expose $PATH? Or, even better, to expose the tools provided by a toolchain like @rules_foreign_cc//toolchains:current_cmake_toolchain. Something like how rules_rust seems to support: https://github.com/bazel-contrib/rules_foreign_cc/blob/ed5f7c974f67d0201be8d87780dcb0fe9557ea6d/examples/MODULE.bazel#L46

I don't see a reason why it would not be possible, but I am not sure how you would expose dependencies (e.g. libxml2). Are you up to diving deeper into the topic and potentially preparing a PR adding these features?