Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building as static library fails #686

Closed
hoho opened this issue Feb 1, 2015 · 23 comments
Closed

Building as static library fails #686

hoho opened this issue Feb 1, 2015 · 23 comments
Labels
build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. confirmed-bug Issues with confirmed bugs.

Comments

@hoho
Copy link
Contributor

hoho commented Feb 1, 2015

This issue is probably not io.js-specific, and it might be the one to be reported to gyp, but I'll try here first.

I'm trying to build io.js as static library, I have just changed 'type': 'executable' to 'type': 'static_library' in node.gyp.

I get the following error in the end:

error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: unknown option character `W' in: -Wl,-force_load,/Users/hoho/iojs/out/Release/libopenssl.a
Usage: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -static [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-sacLT] [-no_warning_for_no_symbols]
Usage: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -dynamic [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-o output] [-install_name name] [-compatibility_version #] [-current_version #] [-seg1addr 0x#] [-segs_read_only_addr 0x#] [-segs_read_write_addr 0x#] [-seg_addr_table <filename>] [-seg_addr_table_filename <file_system_path>] [-all_load] [-noall_load]
make[1]: *** [/Users/hoho/iojs/out/Release/libiojs.a] Error 1
make: *** [iojs] Error 2

OS X Yosemite 10.10.2.

@Fishrock123
Copy link
Member

Is this related to nodejs/roadmap#9 ?

I don't think this is currently possible.

@vinz243
Copy link

vinz243 commented Feb 1, 2015

Why static and not shared? With shared I know it works.

See #537

For now, you can try patching node.gyp by hand: set the type of the 'iojs' target in node.gyp to 'shared_library' ('static_library' won't work) and drop src/node_main.cc from the sources

@hoho
Copy link
Contributor Author

hoho commented Feb 1, 2015

Yes, building shared library works and I'll be using it for a while. It's just shared library also requires -fPIC on Linux. And gyp is not passing it by default. And the result with -fPIC is probably not as fast.

@vinz243
Copy link

vinz243 commented Feb 1, 2015

I've just seen this thread http://stackoverflow.com/q/17739095/2533082. Might help you

@Fishrock123
Copy link
Member

Sounds like it was solved? Maybe we need a guide for this.

@hoho
Copy link
Contributor Author

hoho commented Feb 2, 2015

It was not solved! It is possible to build a shared library, but it is not possible to build a static one because of the error above.

@Fishrock123 Fishrock123 reopened this Feb 2, 2015
@Fishrock123
Copy link
Member

cc @bnoordhuis

(Sorry, my c++ knowledge is still pretty vague)

@hoho
Copy link
Contributor Author

hoho commented Feb 15, 2015

Just a few more notes.
I tried to build a static library on Linux (Ubuntu 14.04). The build process went fine, but the resulting libiojs.a is only 3 megabytes in size (that's obviously not everything) and when I tried to build something with it, I've got plenty of errors like:

string_bytes.cc:(.text._ZN4node11StringBytes5WriteEPN2v87IsolateEPcmNS1_6HandleINS1_5ValueEEENS_8encodingEPi+0x3b7): undefined reference to `v8::String::Value::~Value()'

Not just this reference, plenty of other ones too.

I tried to build a shared library on Linux too, I've got tons of other errors like:

/home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a(debug-support.o):(.data.v8dbg_frametype_InternalFrame+0x0): multiple definition of `v8dbg_frametype_InternalFrame'
/home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a(debug-support.o):(.data.v8dbg_frametype_InternalFrame+0x0): first defined here

Summarizing, from two platforms I'm interested in, I can only build working shared library on OSX and nothing on Linux.

I would appreciate any helpful workarounds (at least to have a shared library on both platforms).

@bnoordhuis
Copy link
Member

@hoho You probably also need to link in the other static libraries that reside in out/Release (and specified in the right order if linking with --as-needed.)

I'm not sure why you get that error with the shared library. Maybe object files are getting linked in twice?

@hoho
Copy link
Contributor Author

hoho commented Feb 16, 2015

@bnoordhuis
as for the static library — I'll try to figure out what the right order is (definitely not alphabetic, because I tried it already).
as for the shared library — I'm not linking something specially and it works on OSX, here is my diff from the mainline (I checked out v1.2.0):

diff --git a/common.gypi b/common.gypi
index ea7779e..47484db 100644
--- a/common.gypi
+++ b/common.gypi
@@ -47,6 +47,7 @@
   },

   'target_defaults': {
+    'cflags': ['-fPIC'],
     'default_configuration': 'Release',
     'configurations': {
       'Debug': {
diff --git a/node.gyp b/node.gyp
index 8506b28..987df56 100644
--- a/node.gyp
+++ b/node.gyp
@@ -75,7 +75,7 @@
   'targets': [
     {
       'target_name': 'iojs',
-      'type': 'executable',
+      'type': 'shared_library',

       'dependencies': [
         'node_js2c#host',
@@ -102,7 +102,7 @@
         'src/node_file.cc',
         'src/node_http_parser.cc',
         'src/node_javascript.cc',
-        'src/node_main.cc',
+#        'src/node_main.cc',
         'src/node_os.cc',
         'src/node_v8.cc',
         'src/node_v8_platform.cc',

@bnoordhuis
Copy link
Member

@hoho RE: linking order, if library A defines a symbol that library B needs, then B should come before A on the command line. E.g. libv8.a comes before libv8_base.a because the former uses symbols from the latter but not the other way around. You can also try linking with --no-as-needed

@hoho
Copy link
Contributor Author

hoho commented Feb 16, 2015

@bnoordhuis it looks like I've managed to link statically on Linux, it wasn't a trivial task though, that's what linker needs (in addition to everything else) to link with C (not C++) application on Linux (Ubuntu 14.04):

    -Wl,--whole-archive ../iojs/out/Release/libiojs.a -Wl,--no-whole-archive \
    ../iojs/out/Release/libuv.a \
    ../iojs/out/Release/libcares.a \
    ../iojs/out/Release/libopenssl.a \
    ../iojs/out/Release/libzlib.a \
    ../iojs/out/Release/libhttp_parser.a \
    ../iojs/out/Release/libv8_base.a \
    ../iojs/out/Release/libv8_libbase.a \
    ../iojs/out/Release/libv8_nosnapshot.a \
    -lstdc++ -lm -ldl

I haven't tested if the result is stable enough — just tried it with simple scripts, so I'll do further testing with many requires and native modules.

And, inspired by the elaboration above, I've managed to build static library on OSX too. It's a bit more tricky though. First of all, OTHER_LDFLAGS ('-Wl,-force_load,<(PRODUCT_DIR)/libopenssl.a' and '-Wl,-force_load,<(V8_BASE)') should be removed from node.gyp. Second of all, to link with the resulted static libraries (again, I link them into C application) the following should be added to the linker invocation

    -Wl,-force_load,../iojs/out/Release/libiojs.a \
    ../iojs/out/Release/libuv.a \
    ../iojs/out/Release/libcares.a \
    ../iojs/out/Release/libopenssl.a \
    ../iojs/out/Release/libzlib.a \
    ../iojs/out/Release/libhttp_parser.a \
    ../iojs/out/Release/libv8_base.a \
    ../iojs/out/Release/libv8_libbase.a \
    ../iojs/out/Release/libv8_nosnapshot.a \
    -lstdc++.6.0.9 -lm -ldl"

The funny thing is the necessity to explicitly add libstdc++ version because of some strangle linker bug (OSX has a symbolic link libstdc++.dylib to libstdc++.6.0.9.dylib, but the linker doesn't follow this symbolic link).

--whole-archive/-force_load is required to include native modules to the result. Otherwise everything dies at runtime with an exception like:

node.js:724
  var ContextifyScript = process.binding('contextify').ContextifyScript;
                                 ^
Error: No such module: contextify
    at Error (native)
    at node.js:724:34

Further testing might show that I need to force_load some other libraries too.

So, in the end, to fix this issue I might create a pull request to modify node.gyp the following way:

  • Add a variable iojs_target_type with the default value equals to executable.
  • Conditionally skip -force_loads in xcode_settings when iojs_target_type equals to static_library.

Optionally, I could describe the contents of this comment (about how to link with io.js as the static library) in some appropriate place for the future generations :). I don't know what place is the best though.

Is it ok? Does this pull request have a chance to be accepted?

And there still is an issue about errors while building shared library on Linux.

@bnoordhuis
Copy link
Member

First of all, OTHER_LDFLAGS ('-Wl,-force_load,<(PRODUCT_DIR)/libopenssl.a' and '-Wl,-force_load,<(V8_BASE)') should be removed from node.gyp.

Can you explain why that is necessary? Was that the cause of the redefinition errors?

@hoho
Copy link
Contributor Author

hoho commented Feb 16, 2015

It's not about redefinition. Redefinition is a shared-library-on-linux bug. I did not dig into it.

The initial cause of this issue is in my first message:

unknown option character `W' in: -Wl,-force_load,/Users/hoho/iojs/out/Release/libopenssl.a

— it's because it's not supported to bundle one one static library into another — the result of building as a static library is actually a bunch of static libraries. -force_load should be removed in this case, it should be moved into the place where you link a binary with this bunch of static libraries.

@hoho
Copy link
Contributor Author

hoho commented Feb 16, 2015

By «should be removed» I meant conditionally removed. It's still required for the executable and for the shared library. But it's unsupported for the static library.

@hoho
Copy link
Contributor Author

hoho commented Feb 17, 2015

And back to the redefinition problem. That's how the link command looks like to build the executable:

flock /home/hoho/iojs/out/Release/linker.lock g++ -pthread -rdynamic -m64 -Wl,--whole-archive /home/hoho/iojs/out/Release/libopenssl.a -Wl,--no-whole-archive -Wl,-z,noexecstack -Wl,--whole-archive /home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a -Wl,--no-whole-archive -pthread -o /home/hoho/iojs/out/Release/iojs -Wl,--start-group /home/hoho/iojs/out/Release/obj.target/iojs/src/debug-agent.o /home/hoho/iojs/out/Release/obj.target/iojs/src/async-wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/fs_event_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/cares_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/handle_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_buffer.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_constants.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_contextify.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_file.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_http_parser.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_javascript.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_main.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_os.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_v8.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_v8_platform.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_stat_watcher.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_watchdog.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_zlib.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_i18n.o /home/hoho/iojs/out/Release/obj.target/iojs/src/pipe_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/signal_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/smalloc.o /home/hoho/iojs/out/Release/obj.target/iojs/src/spawn_sync.o /home/hoho/iojs/out/Release/obj.target/iojs/src/string_bytes.o /home/hoho/iojs/out/Release/obj.target/iojs/src/stream_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/tcp_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/timer_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/tty_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/process_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/udp_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/uv.o /home/hoho/iojs/out/Release/obj.target/iojs/src/util.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_crypto.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_crypto_bio.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_crypto_clienthello.o /home/hoho/iojs/out/Release/obj.target/iojs/src/tls_wrap.o /home/hoho/iojs/out/Release/obj.target/deps/cares/libcares.a /home/hoho/iojs/out/Release/obj.target/deps/openssl/libopenssl.a /home/hoho/iojs/out/Release/obj.target/deps/zlib/libzlib.a /home/hoho/iojs/out/Release/obj.target/deps/http_parser/libhttp_parser.a /home/hoho/iojs/out/Release/obj.target/deps/uv/libuv.a /home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a /home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_libbase.a /home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_nosnapshot.a -Wl,--end-group -lrt -lm -ldl

And that's the one for the shared library:

flock /home/hoho/iojs/out/Release/linker.lock g++ -shared -pthread -rdynamic -m64 -Wl,--whole-archive /home/hoho/iojs/out/Release/libopenssl.a -Wl,--no-whole-archive -Wl,-z,noexecstack -Wl,--whole-archive /home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a -Wl,--no-whole-archive -pthread -Wl,-soname=libiojs.so -o /home/hoho/iojs/out/Release/obj.target/libiojs.so -Wl,--whole-archive /home/hoho/iojs/out/Release/obj.target/iojs/src/debug-agent.o /home/hoho/iojs/out/Release/obj.target/iojs/src/async-wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/fs_event_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/cares_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/handle_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_buffer.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_constants.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_contextify.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_file.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_http_parser.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_javascript.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_os.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_v8.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_v8_platform.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_stat_watcher.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_watchdog.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_zlib.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_i18n.o /home/hoho/iojs/out/Release/obj.target/iojs/src/pipe_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/signal_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/smalloc.o /home/hoho/iojs/out/Release/obj.target/iojs/src/spawn_sync.o /home/hoho/iojs/out/Release/obj.target/iojs/src/string_bytes.o /home/hoho/iojs/out/Release/obj.target/iojs/src/stream_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/tcp_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/timer_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/tty_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/process_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/udp_wrap.o /home/hoho/iojs/out/Release/obj.target/iojs/src/uv.o /home/hoho/iojs/out/Release/obj.target/iojs/src/util.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_crypto.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_crypto_bio.o /home/hoho/iojs/out/Release/obj.target/iojs/src/node_crypto_clienthello.o /home/hoho/iojs/out/Release/obj.target/iojs/src/tls_wrap.o /home/hoho/iojs/out/Release/obj.target/deps/cares/libcares.a /home/hoho/iojs/out/Release/obj.target/deps/openssl/libopenssl.a /home/hoho/iojs/out/Release/obj.target/deps/zlib/libzlib.a /home/hoho/iojs/out/Release/obj.target/deps/http_parser/libhttp_parser.a /home/hoho/iojs/out/Release/obj.target/deps/uv/libuv.a /home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_base.a /home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_libbase.a /home/hoho/iojs/out/Release/obj.target/deps/v8/tools/gyp/libv8_nosnapshot.a -Wl,--no-whole-archive -lrt -lm -ldl

The difference is that the executable is being linked with everything wrapped into -Wl,--start-group/-Wl,--end-group. And for the shared library it's -Wl,--whole-archive/-Wl,--no-whole-archive. This (in a combination with explicitly declared '-Wl,--whole-archive <(V8_BASE) -Wl,--no-whole-archive' and '-Wl,--whole-archive <(PRODUCT_DIR)/libopenssl.a -Wl,--no-whole-archive' from node.gyp) gives redefinitions.

Finally, I'm not sure I want to dig deep into the shared library problem right now (we could create another issue for it). But I'm definitely ready to make a pull request to fix the static library issue and make some docs about linking statically in some appropriate place.

This was referenced Feb 20, 2015
@mscdex mscdex added confirmed-bug Issues with confirmed bugs. c++ Issues and PRs that require attention from people who are familiar with C++. build Issues and PRs related to build files or the CI. labels Mar 12, 2015
This was referenced Jul 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. confirmed-bug Issues with confirmed bugs.
Projects
None yet
Development

No branches or pull requests

6 participants