Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Really slow r.js performance depending on build #3757

Closed
danielmewes opened this issue Feb 11, 2015 · 7 comments
Closed

Really slow r.js performance depending on build #3757

danielmewes opened this issue Feb 11, 2015 · 7 comments
Assignees
@AtnNn
Labels
pr:high tp:performance
Milestone
1.16.2

Comments

@danielmewes
Copy link
Member

Noticed while trying to reproduce #3703 .

Compiling from source on Ubuntu precise:

r.table('en2').merge( r.js("(function(row) {return {mime2: row.mime}; })") ).coerceTo('ARRAY').count()

takes 0.85 seconds.

Using our official package, the same query takes 11.5 seconds. It uses 100% of all 8 CPU cores while running.

The same is true with our CentOS 6 binary.

./configure --allow-fetch output:

* Detecting system configuration
Bash:                           4.2.25(1)-release
Use ccache:                     no
C++ Compiler:                   GCC 4.6.3 (/usr/bin/c++)
Host System:                    x86_64-linux-gnu
Build System:                   Linux 3.13.0-29-generic x86_64
Cross-compiling:                no
Host Operating System:          Linux
Build Architecture:             x86_64
C++11:                          ok
Precompiled web assets:         no
Protobuf compiler:              /usr/bin/protoc
python:                         python 2.7.3
Node.js package manager:        external/node_0.10.15
LESS css:                       lessc 1.3.1
CoffeeScript:                   external/coffee-script_1.7.1
Handlebars:                     external/handlebars_1.3.0
Browserify:                     external/browserify_3.24.13
bluebird:                       external/bluebird_2.3.2
wget:                           /usr/bin/wget
curl:                           /usr/bin/curl
Google Test:                    external/gtest_1.6.0
termcap:                        -ltermcap
boost_system:                   -lboost_system
ssl:                            -lssl
protobuf:                       -lprotobuf
v8:                             -lv8
RE2:                            external/re2_20140111
z:                              -lz
crypto:                         -lcrypto
curl:                           -lcurl
Use pre-3.19 v8 API:            yes
malloc:                         jemalloc
jemalloc (static):              external/jemalloc_3.6.0
Test protobuf:                  ok
Test boost:                     ok
Installation prefix:            /usr/local
Configuration prefix:           /usr/local/etc
Runtime data prefix:            /usr/local/var
* Wrote configuration to config.mk
@danielmewes danielmewes added this to the 2.0 milestone Feb 11, 2015
@danielmewes danielmewes mentioned this issue Feb 11, 2015
Closed
@danielmewes
Copy link
Member Author

Seems to be related to the V8 version.
./configure --allow-fetch --static v8 fetches version 3.22.24.17 and is slow.

Changing mk/support/pkg/v8.sh to fetch 3.7.12.22 makes the build fast.

Interestingly it's even faster when linking with V8 3.7.12.22 statically, even though that's the same version I've installed on my system. The same query now only takes 0.21s instead of 0.85s with the system-wide libv8 of the same version.

@danielmewes
Copy link
Member Author

Weirdly enough linking with V8 3.22.24.17 statically is fast on deadshot (one of our servers), and even faster (0.38s) than linking with the system's 3.7.12.22 (~0.9s).

@danielmewes
Copy link
Member Author

The expensive V8 call is

v8::Context::New(v8::Isolate::GetCurrent())

This call takes 10-30 ms in the slow case.

@danielmewes
Copy link
Member Author

It seems like V8's snapshot feature that exists for speeding up context creation isn't working on my computer for some reason, even though it seems to be enabled when building V8.

It's working fine on deadshot.

@AtnNn
Copy link
Member

AtnNn commented Feb 11, 2015

Perhaps upgrading to a newer version of v8 or changing how we build v8 would help with this issue (#3423, #3472).

@danielmewes danielmewes modified the milestones: 1.16.x, 2.0 Feb 11, 2015
@danielmewes
Copy link
Member Author

@AtnNn found the problem: V8 compiles first without a snapshot, and then in a second step generates the snapshot. Whether the snapshot is used or not depends on which subset of the .o files it generates are linked.
We currently link all of them, making it depend on the linking order.

@AtnNn already has fix for this in his branch for #3423. He's going to backport it to v1.16.x.

@AtnNn
Copy link
Member

AtnNn commented Feb 13, 2015

Fixed in next (f3a38a9) and v1.16.x (9292feb)

@AtnNn AtnNn closed this as completed Feb 13, 2015
@AtnNn AtnNn modified the milestones: 1.16.x, 1.16.2 Feb 13, 2015
@AtnNn AtnNn mentioned this issue Feb 16, 2015
Closed
@danielmewes danielmewes mentioned this issue Mar 24, 2015
Closed
nighelles pushed a commit that referenced this issue Sep 15, 2016
Fixing eq_join for #6108
a940ede 
Implementation in nighelles/6108, Review #3757 by @danielmewes
AtnNn pushed a commit that referenced this issue Jan 28, 2017
@AtnNn
Fixing eq_join for #6108
7b57928 
Implementation in nighelles/6108, Review #3757 by @danielmewes
AtnNn pushed a commit that referenced this issue Feb 7, 2017
@AtnNn
Fixing eq_join for #6108
49edf90 
Implementation in nighelles/6108, Review #3757 by @danielmewes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AtnNn AtnNn

Labels
pr:high tp:performance
Projects
None yet
Milestone
1.16.2
Development

No branches or pull requests
2 participants
@danielmewes @AtnNn