Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash in worker process using table_fuzzer #4879

Closed
danielmewes opened this issue Sep 23, 2015 · 4 comments
Closed

Crash in worker process using table_fuzzer #4879

danielmewes opened this issue Sep 23, 2015 · 4 comments
Assignees
@danielmewes
Labels
tp:bug
Milestone
2.1.5

Comments

@danielmewes
Copy link
Member

I'm not sure if this has anything to do with what the table fuzzer does or if it's a more general problem.

On next, I changed the following line in the table_fuzzer.py script:

diff --git a/test/interface/table_fuzzer.py b/test/interface/table_fuzzer.py
index 21f6a8f..5da959a 100755
--- a/test/interface/table_fuzzer.py
+++ b/test/interface/table_fuzzer.py
@@ -331,7 +331,7 @@ def run_changefeed(query, host, port):
         end_time = time.time() + duration
         try:
             conn = r.connect(host, port)
-            feed = query.changes().run(conn)
+            feed = query.changes().filter(r.js("(function(x) { return true; })")).run(conn)
             while time.time() < end_time:
                 try:
                     feed.next(wait=duration)

I then ran the fuzzer as follows:

$ ../table_fuzzer.py --changefeeds --servers 3

This caused a bunch of errors like this one:

Feed resulted in Exception: ReqlInternalError(u'Javascript query `(function(x) { return true; })` caused a crash in a worker process.',)

I used a release-mode server for this.

@Tryneus any ideas?
@danielmewes danielmewes added this to the 2.1.x milestone Sep 23, 2015
@danielmewes
Copy link
Member Author

Error and backtrace:

2015-09-23T12:17:14.978268058 33.971848s error: Error in src/extproc/js_job.cc at line 711:
2015-09-23T12:17:14.984473802 33.978048s error: Guarantee failed: [!key.IsEmpty() && !val.IsEmpty()] 
2015-09-23T12:17:14.988411583 33.981986s error: Backtrace:
2015-09-23T12:17:15.437433606 34.431007s info: Table f5626d31-d85f-45ed-9c83-3632b940d295: Configuration is changing.
2015-09-23T12:17:15.520931455 34.514506s error: Wed Sep 23 12:17:14 2015

1: backtrace_t::backtrace_t() at 0xc3f1b0 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
2: lazy_backtrace_formatter_t::lazy_backtrace_formatter_t() at 0xc3f8c3 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
3: format_backtrace(bool) at 0xc3ec14 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
4: report_fatal_error(char const*, int, char const*, ...) at 0x7c3793 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
5: js_from_datum(ql::datum_t const&, std::string*) at 0xa04956 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
6: js_from_datum(ql::datum_t const&, std::string*) at 0xa04925 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
7: js_from_datum(ql::datum_t const&, std::string*) at 0xa0461b (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
8: js_from_datum(ql::datum_t const&, std::string*) at 0xa04925 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
9: js_from_datum(ql::datum_t const&, std::string*) at 0xa0461b (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
10: js_from_datum(ql::datum_t const&, std::string*) at 0xa04925 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
11: js_from_datum(ql::datum_t const&, std::string*) at 0xa04925 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
12: run_js_func(v8::Handle<v8::Function>, std::vector<ql::datum_t, std::allocator<ql::datum_t> > const&, std::string*) at 0xa043da (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
13: js_env_t::call(unsigned long, std::vector<ql::datum_t, std::allocator<ql::datum_t> > const&, ql::configured_limits_t const&) at 0xa03a42 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
14: run_call(read_stream_t*, write_stream_t*, js_env_t*, unsigned long) at 0xa03544 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
15: js_job_t::worker_fn(read_stream_t*, write_stream_t*) at 0xa01f85 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
16: worker_run_t::main_loop() at 0x9f5122 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
17: spawner_run_t::main_loop() at 0x9f502f (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
18: extproc_spawner_t::fork_spawner() at 0x9f4a99 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
19: extproc_spawner_t::extproc_spawner_t() at 0x9f4932 (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
20: main_rethinkdb_serve(int, char**) at 0x7f8d1e (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
21: main+0x33e at 0x7c40de (/home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb)
22: __libc_start_main+0xed at 0x7f54aa66476d (/lib/x86_64-linux-gnu/libc.so.6)
23: /home/ssd3/daniel/rethinkdb/build/release_clang/rethinkdb() [0x6e17d9] at 0x6e17d9 ()
2015-09-23T12:17:15.526357864 34.519932s error: Exiting.

@danielmewes
Copy link
Member Author

I investigated this a bit more, and the actual error seems to be: "r.binary data cannot be used in r.js."

Because of this guarantee (and a similar one a few lines up), that error never reaches the user if the illegal value is inside an array of object.

@danielmewes danielmewes assigned danielmewes and unassigned Tryneus Sep 23, 2015
@danielmewes
Copy link
Member Author

Reassigning to myself. I'll prepare a fix so we pass the error through properly.

danielmewes pushed a commit that referenced this issue Sep 23, 2015
Pass datum decoding errors in r.js helper up.
3bb9144 
Fixes #4879.
@danielmewes
Copy link
Member Author

Fixed in next 1fc1004 and cherry-picked into v2.1.x 3bb9144 .

@danielmewes danielmewes modified the milestones: 2.1.x, 2.1.5 Oct 1, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@danielmewes danielmewes

Labels
tp:bug
Projects
None yet
Milestone
2.1.5
Development

No branches or pull requests
2 participants
@danielmewes @Tryneus