New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Segmentation fault in table_fuzzer.py
when killing/reviving servers
#4972
Comments
Just got another crash, probably the same problem but a slightly different code path: error: Error in ../src/arch/runtime/thread_pool.cc at line 359:
error: Segmentation fault from reading the address (nil).
error: Backtrace:
error: Tue Oct 20 16:32:42 2015
1 [0x1d1fbf8]: rethinkdb_backtrace(void**, int) at rethinkdb_backtrace.cc:101
2 [0x19085c1]: backtrace_t::backtrace_t() at backtrace.cc:203
3 [0x1908c2f]: lazy_backtrace_formatter_t::lazy_backtrace_formatter_t() at backtrace.cc:283
4 [0x190850f]: format_backtrace(bool) at backtrace.cc:198
5 [0x18fc3ce]: report_fatal_error(char const*, int, char const*, ...) at errors.cc:83
6 [0x172eac8]: linux_thread_pool_t::fatal_signal_handler(int, siginfo*, void*) at thread_pool.cc:359
7 [0x7f2b3fd7acb0]: /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0) [0x7f2b3fd7acb0] at 0x7f2b3fd7acb0 (/lib/x86_64-linux-gnu/libpthread.so.0)
8 [0x18aebd0]: /home/ssd2/grey/rethinkdb/build/debug/rethinkdb() [0x18aebd0] at 0x18aebd0 ()
9 [0x18ae5a4]: mailbox_manager_t::on_message(connectivity_cluster_t::connection_t*, auto_drainer_t::lock_t, read_stream_t*) at mailbox.cc:245
10 [0x183cb14]: connectivity_cluster_t::run_t::handle(keepalive_tcp_conn_stream_t*, boost::optional<peer_id_t>, boost::optional<peer_address_t>, auto_drainer_t::lock_t, bool*) at cluster.cc:1192
11 [0x183a850]: connectivity_cluster_t::run_t::connect_to_peer(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*) at cluster.cc:333
12 [0x1852f88]: std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)>::operator()(connectivity_cluster_t::run_t*, peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*) const at functional:551
13 [0x1850eab]: void std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)>::__call_c<void, long&, 0, 1, 2, 3, 4, 5, 6>(std::tuple<long&>&&, std::_Index_tuple<0, 1, 2, 3, 4, 5, 6>) const at functional:1155
14 [0x184e267]: void std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)>::operator()<long&, void>(long&) const at functional:1220
15 [0x184a611]: pmap_runner_one_arg_t<std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)>, long>::operator()() at pmap.hpp:21
16 [0x1855c10]: callable_action_instance_t<pmap_runner_one_arg_t<std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)>, long> >::run_action() at callable_action.hpp:31
17 [0x1729928]: callable_action_wrapper_t::run() at runtime_utils.cc:43
18 [0x1731b2f]: coro_t::run() at coroutines.cc:207
19 [0x184a5db]: void coro_t::spawn_now_dangerously<pmap_runner_one_arg_t<std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)>, long> >(pmap_runner_one_arg_t<std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)>, long>&&) at coroutines.hpp:53
20 [0x1846e03]: void pmap<std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)> >(long, long, std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)> const&) at pmap.hpp:43
21 [0x1842da5]: void pmap<std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)> >(long, std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t const*, int, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, co_semaphore_t*)> (connectivity_cluster_t::run_t*, peer_address_t const*, std::_Placeholder<1>, boost::optional<peer_id_t>, auto_drainer_t::lock_t, bool*, static_semaphore_t*)> const&) at pmap.hpp:54
22 [0x183ad0b]: connectivity_cluster_t::run_t::join_blocking(peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t) at cluster.cc:376
23 [0x1856969]: std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t)>::operator()(connectivity_cluster_t::run_t*, peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t) const at functional:551
24 [0x18563aa]: void std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t)> (connectivity_cluster_t::run_t*, peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t)>::__call<void, , 0, 1, 2, 3>(std::tuple<>&&, std::_Index_tuple<0, 1, 2, 3>) at functional:1146
25 [0x185604c]: void std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t)> (connectivity_cluster_t::run_t*, peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t)>::operator()<, void>() at functional:1206
26 [0x1855c2e]: callable_action_instance_t<std::_Bind<std::_Mem_fn<void (connectivity_cluster_t::run_t::*)(peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t)> (connectivity_cluster_t::run_t*, peer_address_t, boost::optional<peer_id_t>, auto_drainer_t::lock_t)> >::run_action() at callable_action.hpp:31
27 [0x1729928]: callable_action_wrapper_t::run() at runtime_utils.cc:43
28 [0x1731b2f]: coro_t::run() at coroutines.cc:207 |
gdb analysis of the core file shows a completely different backtrace: #3 0x0000000001b0b8c0 in operator() (this=0x7f2b0c7f1700, sub=0x7f2b28187c00) at ../src/rdb_protocol/changefeed.cc:2308
#4 0x0000000001b26f20 in std::_Function_handler<void(ql::changefeed::range_sub_t*), ql::changefeed::msg_visitor_t::operator()(const ql::changefeed::msg_t::change_t&) const::<lambda(ql::changefeed::range_sub_t*)> >::_M_invoke(const std::_Any_data &, ql::changefeed::range_sub_t *) (__functor=..., __args#0=0x7f2b28187c00) at /usr/include/c++/4.6/functional:1778
#5 0x0000000001b3a071 in std::function<void(ql::changefeed::range_sub_t*)>::operator()(ql::changefeed::range_sub_t *) const (this=0x7f2b0ab04e90, __args#0=0x7f2b28187c00)
at /usr/include/c++/4.6/functional:2161
#6 0x0000000001b28ca8 in ql::changefeed::feed_t::each_sub_in_vec_cb<ql::changefeed::range_sub_t>(const std::function<void(ql::changefeed::range_sub_t*)> &, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > &, const std::vector<int, std::allocator<int> > &, int) (this=0x7f2b0d7ff200, f=...,
vec=std::vector of length 25, capacity 25 = {...}, subscription_threads=std::vector of length 1, capacity 1 = {...}, i=0) at ../src/rdb_protocol/changefeed.cc:2933
#7 0x0000000001b5ec3b in std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>::operator()(ql::changefeed::feed_t *, const std::function<void(ql::changefeed::range_sub_t*)> &, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > &, const std::vector<int, std::allocator<int> > &, int) const (this=0x7f2b0ab04800,
__object=0x7f2b0d7ff200, __args#0=..., __args#1=std::vector of length 25, capacity 25 = {...}, __args#2=std::vector of length 1, capacity 1 = {...}, __args#3=0)
at /usr/include/c++/4.6/functional:551
#8 0x0000000001b58a07 in std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)>::__call_c<void, long&, 0, 1, 2, 3, 4>(std::tuple<long&> &, std::_Index_tuple<0, 1, 2, 3, 4>) const (this=0x7f2b0ab04800,
__args=empty std::tuple) at /usr/include/c++/4.6/functional:1155
#9 0x0000000001b512c5 in std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)>::operator()<long int&, void>(void) const (this=0x7f2b0ab04800) at /usr/include/c++/4.6/functional:1219
#10 0x0000000001b46f09 in pmap_runner_one_arg_t<std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)>, long int>::operator()(void) (this=0x7f2b0ab04750) at ../src/concurrency/pmap.hpp:20
#11 0x0000000001b3a333 in pmap<std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*> >&, const std::vector<int>&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*> > >, std::reference_wrapper<const std::vector<int> >, std::_Placeholder<1>)> >(int64_t, int64_t, const std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)> &) (begin=0,
end=1, c=...) at ../src/concurrency/pmap.hpp:47
#12 0x0000000001b28e00 in pmap<std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*> >&, const std::vector<int>&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*> > >, std::reference_wrapper<const std::vector<int> >, std::_Placeholder<1>)> >(int64_t, const std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)> &) (count=1, c=...)
at ../src/concurrency/pmap.hpp:53
#13 0x0000000001b1a01a in ql::changefeed::feed_t::each_sub_in_vec<ql::changefeed::range_sub_t>(const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > &, rwlock_in_line_t *, const auto_drainer_t::lock_t &, const std::function<void(ql::changefeed::range_sub_t*)> &) (this=0x7f2b0d7ff200,
vec=std::vector of length 25, capacity 25 = {...}, spot=0x7f2b0ab048f0, lock=..., f=...) at ../src/rdb_protocol/changefeed.cc:2916
#14 0x0000000001afd95c in ql::changefeed::feed_t::each_range_sub(const auto_drainer_t::lock_t &, const std::function<void(ql::changefeed::range_sub_t*)> &) (this=0x7f2b0d7ff200, lock=...,
f=...) at ../src/rdb_protocol/changefeed.cc:2942 This looks a little more useful (although it's hard to say if it's more correct than the other backtraces). Inspecting |
Are these crashes on |
These were on |
I got one of these on the Backtrace from gdb: #0 0x0000000001604ba5 in linux_thread_pool_t::fatal_signal_handler (signum=11, info=0x7fb00903e570, data=0x7fb00903e440)
at ../src/arch/runtime/thread_pool.cc:359
#1 <signal handler called>
#2 0x00000000019017b5 in operator() (this=0x7fafe9d63860, sub=0x7fb009217800)
at ../src/rdb_protocol/changefeed.cc:2306
#3 0x000000000191deae in std::_Function_handler<void(ql::changefeed::range_sub_t*), ql::changefeed::msg_visitor_t::operator()(const ql::changefeed::msg_t::change_t&) const::<lambda(ql::changefeed::range_sub_t*)> >::_M_invoke(const std::_Any_data &, ql::changefeed::range_sub_t *) (__functor=..., __args#0=0x7fb009217800)
at /usr/include/c++/4.6/functional:1778
#4 0x0000000001931cfb in std::function<void(ql::changefeed::range_sub_t*)>::operator()(ql::changefeed::range_sub_t *) const (this=0x7faff6d84e90, __args#0=0x7fb009217800)
at /usr/include/c++/4.6/functional:2161
#5 0x000000000191fc36 in ql::changefeed::feed_t::each_sub_in_vec_cb<ql::changefeed::range_sub_t>(const std::function<void(ql::changefeed::range_sub_t*)> &, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > &, const std::vector<int, std::allocator<int> > &, int) (this=0x7fafe2bfcf00, f=..., vec=std::vector of length 25, capacity 25 = {...}, subscription_threads=std::vector of length 1, capacity 1 = {...}, i=0)
at ../src/rdb_protocol/changefeed.cc:2932
#6 0x00000000019555bb in std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>::operator()(ql::changefeed::feed_t *, const std::function<void(ql::changefeed::range_sub_t*)> &, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > &, const std::vector<int, std::allocator<int> > &, int) const (this=0x7faff6d84800, __object=0x7fafe2bfcf00, __args#0=..., __args#1=std::vector of length 25, capacity 25 = {...}, __args#2=std::vector of length 1, capacity 1 = {...}, __args#3=0)
at /usr/include/c++/4.6/functional:551
#7 0x000000000194fb4b in std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)>::__call_c<void, long&, 0, 1, 2, 3, 4>(std::tuple<long&> &, std::_Index_tuple<0, 1, 2, 3, 4>) const (this=0x7faff6d84800, __args=empty std::tuple)
at /usr/include/c++/4.6/functional:1155
#8 0x0000000001948a71 in std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)>::operator()<long int&, void>(void) const (this=0x7faff6d84800)
at /usr/include/c++/4.6/functional:1219
#9 0x000000000193f2e3 in pmap_runner_one_arg_t<std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)>, long int>::operator()(void) (this=0x7faff6d84750)
at ../src/concurrency/pmap.hpp:20
#10 0x0000000001931fbd in pmap<std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*> >&, const std::vector<int>&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*> > >, std::reference_wrapper<const std::vector<int> >, std::_Placeholder<1>)> >(int64_t, int64_t, const std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)> &) (begin=0, end=1, c=...)
at ../src/concurrency/pmap.hpp:47
#11 0x000000000191fd8e in pmap<std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*> >&, const std::vector<int>&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*> > >, std::reference_wrapper<const std::vector<int> >, std::_Placeholder<1>)> >(int64_t, const std::_Bind<std::_Mem_fn<void (ql::changefeed::feed_t::*)(const std::function<void(ql::changefeed::range_sub_t*)>&, const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > >&, const std::vector<int, std::allocator<int> >&, int)>(ql::changefeed::feed_t*, std::reference_wrapper<const std::function<void(ql::changefeed::range_sub_t*)> >, std::reference_wrapper<const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > >, std::reference_wrapper<const std::vector<int, std::allocator<int> > >, std::_Placeholder<1>)> &) (count=1, c=...)
at ../src/concurrency/pmap.hpp:53
#12 0x00000000019106f0 in ql::changefeed::feed_t::each_sub_in_vec<ql::changefeed::range_sub_t>(const std::vector<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> >, std::allocator<std::set<ql::changefeed::range_sub_t*, std::less<ql::changefeed::range_sub_t*>, std::allocator<ql::changefeed::range_sub_t*> > > > &, rwlock_in_line_t *, const auto_drainer_t::lock_t &, const std::function<void(ql::changefeed::range_sub_t*)> &) (this=0x7fafe2bfcf00, vec=std::vector of length 25, capacity 25 = {...}, spot=0x7faff6d848f0, lock=..., f=...)
at ../src/rdb_protocol/changefeed.cc:2915
#13 0x00000000018f386e in ql::changefeed::feed_t::each_range_sub(const auto_drainer_t::lock_t &, const std::function<void(ql::changefeed::range_sub_t*)> &) (this=0x7fafe2bfcf00, lock=..., f=...)
at ../src/rdb_protocol/changefeed.cc:2941
#14 0x0000000001902d5d in ql::changefeed::msg_visitor_t::operator() (this=0x7faff6d85380, change=...)
at ../src/rdb_protocol/changefeed.cc:2392
#15 0x0000000001958f9b in boost::detail::variant::invoke_visitor<ql::changefeed::msg_visitor_t>::internal_visit<ql::changefeed::msg_t::change_t const> (this=0x7faff6d85120, operand=...)
at /usr/local/include/boost/variant/variant.hpp:858
... This is on the same |
Assigning @mlucy . |
This looks like #4878 to me, so I guess that probably wasn't fixed by the changes in the patch. |
I think I found it (and the fuzzer doesn't seem to be producing this crash for me any more). This is in CR 3296 by @danielmewes . |
A fix for this is in next and v2.1.x. |
(Or what I believe is a fix. @Tryneus, let me know if you encounter this again.) |
I was testing the new
--kill
option fortable_fuzzer.py
, and ran into this crash with the command linetable_fuzzer.py --servers 7 --threads 48 --changefeeds --kill
The text was updated successfully, but these errors were encountered: