+1 @errordaiwa how is the cpu usage with high bolt count when idle ?

@errordaiwa @arunmahadevan
I'm aware of unknown bug in Disruptor Queue, so I'd like to specify timeout value though some race conditions are fixed.
We can have an experiment with adjusting timeout from 100 to 1000 and look at CPU usage.

In short, applying both configuring timeout and upgrading Disruptor to 2.10.4 makes sense for me.

@errordaiwa
If you don't mind, I'd like to ask you a favor to experiment such things, since I'm not ready to experiment.

@HeartSaVioR @arunmahadevan I do a performance test using storm 0.9.3 with disruptor queue 2.10.4. The target topology is metioned in STORM-503. To make result more clear, I raise bolt num to 1000.

Here is the CPU usage.

• base (Storm running with nothing)
  • user: 2%
  • sys: 1.5%
• no timeout
  • user: 4%
  • sys: 1.5%
• 1000ms timeout
  • user: 5%
  • sys: 2%
• 100ms timeout
  • user: 6%
  • sys: 4.5%
• 10ms timeout
  • user: 17%
  • sys: 26%

@errordaiwa @amontalenti
1000ms timeout makes sense to me.
Actually 100ms timeout also makes sense to me, but I'd like to know our opinions about load aware.

We're having a option to set timeout now, so it would be no issue.
I'll run some performance tests and check no tuples are failed.

I am fine with whichever timeout we pick. I am +1 for merging this in pending the results of the tests @HeartSaVioR is running.

@revans2 Thanks for confirming. :)

During running performance test I can't see any failed tuples from the UI in 10 minutes, so I think it is safe to merge.
LGTM, +1.

Btw, it would be better for users to change default timeout in order to let users not suffering STORM-503 with default setting.

@errordaiwa Could you change default timeout to 1000ms?

After changing default timeout I'll merge.

done.

@HeartSaVioR @revans2 Thanks for validation and confirming.

+1.

Since many users are already affected this situation, it should be included to 0.9.x line.

It is merged to master, and I'm working on backport to 0.10.x and 0.9.x.

I merged this into 0.10.x-branch and 0.9.x-branch by cherry-picked.

Since there're different available sets of config validators among branches - master, 0.10.x, 0.9.x, I take the best proper validator to each branch.

• master: NotNullPosIntegerValidator
• 0.10.x: PositiveIntegerValidator
• 0.9.x: IntegerValidator

Great!

@HeartSaVioR Sincerely thank you for working on this!