Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit x-death header length growth #78

Closed
michaelklishin opened this issue Mar 18, 2015 · 9 comments
Closed

Limit x-death header length growth #78

michaelklishin opened this issue Mar 18, 2015 · 9 comments
Assignees
Milestone

Comments

@michaelklishin
Copy link
Member

Original bug: 26538.

See https://groups.google.com/forum/#!topic/rabbitmq-users/dwdE-DySy3E.

After discussing a few options, we've decided to cap the growth by having one value per queue (the most recent one).

@simonmacmullen
Copy link
Contributor

The new code doesn't count the number of times we have been at a certain queue, it just records the fact that that we have been there. That's removing a bit too much information IMO, a counter would be useful, non-leaky, and easy to add.

I am not completely convinced this has not broken cycle detection for some more arcane types of cycles. I shall try to have a think about that.

The Java tests introduce new Channel APIs! That's an unexpected side effect of this bug 😄 I don't think Channel.queueBind(String queue, String exchange) is actually a great idea; it de-emphasises routing keys and thus might rather confuse beginners.

@dumbbell
Copy link
Member

I agree, the number of times would be nice. The user who reported the problem mentionned it as well.

After reading the cycle detection code, I think it still works: it only looks for one entry for the current queue.

@simonmacmullen
Copy link
Contributor

Yes, but cycle detection makes decisions based on the {reason, queue} pair, but the branch only keeps one header per queue. That smells dubious to me.

So for example if you have a message which (for some odd reason) keeps getting dead lettered to the same queue, but with alternating reasons of expired and rejected then this branch changes the behaviour - we used to not drop it (since we only drop messages on dead-letter cycles that are fully automatic, requiring no client intervention to keep them going) but on this branch we would drop it.

I'm not sure if I can get a change in behaviour the other way round (which would be much more dangerous of course). And that is a bit of a contrived example. But it's still a real behaviour change that I think we can avoid.

What I would like to see when adding a new x-death entry:

  • Look for a matching one, matching on both queue and reason
  • If there, increment its count and bring to the front of the list
  • If not, create one with count=1 and add to the front of the list

...which I think is what @michaelklishin has done but with the addition of a counter and matching on reason too.

@michaelklishin
Copy link
Member Author

Matching on {queue, reason} and having a counter makes sense. Thanks.

@michaelklishin
Copy link
Member Author

Ready for another review:

  • x-death events are now identified by {queue, reason}
  • events now have a counter
  • The docs are updated accordingly
  • Channel#queueBind/2 was taken out

@simonmacmullen
Copy link
Contributor

The logic of this looks OK to me now. Admittedly I haven't actually run it.

I have a bunch of style complaints, take them as seriously as you wish:

AutorecoveringChannel has grown an import for no obvious reason.

I would inline x_deaths_from_header/1 and x_death_not_for_queue/2, I don't think they make it any clearer. I also wonder if x_death_header/1 could become header(HeaderName, Headers, Default) and be moved to rabbit_basic, since I suspect it would be more generally useful.

Personally I don't find abstracting away the names of headers to be any clearer either (since you always have to mentally look it up) so I'd kill ?X_DEATH_HEADER and friends too. I know we do that elsewhere, but I don't like it elsewhere either 😄

I would rather the new field be called count rather than counter: it reads more naturally to me.

@dumbbell
Copy link
Member

I gave the branch a try and did:

  1. From the management UI, I created:

    • a my-dlx fanout exchange
    • a my-dlq queue bound to my-dlx
    • a test queue with a queue TTL of 1" and my-dlx as its dead-letter exchange
  2. I used PerfTest to publish one message to test

  3. The message ended in my-dlq after the 1" TTL. The x-death looked like:

    counter:        1
    reason:         expired
    queue:          test
    time:           1427292711
    exchange:       direct
    routing-keys:   e3e47d2e-8834-481f-8969-ec4125c1847c
    
  4. Still using the management UI, I moved the message from my-dlq to test. Again, the message expired and ended up in my-dlq. The x-death counter has been incremented:

    counter:        2
    reason:         expired
    queue:          test
    time:           1427292711
    exchange:       direct
    routing-keys:   e3e47d2e-8834-481f-8969-ec4125c1847c
    

I tried again but this time with my-dlq configured like test to create a cycle. It was properly detected and the message was dropped.

@michaelklishin
Copy link
Member Author

I've renamed the field, removed the unused import and did some refactoring (introducing rabbit_basic:header/{2,3} and inlining).

dumbbell added a commit that referenced this issue Mar 26, 2015
While here, wrap long lines to fit 80 columns.

References #78.
@dumbbell dumbbell added bug and removed enhancement labels Mar 26, 2015
@dumbbell dumbbell added this to the 3.5.1 milestone Mar 26, 2015
@dumbbell
Copy link
Member

The associated PR (#79) was merged to stable.

dcorbacho pushed a commit that referenced this issue Jul 5, 2023
file:consult/1 is OK to parse a configuration file with some
TLS settings, but it does not support parsing "complex"
settings like a function to validate the hostname.

This commit introduces a hook so that the calling application
can provide a custom function instead of the default file:consult/1
and this way easily support parsing the app settings.

Fixes #78
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants