Pluggable logging backends #94
Comments
|
👍 |
|
@michaelklishin are there plans to implement this any time soon? I am very interested in it! |
|
@willejs we are pretty busy with other things. It should be very realistic to ship this with |
|
Cheers @michaelklishin its good enough that this is on the horizon 😄 . Adding this issue to the 3.6.0 milestone would be cool 😉 |
|
@michaelklishin sorry to keep bothering you! Is it on a roadmap? Is there anyway to speed this up, apart from learning erlang? ;) I'l donate to the project in any way if i can! |
|
There's this already that can help a bit: https://github.com/hyperthunk/rabbitmq-lager I'm using it on a plugin that I'm building |
|
@willejs this is not getting any attention yet for one reason: we are a tiny team. It's still on the roadmap for 3.6.0 and we'll see if we can squeeze it into the list of things for July. |
|
I understand, hence apologising! July would be great! |
|
@michaelklishin me again :) Hows this looking? |
|
@willejs it will not be in 3.6. However, now that we've moved to erlang.mk for the build system, introducing new dependencies is trivial, so this becomes easier. |
|
There were logging changes in Erlang/OTP 18.1, which 3.7 will likely require, that make me wonder if Lager is really necessary for this to work. |
|
Lager receives messages from |
|
Do anyone know why is there |
|
@hairyhum see the comment on |
|
We have following config variables: RABBITMQ_LOG_BASE RABBITMQ_LOGS and RABBITMQ_SASL_LOGS can be set to absolute filename or Lager log_root and handlers for logs can be configured from .config file for different loglevels and console/file/something else backends. There are change in log rotation. Customers can rely on logrotate with 'suffix', which While using lager it is possible just to move file or delete it (if suffix = '') Lager itself supports log rotation by date and file size and can keep given count of So main changes so far will be: |
|
And I didn't found out yet how to separate sasl and general logs so far. |
|
@hairyhum note that we can log something after rotation, e.g. "opening a new log file after rotation". |
|
Found an option to separate sasl and general logs. It is lager |
|
Currently lager has it's own way to format |
|
@hairyhum can it be overridden? |
|
@michaelklishin unfortunately, not in current implementation. And they are not too quick at accepting pull requests right now. But we can use forked version, if it's an option. |
|
@dumbbell @videlalvaro @essen what are your thoughts on using a Lager fork to get more informative crash reports (with stack traces)? @hairyhum can you be more specific what the "some cases" are? |
|
As I can tell from code, stacktraces are omitted if errors are from |
|
We can live without stack traces in the regular log but SASL log really could benefit from having stack traces. I'm not a fan of using forks but if that's what we have to do, I'm fine with it. |
|
I think whatever we fork we should actively try to incorporate upstream,
|
|
I mentioned fork because lager team are too busy right now to deal with PR, like they said. So I hope it could be merged eventually. |
|
This is on its finish line. As part of this, we will pipe the majority of log entries (everything but Erlang process crash reports) to See #491 for more details. |
By default, RabbitMQ now logs messages to a single file
($RABBITMQ_LOGS). The $RABBITMQ_SASL_LOGS variable is unused. To
configure how and which messages are logged, it's recommended to do it
from rabbitmq.config, not from the environment variable.
The old `log_levels` parameter is unsupported and categories are
replaced by Lager extra sinks. If you had in your rabbitmq.config:
{rabbit, [
{log_levels, [{connection, info}]}
]}
You can now configure Lager like this:
{lager, [
{extra_sinks, [
{rabbit_connection_lager_event, [
{handlers, [{lager_forwarder_backend, [lager_event, info]}]}
]}
]}
]}
rabbitmq-build.mk from rabbitmq-common is included in the top-level
Makefile. It sets the appropriate compiler options to enable Lager's
lager_transform parse_transform module.
rabbit_log calls are now converted by this parse_transform to direct
calls to lager:log(). To keep backward compatibility with other plugins,
the rabbit_log module still implements all the <level>() functions.
Compared to the parse_transformed calls, the main difference is the
logged message does not carry the file:line metadata.
Fixes #94.
rabbitmq-build.mk centralizes common compiler options for all RabbitMQ
components. Currently, it includes options specific to Lager:
o It enables lager_transform parse_transform module.
o It lists all our specific extra sinks
rabbit_log is the default sink we use for messages logged using
rabbit_log:<level>().
The other sinks correspond to the previous categories we supported. To
use those sinks, one needs to log messages using eg.:
rabbit_channel:info("Message")
lager_forwarder_backend forwards messages from one sink to another. This
allows RabbitMQ to use a single file to for several sinks:
lager_event (default sink) --> lager_file_backend --> "rabbit.log"
^
|
+------------------------+---+--+
| | |
rabbit_log_lager_event --+ | |
rabbit_channel_lager_event --+ |
rabbit_connection_lager_event --+
rabbit_channel.erl and rabbit_reader.erl are modified to use the new
API.
References rabbitmq/rabbitmq-server#94.
By default, RabbitMQ now logs messages to a single file
($RABBITMQ_LOGS). The $RABBITMQ_SASL_LOGS variable is unused. To
configure how and which messages are logged, it's recommended to do it
from rabbitmq.config, not from the environment variable.
The old `log_levels` parameter is unsupported and categories are
replaced by Lager extra sinks. If you had in your rabbitmq.config:
{rabbit, [
{log_levels, [{connection, info}]}
]}
You can now configure Lager like this:
{lager, [
{extra_sinks, [
{rabbit_connection_lager_event, [
{handlers, [{lager_forwarder_backend, [lager_event, info]}]}
]}
]}
]}
rabbitmq-build.mk from rabbitmq-common is included in the top-level
Makefile. It sets the appropriate compiler options to enable Lager's
lager_transform parse_transform module.
rabbit_log calls are now converted by this parse_transform to direct
calls to lager:log(). To keep backward compatibility with other plugins,
the rabbit_log module still implements all the <level>() functions.
Compared to the parse_transformed calls, the main difference is the
logged message does not carry the file:line metadata.
Fixes #94.
References rabbitmq/rabbitmq-server#94.
Keys `log_file` and `sasl_log_file` are replaced by a single key `log_files` which returns a list. References rabbitmq/rabbitmq-server#94.
This key contains a list of files. If the node sends messages to stdout, the list contains "<stdout>", therefore it needs to be escaped in an HTML context. While here, escape `config_files` as well. References rabbitmq/rabbitmq-server#94.
Do not use ?= operator in make target env variables setup
That change was already committed in commit 9b923e2. However, it was lost later, probably because we merge `stable` into `master. References rabbitmq/rabbitmq-server#94.
* Handle potential retention race condition If a stream member is being deleted whilst retention is evaluating the retention process could crash if there are no index / segment files. * Add a file size function that doesn't just return 0 for files that do not exist. This throws "missing_file" instead to allow retries higher up the stack. * Change where errors are caught in init_offset_reader And implement a simple retry limit Throw missing_file whenever a file_size operation fails instead of returning 0. Remove all nested missing_file handling in favour of entry point handling + retry * improve some logging * Delete index file before segment As this should somewhat reduce the chances of hitting missing_file issues. * Use a different file_size function when evaluating retention As returning 0 when segment file is missing is ok in this case.
People keep asking for it. We should make Lager dead easy to use with RabbitMQ. Or even consider switching to it entirely.
The text was updated successfully, but these errors were encountered: