Skip to content
This repository has been archived by the owner on Apr 23, 2024. It is now read-only.

Loggly

Jump to bottom
Yulian edited this page Nov 25, 2021 · 16 revisions

The Logback Loggly extension enables Logback to post messages to Loggly, a cloud logging service.

Check out Loggly's Java Logback logging documentation to learn more.

Usage

Add the following .jar to your application's classpath:

Jar Maven or Ant+Ivy
logback-ext-loggly-version.jar 
<dependency>
    <groupId>org.logback-extensions</groupId>
    <artifactId>logback-ext-loggly</artifactId>
    <version>version</version>
    <scope>runtime</scope>
</dependency>

Pattern-formatted Messages

Simple logback.xml configuration:

<configuration>
    <appender name="logglyAppender" class="ch.qos.logback.ext.loggly.LogglyAppender">
        <endpointUrl>https://logs-01.loggly.com/bulk/TOKEN/tag/TAGS</endpointUrl>
    </appender>
    ...
</configuration>

By default the Loggly Appender uses a PatternLayout to control log event formatting. The default pattern normalizes timestamps to UTC time, which is usually best for clustered environments that process requests from around the world. The default pattern is:

%d{"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",UTC} %-5level [%thread] %logger: %m%n

If this is not suitable, you can configure the pattern directly on the appender:

<configuration>
    <appender name="logglyAppender" class="ch.qos.logback.ext.loggly.LogglyAppender">
        <endpointUrl>https://logs-01.loggly.com/bulk/TOKEN/tag/TAGS</endpointUrl>
        <pattern>%date %-5level [%thread] %logger{36} %m%n</pattern>
    </appender>
    ...
</configuration>

JSON Messages

Loggly enables some powerful features when you send it JSON. To enable JSON messages:

<configuration>
    <appender name="logglyAppender" class="ch.qos.logback.ext.loggly.LogglyAppender">
        <endpointUrl>https://logs-01.loggly.com/bulk/TOKEN/tag/TAGS</endpointUrl>
        <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
            <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
                <!-- prettyPrint is probably ok in dev, but usually not ideal in production: -->
                <prettyPrint>false</prettyPrint>
            </jsonFormatter>
        </layout>
    </appender>
    ...
</configuration>

You will also need to add the following jars to your application's classpath:

Jar Maven or Ant+Ivy
logback-json-classic-version.jar 
<dependency>
    <groupId>ch.qos.logback.contrib</groupId>
    <artifactId>logback-json-classic</artifactId>
    <version>version</version>
    <scope>runtime</scope>
</dependency>
logback-jackson-version.jar 
<dependency>
    <groupId>ch.qos.logback.contrib</groupId>
    <artifactId>logback-jackson</artifactId>
    <version>version</version>
    <scope>runtime</scope>
</dependency>

JsonFormatter

As you can see, we've chosen to use the Jackson JSON Processor (from logback-contrib) in the above example. You may provide your own JsonFormatter implementation if you do not want to use Jackson. A JsonFormatter must be specified to direct which processor should be used at runtime since the JDK does not provide one by default.

Timestamp Formatting

By default the JsonLayout will NOT format the log event's timestamp. It will just print the timestamp as a long (milliseconds since epoch). If you want a nicely formatted timestamp string, you can specify the timestamp format (as understood by SimpleDateFormat) and time zone ID (as understood by the JDK TimeZone class). If a timestamp format is specified, but a time zone ID is not specified, the system default time zone will be assumed.

Example config normalizing timestamps to UTC time:

<configuration>
    <appender name="logglyAppender" class="ch.qos.logback.ext.loggly.LogglyAppender">
        <endpointUrl>https://logs-01.loggly.com/bulk/TOKEN/tag/TAGS</endpointUrl>
        <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
            <jsonFormatter class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
                <!-- prettyPrint is probably ok in dev, but usually not ideal in production: -->
                <prettyPrint>false</prettyPrint>
            </jsonFormatter>
            <timestampFormat>yyyy-MM-dd'T'HH:mm:ss.SSSZ</timestampFormat>
            <timestampFormatTimezoneId>UTC</timestampFormatTimezoneId>
        </layout>
    </appender>
    ...
</configuration>

Batched Mode

Note: Loggly's Syslog API is much more scalable than the HTTP API which should mostly be used in low-volume or non-production systems. The HTTP API can be very convenient to workaround firewalls

If the LogglyBatchAppender saturates and discards log messages, the following warning message is appended to both Loggly and Java's System.err: $date - OutputStream is full, discard previous logs.

Configuration settings

Property Name Type Description
inputKey String Loggly input key. Either inputKey or endpointUrl is required. Sample: 12345678-90ab-cdef-1234-567890abcdef
endpointUrl String Loggly HTTP API endpoint URL. Either inputKey or endpointUrl is required. By specifying endpointUrl, the user has the option of turning HTTPS on or off for debugging purposes. Otherwise, the plugin will default to sending logs via HTTPS.

Sample: https://logs.loggly.com/bulk/12345678-90ab-cdef-1234-567890abcdef
pattern String Pattern used for Loggly log messages. Default value is: %d{"yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",UTC} %-5level [%thread] %logger: %m%n
proxyHost String hostname of a proxy server. If blank, no proxy is used.
proxyPort int port of a proxy server. Must be a valid int but is ignored if proxyHost is blank or null.
jmxMonitoring boolean Enable registration of a monitoring MBean named ch.qos.logback:type=LogglyBatchAppender,name=LogglyBatchAppender@#hashcode#. Default: true.
maxNumberOfBuckets int Max number of buckets of in the byte buffer. Default value: 8.
maxBucketSizeInKilobytes int Max size of each bucket. Default value: 1024 Kilobytes (1Mo).
flushIntervalInSeconds int Interval of the buffer flush to Loggly API. Default value: 3.

Performance Notes

  • The Default configuration consumes up to 8 buffers of 1024 Kilobytes (1Mo) each which seemed very reasonable even for small JVMs.
  • If logs are discarded, try first to shorten the flushIntervalInSeconds parameter to 2 or even 1 so that the buffers are flushed before they are filled.

Configuration Sample

<configuration scan="true" scanPeriod="30 seconds" debug="true">
  <if condition='isDefined("logback.loggly.inputKey")'>
    <then>
      <appender name="loggly" class="ch.qos.logback.ext.loggly.LogglyBatchAppender">
        <inputKey>${logback.loggly.inputKey}</inputKey>
        <pattern>%d{yyyy/MM/dd HH:mm:ss,SSS} [${HOSTNAME}] [%thread] %-5level %logger{36} - %m %throwable{5}%n</pattern>
        <proxyHost>${logback.loggly.proxy.host:-}</proxyHost>
        <proxyPort>${logback.loggly.proxy.port:-8080}</proxyPort>
        <debug>${logback.loggly.debug:-false}</debug>
      </appender>
      <root level="WARN">
        <appender-ref ref="loggly"/>
      </root>
    </then>
  </if>
</configuration>
Clone this wiki locally