WL#7793: Add native support for syslog on unixoid platforms

Status: Complete

mysqld did not have native support for traditional unix-style
syslog; it instead used a shell-script wrapper that would pipe
the daemon's output to the syslog.  This implied certain limitations:

- it was impossible to see the syslog status at runtime
- it was impossible to change the syslog status at runtime
- the shell script could interfere with log rotation
- the setup would often result in log lines containing two timestamps

Clients and mysql/sun/orcl employees alike have therefore suggested to
add native support of syslog on unixoid systems.

cf. BUG#55370 mysqld general log output to syslog et al.
Functional requirements

FR1 Logging to system facilities

  The server SHALL support output of diagnostic information to
  well-known system facilities:

FR1.1 UNIX syslog

  On unixoid/POSIX systems, output to syslog SHALL be supported.

FR1.2 Windows NT EventLog

  On Windows systems, output to NT EventLog SHALL be supported.

FR2 Diagnostic information

  "Diagnostic information" in the context of this WL is all
  information already sent to stderr (and possibly piped to
  syslog by a wrapper), usually with a timestamp, a severity
  tag (error/warning/notice), and possibly other information.

  "Diagnostic information" for the purpose of this WL does not
  include information sent to auxiliary logs such as the slow
  query log or the general log, but MAY be extended to include
  such logs at a later date.

FR2.1 Redundant information

  The implementation SHOULD take care not to duplicate information,
  for instance it SHOULD NOT include a timestamp in a log line sent
  to the system facilities when it would be reasonable to assume that
  the system facilities will add their own timestamp.

FR2.2 Severity levels

  Where system facilities support this, the implementation SHOULD
  attempt to use severity levels that mirror those used in current 
  Diagnostic Information and those sent to the client.

  Where the system facilities do not support an exact mapping or
  the use of such a mapping seems otherwise inadvisable, the
  implementation MAY opt to approximate such a mapping instead.

FR3 Controls

  The implementation SHALL give the user access to / control over
  these features at start-up and at run-time.

FR3.1 Start-up

  Parameters SHOULD be settable at start-up.

FR3.2 Run-time

FR3.2.1 Read-access

  Parameters SHALL be readable at run-time.

FR3.2.2 Write-access

  Paramaters SHOULD be modifiable at run-time where supported by
  the underlying system facilities.



Non-functional requirements

NFR1

  The implementation SHOULD attempt clean up and unify the 
  rudimentary interfaces to system facilities present in 
  my_syslog.c and log.cc, respectively.
USER VISIBLE CHANGES

Add native syslog support for unixoid OSs.

#1 mysqld sysvars / options

   The following can be set a start-up as well as viewed
   and set at runtime via system variables (requiring the
   SUPER privilege).
 
   --log_syslog[=0|1]
     on/off switch, defaults to off
 
   --log_syslog_include_pid[=0|1]
     Include mysqld's process ID on each syslog line?
     Defaults to on.
 
   --log_syslog-facility=local5
     choose the syslog "facility". Well-known facilities
     may be selected by name, all others by numbers.
     Defaults to "daemon".
 
   --log_syslog-tag=
     If empty and by default, an ident of "mysqld" is used.
     If e.g. tag "abcd" is supplied, "mysqld-abcd" will be used.
     (The hyphen will be used automatically, the user need not
     supply it.)

     This item is designed to closely emulate the syntax used
     by the current mysqld_safe script, so updating the script
     to use the server-side (rather than script-side) facilities
     is made easier, with an eye to keeping the script's parameters
     stable so its callers can remain unchanged in most cases, i.e.
     in the interest of making the transition as smooth as possible
     for the DBAs.

     Finally, moving this item into the server should simplify the
     planned rewrite of the wrapper scripts (mysqld_safe, 
     mysqld_multi).

#2 Severity mapping

   The severity mapping on unixoid platforms shall be as follows:

     ERROR_LEVEL       -> syslog::error   (3)
     WARNING_LEVEL     -> syslog::warning (4)
     INFORMATION_LEVEL -> syslog::notice  (5)

   This will make things uniform within the platform;
   i.e. something shown as "[Notice]" in the log file
   would show as "Notice" in syslog, and would further-
   more reflect the levels (error, warning, notice) sent
   to the client.

   For Windows NT Eventlog, the mapping would be as follows:

     ERROR_LEVEL       -> EVENTLOG_ERROR_TYPE
     WARNING_LEVEL     -> EVENTLOG_WARNING_TYPE
     INFORMATION_LEVEL -> EVENTLOG_INFORMATION_TYPE

   (I.e. the least severe message type would show as "information"
   rather than as "notice"; this is owed to the fact that a) that
   mapping is already used in GA servers, and b) no other suitable
   severity levels (especially one corresponding to "notice") exist
   in classic NT eventlog.)

THE FUTURE / DEFERRED

Future WL tasks may extend logging to syslog/eventlog by
offering to also redirect the general and slow query logs.
The present WL task was designed with this in mind and tries
not to create hindrances for the possible later task.

An additional log_output target "SYSLOG" could be added
(while keeping the existing FILE and TABLE).  If this
target is active, general log and slow log items could
be sent to syslog / EventLog in a way that makes them
distinguishable from the error/warning/information item
used by "error log" type items. One way of achieving this
would be to use a separate EventSource / facility for the
error log on one hand, and general/slow log on the other
(or even to split them 3 way, error <-> slow <-> general).

#3 --log_syslog-facility=
   would then become
   --log_syslog-facility=[,]

#4 --log_output=TABLE|FILE|NONE
   would become
   --log_output=TABLE|FILE|SYSLOG|NONE

#5 my_openlog() / my_syslog() would have to be upgraded for
   multi-facility / multi-EventSource use. (Any further details
   on this to be placed in Low Level Design.)

#6 The mapping for the POSIX syslog severity levels in #2 would
   allow for the remaining two levels to be mapped like so,

     slow log    -> syslog::information (6)
     general log -> syslog::debug       (7)

   This however does not readily translate to NT eventlog,
   so the "two facilities / two event-sources" design outlined
   above may prove more portable.
ENGINEERING CONSIDERATIONS

#1 mysys/my_syslog.c already contains some code for
   syslog support (currently only used by the command
   line client).  This code should be reworked to support
   the features configurable according to #1.

#2 mysys/my_syslog.cc and sql/log.cc has some duplication
   of effort with regard to the Windows EventLog.  Eliminate
   the duplicates from log.cc, and merge the remaining
   useful code into my_syslog.cc so that other binaries
   may have a chance to benefit from it.

#3 As a special case, sql/conn_handler/socket_connection.cc
   when compiled with HAVE_LIBWRAP, will log auth failures
   to syslog. This feature currently uses the libC calls
   directly, and should be modified so it uses the corresponding
   my_*() calls, and does not interfere with other syslogging
   in mysqld.