Configuring logactio
********************

Logactio makes use of a single ini-styled configuration file.  The
default location for this file is */etc/logactio.cfg*.  This file uses
a separate section, identified by *[section-name]*.  There are three
different groups of sections which you need to configure, *Reporter*,
*Logfile* and *Rule*. Each of these group names have a *label*, where
the group name and the label are separated by a colon (:)


Configuring Reporters
=====================

There are three different reporters available in logactio, and each of
them take different configuration parameters.  If no reporter is
configured, a built-in reporter called *Default* will be used.

* The Default reporter

     This reporter will only write data via the configured logging
     methods.  This reporter also does not have any configuration
     settings and does not require any specific declaration.

* HTTPreporter

     This reporter will send the extracted log data to a web server.
     Both HTTP and HTTPS may be used.

* SMTPreporter

     This reporter will send the extracted log data via SMTP to one or
     more e-mail recipients.  Support for SSL and STARTTLS in addition
     to SMTP-AUTH are availble as well.

* IPTipset

     This "reporter" can add extracted IP addresses from a log file
     and add it to an ipset table.  This table can be used by iptables
     to for example block access.

* NftablesSet

     This "reporter" can add extracted IP addresses from a log file
     and add them to a pre-existing nftables set.


HTTPreporter
------------

This reporter takes the following configuration variables

* method

     This defaults to **GET** if this is not set.  But can be set to
     **POST** if you want the reports to be sent via HTTP POST instead
     of HTTP GET.

* url

     This is a required setting.  This is the URL where to submit the
     reports.

* format

     This defaults to **plain** if not set.  Controls how the report
     data is encoded in the request:

     * **plain**: data is URL-encoded ("application/x-www-form-
       urlencoded"). Works with both GET and POST.

     * **json**: data is sent as a JSON object ("application/json").
       Requires **method** to be set to **POST**.


HTTPreporter example
~~~~~~~~~~~~~~~~~~~~

   [Reporter:HTTP-DEMO-1]
   module: HTTPreporter
   method: POST
   url: http://logactio.example.com/alert/

   [Reporter:HTTP-DEMO-2]
   module: HTTPreporter
   url: http://logactio.example.com/info/

   [Reporter:HTTP-DEMO-3]
   module: HTTPreporter
   method: POST
   url: http://logactio.example.com/alert/json/
   format: json

Here three reporters are configured.  *HTTP-DEMO-1* will use HTTP POST
with URL-encoded data, *HTTP-DEMO-2* will use HTTP GET with URL-
encoded data, and *HTTP-DEMO-3* will use HTTP POST sending a JSON
body.


SMTPreporter
------------

This reporter requires the following configuration variables:

* sender

  The e-mail address which will be used in the "From:" field when
  sending mails

* recipients

  This contains a comma separated list with all e-mail addresses who
  will get the logactio reports.

* smtp_host

  This declares which SMTP server to use when sending the reports.

In addition the SMTPreporter supports these optional variables:

* subject_prefix

     The default subject prefix is set to 'LogActio Alert: '.  By
     setting this variable, the subject prefix will be changed
     accordingly.

* smtp_port

     The default value is set to port 25.

* smtp_username

     If the SMTP server requires authentication to relay messages,
     this variable sets the SMTP user name.  To use this feature, you
     must also set the smtp_password.

* smtp_password

     This sets the SMTP password to use for the authentication

* sslmode

     This is not set by default, so everything goes in clear text.  If
     your SMTP server supports either SSL or STARTTLS, you can set it
     to SSL or STARTTLS. In SSL mode the SMTP library expects to the
     server to do the SSL handshake before the SMTP commands can be
     sent.  In STARTTLS mode, the SMTP library will connect to the
     SMTP server in clear text and if the server supports STARTTLS, it
     will send the STARTTLS command and start the SSL handshake.


SMTPreporter example
~~~~~~~~~~~~~~~~~~~~

   [Reporter:SMTP-DEMO-1]
   module: SMTPreporter
   sender: logactio@example.com
   recipients: john.doe@example.com, jane.doe@example.com
   smtp_host: localhost

   [Reporter:SMTP-DEMO-2]
   module: SMTPreporter
   sender: john.doe@example.com
   recipients: bob.external@acme.com
   smtp_host: smtp.example.com
   smtp_port: 587
   sslmode: STARTTLS
   smtp_username: logactioSMTP
   smtp_password: S3cretP4ssw0rd
   subject_prefix: Issues at Example Corp:

Here two more reporters are configured.  SMTP-DEMO-1 will use the SMTP
MTA running locally on the system, and this will not require any
authentication or SSL functionality.  When this reporter is triggered,
it will send mails to john.doe and jane.doe with the From field set to
logactio@example.com.

The SMTP-DEMO-2 reporter will send mails using an external SMTP server
using port 587 and which requires STARTTLS and authentication.  The
subject line is also prefixed differently.


IPTipset
--------

To use this module, you need ipset and iptables installed and
available.  This will also require logactio to run with root
privileges, otherwise it lacks privileges to update the ipset and
iptables rules.

Important:

  This module expects only a single regex group to be extracted.  This
  value **has** to be an IP address, otherwise the IPTipset module
  will fail when executing the ipset command.

The required configuration variables are:

* ipset-name This is the name of the ipset table to this specific
  section will use.

Optional settings are:

* ipset-create

  Boolean variable which tells wether the IPTipset module should
  create the ipset table when starting or not.  The values of "true"
  (case or "1" will be understood as True, all other values as False.

* ipset-hashsize, ipset-timeout, ipset-counters

  These variables will be sent further to the 'ipset create' command
  and will set hashsize, timeout and the counters flag.  ipset-
  counters is a boolean flag, where the value of "true" (case
  insensitive) or "1" will be understood as True, all other values as
  False.  The default value for ipset-hashsize is 1024, ipset-timeout
  is not defined and ipset-counters is False by default.

* ipset-save

  This will enable the IPTipset module to save the state of the ipset
  table when exiting.  If this file exists when LogActio is started
  again later on, this state will be loaded automatically.

* iptables-chains

  If set, LogActio will insert the ipset checks into the listed
  iptables chains.  Multiple chains can be listed, separated by comma.

* iptables-insert-points

  This is optional but only available if iptables-chains are used.
  This defines where in the different iptables chains the ipset rules
  will be inserted.  By default it will be inserted at the top of the
  chain.  The syntax is CHAIN:NUM, where CHAIN is the name provided in
  *iptables-chains* and NUM is the value used when calling iptables.
  The result is something like 'iptables -I CHAIN NUM ...'.  Multiple
  chains can be defined, separated by comma.

* iptables-jump

  This is required when using *iptables-chains*.  This defines what to
  do when iptables get a positive match against the ipset table.  This
  sets the '-j' options when iptables commands are executed.


IPTipset example
~~~~~~~~~~~~~~~~

   [Reporter:ipsetblock]
   module: IPTipset
   ipset-name: BlockList
   ipset-create: True
   ipset-hashsize: 2048
   ipset-timeout: 3600
   ipset-counters: False
   ipset-save: /var/lib/ipset/logactio-ipset.save
   iptables-chains: INPUT,OUTPUT,FORWARD
   iptables-insert-points: INPUT:2,FORWARD:4
   iptables-jump: DROP

This will configure a reported named ipsetblock.  Whenever this
reporter is called, it will add an IP address to an ipset table named
'Blocklist'. This list will be created when LogActio starts, with a
hashsize of 2048 and a timeout of 1 hour (3600 seconds).  The ipset
table state will be saved to /var/lib/ipset/logactio-ipset.save.  When
starting LogActio the following commands will be run:

   # ipset --exist create BlockList hash:ipset hashsize 2048 timeout 3600
   # iptables -I INPUT 2 -m set --match-set BlockList src -j DROP
   # iptables -I OUTPUT -m set --match-set BlockList src -j DROP
   # iptables -I FORWARD 4 -m set --match-set BlockList src -j DROP


NftablesSet
-----------

To use this module, you need nftables installed and available.  This
will also require logactio to run with root privileges, otherwise it
lacks privileges to update the nftables set.

The nftables set must be pre-created before starting logactio.  This
module will not create or destroy the set.

Important:

  This module expects only a single regex group to be extracted.  This
  value **has** to be an IP address, otherwise the NftablesSet module
  will fail when executing the nft command.

The required configuration variables are:

* nft-table

  The name of the nftables table containing the set.

* nft-set

  The name of the nftables set to add IP addresses to.

Optional settings are:

* nft-family

  The nftables family of the table.  Defaults to **inet**.

* nft-save

  If set, the state of the nftables set is saved to this file in
  native nft format on every update.  Restoring the set from this file
  at startup is left to an external tool (e.g. a systemd service or
  init script).


NftablesSet example
~~~~~~~~~~~~~~~~~~~

   [Reporter:nftblock]
   module: NftablesSet
   nft-family: inet
   nft-table: filter
   nft-set: blocklist
   nft-save: /var/lib/nftables/logactio-blocklist.nft

This configures a reporter named nftblock.  Whenever triggered, it
adds an IP address to the pre-existing "inet filter blocklist"
nftables set.  The current set state is saved to the given file after
each update.

To use native expiry, declare the set with "flags timeout" and set a
default "timeout" in the nftables ruleset — nft will then expire
entries automatically without any additional logactio configuration.


Configuring log files
=====================

The configuration sections for the log files are similar to the
reporters. For each configured logfile section logactio will start a
separate worker thread watching the file for new content using
inotify.

There are two configuration variables logactio supports for logfiles:

* file

  This is mandatory, and declares the log file it should watch.

* reporters

  This is optional, but declares the default reporter module(s) to use
  if an event happens to this file.  You may list more reporter
  modules, separated by comma.


Logfile example
---------------

A typical configuration for log file sections would look something
like this:

   [Logfile:messages]
   file: /var/log/messages
   reporters: SMTP-DEMO-1

   [Logfile:maillog]
   file: /var/log/maillog
   reporters: HTTP-DEMO-1, HTTP-DEMO-3

In this example we have configured two Logfile groups, *messages* and
*maillog*. Any event happening in /var/log/messages will by default be
reported using the configured SMTP-DEMO-1 reporter setup.  On changes
in the *maillog*, both the HTTP-DEMO-1 and HTTP-DEMO-3 reporters will
be triggered.


Configuring watch rules
=======================

If you only configure Logfile and Reporter sections, logactio will not
trigger at all.  You need to configure some rules what logactio should
react to.

The rules are based on regular expressions.  And if there is a match
on the log lines received, each of these lines will be acted upon
separately.

A Rules section consists of two required configuration variables:

* logfile

  This is the log file this rule is to be used against

* regex

  This is the regular expression which needs to match to cause the
  reporter to be triggered.  You can also use regex groups, like (.*),
  to extract information from the log line which will be sent to the
  reporter.  If you use multiple groups, all of them will be sent to
  reporter.

* threshold:

  This sets how many times this event should match before triggering
  the reporter.

* threshold-type:

  This defines how the threshold counter works.  By default, it is set
  to *rule*.  This will increase the "hit counter" each time this
  watch rule is triggered.  By setting the *threshold-type* parameter
  to *exact* it will also consider the regex groups defined in the
  *regex* parameter. When using the *exact* type, it will have
  individual threshold counters per group contents for each time this
  rule is triggered.

The optional settings are:

* reporters

  This overrides the default reporters configured in the log file's
  Logfile section.  This can be used to add exceptions or report an
  event differently in special cases.  For example you might want
  developers to get an automatic mail if their program causes an
  exception which is logged.  While a system- administrator might only
  want reports if someone tries to log into a system unsuccessfully
  more than 3 times.  Setting up different Rule sections with
  different reporters and thresholds brings you this power, even if
  everything is logged to the same file.

* time-frame

  This extends the threshold trigger to also consider a time frame
  before trigging an action.  If the threshold is set to 3 and time-
  frame is set to 10, logactio will not trigger an action unless there
  are 3 events within the last 10 seconds.

* rate-limit

  This will restrict logactio from any flood actions.  If this value
  is set to 10 and you have log changes which matches this rule once
  every second, logactio will only perform the configured action once
  per 10 second.

* reset-rule-rate-limits

  This takes a comma separated list of rule names, but only for the
  same log file this rule uses.  This can be used to "unlock" another
  rule's rate-limit restriction.

  This is useful where you might report connection issues only once an
  hour, even though the failed reconnection attempts are logged every
  minute.  But in the moment the connection really is restored you can
  trigger a logactio action informing the connection is back again.
  But if this connection drops after a few minutes again - it might be
  the "connection-failed" rule won't trigger before an hour later.  By
  adding reset-rule-rate-limits on the "connection-is-back" rule, it
  can reset the "connection-failed"'s rate-limit check, so it that
  rule will trigger instantly.


Rule examples
-------------

   [Rule:iptables]
   logfile: messages
   regex: .* (.*) kernel: .* IN=(\S*) OUT=.* MAC=.* SRC=(\S* )DST=(\S* ).* PROTO=(\S* )SPT=(\d* )DPT=(\d* ).*
   threshold: 1

   [Rule:lost-connection]
   logfile: maillog
   regex: lost connection after (.*) from (.*)
   reporters: HTTP-DEMO-2
   threshold: 5

We declare two rules here, one which looks for a certain pattern which
matches iptables and uses the default reporters.  Each time log line
in /var/log/messages matches this rule, the reporters are triggered.
It will report the hostname of the server this happened, the input
device, source and destination IP addresses as well as protocol,
source and destination ports.

The second rule looks for connection issues in /var/maillog and will
report these every 5th issue using only the configured HTTP-DEMO-2
reporter.
