.. SPDX-FileCopyrightText: Copyright 2012-  David Sommerseth <dazo@eurephia.org>
   SPDX-License-Identifier: GPL-2.0-only

.. For notes on how to document Python in RST form, see e.g.:
.. http://sphinx.pocoo.org/domains.html#the-python-domain

Configuring logactio
====================

Logactio makes use of a single ini-styled configuration file.  The default
location for this file is */etc/logactio.cfg*.  This file uses a separate
section, identified by *[section-name]*.  There are three different groups
of sections which you need to configure, *Reporter*, *Logfile* and *Rule*.
Each of these group names have a *label*, where the group name and the label
are separated by a colon (:)


.. index::
   pair: Configuration; Reporters

Configuring Reporters
*********************

There are three different reporters available in logactio, and each of them take
different configuration parameters.  If no reporter is configured, a built-in
reporter called *Default* will be used.

* The Default reporter

   This reporter will only write data via the configured logging methods.  This
   reporter also does not have any configuration settings and does not require
   any specific declaration.

* :ref:`lnk-httpreporter`

   This reporter will send the extracted log data to a web server.  Both
   HTTP and HTTPS may be used.

* :ref:`lnk-smtpreporter`

   This reporter will send the extracted log data via SMTP to one or more
   e-mail recipients.  Support for SSL and STARTTLS in addition to SMTP-AUTH
   are availble as well.

* :ref:`lnk-IPTipset`

   This "reporter" can add extracted IP addresses from a log file and add
   it to an ipset table.  This table can be used by iptables to for example
   block access.

* :ref:`lnk-NftablesSet`

   This "reporter" can add extracted IP addresses from a log file and add
   them to a pre-existing nftables set.

.. index::
   pair: Configuration; HTTPreporter
.. _lnk-httpreporter:

HTTPreporter
------------

This reporter takes the following configuration variables

* method

   This defaults to **GET** if this is not set.  But can be set to **POST**
   if you want the reports to be sent via HTTP POST instead of HTTP GET.

* url

   This is a required setting.  This is the URL where to submit the reports.

* format

   This defaults to **plain** if not set.  Controls how the report data is
   encoded in the request:

   * **plain**: data is URL-encoded (``application/x-www-form-urlencoded``).
     Works with both GET and POST.

   * **json**: data is sent as a JSON object (``application/json``).
     Requires **method** to be set to **POST**.

.. index::
   pair: Examples; HTTPreporter

HTTPreporter example
~~~~~~~~~~~~~~~~~~~~~~

.. code-block :: ini

  [Reporter:HTTP-DEMO-1]
  module: HTTPreporter
  method: POST
  url: http://logactio.example.com/alert/

  [Reporter:HTTP-DEMO-2]
  module: HTTPreporter
  url: http://logactio.example.com/info/

  [Reporter:HTTP-DEMO-3]
  module: HTTPreporter
  method: POST
  url: http://logactio.example.com/alert/json/
  format: json

Here three reporters are configured.  *HTTP-DEMO-1* will use HTTP POST with
URL-encoded data, *HTTP-DEMO-2* will use HTTP GET with URL-encoded data, and
*HTTP-DEMO-3* will use HTTP POST sending a JSON body.

.. index::
   pair: Configuration; SMTPreporter
.. _lnk-smtpreporter:

SMTPreporter
------------
This reporter requires the following configuration variables:

* sender

  The e-mail address which will be used in the "From:" field when sending
  mails

* recipients

  This contains a comma separated list with all e-mail addresses who will
  get the logactio reports.

* smtp_host

  This declares which SMTP server to use when sending the reports.

In addition the SMTPreporter supports these optional variables:

* subject_prefix

   The default subject prefix is set to 'LogActio Alert: '.  By setting
   this variable, the subject prefix will be changed accordingly.

* smtp_port

   The default value is set to port 25.

* smtp_username

   If the SMTP server requires authentication to relay messages, this
   variable sets the SMTP user name.  To use this feature, you must also
   set the smtp_password.

* smtp_password

   This sets the SMTP password to use for the authentication

* sslmode

   This is not set by default, so everything goes in clear text.  If your SMTP
   server supports either SSL or STARTTLS, you can set it to SSL or STARTTLS.
   In SSL mode the SMTP library expects to the server to do the SSL handshake
   before the SMTP commands can be sent.  In STARTTLS mode, the SMTP library
   will connect to the SMTP server in clear text and if the server supports
   STARTTLS, it will send the STARTTLS command and start the SSL handshake.

.. index::
   pair: Examples; SMTPreporter

SMTPreporter example
~~~~~~~~~~~~~~~~~~~~~~

.. code-block :: ini

   [Reporter:SMTP-DEMO-1]
   module: SMTPreporter
   sender: logactio@example.com
   recipients: john.doe@example.com, jane.doe@example.com
   smtp_host: localhost

   [Reporter:SMTP-DEMO-2]
   module: SMTPreporter
   sender: john.doe@example.com
   recipients: bob.external@acme.com
   smtp_host: smtp.example.com
   smtp_port: 587
   sslmode: STARTTLS
   smtp_username: logactioSMTP
   smtp_password: S3cretP4ssw0rd
   subject_prefix: Issues at Example Corp:

Here two more reporters are configured.  SMTP-DEMO-1 will use the SMTP MTA
running locally on the system, and this will not require any authentication
or SSL functionality.  When this reporter is triggered, it will send mails
to john.doe and jane.doe with the From field set to logactio@example.com.

The SMTP-DEMO-2 reporter will send mails using an external SMTP server using
port 587 and which requires STARTTLS and authentication.  The subject line
is also prefixed differently.


.. index::
   pair: Configuration; IPTipset
.. _lnk-IPTipset:

IPTipset
--------
To use this module, you need ipset and iptables installed and available.  This
will also require logactio to run with root privileges, otherwise it lacks
privileges to update the ipset and iptables rules.

.. IMPORTANT::
   This module expects only a single regex group to be extracted.  This value
   **has** to be an IP address, otherwise the IPTipset module will fail when
   executing the ipset command.

The required configuration variables are:

* ipset-name
  This is the name of the ipset table to this specific section will use.

Optional settings are:

* ipset-create

  Boolean variable which tells wether the IPTipset module should create
  the ipset table when starting or not.  The values of "true" (case
  or "1" will be understood as True, all other values as False.

* ipset-hashsize, ipset-timeout, ipset-counters

  These variables will be sent further to the 'ipset create' command and will
  set hashsize, timeout and the counters flag.  ipset-counters is a boolean
  flag, where the value of "true" (case insensitive) or "1" will be understood
  as True, all other values as False.  The default value for ipset-hashsize
  is 1024, ipset-timeout is not defined and ipset-counters is False by default.

* ipset-save

  This will enable the IPTipset module to save the state of the ipset table
  when exiting.  If this file exists when LogActio is started again later on,
  this state will be loaded automatically.

* iptables-chains

  If set, LogActio will insert the ipset checks into the listed iptables
  chains.  Multiple chains can be listed, separated by comma.

* iptables-insert-points

  This is optional but only available if iptables-chains are used.   This
  defines where in the different iptables chains the ipset rules will be
  inserted.  By default it will be inserted at the top of the chain.  The
  syntax is CHAIN:NUM, where CHAIN is the name provided in *iptables-chains*
  and NUM is the value used when calling iptables.  The result is something
  like 'iptables -I CHAIN NUM ...'.  Multiple chains can be defined,
  separated by comma.

* iptables-jump

  This is required when using *iptables-chains*.  This defines what to do
  when iptables get a positive match against the ipset table.  This sets the
  '-j' options when iptables commands are executed.

.. index::
   pair: Examples; IPTipset

IPTipset example
~~~~~~~~~~~~~~~~

.. code-block :: ini

    [Reporter:ipsetblock]
    module: IPTipset
    ipset-name: BlockList
    ipset-create: True
    ipset-hashsize: 2048
    ipset-timeout: 3600
    ipset-counters: False
    ipset-save: /var/lib/ipset/logactio-ipset.save
    iptables-chains: INPUT,OUTPUT,FORWARD
    iptables-insert-points: INPUT:2,FORWARD:4
    iptables-jump: DROP

This will configure a reported named ipsetblock.  Whenever this reporter
is called, it will add an IP address to an ipset table named 'Blocklist'.
This list will be created when LogActio starts, with a hashsize of 2048
and a timeout of 1 hour (3600 seconds).  The ipset table state will be
saved to /var/lib/ipset/logactio-ipset.save.  When starting LogActio
the following commands will be run:

.. code-block :: shell

    # ipset --exist create BlockList hash:ipset hashsize 2048 timeout 3600
    # iptables -I INPUT 2 -m set --match-set BlockList src -j DROP
    # iptables -I OUTPUT -m set --match-set BlockList src -j DROP
    # iptables -I FORWARD 4 -m set --match-set BlockList src -j DROP


.. index::
   pair: Configuration; NftablesSet
.. _lnk-NftablesSet:

NftablesSet
-----------
To use this module, you need nftables installed and available.  This will also
require logactio to run with root privileges, otherwise it lacks privileges to
update the nftables set.

The nftables set must be pre-created before starting logactio.  This module
will not create or destroy the set.

.. IMPORTANT::
   This module expects only a single regex group to be extracted.  This value
   **has** to be an IP address, otherwise the NftablesSet module will fail when
   executing the nft command.

The required configuration variables are:

* nft-table

  The name of the nftables table containing the set.

* nft-set

  The name of the nftables set to add IP addresses to.

Optional settings are:

* nft-family

  The nftables family of the table.  Defaults to **inet**.

* nft-save

  If set, the state of the nftables set is saved to this file in native nft
  format on every update.  Restoring the set from this file at startup is
  left to an external tool (e.g. a systemd service or init script).

.. index::
   pair: Examples; NftablesSet

NftablesSet example
~~~~~~~~~~~~~~~~~~~

.. code-block :: ini

    [Reporter:nftblock]
    module: NftablesSet
    nft-family: inet
    nft-table: filter
    nft-set: blocklist
    nft-save: /var/lib/nftables/logactio-blocklist.nft

This configures a reporter named nftblock.  Whenever triggered, it adds an
IP address to the pre-existing ``inet filter blocklist`` nftables set.  The
current set state is saved to the given file after each update.

To use native expiry, declare the set with ``flags timeout`` and set a default
``timeout`` in the nftables ruleset — nft will then expire entries automatically
without any additional logactio configuration.


.. index::
   single: Log files

Configuring log files
*********************
The configuration sections for the log files are similar to the reporters.
For each configured logfile section logactio will start a separate worker
thread watching the file for new content using inotify.

There are two configuration variables logactio supports for logfiles:

* file

  This is mandatory, and declares the log file it should watch.

* reporters

  This is optional, but declares the default reporter module(s) to use if an
  event happens to this file.  You may list more reporter modules, separated
  by comma.

Logfile example
---------------

A typical configuration for log file sections would look something like this:

.. code-block :: ini

   [Logfile:messages]
   file: /var/log/messages
   reporters: SMTP-DEMO-1

   [Logfile:maillog]
   file: /var/log/maillog
   reporters: HTTP-DEMO-1, HTTP-DEMO-3

In this example we have configured two Logfile groups, *messages* and *maillog*.
Any event happening in /var/log/messages will by default be reported using the
configured SMTP-DEMO-1 reporter setup.  On changes in the *maillog*, both the
HTTP-DEMO-1 and HTTP-DEMO-3 reporters will be triggered.

.. index::
   single: Rules

Configuring watch rules
***********************
If you only configure Logfile and Reporter sections, logactio will not trigger
at all.  You need to configure some rules what logactio should react to.

The rules are based on regular expressions.  And if there is a match on the
log lines received, each of these lines will be acted upon separately.

A Rules section consists of two required configuration variables:

* logfile

  This is the log file this rule is to be used against

* regex

  This is the regular expression which needs to match to cause the reporter
  to be triggered.  You can also use regex groups, like (.*), to extract
  information from the log line which will be sent to the reporter.  If
  you use multiple groups, all of them will be sent to reporter.

* threshold:

  This sets how many times this event should match before triggering the
  reporter.

* threshold-type:

  This defines how the threshold counter works.  By default, it is set to
  *rule*.  This will increase the "hit counter" each time this watch rule
  is triggered.  By setting the *threshold-type* parameter to *exact* it
  will also consider the regex groups defined in the *regex* parameter.
  When using the *exact* type, it will have individual threshold counters
  per group contents for each time this rule is triggered.

The optional settings are:

* reporters

  This overrides the default reporters configured in the log file's Logfile
  section.  This can be used to add exceptions or report an event differently
  in special cases.  For example you might want developers to get an automatic
  mail if their program causes an exception which is logged.  While a system-
  administrator might only want reports if someone tries to log into a system
  unsuccessfully more than 3 times.  Setting up different Rule sections with
  different reporters and thresholds brings you this power, even if everything
  is logged to the same file.

* time-frame

  This extends the threshold trigger to also consider a time frame before
  trigging an action.  If the threshold is set to 3 and time-frame is set
  to 10, logactio will not trigger an action unless there are 3 events
  within the last 10 seconds.

* rate-limit

  This will restrict logactio from any flood actions.  If this value is set to
  10 and you have log changes which matches this rule once every second,
  logactio will only perform the configured action once per 10 second.

* reset-rule-rate-limits

  This takes a comma separated list of rule names, but only for the same log
  file this rule uses.  This can be used to "unlock" another rule's rate-limit
  restriction.

  This is useful where you might report connection issues only once an hour,
  even though the failed reconnection attempts are logged every minute.  But
  in the moment the connection really is restored you can trigger a logactio
  action informing the connection is back again.  But if this connection drops
  after a few minutes again - it might be the "connection-failed" rule won't
  trigger before an hour later.  By adding reset-rule-rate-limits on the
  "connection-is-back" rule, it can reset the "connection-failed"'s rate-limit
  check, so it that rule will trigger instantly.

Rule examples
-------------
.. code-block :: ini

   [Rule:iptables]
   logfile: messages
   regex: .* (.*) kernel: .* IN=(\S*) OUT=.* MAC=.* SRC=(\S* )DST=(\S* ).* PROTO=(\S* )SPT=(\d* )DPT=(\d* ).*
   threshold: 1

   [Rule:lost-connection]
   logfile: maillog
   regex: lost connection after (.*) from (.*)
   reporters: HTTP-DEMO-2
   threshold: 5

We declare two rules here, one which looks for a certain pattern which matches
iptables and uses the default reporters.  Each time log line in /var/log/messages
matches this rule, the reporters are triggered.  It will report the hostname of
the server this happened, the input device, source and destination IP addresses
as well as protocol, source and destination ports.

The second rule looks for connection issues in /var/maillog and will report
these every 5th issue using only the configured HTTP-DEMO-2 reporter.
