SYSLOGCEF(1)

NAME
syslogcef - convert syslog events to ArcSight Common Event Format (CEF)

SYNOPSIS
syslogcef [OPTIONS] [PATHS...]
syslogcef --listen PROTO:PORT [OPTIONS]

DESCRIPTION
syslogcef converts syslog events into ArcSight Common Event Format (CEF) for consumption by SIEM platforms. It reads from files, standard input, or the network, and writes CEF records to standard output, a file, a remote SIEM, or a Kafka topic.

Ten input formats are detected automatically: RFC3164 (BSD, with or without a year in the timestamp), RFC5424 (including structured data), rsyslog JSON and file formats, systemd journal exports (JSON, short, and ISO), native Cisco console output (sequence numbers and clock-state markers), ISO-timestamp syslog with PRI, and key=value streams (Fortinet FortiGate, Sophos XG). Lines matching none of these are analyzed adaptively: a timestamp shape and host candidate are located and a pattern for the layout is synthesized and cached. Unparseable lines never abort the stream; they fall back to a raw-message event.

Vendor mappings for Cisco ASA/IOS, F5 BIG-IP, Fortinet, Sophos, VMware, and generic Linux are bundled and selected automatically from message content. Log content is treated as untrusted input: CEF header and extension values are escaped so crafted messages cannot forge header fields or split records.

OPTIONS

  Input

PATHS
    Input files. When omitted and --listen is not given, lines are read from standard input.

--tail
    Follow the input files like tail -f; all given files are tailed concurrently.

--listen PROTO:PORT
    Receive syslog over the network instead of reading files or standard input. Accepts udp:PORT, tcp:PORT, or PROTO:HOST:PORT to bind a specific address (default: all interfaces). TCP accepts multiple concurrent connections with newline-delimited messages. Overrides PATHS and --tail.

--mode MODE
    Force a specific parser instead of auto-detection. One of rfc3164, rfc5424, rsyslog_json, rsyslog_file, journald_json, journald_short, journald_iso, cisco_seq, iso_syslog, kv.

  Conversion

--mapping FILE
    Use the mapping JSON file FILE. When omitted, a bundled mapping is selected automatically from message content.

--patterns FILE
    Load custom parser patterns from the JSON file FILE (repeatable). Each pattern has a name, a regex whose named groups map to event fields (pri, host, app, pid, msgid, msg, ts), a timestamp_format (strptime format, iso8601, or epoch; required when a ts group is present), and an optional priority: after (default; tried after the built-in parsers, before the adaptive fallback) or before (tried first). Pattern names are accepted by --mode. Files are validated at startup; invalid regexes, unknown groups, or duplicate names fail immediately with a clear message. Patterns run against untrusted log content — keep them anchored and avoid nested quantifiers.

--validate
    Check CEF extension values against the ArcSight extension dictionary (known keys, data types, port ranges, IP/MAC/timestamp formats, maximum string lengths). Violations are logged as warnings; conversion continues. Unknown keys are advisory only, as the CEF standard permits custom keys.

--strict
    Like --validate, but exit with status 1 on the first type or length violation.

--multiprocess
    Convert lines using a process pool.

--pool-size N
    Worker count for --multiprocess (default: CPU count minus one).

  Output

-o, --output FILE
    Write CEF lines to FILE instead of standard output. Flushed per line in --tail and --listen modes. FILE may contain strftime(3) codes, for example /var/log/syslogcef/%Y-%m-%d/events-%H.cef: the file is reopened whenever the rendered path changes (hourly in this example), parent directories are created as needed, and templated paths are always opened in append mode. Unsupported or platform-specific codes are rejected at startup; write %% for a literal percent. An empty FILE is treated as standard output.

-a, --append
    Append to the output file instead of truncating it (implied for strftime-templated paths). Used by the system service so restarts never lose archived events.

--send URL
    Forward each CEF record to a remote destination:

    udp://HOST:PORT
        one datagram per record.

    tcp://HOST:PORT
        newline-delimited stream with automatic reconnect and backoff.

    kafka://BROKER:PORT/TOPIC
        produce to a Kafka topic; requires the optional dependency (pip install syslog2cef[kafka]).

    Combinable with --output to archive locally while forwarding, and with --listen for a complete receive-convert-forward daemon.

--eps N
    Rate-limit --send to at most N events per second.

  General

--log-level LEVEL
    Python logging level (default WARNING). Validation findings are emitted at WARNING.

EXIT STATUS

0   Success.

1   Strict validation failed, an input file could not be read, or a network destination was unreachable after retries.

2   Invalid command-line arguments.

EXAMPLES
Convert a log file to CEF on standard output:

    syslogcef < /var/log/syslog

Convert several files into one archive:

    syslogcef /var/log/messages /var/log/secure -o events.cef

Follow two files in real time, appending CEF to an archive:

    syslogcef /var/log/asa.log /var/log/messages --tail --append -o events.cef

Act as a syslog receiver and forward CEF to an ArcSight connector:

    syslogcef --listen udp:514 --send tcp://siem.example.com:514

Receive on a high port (no privileges needed), archive locally, and forward rate-limited to 5000 events per second:

    syslogcef --listen tcp:5514 --append -o /var/log/syslogcef/events.cef \
              --send udp://siem.example.com:514 --eps 5000

Stream converted events into Kafka:

    syslogcef /var/log/messages --tail --send kafka://broker.example.com:9092/cef-events

Force the Cisco ASA mapping and validate the output strictly:

    syslogcef asa.log --mapping /etc/syslogcef/mapping.json --strict

High-volume batch conversion with a process pool:

    syslogcef big.log --multiprocess --pool-size 8 -o big.cef

FILES

/etc/syslogcef/syslogcef.conf
    Configuration for the system service (environment file): input file or network endpoints, output file, and extra arguments. To run the service as a network daemon, leave INPUT_FILE empty and set, for example, EXTRA_ARGS="--listen udp:514 --send tcp://siem:514".

/etc/syslogcef/conf.d/*.conf
    Per-instance configuration for the syslogcef@.service template unit. Each file defines one independent pipeline (its own input, output, mapping, and restart policy); start one with systemctl enable --now syslogcef@name, where the instance name selects the matching conf.d file. A commented sample is installed as example.conf.sample. On Alpine, create an OpenRC instance by symlinking /etc/init.d/syslogcef to syslogcef.name with its own /etc/conf.d/syslogcef.name.

/etc/logrotate.d/syslogcef
    Rotates flat .cef archives under /var/log/syslogcef daily using copytruncate (the service keeps its output open). Outputs using strftime templates rotate themselves and are not matched.

syslogcef.service
    systemd unit (RPM and Debian packages). By default the service runs as root, which is what allows it to read arbitrary log files. The unit grants CAP_NET_BIND_SERVICE so that administrators who drop privileges (by setting User= in an override, with a group able to read the input logs) can still bind ports below 1024. On Alpine the equivalent OpenRC service is configured via /etc/conf.d/syslogcef.

syslogcef.mappings
    Bundled mapping files, importable from the Python package (from syslogcef.mappings import load_mapping).

SECURITY
All log content is treated as untrusted. Pipes and backslashes are escaped in CEF header fields (CR/LF stripped) and pipes, equals signs, backslashes, and CR/LF are escaped in extension values, preventing header forgery and record splitting. The listener decodes input as UTF-8 with replacement and never evaluates message content. Report vulnerabilities privately; see the SECURITY.md file in the source distribution.

SEE ALSO
rsyslogd(8), journalctl(1), logger(1)

Project homepage and issue tracker: https://github.com/allamiro/syslogcef

AUTHOR
Tamir Suliman <allamiro@gmail.com>