nvme-stas 1.0

Name

stacd.conf — stacd(8) configuration file

Synopsis

/etc/stas/stacd.conf

Description

When stacd(8) starts up, it reads its configuration from stacd.conf.

Configuration File Format

stacd.conf is a plain text file divided into sections, with configuration entries in the style key=value. Whitespace immediately before or after the "=" is ignored. Empty lines and lines starting with "#" are ignored, which may be used for commenting.

Options

[Global] section

The following options are available in the "[Global]"section:

tron=

Trace ON. Takes a boolean argument. If "true", enables full code tracing. The trace will be displayed in the system log such as systemd's journal. Defaults to "false".

hdr-digest=

Enable Protocol Data Unit (PDU) Header Digest. Takes a boolean argument. NVMe/TCP facilitates an optional PDU Header digest. Digests are calculated using the CRC32C algorithm. If "true", Header Digests are inserted in PDUs and checked for errors. Defaults to "false".

data-digest=

Enable Protocol Data Unit (PDU) Data Digest. Takes a boolean argument. NVMe/TCP facilitates an optional PDU Data digest. Digests are calculated using the CRC32C algorithm. If "true", Data Digests are inserted in PDUs and checked for errors. Defaults to "false".

kato=

Keep Alive Timeout (KATO) in seconds. Takes an unsigned integer. This field specifies the timeout value for the Keep Alive feature in seconds. Defaults to 30 seconds for Discovery Controller connections and 120 seconds for I/O Controller connections.

ip-family=

Takes a string argument. With this you can specify whether IPv4, IPv6, or both are supported when connecting to a Controller. Connections will not be attempted to IP addresses (whether discovered or manually configured with the 'controller') if those IP addresses are disabled by this option. If an invalid value is entered, then "ipv4+ipv6" will be used by default.

Choices are "ipv4", "ipv6", or "ipv4+ipv6".

Defaults to "ipv4+ipv6".

ignore-iface=

Takes a boolean argument. This option controls how connections with I/O Controllers (IOC) are made.

There is no guarantee that there will be a route to reach that IOC. However, we can use the socket option SO_BINDTODEVICE to force the connection to be made on a specific interface instead of letting the routing tables decide where to make the connection.

This option determines whether stacd will use SO_BINDTODEVICE to force connections on an interface or just rely on the routing tables. The default is to use SO_BINDTODEVICE, in other words, stacd does not ignore the interface.

BACKGROUND: By default, stacd will connect to IOCs on the same interface that was used to retrieve the discovery log pages. If stafd discovers a DC on an interface using mDNS, and stafd connects to that DC and retrieves the log pages, it is expected that the storage subsystems listed in the log pages are reachable on the same interface where the DC was discovered.

For example, let's say a DC is discovered on interface ens102. Then all the subsystems listed in the log pages retrieved from that DC must be reachable on interface ens102. If this doesn't work, for example you cannot "ping -I ens102 [storage-ip]", then the most likely explanation is that arp proxy is not enabled on the switch that the host is connected to on interface ens102. Whatever you do, resist the temptation to manually set up the routing tables or to add alternate routes going over a different interface than the one where the DC is located. That simply won't work. Make sure arp proxy is enabled on the switch first.

Setting routes won't work because, by default, stacd uses the SO_BINDTODEVICE socket option when it connects to IOCs. This option is used to force a socket connection to be made on a specific interface instead of letting the routing tables decide where to connect the socket. Even if you were to manually configure an alternate route on a different interface, the connections (i.e. host to IOC) will still be made on the interface where the DC was discovered by stafd.

Defaults to "false".

[Controllers] section

The following options are available in the "[Controllers]" section:

controller=

Controllers are specified with the "controller" option. This option may be specified more than once to specify more than one controller. The format is one line per Controller composed of a series of fields separated by semi-colons as follows:

controller=transport=[trtype];traddr=[traddr];trsvcid=[trsvcid];host-traddr=[traddr],host-iface=[iface];nqn=[nqn]
                

Fields

transport=

This is a mandatory field that specifies the network fabric being used for a NVMe-over-Fabrics network. Current "trtype" values understood are:

Table 1. Transport type

trtypeDefinition
rdma The network fabric is an rdma network (RoCE, iWARP, Infiniband, basic rdma, etc)
fc The network fabric is a Fibre Channel network.
tcp The network fabric is a TCP/IP network.
loop Connect to a NVMe over Fabrics target on the local host

traddr=

This is a mandatory field that specifies the network address of the Controller. For transports using IP addressing (e.g. rdma) this should be an IP-based address (ex. IPv4, IPv6). It could also be a resolvable host name (e.g. localhost).

trsvcid=

This is an optional field that specifies the transport service id. For transports using IP addressing (e.g. rdma, tcp) this field is the port number.

Depending on the transport type, this field will default to either 8009 or 4420 as follows.

UDP port 4420 and TCP port 4420 have been assigned by IANA for use by NVMe over Fabrics. NVMe/RoCEv2 controllers use UDP port 4420 by default. NVMe/iWARP controllers use TCP port 4420 by default.

TCP port 4420 has been assigned for use by NVMe over Fabrics and TCP port 8009 has been assigned by IANA for use by NVMe over Fabrics discovery. TCP port 8009 is the default TCP port for NVMe/TCP discovery controllers. There is no default TCP port for NVMe/TCP I/O controllers, the Transport Service Identifier (TRSVCID) field in the Discovery Log Entry indicates the TCP port to use.

The TCP ports that may be used for NVMe/TCP I/O controllers include TCP port 4420, and the Dynamic and/or Private TCP ports (i.e., ports in the TCP port number range from 49152 to 65535). NVMe/TCP I/O controllers should not use TCP port 8009. TCP port 4420 shall not be used for both NVMe/iWARP and NVMe/TCP at the same IP address on the same network.

Ref: IANA Service names port numbers

nqn=

This is an optional field that specifies the Discovery Controller's NVMe Qualified Name. If not specified, this will default to the well-known DC NQN: "nqn.2014-08.org.nvmexpress.discovery".

host-traddr=

This is an optional field that specifies the network address used on the host to connect to the Controller. For TCP, this sets the source address on the socket.

host-iface=

This is an optional field that specifies the network interface used on the host to connect to the Controller (e.g. IP eth1, enp2s0, enx78e7d1ea46da). This forces the connection to be made on a specific interface instead of letting the system decide.

Examples:

controller = transport=tcp;traddr=localhost;trsvcid=8009
controller = transport=tcp;traddr=[2001:db8:::370:7334];host-iface=enp0s8
controller = transport=fc;traddr=nn-0x204600a098cbcac6:pn-0x204700a098cbcac6
                    

blacklist=

Blacklisted controllers can be specified with the "blacklist" option. Using mDNS to automatically discover and connect to controllers, can result in unintentional connections being made. This keyword allows configuring the controllers that should not be connected to (whatever the reason may be).

The syntax is the same as for "controller", except that the key "host-traddr" does not apply. Multiple "blacklist" keywords may appear in the config file to specify more than 1 blacklisted controller.

Note 1: A minimal match approach is used to eliminate unwanted controllers. That is, you do not need to specify all the parameters to identify a controller. Just specifying the "host-iface", for example, can be used to blacklist all controllers on an interface.

Note 2: "blacklist" takes precedence over "controller". A controller specified by the "controller" keyword, can be eliminated by the "blacklist" keyword.

Examples:

blacklist = transport=tcp;traddr=fe80::2c6e:dee7:857:26bb # Eliminate a specific address
blacklist = host-iface=enp0s8                             # Eliminate everything on this interface
                    

See Also

stacd(8)