construct.core
– entire module¶
-
exception
construct.core.
ConstructError
¶
-
exception
construct.core.
FieldError
¶
-
exception
construct.core.
SizeofError
¶
-
exception
construct.core.
AdaptationError
¶
-
exception
construct.core.
RangeError
¶
-
exception
construct.core.
SwitchError
¶
-
exception
construct.core.
SelectError
¶
-
exception
construct.core.
UnionError
¶
-
exception
construct.core.
FocusedError
¶
-
exception
construct.core.
TerminatedError
¶
-
exception
construct.core.
OverwriteError
¶
-
exception
construct.core.
PaddingError
¶
-
exception
construct.core.
ConstError
¶
-
exception
construct.core.
StringError
¶
-
exception
construct.core.
ChecksumError
¶
-
exception
construct.core.
ValidationError
¶
-
exception
construct.core.
BitIntegerError
¶
-
exception
construct.core.
MappingError
¶
-
exception
construct.core.
ExplicitError
¶
-
class
construct.core.
Construct
¶ The mother of all constructs.
This object is generally not directly instantiated, and it does not directly implement parsing and building, so it is largely only of interest to subclass implementors. There are also other abstract classes.
The external user API:
parse()
parse_stream()
build()
build_stream()
sizeof()
Subclass authors should not override the external methods. Instead, another API is available:
_parse()
_build()
_sizeof()
There is also a flag API:
_inherit_flags()
And stateful copying:
__getstate__()
__setstate__()
All constructs have a name and flags. The name is used for naming struct members and context dictionaries. Note that the name can either be a string, or None if the name is not needed. A single underscore (“_”) is a reserved name, and so are names starting with a less-than character (“<”). The name should be descriptive, short, and valid as a Python identifier, although these rules are not enforced.
The flags specify additional behavioral information about this construct. Flags are used by enclosing constructs to determine a proper course of action. Flags are inherited by default, from inner subconstructs to outer constructs. The enclosing construct may set new flags or clear existing ones, as necessary.
-
parse
(data, context=None, **kw)¶ Parse an in-memory buffer.
Strings, buffers, memoryviews, and other complete buffers can be parsed with this method.
-
parse_stream
(stream, context=None, **kw)¶ Parse a stream.
Files, pipes, sockets, and other streaming sources of data are handled by this method.
-
build
(obj, context=None, **kw)¶ Build an object in memory.
- Returns
bytes
-
build_stream
(obj, stream, context=None, **kw)¶ Build an object directly into a stream.
- Returns
None
-
sizeof
(context=None, **kw)¶ Calculate the size of this object, optionally using a context.
Some constructs have no fixed size and can only know their size for a given hunk of data. These constructs will raise an error if they are not passed a context.
- Parameters
context – a container
- Returns
int of the length of this construct
- Raises
SizeofError – the size could not be determined
-
class
construct.core.
Subconstruct
(subcon)¶ Abstract subconstruct (wraps an inner construct, inheriting its name and flags). Parsing and building is by default deferred to subcon, so it sizeof.
Subconstructs wrap an inner Construct, inheriting its name and flags.
- Parameters
subcon – the construct to wrap
-
class
construct.core.
Adapter
(subcon)¶ Abstract adapter parent class.
Needs to implement
_decode()
and_encode()
.- Parameters
subcon – the construct to wrap
-
class
construct.core.
SymmetricAdapter
(subcon)¶ Abstract adapter parent class.
Needs to implement
_decode()
only. Encoding is done by same method.- Parameters
subcon – the construct to wrap
-
class
construct.core.
Validator
(subcon)¶ Abstract class: validates a condition on the encoded/decoded object.
Needs to implement
_validate()
that returns bool.- Parameters
subcon – the subcon to validate
-
class
construct.core.
Tunnel
(subcon)¶
-
class
construct.core.
Bytes
(length)¶ A field consisting of a specified number of bytes. Builds from a b-string, or an integer (although deprecated and BytesInteger should be used).
See also
Analog
BytesInteger()
that parses and builds from integers.- Parameters
length – an int or a function that takes context and returns int
Example:
>>> Bytes(4).parse(b"beef") b'beef' >>> Bytes(4).build(_) b'beef' >>> Bytes(4).build(255) b'\x00\x00\x00\xff' >>> Bytes(4).sizeof() 4
-
class
construct.core.
FormatField
(endianity, format)¶ A field that uses
struct
module to pack and unpack data. This is used to implement basic Int* fields.See
struct
documentation for instructions on crafting format strings.- Parameters
endianity – format endianness string as one of: < > =
format – single format character like: f d B H L Q b h l q
Example:
>>> FormatField(">","H").parse(b"\x01\x00") 256 >>> FormatField(">","H").build(18) b'\x00\x12' >>> FormatField(">","H").sizeof() 2
-
construct.core.
Bitwise
(subcon)¶ Converts the stream from bytes to bits, and passes the bitstream to underlying subcon.
See also
Analog
Bytewise()
that transforms subset of bits back to bytes.Warning
Do not use pointers inside.
- Parameters
subcon – any field that works with bits like: BitStruct BitsNumber Bit Nibble Octet
Example:
>>> Bitwise(Octet).parse(b"\xff") 255 >>> Bitwise(Octet).build(1) b'\x01' >>> Bitwise(Octet).sizeof() 1
-
construct.core.
Bytewise
(subcon)¶ Converts the stream from bits back to bytes. Needs to be used within Bitwise.
- Parameters
subcon – any field that works with bytes like: Bytes BytesInteger Int* Struct
Example:
>>> Bitwise(Bytewise(Byte)).parse(b"\xff") 255 >>> Bitwise(Bytewise(Byte)).build(63) b'?' >>> Bitwise(Bytewise(Byte)).sizeof() 1
-
class
construct.core.
BytesInteger
(length, signed=False, swapped=False, bytesize=1)¶ A byte field, that parses into and builds from integers as opposed to b-strings. This is similar to Int* fields but can be much longer than 4 or 8 bytes.
See also
Analog
BitsInteger()
that operatoes on bits.- Parameters
length – number of bytes in the field, or a function that takes context and returns int
signed – whether the value is signed (two’s complement), default is False (unsigned)
swapped – whether to swap byte order (little endian), default is False (big endian)
bytesize – size of byte as used for byte swapping (if swapped), default is 1
Example:
>>> BytesInteger(4).parse(b"abcd") 1633837924 >>> BytesInteger(4).build(1) b'\x00\x00\x00\x01' >>> BytesInteger(4).sizeof() 4
-
class
construct.core.
BitsInteger
(length, signed=False, swapped=False, bytesize=8)¶ A byte field, that parses into and builds from integers as opposed to b-strings. This is similar to Bit/Nibble/Octet fields but can be much longer than 1/4/8 bits. This must be encosed in Bitwise.
- Parameters
length – number of bits in the field, or a function that takes context and returns int
signed – whether the value is signed (two’s complement), default is False (unsigned)
swapped – whether to swap byte order (little endian), default is False (big endian)
bytesize – size of byte as used for byte swapping (if swapped), default is 8
Example:
>>> Bitwise(BitsInteger(8)).parse(b"\x10") 16 >>> Bitwise(BitsInteger(8)).build(255) b'\xff' >>> Bitwise(BitsInteger(8)).sizeof() 1
-
class
construct.core.
Struct
(*subcons, **kw)¶ A sequence of usually named constructs, similar to structs in C. The elements are parsed and built in the order they are defined.
Some fields do not need to be named, since they are built from None anyway. See Const Padding Pass Terminated.
See also
Can be nested easily, and embedded using
Embedded()
wrapper that merges members into parent’s members.- Parameters
subcons – a sequence of subconstructs that make up this structure
Example:
>>> Struct("a"/Int8ul, "data"/Bytes(2), "data2"/Bytes(this.a)).parse(b"\x01abc") Container(a=1)(data=b'ab')(data2=b'c') >>> Struct("a"/Int8ul, "data"/Bytes(2), "data2"/Bytes(this.a)).build(_) b'\x01abc' >>> Struct("a"/Int8ul, "data"/Bytes(2), "data2"/Bytes(this.a)).build(dict(a=5, data=b"??", data2=b"hello")) b'\x05??hello' >>> Struct(Const(b"MZ"), Padding(2), Pass, Terminated).build({}) b'MZ\x00\x00' >>> Struct(Const(b"MZ"), Padding(2), Pass, Terminated).parse(_) Container() >>> Struct(Const(b"MZ"), Padding(2), Pass, Terminated).sizeof() 4 Note that this syntax works ONLY on python 3.6 and pypy due to unordered keyword arguments: >>> Struct(a=Byte, b=Byte, c=Byte, d=Byte)
-
class
construct.core.
Sequence
(*subcons, **kw)¶ A sequence of unnamed constructs. The elements are parsed and built in the order they are defined.
See also
Can be nested easily, and embedded using
Embedded()
wrapper that merges entries into parent’s entries.- Parameters
subcons – a sequence of subconstructs that make up this sequence
Example:
>>> (Byte >> Byte).build([1, 2]) b'\x01\x02' >>> (Byte >> Byte).parse(_) [1, 2] >>> (Byte >> Byte).sizeof() 2 >>> Sequence(Byte, CString(), Float32b).build([255, b"hello", 123]) b'\xffhello\x00B\xf6\x00\x00' >>> Sequence(Byte, CString(), Float32b).parse(_) [255, b'hello', 123.0]
-
class
construct.core.
Range
(min, max, subcon)¶ A homogenous array of elements. The array will iterate through between
min
tomax
times. If an exception occurs (EOF, validation error), the repeater exits cleanly. If less thanmin
units have been successfully parsed, a RangeError is raised.See also
Analog
GreedyRange()
that parses until end of stream.Note
This object requires a seekable stream for parsing.
- Parameters
min – the minimal count
max – the maximal count
subcon – the subcon to process individual elements
Example:
>>> Range(3, 5, Byte).build([1,2,3,4]) b'\x01\x02\x03\x04' >>> Range(3, 5, Byte).parse(_) [1, 2, 3, 4] >>> Range(3, 5, Byte).build([1,2]) construct.core.RangeError: expected from 3 to 5 elements, found 2 >>> Range(3, 5, Byte).build([1,2,3,4,5,6]) construct.core.RangeError: expected from 3 to 5 elements, found 6
-
construct.core.
GreedyRange
(subcon)¶ A homogenous array of elements that parses until end of stream and builds from all elements.
- Parameters
subcon – the subcon to process individual elements
Example:
>>> GreedyRange(Byte).build(range(10)) b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\t' >>> GreedyRange(Byte).parse(_) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
-
construct.core.
Array
(count, subcon)¶ A homogenous array of elements. The array will iterate through exactly
count
elements. Will raise RangeError if less elements are found.See also
Base
Range()
construct.- Parameters
count – int or a function that takes context and returns the number of elements
subcon – the subcon to process individual elements
Example:
>>> Byte[5].build(range(5)) b'\x00\x01\x02\x03\x04' >>> Byte[5].parse(_) [0, 1, 2, 3, 4] >>> Array(5, Byte).build(range(5)) b'\x00\x01\x02\x03\x04' >>> Array(5, Byte).parse(_) [0, 1, 2, 3, 4]
-
class
construct.core.
PrefixedArray
(lengthfield, subcon)¶ An array prefixed by a length field.
See also
Analog
Array()
construct.- Parameters
lengthfield – a field parsing and building an integer
subcon – the subcon to process individual elements
Example:
>>> PrefixedArray(Byte, Byte).build(range(5)) b'\x05\x00\x01\x02\x03\x04' >>> PrefixedArray(Byte, Byte).parse(_) [0, 1, 2, 3, 4]
-
class
construct.core.
RepeatUntil
(predicate, subcon)¶ An array that repeats until the predicate indicates it to stop. Note that the last element (which caused the repeat to exit) is included in the return value.
- Parameters
predicate – a predicate function that takes (obj, context) and returns True to break, or False to continue
subcon – the subcon used to parse and build each element
Example:
>>> RepeatUntil(lambda x,ctx: x>7, Byte).build(range(20)) b'\x00\x01\x02\x03\x04\x05\x06\x07\x08' >>> RepeatUntil(lambda x,ctx: x>7, Byte).parse(b"\x01\xff\x02") [1, 255]
-
class
construct.core.
Padded
(length, subcon, pattern=b'x00', strict=False)¶ Appends additional null bytes to achieve a fixed length.
Example:
>>> Padded(4, Byte).build(255) b'\xff\x00\x00\x00' >>> Padded(4, Byte).parse(_) 255 >>> Padded(4, Byte).sizeof() 4 >>> Padded(4, VarInt).build(1) b'\x01\x00\x00\x00' >>> Padded(4, VarInt).build(70000) b'\xf0\xa2\x04\x00'
-
class
construct.core.
Aligned
(modulus, subcon, pattern=b'x00')¶ Appends additional null bytes to achieve a length that is shortest multiple of a modulus.
- Parameters
modulus – the modulus to final length, an int or a context->int function
subcon – the subcon to align
pattern – optional, the padding pattern (default is x00)
Example:
>>> Aligned(4, Int16ub).build(1) b'\x00\x01\x00\x00' >>> Aligned(4, Int16ub).parse(_) 1 >>> Aligned(4, Int16ub).sizeof() 4
-
construct.core.
AlignedStruct
(modulus, *subcons, **kw)¶ Makes a structure where each field is aligned to the same modulus.
See also
Uses
Aligned()
and ~construct.core.Struct.- Parameters
modulus – passed to each member
*subcons – the subcons that make up this structure
pattern – optional, keyword parameter passed to each member
Example:
>>> AlignedStruct(4, "a"/Int8ub, "b"/Int16ub).build(dict(a=1,b=5)) b'\x01\x00\x00\x00\x00\x05\x00\x00' >>> AlignedStruct(4, "a"/Int8ub, "b"/Int16ub).parse(_) Container(a=1)(b=5) >>> AlignedStruct(4, "a"/Int8ub, "b"/Int16ub).sizeof() 8
-
construct.core.
BitStruct
(*subcons)¶ Makes a structure inside a Bitwise.
- Parameters
*subcons – the subcons that make up this structure
Example:
>>> BitStruct("field"/Octet).build(dict(field=5)) b'\x05' >>> BitStruct("field"/Octet).parse(_) Container(field=5) >>> BitStruct("field"/Octet).sizeof() 1 >>> format = BitStruct( ... "a" / Flag, ... "b" / Nibble, ... "c" / BitsInteger(10), ... "d" / Padding(1), ... ) >>> format.parse(b"\xbe\xef") Container(a=True)(b=7)(c=887)(d=None) >>> format.sizeof() 2
-
construct.core.
EmbeddedBitStruct
(*subcons)¶ Makes an embedded BitStruct.
See also
Uses
Bitwise()
andEmbedded()
andStruct()
.- Parameters
*subcons – the subcons that make up this structure
-
class
construct.core.
Union
(*subcons, **kw)¶ Treats the same data as multiple constructs (similar to C union statement) so you can “look” at the data in multiple views.
When parsing, all fields read the same data bytes, but stream remains at same offset by default, unless buildfrom selects a subcon. When building, either the first subcon that can find an entry in given dict is allowed to put into the stream, or the subcon is selected by index or name, or builds nothing.
- Parameters
subcons – subconstructs (order and name sensitive)
buildfrom – optional, how to build, the subcon used for building and calculating size, can be integer index or string name selecting a subcon, None (then tries each subcon in sequence, the default), Pass (builds nothing), a context lambda returning either of previously mentioned
Example:
>>> Union("raw"/Bytes(8), "ints"/Int32ub[2], "shorts"/Int16ub[4], "chars"/Byte[8]).parse(b"12345678") Container(raw=b'12345678')(ints=[825373492, 892745528])(shorts=[12594, 13108, 13622, 14136])(chars=[49, 50, 51, 52, 53, 54, 55, 56]) >>> Union("raw"/Bytes(8), "ints"/Int32ub[2], "shorts"/Int16ub[4], "chars"/Byte[8], buildfrom=3).build(dict(chars=range(8))) b'\x00\x01\x02\x03\x04\x05\x06\x07' >>> Union("raw"/Bytes(8), "ints"/Int32ub[2], "shorts"/Int16ub[4], "chars"/Byte[8], buildfrom="chars").build(dict(chars=range(8))) b'\x00\x01\x02\x03\x04\x05\x06\x07' Note that this syntax works ONLY on python 3.6 and pypy due to unordered keyword arguments: >>> Union(raw=Bytes(8), ints=Int32ub[2], shorts=Int16ub[4], chars=Byte[8], buildfrom=3)
-
class
construct.core.
Select
(*subcons, **kw)¶ Selects the first matching subconstruct. It will literally try each of the subconstructs, until one matches.
- Parameters
subcons – the subcons to try (order sensitive)
includename – indicates whether to include the name of the selected subcon in the return value of parsing, default is false
Example:
>>> Select(Int32ub, CString(encoding="utf8")).build(1) b'\x00\x00\x00\x01' >>> Select(Int32ub, CString(encoding="utf8")).build("Афон") b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00' Note that this syntax works ONLY on python 3.6 and pypy due to unordered keyword arguments: >>> Select(num=Int32ub, text=CString(encoding="utf8"))
-
construct.core.
Optional
(subcon)¶ Makes an optional construct, that tries to parse the subcon. If parsing fails, returns None. If building fails, writes nothing.
Note: sizeof returns subcon size, although no bytes could be consumed or produced. Just something to consider.
- Parameters
subcon – the subcon to optionally parse or build
Example:
>>> Optional(Int64ul).parse(b"1234") >>> Optional(Int64ul).parse(b"12345678") 4050765991979987505 >>> Optional(Int64ul).build(1) b'\x01\x00\x00\x00\x00\x00\x00\x00' >>> Optional(Int64ul).build("1") b''
-
class
construct.core.
Switch
(keyfunc, cases, default=<NoDefault: None>, includekey=False)¶ A conditional branch. Switch will choose the case to follow based on the return value of keyfunc. If no case is matched, and no default value is given, SwitchError will be raised.
See also
The
Pass
singleton.- Parameters
keyfunc – a function that takes the context and returns a key, which will be used to choose the relevant case
cases – a dictionary mapping keys to constructs. the keys can be any values that may be returned by keyfunc
default – a default field to use when the key is not found in the cases. if not supplied, an exception will be raised when the key is not found. Pass can be used for do-nothing
includekey – whether to include the key in the return value of parsing, defualt is False
Example:
>>> Switch(this.n, { 1:Byte, 2:Int32ub }).build(5, dict(n=1)) b'\x05' >>> Switch(this.n, { 1:Byte, 2:Int32ub }).build(5, dict(n=2)) b'\x00\x00\x00\x05'
-
construct.core.
IfThenElse
(predicate, thensubcon, elsesubcon)¶ An if-then-else conditional construct. If the predicate indicates True, thensubcon will be used, otherwise elsesubcon will be used.
- Parameters
predicate – a function taking context and returning a bool
thensubcon – the subcon that will be used if the predicate indicates True
elsesubcon – the subcon that will be used if the predicate indicates False
Example:
>>> IfThenElse(this.x > 0, VarInt, Byte).build(255, dict(x=1)) b'\xff\x01' >>> IfThenElse(this.x > 0, VarInt, Byte).build(255, dict(x=0)) b'\xff'
-
construct.core.
If
(predicate, subcon)¶ An if-then conditional construct. If the predicate indicates True, the subcon will be used for parsing and building, otherwise parsing returns None and building is no-op.
- Parameters
predicate – a function taking context and returning a bool
subcon – the subcon that will be used if the predicate returns True
Example:
>>> If(this.x > 0, Byte).build(255, dict(x=1)) b'\xff' >>> If(this.x > 0, Byte).build(255, dict(x=0)) b''
-
class
construct.core.
Pointer
(offset, subcon)¶ Changes the stream position to a given offset, where the construction should take place, and restores the stream position when finished.
See also
Analog
OnDemandPointer()
field, which also seeks to a given offset.- Parameters
offset – an int or a function that takes context and returns absolute stream position, where the construction would take place, can return negative integer as position from the end backwards
subcon – the subcon to use at the offset
Example:
>>> Pointer(8, Bytes(1)).parse(b"abcdefghijkl") b'i' >>> Pointer(8, Bytes(1)).build(b"x") b'\x00\x00\x00\x00\x00\x00\x00\x00x' >>> Pointer(8, Bytes(1)).sizeof() 0
-
class
construct.core.
Peek
(subcon)¶ Peeks at the stream. Parses without changing the stream position. If the end of the stream is reached when peeking, returns None. Sizeof returns 0 by design because build does not put anything into the stream. Building is no-op.
See also
The
Union()
class.- Parameters
subcon – the subcon to peek at
Example:
>>> Sequence(Peek(Byte), Peek(Int16ub)).parse(b"\x01\x02") [1, 258] >>> Sequence(Peek(Byte), Peek(Int16ub)).sizeof() 0
-
class
construct.core.
Seek
(at, whence=0)¶ Sets a new stream position when parsing or building. Seeks are useful when many other fields follow the jump. Pointer works when there is only one field to look at, but when there is more to be done, Seek may come useful.
See also
Analog
Pointer()
wrapper that has same side effect but also processed a subcon.- Parameters
at – where to jump to, can ne an int or a context lambda
whence – is the offset from beginning (0) or from current position (1) or from ending (2), can be an int or a context lambda, default is 0
Example:
>>> (Seek(5) >> Byte).parse(b"01234x") [5, 120] >>> (Bytes(10) >> Seek(5) >> Byte).build([b"0123456789", None, 255]) b'01234\xff6789'
-
class
construct.core.
Restreamed
(subcon, encoder, encoderunit, decoder, decoderunit, sizecomputer)¶ Transforms bytes between the underlying stream and the subcon.
When the parsing or building is done, the wrapper stream is closed. If read buffer or write buffer is not empty, error is raised.
See also
Both
Bitwise()
andBytewise()
are implemented using Restreamed.Warning
Do not use pointers inside.
- Parameters
subcon – the subcon which will operate on the buffer
encoder – a function that takes a b-string and returns a b-string (used when building)
encoderunit – ratio as int, encoder takes that many bytes at once
decoder – a function that takes a b-string and returns a b-string (used when parsing)
decoderunit – ratio as int, decoder takes that many bytes at once
Example:
Bitwise is implemented as Restreamed(subcon, bits2bytes, 8, bytes2bits, 1, lambda n: n//8) Bytewise is implemented as Restreamed(subcon, bytes2bits, 1, bits2bytes, 8, lambda n: n*8)
-
class
construct.core.
Rebuffered
(subcon, tailcutoff=None)¶ Caches bytes from the underlying stream, so it becomes seekable and tellable. Also makes the stream blocking, in case it came from a socket or a pipe. Optionally, stream can forget bytes that went a certain amount of bytes beyond the current offset, allowing only a limited seeking capability while allowing to process an endless stream.
Warning
Experimental implementation. May not be mature enough.
- Parameters
subcon – the subcon which will operate on the buffered stream
Example:
Rebuffered(RepeatUntil(lambda obj,ctx: ?,Byte), tailcutoff=1024).parse_stream(endless_nonblocking_stream)
-
construct.core.
Padding
(length, pattern=b'\x00', strict=False)¶ A padding field that adds bytes when building, discards bytes when parsing.
- Parameters
length – length of the padding, an int or a function taking context and returning an int
pattern – padding pattern as b-string character, default is b”x00” null character
strict – whether to verify during parsing that the stream contains the pattern, raises an exception if actual padding differs from the pattern, default is False
Example:
>>> (Padding(4) >> Bytes(4)).parse(b"????abcd") [None, b'abcd'] >>> (Padding(4) >> Bytes(4)).build(_) b'\x00\x00\x00\x00abcd' >>> (Padding(4) >> Bytes(4)).sizeof() 8 >>> Padding(4).build(None) b'\x00\x00\x00\x00' >>> Padding(4, strict=True).parse(b"****") construct.core.PaddingError: expected b'\x00\x00\x00\x00', found b'****'
-
class
construct.core.
Const
(subcon, value=None)¶ Constant field enforcing a constant value. It is used for file signatures, to validate that the given pattern exists. When parsed, the value must match.
Note that a variable length subcon may still provide positive verification. Const does not consume a precomputed amount of bytes, but depends on the subcon to read the appropriate amount. Consider for example, a field that eats null bytes and returns following byte. When parsing, both b”x00x00x01” and b”x01” will be parsed and checked OK.
- Parameters
subcon – the subcon used to build value from, or a b-string value itself
value – optional, the expected value
- Raises
ConstError – when parsed data does not match specified value, or building from wrong value
Example:
>>> Const(b"MZ").parse(b"MZ") b'MZ' >>> Const(b"MZ").parse(b"??") construct.core.ConstError: expected b'MZ' but parsed b'??' >>> Const(b"MZ").build(None) b'MZ' >>> Const(b"MZ").sizeof() 2 >>> Const(Int32ul, 16).build(None) b'\x10\x00\x00\x00' >>> Const(Int32ul, 16).parse(_) 16 >>> Const(Int32ul, 16).sizeof() 4
-
class
construct.core.
Computed
(func)¶ A computed value. Underlying byte stream is unaffected. When parsing func(context) provides the value.
- Parameters
func – a function that takes context and returns the computed value
- Example::
>>> st = Struct( ... "width" / Byte, ... "height" / Byte, ... "total" / Computed(this.width * this.height), ... ) >>> st.parse(b"12") Container(width=49)(height=50)(total=2450) >>> st.build(dict(width=4,height=5)) b'\x04\x05'
>>> Computed(lambda ctx: os.urandom(10)).parse(b"") b'[\x86\xcc\xf1b\xd9\x10\x0f?\x1a'
-
class
construct.core.
NamedTuple
(tuplename, tuplefields, subcon)¶ Both arrays, structs and sequences can be mapped to a namedtuple from collections module. To create a named tuple, you need to provide a name and a sequence of fields, either a string with space-separated names or a list of strings. Just like the std library namedtuple does.
Example:
>>> NamedTuple("coord", "x y z", Byte[3]).parse(b"123") coord(x=49, y=50, z=51) >>> NamedTuple("coord", "x y z", Byte >> Byte >> Byte).parse(b"123") coord(x=49, y=50, z=51) >>> NamedTuple("coord", "x y z", Struct("x"/Byte, "y"/Byte, "z"/Byte)).parse(b"123") coord(x=49, y=50, z=51)
-
class
construct.core.
Rebuild
(subcon, func)¶ Parses the field like normal, but computes the value for building from a function. Useful for length and count fields when Prefixed and PrefixedArray cannot be used.
Example:
>>> st = Struct( ... "count" / Rebuild(Byte, len_(this.items)), ... "items" / Byte[this.count], ... ) >>> st.build(dict(items=[1,2,3])) b'\x03\x01\x02\x03'
-
class
construct.core.
Default
(subcon, value)¶ Allows to make a field have a default value, which comes handly when building a Struct from a dict with missing keys.
Example:
>>> Struct("a"/Default(Byte,0), "b"/Default(Byte,0)).build(dict(a=1)) b'\x01\x00'
-
class
construct.core.
RawCopy
(subcon)¶ Returns a dict containing both parsed subcon, the raw bytes that were consumed by it, starting and ending offset in the stream, and the amount of bytes. Builds either from raw bytes or a value used by subcon.
Context does contain a dict with data (if built from raw bytes) or with both (if built from value) during building.
Example:
>>>> RawCopy(Byte).parse(b"\xff") Container(data='\xff')(value=255)(offset1=0L)(offset2=1L)(length=1L) ... >>>> RawCopy(Byte).build(dict(data=b"\xff")) '\xff' >>>> RawCopy(Byte).build(dict(value=255)) '\xff'
-
construct.core.
ByteSwapped
(subcon)¶ Swap the byte order within boundaries of the given subcon.
- Parameters
subcon – the subcon on top of byte swapped bytes
Example:
Int24ul <--> ByteSwapped(Int24ub)
-
construct.core.
BitsSwapped
(subcon)¶ Swap the bit order within each byte within boundaries of the given subcon.
- Parameters
subcon – the subcon on top of byte swapped bytes
Example:
>>>> Bitwise(Bytes(8)).parse(b"\x01") '\x00\x00\x00\x00\x00\x00\x00\x01' >>>> BitsSwapped(Bitwise(Bytes(8))).parse(b"\x01") '\x01\x00\x00\x00\x00\x00\x00\x00'
-
class
construct.core.
Prefixed
(lengthfield, subcon)¶ Parses the length field. Then reads that amount of bytes and parses the subcon using only those bytes. Constructs that consume entire remaining stream are constrained to consuming only the specified amount of bytes. When building, data is prefixed by its length.
See also
The
VarInt
encoding should be preferred overByte
and fixed size fields. VarInt is more compact and does never overflow.Note
If lengthfield is fixed size, Prefixed will seek back to write the length afterwards, which will break on non-seekable streams.
- Parameters
lengthfield – a subcon used for storing the length
subcon – the subcon used for storing the value
Example:
>>> Prefixed(VarInt, GreedyBytes).parse(b"\x05hello????remainins") b'hello' >>>> Prefixed(VarInt, Byte[:]).parse(b"\x03\x01\x02\x03????following") [1, 2, 3]
-
class
construct.core.
Checksum
(checksumfield, hashfunc, bytesfunc)¶ A field that is build or validated by a hash of a given byte range.
- Parameters
checksumfield – a subcon field that reads the checksum, usually Bytes(int)
hashfunc – a function taking bytes and returning whatever checksumfield takes when building
bytesfunc – a function taking context and returning the bytes to be hashed, usually this.rawcopy1.data alike
Example:
import hashlib d = Struct( "fields" / RawCopy(Struct( "a" / Byte, "b" / Byte, )), "checksum" / Checksum(Bytes(64), lambda data: hashlib.sha512(data).digest(), this.fields.data), ) data = d.build(dict(fields=dict(value=dict(a=1,b=2)))) # returned b'\x01\x02\xbd\xd8\x1a\xb23\xbc\xebj\xd23\xcd\x18qP\x93 \xa1\x8d\x035\xa8\x91\xcf\x98s\t\x90\xe8\x92>\x1d\xda\x04\xf35\x8e\x9c~\x1c=\x16\xb1o@\x8c\xfa\xfbj\xf52T\xef0#\xed$6S8\x08\xb6\xca\x993'
-
class
construct.core.
Compressed
(subcon, encoding, level=None)¶ Compresses or decompresses underlying stream when processing the subcon. When parsing, entire stream is consumed. When building, puts compressed bytes without marking the end.
See also
This construct should either be used with
Prefixed()
or on entire stream.- Parameters
subcon – the subcon used for storing the value
encoding – any of the module names like zlib/gzip/bzip2/lzma, otherwise any of codecs module bytes<->bytes encodings
level – optinal, an int between 0..9, lzma discards that
Example:
Compressed(GreedyBytes, "zlib") Prefixed(VarInt, Compressed(GreedyBytes, "zlib")) Struct("inner"/above) Compressed(Struct(...), "zlib")
-
class
construct.core.
LazyStruct
(*subcons, **kw)¶ Equivalent to Struct construct, however fixed size members are parsed on demend, others are parsed immediately. If entire struct is fixed size then entire parse is essentially one seek.
See also
Equivalent to
Struct()
.
-
class
construct.core.
LazyRange
(min, max, subcon)¶ Equivalent to Range construct, but members are parsed on demand. Works only with fixed size subcon.
See also
Equivalent to
Range()
.
-
class
construct.core.
LazySequence
(*subcons, **kw)¶ Equivalent to Sequence construct, however fixed size members are parsed on demand, others are parsed immediately. If entire sequence is fixed size then entire parse is essentially one seek.
See also
Equivalent to
Sequence()
.
-
class
construct.core.
OnDemand
(subcon)¶ Allows for on-demand (lazy) parsing. When parsing, it will return a parameterless function that when called, will return the parsed value. Object is cached after first parsing, so non-deterministic subcons will be affected. Works only with fixed size subcon.
- Parameters
subcon – the subcon to read/write on demand, must be fixed size
Example:
>>> OnDemand(Byte).parse(b"\xff") <function OnDemand._parse.<locals>.<lambda> at 0x7fdc241cfc80> >>> _() 255 >>> OnDemand(Byte).build(16) b'\x10' Can also re-build from the lambda returned at parsing. >>> OnDemand(Byte).parse(b"\xff") <function OnDemand._parse.<locals>.<lambda> at 0x7fcbd9855f28> >>> OnDemand(Byte).build(_) b'\xff'
-
construct.core.
OnDemandPointer
(offset, subcon)¶ An on-demand pointer. Is both lazy and jumps to a position before reading.
See also
Base
OnDemand()
andPointer()
construct.- Parameters
offset – an int or a context function that returns absolute stream position, where the construction would take place, can return negative integer as position from the end backwards
subcon – the subcon that will be parsed or built at the offset stream position
Example:
>>> OnDemandPointer(lambda ctx: 2, Byte).parse(b"\x01\x02\x03garbage") <function OnDemand._parse.<locals>.effectuate at 0x7f6f011ad510> >>> _() 3
-
class
construct.core.
LazyBound
(subconfunc)¶ A lazy-bound construct that binds to the construct only at runtime. Useful for recursive data structures (like linked lists or trees), where a construct needs to refer to itself (while it doesn’t exist yet).
- Parameters
subconfunc – a context function returning a Construct (derived) instance, can also return Pass or itself
Example:
>>> st = Struct( ... "value"/Byte, ... "next"/If(this.value > 0, LazyBound(lambda ctx: st)), ... ) ... >>> st.parse(b"\x05\x09\x00") Container(value=5)(next=Container(value=9)(next=Container(value=0)(next=None))) ... >>> print(st.parse(b"\x05\x09\x00")) Container: value = 5 next = Container: value = 9 next = Container: value = 0 next = None
-
class
construct.core.
Embedded
(subcon)¶ Embeds a struct into the enclosing struct, merging fields. Can also embed sequences into sequences. Name is also inherited.
- Parameters
subcon – the struct to embed
Example:
>>> Struct("a"/Byte, Embedded(Struct("b"/Byte)), "c"/Byte).parse(b"abc") Container(a=97)(b=98)(c=99) >>> Struct("a"/Byte, Embedded(Struct("b"/Byte)), "c"/Byte).build(_) b'abc'
-
class
construct.core.
Renamed
(newname, subcon)¶ Renames an existing construct. This creates a wrapper so underlying subcon retains it’s original name, which in general means just a None. Can be used to give same construct few different names. Used internally by / operator.
Also this wrapper is responsible for building a path (a chain of names) that gets attached to error message when parsing, building, or sizeof fails. A field that is not named does not appear on the path.
- Parameters
newname – the new name
subcon – the subcon to rename
Example:
>>> "name" / Int32ul <Renamed: name> >>> Renamed("name", Int32ul) <Renamed: name>
-
construct.core.
Alias
(newname, oldname)¶ Creates an alias for an existing element in a struct. When parsing, value is available under both keys. Building does nothing. Deprecated meaning there is no real use for it.
See also
Note that
Computed()
is more powerful.- Parameters
newname – the new name
oldname – the name of an existing element, must be on same context level
-
class
construct.core.
Mapping
(subcon, decoding, encoding, decdefault=NotImplemented, encdefault=NotImplemented)¶ Adapter that maps objects to other objects. Translates objects before parsing and before
- Parameters
subcon – the subcon to map
decoding – the decoding (parsing) mapping as a dict
encoding – the encoding (building) mapping as a dict
decdefault – the default return value when object is not found in the mapping, if no object is given an exception is raised, if
Pass
is used, the unmapped object will be passed as-isencdefault – the default return value when object is not found in the mapping, if no object is given an exception is raised, if
Pass
is used, the unmapped object will be passed as-is
Example:
???
-
construct.core.
SymmetricMapping
(subcon, mapping, default=NotImplemented)¶ Defines a symmetrical mapping, same mapping is used on parsing and building.
See also
Based on
Mapping()
.- Parameters
subcon – the subcon to map
encoding – the mapping as a dict
decdefault – the default return value when object is not found in the mapping, if no object is given an exception is raised, if
Pass
is used, the unmapped object will be passed as-is
Example:
???
-
construct.core.
Enum
(subcon, default=NotImplemented, **mapping)¶ A set of named values mapping. Can build both from names and values.
- Parameters
subcon – the subcon to map
**mapping – keyword arguments which serve as the encoding mapping
default – an optional, keyword-only argument that specifies the default value to use when the mapping is undefined. if not given, and exception is raised when the mapping is undefined. use Pass topass the unmapped value as-is
Example:
>>> Enum(Byte,a=1,b=2).parse(b"\x01") 'a' >>> Enum(Byte,a=1,b=2).parse(b"\x08") construct.core.MappingError: no decoding mapping for 8 >>> Enum(Byte,a=1,b=2).build("a") b'\x01' >>> Enum(Byte,a=1,b=2).build(1) b'\x01'
-
class
construct.core.
FlagsEnum
(subcon, **flags)¶ A set of flag values mapping. Each flag is extracted from the number, resulting in a FlagsContainer dict that has each key assigned True or False.
- Parameters
subcon – the subcon to extract
**flags – a dictionary mapping flag-names to their value
Example:
>>> FlagsEnum(Byte,a=1,b=2,c=4,d=8).parse(b"\x03") Container(c=False)(b=True)(a=True)(d=False) >>> FlagsEnum(Byte,a=1,b=2,c=4,d=8).build(_) b'\x03'
-
class
construct.core.
ExprAdapter
(subcon, encoder, decoder)¶ A generic adapter that takes
encoder
anddecoder
as parameters. You can use ExprAdapter instead of writing a full-blown class when only a simple expression is needed.- Parameters
subcon – the subcon to adapt
encoder – a function that takes (obj, context) and returns an encoded version of obj, or None for identity
decoder – a function that takes (obj, context) and returns an decoded version of obj, or None for identity
Example:
Ident = ExprAdapter(Byte, encoder = lambda obj,ctx: obj+1, decoder = lambda obj,ctx: obj-1, )
-
class
construct.core.
ExprSymmetricAdapter
(subcon, encoder)¶
-
class
construct.core.
ExprValidator
(subcon, validator)¶ A generic adapter that takes
validator
as parameter. You can use ExprValidator instead of writing a full-blown class when only a simple expression is needed.- Parameters
subcon – the subcon to adapt
encoder – a function that takes (obj, context) and returns a bool
Example:
OneOf = ExprValidator(Byte, validator = lambda obj,ctx: obj in [1,3,5])
-
construct.core.
Hex
(subcon)¶ Adapter for hex-dumping b-strings. It returns a hex dump when parsing, and un-dumps when building.
Example:
>>> Hex(GreedyBytes).parse(b"abcd") b'61626364' >>> Hex(GreedyBytes).build("01020304") b'\x01\x02\x03\x04'
-
construct.core.
HexDump
(subcon, linesize=16)¶ Adapter for hex-dumping b-strings. It returns a hex dump when parsing, and un-dumps when building.
- Parameters
linesize – default 16 bytes per line
buildraw – by default build takes the same format that parse returns, set to build from a b-string directly
Example:
>>> HexDump(Bytes(10)).parse(b"12345abc;/") '0000 31 32 33 34 35 61 62 63 3b 2f 12345abc;/ \n'
-
class
construct.core.
Slicing
(subcon, count, start, stop, step=1, empty=None)¶ Adapter for slicing a list (getting a slice from that list). Works with Range and Sequence and their lazy equivalents.
- Parameters
subcon – the subcon to slice
count – expected number of elements, needed during building
start – start index (or None for entire list)
stop – stop index (or None for up-to-end)
step – step (or 1 for every element)
empty – value to fill the list with during building
Example:
???
-
class
construct.core.
Indexing
(subcon, count, index, empty=None)¶ Adapter for indexing a list (getting a single item from that list). Works with Range and Sequence and their lazy equivalents.
- Parameters
subcon – the subcon to index
count – expected number of elements, needed during building
index – the index of the list to get
empty – value to fill the list with during building
Example:
???
-
class
construct.core.
FocusedSeq
(parsebuildfrom, *subcons, **kw)¶ Parses and builds a sequence where only one subcon value is returned from parsing or taken into building, other fields are parsed and discarded or built from nothing. This is a replacement for SeqOfOne.
- Parameters
parsebuildfrom – which subcon to use, an int or str, or a context lambda returning an int or str
*subcons – a list of members
**kw – a list of members (works ONLY on python 3.6 and pypy)
Excample:
>>> d = FocusedSeq("num", Const(b"MZ"), "num"/Byte, Terminated) >>> d = FocusedSeq(1, Const(b"MZ"), "num"/Byte, Terminated) >>> d.parse(b"MZ\xff") 255 >>> d.build(255) b'MZ\xff'
-
construct.core.
OneOf
(subcon, valids)¶ Validates that the object is one of the listed values, both during parsing and building.
- Parameters
subcon – a construct to validate
valids – a collection implementing in
Example:
>>> OneOf(Byte, [1,2,3]).parse(b"\x01") 1 >>> OneOf(Byte, [1,2,3]).parse(b"\x08") construct.core.ValidationError: ('invalid object', 8) >>> OneOf(Bytes(1), b"1234567890").parse(b"4") b'4' >>> OneOf(Bytes(1), b"1234567890").parse(b"?") construct.core.ValidationError: ('invalid object', b'?') >>> OneOf(Bytes(2), b"1234567890").parse(b"78") b'78' >>> OneOf(Bytes(2), b"1234567890").parse(b"19") construct.core.ValidationError: ('invalid object', b'19')
-
construct.core.
NoneOf
(subcon, invalids)¶ Validates that the object is none of the listed values, both during parsing and building.
- Parameters
subcon – a construct to validate
invalids – a collection implementing in
See also
Look at
OneOf()
for examples, works the same.
-
construct.core.
Filter
(predicate, subcon)¶ Filters a list leaving only the elements that passed through the validator.
- Parameters
subcon – a construct to validate, usually a Range or Array or Sequence
predicate – a function taking (obj, context) and returning a bool
Example:
>>> Filter(obj_ != 0, Byte[:]).parse(b"\x00\x02\x00") [2] >>> Filter(obj_ != 0, Byte[:]).build([0,1,0,2,0]) b'\x01\x02'
-
class
construct.core.
Check
(func)¶ Checks for a condition and raises ValidationError if the check fails.
Example:
Check(lambda ctx: len(ctx.payload.data) == ctx.payload_len) Check(len_(this.payload.data) == this.payload_len)
-
construct.core.
setglobalstringencoding
(encoding)¶ Sets the encoding globally for all String PascalString CString GreedyString instances.
- Parameters
encoding – a string like “utf8” etc or None, which means working with bytes
-
class
construct.core.
StringEncoded
(subcon, encoding)¶ Used internally.
-
class
construct.core.
StringPaddedTrimmed
(length, subcon, padchar=b'x00', paddir='right', trimdir='right')¶ Used internally.
-
construct.core.
String
(length, encoding=None, padchar=b'\x00', paddir='right', trimdir='right')¶ A configurable, fixed-length or variable-length string field.
When parsing, the byte string is stripped of pad character (as specified) from the direction (as specified) then decoded (as specified). Length is a constant integer or a function of the context. When building, the string is encoded (as specified) then padded (as specified) from the direction (as specified) or trimmed as bytes (as specified).
The padding character and direction must be specified for padding to work. The trim direction must be specified for trimming to work.
- Parameters
length – length in bytes (not unicode characters), as int or context function
encoding – encoding (e.g. “utf8”) or None for bytes
padchar – b-string character to pad out strings (by default b”x00”)
paddir – direction to pad out strings (one of: right left both)
trimdir – direction to trim strings (one of: right left)
Example:
>>> String(10).build(b"hello") b'hello\x00\x00\x00\x00\x00' >>> String(10).parse(_) b'hello' >>> String(10).sizeof() 10 >>> String(10, encoding="utf8").build("Афон") b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00\x00' >>> String(10, encoding="utf8").parse(_) 'Афон' >>> String(10, padchar=b"XYZ", paddir="center").build(b"abc") b'XXXabcXXXX' >>> String(10, padchar=b"XYZ", paddir="center").parse(b"XYZabcXYZY") b'abc' >>> String(10, trimdir="right").build(b"12345678901234567890") b'1234567890'
-
construct.core.
PascalString
(lengthfield, encoding=None)¶ A length-prefixed string.
PascalString
is named after the string types of Pascal, which are length-prefixed. Lisp strings also follow this convention.The length field will not appear in the same dict, when parsing. Only the string will be returned. When building, actual length is prepended before the encoded string. The length field can be variable length (such as VarInt). Stored length is in bytes, not characters.
- Parameters
lengthfield – a field used to parse and build the length
encoding – encoding (e.g. “utf8”) or None for bytes
Example:
>>> PascalString(VarInt, encoding="utf8").build("Афон") b'\x08\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd' >>> PascalString(VarInt, encoding="utf8").parse(_) 'Афон'
-
construct.core.
CString
(terminators=b'\x00', encoding=None)¶ A string ending in a terminator b-string character.
CString
is similar to the strings of C.By default, the terminator is the NULL byte (b’x00’). Terminators field can be a longer b-string, and any of the characters breaks parsing. First terminator byte is used when building.
- Parameters
terminators – sequence of valid terminators, first is used when building, all are used when parsing
encoding – encoding (e.g. “utf8”) or None for bytes
Example:
>>> CString(encoding="utf8").build("Афон") b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd\x00' >>> CString(encoding="utf8").parse(_) 'Афон'
-
construct.core.
GreedyString
(encoding=None)¶ A string that reads the rest of the stream until EOF, and writes a given string as is. If no encoding is given, this is essentially GreedyBytes.
- Parameters
encoding – encoding (e.g. “utf8”) or None for bytes
See also
Analog to
GreedyBytes
and the same when no enoding is used.Example:
>>> GreedyString(encoding="utf8").build("Афон") b'\xd0\x90\xd1\x84\xd0\xbe\xd0\xbd' >>> GreedyString(encoding="utf8").parse(_) 'Афон'