Core API: Tunneling¶
-
construct.
RawCopy
(subcon)¶ Returns a dict containing both parsed subcon, the raw bytes that were consumed by it, starting and ending offset in the stream, and the amount of bytes. Builds either from raw bytes or a value used by subcon.
Context does contain a dict with data (if built from raw bytes) or with both (if built from value) during building.
Example:
>>>> RawCopy(Byte).parse(b"\xff") Container(data='\xff')(value=255)(offset1=0L)(offset2=1L)(length=1L) ... >>>> RawCopy(Byte).build(dict(data=b"\xff")) '\xff' >>>> RawCopy(Byte).build(dict(value=255)) '\xff'
-
construct.
ByteSwapped
(subcon)¶ Swap the byte order within boundaries of the given subcon.
- Parameters
subcon – the subcon on top of byte swapped bytes
Example:
Int24ul <--> ByteSwapped(Int24ub)
-
construct.
BitsSwapped
(subcon)¶ Swap the bit order within each byte within boundaries of the given subcon.
- Parameters
subcon – the subcon on top of byte swapped bytes
Example:
>>>> Bitwise(Bytes(8)).parse(b"\x01") '\x00\x00\x00\x00\x00\x00\x00\x01' >>>> BitsSwapped(Bitwise(Bytes(8))).parse(b"\x01") '\x01\x00\x00\x00\x00\x00\x00\x00'
-
construct.
Prefixed
(lengthfield, subcon)¶ Parses the length field. Then reads that amount of bytes and parses the subcon using only those bytes. Constructs that consume entire remaining stream are constrained to consuming only the specified amount of bytes. When building, data is prefixed by its length.
See also
The
VarInt
encoding should be preferred overByte
and fixed size fields. VarInt is more compact and does never overflow.Note
If lengthfield is fixed size, Prefixed will seek back to write the length afterwards, which will break on non-seekable streams.
- Parameters
lengthfield – a subcon used for storing the length
subcon – the subcon used for storing the value
Example:
>>> Prefixed(VarInt, GreedyBytes).parse(b"\x05hello????remainins") b'hello' >>>> Prefixed(VarInt, Byte[:]).parse(b"\x03\x01\x02\x03????following") [1, 2, 3]
-
construct.
Checksum
(checksumfield, hashfunc, bytesfunc)¶ A field that is build or validated by a hash of a given byte range.
- Parameters
checksumfield – a subcon field that reads the checksum, usually Bytes(int)
hashfunc – a function taking bytes and returning whatever checksumfield takes when building
bytesfunc – a function taking context and returning the bytes to be hashed, usually this.rawcopy1.data alike
Example:
import hashlib d = Struct( "fields" / RawCopy(Struct( "a" / Byte, "b" / Byte, )), "checksum" / Checksum(Bytes(64), lambda data: hashlib.sha512(data).digest(), this.fields.data), ) data = d.build(dict(fields=dict(value=dict(a=1,b=2)))) # returned b'\x01\x02\xbd\xd8\x1a\xb23\xbc\xebj\xd23\xcd\x18qP\x93 \xa1\x8d\x035\xa8\x91\xcf\x98s\t\x90\xe8\x92>\x1d\xda\x04\xf35\x8e\x9c~\x1c=\x16\xb1o@\x8c\xfa\xfbj\xf52T\xef0#\xed$6S8\x08\xb6\xca\x993'
-
construct.
Compressed
(subcon, encoding, level=None)¶ Compresses or decompresses underlying stream when processing the subcon. When parsing, entire stream is consumed. When building, puts compressed bytes without marking the end.
See also
This construct should either be used with
Prefixed()
or on entire stream.- Parameters
subcon – the subcon used for storing the value
encoding – any of the module names like zlib/gzip/bzip2/lzma, otherwise any of codecs module bytes<->bytes encodings
level – optinal, an int between 0..9, lzma discards that
Example:
Compressed(GreedyBytes, "zlib") Prefixed(VarInt, Compressed(GreedyBytes, "zlib")) Struct("inner"/above) Compressed(Struct(...), "zlib")