Core API: Streaming¶
-
construct.
Pointer
(offset, subcon)¶ Changes the stream position to a given offset, where the construction should take place, and restores the stream position when finished.
See also
Analog
OnDemandPointer()
field, which also seeks to a given offset.- Parameters
offset – an int or a function that takes context and returns absolute stream position, where the construction would take place, can return negative integer as position from the end backwards
subcon – the subcon to use at the offset
Example:
>>> Pointer(8, Bytes(1)).parse(b"abcdefghijkl") b'i' >>> Pointer(8, Bytes(1)).build(b"x") b'\x00\x00\x00\x00\x00\x00\x00\x00x' >>> Pointer(8, Bytes(1)).sizeof() 0
-
construct.
Peek
(subcon)¶ Peeks at the stream. Parses without changing the stream position. If the end of the stream is reached when peeking, returns None. Sizeof returns 0 by design because build does not put anything into the stream. Building is no-op.
See also
The
Union()
class.- Parameters
subcon – the subcon to peek at
Example:
>>> Sequence(Peek(Byte), Peek(Int16ub)).parse(b"\x01\x02") [1, 258] >>> Sequence(Peek(Byte), Peek(Int16ub)).sizeof() 0
-
construct.
Tell
()¶ Gets the stream position when parsing or building.
Tells are useful for adjusting relative offsets to absolute positions, or to measure sizes of Constructs. To get an absolute pointer, use a Tell plus a relative offset. To get a size, place two Tells and measure their difference using a Compute.
See also
Better to use
CopyRaw()
wrapper in almost any case.Example:
>>> Struct("num"/VarInt, "offset"/Tell).build(dict(num=88)) b'X' >>> Struct("num"/VarInt, "offset"/Tell).parse(_) Container(num=88)(offset=1)
-
construct.
Seek
(at, whence=0)¶ Sets a new stream position when parsing or building. Seeks are useful when many other fields follow the jump. Pointer works when there is only one field to look at, but when there is more to be done, Seek may come useful.
See also
Analog
Pointer()
wrapper that has same side effect but also processed a subcon.- Parameters
at – where to jump to, can ne an int or a context lambda
whence – is the offset from beginning (0) or from current position (1) or from ending (2), can be an int or a context lambda, default is 0
Example:
>>> (Seek(5) >> Byte).parse(b"01234x") [5, 120] >>> (Bytes(10) >> Seek(5) >> Byte).build([b"0123456789", None, 255]) b'01234\xff6789'
-
construct.
Restreamed
(subcon, encoder, encoderunit, decoder, decoderunit, sizecomputer)¶ Transforms bytes between the underlying stream and the subcon.
When the parsing or building is done, the wrapper stream is closed. If read buffer or write buffer is not empty, error is raised.
See also
Both
Bitwise()
andBytewise()
are implemented using Restreamed.Warning
Do not use pointers inside.
- Parameters
subcon – the subcon which will operate on the buffer
encoder – a function that takes a b-string and returns a b-string (used when building)
encoderunit – ratio as int, encoder takes that many bytes at once
decoder – a function that takes a b-string and returns a b-string (used when parsing)
decoderunit – ratio as int, decoder takes that many bytes at once
Example:
Bitwise is implemented as Restreamed(subcon, bits2bytes, 8, bytes2bits, 1, lambda n: n//8) Bytewise is implemented as Restreamed(subcon, bytes2bits, 1, bits2bytes, 8, lambda n: n*8)
-
construct.
Rebuffered
(subcon, tailcutoff=None)¶ Caches bytes from the underlying stream, so it becomes seekable and tellable. Also makes the stream blocking, in case it came from a socket or a pipe. Optionally, stream can forget bytes that went a certain amount of bytes beyond the current offset, allowing only a limited seeking capability while allowing to process an endless stream.
Warning
Experimental implementation. May not be mature enough.
- Parameters
subcon – the subcon which will operate on the buffered stream
Example:
Rebuffered(RepeatUntil(lambda obj,ctx: ?,Byte), tailcutoff=1024).parse_stream(endless_nonblocking_stream)