class Kaitai::Struct::Stream
Kaitai::Struct::Stream
is an implementation of {github.com/kaitai-io/kaitai_struct/wiki/Kaitai-Struct-stream-API Kaitai
Struct
stream API} for Ruby. It's implemented as a wrapper for generic IO objects.
It provides a wide variety of simple methods to read (parse) binary representations of primitive types, such as integer and floating point numbers, byte arrays and strings, and also provides stream positioning / navigation methods with unified cross-language and cross-toolkit semantics.
Typically, end users won't access Kaitai
Stream
class manually, but would describe a binary structure format using .ksy language and then would use Kaitai
Struct
compiler to generate source code in desired target language. That code, in turn, would use this class and API to do the actual parsing job.
Constants
- SIGN_MASK_16
¶ ↑
- SIGN_MASK_32
- SIGN_MASK_64
Public Class Methods
# File lib/kaitai/struct/struct.rb, line 420 def self.bytes_strip_right(bytes, pad_byte) new_len = bytes.length while new_len > 0 and bytes.getbyte(new_len - 1) == pad_byte new_len -= 1 end bytes[0, new_len] end
# File lib/kaitai/struct/struct.rb, line 429 def self.bytes_terminate(bytes, term, include_term) new_len = 0 max_len = bytes.length while bytes.getbyte(new_len) != term and new_len < max_len new_len += 1 end new_len += 1 if include_term and new_len < max_len bytes[0, new_len] end
Constructs new Kaitai
Stream
object. @param arg [String, IO] if String, it will be used as byte array to read data from;
if IO, if will be used literally as source of data
# File lib/kaitai/struct/struct.rb, line 98 def initialize(arg) if arg.is_a?(String) @_io = StringIO.new(arg) elsif arg.is_a?(IO) @_io = arg else raise TypeError.new('can be initialized with IO or String only') end align_to_byte end
Performs a circular left rotation shift for a given buffer by a given amount of bits, using groups of groupSize bytes each time. Right circular rotation should be performed using this procedure with corrected amount. @param data [String] source data to process @param amount [Fixnum] number of bits to shift by @param group_size [Fixnum] number of bytes per group to shift @return [String] copy of source array with requested shift applied
# File lib/kaitai/struct/struct.rb, line 496 def self.process_rotate_left(data, amount, group_size) raise NotImplementedError.new("unable to rotate group #{group_size} bytes yet") unless group_size == 1 mask = group_size * 8 - 1 anti_amount = -amount & mask # NB: actually, left bit shift (<<) in Ruby would have required # truncation to type_bits size (i.e. something like "& 0xff" for # group_size == 8), but we can skip this one, because later these # number would be packed with Array#pack, which will do truncation # anyway data.bytes.map { |x| (x << amount) | (x >> anti_amount) }.pack('C*') end
Performs a XOR processing with given data, XORing every byte of input with a key array, repeating key array many times, if necessary (i.e. if data array is longer than key array). Uses pure Ruby implementation suggested by [Thomas Leitner](github.com/gettalong), borrowed from github.com/fny/xorcist/blob/master/bin/benchmark @param data [String] data to process @param key [String] array of bytes to XOR with @return [String] processed data
# File lib/kaitai/struct/struct.rb, line 472 def self.process_xor_many(data, key) out = data.dup kl = key.length ki = 0 i = 0 max = data.length while i < max out.setbyte(i, data.getbyte(i) ^ key.getbyte(ki)) ki += 1 ki = 0 if ki >= kl i += 1 end out end
Performs a XOR processing with given data, XORing every byte of input with a single given value. Uses pure Ruby implementation suggested by [Thomas Leitner](github.com/gettalong), borrowed from github.com/fny/xorcist/blob/master/bin/benchmark @param data [String] data to process @param key [Fixnum] value to XOR with @return [String] processed data
# File lib/kaitai/struct/struct.rb, line 451 def self.process_xor_one(data, key) out = data.dup i = 0 max = data.length while i < max out.setbyte(i, data.getbyte(i) ^ key) i += 1 end out end
Resolves value using enum: if the value is not found in the map, we'll just use literal value per se.
# File lib/kaitai/struct/struct.rb, line 516 def self.resolve_enum(enum_map, value) enum_map[value] || value end
Private Class Methods
# File lib/kaitai/struct/struct.rb, line 531 def self.format_hex(arr) arr.unpack('H*')[0].gsub(/(..)/, '\1 ').chop end
Public Instance Methods
@!group Unaligned bit values
# File lib/kaitai/struct/struct.rb, line 290 def align_to_byte @bits_left = 0 @bits = 0 end
Closes underlying IO object.
# File lib/kaitai/struct/struct.rb, line 119 def close @_io.close end
Unused since Kaitai
Struct
Compiler v0.9+ - compatibility with older versions.
Reads next len bytes from the stream and ensures that they match expected fixed byte array. If they differ, throws a {UnexpectedDataError} runtime exception. @param expected [String] contents to be expected @return [String] read bytes as byte array, which are guaranteed to
equal to expected
@raise [UnexpectedDataError]
# File lib/kaitai/struct/struct.rb, line 413 def ensure_fixed_contents(expected) len = expected.bytesize actual = @_io.read(len) raise UnexpectedDataError.new(actual, expected) if actual != expected actual end
Check if stream pointer is at the end of stream. @return [true, false] true if we are located at the end of the stream
# File lib/kaitai/struct/struct.rb, line 131 def eof?; @_io.eof? and @bits_left == 0; end
Get current position of a stream pointer. @return [Fixnum] pointer position, number of bytes from the beginning of the stream
# File lib/kaitai/struct/struct.rb, line 141 def pos; @_io.pos; end
# File lib/kaitai/struct/struct.rb, line 295 def read_bits_int_be(n) bits_needed = n - @bits_left if bits_needed > 0 # 1 bit => 1 byte # 8 bits => 1 byte # 9 bits => 2 bytes bytes_needed = ((bits_needed - 1) / 8) + 1 buf = read_bytes(bytes_needed) buf.each_byte { |byte| @bits <<= 8 @bits |= byte @bits_left += 8 } end # raw mask with required number of 1s, starting from lowest bit mask = (1 << n) - 1 # shift @bits to align the highest bits with the mask & derive reading result shift_bits = @bits_left - n res = (@bits >> shift_bits) & mask # clear top bits that we've just read => AND with 1s @bits_left -= n mask = (1 << @bits_left) - 1 @bits &= mask res end
# File lib/kaitai/struct/struct.rb, line 329 def read_bits_int_le(n) bits_needed = n - @bits_left if bits_needed > 0 # 1 bit => 1 byte # 8 bits => 1 byte # 9 bits => 2 bytes bytes_needed = ((bits_needed - 1) / 8) + 1 buf = read_bytes(bytes_needed) buf.each_byte { |byte| @bits |= (byte << @bits_left) @bits_left += 8 } end # raw mask with required number of 1s, starting from lowest bit mask = (1 << n) - 1 # derive reading result res = @bits & mask # remove bottom bits that we've just read by shifting @bits >>= n @bits_left -= n res end
Reads designated number of bytes from the stream. @param n [Fixnum] number of bytes to read @return [String] read bytes as byte array @raise [EOFError] if there were less bytes than requested
available in the stream
# File lib/kaitai/struct/struct.rb, line 364 def read_bytes(n) r = @_io.read(n) if r rl = r.bytesize else rl = 0 end raise EOFError.new("attempted to read #{n} bytes, got only #{rl}") if rl < n r end
Reads all the remaining bytes in a stream as byte array. @return [String] all remaining bytes in a stream as byte array
# File lib/kaitai/struct/struct.rb, line 378 def read_bytes_full @_io.read end
# File lib/kaitai/struct/struct.rb, line 382 def read_bytes_term(term, include_term, consume_term, eos_error) r = '' loop { if @_io.eof? if eos_error raise EOFError.new("end of stream reached, but no terminator #{term} found") else return r end end c = @_io.getc if c.ord == term r << c if include_term @_io.seek(@_io.pos - 1) unless consume_term return r end r << c } end
Big-endian
# File lib/kaitai/struct/struct.rb, line 266 def read_f4be read_bytes(4).unpack('g')[0] end
Little-endian
# File lib/kaitai/struct/struct.rb, line 278 def read_f4le read_bytes(4).unpack('e')[0] end
# File lib/kaitai/struct/struct.rb, line 270 def read_f8be read_bytes(8).unpack('G')[0] end
# File lib/kaitai/struct/struct.rb, line 282 def read_f8le read_bytes(8).unpack('E')[0] end
Signed
# File lib/kaitai/struct/struct.rb, line 156 def read_s1 read_bytes(1).unpack('c')[0] end
.….….….….….….….….….….….….….….….….….… Big-endian .….….….….….….….….….….….….….….….….….…
# File lib/kaitai/struct/struct.rb, line 164 def read_s2be to_signed(read_u2be, SIGN_MASK_16) end
.….….….….….….….….….….….….….….….….….… Little-endian .….….….….….….….….….….….….….….….….….…
# File lib/kaitai/struct/struct.rb, line 186 def read_s2le to_signed(read_u2le, SIGN_MASK_16) end
# File lib/kaitai/struct/struct.rb, line 168 def read_s4be to_signed(read_u4be, SIGN_MASK_32) end
# File lib/kaitai/struct/struct.rb, line 190 def read_s4le to_signed(read_u4le, SIGN_MASK_32) end
# File lib/kaitai/struct/struct.rb, line 173 def read_s8be read_bytes(8).unpack('q')[0] end
# File lib/kaitai/struct/struct.rb, line 195 def read_s8le read_bytes(8).unpack('q')[0] end
Unsigned
# File lib/kaitai/struct/struct.rb, line 208 def read_u1 read_bytes(1).unpack('C')[0] end
.….….….….….….….….….….….….….….….….….… Big-endian .….….….….….….….….….….….….….….….….….…
# File lib/kaitai/struct/struct.rb, line 216 def read_u2be read_bytes(2).unpack('n')[0] end
.….….….….….….….….….….….….….….….….….… Little-endian .….….….….….….….….….….….….….….….….….…
# File lib/kaitai/struct/struct.rb, line 239 def read_u2le read_bytes(2).unpack('v')[0] end
# File lib/kaitai/struct/struct.rb, line 220 def read_u4be read_bytes(4).unpack('N')[0] end
# File lib/kaitai/struct/struct.rb, line 243 def read_u4le read_bytes(4).unpack('V')[0] end
# File lib/kaitai/struct/struct.rb, line 225 def read_u8be read_bytes(8).unpack('Q')[0] end
# File lib/kaitai/struct/struct.rb, line 248 def read_u8le read_bytes(8).unpack('Q')[0] end
Set stream pointer to designated position. @param x [Fixnum] new position (offset in bytes from the beginning of the stream)
# File lib/kaitai/struct/struct.rb, line 136 def seek(x); @_io.seek(x); end
Get total size of the stream in bytes. @return [Fixnum] size of the stream in bytes
# File lib/kaitai/struct/struct.rb, line 146 def size; @_io.size; end
Private Instance Methods
# File lib/kaitai/struct/struct.rb, line 527 def to_signed(x, mask) (x & ~mask) - (x & mask) end