class EventMachine::Protocols::Postgres3

PROVISIONAL IMPLEMENTATION of an evented Postgres client. This implements version 3 of the Postgres wire protocol, which will work with any Postgres version from roughly 7.4 onward.

Objective: we want to access Postgres databases without requiring threads. Until now this has been a problem because the Postgres client implementations have all made use of blocking I/O calls, which is incompatible with a thread-free evented model.

But rather than re-implement the Postgres Wire3 protocol, we’re taking advantage of the existing postgres-pr library, which was originally written by Michael Neumann but (at this writing) appears to be no longer maintained. Still, it’s in basically a production-ready state, and the wire protocol isn’t that complicated anyway.

We’re tucking in a bunch of require statements that may not be present in garden-variety EM installations. Until we find a good way to only require these if a program requires postgres, this file will need to be required explicitly.

We need to monkeypatch StringIO because it lacks the readbytes method needed by postgres-pr. The StringIO monkeypatch is lifted from the standard library readbytes.rb, which adds method readbytes directly to class IO. But StringIO is not a subclass of IO. It is modified to raise an IOError instead of TruncatedDataException since the exception is unused.

We cloned the handling of postgres messages from lib/postgres-pr/connection.rb in the postgres-pr library, and modified it for event-handling.

TODO: The password handling in dispatch_conn_message is totally incomplete.

We return Deferrables from the user-level operations surfaced by this interface. Experimentally, we’re using the pattern of always returning a boolean value as the first argument of a deferrable callback to indicate success or failure. This is instead of the traditional pattern of calling Deferrable#succeed or fail, and requiring the user to define both a callback and an errback function.

Usage

EM.run {
  db = EM.connect_unix_domain( "/tmp/.s.PGSQL.5432", EM::P::Postgres3 )
  db.connect( dbname, username, psw ).callback do |status|
    if status
      db.query( "select * from some_table" ).callback do |status, result, errors|
        if status
          result.rows.each do |row|
            p row
          end
        end
      end
    end
  end
}

Public Class Methods

new() click to toggle source
# File lib/em/protocols/postgres3.rb, line 109
def initialize
  @data = ""
  @params = {}
end

Public Instance Methods

connect(db, user, psw=nil) click to toggle source
# File lib/em/protocols/postgres3.rb, line 114
def connect db, user, psw=nil
  d = EM::DefaultDeferrable.new
  d.timeout 15

  if @pending_query || @pending_conn
    d.succeed false, "Operation already in progress"
  else
    @pending_conn = d
    prms = {"user"=>user, "database"=>db}
    @user = user
    if psw
      @password = psw
      #prms["password"] = psw
    end
    send_data PostgresPR::StartupMessage.new( 3 << 16, prms ).dump
  end

  d
end
dispatch_conn_message(msg) click to toggle source

Cloned and modified from the postgres-pr.

# File lib/em/protocols/postgres3.rb, line 179
def dispatch_conn_message msg
  case msg
  when AuthentificationClearTextPassword
    raise ArgumentError, "no password specified" if @password.nil?
    send_data PasswordMessage.new(@password).dump

  when AuthentificationCryptPassword
    raise ArgumentError, "no password specified" if @password.nil?
    send_data PasswordMessage.new(@password.crypt(msg.salt)).dump

  when AuthentificationMD5Password
    raise ArgumentError, "no password specified" if @password.nil?
    require 'digest/md5'

    m = Digest::MD5.hexdigest(@password + @user)
    m = Digest::MD5.hexdigest(m + msg.salt)
    m = 'md5' + m
    send_data PasswordMessage.new(m).dump

  when AuthentificationKerberosV4, AuthentificationKerberosV5, AuthentificationSCMCredential
    raise "unsupported authentification"

  when AuthentificationOk
  when ErrorResponse
    raise msg.field_values.join("\t")
  when NoticeResponse
    @notice_processor.call(msg) if @notice_processor
  when ParameterStatus
    @params[msg.key] = msg.value
  when BackendKeyData
    # TODO
    #p msg
  when ReadyForQuery
    # TODO: use transaction status
    pc,@pending_conn = @pending_conn,nil
    pc.succeed true
  else
    raise "unhandled message type"
  end
end
dispatch_query_message(msg) click to toggle source

Cloned and modified from the postgres-pr.

# File lib/em/protocols/postgres3.rb, line 221
def dispatch_query_message msg
  case msg
  when DataRow
    @r.rows << msg.columns
  when CommandComplete
    @r.cmd_tag = msg.cmd_tag
  when ReadyForQuery
    pq,@pending_query = @pending_query,nil
    pq.succeed true, @r, @e
  when RowDescription
    @r.fields = msg.fields
  when CopyInResponse
  when CopyOutResponse
  when EmptyQueryResponse
  when ErrorResponse
    # TODO
    @e << msg
  when NoticeResponse
    @notice_processor.call(msg) if @notice_processor
  else
    # TODO
  end
end
query(sql) click to toggle source
# File lib/em/protocols/postgres3.rb, line 134
def query sql
  d = EM::DefaultDeferrable.new
  d.timeout 15

  if @pending_query || @pending_conn
    d.succeed false, "Operation already in progress"
  else
    @r = PostgresPR::Connection::Result.new
    @e = []
    @pending_query = d
    send_data PostgresPR::Query.dump(sql)
  end

  d
end
receive_data(data) click to toggle source
# File lib/em/protocols/postgres3.rb, line 151
def receive_data data
  @data << data
  while @data.length >= 5
    pktlen = @data[1...5].unpack("N").first
    if @data.length >= (1 + pktlen)
      pkt = @data.slice!(0...(1+pktlen))
      m = StringIO.open( pkt, "r" ) {|io| PostgresPR::Message.read( io ) }
      if @pending_conn
        dispatch_conn_message m
      elsif @pending_query
        dispatch_query_message m
      else
        raise "Unexpected message from database"
      end
    else
      break # very important, break out of the while
    end
  end
end
unbind() click to toggle source
# File lib/em/protocols/postgres3.rb, line 172
def unbind
  if o = (@pending_query || @pending_conn)
    o.succeed false, "lost connection"
  end
end