API¶
This part of the documentation covers all the interfaces of AhoyHoy.
Session proxy¶
-
class
ahoyhoy.http.proxy.
SessionProxy
(session=None, pre_callback=None, post_callback=None, exception_callback=None)[source]¶ Proxy adapter so a subclass can proxy methods of a Requests Session, but can alter behavior via pre, post, and exception callbacks.
Can be used by itself, but is intended to be subclassed with overwritten callbacks.
>>> from ahoyhoy.http import SessionProxy >>> sp = SessionProxy() >>> sp.get('http://google.com') <Response [200]>
In order to be a bit more transparent, excpetion_callback will raise the thrown exception.
>>> from ahoyhoy.http import SessionProxy >>> import requests >>> sp = SessionProxy(requests.Session()) >>> try: ... sp.get('http://wwwwww.google.com') ... except requests.exceptions.ConnectionError as e: ... print("Error raised") ... Error raised
You may override the pre|post|exception callbacks either by subclassing them or by runtime configuration.
>>> from ahoyhoy.http import SessionProxy >>> import requests >>> >>> def pre_callback(url): ... print('pre') ... return url >>> >>> def post_callback(res): ... print('post') >>> >>> def exception_callback(e): ... print('Exception!!') >>> >>> sp = SessionProxy(requests.Session(), pre_callback=pre_callback, post_callback=post_callback, exception_callback=exception_callback) >>> sp.get('http://google.com') pre post >>> sp.get('http://google1.com') pre Exception!! post
Test proxy for other methods and attributes:
>>> sp.headers {...'User-Agent': ...}
Parameters: - session – custom session
- pre_callback – executed before the proxied method
- post_callback – executed after the proxied method
- exception_callback – the proxied method is wrapped in a try/except block, and when an exception occurs, the exception is passed to this callback
-
exception_callback
(exc)[source]¶ Executed when an exception is thrown by the proxied Requests Session method.
Intended to be overwritten or set by a derived class.
Parameters: exc – the exception raised by the proxied Requests Session method. By default, returns what’s passed in.
-
post_callback
(result)[source]¶ Executed after the proxied Requests method.
Intended to be overwritten or set by a derived class.
Parameters: result – the return value of the proxied Requests method. By default, returns what’s passed in.
-
pre_callback
(urlpath)[source]¶ Executed before the proxied Requests method.
Intended to be overwritten or set by a derived class.
Parameters: urlpath – the arg[0] of the called proxied method. Requests Session methods all take ‘url’ as the first positional parameter. By default, returns what’s passed in.
Client¶
Client is a wrapper for HTTP calls which allows to use load balancers, circuit breakers and retries.
-
class
ahoyhoy.client.
Client
(lb, ep_list_update_tries=1, **ep_kwargs)[source]¶ A load balancing, circuit breaking client.
Accepts load balancer instance as an argument. Client can be a duck-typed requests object.
Usage examples:
- Client with RoundRobin LB and bad hosts
>>> from ahoyhoy.utils import Host >>> from ahoyhoy.lb.providers import ListProvider >>> from ahoyhoy.lb import RoundRobinLB
>>> bad_host1 = Host('google1.com1', '80') >>> bad_host2 = Host('google2.com2', '80') >>> good_host = Host('google.com', '80') >>> rrlb = RoundRobinLB(ListProvider(bad_host1, bad_host2, good_host)) >>> client = Client(rrlb) >>> client.get('/') <Response [200]>
Note! Client only accepts HTTP calls for now.
Session’s attributes and methods (except HTTP calls) are unavailable for calls through the Client’s instance. Because of the dynamic nature of
Endpoint
s, all parameters (such as headers etc.) have to be changed through the client-specific API.Consider these examples:
>>> client.headers.update({'bla': 'foo'}) Traceback (most recent call last): ... AttributeError: No such attribute: 'headers'. Only HTTP methods can be called for now.
>>> c = Client(rrlb, headers={'bla': 'foo'}) >>> response = c.get('/') >>> assert 'bla' in response.request.headers.keys()
Parameters: - lb – instance of
ILoadBalancer
- ep_list_update_tries – how many times to try to update endpoints list in LB
- ep_kwargs – arguments to pass to an Endpoint (‘retry_http_call_func’, ‘headers’)
Client Builders¶
ClientBuilder provides convenient api for creating the specific Client
instance.
Abstract class:¶
-
class
ahoyhoy.client.builder.
IClientBuilder
[source]¶ Abstract Builder class. Contains required methods for all other client builders.
-
add_retries
(retries)[source]¶ Custom retries function. It has to accept function to retry w/ its args and kwargs. It’s highly recommended to use
ahoyhoy.retries.Retry()
function with custom parameters.
-
-
class
ahoyhoy.client.
LBClientBuilder
[source]¶ Bases:
ahoyhoy.client.builder.IClientBuilder
Usage examples:
Round Robin LB
>>> from ahoyhoy.utils import Host >>> from ahoyhoy.lb.providers import ListProvider >>> from ahoyhoy.lb import RoundRobinLB >>> >>> rrlb = RoundRobinLB(ListProvider(Host('badhost2.bla', 80), Host('google.com', 80), Host('badhost3.bla', 80))) >>> >>> client = LBClientBuilder().add_lb(rrlb).build() >>> client.get('/') <Response [200]>
Round Robin LB and custom session
>>> import requests >>> s = requests.Session() >>> s.headers.update({'bla': 'foo'}) >>> client = LBClientBuilder().add_lb(rrlb).add_session(s).build() >>> client.get('/') <Response [200]>
Round Robin LB with custom HTTP retries
>>> from ahoyhoy.retries import Retry >>> from requests.exceptions import ConnectTimeout >>> >>> retries = Retry(exceptions=ConnectTimeout, tries=3, delay=0, max_delay=None, backoff=2, jitter=0) >>> rrlb = RoundRobinLB(ListProvider(Host('google.com', 80))) >>> >>> client = LBClientBuilder().add_lb(rrlb).add_retries(retries).build() >>> client.get('/') <Response [200]>
Set multiple updates for endpoints list
>>> client = LBClientBuilder().add_lb(rrlb).set_endpoint_updates(3)
-
class
ahoyhoy.client.
SessionClientBuilder
[source]¶ Bases:
ahoyhoy.client.builder.IClientBuilder
Usage example:
>>> import requests >>> from ahoyhoy.client.builder import SessionClientBuilder >>> >>> s = requests.Session() >>> >>> client = SessionClientBuilder().add_session(s).build() >>> client.get('http://google.com') <Response [200]>
Custom headers:
>>> s.headers.update({'bla': 'foo'}) >>> client = SessionClientBuilder().add_session(s).build() >>> response = client.get('http://google.com') >>> assert 'bla' in response.request.headers
or
>>> client = SessionClientBuilder().add_session(s).add_headers({'foo': 'bar'}).build() >>> response = client.get('http://google.com') >>> assert 'foo' in response.request.headers
Custom retries:
>>> from ahoyhoy.retries import Retry >>> from requests.exceptions import ConnectTimeout >>> >>> retries = Retry(exceptions=ConnectTimeout, tries=3, delay=0, max_delay=None, backoff=2, jitter=0) >>> >>> client = SessionClientBuilder().add_session(s).add_retries(retries).build() >>> client.get('http://google.com') <Response [200]>
Load Balancers¶
There are two load balancing algorithms available for now:
- random
- round robin
Base class¶
-
class
ahoyhoy.lb.iloadbalancer.
ILoadBalancer
(provider, session=None)[source]¶ Base class for load balancers.
Parameters: - provider – any instance of
IProvider
- session – custom session
- provider – any instance of
Load blancers algorithms¶
-
class
ahoyhoy.lb.
RandomLB
(provider, session=None, random_function=<bound method Random.randint of <random.Random object at 0x254fa28>>)[source]¶ Bases:
ahoyhoy.lb.iloadbalancer.ILoadBalancer
Implements random algorithm for chosing a host from the list.
>>> from ahoyhoy.utils import Host >>> from ahoyhoy.lb.providers import ListProvider >>> from ahoyhoy.lb import RandomLB >>> rrlb = RandomLB(ListProvider(Host('google1.com1', '80'), Host('google.com', '80'))) >>> rrlb.pick() <Endpoint/.../Host(address='google...', port='80')/<class 'ahoyhoy.circuit.circuit.ClosedState'>
Custom random function:
>>> def my_random(*args): ... return 0 >>> rrlb = RandomLB( ... ListProvider(Host('google1.com1', '80'), Host('google.com', '80')), ... random_function=my_random) >>> rrlb.pick() <Endpoint/.../Host(address='google1.com1', port='80')/<class 'ahoyhoy.circuit.circuit.ClosedState'>
Parameters: - provider –
ListProvider
instance - random_function – function which returns random number for the given range. By default:
random.randint
- provider –
-
class
ahoyhoy.lb.
RoundRobinLB
(provider, session=None)[source]¶ Bases:
ahoyhoy.lb.iloadbalancer.ILoadBalancer
Implement round robin load balancing algorythm.
>>> from ahoyhoy.utils import Host >>> from ahoyhoy.lb.providers import ListProvider >>> from ahoyhoy.lb import RoundRobinLB >>> rrlb = RoundRobinLB(ListProvider(Host('google1.com1', '80'), Host('google.com', '80'))) >>> rrlb.pick() <Endpoint/.../Host(address='google1.com1', port='80')/<class 'ahoyhoy.circuit.circuit.ClosedState'> >>> rrlb.pick() <Endpoint/.../Host(address='google.com', port='80')/<class 'ahoyhoy.circuit.circuit.ClosedState'> >>> rrlb.pick() <Endpoint/.../Host(address='google1.com1', port='80')/<class 'ahoyhoy.circuit.circuit.ClosedState'>
Parameters: provider – IProvider
instance
Providers¶
Providers exist to give lists of hosts
Base class¶
List Provider¶
-
class
ahoyhoy.lb.providers.
ListProvider
(*args)[source]¶ Bases:
ahoyhoy.lb.providers.iprovider.IProvider
A simple list verison of the IProvider interface
Accepts a number of items to store as a list
>>> from ahoyhoy.utils import Host >>> from ahoyhoy.lb.providers import ListProvider >>> lp = ListProvider(Host('google1.com1', '80'), Host('google.com', '80')) >>> lp.get_list() (Host(address='google1.com1', port='80'), Host(address='google.com', port='80'))
Parameters: args – an iterable of items (hosts)
Retries¶
-
ahoyhoy.retries.
Retry
(retry_func=<function retry_call>, **kwargs)[source]¶ Usage example:
>>> import requests >>> s = requests.Session() >>> retry = Retry(exceptions=Exception, tries=-1, delay=0, max_delay=None, backoff=1, jitter=0) >>> response = retry(s.get, fargs=('http://google.com', ), fkwargs={'headers': {'bla': 'foo'}})
Parameters: - callable (retry_func) – function which accepts input function and its parameters
- kwargs – retry_func kwargs
Circuit breaker¶
-
class
ahoyhoy.circuit.
Circuit
[source]¶ Simple base class to handle the transition between closed and open circuit states.
-
class
ahoyhoy.circuit.
OpenState
[source]¶ Bases:
ahoyhoy.circuit.circuit.State
Circut in an open state. Opening as well as all other methods will throw RuntimeError.
-
class
ahoyhoy.circuit.
ClosedState
[source]¶ Bases:
ahoyhoy.circuit.circuit.State
Circuit in a closed state. Closing will throw RuntimeError.
Service discovery¶
-
class
ahoyhoy.servicediscovery.
ServiceDiscoveryHttpClient
(*args, **kwargs)[source]¶ Bases:
ahoyhoy.servicediscovery.servicediscovery.ServiceDiscoveryAdapter
,ahoyhoy.http.proxy.SessionProxy
An object that transparently allows Service Discovery while preserving the same API as a Requests Session object, so typical Requests methods take server-relative paths rather than full URLs.
For instance, here’s how you can make a Http GET request using Requests’s standard get method.
>>> from ahoyhoy.servicediscovery import ServiceDiscoveryHttpClient >>> from ahoyhoy.utils import Host >>> host = Host('google.com', '80') >>> sdhc = ServiceDiscoveryHttpClient(host) >>> sdhc.get('/') <Response [200]>
Note the fact that we’re passing get a path, _not_ a URL.
Why?
When using a form of Service Discovery, host/port (and sometimes protocol) portions of the URL are resolved at runtime. This class adapts the a Requests Session in order to support this runtime service resolution.
-
class
ahoyhoy.servicediscovery.servicediscovery.
ServiceDiscoveryAdapter
(host, *args, **kwargs)[source]¶ Adapter intended to be used as a mixin along with a
SessionProxy
in order to calculate a protocol://host:port string to be added to the path for a complete url.-
address
¶ Returns the endpoint’s address
-
host
¶ Returns the endpoint’s host
-
port
¶ Returns the endpoint’s port
-
pre_callback
(urlpath)[source]¶ Calculate and return the url to be passed to the Requests Session, using self.url (calculated from self.[host|port|protocol]).
-
url
¶ Returns the ‘protocol://host:port’ url
-
Endpoint¶
Endpoint is a derivative class from the Circuit
. It delegates all HTTP calls ot the session (which is one of the Endpoint’s attributes), but can also keep a state. If Endpoint was called and request failed, it’ll change its state from closed to open.
Another feature of Endpoint is callbacks. It accepts pre-, post- and exception- callbacks, which allows to do corresponding actions with request, response and exceptions handling.
-
class
ahoyhoy.endpoints.
Endpoint
(host=None, pre_callback=None, post_callback=None, exception_callback=None, classify=None, retry=None, session=None, *args, **kwargs)[source]¶ Bases:
ahoyhoy.circuit.circuit.Circuit
Accepts a duck-typed session (a “Session” in Requests terms) and allows it to work as a Circuit (open|closed state).
Endpoint simply proxies to session methods, so it’s just as easy to use as ServiceDiscoveryHttpClient.
>>> from ahoyhoy.endpoints import Endpoint >>> from ahoyhoy.utils import Host >>> host = Host('google.com', '443') >>> ep = Endpoint(host) >>> ep.get('/') <Response [200]> >>> ep.open() >>> ep.get('/') Traceback (most recent call last): ... RuntimeError: Circuit state is open, no connections possible.
When using service discovery, this fits nicely with the way that Load Balancers work.
>>> from ahoyhoy.lb import RoundRobinLB >>> from ahoyhoy.lb.providers import ListProvider >>> from ahoyhoy.utils import Host >>> lb = RoundRobinLB(ListProvider(Host('google.com', '80'))) >>> ep = lb.pick() >>> ep.get('/') <Response [200]> >>> ep.open() >>> ep.get('/') Traceback (most recent call last): ... RuntimeError: Circuit state is open, no connections possible.
Here’s an example of how circuit opens automatically:
>>> from ahoyhoy.lb import RoundRobinLB >>> from ahoyhoy.lb.providers import ListProvider >>> from ahoyhoy.utils import Host >>> lb = RoundRobinLB(ListProvider(Host('google1.com1', '80'))) >>> ep = lb.pick() >>> ep <Endpoint/.../Host(address='google1.com1', port='80')/<class 'ahoyhoy.circuit.circuit.ClosedState'> >>> ep.get('/') Traceback (most recent call last): ... requests.exceptions.ConnectionError: HTTPConnectionPool(host='google1.com1', port=80): Max retries exceeded with url:... >>> ep <Endpoint/.../Host(address='google1.com1', port='80')/<class 'ahoyhoy.circuit.circuit.OpenState'>
Before you gasp at the number of lines there, remember that Endpoint is a relatively low-level component. Higher-level components are easier to use, but Endpoints allow full flexibility.
The SimpleHttpEndpoint factory function can be used when you don’t need service discovery.
>>> from ahoyhoy.endpoints import SimpleHttpEndpoint >>> sep = SimpleHttpEndpoint() >>> sep.get('http://google.com') <Response [200]>
Custom exception callback function
>>> def exc(e): ... return 'I caught it!' >>> ep = Endpoint(Host('google1.com1', '80'), exception_callback=exc) >>> ep.get('/') 'I caught it!'
Parameters: - host – collections.namedtuple, Host(address, port)
- pre_callback –
- post_callback –
- exception_callback –
- classify – response clissifier. By default it’s
Circuit's classify
. - retry – function for retrying HTTP calls
- session – custom session
- args – positional argument for ServiceDiscoveryHttpClient
- kwargs – keyword argument for ServiceDiscoveryHttpClient
-
host
¶
-
state
¶