ZeroMQ notes

Write down any known technical requirements in terms of throughput, latency, reliability, and so on.

OS can handle 4-5K threads. The general rule of thumb is to allow one I/O thread per gigabyte of data in or out per second.

The client waits for the server to reply with state, and meanwhile queues all updates. It does this simply by not reading them: ZeroMQ keeps them queued on the socket queue.

When someone asks for “reliability”, ask them to list the failures they want to handle

“credit-based flow control” effectively removes the need for high-water marks, and any risk of memory overflow

SOCKETS:
REQ
– adds empty frame in front of a new message
– REQ socket in ZeroMQ cannot be really used in real-world environments, as they get stuck if message is lost due to service failure or similar. Users have to use XREQ instead and implement the request re-trying themselves. With nanomsg, the re-try functionality is built into REQ socket
REP
DEALER
– PUSH and DEALER rotate messages to “one of many”
– PUSH and DEALER block if there is no peer ready to receive a message
– keeps messages in internal queue
ROUTER
– needs the address of the peer before it can send it a message
– keeps conterparty’s info
– discards messages for unknown peers
– ZMQ_HWM option action: Drop
DEALER and ROUTER sockets are fully asynchronous
SUB
– cannot talk back to PUB
PUB
– sends each message to “all of many”
XSUB
XPUB
– does not report duplicate subscriptions
PULL
PUSH
– PUSH and DEALER rotate messages to “one of many”
– PUSH and DEALER block if there is no peer ready to receive a message
– keeps messages in internal queue
PAIR
– does not reconnect if the peer disappears and comes back

PUB-SUB
– The ZeroMQ pub-sub pattern will lose messages arbitrarily when a subscriber is connecting, when a network failure occurs, or just if the subscriber or network can’t keep up with the publisher.

ZMQ patterns:

PUB-SUB:
Espresso – proxy between subs and pubs; may cache last value
Suicidal Snail – subscriber kills itself if recv message timestamp differnece with the current timestamp is greater than some reasonable value
Black Box – high speed subscriber that distributes messages to workers (multithreaded implementation)
Clone – replicate some state to every client via single (double) server