| commit | 6703c36d3cd9e83ec3375e5dce04676fa84b0e67 | [log] [tgz] |
|---|---|---|
| author | Bob Dionne <bob@cloudant.com> | Sat Dec 01 07:46:34 2012 -0500 |
| committer | Bob Dionne <bob@cloudant.com> | Thu Feb 07 13:55:23 2013 -0500 |
| tree | 40bde14721b0d2cfcefb232729f195344eed1376 | |
| parent | b952dbbc73d884752ce62023d510c4e6c8b14865 [diff] |
Manage Excessive amount of spawned pids rexi_utils:send uses the noconnect and nosuspend options in order to immediately return to the controller and spawns new processes to actually send the messages, assuming the remote node is only temporarily down. In the nosuspend case these pids can hang around indefinitely and build up. This patch sends the messages that need to be sent from spawned processes to a new gen_server, rexi_manager, which manages a group of rexi_governors, one for each node. The governor does the spawning and monitoring of the pids and keeps track of how many are sent. If a node down message is received the manager sets a timer which when expired tells the appropriate governor to kill all the pids waiting on that node. A cap prevents spawning of processes above a certain amount, after which messages are dropped on the floor. BugzID: 15608
Rexi is a tailor-made RPC server application for sending CouchDB operations to nodes in a cluster. It is used in BigCouch as the remote procedure vehicle to get fabric functions to execute on remote cluster nodes.
Rexi better fits the needs of the BigCouch distributed data store by dropping some unneeded overhead in rex, the RPC server that ships with Erlang/OTP. Rexi is optimized for the case when you need to spawn a bunch of remote processes. Cast messages are sent from the origin to the remote rexi server, and local processes are spawned from there, which is vastly more efficient than spawning remote processes from the origin. You still get monitoring of the remote processes, but the request-handling process doesn't get stuck trying to connect to an overloaded/dead node. ‘rexi_DOWN’ messages will arrive at the client eventually. This has been an extremely advantageous mix of latency and failure detection, vastly improving the performance of BigCouch.
Rexi is used in conjunction with ‘Fabric’ which is also an application within BigCouch, but can be used on a stand-alone basis.
Rexi requires R13B03 or higher and can be built with rebar, which comes bundled in the repository.