[OracleOSS] [TitleIndex] [WordIndex]

OCFS2/DesignDocs/UserspaceClustering/UsingCPG

Concurrent Process Groups

The old approach used libgroup to manage mount groups. We're going to use the lower-level CPG functionality of OpenAIS. groupd is already using it, why fight with libgroup's API?

Notes

Some notes on using CPG. Mostly this is stuff I learned while looking at groupd.

When groupd gets a PROCDOWN event, it kills the related node. CPG has two DOWN events - PROCDOWN and NODEDOWN. PROCDOWN means the application died (eg, groupd on one node gets PROCDOWN when groupd dies on another node) but the node is still alive and well. NODEDOWN means the node is down and can't be found. NODEDOWN is easy - it's fencing, crashing, or whatever. The stack is going to make sure the node is fenced and safe. With PROCDOWN, the other node might be quite happy. fenced can talk, so no fencing happens. But if groupd is down on that node, the local groupd cannot trust group state. Thus, groupd will issue cman_kill_node() to ensure the other node is fenced. We need to do the same with ocfs2_controld - assuming that node is part of a mount group.

Groupd has the concept of recovery sets. When a process goes down on another node (groupd or otherwise, anything backed by cpg), groupd has to manage a layered recovery. Thus, it saves off the set of dead nodes/processes for later processing. We don't need this for ocfs2_controld right yet - node down is just communicated to ocfs2 - but we might care later when we support plocks.


2011-12-23 01:01