[Ocfs2-users] read/write performance across cluster

[Srinivas Eeda]

Florin Andrei wrote:
> On 05/04/2011 09:44 AM, Srinivas Eeda wrote:
>   
>> Yes, there is locking involved. Extending a file needs an exclusive
>> lock. Grepping a file needs read lock. If the same node(lets call it
>> writer node) does extending and grepping, then grep already has a
>> compatible lock and the data is also cached on this node. When another
>> node does the grep(lets call it reader node), then it needs to acquire
>> read lock. This requires the writer node to down grade the lock which
>> requires the node to flush data to the disk. Then the reader node has to
>> read the data to the cache. So when grep is done on the reader node,
>> there are additional messages and additional I/O's to the disk are done.
>>     
>
> Any workarounds?
>
> Obviously doing read and append on the same node is one way.
>
> Could the logger extend the file in a different way? The software is 
> homegrown, we could modify it to make it more OCFS2-friendly, if that's 
> possible.
>
> OCFS2 settings to tinker with?
>   
the above behavior is not particular to OCFS2 file system. It's a 
general behavior wrt cluster file systems. The tuning to some extent can 
come from your application. Like making every write hit the disk(by 
direct io or sync io) so the data doesn't need to be flushed when the 
lock needs to be down graded. Cache the inodes on the other nodes(by 
running ls, find, ..). This could avoid few lock messages. These changes 
may or may not have much improvements, but you could try.
>> Just curious, do you have any numbers to show the time difference?
>>     
>
> Not as of now. I switched it back to "fast mode" (read and append on the 
> same node) because it was pretty annoying. I'll see if I can run some 
> tests one of these days.
>
>