[Ocfs2-devel] Mixed mounts w/ different physical block sizes (long post)

[Changwei Ge]

On 2017/9/19 14:47, Michael Ulbrich wrote:
> Hi Changwei,
> 
> thanks for looking into this!
> 
> On 19/09/17 05:32, Changwei Ge wrote:
> 
>> Could you please also provide information about *slot_map*, just type
>> "slotmap" in debugfs.ocfs2 tool. This will be helpful to analysis your case.
>>
>> Please also paste output generated by :
>> cat /sys/kernel/config/cluster/<you cluster name>/heartbeat/<file system
>> UUID>
>> So we see how your cluster is configured.
>> Files like block_bytes, blocks and start_block are preferred.
> 
> Ok, here we go. The 4k fs is currently mounted on 6 nodes:
> 
> hb
>          node: node              seq       generation checksum
>             8:    1 0000000059c0b87d 4b54662f8a10a4c6 ce05089c
>            16:    2 0000000059c0b87d 2e78b067074950f9 057c9608
>            24:    3 0000000059c0b87c 46f4e173012a4b7b a2073bec
>            40:    5 0000000059c0b87b c0a8e3023e9edaa6 bbca2048
>            48:    6 0000000059c0b87c 304d8bc8e22383a2 2f6002e6
>            64:    8 0000000059c0b87c 8d9c95c4b0296c70 f5e8d50a
> 
> And the associated slot map:
> 
> slotmap
> 	Slot#   Node#
> 	    0       1
> 	    1       2
> 	    2       3
> 	    3       6
> 	    4       5
> 	    5       8
> 
> Info from /sys/kernel/config/cluster/ocfs1_cluster/heartbeat/<UUID>
Um.
You mentioned that when the 8th node mounted the file system heartbeat 
sequence mismatching showed up.
But I can only see 6 nodes here.
Is this the scenario where your case happened?


> 
> block_bytes: 4096
> blocks:       255
> start_block:  273

> 
>>> Operation is stable so far. No 'Heartbeat sequence mismatch' errors.
>>> Still strange the "times 8" values in column "node:" but this may be a
>>> purely aesthetical issue.
>> I suppose this is because debugfs.ocfs2 *assumes* that block devices are
>> all 512 bytes formatted.
>> Perhaps we can improve this.
> 
> Yep, that would be great!
> 
> And what about this scenario of mixed mounts partly from nodes accessing
> the device based on 512 byte sectors and partly from nodes seeing it as
> a 4k device: should this be avoided or is it supposed to work because
> ocfs2 internally maps the different sized sectors to a common structure
> in the heartbeat region?
> 
How can this happen?
Do you mean that a unique physical disk device which is accessed by 
different nodes shows different block sizes (512 bytes and 4k)?

For example, node A accesses disk by block size - 512 bytes, however, 
node B accesses the *same* disk by block size - 4k.

Thanks,
Changwei.
> Thanks again ... Michael
>