[Ocfs2-tools-devel] [PATCH 0/4] defrag.ocfs2
Goldwyn Rodrigues
rgoldwyn at gmail.com
Tue Nov 23 06:37:02 PST 2010
Hi Tristan,
On Tue, Nov 23, 2010 at 1:15 AM, Tristan Ye <tristan.ye at oracle.com> wrote:
> Goldwyn Rodrigues wrote:
>>
>> These patches add the defrag.ocfs2 utility to ocfs2-tools. The defrag
>> utility defrags the files of the filessystem, by consolidating the
>> extents of the files and consolidating the dirents of the directory. It
>> also clears up unused allocation groups.
>>
>> This utility requires the patch to fix alloc_clusters to allocate the
>> best clusters. I have sent this patch independently, since it is
>> not part of the defragmentation process.
>>
>> Changes:
>> - incorporated Joel's comments
>>
> Hi Goldwyn,
>
> Thanks for your persistent efforts on making the defrag.ocfs2 tools
> reality;)
>
> I'm playing it around with my current o2info patches, without quite diving
> into
> your codes deeply now, per my understanding, your patches seems aiming at
> mainly
> at defraging files, to reduce the extent records as your main goal in other
> words.
> which seems be changed a lot since your first version from the very
> beginning, right?
> after pass1/2, we reduce the number of extents for files and directories,
> the pass3
> can hopefully make some extent blocks be released, right?
Thanks for reviewing. There is no pass 3 in the defrag utility. Pass1
which defrags files and directories and pass 2 which frees unused
allocation groups.
After the first post, when you pointed out similar numbers, I added
the pass. In the post of the utility in July, I had posted pass 3, to
consolidate the filesystem as a whole, but Joel did not like the idea
of copying extents across filesystem, so I dropped it in the post of
September onwards. I guess, you and Joel can discuss what you really
want in the utility ;)
>
> Following are some interesting numbers from 'o2info' and 'debugfs.ocfs2'
>
> [1]. Free chunks number/size being reported against global bitmap:
>
> -. Before defrag:
> --------------------------------------------------------
> # o2info --freefrag 32 /dev/sda9
> Blocksize: 4096 bytes
> Clustersize: 32768 bytes
> Total clusters: 1250307
> Free clusters: 406165 (32.5%)
>
> Min. free extent: 32 KB
> Max. free extent: 184288 KB
> Avg. free extent: 320 KB
>
> Chunksize: 32768 bytes (1 clusters)
> Total chunks: 1250308
> Free chunks: 406165 (32.5%)
>
> HISTOGRAM OF FREE EXTENT SIZES:
> Extent Size Range : Free extents Free Clusters Percent
> 32K... 64K- : 5237 5237 1.29%
> 64K... 128K- : 7438 17878 4.40%
> 128K... 256K- : 14372 79073 19.47%
> 256K... 512K- : 6455 74499 18.34%
> 512K... 1024K- : 2060 43276 10.65%
> 1M... 2M- : 1134 47662 11.73%
> 2M... 4M- : 888 64649 15.92%
> 4M... 8M- : 24 3463 0.85%
> 8M... 16M- : 4 1324 0.33%
> 16M... 32M- : 1 764 0.19%
> 128M... 256M- : 12 68340 16.83%
> --------------------------------------------------------
>
> -. After defrag:
> --------------------------------------------------------
> # o2info --freefrag 32 /dev/sda9
> Blocksize: 4096 bytes
> Clustersize: 32768 bytes
> Total clusters: 1250307
> Free clusters: 408550 (32.7%)
>
> Min. free extent: 32 KB
> Max. free extent: 184288 KB
> Avg. free extent: 320 KB
>
> Chunksize: 32768 bytes (1 clusters)
> Total chunks: 1250308
> Free chunks: 407869 (32.6%)
>
> HISTOGRAM OF FREE EXTENT SIZES:
> Extent Size Range : Free extents Free Clusters Percent
> 32K... 64K- : 5872 5872 1.44%
> 64K... 128K- : 7864 18959 4.64%
> 128K... 256K- : 15009 82058 20.09%
> 256K... 512K- : 6321 73165 17.91%
> 512K... 1024K- : 1980 41703 10.21%
> 1M... 2M- : 1123 47472 11.62%
> 2M... 4M- : 886 64717 15.84%
> 4M... 8M- : 27 4080 1.00%
> 8M... 16M- : 6 2055 0.50%
> 16M... 32M- : 3 2128 0.52%
> 32M... 64M- : 2 2951 0.72%
> 128M... 256M- : 11 62709 15.35%
>
> --------------------------------------------------------
>
>
> [2]. Extent allocator layout:
> -. Before defrag:
> --------------------------------------------------------------------
> debugfs: stat extent_alloc:0000
> Inode: 28 Mode: 0644 Generation: 2820423521 (0xa81c3f61)
> FS Generation: 2820423521 (0xa81c3f61)
> CRC32: edba1e7d ECC: 0270
> Type: Regular Attr: 0x0 Flags: Valid System Allocbitmap Chain
> Dynamic Features: (0x0)
> User: 0 (root) Group: 0 (root) Size: 12582912
> Links: 1 Clusters: 384
> ctime: 0x4ceb3934 -- Tue Nov 23 11:47:00 2010
> atime: 0x4ceb3934 -- Tue Nov 23 11:47:00 2010
> mtime: 0x4ceb3934 -- Tue Nov 23 11:47:00 2010
> dtime: 0x0 -- Thu Jan 1 08:00:00 1970
> ctime_nsec: 0x00000000 -- 0
> atime_nsec: 0x00000000 -- 0
> mtime_nsec: 0x00000000 -- 0
> Refcount Block: 0
> Last Extblk: 0 Orphan Slot: 0
> Sub Alloc Slot: Global Sub Alloc Bit: 12
> Bitmap Total: 3072 Used: 100 Free: 2972
> Clusters per Group: 128 Bits per Cluster: 8
> Count: 243 Next Free Rec: 3
> ## Total Used Free Block#
> 0 1024 32 992 250624
> 1 1024 34 990 251648
> 2 1024 34 990 252672
>
> Group Chain: 0 Parent Inode: 28 Generation: 2820423521
> CRC32: a492f1a3 ECC: 00f8
> ## Block# Total Used Free Contig Size 0
> 250624 1024 32 992 983 4032
> Group Chain: 1 Parent Inode: 28 Generation: 2820423521
> CRC32: 3dedeb22 ECC: 01a0
> ## Block# Total Used Free Contig Size 0
> 251648 1024 34 990 983 4032
> Group Chain: 2 Parent Inode: 28 Generation: 2820423521
> CRC32: 059ae49a ECC: 0187
> ## Block# Total Used Free Contig Size 0
> 252672 1024 34 990 984 4032
> --------------------------------------------------------------------
> -. After defrag:
> --------------------------------------------------------------------
> debugfs: stat extent_alloc:0000
> Inode: 28 Mode: 0644 Generation: 2820423521 (0xa81c3f61)
> FS Generation: 2820423521 (0xa81c3f61)
> CRC32: dec80e05 ECC: 00b7
> Type: Regular Attr: 0x0 Flags: Valid System Allocbitmap Chain
> Dynamic Features: (0x0)
> User: 0 (root) Group: 0 (root) Size: 4194304
> Links: 1 Clusters: 128
> ctime: 0x4ceb3934 -- Tue Nov 23 11:47:00 2010
> atime: 0x4ceb3934 -- Tue Nov 23 11:47:00 2010
> mtime: 0x4ceb3934 -- Tue Nov 23 11:47:00 2010
> dtime: 0x0 -- Thu Jan 1 08:00:00 1970
> ctime_nsec: 0x00000000 -- 0
> atime_nsec: 0x00000000 -- 0
> mtime_nsec: 0x00000000 -- 0
> Refcount Block: 0
> Last Extblk: 0 Orphan Slot: 0
> Sub Alloc Slot: Global Sub Alloc Bit: 12
> Bitmap Total: 1024 Used: 98 Free: 926
> Clusters per Group: 128 Bits per Cluster: 8
> Count: 243 Next Free Rec: 1
> ## Total Used Free Block#
> 0 1024 98 926 250624
>
> Group Chain: 0 Parent Inode: 28 Generation: 2820423521
> CRC32: 1aec9e0b ECC: 00a0
> ## Block# Total Used Free Contig Size 0
> 250624 1024 98 926 923 4032
> --------------------------------------------------------------------
>
>
> From above statistics, the awesome thing is that we do reduce the number of
> extents
> for files, and gained 2 extent block being freed, while looking at the
> global_bitmap,
> we even found more smaller free chunks at the cost of eating bigger free
> chunks up, it
> somehow makes the volume look more fragmented in terms of 'how many free
> contiguous chunks
> and their size, in global_bitmap for our allocation'.
>
>
Yes, I understand. Having pass 3 of defragging the filesystem as a
whole would have shown better numbers with your utility.
--
Goldwyn
More information about the Ocfs2-tools-devel
mailing list