[Ocfs2-tools-devel] [PATCH 0/4] defrag.ocfs2

[Goldwyn Rodrigues]

Hi Tristan,

Thanks for reviewing.

On Thu, Mar 18, 2010 at 5:38 AM, tristan <tristan.ye at oracle.com> wrote:
> Hi goldwyn,
>
> Per my understanding,
>
> pass1 disclaim the unused inode block groups back to global bitmap, the
> intention here is to leave more contiguous space in gb, and pass3 therefore
> can perfrom a better defragmentation, right?

Correct.

>
> One question, why not also disclaim the unused extent block groups?

Somehow, My small device did not show a great number of extent block
groups, and I concluded (incorrectly) that data allocations are
happening from global bitmap directly. Is there an option I am missing
or is it because I am kernel 2.6.31?
Yes, it would be nice to have that as well, and it should be just a
matter of adding another loop. I'll do that.

>
> pass2 is packing the dirents from one extent to another, the sliding windows
> method looks awesome:)
>

Thanks

>
> pass3 is trying to fill the holes in bg by latter extents which fits best.
>
> Am I understanding these in a wrong way?
>

No, you are correct.

> And goldwyn,
>
> I'm also wondering if you've got some interesting benchmark data during your
> lab testing to show the power of patch persuasively;)

You can enable DEBUG compilation/define flag to get an insight of the
group descriptors before and after the defraging the filesystem
(pass3) This should show more usage of earlier group descriptors and
lesser usage of later ones.


>
> Per my testing, I'm sorry to say that I even found a negative effect brought
> by your patches:(

Hmm, I know why this is happening, most of this is contributed by the
later gds. We could coalesce the extent_recs in all group descriptors
(as Wengang suggested as a first pass). Actually, I started with
Wengang's algorithm but realized there would be more data moves that
way.


>
> ==============================================
> Fs without defrag
> ==============================================
> Blocksize: 4096 bytes
> Clustersize: 4096 bytes
> Total clusters: 7325632
> Free clusters: 6811198 (93.0%)
>
> Min. free extent: 2044 KB
> Max. free extent: 129020 KB
> Avg. free extent: 120552 KB
>
> Chunksize: 1048576 bytes (256 clusters)
> Total chunks: 28616
> Free chunks: 26381 (92.2%)
>
> HISTOGRAM OF FREE EXTENT SIZES:
> Extent Size Range : Free extents Free Clusters Percent
> 1M... 2M- : 1 511 0.01%
> 2M... 4M- : 2 1668 0.02%
> 4M... 8M- : 9 13815 0.20%
> 8M... 16M- : 2 7614 0.11%
> 32M... 64M- : 2 32254 0.47%
> 64M... 128M- : 210 6755336 99.18%
>
>
> ==============================================
> Fs after defrag
> ==============================================
> Blocksize: 4096 bytes
> Clustersize: 4096 bytes
> Total clusters: 7325632
> Free clusters: 6811198 (93.0%)
>
> Min. free extent: 4 KB
> Max. free extent: 129020 KB
> Avg. free extent: 101280 KB
>
> Chunksize: 1048576 bytes (256 clusters)
> Total chunks: 28616
> Free chunks: 26379 (92.2%)
>
> HISTOGRAM OF FREE EXTENT SIZES:
> Extent Size Range : Free extents Free Clusters Percent
> 4K... 8K- : 1 1 0.00%
> 8K... 16K- : 3 8 0.00%
> 16K... 32K- : 9 50 0.00%
> 32K... 64K- : 5 48 0.00%
> 64K... 128K- : 6 135 0.00%
> 128K... 256K- : 6 294 0.00%
> 256K... 512K- : 8 725 0.01%
> 512K... 1024K- : 2 292 0.00%
> 1M... 2M- : 2 846 0.01%
> 2M... 4M- : 3 2414 0.04%
> 4M... 8M- : 8 11752 0.17%
> 8M... 16M- : 2 7443 0.11%
> 16M... 32M- : 1 7022 0.10%
> 32M... 64M- : 4 52979 0.78%
> 64M... 128M- : 209 6727189 98.77%
>
>
> Disk layout reported by 'o2info' show us that the fs was even more
> fragmented after being defraged.
>

The 64-128M and 4-8M has decreased, but the rest have increased.

Thanks for pointing this out. I will improvise by adding another pass
of coalescing the gds as a whole as well. Besides, I expect some more
improvements after file defragmentation is in place.

>
> Anyway, I'll take much closer look on your 3 patches in detail.
>
>
> Regards,
> Tristan.
>
> Goldwyn Rodrigues wrote:
>>
>> These patches adds a new tool to defrag ocfs2 filesystem.
>> There are 3 passes - Pass 1: Clears unused inode groups
>> Pass 2: Defrags the directories, coalescing the directory entries
>> Pass 3: Defrags the filesystem as a whole
>>
>> I haven't done defraging of files (merging of file extents) as yet.
>> Algorithms and limitations are in patch headers of the pass they
>> introduce.
>>
>> Disclaimer: Don't blame me if it eats your data ;)
>>
>> Comments/Suggestions welcome.
>>
>>
>
>



-- 
Goldwyn