[Ocfs2-devel] 40TB RAID and OCFS2 woes (inode64, JDB2, huge partition support, Volume might try to write to blocks beyond what jbd can address in 32 bits)
Robert Smith
spamfree at wansecurity.com
Sun Jan 3 17:09:11 PST 2010
Sure, the easiest first:
#
# debufs.ocfs2
#
root at s2-replay02:~# echo 'stat biggest_yet_file'|debugfs.ocfs2 /dev/replays/ReplayDataVolume001
debugfs.ocfs2 1.4.3
debugfs: stat biggest_yet_file
Inode: 17669 Mode: 0644 Generation: 2699404090 (0xa0e5a33a)
FS Generation: 2687878495 (0xa035c55f)
CRC32: 36389313 ECC: 07e3
Type: Regular Attr: 0x0 Flags: Valid
Dynamic Features: (0x0)
User: 0 (root) Group: 0 (root) Size: 20971520000006
Links: 1 Clusters: 20000001
ctime: 0x4b3e2ada -- Fri Jan 1 11:03:22 2010
atime: 0x4b3e2ae3 -- Fri Jan 1 11:03:31 2010
mtime: 0x4b3e2ada -- Fri Jan 1 11:03:22 2010
dtime: 0x0 -- Wed Dec 31 18:00:00 1969
ctime_nsec: 0x326a39d4 -- 845822420
atime_nsec: 0x051d8c1a -- 85822490
mtime_nsec: 0x326a39d4 -- 845822420
Last Extblk: 2213020163
Sub Alloc Slot: 0 Sub Alloc Bit: 5
Tree Depth: 1 Count: 243 Next Free Rec: 3
## Offset Clusters Block#
0 0 7821880 2213020161
1 7821880 8126496 2213020162
2 15948376 4051625 2213020163
SubAlloc Bit: 1 SubAlloc Slot: 0
Blknum: 2213020161 Next Leaf: 2213020162
CRC32: ef11afcc ECC: 071c
Tree Depth: 0 Count: 252 Next Free Rec: 252
## Offset Clusters Block# Flags
0 0 3004 317517056 0x0
1 3004 14568 313786624 0x0
2 17572 14672 318286080 0x0
3 32244 32248 49545472 0x0
4 64492 32248 57803008 0x0
5 96740 32248 74318080 0x0
6 128988 32248 82575616 0x0
7 161236 32248 90833152 0x0
8 193484 32248 99090688 0x0
9 225732 32248 107348224 0x0
10 257980 32248 115605760 0x0
11 290228 32248 123863296 0x0
12 322476 32248 132120832 0x0
13 354724 32248 140378368 0x0
14 386972 32248 148635904 0x0
15 419220 32248 156893440 0x0
16 451468 32248 165150976 0x0
17 483716 32248 173408512 0x0
18 515964 32248 181666048 0x0
19 548212 32248 189923584 0x0
20 580460 32248 198181120 0x0
21 612708 32248 206438656 0x0
22 644956 32248 214696192 0x0
23 677204 32248 222953728 0x0
24 709452 32248 231211264 0x0
25 741700 32248 239468800 0x0
26 773948 32248 247726336 0x0
27 806196 32248 255983872 0x0
28 838444 32248 272498944 0x0
29 870692 32248 280756480 0x0
30 902940 32248 289014016 0x0
31 935188 32248 297271552 0x0
32 967436 32248 305529088 0x0
33 999684 32248 322044160 0x0
34 1031932 32248 330301696 0x0
35 1064180 32248 338559232 0x0
36 1096428 32248 346816768 0x0
37 1128676 32248 355074304 0x0
38 1160924 32248 363331840 0x0
39 1193172 32248 371589376 0x0
40 1225420 32248 379846912 0x0
41 1257668 32248 388104448 0x0
42 1289916 32248 396361984 0x0
43 1322164 32248 404619520 0x0
44 1354412 32248 412877056 0x0
45 1386660 32248 421134592 0x0
46 1418908 32248 429392128 0x0
47 1451156 32248 437649664 0x0
48 1483404 32248 445907200 0x0
49 1515652 32248 454164736 0x0
50 1547900 32248 462422272 0x0
51 1580148 32248 470679808 0x0
52 1612396 32248 478937344 0x0
53 1644644 32248 487194880 0x0
54 1676892 32248 495452416 0x0
55 1709140 32248 503709952 0x0
56 1741388 32248 511967488 0x0
57 1773636 32248 520225024 0x0
58 1805884 32248 528482560 0x0
59 1838132 32248 536740096 0x0
60 1870380 32248 544997632 0x0
61 1902628 32248 553255168 0x0
62 1934876 32248 561512704 0x0
63 1967124 32248 569770240 0x0
64 1999372 32248 578027776 0x0
65 2031620 32248 586285312 0x0
66 2063868 32248 594542848 0x0
67 2096116 32248 602800384 0x0
68 2128364 32248 611057920 0x0
69 2160612 32248 619315456 0x0
70 2192860 32248 627572992 0x0
71 2225108 32248 635830528 0x0
72 2257356 32248 644088064 0x0
73 2289604 32248 652345600 0x0
74 2321852 32248 660603136 0x0
75 2354100 32248 668860672 0x0
76 2386348 32248 677118208 0x0
77 2418596 32248 685375744 0x0
78 2450844 32248 693633280 0x0
79 2483092 32248 701890816 0x0
80 2515340 32248 710148352 0x0
81 2547588 32248 718405888 0x0
82 2579836 32248 726663424 0x0
83 2612084 32248 734920960 0x0
84 2644332 32248 743178496 0x0
85 2676580 32248 751436032 0x0
86 2708828 32248 759693568 0x0
87 2741076 32248 767951104 0x0
88 2773324 32248 776208640 0x0
89 2805572 32248 784466176 0x0
90 2837820 32248 792723712 0x0
91 2870068 32248 800981248 0x0
92 2902316 32248 809238784 0x0
93 2934564 32248 817496320 0x0
94 2966812 32248 825753856 0x0
95 2999060 32248 834011392 0x0
96 3031308 32248 842268928 0x0
97 3063556 32248 850526464 0x0
98 3095804 32248 858784000 0x0
99 3128052 32248 867041536 0x0
100 3160300 32248 875299072 0x0
101 3192548 32248 883556608 0x0
102 3224796 32248 891814144 0x0
103 3257044 32248 900071680 0x0
104 3289292 32248 908329216 0x0
105 3321540 32248 916586752 0x0
106 3353788 32248 924844288 0x0
107 3386036 32248 933101824 0x0
108 3418284 32248 941359360 0x0
109 3450532 32248 949616896 0x0
110 3482780 32248 957874432 0x0
111 3515028 32248 966131968 0x0
112 3547276 32248 974389504 0x0
113 3579524 32248 982647040 0x0
114 3611772 32248 990904576 0x0
115 3644020 32248 999162112 0x0
116 3676268 32248 1007419648 0x0
117 3708516 32248 1015677184 0x0
118 3740764 32248 1023934720 0x0
119 3773012 32248 1032192256 0x0
120 3805260 32248 1040449792 0x0
121 3837508 32248 1048707328 0x0
122 3869756 32248 1056964864 0x0
123 3902004 32248 1065222400 0x0
124 3934252 32248 1073479936 0x0
125 3966500 32248 1081737472 0x0
126 3998748 32248 1089995008 0x0
127 4030996 32248 1098252544 0x0
128 4063244 32248 1106510080 0x0
129 4095492 32248 1114767616 0x0
130 4127740 32248 1123025152 0x0
131 4159988 32248 1131282688 0x0
132 4192236 32248 1139540224 0x0
133 4224484 32248 1147797760 0x0
134 4256732 32248 1156055296 0x0
135 4288980 32248 1164312832 0x0
136 4321228 32248 1172570368 0x0
137 4353476 32248 1180827904 0x0
138 4385724 32248 1189085440 0x0
139 4417972 32248 1197342976 0x0
140 4450220 32248 1205600512 0x0
141 4482468 32248 1213858048 0x0
142 4514716 32248 1222115584 0x0
143 4546964 32248 1230373120 0x0
144 4579212 32248 1238630656 0x0
145 4611460 32248 1246888192 0x0
146 4643708 32248 1255145728 0x0
147 4675956 32248 1263403264 0x0
148 4708204 32248 1271660800 0x0
149 4740452 32248 1279918336 0x0
150 4772700 32248 1288175872 0x0
151 4804948 32248 1296433408 0x0
152 4837196 32248 1304690944 0x0
153 4869444 32248 1312948480 0x0
154 4901692 32248 1321206016 0x0
155 4933940 32248 1329463552 0x0
156 4966188 32248 1337721088 0x0
157 4998436 32248 1345978624 0x0
158 5030684 32248 1354236160 0x0
159 5062932 32248 1362493696 0x0
160 5095180 32248 1370751232 0x0
161 5127428 32248 1379008768 0x0
162 5159676 32248 1387266304 0x0
163 5191924 32248 1395523840 0x0
164 5224172 32248 1403781376 0x0
165 5256420 32248 1412038912 0x0
166 5288668 32248 1420296448 0x0
167 5320916 32248 1428553984 0x0
168 5353164 32248 1436811520 0x0
169 5385412 32248 1445069056 0x0
170 5417660 32248 1453326592 0x0
171 5449908 32248 1461584128 0x0
172 5482156 32248 1469841664 0x0
173 5514404 32248 1478099200 0x0
174 5546652 32248 1486356736 0x0
175 5578900 32248 1494614272 0x0
176 5611148 32248 1502871808 0x0
177 5643396 32248 1511129344 0x0
178 5675644 32248 1519386880 0x0
179 5707892 32248 1527644416 0x0
180 5740140 32248 1535901952 0x0
181 5772388 32248 1544159488 0x0
182 5804636 32248 1552417024 0x0
183 5836884 32248 1560674560 0x0
184 5869132 32248 1568932096 0x0
185 5901380 32248 1577189632 0x0
186 5933628 32248 1585447168 0x0
187 5965876 32248 1593704704 0x0
188 5998124 32248 1601962240 0x0
189 6030372 32248 1610219776 0x0
190 6062620 32248 1618477312 0x0
191 6094868 32248 1626734848 0x0
192 6127116 32248 1634992384 0x0
193 6159364 32248 1643249920 0x0
194 6191612 32248 1651507456 0x0
195 6223860 32248 1659764992 0x0
196 6256108 32248 1668022528 0x0
197 6288356 32248 1676280064 0x0
198 6320604 32248 1684537600 0x0
199 6352852 32248 1692795136 0x0
200 6385100 32248 1701052672 0x0
201 6417348 32248 1709310208 0x0
202 6449596 32248 1717567744 0x0
203 6481844 32248 1725825280 0x0
204 6514092 1016 16515328 0x0
205 6515108 31224 16777472 0x0
206 6546332 4088 66060544 0x0
207 6550420 28152 67109120 0x0
208 6578572 16376 264241408 0x0
209 6594948 15864 268435712 0x0
210 6610812 944 18688 0x0
211 6611756 3064 262400 0x0
212 6614820 12280 1048832 0x0
213 6627100 15864 4194560 0x0
214 6642964 32248 1734082816 0x0
215 6675212 17992 44938496 0x0
216 6693204 32248 2006581504 0x0
217 6725452 32248 2023096576 0x0
218 6757700 32248 2072641792 0x0
219 6789948 32248 2270822656 0x0
220 6822196 32248 2014839040 0x0
221 6854444 32248 2320367872 0x0
222 6886692 32248 2031354112 0x0
223 6918940 32248 2039611648 0x0
224 6951188 32248 2056126720 0x0
225 6983436 32248 2064384256 0x0
226 7015684 32248 2080899328 0x0
227 7047932 32248 2089156864 0x0
228 7080180 32248 2097414400 0x0
229 7112428 32248 2105671936 0x0
230 7144676 32248 2113929472 0x0
231 7176924 32248 2122187008 0x0
232 7209172 32248 2130444544 0x0
233 7241420 32248 2138702080 0x0
234 7273668 32248 2146959616 0x0
235 7305916 32248 2155217152 0x0
236 7338164 32248 2163474688 0x0
237 7370412 32248 2171732224 0x0
238 7402660 32248 2179989760 0x0
239 7434908 32248 2188247296 0x0
240 7467156 32248 2196504832 0x0
241 7499404 32248 2204762368 0x0
242 7531652 1 2213019904 0x0
243 7531653 32243 2213021184 0x0
244 7563896 32248 2221277440 0x0
245 7596144 32248 2229534976 0x0
246 7628392 32248 2237792512 0x0
247 7660640 32248 2246050048 0x0
248 7692888 32248 2254307584 0x0
249 7725136 32248 2262565120 0x0
250 7757384 32248 2279080192 0x0
251 7789632 32248 2287337728 0x0
SubAlloc Bit: 2 SubAlloc Slot: 0
Blknum: 2213020162 Next Leaf: 2213020163
CRC32: 95e1aa74 ECC: 4a6b
Tree Depth: 0 Count: 252 Next Free Rec: 252
## Offset Clusters Block# Flags
0 7821880 32248 2295595264 0x0
1 7854128 32248 2303852800 0x0
2 7886376 32248 2312110336 0x0
3 7918624 32248 2328625408 0x0
4 7950872 32248 2336882944 0x0
5 7983120 32248 2345140480 0x0
6 8015368 32248 2353398016 0x0
7 8047616 32248 2361655552 0x0
8 8079864 32248 2369913088 0x0
9 8112112 32248 2378170624 0x0
10 8144360 32248 2386428160 0x0
11 8176608 32248 2394685696 0x0
12 8208856 32248 2402943232 0x0
13 8241104 32248 2411200768 0x0
14 8273352 32248 2419458304 0x0
15 8305600 32248 2427715840 0x0
16 8337848 32248 2435973376 0x0
17 8370096 32248 2444230912 0x0
18 8402344 32248 2452488448 0x0
19 8434592 32248 2460745984 0x0
20 8466840 32248 2469003520 0x0
21 8499088 32248 2477261056 0x0
22 8531336 32248 2485518592 0x0
23 8563584 32248 2493776128 0x0
24 8595832 32248 2502033664 0x0
25 8628080 32248 2510291200 0x0
26 8660328 32248 2518548736 0x0
27 8692576 32248 2526806272 0x0
28 8724824 32248 2535063808 0x0
29 8757072 32248 2543321344 0x0
30 8789320 32248 2551578880 0x0
31 8821568 32248 2559836416 0x0
32 8853816 32248 2568093952 0x0
33 8886064 32248 2576351488 0x0
34 8918312 32248 2584609024 0x0
35 8950560 32248 2592866560 0x0
36 8982808 32248 2601124096 0x0
37 9015056 32248 2609381632 0x0
38 9047304 32248 2617639168 0x0
39 9079552 32248 2625896704 0x0
40 9111800 32248 2634154240 0x0
41 9144048 32248 2642411776 0x0
42 9176296 32248 2650669312 0x0
43 9208544 32248 2658926848 0x0
44 9240792 32248 2667184384 0x0
45 9273040 32248 2675441920 0x0
46 9305288 32248 2683699456 0x0
47 9337536 32248 2691956992 0x0
48 9369784 32248 2700214528 0x0
49 9402032 32248 2708472064 0x0
50 9434280 32248 2716729600 0x0
51 9466528 32248 2724987136 0x0
52 9498776 32248 2733244672 0x0
53 9531024 32248 2741502208 0x0
54 9563272 32248 2749759744 0x0
55 9595520 32248 2758017280 0x0
56 9627768 32248 2766274816 0x0
57 9660016 32248 2774532352 0x0
58 9692264 32248 2782789888 0x0
59 9724512 32248 2791047424 0x0
60 9756760 32248 2799304960 0x0
61 9789008 32248 2807562496 0x0
62 9821256 32248 2815820032 0x0
63 9853504 32248 2824077568 0x0
64 9885752 32248 2832335104 0x0
65 9918000 32248 2840592640 0x0
66 9950248 32248 2848850176 0x0
67 9982496 32248 2857107712 0x0
68 10014744 32248 2865365248 0x0
69 10046992 32248 2873622784 0x0
70 10079240 32248 2881880320 0x0
71 10111488 32248 2890137856 0x0
72 10143736 32248 2898395392 0x0
73 10175984 32248 2906652928 0x0
74 10208232 32248 2914910464 0x0
75 10240480 32248 2923168000 0x0
76 10272728 32248 2931425536 0x0
77 10304976 32248 2939683072 0x0
78 10337224 32248 2947940608 0x0
79 10369472 32248 2956198144 0x0
80 10401720 32248 2964455680 0x0
81 10433968 32248 2972713216 0x0
82 10466216 32248 2980970752 0x0
83 10498464 32248 2989228288 0x0
84 10530712 32248 2997485824 0x0
85 10562960 32248 3005743360 0x0
86 10595208 32248 3014000896 0x0
87 10627456 32248 3022258432 0x0
88 10659704 32248 3030515968 0x0
89 10691952 32248 3038773504 0x0
90 10724200 32248 3047031040 0x0
91 10756448 32248 3055288576 0x0
92 10788696 32248 3063546112 0x0
93 10820944 32248 3071803648 0x0
94 10853192 32248 3080061184 0x0
95 10885440 32248 3088318720 0x0
96 10917688 32248 3096576256 0x0
97 10949936 32248 3104833792 0x0
98 10982184 32248 3113091328 0x0
99 11014432 32248 3121348864 0x0
100 11046680 32248 3129606400 0x0
101 11078928 32248 3137863936 0x0
102 11111176 32248 3146121472 0x0
103 11143424 32248 3154379008 0x0
104 11175672 32248 3162636544 0x0
105 11207920 32248 3170894080 0x0
106 11240168 32248 3179151616 0x0
107 11272416 32248 3187409152 0x0
108 11304664 32248 3195666688 0x0
109 11336912 32248 3203924224 0x0
110 11369160 32248 3212181760 0x0
111 11401408 32248 3220439296 0x0
112 11433656 32248 3228696832 0x0
113 11465904 32248 3236954368 0x0
114 11498152 32248 3245211904 0x0
115 11530400 32248 3253469440 0x0
116 11562648 32248 3261726976 0x0
117 11594896 32248 3269984512 0x0
118 11627144 32248 3278242048 0x0
119 11659392 32248 3286499584 0x0
120 11691640 32248 3294757120 0x0
121 11723888 32248 3303014656 0x0
122 11756136 32248 3311272192 0x0
123 11788384 32248 3319529728 0x0
124 11820632 32248 3327787264 0x0
125 11852880 32248 3336044800 0x0
126 11885128 32248 3344302336 0x0
127 11917376 32248 3352559872 0x0
128 11949624 32248 3360817408 0x0
129 11981872 32248 3369074944 0x0
130 12014120 32248 3377332480 0x0
131 12046368 32248 3385590016 0x0
132 12078616 32248 3393847552 0x0
133 12110864 32248 3402105088 0x0
134 12143112 32248 3410362624 0x0
135 12175360 32248 3418620160 0x0
136 12207608 32248 3426877696 0x0
137 12239856 32248 3435135232 0x0
138 12272104 32248 3443392768 0x0
139 12304352 32248 3451650304 0x0
140 12336600 32248 3459907840 0x0
141 12368848 32248 3468165376 0x0
142 12401096 32248 3476422912 0x0
143 12433344 32248 3484680448 0x0
144 12465592 32248 3492937984 0x0
145 12497840 32248 3501195520 0x0
146 12530088 32248 3509453056 0x0
147 12562336 32248 3517710592 0x0
148 12594584 32248 3525968128 0x0
149 12626832 32248 3534225664 0x0
150 12659080 32248 3542483200 0x0
151 12691328 32248 3550740736 0x0
152 12723576 32248 3558998272 0x0
153 12755824 32248 3567255808 0x0
154 12788072 32248 3575513344 0x0
155 12820320 32248 3583770880 0x0
156 12852568 32248 3592028416 0x0
157 12884816 32248 3600285952 0x0
158 12917064 32248 3608543488 0x0
159 12949312 32248 3616801024 0x0
160 12981560 32248 3625058560 0x0
161 13013808 32248 3633316096 0x0
162 13046056 32248 3641573632 0x0
163 13078304 32248 3649831168 0x0
164 13110552 32248 3658088704 0x0
165 13142800 32248 3666346240 0x0
166 13175048 32248 3674603776 0x0
167 13207296 32248 3682861312 0x0
168 13239544 32248 3691118848 0x0
169 13271792 32248 3699376384 0x0
170 13304040 32248 3707633920 0x0
171 13336288 32248 3715891456 0x0
172 13368536 32248 3724148992 0x0
173 13400784 32248 3732406528 0x0
174 13433032 32248 2047869184 0x0
175 13465280 32248 1742340352 0x0
176 13497528 32248 1750597888 0x0
177 13529776 32248 1758855424 0x0
178 13562024 32248 1767112960 0x0
179 13594272 32248 1775370496 0x0
180 13626520 32248 1783628032 0x0
181 13658768 32248 1791885568 0x0
182 13691016 32248 1800143104 0x0
183 13723264 32248 1808400640 0x0
184 13755512 32248 1816658176 0x0
185 13787760 32248 1824915712 0x0
186 13820008 32248 1833173248 0x0
187 13852256 32248 1841430784 0x0
188 13884504 32248 1849688320 0x0
189 13916752 32248 1857945856 0x0
190 13949000 32248 1866203392 0x0
191 13981248 32248 1874460928 0x0
192 14013496 32248 1882718464 0x0
193 14045744 32248 1890976000 0x0
194 14077992 32248 1899233536 0x0
195 14110240 32248 1907491072 0x0
196 14142488 32248 1915748608 0x0
197 14174736 32248 1924006144 0x0
198 14206984 32248 1932263680 0x0
199 14239232 32248 1940521216 0x0
200 14271480 32248 1948778752 0x0
201 14303728 32248 1957036288 0x0
202 14335976 32248 1965293824 0x0
203 14368224 32248 1973551360 0x0
204 14400472 32248 1981808896 0x0
205 14432720 32248 1990066432 0x0
206 14464968 32248 1998323968 0x0
207 14497216 32248 3740664064 0x0
208 14529464 32248 4013162752 0x0
209 14561712 32248 4029677824 0x0
210 14593960 32248 4079223040 0x0
211 14626208 32248 4277403904 0x0
212 14658456 32248 4021420288 0x0
213 14690704 32248 4326949120 0x0
214 14722952 32248 4037935360 0x0
215 14755200 32248 4046192896 0x0
216 14787448 32248 4062707968 0x0
217 14819696 32248 4070965504 0x0
218 14851944 32248 4087480576 0x0
219 14884192 32248 4095738112 0x0
220 14916440 32248 4103995648 0x0
221 14948688 32248 4112253184 0x0
222 14980936 32248 4120510720 0x0
223 15013184 32248 4128768256 0x0
224 15045432 32248 4137025792 0x0
225 15077680 32248 4145283328 0x0
226 15109928 32248 4153540864 0x0
227 15142176 32248 4161798400 0x0
228 15174424 32248 4170055936 0x0
229 15206672 32248 4178313472 0x0
230 15238920 32248 4186571008 0x0
231 15271168 32248 4194828544 0x0
232 15303416 32248 4203086080 0x0
233 15335664 32248 4211343616 0x0
234 15367912 32248 4219601152 0x0
235 15400160 32248 4227858688 0x0
236 15432408 32248 4236116224 0x0
237 15464656 32248 4244373760 0x0
238 15496904 32248 4252631296 0x0
239 15529152 32248 4260888832 0x0
240 15561400 32248 4269146368 0x0
241 15593648 32248 4285661440 0x0
242 15625896 32248 4293918976 0x0
243 15658144 32248 4302176512 0x0
244 15690392 32248 4310434048 0x0
245 15722640 32248 4318691584 0x0
246 15754888 32248 4335206656 0x0
247 15787136 32248 4343464192 0x0
248 15819384 32248 4351721728 0x0
249 15851632 32248 4359979264 0x0
250 15883880 32248 4368236800 0x0
251 15916128 32248 4376494336 0x0
SubAlloc Bit: 3 SubAlloc Slot: 0
Blknum: 2213020163 Next Leaf: 0
CRC32: c8c070b2 ECC: 63ca
Tree Depth: 0 Count: 252 Next Free Rec: 126
## Offset Clusters Block# Flags
0 15948376 32248 4384751872 0x0
1 15980624 32248 4393009408 0x0
2 16012872 32248 4401266944 0x0
3 16045120 32248 4409524480 0x0
4 16077368 32248 4417782016 0x0
5 16109616 32248 4426039552 0x0
6 16141864 32248 4434297088 0x0
7 16174112 32248 4442554624 0x0
8 16206360 32248 4450812160 0x0
9 16238608 32248 4459069696 0x0
10 16270856 32248 4467327232 0x0
11 16303104 32248 4475584768 0x0
12 16335352 32248 4483842304 0x0
13 16367600 32248 4492099840 0x0
14 16399848 32248 4500357376 0x0
15 16432096 32248 4508614912 0x0
16 16464344 32248 4516872448 0x0
17 16496592 32248 4525129984 0x0
18 16528840 32248 4533387520 0x0
19 16561088 32248 4541645056 0x0
20 16593336 32248 4549902592 0x0
21 16625584 32248 4558160128 0x0
22 16657832 32248 4566417664 0x0
23 16690080 32248 4574675200 0x0
24 16722328 32248 4582932736 0x0
25 16754576 32248 4591190272 0x0
26 16786824 32248 4599447808 0x0
27 16819072 32248 4607705344 0x0
28 16851320 32248 4615962880 0x0
29 16883568 32248 4624220416 0x0
30 16915816 32248 4632477952 0x0
31 16948064 32248 4640735488 0x0
32 16980312 32248 4648993024 0x0
33 17012560 32248 4657250560 0x0
34 17044808 32248 4665508096 0x0
35 17077056 32248 4673765632 0x0
36 17109304 32248 4682023168 0x0
37 17141552 32248 4690280704 0x0
38 17173800 32248 4698538240 0x0
39 17206048 32248 4706795776 0x0
40 17238296 32248 4715053312 0x0
41 17270544 32248 4723310848 0x0
42 17302792 32248 4731568384 0x0
43 17335040 32248 4739825920 0x0
44 17367288 32248 4748083456 0x0
45 17399536 32248 4756340992 0x0
46 17431784 32248 4764598528 0x0
47 17464032 32248 4772856064 0x0
48 17496280 32248 4781113600 0x0
49 17528528 32248 4789371136 0x0
50 17560776 32248 4797628672 0x0
51 17593024 32248 4805886208 0x0
52 17625272 32248 4814143744 0x0
53 17657520 32248 4822401280 0x0
54 17689768 32248 4830658816 0x0
55 17722016 32248 4838916352 0x0
56 17754264 32248 4847173888 0x0
57 17786512 32248 4855431424 0x0
58 17818760 32248 4863688960 0x0
59 17851008 32248 4871946496 0x0
60 17883256 32248 4880204032 0x0
61 17915504 32248 4888461568 0x0
62 17947752 32248 4896719104 0x0
63 17980000 32248 4904976640 0x0
64 18012248 32248 4913234176 0x0
65 18044496 32248 4921491712 0x0
66 18076744 32248 4929749248 0x0
67 18108992 32248 4938006784 0x0
68 18141240 32248 4946264320 0x0
69 18173488 32248 4954521856 0x0
70 18205736 32248 4962779392 0x0
71 18237984 32248 4971036928 0x0
72 18270232 32248 4979294464 0x0
73 18302480 32248 4987552000 0x0
74 18334728 32248 4995809536 0x0
75 18366976 32248 5004067072 0x0
76 18399224 32248 5012324608 0x0
77 18431472 32248 5020582144 0x0
78 18463720 32248 5028839680 0x0
79 18495968 32248 5037097216 0x0
80 18528216 32248 5045354752 0x0
81 18560464 32248 5053612288 0x0
82 18592712 32248 5061869824 0x0
83 18624960 32248 5070127360 0x0
84 18657208 32248 5078384896 0x0
85 18689456 32248 5086642432 0x0
86 18721704 32248 5094899968 0x0
87 18753952 32248 5103157504 0x0
88 18786200 32248 5111415040 0x0
89 18818448 32248 5119672576 0x0
90 18850696 32248 5127930112 0x0
91 18882944 32248 5136187648 0x0
92 18915192 32248 5144445184 0x0
93 18947440 32248 5152702720 0x0
94 18979688 32248 5160960256 0x0
95 19011936 32248 5169217792 0x0
96 19044184 32248 5177475328 0x0
97 19076432 32248 5185732864 0x0
98 19108680 32248 5193990400 0x0
99 19140928 32248 5202247936 0x0
100 19173176 32248 5210505472 0x0
101 19205424 32248 5218763008 0x0
102 19237672 32248 5227020544 0x0
103 19269920 32248 5235278080 0x0
104 19302168 32248 5243535616 0x0
105 19334416 32248 5251793152 0x0
106 19366664 32248 5260050688 0x0
107 19398912 32248 5268308224 0x0
108 19431160 32248 5276565760 0x0
109 19463408 32248 5284823296 0x0
110 19495656 32248 5293080832 0x0
111 19527904 32248 5301338368 0x0
112 19560152 32248 5309595904 0x0
113 19592400 32248 5317853440 0x0
114 19624648 32248 5326110976 0x0
115 19656896 32248 5334368512 0x0
116 19689144 32248 5342626048 0x0
117 19721392 32248 5350883584 0x0
118 19753640 32248 5359141120 0x0
119 19785888 32248 5367398656 0x0
120 19818136 32248 5375656192 0x0
121 19850384 32248 5383913728 0x0
122 19882632 32248 5392171264 0x0
123 19914880 32248 5400428800 0x0
124 19947128 32248 5408686336 0x0
125 19979376 20625 5416943872 0x0
debugfs:
root at s2-replay02:~#
#
# Append a line
#
root at s2-replay02:~# strace -ttt echo "hey there, second appendage of one line" >> /data/storage/ReplayDataVolume001/biggest_yet_file
1262563929.454995 execve("/bin/echo", ["echo", "hey there, second appendage of o"...], [/* 14 vars */]) = 0
1262563929.456493 brk(0) = 0x1935000
1262563929.456772 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff41769e000
1262563929.457075 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
1262563929.457361 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff41769c000
1262563929.457623 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
1262563929.457812 open("/etc/ld.so.cache", O_RDONLY) = 3
1262563929.457932 fstat(3, {st_mode=S_IFREG|0644, st_size=14369, ...}) = 0
1262563929.458023 mmap(NULL, 14369, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff417698000
1262563929.458101 close(3) = 0
1262563929.458174 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
1262563929.458269 open("/lib/libc.so.6", O_RDONLY) = 3
1262563929.458342 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\353\1\0\0\0\0\0"..., 832) = 832
1262563929.458444 fstat(3, {st_mode=S_IFREG|0755, st_size=1490312, ...}) = 0
1262563929.458546 mmap(NULL, 3598344, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff417112000
1262563929.458609 mprotect(0x7ff417278000, 2093056, PROT_NONE) = 0
1262563929.458686 mmap(0x7ff417477000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x165000) = 0x7ff417477000
1262563929.458783 mmap(0x7ff41747c000, 18440, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff41747c000
1262563929.458873 close(3) = 0
1262563929.458952 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff417697000
1262563929.459043 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff417696000
1262563929.459113 arch_prctl(ARCH_SET_FS, 0x7ff4176966f0) = 0
1262563929.459317 mprotect(0x7ff417477000, 16384, PROT_READ) = 0
1262563929.459385 mprotect(0x607000, 4096, PROT_READ) = 0
1262563929.459462 mprotect(0x7ff41769f000, 4096, PROT_READ) = 0
1262563929.459528 munmap(0x7ff417698000, 14369) = 0
1262563929.459801 brk(0) = 0x1935000
1262563929.459856 brk(0x1956000) = 0x1956000
1262563929.459966 open("/usr/lib/locale/locale-archive", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.460074 open("/usr/share/locale/locale.alias", O_RDONLY) = 3
1262563929.460232 fstat(3, {st_mode=S_IFREG|0644, st_size=2570, ...}) = 0
1262563929.460320 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff41769b000
1262563929.460381 read(3, "# Locale name alias data base.\n#"..., 4096) = 2570
1262563929.460561 read(3, "", 4096) = 0
1262563929.460618 close(3) = 0
1262563929.460673 munmap(0x7ff41769b000, 4096) = 0
1262563929.460901 open("/usr/lib/locale/en_US.UTF-8/LC_IDENTIFICATION", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.460999 open("/usr/lib/locale/en_US.utf8/LC_IDENTIFICATION", O_RDONLY) = 3
1262563929.461083 fstat(3, {st_mode=S_IFREG|0644, st_size=373, ...}) = 0
1262563929.461157 mmap(NULL, 373, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41769b000
1262563929.461229 close(3) = 0
1262563929.461297 open("/usr/lib/gconv/gconv-modules.cache", O_RDONLY) = 3
1262563929.461370 fstat(3, {st_mode=S_IFREG|0644, st_size=26048, ...}) = 0
1262563929.461442 mmap(NULL, 26048, PROT_READ, MAP_SHARED, 3, 0) = 0x7ff41768f000
1262563929.461514 close(3) = 0
1262563929.461675 open("/usr/lib/locale/en_US.UTF-8/LC_MEASUREMENT", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.461751 open("/usr/lib/locale/en_US.utf8/LC_MEASUREMENT", O_RDONLY) = 3
1262563929.461835 fstat(3, {st_mode=S_IFREG|0644, st_size=23, ...}) = 0
1262563929.461906 mmap(NULL, 23, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41769a000
1262563929.461977 close(3) = 0
1262563929.462127 open("/usr/lib/locale/en_US.UTF-8/LC_TELEPHONE", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.462195 open("/usr/lib/locale/en_US.utf8/LC_TELEPHONE", O_RDONLY) = 3
1262563929.462252 fstat(3, {st_mode=S_IFREG|0644, st_size=59, ...}) = 0
1262563929.462319 mmap(NULL, 59, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff417699000
1262563929.462367 close(3) = 0
1262563929.462487 open("/usr/lib/locale/en_US.UTF-8/LC_ADDRESS", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.462545 open("/usr/lib/locale/en_US.utf8/LC_ADDRESS", O_RDONLY) = 3
1262563929.462600 fstat(3, {st_mode=S_IFREG|0644, st_size=155, ...}) = 0
1262563929.462666 mmap(NULL, 155, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff417698000
1262563929.462714 close(3) = 0
1262563929.462833 open("/usr/lib/locale/en_US.UTF-8/LC_NAME", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.462891 open("/usr/lib/locale/en_US.utf8/LC_NAME", O_RDONLY) = 3
1262563929.462947 fstat(3, {st_mode=S_IFREG|0644, st_size=77, ...}) = 0
1262563929.463012 mmap(NULL, 77, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41768e000
1262563929.463062 close(3) = 0
1262563929.463178 open("/usr/lib/locale/en_US.UTF-8/LC_PAPER", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.463236 open("/usr/lib/locale/en_US.utf8/LC_PAPER", O_RDONLY) = 3
1262563929.463323 fstat(3, {st_mode=S_IFREG|0644, st_size=34, ...}) = 0
1262563929.463389 mmap(NULL, 34, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41768d000
1262563929.463437 close(3) = 0
1262563929.463559 open("/usr/lib/locale/en_US.UTF-8/LC_MESSAGES", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.463616 open("/usr/lib/locale/en_US.utf8/LC_MESSAGES", O_RDONLY) = 3
1262563929.463672 fstat(3, {st_mode=S_IFDIR|0755, st_size=80, ...}) = 0
1262563929.463735 close(3) = 0
1262563929.463781 open("/usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES", O_RDONLY) = 3
1262563929.463838 fstat(3, {st_mode=S_IFREG|0644, st_size=52, ...}) = 0
1262563929.463904 mmap(NULL, 52, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41768c000
1262563929.463952 close(3) = 0
1262563929.464073 open("/usr/lib/locale/en_US.UTF-8/LC_MONETARY", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.464131 open("/usr/lib/locale/en_US.utf8/LC_MONETARY", O_RDONLY) = 3
1262563929.464186 fstat(3, {st_mode=S_IFREG|0644, st_size=286, ...}) = 0
1262563929.464252 mmap(NULL, 286, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41768b000
1262563929.464300 close(3) = 0
1262563929.464431 open("/usr/lib/locale/en_US.UTF-8/LC_COLLATE", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.464489 open("/usr/lib/locale/en_US.utf8/LC_COLLATE", O_RDONLY) = 3
1262563929.464544 fstat(3, {st_mode=S_IFREG|0644, st_size=966938, ...}) = 0
1262563929.464609 mmap(NULL, 966938, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41759e000
1262563929.464657 close(3) = 0
1262563929.466654 open("/usr/lib/locale/en_US.UTF-8/LC_TIME", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.466858 open("/usr/lib/locale/en_US.utf8/LC_TIME", O_RDONLY) = 3
1262563929.467079 fstat(3, {st_mode=S_IFREG|0644, st_size=2454, ...}) = 0
1262563929.467296 mmap(NULL, 2454, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41759d000
1262563929.467500 close(3) = 0
1262563929.467830 open("/usr/lib/locale/en_US.UTF-8/LC_NUMERIC", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.468033 open("/usr/lib/locale/en_US.utf8/LC_NUMERIC", O_RDONLY) = 3
1262563929.468150 fstat(3, {st_mode=S_IFREG|0644, st_size=54, ...}) = 0
1262563929.468233 mmap(NULL, 54, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41759c000
1262563929.468308 close(3) = 0
1262563929.468476 open("/usr/lib/locale/en_US.UTF-8/LC_CTYPE", O_RDONLY) = -1 ENOENT (No such file or directory)
1262563929.468553 open("/usr/lib/locale/en_US.utf8/LC_CTYPE", O_RDONLY) = 3
1262563929.468639 fstat(3, {st_mode=S_IFREG|0644, st_size=256316, ...}) = 0
1262563929.468715 mmap(NULL, 256316, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff41755d000
1262563929.468790 close(3) = 0
1262563929.468867 fstat(1, {st_mode=S_IFREG|0644, st_size=20971520000006, ...}) = 0
1262563929.468949 mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff417012000
1262563929.469048 write(1, "hey there, second appendage of o"..., 40) = 40
1262563929.469183 close(1) = 0
1262563929.469233 munmap(0x7ff417012000, 1048576) = 0
1262563929.469314 close(2) = 0
1262563929.469390 exit_group(0) = ?
root at s2-replay02:~#
#
# strace tail -n 2
#
root at s2-replay02:~# time strace -ttt tail -n 2 /data/storage/ReplayDataVolume001/biggest_yet_file 2> strace_tail_biggest_yet_file.out
^C
real 9m22.950s
user 0m44.300s
sys 1m24.470s
root at s2-replay02:~# ls -aFl strace_tail_biggest_yet_file.out
-rw-r--r-- 1 root root 187251511 2010-01-03 19:07 strace_tail_biggest_yet_file.out
root at s2-replay02:~# tail -f strace_tail_biggest_yet_file.out
1262567242.186934 lseek(3, 20962998501376, SEEK_SET) = 20962998501376
1262567242.187022 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192) = 8192
1262567242.187433 lseek(3, 20962998493184, SEEK_SET) = 20962998493184
1262567242.187524 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192) = 8192
1262567242.187928 lseek(3, 20962998484992, SEEK_SET) = 20962998484992
1262567242.188017 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192) = 8192
1262567242.188424 lseek(3, 20962998476800, SEEK_SET) = 20962998476800
1262567242.188513 read(3, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192) = 8192
1262567242.188946 lseek(3, 20962998468608, SEEK_SET) = 20962998468608
1262567242.189034 read(3, <unfinished ...>
^C
root at s2-replay02:~# tail -f strace_tail_biggest_yet_file.out
That was 10 minutes of seek time. I can let it run longer if you want.
-Robert
On Jan 2, 2010, at 4:33 PM, Tao Ma wrote:
> Hi Robert,
> Great thanks for your test. I haven't met with such a big volume ever before. ;)
>
> Robert Smith wrote:
>> Just thought I would let you guys know that creating a 20TB file was successful. I even appended data to the end of it. Any operations on the file are completely useless because they take way to long. A appended "hello" to the end of the file no problem, but tail -n 1 {filename} yielded nothing except a lot of disk read after 159minutes of waiting.
> I don't think 159 minutes is a good number.
> So could you please use "strace -tttt tail -n 1 biggest_yet_file"? Just want to find out which system call last such a long time.
> And also could you please run such command to find the disk layout of that file.
> echo 'stat biggest_yet_file'|debugfs.ocfs2 /dev/sdx
> sdx is your device of course.
>
> btw, you said appending has no problem, so how long does it take?
> And also please run "strace -ttt" to it.
>
> Regards,
> Tao
>
>> I don't really even know if this is good information or common knowledge.
>> dd bs=1000M count=20000 if=/dev/zero of=/data/storage/ReplayDataVolume001/biggest_yet_file
>> root at s2-replay02:/data/storage/ReplayDataVolume001# ls -aFl
>> total 1266312192
>> drwxr-xr-x 3 root root 3896 2009-12-31 23:31 ./
>> drwxr-xr-x 3 root root 88 2010-01-01 11:02 ../
>> -rw-r--r-- 1 root root 1048576000 2009-12-31 23:23 big_file
>> -rw-r--r-- 1 root root 10485760000 2009-12-31 23:24 bigger_file
>> -rw-r--r-- 1 root root 104857600000 2009-12-31 23:28 biggest_file
>> -rw-r--r-- 1 root root 20971520000006 2010-01-01 11:03 biggest_yet_file
>> drwxr-xr-x 2 root root 3896 2009-12-31 11:53 lost+found/
>> root at s2-replay02:/data/storage/ReplayDataVolume001#
>> -Robert
>> On Jan 1, 2010, at 5:08 AM, Joel Becker wrote:
>>> On Fri, Jan 01, 2010 at 04:36:02AM +0900, Robert Smith wrote:
>>>> Oh, I found it at line #2163 of fs/ocfs2/super.c.
>>>>
>>>> I imagine that something as simple as the following would work, but perhaps I'll wait for your feedback.
>>>>
>>>>
>>>> /*
>>>> if (ocfs2_clusters_to_blocks(osb->sb, le32_to_cpu(di->i_clusters) - 1)
>>>>> (u32)~0UL) {
>>>> mlog(ML_ERROR, "Volume might try to write to blocks beyond "
>>>> "what jbd can address in 32 bits.\n");
>>>> status = -EINVAL;
>>>> goto bail;
>>>> }
>>>> */
>>> That should work. The real solution will check based on the
>>> journal flags. Be warned, there be tygers in here.
>>>
>>> Joel
>>>
>>> --
>>>
>>> "But all my words come back to me
>>> In shades of mediocrity.
>>> Like emptiness in harmony
>>> I need someone to comfort me."
>>>
>>> Joel Becker
>>> Principal Software Developer
>>> Oracle
>>> E-mail: joel.becker at oracle.com
>>> Phone: (650) 506-8127
>> _______________________________________________
>> Ocfs2-devel mailing list
>> Ocfs2-devel at oss.oracle.com
>> http://oss.oracle.com/mailman/listinfo/ocfs2-devel
More information about the Ocfs2-devel
mailing list