<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style>body { line-height: 1.5; }blockquote { margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em; }p { margin-top: 0px; margin-bottom: 0px; }div.foxdiv20170527135032722573 { }body { font-size: 10.5pt; font-family: 'Times New Roman'; color: rgb(0, 0, 0); line-height: 1.5; }</style>
</head>
<body>
<div><span style="font-family: ''; font-size: 10.5pt; line-height: 1.5; background-color: window;">--- a/aops.c 2017-05-27 01:23:35.591274026 -0400 </span></div>
<div><span style="font-family: "" timesnewroman??;font-size:14px;color:rgb(0,0,0);background-color:rgba(0,font-weight:normal;font-style:normal;text-decoration:none;?="">+++ b/aops.c 2017-05-27 01:29:44.743285821 -0400
<br>
@@ -2396,6 +2396,35 @@ <br>
return 0; <br>
} </span></div>
<div><span style="font-family: "" timesnewroman??;font-size:14px;color:rgb(0,0,0);background-color:rgba(0,font-weight:normal;font-style:normal;text-decoration:none;?=""><br>
+/* <br>
+ * TODO: Make this into a generic get_blocks function. <br>
+ * <br>
+ * In ocfs2, ip_alloc_sem is used to protect allocation changes on the node. <br>
+ * In direct IO, we add ip_alloc_sem to protect date consistent between <br>
+ * direct-io and ocfs2_truncate_file race (buffer io use ip_alloc_sem <br>
+ * already). Although inode->i_mutex lock is used to avoid concurrency of <br>
+ * above situation, i think ip_alloc_sem is still needed because protect <br>
+ * allocation changes is significant. <br>
+ * <br>
+ * This function is called directly from get_more_blocks in direct-io.c. <br>
+ * <br>
+ * called like this: dio->get_blocks(dio->inode, fs_startblk, <br>
+ * fs_count, map_bh, dio->rw == READ); <br>
+ */ <br>
+static int ocfs2_dio_read_get_block(struct inode *inode, sector_t iblock, <br>
+ struct buffer_head *bh_result, int create) <br>
+{ <br>
+ struct ocfs2_inode_info *oi = OCFS2_I(inode); <br>
+ int ret = 0; <br>
+ <br>
+ down_read(&oi->ip_alloc_sem); <br>
+ /* This is the fast path for direct-io reading. */ <br>
+ ret = ocfs2_get_block(inode, iblock, bh_result, create); <br>
+ up_read(&oi->ip_alloc_sem); <br>
+ <br>
+ return ret; <br>
+} <br>
+ <br>
static ssize_t ocfs2_direct_IO(struct kiocb *iocb, struct iov_iter *iter) <br>
{ <br>
struct file *file = iocb->ki_filp; <br>
@@ -2416,7 +2445,7 @@ <br>
return 0; <br>
<br>
if (iov_iter_rw(iter) == READ) <br>
- get_block = ocfs2_get_block; <br>
+ get_block = ocfs2_dio_read_get_block; <br>
else <br>
get_block = ocfs2_dio_get_block; <br>
</span><span></span></div>
<div><span timesnewroman??;font-size:14px;color:rgb(0,0,0);background-color:rgba(0,font-weight:normal;font-style:normal;text-decoration:none;?=""><font face="\Times New Roman\"><br>
</font></span></div>
<div><span timesnewroman??;font-size:14px;color:rgb(0,0,0);background-color:rgba(0,font-weight:normal;font-style:normal;text-decoration:none;?=""><span style="line-height: normal;"><font face="\Times New Roman\">comments and questions are, as always, welcome.
Thanks </font></span></span></div>
<div><span style="font-family: "" timesnewroman??;font-size:14px;color:rgb(0,0,0);background-color:rgba(0,font-weight:normal;font-style:normal;text-decoration:none;?=""><span style="font-size: 16px; line-height: normal;"><br>
</span></span></div>
<hr style="width: 210px; height: 1px;" color="#b5c4df" size="1" align="left">
<div><span>
<div style="margin: 10px; font-size: 10pt;">
<div style="font-size: 16px;"><font face="verdana" color="#000080"><i>All the best wishes for you.</i></font></div>
<div style="font-size: 16px;"><i style="line-height: 1.5; font-family: verdana; background-color: window;"><font color="#000080">zhangguanghui</font></i></div>
</div>
</span></div>
<blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;">
<div> </div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div style="PADDING-RIGHT: 8px; PADDING-LEFT: 8px; FONT-SIZE: 12px;FONT-FAMILY:tahoma;COLOR:#000000; BACKGROUND: #efefef; PADDING-BOTTOM: 8px; PADDING-TOP: 8px">
<div><b>From:</b> <a href="mailto:zhang.guanghui@h3c.com">zhangguanghui 10102 (Cloud)</a></div>
<div><b>Date:</b> 2017-05-26 16:21</div>
<div><b>To:</b> <a href="mailto:ocfs2-devel@oss.oracle.com">ocfs2-devel@oss.oracle.com</a></div>
<div><b>CC:</b> <a href="mailto:ryan.ding@oracle.com">ryan.ding</a>; <a href="mailto:akpm@linux-foundation.org">
Andrew Morton</a>; <a href="mailto:wangww631@huawei.com">wangww631</a>; <a href="mailto:jlbec@evilplan.org">
Joel Becker</a>; <a href="mailto:mfasheh@suse.com">Mark Fasheh</a></div>
<div><b>Subject:</b> Re: ocfs2: fix sparse file & data ordering issue in direct io. review</div>
</div>
</div>
<div>
<div class="FoxDiv20170527135032722573">
<div><span></span></div>
<blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;">
<div class="FoxDiv20170526162001675386">
<blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;">
<div class="FoxDiv20170526114444302913">
<div>
<blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;">
<div class="FoxDiv20170422163954881342">
<p style="font-size: 16px; margin-top: 0px; margin-bottom: 0px;"><font face="\Times New Roman\" size="2" style="font-size: 16px;">Hi </font></p>
<p style="font-size: 16px; margin-top: 0px; margin-bottom: 0px;"><font face="\Times New Roman\" size="2" style="font-size: 16px;">This patch r<span style="line-height: 24px; widows: 1;">eplace that </span>function ocfs2_direct_IO_get_blocks with </font></p>
<p style="font-size: 16px; margin-top: 0px; margin-bottom: 0px;"><font face="\Times New Roman\" size="2" style="font-size: 16px;"><span style="line-height: 24px; widows: 1;">this function </span>ocfs2_get_blocks in ocfs2_direct_IO, </font><span style="line-height: 1.5; background-color: window;">and
remove the ip_alloc_sem.</span></p>
<p style="font-size: 16px; margin-top: 0px; margin-bottom: 0px;"><font face="\Times New Roman\" size="2" style="font-size: 16px;">but i think ip_alloc_sem is still needed because protect </font><span style="line-height: 19px; background-color: window;">allocation
changes is very correct. </span></p>
<p style="margin-top: 0px; margin-bottom: 0px;"><span style="line-height: 19px; background-color: rgb(255, 255, 255);"><font face="\Times New Roman\" size="3">Now, BUG_ON hav</font></span><span style="font-size: 16px; line-height: 19px; background-color: window;">e
been tiggered </span><span style="font-size: 16px; line-height: 1.5; background-color: window;"> in the process of testing direct-io. </span></p>
</div>
</blockquote>
<blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;">
<div class="FoxDiv20170422163954881342" style="font-size: 16px;"><span style="line-height: normal;"><font face="\Times New Roman\" size="2" style="font-size: 16px;">Comments and questions are, as always, welcome. Thanks </font></span></div>
<div class="FoxDiv20170422163954881342" style="font-size: 16px;"><span style="line-height: normal;"><font face="\Times New Roman\" size="2" style="font-size: 16px;"><br>
</font></span></div>
<div class="FoxDiv20170422163954881342" style="font-size: 16px;">
<p style="font-size: 14px; line-height: 21px; margin-top: 0px; margin-bottom: 0px;">
<span style="line-height: normal;"><font face="\Times New Roman\" size="3">As wangww631 described</font></span></p>
<p style="margin-top: 0px; margin-bottom: 0px;"><span style="line-height: 19px;">In ocfs2, ip_alloc_sem is used to protect allocation changes on the node.</span><br style="line-height: 19px;">
<span style="line-height: 19px;">In direct IO, we add ip_alloc_sem to protect date consistent between</span><br style="line-height: 19px;">
<span style="line-height: 19px;">direct-io and ocfs2_truncate_file race (buffer io use ip_alloc_sem</span><br style="line-height: 19px;">
<span style="line-height: 19px;">already). Although inode->i_mutex lock is used to avoid concurrency of</span><br style="line-height: 19px;">
<span style="line-height: 19px;">above situation, i think ip_alloc_sem is still needed because protect</span><br style="line-height: 19px;">
<span style="line-height: 19px;">allocation changes is significant.</span><br style="line-height: 19px;">
<br style="line-height: 19px;">
<span style="line-height: 19px;">Other filesystem like ext4 also uses rw_semaphore to protect data</span><br style="line-height: 19px;">
<span style="line-height: 19px;">consistent between get_block-vs-truncate race by other means, So</span><br style="line-height: 19px;">
<span style="line-height: 19px;">ip_alloc_sem in ocfs2 direct io is needed.</span></p>
<p style="margin-top: 0px; margin-bottom: 0px;"><span style="line-height: 19px;"><br>
</span></p>
</div>
<div class="FoxDiv20170422163954881342"><span style="line-height: normal;"><font face="\Times New Roman\" size="2"><br>
</font></span></div>
</blockquote>
</div>
<div><span style="font-size: small; line-height: 19px;">Date: Fri, 11 Sep 2015 16:19:18 +0800</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">From: Ryan Ding <ryan.ding@oracle.com></span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">Subject: [Ocfs2-devel] [PATCH 7/8] ocfs2: fix sparse file & data</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> ordering issue in direct io.</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">To: ocfs2-devel@oss.oracle.com</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">Cc: mfasheh@suse.de</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">Message-ID: <1441959559-29947-8-git-send-email-ryan.ding@oracle.com></span><br style="font-size: small; line-height: 19px;">
<br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">There are mainly 3 issue in the direct io code path after commit 24c40b329e03 ("ocfs2: implement ocfs2_direct_IO_write"):</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> * Do not support sparse file.</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> * Do not support data ordering. eg: when write to a file hole, it will alloc</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> extent first. If system crashed before io finished, data will corrupt.</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> * Potential risk when doing aio+dio. The -EIOCBQUEUED return value is likely</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> to be ignored by ocfs2_direct_IO_write().</span><br style="font-size: small; line-height: 19px;">
<br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">To resolve above problems, re-design direct io code with following ideas:</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> * Use buffer io to fill in holes. And this will make better performance also.</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> * Clear unwritten after direct write finished. So we can make sure meta data</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> changes after data write to disk. (Unwritten extent is invisible to user,</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> from user's view, meta data is not changed when allocate an unwritten</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> extent.)</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;"> * Clear ocfs2_direct_IO_write(). Do all ending work in end_io.</span><br style="font-size: small; line-height: 19px;">
<br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">This patch has passed fs,dio,ltp-aiodio.part1,ltp-aiodio.part2,ltp-aiodio.part4</span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">test cases of ltp.</span><br style="font-size: small; line-height: 19px;">
<br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">Signed-off-by: Ryan Ding <ryan.ding@oracle.com></span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">Reviewed-by: Junxiao Bi <junxiao.bi@oracle.com></span><br style="font-size: small; line-height: 19px;">
<span style="font-size: small; line-height: 19px;">cc: Joseph Qi <joseph.qi@huawei.com></span><br style="font-size: small; line-height: 19px;">
<br>
</div>
<hr style="width: 210px; height: 1px;" color="#b5c4df" size="1" align="left">
<div><span>
<div style="margin: 10px; font-size: 10pt;">
<div style="font-size: 16px;"><font face="verdana" color="#000080"><i>All the best wishes for you.</i></font></div>
<div style="font-size: 16px;"><i style="line-height: 1.5; font-family: verdana; background-color: window;"><font color="#000080">zhangguanghui</font></i></div>
</div>
</span></div>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</blockquote>
<span style="font-size:7.5pt;font-family:华文细黑;color:gray"><span lang="EN-US">-------------------------------------------------------------------------------------------------------------------------------------<br>
</span>本邮件及其附件含有新华三技术有限公司的保密信息，仅限于发送给上面地址中列出<span lang="EN-US"><br>
</span>的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、<span lang="EN-US"><br>
</span>或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本<span lang="EN-US"><br>
</span>邮件！<span lang="EN-US"><br>
</span></span><span lang="EN-US" style="font-size:7.5pt;font-family:"Arial","sans-serif";color:gray">This e-mail and its attachments contain confidential information from New H3C, which is
<br>
intended only for the person or entity whose address is listed above. Any use of the
<br>
information contained herein in any way (including, but not limited to, total or partial
<br>
disclosure, reproduction, or dissemination) by persons other than the intended <br>
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
<br>
by phone or email immediately and delete it!</span>
</body>
</html>