[Btrfs-devel] Re: list of things to do

[Josef Bacik]

On Thu, Jul 05, 2007 at 09:07:21AM -0400, Chris Mason wrote:
> On Tue, 3 Jul 2007 21:00:05 -0400
> Josef Bacik <jwhiter at redhat.com> wrote:
> 
> > > Here are a few ideas of things that won't conflict too much with my
> > > current enospc work.  But, if you have other ideas or bits you would
> > > like to improve, please feel free to suggest them, I'm happy to get
> > > all the help I can.
> > > 
> > > 1) On fsync, ext3 and reiserfs have code to let more writers try and
> > > come into the current transaction before a commit is forced (this is
> > > called let_transaction_grow in reiserfs).  btrfs needs something
> > > similar.  It's fairly simple, you look at the count of procs
> > > currently in the transaction, and if it is non-zero you sleep for a
> > > bit before forcing the commit.
> > > 
> > > It's important to sleep without any locks held and without being in
> > > the transaction.  Once coded, you can benchmark with Ric Wheeler's
> > > fs_mark:
> > > 
> > > http://developer.osdl.org/dev/doubt/fs_mark/index.html
> > > 
> > > If you're comparing btrfs to ext3 on a sata or ide drive, make sure
> > > to mount ext3 with -o barrier=1
> > >
> > 
> > Ok I started looking into this, and I figured the best place to do
> > this would be in btrfs_commit_transaction, but it appears it already
> > does something like this, where we check
> > trans->transaction->num_writers > 1, and if it is we declare a
> > watiqueue  and sleep for a bit, and keep doing this until we have
> > only 1 or 0 writers.  So it appears you've already done this.  Now
> > I'm getting ready for my wedding so its entirely possible that I've
> > lost my mind, and if so please let me know :).
> 
> Congrats on the wedding ;)  I'm sure it will be far more fun than
> filesystem hacking.  btrfs_commit_transaction does check for more than
> one writer, but reiserfs (and I think ext3) also has a check to see if
> anyone did work while we were sleeping.  This is something of a gray
> area because we're increasing fsync latency slightly in hopes of
> increasing overall system throughput.  It's a tradeoff and you don't
> always make the right choice.
> 
> So, I think the case we're missing is the writer count has gone down to
> one (the proc running btrfs_commit_transaction is the only writer), but
> while we were sleeping other procs came in and did work.  If we haven't
> been waiting too horribly long, we want to sleep again and give more
> procs the chance to do something useful.  And, we probably want to
> always sleep at least once.
> 
> (I'm being intentionally vague in the description because I'm not sure
> what will work best on btrfs ;)
>

Ahh ok, that shouldn't be too hard do, I'll see if I can crank this out today,
thanks much for the clarificaiton.
 
> > I will look into these other things as I get time, but you probably
> > won't hear anything out of me for about a week.  If you manage to do
> > these yourself while I'm out let me know and I'll start tackling
> > things on the TODO list.  Thanks much,
> 
> Enjoy the wedding, best wishes to you and your family.
>

Thank you :),

Josef