[Btrfs-devel] 3 thoughts about important and outstanding features
chris.mason at oracle.com
Mon Jan 21 08:46:16 PST 2008
On Monday 21 January 2008, myLC at gmx.net wrote:
> 'lo there, =)
> having a DVB-S receiver running Linux (PPC) I found myself
> wondering how to delete data from the middle of a large file
> (stripping a recording of ads, for example - or messing
> around in a virtual disk file, etc.). Currently, the common
> way of doing this seems to be by copying the file (leaving a
> part behind) and then deleting the original. Of course, on a
> large file (say 12 GB or more) this can take an eternity;
> also you can run into trouble if the filesystem is nearly
> Is it me, or doesn't that make any sense?
> Having a block-oriented filesystem, operations like this
> should only take an instance.
> So basically I'm looking for functions to:
> - insert a chunk into a file
> - delete a chunk from a file
> - move a chunk from one file into another
Like most filesystems, btrfs stores metadata for extents keyed by the offset
into the file. So, in order to insert one byte in the middle of the file,
you have to change all of the extent pointers after that byte in the file by
one. This can be slow, although it is certainly faster than copying all the
It is very hard to provide slicing operations in place due to races with
truncate. I'm not eager to dive into all of those corner cases.
But, what Btrfs can do is provide a few building blocks that applications
could use to do this much much faster than it is done today. It relates to
the cow one file ioctl as well. On disk, btrfs file data extent pointers
store 4 numbers:
[ start of the extent on disk, length of the extent on disk, offset into the
extent, length ]
This is currently used when doing copy on write operations in the middle of an
extent. A new extent is created with the modified data, and pointers are
setup to the old extent for the bytes surrounding the new data.
The Btrfs disk format allows one file to reference extents made by another
file. So, we could easily write an ioctl that created a new file referencing
all of the extents of an existing file. That ioctl could also provide the
ability to reference only parts of an existing file.
Punching holes in an existing file is something I think XFS can do now via an
ioctl. Shifting bytes around is a different (and much more complex) story
unless you're willing to shift them to a new file.
> There is a HUGE difference in performance when it comes to
> harddrives and their outer versus their inner tracks.
> A "self-optimizing" filesystem could make use of this by
> allowing the user/administrator to specify preferences about
> where to put certain files.
Different disks do different things (including inverting where block 0 lies).
Btrfs will have flexible allocation policies that help admins make their own
decisions about these things, but inner/outer optimizations probably won't be
a focus of automatic tuning.
More information about the Btrfs-devel