[Ocfs2-tools-devel] [RFC] changing default block device detection method and more

[Fabio Massimo Di Nitto]

Joel Becker wrote:
> [Sorry I took so long to respond.  Out of town and all.]
> 

No problem. Nobody is rushing after us except my wife about to deliver
fabbione v3.0 :)

> On Tue, Nov 20, 2007 at 08:05:24AM +0100, Fabio Massimo Di Nitto wrote:
>> so our tools in general uses /proc/partitions to detect available block devices
>> on the system. While this is quick and somewhat a not expensive operation, it
>> has a major limitation of not discovering devices that are not exported to
>> /proc/partitions. This generally fits well most of the users out there but of
>> course not me :)
> 
> 	We should, of course, detect available devices.  If they are
> not in /proc/partitions, we need to fix.

agreed.

> 	Do we know what devices don't appear there?

nbd is one of them. I don't have a complete list.

>  You mentioned that
> some folks are saying they "won't add them", do you have the email
> thread to reference?  I'd like to read up on it.

That's not exactly what I said. I mentioned that our udev maintainer said that
/proc/partitions will be phased out at some point, since /sys/ has the same
informations. I will ask him for a reference.

> 	The following is a bunch of observations, I don't think I know
> what approach is best, other than that "/proc/partitions is missing some
> items, we need to do something" is a driving statement.
> 
>> One of them could be to scan directly /dev for block devices and that is the
>> same approach I used in another piece of software that was using
>> /proc/partitions. The general experience is that it is still very fast to stat
>> the bits we need, we catch all the devices around and sometimes a bit too many
>> (for example all loop devices are also shown).
> 
> 	Well, we want to catch *active* loop devices.
> 	The basic concern is what happens for expensive scans (many
> non-active devices in /dev).  Some people keep a static /dev (turning
> off udev).  That scan will be expensive, but perhaps we say "those
> people can be ignored", as udev is the common case.  Debian (I don't know
> about Ubuntu) boots with a static /dev, moving it to /dev/.static when
> udev starts up.  On my laptop:
> 
>   # find /dev  -ls | grep -v '\/dev\/.static' | grep 'brw' | wc -l
>   32
>   # find /dev  -ls  | grep 'brw' | wc -l
>   4566

note that I already applied a "udev filter" in /dev scanning to skip .static.

+      if (strcmp(namelist[n]->d_name, ".") &&
+         strcmp(namelist[n]->d_name, "..") &&
+         strcmp(namelist[n]->d_name, ".static") && <----
+         strcmp(namelist[n]->d_name, ".udev"))



>> Another solution could be to walk /sys/block that indeed exports only available
>> block devices in the system. The problem is that /sys/block does not have any
>> knowledge on how /dev is populated. So once we get to the major/minor
>> information of each single device, we still have the issue to go around /dev to
>> stat all block devices to match them with /sys/block entries.
> 
> 	Actually, from that direction, you can just mknod a temporary
> name with the device number (if you are just checking something).  But
> yes, you'd have to match up as in the "sr0 <-> scd0" case above.
> 	In the reverse, if you scan /dev and then need to check /sys for
> media type (it's really unsafe to blindly open tape or cd devices
> according to Alan and Al, which is why we avoid them in the IDE case),
> you have to do the search in reverse.
> 	I don't know which direction is best, and I don't have a
> solution, but I figured food for thought.
> 

I think I have a solution that I believe is the best compromise.

We start scanning /sys/block for disks (this will solve directly the filtering
problem at the source by skipping tapes, cd-rom's no matter how they are
connected to the system or called in /dev).

We gather the major/minor for each device and by simply doing a stat in /dev
(and filtering .static as in the previous code) we can associate maj/min to the
name in /dev. If we find no devices we can either wait a few secs or just skip
it (thought luck).

It is slightly more expensive because there might be tons of entries in /sys but
none of them will involve a disk access and we will do a real check only on
devices we care. Stat of /dev is cheap as an ls (you can't make it cheaper than
that I believe).

The code will be a bit more complex as we need to work on 2 lists rather than
one, but I doubt this is an issue at all.

How does it sound?

Fabio

-- 
I'm going to make him an offer he can't refuse.