I too forgot to mention that my accommodation at GUADEC was sponsored by the GNOME Foundation. Thanks guys!

#ignore yes

Today I came across this blog post of your design team. In context of the recent criticism you had to endure regarding upstream contributions I am disappointed that you have not bothered to ping anybody from the upstream freedesktop sound theme (for example yours truly) about this in advance. No, you went to cook your own soup. What really disappoints me is that we have asked multiple times for help and support and contributions for the sound theme, to only very little success, and I even asked some of the Canonical engineers about this topic and in particular regarding some clarifications of the licensing of the old Ubuntu sound theme. I am sorry, but if you had listened, or looked, or asked you would have been aware that we were looking for somebody to maintain this actively, upstream -- and because we didn't have the time to maintain this we only did the absolute minimum work necessary and we only maintain this ourselves because noone else wanted to.

It should be upstream first, downstream second.

I am sorry if I sound like an always complaining prick to you. But believe me, I am not saying this because I wouldn't like you or anything like that. I am just saying this because I believe you could do things oh so much better.

Please fix this. We want your contributions. Upstream.

I guess it's a bit beating a dead horse, but I had a good laugh today when I learned that I alone contributed more to GNOME than the entirety of Canonical, and only 800 additional commits seperating me from being more awesome than Nokia.

/me is amused

Here's a podcast interview with yours truly where I speak a little about PulseAudio and systemd. Seek to 64:43 for my lovely impetuous voice. There's also an interview with Owen just before mine.

#nocomments y

The Call for Papers for the Linux Plumbers Conference (LPC) in November in Cambridge, Massachusetts is ending soon, on July 19th 2010 (That's the upcoming monday!). It's a conference about the core infrastructure of Linux systems: the part of the system where userspace and the kernel interface. It's the only conference where the focus is specifically on getting together the kernel people who work on the userspace interfaces and the userspace people who have to deal with kernel interfaces. It's supposed to be a place where all the people doing infrastructure work sit down and talk, so that both parties understand better what the requirements and needs of the other are, and where we can work towards fixing the major problems we currently have with our lower-level infrastructure and APIs.

The two previous LPCs were hugely successful (as reported on LWN on various occasions), and this time we hope to repeat that.

Like the previous years, I will be running the Audio conference track of LPC, this time together with Mark Brown. Audio infrastructure on Linux has been steadily improving the last years all over the place, but there's still a lot to do. Join us at the LPC to discuss the next steps and help improving Linux audio further! If you are doing audio infrastructure work on Linux, make sure to attend and submit a paper!

Sign up soon! Send in your paper quickly! Only three days left to the end of the CFP!

(I am also planning to do a presentation there about systemd, together with Kay. Make sure to attend if you are interested in that topic.)

See you in Boston!

I forgot to mention another central problem in my blog story about file locking on Linux:

Different machines have access to different features of the same file system. Here's an example: let's say you have two machines in your home LAN. You want them to share their $HOME directory, so that you (or your family) can use either machine and have access to all your (or their) data. So you export /home on one machine via NFS and mount it from the other machine.

So far so good. But what happens to file locking now? Programs on the first machine see a fully-featured ext3 or ext4 file system, where all kinds of locking works (even though the API might suck as mentioned in the earlier blog story). But what about the other machine? If you set up lockd properly then POSIX locking will work on both. If you didn't one machine can use POSIX locking properly, the other cannot. And it gets even worse: as mentioned recent NFS implementations on Linux transparently convert client-side BSD locking into POSIX locking on the server side. Now, if the same application uses BSD locking on both the client and the server side from two instances they will end up with two orthogonal locks and although both sides think they have properly acquired a lock (and they actually did) they will overwrite each other's data, because those two locks are independent. (And one wonders why the NFS developers implemented this brokenness nonetheless...).

This basically means that locking cannot be used unless it is verified that everyone accessing a file system can make use of the same file system feature set. If you use file locking on a file system you should do so only if you are sufficiently sure that nobody using a broken or weird NFS implementation might want to access and lock those files as well. And practically that is impossible. Even if fpathconf() was improved so that it could inform the caller whether it can successfully apply a file lock to a file, this would still not give any hint if the same is true for everybody else accessing the file. But that is essential when speaking of advisory (i.e. cooperative) file locking.

And no, this isn't easy to fix. So again, the recommendation: forget about file locking on Linux, it's nothing more than a useless toy.

Also read Jeremy Allison's (Samba) take on POSIX file locking. It's an interesting read.

It's amazing how far Linux has come without providing for proper file locking that works and is usable from userspace. A little overview why file locking is still in a very sad state:

To begin with, there's a plethora of APIs, and all of them are awful:

• POSIX File locking as available with fcntl(F_SET_LK): the POSIX locking API is the most portable one and in theory works across NFS. It can do byte-range locking. So much on the good side. On the bad side there's a lot more however: locks are bound to processes, not file descriptors. That means that this logic cannot be used in threaded environments unless combined with a process-local mutex. This is hard to get right, especially in libraries that do not know the environment they are run in, i.e. whether they are used in threaded environments or not. The worst part however is that POSIX locks are automatically released if a process calls close() on any (!) of its open file descriptors for that file. That means that when one part of a program locks a file and another by coincidence accesses it too for a short time, the first part's lock will be broken and it won't be notified about that. Modern software tends to load big frameworks (such as Gtk+ or Qt) into memory as well as arbitrary modules via mechanisms such as NSS, PAM, gvfs, GTK_MODULES, Apache modules, GStreamer modules where one module seldom can control what another module in the same process does or accesses. The effect of this is that POSIX locks are unusable in any non-trivial program where it cannot be ensured that a file that is locked is never accessed by any other part of the process at the same time. Example: a user managing daemon wants to write /etc/passwd and locks the file for that. At the same time in another thread (or from a stack frame further down) something calls getpwuid() which internally accesses /etc/passwd and causes the lock to be released, the first thread (or stack frame) not knowing that. Furthermore should two threads use the locking fcntl()s on the same file they will interfere with each other's locks and reset the locking ranges and flags of each other. On top of that locking cannot be used on any file that is publicly accessible (i.e. has the R bit set for groups/others, i.e. more access bits on than 0600), because that would otherwise effectively give arbitrary users a way to indefinitely block execution of any process (regardless of the UID it is running under) that wants to access and lock the file. This is generally not an acceptable security risk. Finally, while POSIX file locks are supposedly NFS-safe they not always really are as there are still many NFS implementations around where locking is not properly implemented, and NFS tends to be used in heterogenous networks. The biggest problem about this is that there is no way to properly detect whether file locking works on a specific NFS mount (or any mount) or not.
• The other API for POSIX file locks: lockf() is another API for the same mechanism and suffers by the same problems. One wonders why there are two APIs for the same messed up interface.
• BSD locking based on flock(). The semantics of this kind of locking are much nicer than for POSIX locking: locks are bound to file descriptors, not processes. This kind of locking can hence be used safely between threads and can even be inherited across fork() and exec(). Locks are only automatically broken on the close() call for the one file descriptor they were created with (or the last duplicate of it). On the other hand this kind of locking does not offer byte-range locking and suffers by the same security problems as POSIX locking, and works on even less cases on NFS than POSIX locking (i.e. on BSD and Linux < 2.6.12 they were NOPs returning success). And since BSD locking is not as portable as POSIX locking this is sometimes an unsafe choice. Some OSes even find it funny to make flock() and fcntl(F_SET_LK) control the same locks. Linux treats them independently -- except for the cases where it doesn't: on Linux NFS they are transparently converted to POSIX locks, too now. What a chaos!
• Mandatory locking is available too. It's based on the POSIX locking API but not portable in itself. It's dangerous business and should generally be avoided in cleanly written software.
• Traditional lock file based file locking. This is how things where done traditionally, based around known atomicity guarantees of certain basic file system operations. It's a cumbersome thing, and requires polling of the file system to get notifications when a lock is released. Also, On Linux NFS < 2.6.5 it doesn't work properly, since O_EXCL isn't atomic there. And of course the client cannot really know what the server is running, so again this brokeness is not detectable.

The Disappointing Summary

File locking on Linux is just broken. The broken semantics of POSIX locking show that the designers of this API apparently never have tried to actually use it in real software. It smells a lot like an interface that kernel people thought makes sense but in reality doesn't when you try to use it from userspace.

Here's a list of places where you shouldn't use file locking due to the problems shown above: If you want to lock a file in $HOME, forget about it as $HOME might be NFS and locks generally are not reliable there. The same applies to every other file system that might be shared across the network. If the file you want to lock is accessible to more than your own user (i.e. an access mode > 0700), forget about locking, it would allow others to block your application indefinitely. If your program is non-trivial or threaded or uses a framework such as Gtk+ or Qt or any of the module-based APIs such as NSS, PAM, ... forget about about POSIX locking. If you care about portability, don't use file locking.

Or to turn this around, the only case where it is kind of safe to use file locking is in trivial applications where portability is not key and by using BSD locking on a file system where you can rely that it is local and on files inaccessible to others. Of course, that doesn't leave much, except for private files in /tmp for trivial user applications.

Or in one sentence: in its current state Linux file locking is unusable.

And that is a shame.

Update: Check out the follow-up story on this topic.

When programming software that cooperates with software running on behalf of other users, other sessions or other computers it is often necessary to work with unique identifiers. These can be bound to various hardware and software objects as well as lifetimes. Often, when people look for such an ID to use they pick the wrong one because semantics and lifetime or the IDs are not clear. Here's a little incomprehensive list of IDs accessible on Linux and how you should or should not use them.

Hardware IDs

1. /sys/class/dmi/id/product_uuid: The main board product UUID, as set by the board manufacturer and encoded in the BIOS DMI information. It may be used to identify a mainboard and only the mainboard. It changes when the user replaces the main board. Also, often enough BIOS manufacturers write bogus serials into it. In addition, it is x86-specific. Access for unprivileged users is forbidden. Hence it is of little general use.
2. CPUID/EAX=3 CPU serial number: A CPU UUID, as set by the CPU manufacturer and encoded on the CPU chip. It may be used to identify a CPU and only a CPU. It changes when the user replaces the CPU. Also, most modern CPUs don't implement this feature anymore, and older computers tend to disable this option by default, controllable via a BIOS Setup option. In addition, it is x86-specific. Hence this too is of little general use.
3. /sys/class/net/*/address: One or more network MAC addresses, as set by the network adapter manufacturer and encoded on some network card EEPROM. It changes when the user replaces the network card. Since network cards are optional and there may be more than one the availability if this ID is not guaranteed and you might have more than one to choose from. On virtual machines the MAC addresses tend to be random. This too is hence of little general use.
4. /sys/bus/usb/devices/*/serial: Serial numbers of various USB devices, as encoded in the USB device EEPROM. Most devices don't have a serial number set, and if they have it is often bogus. If the user replaces his USB hardware or plugs it into another machine these IDs may change or appear in other machines. This hence too is of little use.

There are various other hardware IDs available, many of which you may discover via the ID_SERIAL udev property of various devices, such hard disks and similar. They all have in common that they are bound to specific (replacable) hardware, not universally available, often filled with bogus data and random in virtualized environments. Or in other words: don't use them, don't rely on them for identification, unless you really know what you are doing and in general they do not guarantee what you might hope they guarantee.

Software IDs

1. /proc/sys/kernel/random/boot_id: A random ID that is regenerated on each boot. As such it can be used to identify the local machine's current boot. It's universally available on any recent Linux kernel. It's a good and safe choice if you need to identify a specific boot on a specific booted kernel.
2. gethostname(), /proc/sys/kernel/hostname: A non-random ID configured by the administrator to identify a machine in the network. Often this is not set at all or is set to some default value such as localhost and not even unique in the local network. In addition it might change during runtime, for example because it changes based on updated DHCP information. As such it is almost entirely useless for anything but presentation to the user. It has very weak semantics and relies on correct configuration by the administrator. Don't use this to identify machines in a distributed environment. It won't work unless centrally administered, which makes it useless in a globalized, mobile world. It has no place in automatically generated filenames that shall be bound to specific hosts. Just don't use it, please. It's really not what many people think it is. gethostname() is standardized in POSIX and hence portable to other Unixes.
3. IP Addresses returned by SIOCGIFCONF or the respective Netlink APIs: These tend to be dynamically assigned and often enough only valid on local networks or even only the local links (i.e. 192.168.x.x style addresses, or even 169.254.x.x/IPv4LL). Unfortunately they hence have little use outside of networking.
4. gethostid(): Returns a supposedly unique 32-bit identifier for the current machine. The semantics of this is not clear. On most machines this simply returns a value based on a local IPv4 address. On others it is administrator controlled via the /etc/hostid file. Since the semantics of this ID are not clear and most often is just a value based on the IP address it is almost always the wrong choice to use. On top of that 32bit are not particularly a lot. On the other hand this is standardized in POSIX and hence portable to other Unixes. It's probably best to ignore this value and if people don't want to ignore it they should probably symlink /etc/hostid to /var/lib/dbus/machine-id or something similar.
5. /var/lib/dbus/machine-id: An ID identifying a specific Linux/Unix installation. It does not change if hardware is replaced. It is not unreliable in virtualized environments. This value has clear semantics and is considered part of the D-Bus API. It is supposedly globally unique and portable to all systems that have D-Bus. On Linux, it is universally available, given that almost all non-embedded and even a fair share of the embedded machines ship D-Bus now. This is the recommended way to identify a machine, possibly with a fallback to the host name to cover systems that still lack D-Bus. If your application links against libdbus, you may access this ID with dbus_get_local_machine_id(), if not you can read it directly from the file system.
6. /proc/self/sessionid: An ID identifying a specific Linux login session. This ID is maintained by the kernel and part of the auditing logic. It is uniquely assigned to each login session during a specific system boot, shared by each process of a session, even across su/sudo and cannot be changed by userspace. Unfortunately some distributions have so far failed to set things up properly for this to work (Hey, you, Ubuntu!), and this ID is always (uint32_t) -1 for them. But there's hope they get this fixed eventually. Nonetheless it is a good choice for a unique session identifier on the local machine and for the current boot. To make this ID globally unique it is best combined with /proc/sys/kernel/random/boot_id.
7. getuid(): An ID identifying a specific Unix/Linux user. This ID is usually automatically assigned when a user is created. It is not unique across machines and may be reassigned to a different user if the original user was deleted. As such it should be used only locally and with the limited validity in time in mind. To make this ID globally unique it is not sufficient to combine it with /var/lib/dbus/machine-id, because the same ID might be used for a different user that is created later with the same UID. Nonetheless this combination is often good enough. It is available on all POSIX systems.
8. ID_FS_UUID: an ID that identifies a specific file system in the udev tree. It is not always clear how these serials are generated but this tends to be available on almost all modern disk file systems. It is not available for NFS mounts or virtual file systems. Nonetheless this is often a good way to identify a file system, and in the case of the root directory even an installation. However due to the weakly defined generation semantics the D-Bus machine ID is generally preferrable.

Generating IDs

Linux offers a kernel interface to generate UUIDs on demand, by reading from /proc/sys/kernel/random/uuid. This is a very simple interface to generate UUIDs. That said, the logic behind UUIDs is unnecessarily complex and often it is a better choice to simply read 16 bytes or so from /dev/urandom.

Summary

And the gist of it all: Use /var/lib/dbus/machine-id! Use /proc/self/sessionid! Use /proc/sys/kernel/random/boot_id! Use getuid()! Use /dev/urandom! And forget about the rest, in particular the host name, or the hardware IDs such as DMI. And keep in mind that you may combine the aforementioned IDs in various ways to get different semantics and validity constraints.

On popular request, here are my (terse) slides from LinuxTag on systemd.

The upcoming week I'll do two talks at LinuxTag 2010 at the Berlin Fair Grounds. One of them was only added to the schedule today, about systemd. Systemd has never been presented in a public talk before, so make sure to attend this historic moment... ;-). Read about what has been written about systemd so far, so that you can ask the sharpest questions during my presentation.

My second talk might be about stuff a little less reported in the press, but still very interesting, about Surround Sound in Gnome.

See you at LinuxTag!