udev for Security Conscious RAC Sysadmins

I’ve recently installed Oracle RAC on both RedHat 4 and Suse Enterprise 9. The most recent installation reminded me of an issue which has come up more than once: persistance of raw device permissions. If you remember installing Oracle on SLES8, RHEL2 and RHEL3, then you remember that you would just chmod the /dev files and that was it. But with RHEL4 and SLES9 (and all newer releases) everything breaks when you reboot! It doesn’t take long to figure out that the /dev file permissions were reset to their defaults.

The root cause is a change that was introduced several years ago in the 2.6 Linux kernel called udev. Unlike the 2.4 Linux system, where the device nodes in the /dev directory were a static set of files, udev in 2.6 dynamically creates files for the devices that are actually present on a system. It was also designed from the ground up to support persistent device naming. Naturally udev is nothing new to Linux admins; but if you’re not familiar with it and are interested in a good overview then check out Greg Kroah-Hartman’s talk and paper from the 2003 Linux Symposium.

What does this mean to DBAs installing Oracle RAC on Linux? What is the best and most secure way to get the permissions right for Oracle on your system?

The Approach

The most common approach is to add a few lines to the local startup script which will chmod the devices at boot-time. On RHEL, /etc/rc.d/rc.local gets run last (after all the startup scripts) and on SLES, /etc/rc.d/boot.local gets run after all the boot scripts (including the udev setup) but before entering a runlevel. Many people use these files just like “autoexec.bat” from the DOS days to start services without having to write System-V compatible startup scripts. In reality that’s a lazy approach: there is a much better way to do it.

The best way to assign permissions to udev devices is by using udev itself. A scheme for setting default permissions is already built into the carefully designed udev model. Whenever the machine boots or a new device is added, the kernel sends a message to a hotplug script (usually the default /sbin/hotplug but configurable through /proc/sys/kernel/hotplug). This script, among other things, passes the information to udev. First, udev reads its RULES configuration file(s) to determine what filename it should create under /dev for the device. Second, udev checks its PERMISSIONS configuration file(s) to determine the owner, group, and permissions for that device file. Setting the proper boot-time permissions to your raw devices for Oracle is no more complicated than adding a few lines to the right configuration file.

The Landing

First of all, udev is only available in distributions based on a 2.6 or newer kernel – that means RHEL 4+ and SLES 9+. RedHat and Suse take slightly different approaches to udev configuration; we’ll look at both of them.

SLES contains a single file which holds all of the RULES and a single file which holds all of the PERMISSIONS. The filenames are set in /etc/udev/udev.conf and by default /etc/udev/udev.permissions is the PERMISSIONS file. Entries at the top of this file take precedence over those later in the file. Toward the bottom of the file you will see these two lines which establish the default permissions for raw devices:

rawctl*:root:disk:660
raw/raw*:root:disk:660

In the interest of security you should not change the defaults, but instead you should add a line at the top of the file for each raw device with properly restrictive permissions. For example:

# 26-Feb-06 : jschneider : raw device default perms for oracle; 1/2 OCR, 3-5 Vote/Quorum
raw/raw1:root:oinstall:640
raw/raw2:root:oinstall:640
raw/raw3:oracle:oinstall:644
raw/raw4:oracle:oinstall:644
raw/raw5:oracle:oinstall:644

One thing to watch out for on SLES: this file is part of the udev rpm; if you upgrade your system then you will need to make sure that your changes don’t get removed.

RHEL takes a slightly cleaner approach: it sets up a directory which can contain multiple configuration files. This means that you can put your configuration in your own file and not worry about interfering with a system upgrade. By default /etc/udev/permissions.d is the directory. You will see that the default raw device permissions in RedHat’s config file 50-udev.permissions are the same as Suse:

# raw devices
ram*:root:disk:0660
raw/*:root:disk:0660

But now you can create your own file to contain your Oracle updates:

[root@rh4lab15 ~]# cat <<EOF >/etc/udev/permissions.d/10-oracle.permissions
> # 26-Feb-06 : jschneider : raw device default perms for oracle; 1/2 OCR, 3-5 Vote/Quorum
> raw/raw1:root:oinstall:640
> raw/raw2:root:oinstall:640
> raw/raw3:oracle:oinstall:644
> raw/raw4:oracle:oinstall:644
> raw/raw5:oracle:oinstall:644
> EOF

Filenames from the PERMISSIONS directory that end with “.permissions” will be read in lexical order and the first matching entry will be used.

So what are the properly restrictive permissions? The RAC FAQ in metalink note 220970.1 claims that the perms are changed during the installation process… but it lists the exact same perms before and after install. Pre-install the permissions should match the install guide. The OCR should be owned by root:oinstall with permissions 640. The install guide gives two different sets of permissions for the voting disks in two places; you should create them according to the first set as [crs user]:oinstall with permissions 644. The crs user can be obtained be running /etc/init.d/init.cssd user.

As one final note, in 10.2 you no longer need to create raw devices for ASM – but of course you’ll still need to use this same method to make the /dev/sd* files owned by oracle:dba for ASM to read them.

That’s it! Harnessing udev is a cleaner, more secure and less error-prone way to set default boot-time permissions on RHEL4+ and SLES9+.