>
Linux, Oracle, Technical

ASMLIB Performance vs Udev

Posted by ⋅ ⋅ 8 Comments

Is asmlib obsolete on a modern Linux system? I’m still undecided but starting to lean toward “yes”.

Everybody knows that asmlib was very useful when it was first introduced with Oracle 10.1 to simplify a host of issues on Linux: direct async device access without raw devices, file permissions & ownership without custom code, and persistent device naming without devlabel.

But I’m now involved in setting some standards to be used across a large organization for Oracle 10.2 on RHEL5 and I’m wondering if there’s still a case for using asmlib. So I did a little trolling for info – which was suprisingly sparse. Had a hard time finding much, but after a lot of digging I think I’ve compiled a useful bit of information about benefits and drawbacks.

Benefits

It seems to me that the ASMLIB API was originally introduced to do more than just simplify file permissions – sounds like it was an alternative I/O API to the standard unix one, allowing ASM to access the underlying storage more efficiently and completely. I don’t think it’s just an “extra layer” – it’s an alternative code path to the std unix I/O libs. Like an ODM for block devices – and the idea was that there could be additional vendor implementations. And Oracle released an initial generic implementation on Linux under the GPL.

Some theoretical benefits of ASMLIB API:

  • always uses direct, async i/o
  • solves persistent device naming, even if underlying device moves across reboots
  • solves file permissions and ownership
  • reduced user mode to kernel mode context switches during I/O, possibly reducing CPU usage
  • reduced file handle usage
  • pass metadata such as I/O prioritization to storage device (don’t think this is implemented in the Linux version)

Drawbacks

However from what I can tell, the ASMLIB API didn’t quite catch on. Wim recommends it (as of Apr ’07) but Closson doesn’t really talk about it on his blog but has one disparaging off-hand reference about pitying people who use it. There have been no vendor implementations that I could find. (But I wonder what’s going on now with exadata and “iDB” which has I/O prioritization and predicate pre-filtering – could they have used the API here?) Wim’s post on Oracle Forums says that there isn’t “much of an io performance benefit” added by the linux ASMLIB implementation. And the OTN page hasn’t been updated for almost two years.

A RedHat consultant I’m working with recommended against using it and one of his reasons was that it introduces an unnecessary additional layer and dependency in the kernel. There have been a small number of past bugs in asmlib and it doesn’t seem to be frequently updated (once this year according to the changelogs)… but then it’s very simple code (mostly in a single small file).

Conclusions

After all of this, I’m not convinced that it’s worth installing ASMLIB if you’re comfortable with udev. On Linux, udev is definitely a better solution to handle persistent device naming and permissions and I don’t see enough benefits in asmlib to outweigh the very slight additional overhead.

So right now my preference would be to only install asmlib on old systems – Oracle 10.1 or Linux 2.4 – but to use udev and block devices for 10.2 and newer on 2.6 kernels. But the case isn’t closed and I’d love to hear from anyone who agrees or disagrees… thoughts?

References:

About Jeremy

Building and running reliable data platforms that scale and perform. about.me/jeremy_schneider

Discussion

8 thoughts on “ASMLIB Performance vs Udev”

  1. I’ve done the investigation and came to the same conclusion. It’s also part of my ASM presentation I’ve done on openworld.

    There are more things you have to be aware of when using asmlib:
    -the asmlib library is placed in /opt/oracle/extapi/32/asm/orcl/1/libasm.so. No problem, but if you are not aware you have a critical library placed there, it’s possible that someone (a sysadmin) cleanes this up.
    -asmlib adds a kernel dependency. No problem, but you have to be aware you have to upgrade the asmlib kernel package together with upgrading your kernel.
    -what sense does a seperate asmlib name make for a disk? it means you have a device name (which can differ), a ASM diskname, and a asmlib name. seems redundant to me. also, when udev is setup to give the disks for oracle the correct permissions (and the asm discovery string is setup right), ASM reads the diskheaders and recognises them, no matter if disk /dev/sdd1 was /dev/sdb1 before the reboot.

    Like

    Posted by Frits Hoogland | October 8, 2008, 7:46 am

  2. Hello,

    Like Frits, I have also presented on ASM. Though I have a more benign attitude to ASM, but then I found udev to be quite impenetrable to use.

    I have Oracle employees express the opinion the ASMLib enables global opening of devices, so 1 file handler per device which is shared amongst processes rather than each process having to do this.

    I really have to test this so you’d treat this with a large pinch of salt until there is more concrete evidence.

    Of course, whether reducing file handlers, and marginal improvement in context switches (thus reducing cpu) is really a benefit you will notice is a different question.

    jason.

    Like

    Posted by jason arneil | October 9, 2008, 4:15 am

  3. One small update: s/gorman/hall/.

    Tim Hall is widely known as “Tim…”. Though I think he may also be the original and only case of an illness called “Oracle Open World Syndrome (OOWS)” which seems to affect him each year he attends (at least two years running now).

    Thanks for the blog. Hope to see more like this one coming soon!

    Like

    Posted by Dan Norris | October 13, 2008, 1:24 pm

  4. Dan – thanks for stopping by! Fixed the link, thanks for pointing it out… I have it correct on my “Links” page but somehow I got the last name mixed up here.

    Fritz and Jason – thanks for the comments. I agree that udev is a bit difficult; it’s not exactly obvious and the documentation is very poor. But once you figure it out, it’s quite powerful and flexible.

    Like

    Posted by Jeremy | October 14, 2008, 10:05 am

  5. Wow, I didn’t figured out the possibility of using Udev with ASM before reading your post. That’s great.
    I’ll follow your blog with entusiasm!

    Like

    Posted by Ludovico Caldara | December 10, 2008, 2:55 am

  6. One recent update… I haven’t verified this, but:

    I heard that with regular block devices, after dropping or offline-ing a device it might still be help open by other rdbms processes besides the one that did the drop. I’m not sure if or how this might cause problems. (Do you need all the handles cleanly closed for OS operations like detaching the LUN or renaming the device?) I imagine that ASMLIB would not have this problem since each server process uses a single ASMLIB handle instead of direct device handles.

    Have yet to verify the details. :)

    Like

    Posted by Jeremy | March 4, 2009, 8:06 pm

  7. One more caveat. We’ve pushed ahead with using udev here. Every time a new hardware model is introduced we have to figure out new udev rules. (Multipath, SCSI, SSD, HP cciss hardware RAID…) ASMLIB probably would have been a bit easier in this regard. (I’m not certain off-hand, but I think ASMLIB works with any block device.)

    Basically my current thinking is that you should use udev if you have Linux guys around to take care of it, ASMLIB if the DBAs are going to be tending the device permissions. Guess it kinda relates to that whole DBA2.0 thing that was recently making the rounds.

    Like

    Posted by Jeremy | March 12, 2009, 12:24 pm

  8. Working on the 11Gr2 Grid (Linux x86_64), I had the same opinion about ASMlib and its obsolescence.

    But I was getting several issues with open files descriptors left after disks are dropped, making storage migrations complex or even impossible. Chasing every RBDMS process for unclosed fd is an issue.

    After testing ASMlib, the whole problem went away since no POSIX open()/close() calls are used at all.

    I’m forced to conclude that, even if block device name persistence or permissions can be handled by OS (udev), the file handle management via ASMlib do make storage migration easier.

    my2cents

    Like

    Posted by Jean-Francois | September 1, 2010, 10:51 am

Disclaimer

This is my personal website. The views expressed here are mine alone and may not reflect the views of my employer.

contact: 312-725-9249 or schneider @ ardentperf.com


https://about.me/jeremy_schneider

oaktableocmaceracattack

(a)

Enter your email address to receive notifications of new posts by email.

Join 68 other subscribers