trying out dtrace

One of the things we talked about at Oracle Openworld, last week, was Dtrace for Oracle Linux. There's not been much information on it yet so I wanted to write up what you need to do to give it a go.
We released a preview of Dtrace and we will provide updates as we include new features on an ongoing basis. The biggest project to implement is userspace tracing and many side projects to include and enhance various Dtrace providers in kernel. So it's not all there yet but it is going to continuously be enhanced with new updates.
Dtrace is made available to Oracle Linux support subscribers and currently requires you to do the following :
- have your system registered with the Unbreakable Linux Network (ULN) (or a local yum repo that mirrors the channels on ULN)
- run Oracle Linux 6, of course with the Unbreakable Enterprise Kernel
- register your server with the Dtrace channel (ol6_x86_64_Dtrace_BETA)
- install the updated version of 2.6.32 and the dtrace kernel module and userspace programs
- run it

Here are some of the detailed steps :
(1) register your server with ULN, run the uln_register command on your server as the root user. It requires you to enter your single sign-on ID, password and Oracle Linux support ID (CSI).
(2) once your system(s) is registered, login to the ULN website, click on Systems and find the server(s) you just registered
(3) Click on the server and go to Manage Subscriptions
(4) add the Dtrace for Oracle Linux 6 channel to your server and click Save
(5) back on the server, double check if everything worked by typing yum repolist it should now also show ol6_x86_64_Dtrace_BETA
(6) install the required packages :
... (a) yum install dtrace-modules
... (b) yum install dtrace-utils
... (c) yum install kernel-uek-2.6.32-201.0.4.el6uek
You have to install that specific kernel from the dtrace ULN channel because it is the kernel with the instrumentation.
(7) reboot into the new kernel
(8) load the kernel modules modprobe dtrace and modprobe systrace

Now you are ready to start playing with dtrace. We will publish a number of useful dtrace scripts to get you going. I have one quick example. When running an Oracle Database on my machine, I want to find out which files the oracle executable opens on the system. This simple command does it for me :
dtrace -n 'syscall::open*:entry/execname == "oracle"/{ printf("%s %s", execname, copyinstr(arg0)); }'

Any syscall that's open* executed by "oracle" will print the first argument being the path in this case.

A lot more to come but for the existing customers that want to start taking a look at it, check it out. The 2.6.32-201.0.4 kernel source is available on ULN, the dtrace kernel module source is under the CDDL license and is also available on ULN. The userspace tools are as is. Dtrace for our UEK2 Beta kernel is not there yet but will come in an update of the UEK2 beta kernel. Another thing we are evaluating is the use of the ksplice technology to "splice in" the probes/providers at runtime. So you would be able to run a kernel without the extra code, if you want to enable dtrace, you first apply the ksplice dtrace update, run dtrace and after the fact unload it. The word evaluate is key here, however :).


a little sample :

# dtrace -n 'syscall::open*:entry/execname == "oracle"/{ printf("%s %s", execname, copyinstr(arg0)); }'
dtrace: description 'syscall::open*:entry' matched 2 probes
CPU     ID                    FUNCTION:NAME
  2      8                       open:entry oracle /proc/2672/stat
  2      8                       open:entry oracle /proc/2674/stat
  2      8                       open:entry oracle /proc/2678/stat
  2      8                       open:entry oracle /proc/2682/stat
  2      8                       open:entry oracle /proc/2686/stat
  2      8                       open:entry oracle /proc/2688/stat
  2      8                       open:entry oracle /proc/2690/stat
  2      8                       open:entry oracle /proc/2692/stat
  2      8                       open:entry oracle /proc/2694/stat
  3      8                       open:entry oracle /proc/loadavg
  1      8                       open:entry oracle /u01/app/oracle/oradata/XE/system.dbf