Advanced GDB tips and tricks

http://www.openlogic.com/wazi/bid/336594/advanced-gdb-tips-and-tricks


The GNU Debugger (GDB) is one of the most popular debugging tools available on Linux and Unix-like systems. Learn the advanced debugging techniques in this article to improve your development process.
To create the examples here, I used GDB 7.6.1-ubuntu and GCC 4.8.1, and compiled the C code using the -ggdb option.

Conditional breakpoints

Breakpoints are an integral part of a debugger. They let you pause program execution to do things such as examining variable values. While you probably know how to use breakpoints, you can debug your code better and faster by using conditional breakpoints.
Suppose your code crashes within a loop that runs hundreds or thousands of times. It would be impractical to put a simple breakpoint anywhere in that loop to catch a problem on some unknown iteration. With a conditional breakpoint, however, you can pause your program only when some condition is met.
Let's see how it works with the code below, which produces a floating point exception error on execution:

#include 

int main()
{
    int num = -1;
    int total = -1;
    int count = 0;
    int values[] = {10, 256, 55, 67, 43, 89, 78, 78, 89, 0};

    while(count < 10)
    {
            num = values[count];
            total = num + 0xffffffff/num;

            printf("\n result = [%d]\n", total);
            count++;
    }

    return 0;
}

You suspect that the crash happens when num is zero in line 13. You could put a breakpoint on that line, but the program would halt every time the line is executed. Instead, set a conditional breakpoint by specifying the condition subject to which the breakpoint should hit, which in this case is num==0:

$ gdb test
GNU gdb (GDB) 7.6.1-ubuntu
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /home/himanshu/practice/wazi/gdb/test...done.
(gdb) break 13 if num==0
Breakpoint 1 at 0x804849c: file test.c, line 13.
(gdb) run
Starting program: /home/himanshu/practice/wazi/gdb/test

 result = [429496739]

 result = [16777471]

 result = [78090369]

 result = [64104056]

 result = [99883003]

Breakpoint 1, main () at test.c:13
13                total = num + 0xffffffff/num;
(gdb) n

Program received signal SIGFPE, Arithmetic exception.
0x080484aa in main () at test.c:13
13                total = num + 0xffffffff/num;
(gdb)

As you can see, the conditional breakpoint made the program stop when the value of the variable num was zero. I then entered the gdb command n (for next) and confirmed that the crash happens when num is zero.
If you want to cross-check that the debugger stopped the program execution at the correct condition, you can print the value of the variable num through the p 'num' command.

Ignore breakpoints

Sometimes you don't have any clue about the problem condition, so you might want the debugger to tell you the exact number of loop iterations after which the crash occurs, to help you analyze loop conditions. GDB lets you ignore a breakpoint a specified number of times. You can then check the breakpoint information to see the number of loop iterations after which the crash occurs.
For example, the code shown below also gives a floating point exception on execution. But let's suppose that, because of the way the code was written, it is difficult for you to pinpoint a condition under which the crash occurs:

#include 

int main()
{
    int num = -1;
    int total = -1;
    int count = 50;

    while(count--)
    {
        // a lot of code here

        total = num + 0xffffffff/(count-10);
        printf("\n result = [%d]\n", total);

            // a lot of code here
    }

    return 0;
}

You can put a breakpoint at the entry of the loop and ask GDB to ignore it every time. At the end of program execution, use the info command to see how many times the program encountered the breakpoint before the crash happened.
By the way, here and in all the subsequent examples, I used GDB's -q (for quiet) command-line option to suppress introductory and copyright messages.

$ gdb -q test
Reading symbols from /home/himanshu/practice/wazi/gdb/test...done.
(gdb) break 10
Breakpoint 1 at 0x8048440: file test.c, line 10.
(gdb) info breakpoints
Num     Type           Disp Enb Address    What
1       breakpoint     keep y   0x08048440 in main at test.c:10
(gdb) ignore 1 50
Will ignore next 50 crossings of breakpoint 1.
(gdb) run
Starting program: /home/himanshu/practice/wazi/gdb/test

 result = [110127365]

 result = [113025454]

 result = [116080196]

...
...
...
 
 result = [2147483646]

 result = [-2]

Program received signal SIGFPE, Arithmetic exception.
0x08048453 in main () at test.c:13
13            total = num + 0xffffffff/(count-10);
(gdb) info break 1
Num     Type           Disp Enb Address    What
1       breakpoint     keep y   0x08048440 in main at test.c:10
    breakpoint already hit 40 times
    ignore next 10 hits
(gdb)

Note that the number 1, used in the ignore command and later in the info break 1 command, is the breakpoint number, which you can get from the info breakpoints command.
In this example the output of the info break 1 command displayed the exact number of iterations (40) after which crash occurred. You now have an idea that something went wrong after exactly 40 loop iterations, which should lead you to the problematic line total = num + 0xffffffff/(count-10);.

Use watchpoints

Sometimes a variable whose value is not supposed to be changed is passed as an argument into a series of functions, and when the code flow comes back, you observe that the variable's value was changed. To manually debug this kind of problem, you'd have to debug every function to which the variable was passed. A better approach is to use watchpoints, which help you track the value of a specified variable.
To set a watchpoint on a global variable, first set a breakpoint to stop program execution at the entry of the main() function. For a non-global variable, set a breakpoint at the entry of the function where the variable is in scope. In either case, once the breakpoint hits, set a watchpoint.
In the following code, the value of the variable ref_val is passed from the main() function to the func5() function, and when the flow comes back to the main function, we find that the value is changed from 256 to 512.

#include 

void func5(int *ptr)
{
    // a lot of code here
    *ptr = 512;
}

void func4(int *ptr)
{
    // a lot of code here
    func5(ptr);
}

void func3(int *ptr)
{
    // a lot of code here
    func4(ptr);
}

void func2(int *ptr)
{
    // a lot of code here
    func3(ptr);
}

void func1(int *ptr)
{
    // a lot of code here
    func2(ptr);
}

int main()
{
    int ref_val = 256;

    func1(&ref_val);

    printf("\n ref_val = [%d]\n", ref_val);

    return 0;
}

To debug this issue, you can put a watchpoint at the entry of each function involved in the call sequence. Here I test func5() first. I set a breakpoint at its entry, and when it is hit, I put the variable *ptr under watchlist using the watch command:

$ gdb -q test
Reading symbols from /home/himanshu/practice/wazi/gdb/test...done.
(gdb) break test.c:func5
Breakpoint 1 at 0x8048420: file test.c, line 6.
(gdb) run
Starting program: /home/himanshu/practice/wazi/gdb/test

Breakpoint 1, func5 (ptr=0xbffff0bc) at test.c:6
6        *ptr = 512;
(gdb) watch *ptr
Hardware watchpoint 2: *ptr
(gdb) c
Continuing.
Hardware watchpoint 2: *ptr

Old value = 256
New value = 512
func5 (ptr=0xbffff0bc) at test.c:7
7    }
(gdb)

When I continued program execution, GDB displayed the new and old values of the variable being watched, which in this case were different. Just like a breakpoint, a watchpoint stops program execution, but at the point at which the value changes. Once you know the culprit, which is func5() in this case, you can invest your time debugging it.

Call user-defined or system functions

Sometimes you might want to test a function by providing inputs to it. To do this, you could change the code that calls that function every time, or add extra code that makes it possible to send inputs to that function through STDIN, which is usually the command line. Alternatively, you can use GDB's call command.
Suppose you want to test the function user_defined_strlen() defined in the following code. As you can see, it essentially calculates and returns the length of a string passed to it as argument:

#include 

unsigned int user_defined_strlen(char *ptr)
{
    int len = 0;
    printf("\n User-defined strlen() function called with string [%s]\n", ptr);

    if(NULL == ptr)
    {
        printf("\n Invalid string\n");
        return 0;
    }
     
    while(*(ptr++) != '\0')
        len++;

    printf("\n[%u]\n", len);
    return len;
}

int main()
{
    char *ptr = "some-string";
     
    user_defined_strlen(ptr);

    return 0;
}

You can put a breakpoint at the entry of the main() function. When GDB hits the breakpoint, execute the call command by passing to it a function name, along with arguments to test:

$ gdb -q test
Reading symbols from /home/himanshu/practice/wazi/gdb/test...done.
(gdb) break test.c:main
Breakpoint 1 at 0x80484bd: file test.c, line 23.
(gdb) run
Starting program: /home/himanshu/practice/wazi/gdb/test

Breakpoint 1, main () at test.c:23
23        char *ptr = "some-string";
(gdb) call user_defined_strlen("wazi")

 User-defined strlen() function called with string [wazi]

[4]
$1 = 4
(gdb)

You can also call standard library functions using the call command. For instance, at the same breakpoint, you can call the strlen() function to cross-check the output of the function:

(gdb) call strlen("wazi")
$2 = 4
(gdb)

Auto-display variable values

As a complement to a watchpoint, which stops execution whenever the value of a variable or an expression changes, you can use the display command to print the value of a variable or expression to see how it changes.
For example, in the code snippet above, if you wanted to display the value of the len variable as the while loop progresses, you could put a breakpoint at the while condition, execute the display command with len as an argument, and step through the code with the n command – or step through the code with n once, and just press Enter at every subsequent GDB prompt, because the debugger repeats the last command by default.

$ gdb -q test
Reading symbols from /home/himanshu/practice/wazi/gdb/test...done.
(gdb) break 14
Breakpoint 1 at 0x8048486: file test.c, line 14.
(gdb) run
Starting program: /home/himanshu/practice/wazi/gdb/test

 User-defined strlen() function called with string [some-string]

Breakpoint 1, user_defined_strlen (ptr=0x80485c2 "some-string") at test.c:14
14        while(*(ptr++) != '\0')
(gdb) display len
1: len = 0
(gdb) n
15            len++;
1: len = 0
(gdb) n
14        while(*(ptr++) != '\0')
1: len = 1
(gdb) n
15            len++;
1: len = 1
(gdb) n
14        while(*(ptr++) != '\0')
1: len = 2
(gdb) n
15            len++;
1: len = 2
(gdb) n
14        while(*(ptr++) != '\0')
1: len = 3
(gdb) n
15            len++;
1: len = 3
(gdb) n
14        while(*(ptr++) != '\0')
1: len = 4
(gdb) n
15            len++;
1: len = n
(gdb) n
14        while(*(ptr++) != '\0')
1: len = 5
(gdb) n
15            len++;
1: len = 5

The undisplay command removes an auto-displayed variable or expression previously set with display. It expects an expression number, which you can determine with the info command:

(gdb) info display
Auto-display expressions now in effect:
Num Enb Expression
1:   y  len
(gdb) undisplay 1
(gdb) n
15            len++;
(gdb) n
14        while(*(ptr++) != '\0')
(gdb) n
15            len++;
(gdb) n
14        while(*(ptr++) != '\0')
(gdb) n
15            len++;

In conclusion

As you can see, GDB offers several advanced tools that can help you find the flaws in your programs' code. You probably have your own favorite advanced debugging techniques – please share them in the comments below.