TedCarnahan

Debugging long-running perl programs in Debian

A Tech article with View Comments posted 15 December 2006.
Tags: debian, debug, perl, software

One of the slicker ways I’ve seen of trying to figure out what Perl is doing in a long-running process is defining a signal handler, like so:

use Carp ();
$SIG{‘USR2′} = sub {
Carp::confess(“Caught SIGUSR2: Dumping stacktrace and dieing:”);
};

It works great, dumping the current stacktrace to STDERR. But if you’re stuck in a loop inside the perl interpreter, that user-level signal is never handled, and you don’t get anything. Foiled again! You’re going to have to use (gasp!) the GNU debugger (gdb).

While the advice offered in Debugging mod_perl and Debugging mod_perl C internals is useful, most of the magic with gdb doesn’t work unless you are running a perl with debugging symbols on. How do you get that in Debian-based distributions?

sudo aptitude install perl-debug

Then run the process:

/usr/bin/debugperl path/to/program.pl

Attach gdb:

gdb

attach Process ID

Define the curinfo gdb macro. (see Analyzing the Core File.) If you see “my_perl” in a gdb backtrace, you’re running threaded perl, otherwise use the unthreaded version of curinfo.

(gdb) bt
#0 0xffffe410 in __kernel_vsyscall ()
#1 0xb7f78751 in accept () from /lib/tls/i686/cmov/libpthread.so.0
#2 0×081502de in Perl_pp_accept (my_perl=0×81c1008) at pp_sys.c:2565
#3 0×080d10b3 in Perl_runops_debug (my_perl=0×81c1008) at dump.c:1452
#4 0×08065c9b in S_run_body (my_perl=0×81c1008, oldscope=1) at perl.c:1995
#5 0×08065703 in perl_run (my_perl=0×81c1008) at perl.c:1919
#6 0×0805febd in main (argc=3, argv=0xbfccac44, env=0xbfccac54)
at perlmain.c:98

Looks like we’re running threaded perl, so define curinfo:

(gdb) define curinfo
Type commands for definition of “curinfo”.
End with a line saying just “end”.
> printf “%d:%s\n”, my_perl->Tcurcop->cop_line, \
my_perl->Tcurcop->cop_file
> end

Then use it:

(gdb) curinfo
196:/usr/local/share/perl/5.8.7/Catalyst/Engine/HTTP.pm
(gdb)

That’s what line was executing and what file you were in. It’s worth mentioning that this was not a buggy application, just an ordinary Catalyst application waiting for a connection. Not too exciting for ordinary perl debugging (much easier to run the process as perl -d), but when you’ve got an infinite loop in the perl interpreter (for example, an infinite loop in a regex), this can help point you in the right direction.