Tech Debug » perl http://techdebug.com Why talk when you can fly Thu, 26 Aug 2010 07:30:27 +0000 en hourly 1 http://wordpress.org/?v=3.0.1 Perl while loop memory hog http://techdebug.com/blog/2007/10/30/perl-while-loop-memory-hog/ http://techdebug.com/blog/2007/10/30/perl-while-loop-memory-hog/#comments Mon, 29 Oct 2007 23:16:57 +0000 lantrix http://techdebug.com/blog/2007/10/30/perl-while-loop-memory-hog/ I was trying to use someone elses script for logging dansguardian events to an RDBMS.The script was chewing up 99% of my CPU! I got chatting to a friend and a perl coder about this, and got some ideas. He showed me about the perl debugger using the -d switch. Awesome. I finally got my perl script working. Installing the DBI package on OpenBSD was a snap with pkg_add (pkg_add -v p5-DBD-Pg-1.47.tgz). My logical debugging was as follows.

first I read that DBI was a memory hog so I wrote it out and put in a native postgres call, but the pgsql for perl was borked.
(comment from coding friend “DBI is fine man, works for the massess”). Next I changed the while loop over the the file to a Tail::File method, but the module was badly documenting and had issues.

I actually ended up using IO::File to tail the log and looped over the lines (in a mad loop), but it was still broken.
So I went back to DBI, which as my friend pointed out was never a problem. It worked!

It turned out the loop was the CPU hog and all I had to do was put in a sleep 1; if there was no new line in the tail.
In the process I rewrote the whole script and it barely resembles the original except for the idea. Now it has 0.5% CPU load at best. A win for coding. I’ll post the script soon and also send it to dansguardian.

Comment from perl coding friend: of course – i guessed that after u left. :-)

]]> http://techdebug.com/blog/2007/10/30/perl-while-loop-memory-hog/feed/ 0