awk oneliner to find large files

A mega geeky awk one-liner today. Tested on Solaris under bash, so YMMV.

Have you ever found that a filesystem is filling up fast, and dont know what is causing it? This one liner (which can be placed in a cron job if you like) is best run as a super user.

It will:

1. Search for all files in the current dir and subdirs that are modified in the last 3 days;
2. list them, filtering just the filesize and name/path;
3. sort/order by the largest file; and
4. Email you a copy of the report.

find / -type f -mtime -3 -exec ls -l {} \; 2>/dev/null \
| awk '{print $5 " " $9}' \
| awk '{printf "%12d\t%s\n", $1, substr($0,index($0,$2),80)}' \
| sort -r >/tmp/largefiles.txt && \
( uuencode /tmp/largefiles.txt largefiles.txt ) \
| mailx -s 'Large Files Report' -r mail2@youremail.com `cat /tmp/mailrecipients` &

Nice. If you just want to see the response on stdout then try this:

find / -type f -mtime -3 -exec ls -l {} \; 2>/dev/null \
| awk '{print $5 " " $9}' \
| awk '{printf "%12d\t%s\n", $1, substr($0,index($0,$2),80)}' \
| sort -r

Have fun.

Clearcase Tips Number 03 – managing label conventions with perl

I previously showed you how to use a shell script with Rational Clearcase, to alert you when a new branch type was created.

In this post, I will show you how to use a Perl script to enforce Clearcase labeling conventions. This example is directed toward Clearcase on UNIX (i.e. Solaris or similar) and assumes you have Perl installed, working and have a basic knowledge of how to program in Perl. It is a reworked version of the windows script supplied by IBM on Developerworks.

This is a long post, but a good one if you are a new clearcase admin who needs to enforce label names.

Read the rest of this entry »

Perl while loop memory hog

I was trying to use someone elses script for logging dansguardian events to an RDBMS.The script was chewing up 99% of my CPU! I got chatting to a friend and a perl coder about this, and got some ideas. He showed me about the perl debugger using the -d switch. Awesome. I finally got my perl script working. Installing the DBI package on OpenBSD was a snap with pkg_add (pkg_add -v p5-DBD-Pg-1.47.tgz). My logical debugging was as follows.

first I read that DBI was a memory hog so I wrote it out and put in a native postgres call, but the pgsql for perl was borked.
(comment from coding friend “DBI is fine man, works for the massess”). Next I changed the while loop over the the file to a Tail::File method, but the module was badly documenting and had issues.

I actually ended up using IO::File to tail the log and looped over the lines (in a mad loop), but it was still broken.
So I went back to DBI, which as my friend pointed out was never a problem. It worked!

It turned out the loop was the CPU hog and all I had to do was put in a sleep 1; if there was no new line in the tail.
In the process I rewrote the whole script and it barely resembles the original except for the idea. Now it has 0.5% CPU load at best. A win for coding. I’ll post the script soon and also send it to dansguardian.

Comment from perl coding friend: of course – i guessed that after u left.