Server resource utilization monitoring

Pretty much every Linux distribution has tools to help sys admins monitor the utilization of common server resources such as CPU, memory and storage. For advanced monitoring and reporting we may need to install 3rd-party tools (e.g. Nagios or FAN) but sometimes we just want a simple way to get warnings when critical server resources may be close to overload.

Here is a set of simple shell scripts (tested in RHEL and CentOS) you can use to monitor CPU, memory and storage space. If utilization goes over a threshold these scripts will send you an email warning hopefully soon enough so that you can fix it before a system crash. Assuming you have access to basic commands like vmstat, df, mail and crontab these scripts can be setup even if you do not have root access to the server.

Quest to find the best Linux for slow computers, Fedora 12 LXDE upgrade to 13

I replaced Kubuntu Netbook Remix (KNR) on my netbook with the LXDE spin of Fedora 12. KNR looks nice and has many nice features for devices with small screens but it was just painfully slow on my netbook. LXDE is (in theory) a more crappy-hardware-friendly Linux flavor, so I decided to give it a try.

LXDE's UI looks uglier and less flexible than KNR's, but performance is indeed better. With that in mind I put together a development environment based on a Fedora LXDE virtual machine I can use to test my own projects in Linux. Nothing terribly resource-intensive but things like Grails, Gradle, AppFuse, etc. I wanted to know if this VM would be fast enough so I could use it as my development environment.

Click on "Read more" below to check out the rest of this post on a development environment VM and instructions on how to update Fedora 12 LXDE with the unofficial release 13.

UNIX / LINUX quick reference

I have compiled the Unix/Linux code snippets below as a quick reference covering the "how to" on various topics. Please feel free to leave a comment with your own "Unix command line gems".

How to:

# list top memory processes on a Linux machine:
ps aux | awk '{print $4"\t"$11}' | sort | uniq -c | awk '{print $2" "$1" "$3}' | sort -nr

# show how many CPUs on a Linux machine:
cat /proc/cpuinfo

# create a symbolic link:
ln -s originalPath newLinkPath
e.g. ln -s /usr/local/apache/logs /tmp/apachelogs

# list largest files in Linux (good for file system cleanup):
find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | sort -nk 2,2

# list largest directories in Linux:
du -h / | grep ^[0-9.]*G | sort -g
du -h / | grep ^[0-9.]*M | sort -g | more
find / -type d -size +1G

# split big text files into smaller and more manageable files
split --lines=100000 --suffix-length=3 application.log application.log.a

# redirect Linux output:
ls 1> out.log 2> err.log

# do global string replaces in the VI editor:
:%s/old_string/new_string/g

# set executable tag in shell script files recursivelly:
find . -name "*.sh" -exec chmod +x+x+x {} \;

# delete bak and log files recursivelly:
find . -name "*.bak" -exec rm {} \;
find . -name "*.log" -exec rm {} \;

# recursively determine how much space a directory is taking:
df -h : shows how much space we have on each partition
du -sh * : show how much space the current directory is taking
du -h | grep -v '/' | awk '{print $1}' : does the same without iterating thru the subdirs

# find files that contain a given text string
grep -lir "/data/platform" *

# find files that contain a given text string recursively
find . * -exec grep -li "/data" {} \;
find . * -exec grep -li "/data" {} \; > files.txt

# poll the number of files in a folder every 3 seconds
ksh
while true
do
ls *.retry.xml | wc -l
sleep 3
done

# archive a folder and sub-directories in tar/gz
tar -zcvf archive.tgz directory
e.g. tar -zcvf apacheLogs.tgz /usr/local/apache/logs

# extract a tar/gz archive
tar -xvf archive.tgz directory
e.g. tar -xvf  apacheLogs.tgz .

These are just some of the ones I use the most. You can also find many more at the Unix Toolbox here.