I wanted to quickly perform queries on my apache access.log file to see how many of the visitors I was getting were going to the pages I wanted them to go. I also wanted to be able to view this at any location, and to be able to see this activity over time (as far back as a week). I decided I was going to do it in shell script, even though I've never done that before and knew next to nothing about it. So after a few hours, here it is for your convenience:
Code:
#!/bin/sh
#
# log_manager.sh v1.1 by itistoday
# Description: An administrative utility that can be used to view relevant parts
# of apache access.log files. Used with cron it can keep a trail of a fixed
# number of logs to view. This can be modified for many practical purposes because
# all it really does is execute external scripts and increments file numbers.
#
# The results of using it with cron can look something like this:
#
# [shell]$ ls | less
#
# potential-0.log
# potential-1.log
# potential-2.log
# potential-3.log
# potential-4.log
# potential-5.log
# services-0.log
# services-1.log
# services-2.log
# services-3.log
# services-4.log
# services-5.log
#
############### Config area ###############
APACHE_LOG_PATH="path/to/access.log" # path to the access.log file
NEW_LOG_PATH="path/to/new/logs" # path to new logs directory, no trailing slash!
MAX_NUM_LOGS=51 # maximum number of new logs to keep per prefix
############# End Config area #############
if [ -z $1 ] ; then
echo "Please provide a log prefix as an argument."
echo "Be sure to provide the appropriate your_prefix.sh file as well"
echo "in the same folder as this script."
echo "It must take four arguments and look something like this:"
echo ""
echo "grep your_query \$1 > \$2/\$3-\$4.log"
echo ""
echo "The first argument passed is \$APACHE_LOG_PATH (edit this script to change)"
echo "The second is \$NEW_LOG_PATH"
echo "The third is the file prefix that you pass to this script"
echo "The fourth is an incremental number this script passes automatically"
exit
fi
log_prefix=$1
if [ ! -f ${log_prefix}.sh ] ; then
echo "Error: ${log_prefix}.sh does not exist."
echo "Please create it with the correct format."
echo ""
echo "Hint: to see an example run this script without arguments"
exit
fi
num_files=`ls ${NEW_LOG_PATH}/${log_prefix}* 2>/dev/null | wc | awk '{print $1}'`
sh ${log_prefix}.sh $APACHE_LOG_PATH $NEW_LOG_PATH $log_prefix $num_files
if [ $num_files -ge $MAX_NUM_LOGS ] ; then
# get oldest file and delete it, the rename the rest to one less
oldest=`ls ${NEW_LOG_PATH}/${log_prefix}* | head -1`
rm $oldest
list_of_files=`ls ${NEW_LOG_PATH}/${log_prefix}*`
i=0
for ff in $list_of_files ; do
new_name="${NEW_LOG_PATH}/${log_prefix}-${i}.log"
if [ -f $new_name ] ; then
rm $new_name
fi
mv $ff $new_name
i=$(( $i + 1 ))
done
fi
(Sorry about the lack of tabs, it's this board's fault)
It takes one argument, a "prefix". Here's an example:
Say you wanted to see all the people that accessed a certain web page, but only cared about the Mac users and didn't want to see yourself in the logs. You would make a file called macfilter.sh that contained this (where xxx.xxx.xxx.xxx is your IP address):
Code:
grep the_page.html $1 | grep Macintosh | grep -v xxx.xxx.xxx.xxx > $2/$3-$4.log
Make sure this file is in the same directory as this script. Edit the script config values to your liking, and then just run:
Code:
sh log_manager.sh macfilter
or make it executable to just do:
Code:
./log_manager.sh macfilter
You can then set this up to be executed by cron every 30 minutes, and presto! You've got a whole history of the visitors to this page, and if you have it put them in a folder for a website you can view it wherever you are. Sure it's crude, but it's also quick and dirty and gets the job done!
It was a great learning experience and hopefully someone other than me will find it useful
Top
