Apache Logs

Find unique IP addresses that hit your webserver

cat /var/log/httpd/access_log |awk '{print $1}' | sort | uniq | wc -l

1. first we use the cat command to list all the entries in the apache log file.
2. we send that to awk which extracts the first item on each line which should be the IP
3. then we sort all the IPs, this is needed for the uniq command
4. we apply the uniq filtering command to remove duplicate IPs
5. then we count the number of lines left using word count command wc

# filter out unique requests for a given day, in this case Jan 1st 2012

cat /var/log/httpd/access_log |  grep "\[01/Jan/2012" |awk '{print $1}' | sort | uniq | wc -l




# getting a list of unique referrers

cat /var/log/httpd/access_log | awk '{print $11};' | awk -F / '{print $3}' | sort | uniq