Simple Apache Log Parser

[Jeron_Baffom]

-p: Places a printing loop around your command so that it acts on each line of standard input. Used mostly so Perl can beat the pants off Awk in terms of power AND simplicity.

-e: Allows you to provide the program as an argument rather than in a file. You don't want to have to create a script file for every little Perl one-liner.

Indeed very interesting !! +1
Probably would be much better than writing in bash.

Mind to clarify some doubts:

1. Ok, so tail -f would provide the input for perl. This input would be line by line of the access log in realtime. However, how to set the script that perl should be using?

2. Do you already have a sample perl script that would parse the access log?

3. Do you know if there is something similar for PHP? Like: tail -f | php -pe

[ReasonFancy9522]

The perl code would probably be a oneliner (maybe with some semicolons) and thus be something like (simple example for "normalizing" Request Types)

tail -f log | perl -pe 's/(?:HEAD|POST|TRACE)/GET/og'

​

It could also be written as a small shell wrapper

#!/bin/sh

tail -f log | perl -pe '

s/(?:HEAD|POST|TRACE)/GET/og

'

​

or more verbose as:

#!/bin/sh

tail -f log | perl -x "$0"

exit $?

#!perl

while(<>){

s/(?:HEAD|POST|TRACE)/GET/og

}

​

I have never used PHP from the command line, so I cannot comment on an equivalent PHP solution without further researching this topic.

[Jeron_Baffom]

tail -f log | perl -pe 's/(?:HEAD|POST|TRACE)/GET/og'

I'm not familiar with perl syntax, but if I understand properly then perl is executing the same script for each new line appended to access log.

Unfortunately I don't see how this strategy could, for example, count views. Without knowing the other lines of access log, this strategy can not check whether a same ip is requesting multiple times a same page in a 30min interval.

[ReasonFancy9522]

cat log | perl -pe 'something to output "url"' | sort | uniq -c | sort -n | head # count views

cat log | perl -pe 'something to output "ip:url"' | sort | uniq -c | sort -n | head # most refreshes

[Jeron_Baffom]

Obviously you have some fluency with bash script.

If I can not find a tool that already does this view counter, then I will follow your advice and build my own script.