Linux Count lines in file and Sum count

• Tweet

• Tweet

Hi Guys,

To start with I am intermediate user of Linux and use RedHat at work and Ubuntu at home.

I had this job where I have to search my apache server logs for some “text”. Count the number of lines that have this “text” in them.

Well I know basic shell scripting so I just used a bit of basic commands to get what I wanted. If you know other better solution please leave yours in comments.

#!/bin/bash
COUNTTOTAL=0
DIRPATH=/var/www/html/access-logs/tmp/*
for file in $DIRPATH
do
  COUNT=`wc -l $file | cut -d " " -f1`
  COUNTTOTAL=$(($COUNT+$COUNTTOTAL))
done
echo $COUNTTOTAL

So DIRPATH above holds path to my access logs directory and I am iterating over to get line count of files one by one and my COUNTTOTAL variable holds the overall count.

Above example show to count all lines and summing the output.

In the example below lets make a slight change and search for a string “abc.com” and see how many lines have this string present.

#!/bin/bash
COUNTTOTAL=0
DIRPATH=/var/www/html/access-logs/tmp/*
for file in $DIRPATH
do
  COUNT=`grep "abc.com" $file | wc -l | cut -d " " -f1`
  COUNTTOTAL=$(($COUNT+$COUNTTOTAL))
done
echo $COUNTTOTAL

As you can see that we now are searching text “abc.com” in a file and piping the output to wc command adding the output to our global variable COUNTTOTAL and printing it using echo command.

you should change DIRPATH to suit you.

You can use the same fundamentals to create a dynamic list of file e.g. count the number of lines for files that are younger than 30 day, and iterate over them.

We have used the following command in our shell scripts and each of the one have link to their manual page

1. grep
2. wc
3. cut

As my mate David Overton pointed out that above can be achieved without a script so here are our shell commands to do exactly the same thing. I think I am just stuck with scripting mind frame

First command (equivalent to first script):

cat /var/www/html/access-logs/tmp/* | wc -l

Second command (equivalent to second script) :

grep ‘abc\.com’ /var/www/html/access-logs/tmp/* | wc -l

I hope that this helps.

If I made any mistake please let me know.

Cheers