Finding the right information on time can be a challenge when working with large files in the terminal. Especially log files, which contain lots of details about the activity of a system. Luckily, there are also lots of file searching tools for cutting through unnecessary info and getting only the data that interests you. I will make a quick dive into some of the most basic file reading tools that I use on a daily basis for handling log files, as a junior QA tester.
Note: the commands have a fragment from their man description added besides them. If you don’t know what man is, try the man man command in the terminal to find out.
cat – “The cat utility reads files sequentially, writing them to the standard output.”
The most basic command for reading a file from the terminal is cat. As most built-in Linux commands it comes with several flags that can enhance its functionality but it’s frequently used without any of them, to only display the whole content of a file. Here’s an example:
It only displays the contents of sample.txt to the standard output. By using it in combination with the pipe “|” operator it can deliver this output to other terminal commands instead of the standard output.
pipe ‘|’ – “The pipe(8) daemon processes requests from the Postfix queue manager to deliver messages to external commands.”
The pipe operator is used between two commands and as an operator, its function is to pass the output of the first command to the input of the second command. Here’s an example:
The pipe operator already comes in handy and lets us filter the sample1.txt. In the first example it’s used in combination with head to display the first line from sample1.txt and in the second one it’s pretty self-explanatory but you should try it out.
tail – “The tail utility displays the contents of file or, by default, its standard input, to the standard output.”
As you saw in the example from above, tail can be used to display lines of text from a file, starting from the end of the file upwards. In combination with the -f flag, tail can constantly display in the terminal the lines of text that are appended to a file. This is the normal behavior of a log file; information is added constantly as new events happen in the system. Here’s an example of tail -f :
So tail displayed the last 10 lines from sample.log and now waits to display the next lines of text that will be appended.
grep – “The grep utility searches any given input files, selecting lines that match one or more patterns”
Grep is a powerful and easy to use tool for finding the right information from inside a given context. The context can be one or more than one file, and the correct information can be one or more than one pattern that interests you. Here’s an example:
In this example, I used grep to search for all the contexts in sample.log that contain the ‘EVENT‘ pattern. The response was very fast in this case, where the file had about 360 lines of text. You can check how many lines a file has by using cat -n and see how fast a commands’ response is by typing time in front of it and then execute it. You can also check this simple explanation of the levels of a
One of the most useful command combinations for me, when working with log files, is tail and grep. Let’s say you monitor the actions related to a specific ID. The logs can have dense information so you only want those actions displayed and the rest of them ignored. The following command will do exactly that: tail -1f file_name | grep ’42’
It will display the last line from file_name if ’42’ is present in it and any further lines (when appended) in which ’42’ is found. Tail will run until you interrupt it’s execution.
To search for a pattern in multiple files, you can simply type: grep ’42’ file1 file2
To search in a file for multiple patters, type: grep -e ’42’ -e ’43’ file1
“This option is most useful when multiple -e options are used to specify multiple patterns”
Other useful flags for grep are -A and -B. They specify a context to be printed (B) before or (A) after the line that contains the pattern is displayed. You can also use the -C flag which results in a combination of both -A and -B flags.
There are a lot of command combinations that can be done only with the commands mentioned above, especially with the help of the pipe character. Feel free to try them and see how it goes.
Extra: If you’re on MAC OS, try the following command: say hello world