Why is Retrospective Faster than Grep, Smarter than Tail?

urs.minder
centeractive people
Posts: 68
Joined: 06 Jun 2012, 16:45
Contact:

Why is Retrospective Faster than Grep, Smarter than Tail?

Postby urs.minder » 16 Aug 2012, 14:55

One of the first things that you'll see on our Retrospective web site is "Faster than Grep, Smarter than Tail!

You're probably wondering quite what we mean by this, so I thought i'd just write a few lines to explain.

Both grep and tail are well know linux/unix commands that sysadmins, operations people and IT nerds :roll: know pretty well but they are not so well known in the Mac and Windows world.

Grep is basically a pattern matching command, and you use it in a console window (which is like a dos window, remember them? :lol: ) to search through files or directories to find whatever you are looking for.

tail (with -F option) is a command that is also given in a console window and only shows the latest entries that are written to a file so, in essence, it only prints the latest log entries in near real time. Of course, Retrospective is smarter than a simple tail -F and allows further filtering (by entering a search parameter such as a text string) so it will only list the log entries you are interested in (rather than all the junk written to the file) which may contain, for example, the account number or ID you are specifically looking for (and you can use a local filter during a tail to only present a further subset of the tail data, even temporarily!)

For an interesting explanantion of tail -F see the last part of this helpful youtube video: http://www.youtube.com/watch?v=-sACt5yhrUA

As with most unix commands, the full explanantion of how to use grep will overload even R2D2's memory circuits, so retrospective allows similar functionality without needing to have your head plugged into the internet or remember the necessary arguments.

As to why Retrospective is smarter & faster, grep works in a serial way, it will slowly plod through the whole file system and eventually cover all the files you've asked it to search through (and getting grep to cover the specific file set your are interested in is a hell of a mission (take a look at the man page :o ), especially over multiple servers!!), however, what retrospective does is launch a thread for each log file being searched through PLUS Retrospective saves additional time by quickly analysing each matching file to check it's actually got content in the given date range to decide whether to continue to search through that file, or not. ( :idea: Hint: use a date range to reduce the data set Retrospective has to search through)

If you want to read the unix manuals, see below, otherwise you can use Retrospective and spend more time doing something useful than just staring at a console window! :lol:


BTW, the product roadmap includes dynamic inclusion of a file that matches any wild card expression or is contained in a defined directory used by an ongoing tail. Pretty smart huh!? :ugeek:

Return to “General Retrospective Topics”

Who is online

Users browsing this forum: No registered users and 1 guest