Splunk is a software application that uses a search language and log aggregation techniques to assist IT help desk staff and managers get to the root cause of system, application and infrastructure problems. Splunk 2.2 takes log data from almost any source and parses out the time information in the message in order to execute fast searches based on user requests of the basic form "what was happening at such-and-such time."
While Splunk can work without any training and end users should be able to come up to speed on the product with little training, getting the most out of the reams of data that Splunk processes does take skill. Besides knowing the right questions to ask and formatting those problems correctly operators can also tag found events to make the system work even better. Tagging and searching do get better over time and that is part of what I've been working on in eWEEK Labs during our tests of the log search engine.