As most of you know, I’ve been working on a book about logs, logging and log management for some number of years. At this point, the book is almost done, but the author team is having some minor time commitment issues (aka “less time to write than originally estimated”) ).

So, do any of my esteemed blog readers (those adept in the dark arts of log analysis) care to help and write a few chapters here and there, in exchange for (lots of) immortal fame and (admittedly small amount of) cash?

Table of contents is here – if you see any chapters you’d like to help with, please let us know. I will post a list of chapters that really need help soon.

At this point, we have PLENTY of reviewing help, but we sure can use some writing help!

About me: http://www.chuvakin.org

Executive summary: you need to procure services when you buy a SIEM tool, if you don’t – you’d be sorry later.

Even if you are amazingly intelligent and have extensive SIEM experience – see above.  Even if you saw a successful SIEM project that didn’t include vendor or 3rd party services with your very eyes – see above. Even if your SIEM vendor tells you “you don’t need services” – see above. See above! See above!! See above!!!

Let’s analyze this “SIEM services paradox.” A lot of organizations – way too many, in fact – balk at the need to procure related services before, during and after their SIEM purchase. The thinking often goes like this: we need a SIEM and this box <points at the appliance still in the box> is a SIEM. That’s all we need. What services? Why services? Huh?

In reality -  and this is what I sometimes call “secret to SIEM magic” – that box is not a SIEM. That box, when racked and connected to your network, is STILL not quite a SIEM. Only when you “operationalize” it (see picture), then you can say that you have Security Information and Event Management (SIEM) capability in your organization and that you do “real-time” security monitoring.

Now, be honest, do you know how to deploy a SIEM tool and then figure out the shortest path to its operational success? Probably not… thus services/consultants who will work WITH you to make it a reality by arriving at the best possible way of benefitting from SIEM in your environment. Which use case give you the best bang for the SIEM buck? Which one will show a “quick win” to your management? Which one is more likely to detect an attacker in your network?

When a SIEM vendor tries to sell you services, it is NOT vendor greed – but simply common sense. And if you say “no”, it is not “saving money” – but being stupid. SIEM success  out-of-the-box (while real, in some cases!) is a pale shadow of what a well-thought through deployment looks like! My [broken] analogy is: you buying a nice shiny Aston Martin and then only using it to commute to a train station 1 mile from home. Will it work? Yes. Is this a good investment and a good experience? Hell  no!

So, no, SIEM is NOT useless without services, but it is unlikely to reach its full potential. Pitfalls to SIEM success are many, and navigating them requires help.

And, no, outsiders alone cannot do it. You will need to help them  help you.

This also leads to the rise of managed or co-managed SIEM options (which are NOT MSSPs, BTW!) as more people realize that a) they need a SIEM and b) they cannot handle a SIEM. Future cloud SIEM will (when it emerges) try to tackle the same problem of being simpler to operate and thus simpler to operationalize.

Today, most SIEM vendors offer an extensive menu of services to go with a product, and there are also some smart third parties.  Many services around SIEM can be organized as follows.

Pre-sale services examples:

• Product selection help
• Vendor differentiation analysis and shortlist definition
• Regulation analysis and business cases review
• Product strengths/weaknesses analysis
• Product fit for type of project
• Product fit for vertical / business type
• RFP definition assistance
• Current tools vs requirements gap analysis

Services offered during SIEM acquisition and deployment:

• SIEM implementation
• SIEM project planning
• Proof-of-concept deployment management
• Product testing in production environment
• Data source integration and collection architecture
• Default contents tuning

Post-sale, operational services:

• SIEM analyst training
• Performance tuning and capacity planning
• SIEM project management
• Custom content creation
• Custom device integration
• SOC building

Vendors and consulting firms offer other types of services as well all the way up to “co-managed SIEM” where a 3^rd party firm manages your SIEM deployment for you. Will future SIEM work better out of the box? Yes, I think so. Will SIEM ever be as simple as a firewall? No, never: it is inherent complexity of security monitoring that cannot be squeezed out even by creative engineering…

Enjoy ... as this was my final blog post on SIEM.

Possibly related posts on SIEM:

• On Broken SIEM Deployments
• Top 10 Criteria for a SIEM?
• Algorithmic SIEM "Correlation" Is Back?
• How Do I Get The Best SIEM?
• Log Management->SIEM Graduation Criteria: Violate at Your Own Peril!
• How to Replace a SIEM?
• SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?
• How to Write an OK SIEM RFP?
• On Choosing SIEM
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases and Whitepaper with detailed SIEM use cases
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

Imagine you own a broken, dilapidated, failing SIEM crap deployment. What? Really… that, like, never happens, dude! SIEM is what makes unicorns shine and be happy all the time, right?

Well…mmm… no comment. In this post, I want to address one common  #FAIL scenario: a SIEM that is failing because it was deployed with a goal of real-time security monitoring, all the while the company was nowhere near ready (not mature enough) to have any monitoring process and operations (criteria for it).  On my log/SIEM maturity scale (presented here, also see this related post from Raffy), they are either in the ignorance phase or maybe log collection phase.

And herein lies the problem: if you deployed one of the legacy, born in the 1990s SIEMs that are not based on a solid log management platform, the tool will actually suck at the very fundamental level: log collection. The specific issue here is that most of these early tools were designed to only selectively collect what was deemed necessary for real-time security monitoring (vs all log data). In essence, you have a tool with monitoring features (that you don’t use) and with weak collection features (that you can use, but they are weak).

What to do? You have these options:

1. Leave it to rot; you can always keep it just to boast to your friends (and PCI QSAs) that “ye own one of ‘em olde SIEMs”
2. Blow it away and join the “SIEM doesn’t work” crowd – and maybe buy a simple log management tool later
3. Deploy a log management tool to “undergird” your crappy SIEM; you have a choice of buying from the same SIEM vendor (if they have it) or a different vendor
4. Built your own log management layer on syslog and open source tools

I have seen people take either of the above four. Personally, I have seen much more success with the option #3 (buy log management) and not infrequently with #4 (built log management) – BTW, this deck might help you choose. You want to move your SIEM setup from “get some logs – ignore all logs” model to “collect all/more logs – review some logs” which is typically much more aligned with your level of maturity. And then grow and solve more problems with your SIEM and demonstrate “quick wins.” While you are at it, review some architecture choices discussed here.

Enjoy …while it lasts.

Possibly related posts on SIEM:

• Top 10 Criteria for a SIEM?
• Algorithmic SIEM "Correlation" Is Back?
• How Do I Get The Best SIEM?
• Log Management->SIEM Graduation Criteria: Violate at Your Own Peril!
• How to Replace a SIEM?
• SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?
• How to Write an OK SIEM RFP?
• On Choosing SIEM
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases and Whitepaper with detailed SIEM use cases
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

As I am going through my backlog of topics I wanted to blog about (but didn’t have time for the last 4-6 months), this is the one I really wanted to explore. Here is the scenario:

1. Something blows up, hits the fan, starts to smell bad, <insert your favorite incident metaphor> … either in your IT environment or at one of your clients’
2. Logs (mostly) and other evidence is taken from all the components of the affected system and packaged for offline analysis
3. You get a nice 10MB-10GB pile of juicy log data – and they wants “answers”
4. What do you do FIRST? With what tools?

Let’s explore this situation. I know most of you would say “just pile’em into splunk”  and, of course, I will do that. However, that is not a full story. As I point out in this 2007 blog post (“Do You Enjoy Searching?”), to succeed with search you need to know what to search for. At this point of our incident investigation, we actually don’t! Meanwhile, the volume of log data beyond a few megabytes makes “trial and error” approach of searching for common clues fairly ineffective.

If you received any hints with the log pile (“I think the user ‘jsmith’ did it” or “it seems  like 10.1.3.2 IP was involved”), then you can search for this (and then branch out to co-occurring and related issues and drill-down as needed), but then your investigation will suffer from “tunnel vision” of only seeing this initially reported issue and that is, obviously, a bad idea.

Let’s take a step back and think: what do we want here? what is our problem?  We want a way to explore ALL the logs in  a pile, across log types, across devices, across all time AND then also following a timeline of events. In other words, we ain’t in “searchland” here, buddy…

If you have an enterprise SIEM sitting around (and one with well-engineered support for diverse historical log imports – which is NOT a certainty, BTW), you should definitely load the logs there as well. I like this approach since you can then run cross-device summary reports over the entire set, slice the set in various ways (type of log source, log source instance, type of log entry – categorized, time period filter, time trend, etc) and data visualization tools (treemaps, trend lines, link maps, and other advanced visuals on parsed, normalized and categorized) help get a big picture view of our pile.

Looking at the open source log tools, does anything look promising for the task? OSSIM can do the trick (even though their historical log import is not my favorite), but nothing else does. In some cases, I used sawmill (free trial) for my “big picture first look”, but it is not cross-device and only shows reports for each log type individually. If I were feeling really adventurous (and was on hourly billing), I could actually send all the logs via a syslog streamer into OSSEC (in order to see the log entries the tool will flag as interesting/alertable), but this is not really something I’d enjoy doing. I am almost tempted to say that you can use something like afterglow, but it relies on parsed data that you’d sill need to cook somehow (such as again using a SIEM). Log2timeline is useful, but only for one dimension – and the one that splunk actually addresses pretty well already.

To generalize, you need (a) a search tool and (b) an exploration tool. The search tool should help you quickly answer SPECIFIC questions. The exploration tool should use data to generate “hints” on WHAT questions you should start asking…

About me: http://www.chuvakin.org

As I was drinking cognac on the upper deck of a 747, flying TPE-SFO back from a client meeting, the following idea crossed my mind:  CAN one REALLY do a decent job with log management (including log review) if their budget is $0 AND their “time budget” is 1 hour/week? I got asked that when I was teaching my SANS SEC434 class a few months ago and the idea stuck in my head – and now cognac, courtesy of China Airlines, helped  stimulate it into a full blog post.

So, $0 budget points to using open-source,  free tools (duh!), but 1hr/week points in exactly the opposite direction: commercial or even outsourced model.

The only slightly plausible way it that I came up with is:

1. Spend your 1st hour building a syslog server; it can be done, especially if starting from a old Linux box that you found in the basement (at $0); don’t forget logrotate or equivalent
2. Spend a few next weeks (i.e. hours) configuring various Unix, Linux and network devices (essentially, all syslog log sources) to log to it
3. Consider deploying Snare on a few Windows boxes (if needed); it would likely be easier to do than doing remote pull – too much tuning might be needed
4. Next, drop a default OSSEC install on your log server and – gasp! – enable all alerts
5. Spend the next  few hours (in the next few weeks) turning off the ones that are too numerous, irrelevant or don’t trigger any action in your environment.
6. If you log volume fits within a free splunk license size (500MB/day), also spend an hour deploying splunk on your log server and have it index all gathered logs
7. Now you’d be spending your “one log hour each week” on reviewing alerts and (if installed) digging in splunk for additional details
8. Congrats! $0 and 1hr/week gave you semblance of log management and even monitoring….

What do you think? It just might work for organizations with severe time AND money constraints.

Enjoy the post … while it lasts.

BTW, on a completely unrelated note:  do you think EVERY organization above a certain size NEEDS a SIEM? Or WILL NEED a SIEM in the future?

About me: http://www.chuvakin.org

As those in the know already know, NIST has officially released some EMAP materials the other day (see scap.nist.gov/emap/). EMAP stands for “Event Management Automation Protocol” and has the goal of “standardizing the communication of digital event data.” You can think of it as future “SCAP for logs/events” (the SCAP itself is for configurations and vulnerabilities). Obviously, both twin standards will be “Siamese twins” and will have multiple connection points (such as through CVE, CPE and others).

In reality, SCAP and EMAP are more like “standard umbrellas” and cover multiple constituent security data standards – such as CPE, CVE, CVSS, XCCDF, etc for SCAP and CEE for EMAP. As the new EMAP site states:

The Event Management Automation Protocol (EMAP) is a suite of interoperable specifications designed to standardize the communication of event management data. EMAP is an emerging protocol within the NIST Security Automation Program, and is a peer to similar automation protocols such as the Security Content Automation Protocol (SCAP). Where SCAP standardizes the data models of configuration and vulnerability management domains, EMAP will focus on standardizing the data models relating to event and audit management. At a high-level, the goal of EMAP is to enable standardized content, representation, exchange, correlation, searching, storing, prioritization, and auditing of event records within an organizational IT environment.

[emphasis by me]

While CEE team is continuing its work on the log formats, taxonomy, profiles and other fun details of logging events, the broader EMAP effort creates a framework around it as well as proposes a set of additional standards related to correlation, parsing rules, event log filtering, event log storage, etc. The released deck [PDF] has these details as well as some use cases for EMAP such as Audit Management, Regulatory Compliance, Incident Handling, Filtered Event Record Sharing, Forensics, etc.

In the future, I expect EMAP to include event log signing, maybe its own event transport (run under CEE component standard) as well as a bunch of standardized representation for correlation (via CERE component standard) and parsing rules (via OEEL) to simplify SIEM interoperability as well as migration.

Everything public to read on EMAP is linked here (2009), here (2010), here, etc [links are PDFs], if you are into that sort of reading. SIEM/log management vendors, please pay attention - some of you have already started implementation of this stuff….

About me: http://www.chuvakin.org

Somebody asked me that question “Do I need SIEM or do I need log management?” yesterday again, and I figured I’d repost this “bit of Anton’s wisdom” (ego alert!), so that people can just read this instead of repeatedly bugging me with this question.

Q: How do I figure out whether I need SIEM or log management?

A: You need log management – if you have computers, IT, data, etc. Period! This is not really a discussion item at all, since about 1986 or so. But do you also need a SIEM? You might think you need it, but you would only be able to benefit from it and satisfy that need if your organization fits the following "graduation criteria from log management to SIEM:”

1. Response capability: The organization must be ready to respond to alerts soon after they are produced. Incident response process/procedures are a must
2. Monitoring capability: The organization must have or start to a build security monitoring capability such as a Security Operations Center (SOC), or at least a team/person/resource dedicated to ongoing periodic monitoring.
3. Tuning and customization capability: The organization must accept responsibility for tuning and customizing the deployed SIEM tool; pure out-of-the-box SIEM deployments rarely succeed.

(originally written for this paper where the above are clarified in more detail)

Possibly related posts:

• All my posts about SIEM

About me: http://www.chuvakin.org

Here is a hard problem: you MUST log, but there are no logs to enable. Or, what is no less common, logs are so abysmal that they don’t help – and don’t fit the regulatory mold (example: PCI DSS Requirement 10.2 and 10.3). Or, logs are “out there in the cloud” and you cannot get them, but compliance is here and requires them.

What to do?

The answer to this eternal question is in my new whitepaper that I have written for Observe-IT (observeit-sys.com)

Executive summary:

This paper covers the critical challenges implementing PCI DSS controls and suggests creative solutions for related compliance and security issues. Specifically, the hard problem of security monitoring and log review in cloud, legacy, and custom application environment is discussed in depth. Additionally, clarification of key PCI DSS compensating controls is provided. This paper will help you satisfy the regulatory requirements and improve security of your sensitive and regulated data.

Short version [PDF] (5 pages)

Extended version [PDF] (13 pages)

As usual, the vendor was paying the bill, but thinking and research are all mine (SecurityWarrior Consulting)

Enjoy!

Possibly related posts / past whitepapers:

• Two New Logging Resources Published 
• New SIEM Whitepaper on Use Cases In-Depth OUT! 
• Another Fun SIEM Whitepaper

About me: http://www.chuvakin.org

Note: this has been written for “Cisco MARS blog” as a guest post and is reposted here for posterity.

Ouch! That “Venus” SIEM  appliance that we got with routers has finally croaked. That piece of PHP brilliance that pre-pre-previous security engineer wrote has been buried under the thick pile of XML. That managed SIEM provider has annoyed us one last time.

What do the above situations have in common? The unfortunate time to replace your SIEM has come. What to expect, apart from copious amounts of pain? This post will shed some light on this conundrum, based on author’s experiences.

First, it goes without saying that it is better to choose the right SIEM the first time (e.g. see “On Choosing SIEM” and other posts mentioned below) than to migrate from a SIEM that has been collecting logs (and dust) for a few years. However, you might not have any say in the matter – you might have inherited it, your “evil boss” might have procured the previous SIEM without asking you or you might have built it yourself after a particularly bad hangover… Also, your organization might have simply outgrown the SIEM or your early generation SIEM vendor has not kept up with innovation in the space. In any case, you have a SIEM and you need a new one.

Let’s look at the good side of the situation:

• It is very likely that you learned some super-valuable lessons from your previous SIEM experience (other people have to hire consultants to get to those lessons) and now can avoid the common purchasing process pitfalls (some discussed here, BTW)
• You have much more confidence while discussing confusing SIEM features with vendors – speaking from your previous SIEM experience (this alone will make your new SIEM purchase process much less painful)
• You have some semblance of the logging policy across the systems that log into SIEM – that puts you ahead of those organizations who are just getting their first SIEM or log management tool
• It is possible that you built some operational procedures around SIEM (such as for PCI DSS log review or other purposes) and those would be handy for a new SIEM as well
• If you have to write an RFP (as I discuss here), the chances are that your new RFP would be MUCH better and more likely to result in a good vendor short list
• Treat this situation as positive, think “I now know more than 90% of people buying a SIEM, thus my new SIEM project will be a success”

A few things to avoid and pay attention to:

• Suppress that “I’d buy anything but this crap” mentality – think “what problems will a new SIEM solve or solve better?”
• Avoid taking shortcuts (such as not doing a PoC); you are more knowledgeable, but not prescient…

How might a migration process look like? This assumes that you have already selected a new product, tested it in the lab and are ready for production deployment.

• Prepare to run both products for some time – this might range from a few weeks to months
• Draft the new SIEM vendor to help you migrate the data; after all, they are getting the prize
• Potentially, be prepared to keep the old SIEM running (without paying for the support contract, of course) or at least keep the old data backups – this becomes important if complete data migration is impossible due to architecture differences between the new and old SIEMs.  Ideally, your log management tool will hold raw log backups and so keeping the old SIEM in operation won’t be needed.
• One of the biggest migration efforts will be migrating SIEM content: reports, rules, views, alerts, etc. As well all know, such content is not really portable across SIEMs and you should be prepared to simply recreate all the custom content AND all the default content that you used in the the old SIEM and that the new SIEM might lack.

By the way, I have seen more than a few organizations start from an open source SIEM or home-grown log management tool, learn all the lessons they can without paying any license fees – and then migrate to a commercial SIEM tool. Their projects are successful  more often than just pure “buy commercial SIEM on day 1” projects and this might be a model to follow (I once called this  “build then buy” approach)

Possibly related posts:

• SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?
• How to Write an OK SIEM RFP?
• Log Management Tool Selection Checklist Out!
• On Choosing SIEM
• "So, What Should I Want?" or How NOT to Pick a SIEM-III?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• Logging, Log Management and Log Review Maturity
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases
• Whitepaper with detailed SIEM use cases (using a particular SIEM as an example)
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

About me: http://www.chuvakin.org

SANS is almost ready with their 7th Annual Log Management Survey, which would be unveiled at two SANS webcasts on April 25 and April 26 (both at 1PM EST / 10AM PST). The SANS log management survey is a useful measure of what organizations do with logs and how it changes year over year. SANS states that “organizations still want better access to their log data and better integration with third party security software and their SIEM systems and their Windows logs.”

I am allowed to share a few (very few!) bits from a report, but expect full analysis from me when it officially comes out. So:

• Collection has dropped way down among the most challenging tasks related to logs – now categorization, reporting, analysis and other higher level tasks show up as top challenges (good news!)
• Alerting / detection again trumps search / investigations as far as basic log use cases are concerned (it is definitely very interesting since post-incident search requires much less tuning than alerting)
• PCI DSS still rules the roost of “logging for compliance”… which mandate is #2? Well, wait for the survey to come out
• Windows logs still spell “t-r-o-u-b-l-e”, even after Windows Vista and new XML logging (only 10% are happy with it…): “analysis is the top problem that organizations have with Windows log management.” And Snare agent still rules.

Enjoy the webcasts and the report next week!

Possibly related posts:

• SANS Log Management Survey 2010 is Out!

About me: http://www.chuvakin.org

FYI, I have updated my list of free log analysis and log management on my consulting site. Here it is, reposted:

The open source log management tools are:

1. OSSEC (ossec.net)  an open source tool for  analysis of real-time log data from Unix systems, Windows servers and network devices. It includes a set of useful default alerting rules as well as a web-based graphical user interface. This is THE tool to use, if you are starting up your log review program. It even has a book written about it.
2. Snare agent (intersectalliance.com/projects/index.html) and ProjectLasso remote collector (sourceforge.net/projects/lassolog) are used to convert Windows Event Logs into syslog, a key component of any log management infrastructure today (at least until Visa/W7 log aggregation tools become mainstream).
3. syslog-ng (balabit.com/network-security/syslog-ng/) is a replacement and improvement of classic syslog service - it also has a Windows version that can be used the same way as Snare
4. rsyslog (rsyslog.com) is another notable replacement and improvement of syslog service that uses traditional (rather than ng-style) format for syslog.conf configuration files. No Windows version, but it has an associated front-end called phpLogCon
5. Among the somewhat dated tools, Logwatch (logwatch.org), Lire (logreport.org) and LogSurfer (crypt.gen.nz/logsurfer) can all be used to summarize logs into readable reports.
6. sec (simple-evcorr.sourceforge.net) can be used for correlating logs, even though most people will likely find OSSEC correlation a bit easier to use
7. LogHound (ristov.users.sourceforge.net/loghound) and slct (ristov.users.sourceforge.net/slct) are more "research-grade" tools, that are still very useful for going thru a large pool of barely-structured log data.
8. Log2timeline (log2timeline.net/) is a useful tool for investigative review of logs; it can create a timeline view out of raw log data.
9. LogZilla (aka php-syslog-ng) (code.google.com/p/php-syslog-ng) is a simple PHP-based visual front-end for a syslog server to do searches, reports, etc

The next list is "honorable mentions" list which includes logging tools that don't quite fit the definition above:

• Splunk is neither free nor open source, but is has a free version usable for searching up to 500MB of log data per day - think of it as a smart search engine for logs. Splunk includes a tool to extracting parameters out of log data
• Offering both fast index searches and parsed data reports, Novell Sentinel Log Manager 25 is not open source, but can be used for free forever as long as your log data volume does not exceed 25 log messages/second (25 EPS). Unlike splunk above, it includes log data parsing for select log formats and thus can be used for running reports out of the box, not just searching
• Q1Labs is also neither free nor open source, but is has a free version usable for managing up to 50EPS (roughly 2GB/day). It can be downloaded as a virtual appliance
• OSSIM  is not just for logs and also includes OSSEC; it  is an open source SIEM tool and can be used much the same way as commercial Security Information and Event Management tools are used (SIEM use cases)
• Microsoft Log Parser is a handy free tool to cut thru various Windows logs, not just Windows Event Logs. A somewhat similar tool for Windows Event log analysis is Mandiant Highlighter (mandiant.com/products/free_software/highlighter)
• Sguil is not a log analysis tools, but a  network security monitoring (NSM) tool, but it uses logs in its analysis.
• Loggly cloud logging service now offers free developer accounts (at loggly.com/signup) for their cloud log management service. The volume limitation is 200MB/day and retention time limitation is 7 days. If you'd like to collect and search your logs without running any software, this is for you.

P.S. I’d love to finally test GrayLog in my lab since it looks very promising, but – sorry – I was not able to get it to work Too much Ruby and Java for my Linux box… BTW, I got a couple more of fun new tools that I plan to test and then possibly add to this list.

P.P.S. Comment response will be slow, I am away from computers.

Possibly related posts:

• Original On Free Log Management Tools

About me: http://www.chuvakin.org

I did this fun presentation on log forensics (here) and the question of “original” (aka “native”, “raw”, “unmodified”) log events came up again. Since the early days of my involvement in SIEM and log management, this question generated a lot of delusions and just sheer idiocy. A lot of people spout stuff like “you need original logs in court” without having any knowledge about either logs or court – or forensics in general. Or, as I sometimes feel, even computers in general. 

So, WTH is an “original” event? Let’s explore this a bit. 

For example, let’s take Windows 7 Event Logs. Before you read further, without focusing too much on the real meaning of “original”, think what you’d consider an original event log record …

Is this original – the EVTX file itself:

Is this – an XML view via Event Viewer on the computer where the log is produced:

Is this – a “friendly” view in the same Event Viewer on the same “original” computer:

As you might know, the above view is actually enriched i.e. has new information added compared to the EVTX file. Does it break the originality?
What if the EVTX file is copied to another computer and then opened in an Event Viewer? It might look a bit different due to various ID dereference operation, and it might enrich the contents with slightly different information.

How about this – exported to CSV at another computer. Is this still original?

And what about the one that is converted to syslog in a similar fashion? What if, or horror, TABs are replaces with spaces?

So, what’s the lesson here?  Obsession about “original”, “native”, raw” logs is just not a useful pursuit and it dead-ends pretty quickly.

Instead, you need a clearly understood and documented path of all event records that unambiguously tracks all changes to event records (removals, addition of details, modifications of contents, new headers, etc), not fake and impossible quest for “originality.” For additional reference on trusting logs, check out what Eric Fitzerald wrote about log trust back in the days of his ownership of the Event Log.

Possibly related posts:

• Log Trustworthiness Hierarchy

About me: http://www.chuvakin.org

Two of my esteemed colleagues, Misha Govshtein of AlertLogic and Raffael Marty of Loggly had a bit of an argument over something fairly central to logging and log management, especially as it applies to the coming cloud wave. Let’s review what happened.

In 2010, AlertLogic  folks have submitted an IETF draft of what they called “Syslog Extension for Cloud Using Syslog Structured Data”. Draft is available here and AlertLogic team explanation of its mission and purpose can be found here and  here (unfortunately in MP3 form). The draft reads as if they are proposing a new cloud log standard since the very first sentence of the document is: “This document provides an open and extensible log format to be used  by any cloud entity or cloud application to log and trace activities  that occur in the cloud.”

Said draft has found its way to the CEE Editorial Board (via IETF list message) and has caused some interest and, dare I say, unrest. And some disagreements. Raffael Marty of Loggly has published his position on the draft here. Further exchange of opinions can be seen in comments here, as well as heard in the hallways of RSA 2011 conference.

What do I think of this? I think both of these renowned log literati are both right and wrong (at this point, somebody might say “Anton…you are such a consultant”… and I am )

Unquestionably, I believe that the idea of cloud logging having its very own special standard, completely disconnected from all other logs is misguided. Being disconnected from both the rest of the logging domain and current log standardization efforts (like CEE, XDAS, etc) only makes this idea more misguided. In essence, if you grab an example of a current bad application log, add “cloudiness” to it (more on this later) and then publish it as “cloud log standard”, you generate mostly hilarity and not value for the IT community. Logically, it goes like this:

1. Bad log + cloud ID = really bad cloud log.
2. Really bad cloud log + public IETF draft = really bad, standard cloud log, exposed in public
3. Really bad standard log in the cloud EXPOSED in public = stupidity
4. Stupidity –> funny blog posts from Anton, like, for example, this one.

This just reminds me of Chris Hoff saying “Cloud security suffers from the exact same siloed security telemetry problems as legacy operational models…except now it does it at scale.” In fact, here is an example from the draft:

Aug 16 13:34:18 [context aid="149683FC-8DF5-1004-E1A8-00000A000152"
   provider="example.com" rid="1:123"][transit client="172.16.1.82"]
   User authentication successful for 1:123

Would YOU like to spend your mornings analyzing logs like this? If you expose such examples in a purported standard draft, future generations of log analysts will hate you with a passion….

However!

I also happen to think that there are significant differences of logging from/at cloud computing platforms (whether SaaS, PaaS or IaaS) compared to BOTH traditional system logging AND distributed application logging. Cloud computing (as defined by NIST) has inherent multi-tenancy, elasticity, immediate provisioning and other fun properties, not found in traditional applications and platforms – whether distributed or not. All of these happen to affect accountability, auditability and transparency – all the goals logs serve – in a number of big ways. Thus, cloud computing must change how logging is done and it will change it. Specifically, adding a unique ID (“audit identifier which uniquely  identifies an external request for activity”) to logs in order to enable serves a useful purpose.

So, we must change logging for the cloud AND we must improve logging  everywhere through standard work. It will result in GOOD, USEFUL LOGS that ALSO WORK WELL IN THE CLOUD. The caveat? We need it sooner than CEE is finished and adopted on a broad scale. “CloudLog” effort contains useful ideas that need to be implemented in future logs produced by cloud framework components, but the method chosen (uncooked IETF draft choke full of bad log examples) deserves mostly ridicule…

About me: http://www.chuvakin.org

1. We plan for this to happen periodically, such as maybe every three weeks - recorded on Wednesday, posted on Thursday. However, due to our work schedules, irregularities occur all the time. If you have not seen or heard a new LogChat podcast for a few weeks, be aware that we are not dead; just busy taking over the world.
2. No, we are still not ready with transcribing and, yes, we still want it.  I did try Amazon Mechanical Turk, but it didn't turn to be as inexpensive as people claimed. If you have ideas for a good AND cheap transcribing service, we are all ears.
3. Please suggest topics to cover as well - even though we are not likely to run out of ideas for a few years.
4. Any other feedback is HUGELY useful. Is it too long? Too loud? Too rant-y? Too technical? Not enough jokes? Too few mentions of the "cloud"? Feedback please!

• LogChat Podcast 1: Anton Chuvakin and Andrew Hay Talk Logs
• LogChat Podcast 2: Anton Chuvakin and Andrew Hay Talk Logs
• LogChat Podcast 3: Anton Chuvakin and Raffy Marty Talk Logs
• LogChat Podcast 4: Anton Chuvakin and Andrew Hay Talk Logs

About me: http://www.chuvakin.org

Love those easy unscientific quizzes you see all over the Internet? Here is one such quiz on LOGGING and LOG MANAGEMENT that I created specially for LogManagementCentral.

Go check what you really know about logs and figure out whether you are a mere bunny logger or a log management ninja.

Result scales:

• Bunny logger (score of 10%)
• Eager log beaver (score of 20 – 40%)
• I know my way around logs (score of 50 – 70%)
• I changed my name to “Log Logger” (score of 80 – 90%)
• Log management ninja (score of 100.00% and nothing less!)

Don’t be afraid … I did put a couple of tricky questions in there.

About me: http://www.chuvakin.org

FYI, this piece has been especially created for LogManagementCentral (original post). It is reposted here for posterity.

Many organizations talk about “best practices” for security, log management, SIEM, etc. The definition of such practice is often fuzzy (and overrun by marketing influences…) but can be loosely related to what leaders in the field are doing today and what practices generally lead to great results. Following the same model, we can create a definition of a “worst practice”:

· What the losers in the field are doing today

· A practice that generally leads to disastrous results, despite its popularity

Here are some of the “worst practices” in the area of SIEM and log management that I have observed over the years:

1. Skipping the requirement definition stage of SIEM purchase is one of the worst, albeit common, practices one can take. It almost always leads to failed SIEM projects, unmet needs for customers as well as unjustified anger aimed at technology providers. “John said that we need a correlation engine” is not the way to define your requirements, by the way.

2. Postponing the environment sizing until the purchase is another generally disastrous practice. Even if you plan to eventually collect “everything”, the initial implementation will only have a specific smaller set of data. Careful sizing of that initial phase by watching your logs for a week or two is very important.

3. Choosing by price alone has led to many wrong purchases – and not only in the realm of SIEM. SIEM and log management products are priced from $0 to a few hundred to millions – and there is usually a difference in both capability and scalability between tools with dramatically different prices. Remember that tool can be 30% cheaper, but be “only” twice as bad…

4. “Saving time” by not checking references is another common bad practice at purchase stage. Your environment might be unique, but references is one of the few ways to know that the tool you’re planning to purchase has the will of somebody else. Skipping Proof-of-concept is even worse- that is your way to test a complex new tool in your environment!

5. Expecting the vendor to tell you what you need to log happens more frequently than you might think. Sadly, the only person who knows your needs and requirement for logging, log management and log monitoring is you – not the vendor. If you don’t know – then nobody does.

6. SIEM implementation is often a very “political” affair and thinking you can do it alone without involving others from you organization is definitely the worst practices. SIEM touches systems, network devices, possibly IdM systems and many other components – each with their own business owners and administrators. These people and teams have to be involved in SIEM implementation; and there is no way around it. Preparing the infrastructure is key for the deployment, even if you simply need to make sure that all log source systems has their time synchronized.

7. Ignoring your legal team is a quick way to FAIL with SIEM, especially if your project covers log data from multiple countries. Log data is covered by a conflicting laws and regulations and only your organization legal counsel can figure it out.

8. Deploying everywhere at once and not in phases is a way to run out of budget, management patience and other resources. Phased approach – both in terms of log source scope and SIEM capabilities (from simple to more advanced) – is the only way to go. Focus on “quick wins” in each phase.

9. The interface is “intuitive” so who needs training? Avoiding training is not the way to save money on a SIEM tool. SIEM and log management tools connect to many pieces of the infrastructure and applications. The vendor or consultants might teach you how to resolve many of these challenges, based on their experience with other customers

10. Not checking for changed needs as your SIEM implementation expands is another way to fail. Even though your SIEM may have a few problems, it does not necessarily mean that it can solve every problem you have. Notice how some organization deployed log management tools and then had to expand their deployments to full SIEM due to evolving needs. “We made the decision years ago – why fuss over it?” does not work for integration-heavy technologies like SIEM.

11. Finally, expecting immediate reduction in work after deploying a SIEM is unreasonable. Unless you deploy, customize and tune your system, it is likely that you will not see massive resource savings. SIEM is a great example of “to get value you have to work on it” rather than a magic box that “tells you what is wrong”…

What good or bad practices with SIEM and log management can you share?

About me: http://www.chuvakin.org

FYI, this piece has been specially created for LogManagementCentral (original post), an awesome site about logs, log management and SIEM. It is reposted here for posterity.

So, behold 11 log resolutions for 2011!!

1. I will turn logging on the systems I manage: this resolution is about the very first step one must take to using log data for many purposes inside and outside of IT – actually having logs. Start 2011 by committing to enabling logging across the systems you manage or oversee. And, yes, “log everything” is not the answer in most environments (and as all oversimplifications, it is often downright silly – e.g. log every SELECT on a database will lead to your DBAs killin’ ya ) – further resolutions help with figuring out how to do it without killing your systems

2. I will create log policy: this resolution helps you to make a commitment to understanding what you need to log on each system and how to do it. Logging policy starts from reviewing compliance requirements and other “use cases” for log data.

3. I will check for when logging stops: one of the simplest ways to commit to having logging in 2011 (and all years thereafter) is to commit to monitoring when logging stops. Apart from being a violation of a few regulatory compliance mandates, termination of logging – whether due to an attacker all by mistake – is something you need to know right when it happens.

4. I will use compliance intelligently: this resolution draws a line between being a checkbox-following “compliance monkey” and being convinced that “compliance is evil.” Regulation such as PCI DSS contain not just motivation but also some useful advice on how to do logging right (some ideas).

5. I will learn what the logs mean: committing to logs is not simply committing to having logs –you have to know what the log messages actually mean and what they are trying to tell you. In 2011, make sure who that you seek to understand what your systems are trying to tell you in their logs and learn to tell routine messages from critical “system-busting ” alerts.

6. I will at least check logs for intrusions, system and account changes and major errors: one cannot make a resolution to analyze logs without starting small first – if you have to look for some will things first, at least commit to check your logs for intrusions, system and account changes and major errors (this checklist can help)

7. I will review logs: generating, centralizing and storing logs is important. These practices a bed of sensible and mandatory (prescribed by many regulatory mandates). However, main log value lies in interpreting, understanding and then acting upon the information present in the logs. You cannot commit to logging excellence without committing to log review – using automated tools (lots of ideas on log review)

8. I will make sure that I have logs preserved after an incident: leads rarely matter more than in a hectic post-incident environment where every bit of data can help understand the origin and impact of the intrusion. Commit to using logs for incident response in 2011! (useful tips on that)

9. I will train my developers to create useful logs: making – and keeping!-a resolution to collect and review logs is impossible if logs do not exist – as it is often the case for your custom applications. In order to gain benefits of logging in such case, you must make a resolution to train your application developers to create useful logs inside their applications. Use emerging standards such as CEE to guide them towards proper logging practices

10. I will stop complaining about how bad logging is at most organizations: everybody starts somewhere, and many organizations start from a truly abysmal state in regards to logging. Start logging – and stop complaining. Go from log ignorance to near-real-time log enlightenment using a process similar to this

11. Finally, I WILL REMEMBER THESE RESOLUTIONS FOR THE ENTIRE YEAR: unlike some security technologies, logging, log review and log monitoring is a lifetime commitment. To get something useful out of log data, you have to log and review data all the time.

Any other logging resolutions you are making for 2011?

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you well know, tools alone don’t make anybody compliant!

This is the FINAL, 18th post in the long, long series  (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we end our Complete PCI DSS Log Review Procedures. Please start reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts:

References

The following references are useful for PCI DSS log review program and log management in general:

SANS CAG/CSC

“Twenty Critical Security Controls for Effective Cyber Defense: Consensus Audit Guidelines”

http://www.sans.org/critical-security-controls/

Specifically, the relevant control on audit logs is shown below:

“Critical Control 6: Maintenance, Monitoring, and Analysis of Audit Logs”

NIST 800-92 Logging Guide

“Guide to Computer Security Log Management: Recommendations of the National Institute of Standards and Technology by Karen Kent and Murugiah Souppaya”

http://csrc.nist.gov/publications/nistpubs/800-92/SP800-92.pdf

NIST 800-66 HIPAA Guide

“An Introductory Resource Guide for Implementing the Health Insurance Portability and Accountability Act (HIPAA) Security Rule ”

http://csrc.nist.gov/publications/nistpubs/800-66-Rev1/SP-800-66-Revision1.pdf

Appendix A Recommended Logbook Format

Logbook entry:

1. Date/time/time zone this logbook entry was started

2. Name and role of the person starting the logbook entry

3. Reason it is started: log exception (copied from log aggregation tool or from the original log file), make sure that the entire log is copied, especially its time stamp (which is likely to be different from the time of this record) and system from which it came from (what/when/where, etc)

4. Detailed on why the log is not routine and why this analysis is undertaken

5. Information about the system that produced the exception log record or the one this log exception is about

a. Hostname

b. OS

c. Application name

d. IP address(s)

e. Location

f. Ownership (if known)

g. System criticality (if defined and applicable)

h. Under patch management, change management, FIM, etc

6. Information about the user whose activity produced the log (if applicable)

7. Investigation procedure followed, tools used, screenshots, etc

8. Investigative actions taken

9. People contacted in the course of the log analysis

10. Impact determine during the course of the analysis

11. Recommendations for actions, mitigations (if needed)

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

FYI, this piece has been specially created for LogManagementCentral (original post), an awesome resource for all logging things. It is reposted here for posterity.

While many people have seen 10 things that your chef, real-estate agent, wedding planner or pilot won’t tell you, the world has not yet seen Top 10 things your log management vendor won't tell you. Finally, this gap is now closed.

1. “We talk analytics, but really, most of our customers use us for collection only.” While some products within SIEM and log management offer advanced analytics features, many of their customers are not truly ready for them. They need to start dealing with the basics – logging, log collection, log review before delving into advanced areas. Buying a product based on features you won’t use is a mistake. For example, see “Log Management Before SIEM”

2. “Our tool won’t make you PCI compliant. You’d have to do A LOT of things yourself – every day – to get and maintain compliance.” Sadly, many security solutions – and SIEM / log management are no exception – are sometimes sold as “compliance in a box.” You need to be aware that to stay PCI compliance you need to do more than purchase tools. For example, see “How to Stay PCI Compliant”

3. “No, you cannot buy an entire SOC in this small box.” Just as with compliance, you cannot buy an entire Security Operations Center in a box, big or small. However, some will try to sell you their SIEM as “SOC-in-a-box.” Running an effective SOC includes multiple processes and procedures which are just as necessary as a market-leading SIEM tool

4. “We are cloud-ready, because … mmmmm… well, we are ready for it!” Many vendors will tell you that their tools are cloud-ready – without really thinking what they mean. Effectively monitoring traditional and multi-tenant cloud environments distributed across regions and countries requires more than updated marketing materials. You need to carefully test the tool in your own hybrid environment before concluding that it is “cloud ready”

5. “Our SIEM is really just a renamed log management tool. But that’s all you probably need.” The confusion around SIEM and log management functionality rages on – it also allows some tools to be sold as SIEM without having any critical SIEM functionality such as correlation and real-time dashboards. Even though it might be all many customers need, it does not make such tool a SIEM tool. For additional reading, see this whitepaper.

6. “We can do everything with logs, but it might require some SMALL customizations. Our PS team is standing by!” More than a few SIEM vendors will promise support for every possible log – including logs they have never seen. However, fully integrating a new log source for reporting, correlation and visualization will always takes work and cannot be taken for granted.

7. “If you make a mistake with capacity planning, we’d be happy to sell you more log management than you really need.” Many organizations are having trouble estimating how much log data will be coming into their SIEM or log management tools. Both under as to making and overestimating are common. It is recommended that you spend about a week measuring log volumes across the systems that will be reporting to a SIEM.

8. “We think our tool is scalable, but we don’t really have production customers of your size. Our engineers believe that it might work.” Scalability claims are cheap and would often be made by SIEM and log management vendors. However, the only real proof that the tool will scale to your requirements is testing the tool in your environment. Thus, you should insist on performance testing during the pilot if there are any doubts.

9. “Out tool offers predictive security intelligence. No, we don’t know what it means either – and we can’t really predict it.” SIEM is one of the most over-hyped and over-marketed security technologies out there. The only way to get the tool that satisfies your requirements is too carefully spelled out those requirements and then test the tool yourself.

10. “We estimate our performance using really small log messages sizes.” Yes, our tools can do a million message an instant – but these are our special messages that we create in the lab. Nowadays, application logs and proliferation of XML-based logging has pushed the message sizes up to 1 kb or more from a traditional 200 byte logs from firewalls. Thus, you need to be wary of performance estimates based on such artificially short logs.

So what is your vendor NOT tellin’ ya?

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you undoubtfully know, tools alone don’t make anybody compliant!

This is the 17th, one before last,  post in the long series of 18 posts (part 1, part 2, part 3 – all parts) – this is a very important part as it contains the summary of key periodic operational procedures. Please consider reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts. A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures:

Periodic Operational Task Summary

The following chapter contains a summary of operational tasks related to logging and log review. Some of the tasks are described in detail in the document above; others are auxiliary tasks needed for successful implementation of PCI DSS log review program.

Daily Tasks

The table below contains daily tasks, responsible role that performs them as well as what record or evidence is created of their execution:

Task	Responsible Role	Evidence
Review all the types of logs produced over the last day as described in the daily log review procedures	Security administrator, security analyst, (if authorized) application administrator	Record of reports being run on a log management tool
(As needed) investigate the anomalous log entries as described in the investigative procedures	Security administrator, security analyst, (if authorized) application administrator	Recorded logbook entries for investigated events
(As needed) take actions as needed to mitigate, remediate or reconcile the results of the investigations	Security administrator, security analyst, (if authorized) application administrator, other parties	Recorded logbook entries for investigated events and taken actions
Verify that logging is taking place across all in-scope applications	Application administrator	Create a spreadsheet to record such activities for future assessment
(As needed) enabled logging if disabled or stopped	Application administrator	Create a spreadsheet to record such activities for future assessment

Weekly Tasks

The table below contains weekly tasks, responsible role that performs them well as what record or evidence is created of their execution:

Task	Responsible Party	Evidence
(If approved by a QSA) Review all the types of logs produced on less critical application over the last day as described in the daily log review procedures	Security administrator, security analyst, (if authorized) application administrator	· Record of reports being run on a log management tool. · Record of QSA approval for less frequent log reviews and reasons for such approval
(As needed) investigate the anomalous log entries as described in the investigative procedures	Security administrator, security analyst, (if authorized) application administrator	Recorded logbook entries for investigated events
(As needed) take actions as needed to mitigate, remediate or reconcile the results of the investigations	Security administrator, security analyst, (if authorized) application administrator, other parties	Recorded logbook entries for investigated events and taken actions

Monthly Tasks

The table below contains daily tasks, responsible role that performs them as well as what record or evidence is created of their execution:

Task	Responsible Party	Evidence
Prepare a report on investigated log entries	Security analyst, security manager	Prepared report (to be filed)
Report on observed log message types	Security analyst, security manager	Prepared report (to be filed)
Report on observed NEW log message types	Security analyst, security manager	Prepared report (to be filed)
(If approved by a QSA) Review all the types of logs produced on non-critical applications over the last day as described in the daily log review procedures	Security administrator, security analyst, (if authorized) application administrator	· Record of reports being run on a log management tool. · Record of QSA approval for less frequent log reviews and reasons for such approval
(As needed) investigate the anomalous log entries as described in the investigative procedures	Security administrator, security analyst, (if authorized) application administrator	Recorded logbook entries for investigated events
(As needed) take actions as needed to mitigate, remediate or reconcile the results of the investigations	Security administrator, security analyst, (if authorized) application administrator, other parties	Recorded logbook entries for investigated events and taken actions

Quarterly Tasks

The table below contains daily tasks, who performs them as well as what record or evidence is created of their execution:

Task	Responsible Party	Evidence
Verify that all the system in scope for PCI are logging and that logs are being reviewed	Security analyst, security manager	Recorded logbook entries for review and exception follow-up
Review daily log review procedures	Security analyst, security manager	Updates to logging procedures; change log
Review log investigation procedures	Security analyst, security manager	Updates to logging procedures; change log
Review collected compliance evidence	Security analyst, security manager	Compliance evidence; evidence review log
Review compliance evidence collection procedures	Security analyst, security manager	Updates to procedures; change log

Annual Tasks

The table below contains daily tasks, who performs them as well as what record or evidence is created of their execution:

Task	Responsible Party	Evidence
Review logging and log review policy	CSO	Policy changes; change log; policy review meeting minutes
Review compliance evidence before the QSA assessment	PCI DSS compliance project owner	Meeting minutes or other record
Live tests with anomalies	As needed	Logs or other records of such tests

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

Management Reporting

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

This first-ever dedicated log management class teaches system, network, and security logs, their analysis and management and covers the complete lifecycle of dealing with logs: the whys, hows and whats.
You will learn how to enable logging and then how to deal with the resulting data deluge by managing data retention, analyzing data using search, filtering and correlation as well as how to apply what you learned to key business and security problems. The class also teaches applications of logging to forensics, incident response and regulatory compliance.
In the beginning, you will learn what to do with various log types and provide brief configuration guidance for common information systems. Next, you will learn a phased approach to implementing a company-wide log management program, and go into specific log-related tasks that needs to be done on a daily, weekly, and monthly basis in regards to log review and monitoring.
Everyone is looking for a path through the PCI DSS and other regulatory compliance maze and that is what you will learn in the next section of the course. Logs are essential for resolving compliance challenges; this class will teach you what you need to concentrate on and how to make your log management compliance-friendly. And people who are already using log management for compliance will learn how to expand the benefits of you log management tools beyond compliance.
You will learn to leverage logs for critical tasks related to incident response, forensics, and operational monitoring. Logs provide one of the key information sources while responding to an incident and this class will teach you how to utilize various log types in the frenzy of an incident investigation.
Finally, the class author, Dr. Anton Chuvakin, probably has more experience in the application of logs to IT and IT security than anyone else in the industry. This means he and the other instructors chosen to teach this course have made a lot of mistakes along the way. You can save yourself a lot of pain and your organization a lot of money by learning about the common mistakes people make working with logs.

This is a special beta course whose materials are still being fine-tuned. We are offering it at a discount at this event in exchange for the students' detailed feedback, which will help us improve and finalize the course's content and exercises.

A laptop with Windows XP or later or recent Linux operating system installed which can unzip/gunzip compressed files. CD/DVD drive is required. MacOS is not acceptable.

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you undoubtfully know, tools alone don’t make anybody compliant!

This is the 15th post in the long, long series  (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures (please consider reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts):

PCI Compliance Evidence Package

Finally, it is useful to create a “PCI Compliance Evidence Package” based on the established and implemented procedures to show it to the QSA. It will help establish your compliance with three key of PCI DSS logging requirements:

· Presence and adequacy of logging

· Log review

· Exception handling

While it is possible to prepare the evidence package before the assessment, it is much easier to maintain it on the ongoing basis. For example, keep printed or electronic copies of the following:

1. Logging policy that covers all of the PCI DSS in-scope systems

2. Logging and log review procedures (this document)

3. List of log sources – all systems and their components (applications) from the in-scope environment

4. Sampling of configuration files that indicate that logging is configured according to the policy (e.g. /etc/syslog.conf for Unix, screenshots of audit policy for Windows, etc)

5. Sampling of logs from in-scope systems that indicate that logs are being generated according to the policy and satisfy PCI DSS logging requirements

6. Exported or printed report from a log management tools that shows that log reviews are taking place

7. Up-to-date logbook defined above

This will allow always establishing compliant status and proving ongoing compliance.

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org

Just quoted from here:

Christmas in May: Take the SANS 2011 Annual Log Management Survey

Take the 7th Annual Log Management Survey and be entered to win a $250
American Express Gift card.

This comprehensive survey has become a
leading indicator of how well log management and automation helps
organizations with their security and compliance needs. To take our
survey, follow this link: http://www.sans.org/info/68369

The results will be released in early May during a short series of live
webcasts with Jerry Shenk and Dave Shackleford.

Do the survey, please. Past years results have been very insightful due to good participation.

Possibly related posts:

• SANS Log Management Survey 2010 is Out! (contains last year survey analysis)

About me: http://www.chuvakin.org

Once upon a time, I was retained to create a comprehensive PCI DSS-focused log review policies, procedures and practices for a large company.  They are focused on PCI DSS compliance, but based on generally useful log review practices that can be utilized by everybody and with any regulation or without any compliance flavor at all. The practices can be implemented with commercial log management or SIEM tools, open source log analysis tools or manually. As you undoubtfully know, tools alone cannot and do not make anybody compliant!

This is the 14th post in the long, long series  (part 1, part 2, part 3 – all parts). A few tips on how you can use it in your organization can be found in Part 1. You can also retain me to customize or adapt it to your needs.

And so we continue with our Complete PCI DSS Log Review Procedures (please consider reading from Part 1 – at this stage we are deep in the details and these sections might seem out of context without reading earlier parts):

Example Logbook Entry

Here is an example following the above pattern:

1. Date/time/time zone this logbook entry was started: November 23, 2009, 4:15PM PST

2. Name and role of the person starting the logbook entry: Anton Chuvakin, principal consultant.

3. Reason the logbook entry is started: log exception (copied from log aggregation tool or from the original log file), make sure that the entire log is copied, especially its time stamp (which is likely to be different from the time of this record) and system from which it came from (what/when/where, etc):

Time/date of log: 10/21/2009 10:01:23 PM PST

System: OLGA.example.com

4. Detailed on why the log is not routine and why this analysis is undertaken: this event ID (Windows event ID 11) from this application event source (Source crypt32) was never seen before on any of the systems where logs are reviewed across our organization.

5. Information about the system that produced the exception log record or the one this log exception is about

a. Hostname: OLGA.example.com

b. OS: Windows XP SP 3

c. Application name: N/A

d. IP address(s): 10.1.1.1

e. Location: Home office

f. Ownership (if known): Olga Chuvakin, President and CEO

g. System criticality (if defined and applicable): critical, main laptop of the executive

h. Under patch management, change management, FIM, etc: yes

6. Information about the user whose activity produced the log: N/A, no user activity involved

7. Investigation procedure followed, tools used, screenshots, etc: procedure for “Initial Investigation” described above

8. Investigative actions taken: following the procedure for “Initial Investigation” described above, it was determined that this log entry is followed by a successful completion of the action logged. Specifically, on the same day, 1 second later the following log entry appeared:

This entry indicates the successful completion of the action referenced in our exception log entry and thus no adverse impact from the error/failure is present.

9. People contacted in the course of the log analysis: none

10. Impact determine during the course of the analysis: impact was determined to be low to non-existent; no functionality was adversely affected, no system was at risk.

11. Recommendations for actions, mitigations (if needed): no mitigation needed, added this log entry to baseline to be ignored in the future as long as the subsequent log entry exists.

The logbook of that sort is used as compliance evidence since it establishes log exceptions follow-up, required in item 10.6.a of PCI DSS validation procedure, which states “Obtain and examine security policies and procedures to verify that they include procedures to review security logs at least daily and that follow-up to exceptions is required.”

The logbook (whether in electronic or paper form) can be presented to a QSA or other auditor, if requested. I recommend retaining the log book for 3 years or at least 2x of the log retention period (1 year for PCI DSS)

To be continued.

Follow PCI_Log_Review to see all posts.

Possibly related posts:

• Incident Log Review Checklist
• All posts tagged PCI_Log_Review

About me: http://www.chuvakin.org