Doug Lhotka

Technical Storyteller

Opinions and commentary are mine, and do not reflect those of my employer.

(C) Copyright 2019-2023
Doug Lhotka.
All Rights Reserved.
Use of text, images, or other content on this website in generative AI or other machine learning is prohibited.

Commander’s Intent

By

So all your preventative measures have failed – to be fair, they succeeded for the last few thousand hacks, but the bad guys got lucky once, and you now have a full blown incident underway.  Unfortunately you (the CEO) is at 23,000’ knocking K2 off their bucket list.  How does your company execute?

Let’s expand the scenario a bit more.  Turns out it’s a bad one, with serious implications for business operations and significant customer impact.  Decisions need to be made right nowto mitigate and respond – both technically and to your customers.  The security team has their runbooks, notification trees, and incident response plans in place (you do have all that, right?). They’ve notified PR, Legal, the CISO, and the on-deck line of business leadership.  Each of those teams is assembling and starting to launch their own parts of the plan.  So far, so good.

Now’s where it usually breaks down.  You have hundreds of angry customers calling on the phone, and they all want resolution. Response plans rarely extend to business operations, let alone to customer remediation; most organizations try to use existing day-to-day processes, which fail miserably.  When there’s a major disaster, hospitals change their workflow.  They don’t look for insurance cards – they treat the wounded.  Does your call center?

This is not the time to parse expense authority though five layers of management with graduated clip levels, let alone try to run your day-to-day customer care plans.  During a crisis, the goal has to be to resolve the customer’s situation on the very first call.  You might get away with one level of escalation – as long as hold times are short and calls don’t drop – but as soon as you have to call them back, the customer will be fuming, and probably calling your competitors.  And woe to the bottom line if they aren’t called back as promised.   Goodwill doesn’t come back easily, if at all.

Avoiding this starts at the very top.  The commander’s intent has to be clear, concise and easy to understand.  During a recent ransomware outbreak, the CEO told the entire staff to ‘make it right for the customer, we’ll cover the cost’.  Full stop.

Now if you have a strong command and control culture, I’ve probably just caused a heart attack.  But the point is clear – you need a different set of rules on deck when a disaster – cyber or otherwise – strikes.  On declaration, the teams break glass on the case, crack the code books, and execute a streamlined workflow that includes escalated authority for the duration of the crisis.

The next time you do a cyber range drill or tabletop exercise, include an angry customer in the scenario.  See what happens.  I’ll bet that in most organizations your staff will either resort to daily procedures, platitudes, playing hot potato, or just wing it.  Very few teams have the modified workflow in place to handle a disaster when it strikes, let alone have a clear statement of their commander’s intent.

Do yours?

Don’t Poke the Buffalo

By

Ranchers know that there’s little that can stop a determined buffalo – barbed wire is at best a suggestion. That’s why taunting or poking one is just a bad idea (the guy in Yellowstone recently was extremely lucky). Malicious actors can be a bit like that too – that’s why I’ve written before that ‘hacking back’ is a bad idea for all but nation states, but what about as individuals?

Folks in the security and technology industry often get called by obvious (to us) scammers like Windows Tech Support or ‘Rachel from Card Services’, which are two that have plagued my phone with junk calls.  Clearly we’re not going to bite, but should you try to keep them on the phone as long as possible, with the intent that it prevents them from scamming other people?  Tempting…very tempting.

There’s two classes of bad folks out there – the creative, innovative, highly skilled adversary, and what used to be derisively called the ‘script kiddie’.  The latter may or may not have a highly skilled actor behind him pulling the strings, or might just be a run of the mill boiler room operation – mass market cybercrime.  The problem is that you don’t really know which one just called you.  And that’s why as tempting as it is to engage them and burn their time, I recommend against it.

For the vast majority of these scams you’re just another number in the machine.  So when you don’t bite, it’s not unexpected and they just move on.  But do you really want to move from anonymous in the herd to being singled out for attention? Is it worth the risk of angering someone who already has questionable morals, and now decides to target you for specific attacks? You turn a run of the mill attack into a battle of wills against a potentially highly skilled adversary.  That’s why I have tremendous respect for folks like Brian Krebs who do active investigations of malefactors.  He’s a full time professional, and spends a decent amount of his time going above and beyond to protect himself.  For example, he’s recently started decoupling his phone from his accounts, due to SMS spoofing attacks, and has had his home swatted. Do you really want to have to go to that level?  I’d rather not.

I have a lot of respect for our adversaries.  Sometimes that’s due to deep technical capability, and others it’s just because they can make life miserable.  Either way, engaging isn’t something you’re really going to win or make a difference doing. So next time a scammer calls, just hang up.  Don’t poke the buffalo.

How much should you spend on security?

By

(c) Dreamstime / Mosich.com

I regularly get asked by new CISOs for information – benchmarks – on how much organizations like theirs should spend on security.  That’s a deceptively simple question, and while there’s plenty of surveys that you can reference, none of them provide more than a rough starting point – there’s just too many variables.

The classic measure, ALE=ARO*SLE, is a nice fiction when it comes to cybersecurity.  Sure, we all learned it for the CISSP exam – Annual loss expectancy equals annual rate of occurrence times single loss expectancy.  Put simply, if there’s 100 houses in your neighborhood, and 10 burn down a year, that’s a 10% ARO.  If it costs $100K to rebuild the house (we’re ignoring the land and possessions here), then the single loss expectancy is $100k. Multiply and we get $10K per year, so spending up to that amount on fire prevention makes sense.   Over simplified, but you get the gist.

When it comes to a breach, things break down.  Looking at SLE first, there’s hard costs (fines and settlements, and cost of response), soft costs (brand damage).  So far so good – we can calculate the former with a fair degree of accuracy, especially with GDPR and the new California and Colorado legislation.  Soft costs are more difficult, and there’s some argument to be made that they’re much lower than most studies would claim.  Target, Home Depot, Anthem, Equifax, and all the rest may have taken a short term hit, but lasting brand damage has been hard to find, let alone calculate.  Still, for arguments sake, let’s stipulate that we can get to some reasonable semblance of a number.

Where things break down is on probability – annual rate of occurrence.  These concepts came from risk management techniques used to price insurance and similar financial vehicles.  We have a decent historical knowledge of the frequency and severity of accidents, fires, floods, hurricanes and so forth.  While any given event may be an unexpected black swan to an individual, at a population level, the actuarial data is quite sound.

But cybersecurity isn’t there yet.  We’re just at the beginning of the storm and most events are still a black swan even at the population level.  Sure we all know that we’re targets, but our sample size isn’t large enough to do broad analysis that we can leverage.  That’s because unlike all of the other categories, with cyber we have active adversaries working against us.  And remember, they only have to get lucky once.  We have to be perfect – one failure and we lose.  Put it another way, to paraphrase my favorite Jedi Master – It’s data loss, or loss not.  There is no loss partial.

So are we out of luck? Not quite.  There are some things that can guide us.  First, there’s two baselines – audit and hygiene. Spend what it takes to pass the audit.  At the same time, there’s general things that every organizations should be doing today and would be considered negligent if they don’t.  Encrypting laptops, having a firewall, patching quickly, and so forth are more or less in that category.  Call it security hygiene.

Next, it’s time to do that data classification project you’ve been putting off.  Yep it’s hard, but you need at least a rough idea of where your crown jewels (close the door) and critical (expensive to lose) data are located.  Of course, you might need to really think about what those are before going to look (it’s not always what you think).  From that you can do rough calculations about cost of loss, and take what are called ‘commercially reasonable measures’ to protect the information. That’s a fuzzy concept, but there’s usually enough of an idea (and enough work to be done) that it’ll keep you busy for a couple of budget cycles.

Third, is awareness. Do you really know what’s going on in your environment.  Do you have good instrumentation sensors that tell you what’s going on, and can you collect and process that to detect anomalies?  SIEM is one part of it, but security intelligence goes much deeper. Once you have it of course, you need to be able to take action on it, but that’s another topic.

Next is the most amorphous one of all.  I commented recently on a post on LinkedIn, and noted that I essentially presume that all of my personal information is already gone.  I still try to protect it of course, but I can’t avoid doing business with companies that have either poor security practices, or have lost it to active attack (one doesn’t always imply the other) – mainly because I can’t control where my data goes.  That fox left the henhouse when my college professors posted grades by SSN outside their offices, and it’s only gotten worse from there.

So what do I mean by the amorphous one?  It’s the ethical and moral aspect.  How would you want your information protected?  Or your parents?  Or your kids? Ironically some of the best protected data (credit cards) is the least damaging to the person if it’s exposed. Losing a card number is annoying. Losing an SSN is a much bigger deal, but that has already happened for essentially everyone.  Losing medical records, well, that’s a whole different story. Ask any security professional in your organization and they’ll have a pretty good idea if you’re doing a reasonable job protecting information or not.

Last, is the reality check. How much can we afford to spend and stay in business?   How much can we not afford to spend and stay in business?

Put all those together and we have a decent working model to determine our security budgets.  It takes auditors and accountants, technicians and data analysts, attorneys and regulators.  But in the end, it takes a mirror – when we look into it, are we confident we’re doing the right thing for our shareholders and our customers (or products if you’re a data broker)?