Interview: Jeff Schilling, Chief Security Officer at FireHost

Today we're talking with Jeff Schilling. Jeff is the Chief Security Officer at FireHost, and prior to that, he was a director of the global Incident Response practice for Dell SecureWorks. Colonel Schilling retired from the U.S. Army, where in his last assignment, he was the director of the Army's global Security Operations Center under the U.S. Army Cyber Command.

Jeff and I discuss his tenure in the Army doing global Security Operations, Monitoring, and Incident Response, for over a million computer systems. Jeff explains the challenges of bringing network security to a stack that was not designed to be defended. We talk about his current position at FireHost, what makes them different, and why companies can benefit from moving their applications from their internal IT staff. A lot of companies out there are good at running a network for their primary line of business, but the challenges lie in running a secure one. Jeff also shares his thoughts on the future of DevOps and application security and what he sees as the new trends moving forward.

The following is a brief excerpt of our interview.

Jeff Williams: Now, you recently wrote that a lot of network security teams are kind of like bicyclists who go out wearing a bike helmet, but don't bother to strap it on. I've seen this a lot. Skateboarders just love to wear the helmet, but they don't actually strap it on. You know, if they wreck, it just goes flying off. And you said those network security folks have a false sense of protection. Tell me more about that analogy. Why do network security people believe that they've got a helmet on, but really don't?

Jeff Schilling: Well, I think when you go back and you kind of unpack all these major security breaches that happen, what you'll find is that they actually had the technology in place. In a lot of cases, there are one or two problems: either the technology wasn't properly employed, or the technology was not actively monitored; and so, you know, the technology detected it, but no one did anything about it.

I spent a lot of time when I was in the consulting business talking to companies that were - because I have a very, what I guess would be described as a very strong SOC background, - running security operation centers, and then I would sit down and I would say, "Okay, who is daily looking across all the functions that you do across security operations?"

Everything from your vulnerability threat management, to your threat intel, to what you're seeing off of your security controls and the events you're getting, and it's pulling all that together and synthesizing and coming up with a plan of action in what you're doing. And what I found in most cases is that there was no one. You read the job description of a SOC manager and it is all about, you know, management stuff, and not so much about operational stuff.

So I think that a lot of companies think that the technology that they buy is going to protect them, and don't understand that it's the trained people and processes that's actually more important than the technology sometimes. You know, I say to our guys around here - Tiger Woods can pick up my golf clubs and go play scratch golf; I can't pick up Tiger Woods' golf clubs and play scratch golf. [laughs]

At some point, you've got to put the investment into the people and processes, and I saw that as a huge gap across multiple industries, multiple verticals of people spending the time to develop these to make sure that their people had the training that they need, and they had the right experts in the right place, and then had the process to actually execute.

Jeff Williams: Interesting. So, I thought it was fascinating about the level of monitoring that you can get over your networks. I'm wondering if you felt like you have the same level of security insight into the applications that are running on those networks? I've always thought that, you know, network security and application security are a little different to me. It's different people, it's a different set of skills, it's, you know, kind of different vulnerabilities.

So what I'm wondering is, do you feel like you could see the security in the application layer the same way that you get visibility into the security of the network layer?

Jeff Schilling: I kind of look at it this way. I don't know if you remember the old movie called "The Predator" . . .

Jeff Williams: Yeah.

Jeff Schilling: . . . there's that one scene where the Predator's walking around and you can't see the guy that he's chasing, and he changes from infrared, the heat-seeking, you know, the heat signature, he changed his lens.

Jeff Williams: Right.

Jeff Schilling: He was looking for the threat. And that's kind of the way I view the difference between application layer monitoring and networking. You're absolutely right; they're two different skill sets, and probably one of the hardest skill sets to find in the security business are people that really understand how to do network security monitoring and network forensics, especially. That's one of the things I found in Instant Response practice, is to find anybody that really knows how to go take a PCAP dump and really go through it and figure out what's going on in the network. But I think it is important if you want to see everything, is to be able...you know, like the Predator [laughs] did in that movie, be able to flip between those two different types of views to be able to see what the threat actors are doing. You know, third lens of that I would put on there that you didn't mention is that host level indicator.

The things that happened on the host that aren't necessarily always signature-based, but are anomaly-based.

Jeff Williams: Yeah. My feeling is that we don't have that lens for application security. There is no good way to flip on a view that can show you access control violations and various application attacks, like XXE and XXS and SQL injection and so on. A lot of times they're invisible because we don't have the right sensors in place to look for them.

Jeff Schilling: I think that we have some pretty good success. Again, we have such a tight architecture for our customer environment that, you know, in very well-defined aggregation points, that it's easy for us...and we actually have outer parameter, what we call our core web application firewall, which is just looking at the OWASP top 10, and then our customers actually have additional control to be able to write custom, to basically be able to tune to their application; and that's what it really takes, is someone with the knowledge to be able to tune to their application and know what should be allowed and what shouldn't be allowed.

We have a second customer control that's actually down in their customer environment firewall that they can tune, and I think that's probably the best strategy, especially the multi-tenant environment like ours because we can't be everything to everybody.

Jeff Williams: I couldn't agree more. Actually, I think it's so critical to tune application security tools to the applications because I always say, "Every application is a beautiful and unique snowflake", and so you've got to customize and tailored the protections for that particular app.

Jeff Schilling: Yeah, and that's what we recommend to our customers, but we do try to protect them at the perimeter as well, and then for cases like what we're seeing with this Bash vulnerability, it also gives us a central point to protect our whole...you know, put a compensating control. I mean, our guys literally wrote a custom signature last night...

Jeff Williams: Yep.

Jeff Schilling: ...and so I'm sitting here looking at our...inside of our SOC, I'm looking at our spikes [laughs] on the...

Jeff Williams: [laughs]

Jeff Schilling: ...things bouncing off of those signatures, and it's still pretty active.

Jeff Williams: Well, I'm glad we're protected. Hey, let me ask you about this, a little different kind of question. So you've got such a strong background in operational security. What do you think about the recent trend towards DevOps? Is it possible to have a security DevOps, or some kind of rugged DevOps? I'm interested in your thoughts on how development and operations can work together to achieve security.

Jeff Schilling: Well, I think that your software engineers are starting to get more and more savvy about security, and I think that they're seeing that, you know, because they've just got to do it right the first time. I actually spent about four hours with one of our customers this week. I mean, this guy was a software engineer, and we were talking...you know, I was going over our Intelligent Security Model that we apply for our customers, and I was pretty impressed with his level of understanding, and not only application security, but also...I mean, he really had a good grasp of the networking security strategies as well because they host in our environment, they do their dev in our environment, as well as their production, so it was a great conversation.

Jeff Williams: Let me ask you one last question. I'm interested in what you see over the next couple years. How do you see the security landscape evolving?

Jeff Schilling: Well, I think that we're going to see threat actors continue to do the same things, because the [laughs] same things are working.

Jeff Williams: [laughs]

Jeff Schilling: You know, actually over the last five years, I haven't really seen that many innovations in new tradecraft or tools; a lot of it is just kind of rehashing the tools and tradecraft they have. Now what I have seen change with the threat actors, are the threat actors are getting more operationally savvy in having better operational security themselves. It used to be, if you were a security...you know, if there was a hacker forum and there were 10 guys sitting in the hacker's forum, probably seven of them were either F.B.I. or security researchers.

You know, these guys are trading tradecraft back and forth, and they're talking. The security guys are coming and writing papers about it, and then boom, you know, but the threat actors have gotten much better at holding the cards closer to their chest, and I think that the most innovative thing that I saw about the major breach that Target had was not so much the tradecraft, but just the thought and the approach that they used to get a foothold in that network, and then use the infrastructure to spread their malware, was just pure genius. You know, the actual tools that they used was not that sophisticated. I think that we will see the threat actors to continue to up their game and that, and I think that we'll start seeing better software, better hardware that's going to make it tougher for them.