There are recon tools, and there are recon tools. @tomnomnom—also called Tom Hudson—creates the latter. I have great respect for large, multi-use suites like Burp, Amass, and Spiderfoot, but I love tools with the Unix philosophy of doing one specific thing really well. I think this granular approach is especially useful in recon. Related Talk: Mechanizing theMethodology Basically:…
ffuf is an acronym for “fuzz faster you fool!”, and it’s a cli-based web attack tool written in Go. Veteran web testers might think of it as Burp Intruder on the command line. The hardest thing about ffuf is figuring out how to pronounce it. It’s just “fluff”, without the “l”. Once you get the main concept, it’s…
Last Sunday night, while I was lounging on the couch watching some British Bake Off, I got word of the Solar Winds supply chain hack. After kicking back the last of my whiskey, I immediately got on the phone to start IR at work, cuz, yep, we have Solar Winds too.
Who’da thunk it?
Anyway, three days of IR stuff later, I am here to blog on the meanings for the muggles out there after having a conversation with a reporter on what it all meant. The reporter asked me about a tweet that was put out by Richard Blumenthal about needing to know more about this evolving hack and fallout thereof.
While I think that Dick is being a bit hyperbolic here, I also can tell you, gentle reader, that there is a lot to in fact be worried about regarding this instance of adversarial activity (most likely Russia’s APT29 Sluzhba vneshney razvedki Rossiyskoy /SVR group) which managed to break into a system application that many in the government, military, and corporations still run to manage their network.
This system is so prevalent in the space, that even in my environment, we still had it running and man, I thought we had made it go away long ago. So, you might be wondering what does Solar Winds really do? Well, glad you asked, it is a series of applications that help you maintain your large networks.
As you can see from the graphic from their site, the companies software performs a lot of management and monitoring capabilities within a network of individual systems. Servers, routers, databases, service desk applications, resource monitoring, network configuration, and security management. Now, you might be saying; “Ok, well, those are a lot of things that this stuff does, but, what does that mean security wise if the application (Orion) is compromised?” and that is a good question, the primary one I want you to comprehend if you are not in tech or security of the tech. What this means, is that this program suite by SolarWinds, is the ‘skeleton key’ now to a host of around 33k companies/networks that downloaded the tampered with update. This could affect around 300k clients in all, should there be more tampering or vulnerabilities exploited by the adversary now that they have the code base (assuming here) after they spent all that time inside SolarWinds systems.
So, we have a rather prevalent application suite that usually functions on a level of administrative access to do the very things it is bought to do. This means, that the Orion system contains ALL of your admin passwords up to and including domain administrator and enterprise administrator. What does this mean? It means that once the adversary had control over the Orion system, they had control over EVERYTHING that that system touched as well as now, if it did not have direct control, the passwords that would allow access within a network running this compromised system, are in the hands of the enemy.
Put simply, the adversary, has control over pretty much everything you own. They can log in, take data, manipulate data, and in the most extreme, burn your network down using other malware like a wiper or ransomware to do it. All of this, while you may not see the activity because everything is using credentials that are admin level and authenticated on your network. This is why it was so hard to detect this attack and to stop it and why they were inside the systems for so long.
Ok, so, what does that mean from the perspective of damage and about what groups the adversary hit? Well, so far, we know that the following entities were hit in this supply chain attack(s)
- Department of Homeland Security
- The National Security Council
These are all either government agencies or companies that handle a lot of government contracts, so you can kind of get a sense of what it means. However, let me expand on this, DHS and the NSC alone is a treasure trove for the Russians to gather all kinds of unclassified/classified data that they would want. Not only that, but, if you own the Orion systems in places like that, and that systems is in fact running in the CLASSIFIED space, then you have broached into the CLASSIFIED networks of things like NIPRNET and SIPRNET as well probably JWICS.
What does this mean? Lemme put it into internet vernacular for you;
This could be spectacularly bad. This is why so many are freaked out about this supply chain attack and the incident responses are all going on 24×7 now. It has yet to really be determined (at least publicly) how long the adversaries were inside these networks, but, I am going to assume that it was a long time, and a lot of damage has been done. Now all these places have to clean up the mess, re-set their networks and rebuild so that this cannot happen again. Then they have to assess the real damage to our security and perhaps someday give testimony in congress about it.
Now, about the other entities, these are the reasons that this hack is bad;
- FireEye: They do all the pentesting and security work for many of the same orgs as well as incident response. If they were owned as hard as we think, well, there is a lot of data that the adversaries could use on top of using all the tools they stole from them.
- Treasury, well, money right? Plans? Routes? All things monetary that the adversaries could use to mess with the united states up to and including theft of large sums of money potentially.
- Commerce as well, plans and other details that they could use against the US financially internally as well as globally.
Time will tell just how many other orgs got hit and may in fact have had data lost to the attackers. Also, do not forget the potential for further logic bombs out there that might be placed by the actor as well for future fun. Of course I have been hearing stories about power and water companies and systems being affected by this as well. All in all, it could be very bad for us all, and places us in our back foot most solidly globally.
One other aspect here, and this is highly speculative, but, what other secret orgs had connections to others with Orion? What orgs themselves in the secret spaces like FireEye, had the same software as well? What classified intelligence has been lost here?
Let that sink in…
Also, on the critical infrastructure end, I am not worried that the power will go off nationally, but, the Russians could mount more, and working attacks against regions with the right kind of access vis a vis this kind of hack.
Think about that too.
Gotta hand it to the Russians man, they play a good long game. Expect to be hearing about fallout on this for quite a long time. If you want to kind of get a sense of the scope of this, I would recommend watching “Sneakers” the whole McGuffin of the movie is the little black box that the mathematician created that decrypts all the things. This hack is kinda like that. With one box, the Russians decrypted EVERYTHING and then, like the Grinch, took it all up the chimney.
Here’s a reading list too for you all to follow along with:
Someone put out a tweet earlier that is very prescient;
This is an important context to have. Russia has used Ukraine as their down range test bed. If you remember back to NotPetya, you can see this exact supply chain attack cycle being leveraged there first, and tested. The Russians are old hands at this now.
- Adding Authentication
- HTTPS Listeners
- Tunneling SSH
- Tunneling RDP
- Serving Directories
Introduction to Ngrok
This works because Ngrok is calling outbound, and meeting its other side on the internet.
Ngrok is an application that gives you external (internet) access to your private systems that are hidden behind NAT or a firewall. It’s basically a super slick, encrypted TCP tunnel that provides an internet-accessible address that anyone can get to, and then links the other side of that tunnel to functionality running local.
Here’s what it does:
- You run
ngrokfrom a local system with a service you want to make available to people on the internet
- Just run the command and give it the protocol you want to use, along with the local port it’s listening on
- Ngrok then creates an address in the cloud that people can get to
- It then connects those two things over an encrypted tunnel, so when you hit the Internet address, you land on your local service automagically!
Just because hackers use something doesn’t make it automatically malicious.
Two of the examples they give on the site include: 1) public URLs for sending previews to clients, and 2) demoing local functionality with external people.
Those use cases are definitely cool, but if you’re in security like I am, the main benefit is granting external access to internal systems once you’ve, um, gained access.
Ngrok simplifies what used to require lots of trickery, usually involving SSH.
The other cool thing about Ngrok is that it allows you to see the HTTP traffic that’s being tunneled over it via a separate interface, which is by default hosted on
http://127.0.0.1:4040. This especially helps the normies that are using it legitimately for testing and troubleshooting (?).
Let’s say you have Apache or Nginx listening on localhost:80.
To start the application in the most basic mode possible, simply invoke it and tell it the port your local web server is running on.
ngrok http 80
By default Ngrok creates a random subdomain, but you can use the paid version to custom domains and subdomains.
So with this one command you now have a public URL that you can hit from anywhere in the world—that will then access port 80 on your localhost. That’s epic.
Hand-washing after using the bathroom and a decent password are highly recommended.
If you’re going to make stuff behind the firewall (that may or may not have been compromised in some way) available to the internet, you might want to make some attempt to lock it down. You can do that with the
ngrok http -auth=”youruser:yourpassword” 8181
This will at least make it so that people can’t just walk in the front door.
Forwarding to HTTPS servers
This is different than the internet endpoint, which automatically has an HTTP option.
By default your targets are unencrypted (HTTP), so if you want to send to an internal server that uses TLS, you’ll have to specify that.
ngrok http https://localhost:8181
One of the most popular security (ab)use cases.
One of the most flexible things you can do—at least on a Linux-based host—is give yourself SSH access to the system. Because once you have SSH access you can do whatever else you need to from there.
- Make sure SSH is running on the localhost, which is usually on port 22.
ngrok tcp 22
Keep in mind you can do this with any port, e.g., Postgres, MySQL, etc.
Hopefully with SSH and RDP you have authentication already on the landing point, but that says nothing about the quality of the credentials.
ngrok tcp 3389
Serving a directory
Be careful with this, obviously.
You can also specify a local directory, and have that served via the web interface. Super slick. Super scary.
ngrok http https://localhost:8181
Ngrok is fantastic because it simplifies external access to internal things through the magic of outbound connections.
- Tell Ngrok what to push up to the internet.
- It gives you an internet place to go.
- Go there an access the internal resource.
Always use authentication, especially where the endpoint doesn’t have it already.
For web developers wanting external access to an internal web server:
ngrok http 80
For security types wanting an external tunnel to an internal resource:
ngrok tcp 22
- The ngrok.io subdomains appear to have decent entropy, but I wonder if anyone’s trying to search for unsecured endpoints. Of course they are.
- A friend and I were talking about how Ngrok (or any) connections to localhost:22 could be a decent IOC signal.
- This tutorial will get you started, but the Ngrok documentation is some of the best I’ve ever seen on a tool, so more information definitely check that out. More
- Another thing to consider about Ngrok, similar to third-party VPNs, is that Ngrok is closed-source, so you should be aware of what you make available through it. It’s not that open-source automatically means secure, but when you shouldn’t have extreme trust in a closed-source, free tool that offers encrypted tunnels. They could do evil (for whatever reason), get compromised, etc. Not a red flag, but a yellow one.
Become a direct supporter of my content for less than a latte a month ($ 50/year) and get the Unsupervised Learning podcast and newsletter every week instead of just twice a month, plus access to the member portal that includes all member content.