Appliances are really software with hardware vulnerabilities

When you or your managed IT service provider (MSP) buys a fancy new managed appliance, you expect it to come secured out of the box with automatic updates. That might apply to the software that you bought; but it doesn’t apply to the computer inside the case.

The problem

Even though that fancy new box says Datto, Barracuda, Nutanix, or another brand on the outside of the box, chances are that it’s a basic SuperMicro, American Megatrends, other brand of computer inside the box. The company outside of the box or your MSP might be diligent in patching their own software and the fancy cloud infrastructure that supports it — but they probably forgot about the computer they sent you.

How easy is it to attack?

A simple network scan will probably reveal an Intelligent Platform Management Interface (IPMI) on that device. The username and password are probably admin and admin. What’s worse? It’s probably on the standard unencrypted web port 80 so the password wouldn’t be encrypted if you logged into it.

Since you and your MSP have likely never logged into the IPMI, you likely haven’t upgraded the firmware. In most cases, this means that it dates back at least five years with many well-known vulnerabilities that even the least technical attacker can download.

What can bad guys do?

Attackers who can spell “admin” can certainly turn off the device and might be able to access the underlying system as an administrator.
Since most companies treat cloud-based backup devices as “set it and forget it” devices, a backup device that’s turned off might not be caught for months. In many cases, remote console-level access is possible. Since many companies only use Multi-Factor Authentication (MFA) for remote access and not for console access, this makes direct system access much easier – especially since most companies don’t change the console credentials either and rely on web interfaces. This means data theft, corruption, ransomware, or mining for cryptocurrencies and slowing down anything else running on the device.

If the firmware hasn’t been updated, the device can probably be compromised even if the password is changed.

We couldn’t believe our eyes and asked directly

We asked senior engineers at Datto and Nutanix about this issue. Datto said that it was the MSP’s responsibility to change the default IPMI and BIOS passwords. Nutanix said that they are a software company and aren’t responsible for hardware (even though they brand it and sell it as an all-in-one hyperconverged solution). They said that it was the end user’s responsibility. We asked them if they told their clients that they needed to do this. They said no.

What should you do?

1. Change your IPMI and BIOS passwords today.
   You might need to coordinate with your MSP but it’s a low-risk change with a huge payoff.
2. Unplug IPMI, DRAC, ILO, and other management ports if you don’t use them.
   If your MSP needs to access them for remote management, have them call you to plug in the cable. Mark the port and the cable with a bright sticker so you can recognize them to plug them in during emergencies and updates.
3. Add firmware to your monthly patching cycle.
   Your patching tool might not catch these patches so it might need to be a manual process and it’ll likely require a reboot and downtime to patch, but it could also save your company if we’re talking about the hyperconverged device that runs the company or the backup device that you’ll need when ransomware hits.
4. Test your backups.
   Does your system spin up snapshots and take screen captures? Great.
   Do you have some automated scripts running to make sure that the snapshot’s database and applications are working right? Great.
   Do these results go into a log that nobody checks or get funneled into a folder in your email that you don’t read? Not good.
   Make sure to do simulated failovers once a quarter and check those logs as frequently as you can justify. Weekly or monthly checks are a great place to start.