A series of unfortunate commits: software security stories

In March, news broke that two malicious commits were introduced onto the in-development PHP 8.1 branch by perpetrators pretending to be Rasmus Lerdorf and Nikita Popov. The response by the PHP community to this attack was a shining example of the openness and transparency of the open-source world.

We’re going to take a closer look at the PHP 8.1 story to uncover what lessons there are to be learned on the value of visibility in coding. We’ll also retell tales of exploits past whose events were as notable but whose conclusions perhaps not as fortunate.

If you popped in here on your lunch hour to peruse a cheery collection of passed tests and green checks, click away now to something more pleasant. If you are still interested in the maintainer misfortune that follows, then, my macabre and welcome guest, please pull up a seat.

The Zeroed Zerodium: stolen identities and PHP 8.1

In March, two malicious commits were introduced into the PHP source repository. The first, pushed by the imitator of PHP creator Rasmus Lerdorf supposedly to fix a typo in the zlib extension, instead introduced an injection that allowed PHP code to execute within the User-Agent header if it included the string zerodium.

Because it was so blatant, so entirely conspicuous, and so obviously a case of assumed identity, it was immediately reverted by the volunteers who watch over that repository. But then, the faux Nikita Popov reverted that revert commit, reintroducing the zerodium block.

After applying the revert of the revert of the revert, the community, in an effort that characterized the transparency of the entire response, acted quickly to prevent similar exploits.

• The PHP internals project immediately and publicly addressed the event, offering details on the suspected vulnerability (a compromise of the git.php.net server), explaining the containment actions being taken (abandoning private Git server maintenance, and making the GitHub mirror canonical), and supplying clear instructions for how to restore contributor access. Anyone from the PHP community could join the conversation, ask questions, and suggest new practices, such as requiring a signed commit, for avoiding similar events in the future.
• New releases were put on hold as the investigation continued, and users were offered reassurances that the malicious commits never reached their projects.
• Maintainer Sara Golemon sat in on a popular PHP podcast to further inform the community about the timeline and details of the investigation and to clear up the confusion being spread by click-bait articles misrepresenting the event.
• The project then further updated the community that it no longer considered git.php.net to be the source of the event. Rather they suspected that the cause was a leak of the user database for master.php.net in combination with insecurely stored passwords that supported HTTPS authentication on commits. Subsequently, the project removed HTTPS auth, reset all php.net passwords, and updated the vulnerable server and moved it to a new home on main.php.net.

The commits, however unfortunate, were contained, and the maintainers handled it like pros. Here are some of the lessons we can learn from their success:

• Involve your community throughout a security event, including its identification and each step of your response.
• Even if they’re not the source of the leak, maintaining services that are not your project can be more trouble than it's worth. Services increase the maintenance burden and lengthen onboarding. Plus they add places for bugs and potential vulnerabilities to hide. Leave service maintenance to the experts and focus on the work.
• If accounts are affected, reset everyone's password, and make them create a new one. Don’t risk leaving it to the users to take this action.

The Dreaded Drupalgeddon2: sanitize all the things

Next is perhaps the most infamous of recent security events, so much so that it shares its name with the end of days (as well as with the highest-grossing asteroid disaster movie of 1998).

Drupal is a tremendously popular content management system. (In fact, Platform.sh itself was born from support for Drupal commerce.) It's open source and has accepted community contributions for almost twenty years. However, it's had its share of security events along the way.

Drupalgeddon, which occured in 2015, affected millions of Drupal-powered sites. A database abstraction API that sanitized queries was found to be, with the right requests, vulnerable to privilege escalation and arbitrary code execution via SQL injection. It was a big deal, so big in fact that subsequent vulnerabilities of that magnitude continued to carry on its nickname.

Four years later, Druid's Jasper Mattsson reported Drupalgeddon2, which put about one million Drupal installations seriously at risk.

• SA-CORE-2018-002 had a Highly Critical risk score of 24/25.
• About one million Drupal 8, 7, and 6 sites were vulnerable (~9% of the sites running Drupal).
• The vulnerability involved the Form API introduced in Drupal 6, as well as some of the generalizations applied to it in Drupal 7.
• Data could be injected into unsanitized fields visible to anonymous visitors to the site, forcing Drupal to re-render the page, which allowed malicious users to add modules to the installation and assume complete administrative control over the site.

So what’s the takeaway from the Drupalgeddons? The Drupal community did step up upon their discovery. Site maintainers were alerted and provided with patches to fix the issues in their same release. But in the world of open-source software, there’s no guarantee that a maintainer will respond quickly enough to secure their sites before an exploit is attempted. For this reason, many hosting providers maintain Web Application Firewalls (WAFs) that seek to filter out malicious requests specific to these exploits.

In the end the only take away may be to keep your ear to the ground for these alerts when they occur, and do your part to widen the broadcast so no one gets caught.

The Committed Credentials: weak credentials affect everyone

Our next story reveals how a lack of password constraints for npm, the Node.js package manager that's slowly annexing your hard drive, along with poor and widespread Git practices put half of the community in jeopardy.

In 2015, Nikita Andreevich Skovoroda, a member of the Node.js Technical Steering Committee, performed a scan on GitHub search and npm. Afterwards, he was able to obtain over 100 passwords and nearly 200 tokens for accounts associated with a number of frequently installed packages.

With these credentials, he could either publish to npm or sometimes even push to the repository on GitHub. This was the case for not only for those projects he was able to get credentials for, but also for other popular packages that depend on them, such as Express and Bower.

What was the source of these exploits? Committed credentials in .gitconfig, .env, .npmrc, and even in some cases keys .ssh. All the things meant to stay local and private were committed and included in the final npm package. After Skovoroda reported his findings, npm added tools that scanned packages for these kinds of unintentional commits and revoked them when found.

And for a while, all seemed well.

But then Skovoroda asked himself, "How many passwords out there are simple enough to guess or to brute force?" He reworked his search tools to look for npm accounts using password and sequential numbers.

The results? Well, from his original report:

Thanks to the community’s simple habit of setting simple passwords, Skoboroda found himself in control of a massive portion of Node.js code using npm to manage its dependencies. He then submitted his report, demonstrating his continued commitment to the community, as well as the fact that he had no inclinations for world domination.

npm responded by reseting every affected password and strengthening requirements so that simple passwords would no longer be accepted for accounts.

The takeaways here are nothing new: don't use simple passwords or reuse passwords on multiple accounts. More than that, it's essential that you place thoughtful requirements on the passwords users can register with in the first place, because a lack of constraints can make every user vulnerable.

The Buried Baron:

Our final entry involves a malicious commit that modified the default behavior of sudo and managed to go unnoticed for nearly ten years.

At some point in your life you've likely come across sudo—and probably used it too, despite the well-known warnings against doing so. Those warnings are warranted. sudo allows a user to take on the permissions of another user, most often root, giving them complete control over everything on that computer. However, it's sometimes necessary to use sudo, which is why it comes pre-installed on most Unix-based operating systems across the world.

This combination of ubiquity and exploitability made our last vulnerability particularly dangerous.

In January of 2021, the security firm Qualys announced that, sticking to sudo's default configuration, it was possible for an unprivileged user to gain root privileges on any host by exploiting a commit introduced nearly ten years earlier. This was huge. Any computer or server that contained sudo could be completely taken over by a malicious user, who could then delete data or modify software for subsequent attacks.

Baron Samedit (a nickname that combined the vulnerable part of sudo—sudoedit—and the Haitian loa of death Baron Samedi) even shipped with the latest version of Mac OSX. The attack surface was massive, and so a highly coordinated response was planned to patch it.

• Qualys disclosed the discovered vulnerability to the sudo maintainers privately.
• Patches and detailed advisories were sent to the Openwall Project for maximum visibility.
• A coordinated release of all materials was made thirteen days after the original disclosure.

The actions were kept secret while they needed to be. Then as soon as a fix was in hand they were broadcast as far and loudly as possible. There's not much to consider, however, as to what could have been done to avoid it or to prevent similar vulnerabilities appearing in your own code. Sometimes there just simply aren't enough eyes available to catch everything.

Platform.sh takes pride in providing environments that are secured against most of the heinous acts described above. Bad requests get blocked, file systems are locked down, and we're always reassessing what should and shouldn't be made available at runtime to give you the most secure applications possible. But eventually, upstreams can get updated or neglected. Constant vigilance is a necessity in our communities to guard against the evil-doers out there.

So, stalwart readers, having listened to these morbid yet instructive tales of woe and ruin, what must we do to keep future such nightmares at bay? Foremost, do what you can—contribute to those upstreams, share security releases when you're made aware of them— so that we may all continue to sleep soundly. Rest assured that Platform.sh will be doing our part too. If you ever have any questions about the actions we’re taking to keep your code safer, please feel free to contact us.