> If everyone sets a cooldown, there will be no early adopter. Everyone is waiting for everyone else to be the canary, and the canary does not exist.
Except there are researchers chomping at the bit to download every new release of packages in search of vulnerabilities they can exchange for fiat, reputational credit, or both.
I do appreciate these security companies a lot (for example, Snyk), but I feel it hard to believe this is sustainable. Especially in the current world where LLM is devaluing security researchers' work.
If they cannot get enough fiat or credit, this will eventually turn to some incident like OpenSSL heartbleed or so.
It's bold to call something that has prevented real-world attacks "security theater", with the justification that one day the world might change enough that it's less likely to prevent future attacks. Was the printing press worthless, just because we have laser printers nowadays?
Anyhow, If you believe LLMs will kill security research, then surely you must also believe a cooldown of 1 week would be enough time for your favorite LLM to analyze a new package version?
What do you call the time period between "new package is released" and "automated security scanners have analyzed the package"? That sure sounds like a release cooldown to me.
I applaud you if you do setup automated security scanners, without counting on external security groups or individuals (that doesn't have a security contract with your company).
This post is based on an assumption from what I see (I would be very happy if it's wrong) that most companies do not event bother to do these scans. They are merely waiting for the free kindness.
LLM is only devaluing security work by competition. As in, it's making dependency scanning so easy and cheap that researchers don't have much left to do on that front. The end result is very easy and cheap dependency scanning available to everyone. It makes the next heartbleed less likely, not more (though actually the class of bugs it protects against is closer to log4shell).
I'm confused by this take. In general, LLMs don't replace the expertise required to use an LLM.
Knowing the right input to give the LLM is not enough, nor does it ever end. The LLMs themselves need to be continuously updated and/or specialized for use case, and the "right input" is a moving target.
Keeping both ends of this aligned is the exact same expertise we already had before LLMs. These implementation details would only impact the credibility of those researchers who don't adapt to using this new tool. Who would stumble over something as trivial as this? LLMs aren't even the most important tool in the toolbox anyway.
I agree that LLM does not (and believe it never) fully replace researchers, but it produces artifacts (e.g. writeups, PoCs) at a cheaper price.
That makes me think LLM impacts human researchers' rewards, but now I realized the result might actually go in the opposite direction according to Jevons paradox. People still need experienced and professional human security researchers after all.
I would honestly hope that the LLM usage would help with that very thing. If AI systems automatically perform security audits on brand-new NPM releases, doesn't that accomplish a similar purpose?
It's worded a bit strongly, but it encourages critical thinking about downloading dependencies, how vetting should work, and where the human factor comes in. I think the top comment was right that some security researchers will audit packages no matter what, but it's good to not take that for granted.
> it's absurd to think that nobody is going to pull releases before the default cool down period.
Why is that absurd? It seems like a more likely outcome than just assuming there are enough "security researchers" out there doing this for every package?! IMO both sides here seem to have no evidence that the other is wrong.
Because I can always bump up the cooldown period. I'm not naive in thinking that I never have to worry about anything with a cooldown period, but having rolling releases is inherently what cooldowns give you. Companies have been doing rolling releases on their own products for years because it's a lot easier to handle when something falls over for a small percentage of users that when in nukes everyone all at once. That same concept still applies here.
Because security companies/researches are doing just that. The vast majority of recent supply chain attacks have only lasted minutes to hours. A large portion of them were credential losses or tag takeovers that aren't hard to detect, but by the time you do, your CI/CD has already downloaded and ran the attack 200 times.
Actually, you make an excellent point. Account takeovers have been responsible for a number of recent high profile attacks. Even if there are 0 security researchers reviewing new releases, just having time for the real owner to announce that a release was fraudulent before it's downloaded by half the Internet is a huge boon.
This entire article seems to say that if it doesn't prevent 100% of all possible attacks that it's "security theater". That's nonsense. It's just one more tool in the tool belt, and a highly useful one at that.
That's better than nothing, but if you're expecting people to be looking at something before you use it, you should be waiting for a thumbs up from them, not waiting some arbitrary timespan for the absence of a thumbs down.
I don’t think it makes a ton of sense for individual OSS maintainers to pay companies for security. The point of cooldowns is that they’re free and they trend towards security at no logistical cost to maintainers.
(I agree that companies should pay for security products, of course. The bigger problem there is that so many security products are terrible, and supply chain products appear to be no different as a class.)
It's a game of probabilities. Waiting longer means a higher probability that someone found something before you upgrade.
Most of the attacks we have seen lately are from legitimate projects which got compromised. The maintainers of those projects are generally looking out for that.
So I went down the rabbit hole of the meaning of "champ", thinking it might mean something esoteric in the context of "champing at the bit" like "speaking the first line of the skit". But no, "champ" as a verb is just an alternative version of "chomp", or at least a word that differs very subtly from "chomp".
The migration of "champing at the bit" to "chomping at the bit" is inevitable.
There don't seem to be many idioms like this, where the newer version is both semantically intuitive/meaningful and accepted by authorities. The closest is probably "hone in on", but the verb form of "home" as in "homing device" is still current, unlike the verb form of "champ". Another interesting one is the migration from "just deserts", using an archaic meaning of "desert" as in something deserved, to the modern form "just desserts". However there's much greater distance between "desserts" and the archaic meaning of "deserts" than between "chomp" and "champ".
The proliferation of these posts with such obviously terrible argumentation almost has me believing that they're a psy-op from some malicious actor (such as a US agency) which doesn't want people to improve their security. I mean why would you post drivel like this if you don't benefit from the status quo.
The most likely explanation is obviously that it's just a bunch of attention seeking amateur. But damn.
I don't agree with this take, there are a bunch of security companies that are screening new versions of packages looking for vulnerabilities, so waiting a bit before upgrading to the lastest package is miles better than immediately upgrading.
A decent amount of recent issues follow the pattern where someone's credentials are compromised, those compromised credentials are used to release a malicious package, and then the original maintainer is the one to notice it but it takes them a little while to get the bad package pulled. A cooldown pretty much completely solves that problem.
Beyond that, a lot of these cooldown criticisms assume everyone will use the same length, which isn't how it has to work. There are people who run Arch because they want the latest version of everything, and there are people who run Debian because they never want a new version to break their system. Some people camp out for the newest iPhone on day 1, and some people wait to see reviews and find out if the antenna is sensitive to how the phone is held. Adjustable cooldowns let everyone choose their own balance between getting the latest features and getting something that is tested.
Isn't this what alpha/beta/RC releases are for, ensuring that there are no issues before a general release? I just don't get ecosystem has decided to use client-side configurations for this rather than just marking the releases in the ways that already exist.
The archaic model where every tool gets full access to your home directory was never designed for the scenario where a single npm install will install and execute 100s of packages written by various authors.
I also believe sandboxing will get more and more important. There might be some trade-off on user experience or convenience, but given the security enhancement and (LLM agent's) freedom I think it will be well worth it.
Yes I highly suggest working in a container or setting up selinux. Selinux is especially powerful for this kind of thing but also requires a lot of advanced knowledge and skills to setup and maintain sadly.
I don't disagree with this take. But I'm not going to risk installing a package uploaded 1 hour ago considering all of the compromises this year. I think a cooldown period is prudent to let the automated security scanners to do their thing.
It also includes time to allow discovering of compromised accounts. If the owner of a popular package is hacked, this gives the owner a week to raise alarms about this.
How do posts like this make it to the top of HN? Nearly all the comments here are saying the same thing I thought:
1. Security researchers and companies are incentivized to install packages early.
2. Even without security patches, people and teams can still set the cooldown period that is appropriate to their risk tolerance. Even if someone in the wild does "get bit", it's better that a smaller number of people get bit than everyone getting bit at the same time. That's not "security theater", that's limiting the blast radius.
I'd also add that adding in cooldown periods lowers the incentives for bad guys to try to compromise a package in the first place because there is a much greater chance all their work will be for naught.
So I started reading this post with interest, thinking it would have some reason I hadn't thought of, but it just boils down to the faulty premise that "Godot is not coming". Bullshit, at the very least I now have agency over where I am in the download timeline. I just don't understand how such bad posts built on the flimsiest of assumptions make it to the top of HN.
I have no idea either. Every argument I hear about why wait times won't work are just factually wrong and using terrible assumptions they have zero insight into.
And, the complainers are free to set the wait timer to 0 and go back to the way it was. I'll be glad to let them be the first penguin to jump in the water.
Depending on those vaunted “security researchers” to take point and find all vulnerabilities in all packages is also fairly naive. That process costs money and produces a fairly valuable result - why would they give it away for free? So this will inevitably evolve into targeting only the high-value most popular packages and the exploiters will retreat to the long tail of less-popular ones. Lower payoff? Sure, but beats zero payoff.
Even setting aside that there are, in fact, many human and robot eyes staring hard at the firehose of published packages on popular registries, the whole argument feels like a false dichotomy.
If you're going to "DYOR" or whatever, having your automated dependency management on a cooldown timer seems like a snazzy way to smoothly provide a window for all your fancy dependency validation to happen.
The conventional meaning of “security theater” requires performance instead of effective action. But the author doesn’t reach that meaning: they claim that cooldowns don’t universalize, which then gets misconstrued into “security theater.”
I don’t agree with either claim: I don’t think they’re security theater, and I do think they universalize, contingent on there being security parties that are incentivized to scan packages.
The latter is a big assumption, but I think it’s a win for us either way: if supply chain companies are actually capable of defending us, then we win. But if it turns out they can’t serve as the cooldown vanguard, then we have great evidence that they shouldn’t be trusted at all.
Thanks for your explanation on the definition of "security theater"!
> But if it turns out they can’t serve as the cooldown vanguard, then we have great evidence that they shouldn’t be trusted at all.
If they cannot serve as the cooldown vanguard, we are indeed going to realize they are not trustable, but by the time the damage has already been done. Therefore, if companies want to prevent the damage as much as possible, I believe they should do the scan by themselves.
I don’t agree about the “damage being done”: there’s no evidence that a cooldown is itself harmful. The worst thing it can do to you is deliver some updates to you slower-than-instantly.
I agree cooldown is not harmful in general. What I intended to say is that people (mostly) only know a security company is not trustable because they once failed to detect a malware package, and the reason people know a package is malware is that it has caused damaged to at least someone.
Unlike most commenters here, I actually agree with most of this criticism. But there is one simple thing the delay gets you, which is a window for the dev to go on Twitter and announce that they were hacked.
We're so close to a full circle moment. Next up is "npm stable", a curated set of packages (a "distribution", say) known to work well that are supported for a couple of years with security patches.
I actually proposed a cooldown for apt-get in my PhD thesis in 2013 and calculated its impact. Because the meta-data for the Ubuntu apt repository can be completely downloaded you can simulate the impact of a cooldown on how many less packages would be installed and it isn't zero.
There are always people in a community running beta version, or up to date versions. There will be less with a cool down, but then depending on how critical your system is you could adjust cooldown length
What a poor take. Cooldowns certainly don't solve everything but 'out of the box' even with no vetters they'll help catch unauthorised releases on active projects because it's more time to notice.
Account takeovers, deployment exploits whatever the root cause, given a few days it's incredibly likely maintainers will notice they've released an unexpected update and fix/sound an alarm. A lot of existing cases like this people realise pretty fast, but the packages might still be live for a few hours as NPM catches up. Cooldown entirely removes that 'updated at the wrong time' issue, an no-one even needs to actually look at the code at all.
For inactive or intentionally malicious maintainers, yes it relies on vetting. But it's hardly a mythical 'canary' - there are already a lot of companies scanning every popular npm package, auditing new ones, or just generally analysing threats. I can think of at least 4 such companies without even trying, I'm sure there's loads more.
And they'll continue because it's not warm hearted so much as promoting/testing their security scanning services and tools so they can get the kudos from being the first to spot X etc. Which often does hit the front of HN or reddit, but they don't need the massive cyber loudspeaker to be effective. Flagging the issue to maintainers, reporting to NPM security - with a cooldown in place the package can be voided before anyone really gets the chance to install it. This is literally how a lot of things are spotted already, just with that 'updated at the wrong time' issue catching unlucky people out.
Does it solve everything? No. Will it reduce the amount of problems that make it to end users? Obviously yes.
And for all the author's 'no one will do this', 'waiting for Godot' cries - what do they recommend? Running your own audits, using LLMs, static analysis and testing it yourself. Basically being the veter they claim no-one will be! Unless they don't intend to sound the alarm themselves, they are literally the canary they're looking for.
Cooldowns aren't magic, but they're not security theater either.
I wonder if the author has ever worked in IT. The idea of not moving quickly to new software versions has been around for decades.
There are always early adopters and researchers looking at new, publicly available software. Critical industries have long waited for software releases to be stable and "recommended" by the vendor before upgrading, letting those early adopters who can afford some availability risk take time with it.
The observation that "if everyone in production has a cooldown, the benefit is gone" is an absolutist opinion.
I have seen this occur in some fashion - my company's product has a software version that's been out for almost six months and we still tell customers in critical industries to stay on the older version out of an abundance of caution and with the understanding of what upgrades require for them. Stability > features.
> The idea of not moving quickly to new software versions has been around for decades.
There are COBOL users and CentOS 6 users. They aren't affected by this issue.
They might need to face another set of issues like vulnerability backporting and so on. People not worrying about cooldowns might still need to patch their log4j libraries, but that's another thing. This post is not for them.
> There are always early adopters and researchers looking at new, publicly available software.
I agree with that. These hacks are very unlikely to go unnoticed, especially for popular packages (although they do happen; `chalk` has 3M downloads per week when it was hacked).
> The observation that "if everyone in production has a cooldown, the benefit is gone" is an absolutist opinion.
Yes. However, I think this is eventually becoming some type of pointless arms race, as cooldown does not act actively towards reducing ecosystem attacks.
To add a somehow random example, pnpm's default cooldown time is 1 day. Yarn's 7 days. I'm not sure whether one day pnpm would also change it to 7 days because of people asking.
>think this is eventually becoming some type of pointless arms race, as cooldown does not act actively towards reducing ecosystem attacks.
The nature of some of these attacks, it does. If your developers machine with publishing rights gets rooted a cool down makes credential gathering attacks via script in npm so very much harder it's not even funny.
In some of the recent supply chain attacks the entire fiasco was only a few hours long, but in that time tens of hundreds of thousands of credentials may have been stolen.
One day timers have the weakness of weekends and holidays reducing the number of eyes/systems seeing the issue.
I think the author is just totally ignorant, there's a whole community of security companies and researchers who are scanning every single package as it's released and have been doing so for some time now. Some of the details like how long the cooldown should be for are certainly up in the air, but the numbers have reasoning behind them.
I think this is such an nefariously unnecessary negative argument.
Most, if not all, of the shai-hulud attacks that hit npm and other ecosystems were preventable with cooldowns. And these were not detected because regular users reported the worms, but because security researchers did. I don’t think I’ve ever seen an attack that was discovered because a user reported it.
Except there are researchers chomping at the bit to download every new release of packages in search of vulnerabilities they can exchange for fiat, reputational credit, or both.
Anyhow, If you believe LLMs will kill security research, then surely you must also believe a cooldown of 1 week would be enough time for your favorite LLM to analyze a new package version?
> Run LLM-assisted audit on vendored code.
This post is based on an assumption from what I see (I would be very happy if it's wrong) that most companies do not event bother to do these scans. They are merely waiting for the free kindness.
I'm confused by this take. In general, LLMs don't replace the expertise required to use an LLM.
Knowing the right input to give the LLM is not enough, nor does it ever end. The LLMs themselves need to be continuously updated and/or specialized for use case, and the "right input" is a moving target.
Keeping both ends of this aligned is the exact same expertise we already had before LLMs. These implementation details would only impact the credibility of those researchers who don't adapt to using this new tool. Who would stumble over something as trivial as this? LLMs aren't even the most important tool in the toolbox anyway.
That makes me think LLM impacts human researchers' rewards, but now I realized the result might actually go in the opposite direction according to Jevons paradox. People still need experienced and professional human security researchers after all.
I'm glad you brought up Jevons Paradox, because it's for this reason I believe it's an extremely poor description of what's happening with AI.
The broader topic people should be discussing is not Jevons Paradox, but marginal utility.
More artifacts have diminishing utility.
https://en.wikipedia.org/wiki/Marginal_utility
Different groups have different risks tolerances, and it's absurd to think that nobody is going to pull releases before the default cool down period.
Why is that absurd? It seems like a more likely outcome than just assuming there are enough "security researchers" out there doing this for every package?! IMO both sides here seem to have no evidence that the other is wrong.
This entire article seems to say that if it doesn't prevent 100% of all possible attacks that it's "security theater". That's nonsense. It's just one more tool in the tool belt, and a highly useful one at that.
You should probably also be paying them directly.
(I agree that companies should pay for security products, of course. The bigger problem there is that so many security products are terrible, and supply chain products appear to be no different as a class.)
Most of the attacks we have seen lately are from legitimate projects which got compromised. The maintainers of those projects are generally looking out for that.
https://www.merriam-webster.com/dictionary/champing%20at%20t...
https://www.merriam-webster.com/dictionary/champ
The migration of "champing at the bit" to "chomping at the bit" is inevitable.
There don't seem to be many idioms like this, where the newer version is both semantically intuitive/meaningful and accepted by authorities. The closest is probably "hone in on", but the verb form of "home" as in "homing device" is still current, unlike the verb form of "champ". Another interesting one is the migration from "just deserts", using an archaic meaning of "desert" as in something deserved, to the modern form "just desserts". However there's much greater distance between "desserts" and the archaic meaning of "deserts" than between "chomp" and "champ".
https://www.npr.org/sections/memmos/2016/06/09/605796769/che...
The most likely explanation is obviously that it's just a bunch of attention seeking amateur. But damn.
Beyond that, a lot of these cooldown criticisms assume everyone will use the same length, which isn't how it has to work. There are people who run Arch because they want the latest version of everything, and there are people who run Debian because they never want a new version to break their system. Some people camp out for the newest iPhone on day 1, and some people wait to see reviews and find out if the antenna is sensitive to how the phone is held. Adjustable cooldowns let everyone choose their own balance between getting the latest features and getting something that is tested.
This ignores all the security companies & researchers screening packages.
Most maintainer are going to notice that themselves.
It will make credential attacks much harder to pull off successfully and the returns from using them will drop pretty dramatically.
One should sandbox all npm command executions.
https://github.com/ashishb/amazing-sandbox
1. Security researchers and companies are incentivized to install packages early.
2. Even without security patches, people and teams can still set the cooldown period that is appropriate to their risk tolerance. Even if someone in the wild does "get bit", it's better that a smaller number of people get bit than everyone getting bit at the same time. That's not "security theater", that's limiting the blast radius.
I'd also add that adding in cooldown periods lowers the incentives for bad guys to try to compromise a package in the first place because there is a much greater chance all their work will be for naught.
So I started reading this post with interest, thinking it would have some reason I hadn't thought of, but it just boils down to the faulty premise that "Godot is not coming". Bullshit, at the very least I now have agency over where I am in the download timeline. I just don't understand how such bad posts built on the flimsiest of assumptions make it to the top of HN.
And, the complainers are free to set the wait timer to 0 and go back to the way it was. I'll be glad to let them be the first penguin to jump in the water.
Are security researchers going to be doing free labor for you indefinitely for exposure?
I’d never heard of socket until they found and reported shai hulud hiding in pytorch lightning. It pays off.
I think there are diminishing returns to be had.
If you're going to "DYOR" or whatever, having your automated dependency management on a cooldown timer seems like a snazzy way to smoothly provide a window for all your fancy dependency validation to happen.
I don’t agree with either claim: I don’t think they’re security theater, and I do think they universalize, contingent on there being security parties that are incentivized to scan packages.
The latter is a big assumption, but I think it’s a win for us either way: if supply chain companies are actually capable of defending us, then we win. But if it turns out they can’t serve as the cooldown vanguard, then we have great evidence that they shouldn’t be trusted at all.
> But if it turns out they can’t serve as the cooldown vanguard, then we have great evidence that they shouldn’t be trusted at all.
If they cannot serve as the cooldown vanguard, we are indeed going to realize they are not trustable, but by the time the damage has already been done. Therefore, if companies want to prevent the damage as much as possible, I believe they should do the scan by themselves.
Shall have thought about that... but I'm playing too many Unity games recently.
I actually proposed a cooldown for apt-get in my PhD thesis in 2013 and calculated its impact. Because the meta-data for the Ubuntu apt repository can be completely downloaded you can simulate the impact of a cooldown on how many less packages would be installed and it isn't zero.
There are always people in a community running beta version, or up to date versions. There will be less with a cool down, but then depending on how critical your system is you could adjust cooldown length
Account takeovers, deployment exploits whatever the root cause, given a few days it's incredibly likely maintainers will notice they've released an unexpected update and fix/sound an alarm. A lot of existing cases like this people realise pretty fast, but the packages might still be live for a few hours as NPM catches up. Cooldown entirely removes that 'updated at the wrong time' issue, an no-one even needs to actually look at the code at all.
For inactive or intentionally malicious maintainers, yes it relies on vetting. But it's hardly a mythical 'canary' - there are already a lot of companies scanning every popular npm package, auditing new ones, or just generally analysing threats. I can think of at least 4 such companies without even trying, I'm sure there's loads more.
And they'll continue because it's not warm hearted so much as promoting/testing their security scanning services and tools so they can get the kudos from being the first to spot X etc. Which often does hit the front of HN or reddit, but they don't need the massive cyber loudspeaker to be effective. Flagging the issue to maintainers, reporting to NPM security - with a cooldown in place the package can be voided before anyone really gets the chance to install it. This is literally how a lot of things are spotted already, just with that 'updated at the wrong time' issue catching unlucky people out.
Does it solve everything? No. Will it reduce the amount of problems that make it to end users? Obviously yes.
And for all the author's 'no one will do this', 'waiting for Godot' cries - what do they recommend? Running your own audits, using LLMs, static analysis and testing it yourself. Basically being the veter they claim no-one will be! Unless they don't intend to sound the alarm themselves, they are literally the canary they're looking for.
Cooldowns aren't magic, but they're not security theater either.
There are always early adopters and researchers looking at new, publicly available software. Critical industries have long waited for software releases to be stable and "recommended" by the vendor before upgrading, letting those early adopters who can afford some availability risk take time with it.
The observation that "if everyone in production has a cooldown, the benefit is gone" is an absolutist opinion.
I have seen this occur in some fashion - my company's product has a software version that's been out for almost six months and we still tell customers in critical industries to stay on the older version out of an abundance of caution and with the understanding of what upgrades require for them. Stability > features.
There are COBOL users and CentOS 6 users. They aren't affected by this issue.
They might need to face another set of issues like vulnerability backporting and so on. People not worrying about cooldowns might still need to patch their log4j libraries, but that's another thing. This post is not for them.
> There are always early adopters and researchers looking at new, publicly available software.
I agree with that. These hacks are very unlikely to go unnoticed, especially for popular packages (although they do happen; `chalk` has 3M downloads per week when it was hacked).
> The observation that "if everyone in production has a cooldown, the benefit is gone" is an absolutist opinion.
Yes. However, I think this is eventually becoming some type of pointless arms race, as cooldown does not act actively towards reducing ecosystem attacks.
To add a somehow random example, pnpm's default cooldown time is 1 day. Yarn's 7 days. I'm not sure whether one day pnpm would also change it to 7 days because of people asking.
The nature of some of these attacks, it does. If your developers machine with publishing rights gets rooted a cool down makes credential gathering attacks via script in npm so very much harder it's not even funny.
In some of the recent supply chain attacks the entire fiasco was only a few hours long, but in that time tens of hundreds of thousands of credentials may have been stolen.
One day timers have the weakness of weekends and holidays reducing the number of eyes/systems seeing the issue.
- Independent security companies are scanning the packages (be careful if the project is depending on some no-so-popular packages)
- Maintainers react promptly (it would be nonsense if people need to cooldown for 21 days because people may have 21-day vacations)
In no circumstances will everyone. This is just a bad take based on a silly premise.
Most, if not all, of the shai-hulud attacks that hit npm and other ecosystems were preventable with cooldowns. And these were not detected because regular users reported the worms, but because security researchers did. I don’t think I’ve ever seen an attack that was discovered because a user reported it.