So, supposing it's true that these models completely change the security field and humans are ~obsolete other than as pilots guiding them what to crack, you think it's reasonable that Anthropic and OpenAI should unilaterally determine who gets to be a security professional? I hope you do understand that is what you are suggesting.
Why should anyone get to determine that? Do people really want us to move to an exclusionary guild system? I thought the experience with proprietary versus open source over the past 30 years had driven home the point that closed ecosystems are almost always far worse for security.
> the experience with proprietary versus open source over the past 30 years had driven home the point that closed ecosystems are almost always far worse for security.
Has it? Can you prove it? I've been using computers for almost 40 years. I've seen foss-enthusiasts repeat that claim ad-nauseam, without proof. All they ave is the vague, hand-wavy, "millions of people read the code!!11".
I use both proprietary and foss software. I write both proprietary and foss software. I have not noticed a meaningful difference in security.
Then I think you haven't been paying attention. We regularly see examples of companies attempting to cover up vulnerabilities, attacking security researchers, dragging their feet on fixes, etc. Meanwhile you can easily see for yourself how long it takes various FOSS projects to get patched and often what the attitude of the devs is.
You can also take an aggregate view. Presumably skilled developers working on major projects should be expected to have similar rates of security issues. So compare CVE frequency between various FOSS and closed source projects.
A more charitable interpretation might be that a guild would not be expected to passively allow such a situation to continue to exist. I think you'd expect a guild to directly contract for the desired tools or failing that to move into production themselves.
Regulatory capture: A vendor using regulation to prevent potential competitors from producing or selling competitive goods or services
What we're discussing in this thread: A customer compelling a vendor to produce and sell a specific good or service
Do you think these are similar?
The fact that Anthropic is doing one thing (regulatory capture) is entirely irrelevant as to whether they're allowed to engage in a completely different thing (declining to sell their services/products to specific people).
> Additionally, even if there is a guild - no guild ever let a vendor pick and choose what [the guild's] capabilities were, that would be insanely dumb.
But that's not true. Again: Vendors absolutely pick and choose what their customers' capabilities are. Regardless of whether "the guild allows them to." Guilds can't force people to make or sell tools against their will – obviously.
The analog you're trying to describe doesn't exist, which is Anthropic saying nobody else can make and sell an offensive model to "the guild."
No. Customers have never been able to compel their suppliers to make or sell certain products against their will (except in collectivist regimes or like 0.00001% of natsec related instances)
This conversation gets more and more bizarre, but I’ll bite.
1) pharmaceutical companies are regularly compelled to produce specific pharmaceuticals to continue to be allowed to exist.
2) hospitals are regularly compelled to treat patients even if they can’t afford treatment, if it is a life threatening emergency.
3) car manufacturers are always compelled to produce vehicles that meet a litany of safety, weight, and efficiency standards or they can’t produce at all.
4) defense contractors are regularly compelled to produce specific defense related products for long periods of time after they would otherwise have stopped, or else.
5) even your neighborhood gas station is likely compelled to provide air refills, free or at minimal cost, or else.
6) during a wartime (command) economy, which has happened numerous times in the US alone in the last 100 years, companies have to make what their customers (the people of the United States) demand or else.
7) utilities like electric utilities regularly have to give out freebies or take losses on things as demanded by regulators, at customers behest.
Or if we go back a bit, blacksmiths, quarries, masons, etc. all had to deal with producing what the government/lord at the time wanted - often on penalty of death - during wartime, or just because they were ordered to do so.
4) The US government can compel production, but it's extremely rare
5) Not by their customers they're not, lol
6) Yep this can happen, but is extremely unusual
7) Not by their customers they're not, lol
We're illustrating how ridiculous your claim that "guilds have always been able to declare what vendors create for them" is
Now you're talking about government regulations for some reason. Even your examples of customers being able to compel production are actually examples of governments being able to compel production, and in just a few of these scenarios the government is the customer. But it's their power as governments, not their power as customers that can compel production.
As stated: you've lost the thread. You're talking about totally irrelevant stuff.
And regulations are totally not written by people elected to do so - by their customers? And many of those, the customer is the gov’t and literally doing so?
Not to mention how wild it is to operate under the assumption that they won’t give a license to an LLM that can do illegal actions to someone who shouldn’t have it. Offering it at all is an ethically dicey question.
I wish you understood that there are organizations of security professions that are not controlled by Anthropic and OpenAI and that it is a common thing that when companies of any type sell to professionals of any type it is not the companies that determine whether or not the people they sell to are professionals but membership in professional organizations.
As an example the people who sell police uniforms check that the person they are selling to is in fact a policeman (at least in the jurisdictions I have lived in, you may have had a different experience which would certainly explain what to me seems a farcical misapprehension of how modern civilization works)
I mean I just wish you understood, and really that everyone understood, that this kind of three part communication (company selling, buyer, professional organization certifying buyer) is often when buying things that are considered to have security implications.
>So, supposing it's true that these models completely change the security field and humans are ~obsolete
OK, well that strike me as a really crazy level of supposition there.
I would suppose that these models make it easier for people who want to do bad things to do bad things at scale, at the same time allowing people who want to stop bad things to help identify potential targets.
Based on my supposition I would want to stop the first and find a way of helping the second. Also because I have another supposition that the first thing is easier to do than the second.
But you obviously feel differently about this issue, no doubt because of your position of great moral stature and insight, and this no doubt prompts you to wish to me to understand things that from my position seem absolutely ludicrous.
