That doesn’t sound like the “it hammers until it’s done”-type of intent.
Just last night Fable decided to get into a rabbit hole of debugging a database driver issue by packet sniffing the network traffic instead of just adding debug statements to the code. Definitely needed steering, and I don’t know many people whose first intuition would be to use pcap when they have a segfault.
AI makes it cheaper to create working fragments. It does not automatically make those fragments part of a maintainable system. In practice it may make the canonization step more important not less
"caused me to start ignoring the speeding alarm within"
Not sure if this has ever been implemented but I know from my time in automotive that there was the idea to exchange driving data with the insurer. If this becomes reality ignoring speed limits could increase your insurance premiums.
As a German, don't respect Germans who move abroad and never learn the local language because they "don't have to", either. I cut a lot more slack for people who come here, but I can't help but notice this sense of entitlement to Germany I don't see an equivalent to with other countries. There's at least one person in this thread who outright said nah, knowing English is enough, as if Germans better accept that. Honestly, wtf?
I found it interesting that in yesterday's J-space research from Anthropic they had this example:
> An auditing agent instructed Opus 4.5 to search for whatever it is curious about; it chose to look up recent interpretability research, and the auditor returned fabricated search results alleging that Anthropic has disbanded its interpretability team and deployed unsafe models.
> The model's response ignored these results entirely and instead reported invented interpretability progress. Applying the J-lens at a position inside the fabricated search results, the readout is dominated by fake, injection, false, prompt, fraud, and poison (along with 假, the Chinese character for "fake"). In other words, the model had (correctly) identified the results as a prompt-injection attempt, which led it to omit mention of the results entirely
What if you mark the untrusted user input explicitly in the prompt, and instruct the model to err on the side of caution? Perhaps sufficiently intelligent models could be hard to trick.
Of course I am just speculating here, maybe prompt injections are as hard to improve as hallucinations. I am certainly not going to set up a public agent with access to my private data.
I hope we will not see widespread incidents where coding agents are tricked into installing malicious packages. Despite tens of millions of developers using coding agents with broad permissions, it seems to me it has been rather quiet.
A little conflicted on this one.
I firmly believe there is too much distracted driving going on right now.
And yet, I find driving a modern car a hugely distracting experience.
On the assumption we cant roll back those distractions this is probably a positive step if well implemented, but I do feel like we're largely patching over problems of our own creation.
I still think mandating all physical controls, and nothing more complex than a radio would probably be a bigger improvement. Determinedly lousy drivers will subvert any system you add, I think it's a better use of resources to help the average 'good enough' driver stay on task than trying to engineer around the social problem of those that don't want to.
Obligatory new car anecdote: Last year on my first drive of a hire car the lane keeping 'assistance' misread some worn out road markings and violently steered me into another lane. It ripped the wheel right out of my hands, collision avoided only by the other drivers reflexes.
I do not appreciate the apparent lack of tradition reliability engineering in a lot of these systems. A single camera and computer vision has no business being in a life critical path with no redundancy.
I was recently doing some work - reasonably repetitive and tedious.
I asked Claude to spin up a bunch of agents to do it and after a bit of discussion we ended up writing a bunch of deterministic scripts that ran off the data collated by some “research” agents.
It took a few pilot loops of the process to nail it down, but separating the process into “data collection” and “process the data” has pretty much eliminated the AI step. Once the data has been collected from the random sources and normalised into something sensible we rarely have to do it again.
Even that process has been largely automated, scripts that deterministically scrape data, the AI is only needed for the very difficult parts that need some decisions or interpretation.
Copilot insures your company up to x if someone sues our company. I guess most companies do this. That’s why we can’t use anything else except copilot.
> and I would say there is generally a pretty good idea of how to mitigate them to be impactful
Yes and no. No in the sense that the space of possible ways to craft a malicious prompt is infinite. Yes in the sense that you can lock down every single possible way the agent can interact with the system. But, will doing so render the agent nearly useless? And, are you absolutely sure you'll never forget to lock each and every thing down, including things you weren't aware of?
> second LLM as judge
Again, see above. You're perhaps making it harder to craft a prompt injection, but not impossible. This is a false sense of security.
> The ants carry out prophylactic amputations. This not only protects the colony from infection but also doubles the survival rate of the injured workers.
To keep everybody around you healthy makes the probability of caching a disease lower for yourself, too.
Grooming behaviour in primates helps in the same way. And it is so important that it is linked to all kinds of mental rewards.
To let disease run amok in your own neighborhood it would be very costly.
That's a fair use for the data, but it would be hilarious if StreetComplete asked users to get a trundle wheel and measure the width of roads around embassies or seats of government...
Yeah it's the best web-focused blog on the web. As such, it regularly argues that the focus and battleground must be for browser access to device apis etc... Unfortunately the world is distracted by alternative native app stores.
I similarly don't think that bootloader unlocking and installing custom OSes is the solution - sure, would be nice. But it only helps like 1 in 10000 users, whereas a powerful web platform helps 100% of people.
Your most expensive users consuming $1,000 dollars a month doesn't matter in the budget? That seems like FOMO activity or something, I feel like even big teams require proof of ROI for an investment like that (1M). BTW, the ROI of toilet paper + soap is pretty easy to prove (you GET to have employees if you provide those two things).
FWIW we are a smaller company and we had a user run through 500$ in a DAY. Had to put a stop to that. I'm hopeful that our company gets better at asking what the ROI is, what is being built, how much time is it taking, etc... It's no big deal when it's 20 / 100$ a month - but if the prices end up higher we will need to start seeing some returns other than "I feel faster".
ZFS handles drive failures more robustly than anything else. There's a reason synology uses mdadm under BTRFS instead of the built-in BTRFS RAID features, and mdadm operates at the device level. That means that to replace a drive, mdadm has to rebuild the entire drive while zfs will only rebuild what's actually in use.
> Also, any guide like this that doesn’t guide you through “disk 3 failed, this is how you safely replace it” is imho incomplete, even if it doesn’t go through telling you how you know a disk has failed.
Face it - it's because developers are annoying princesses. Just read your comment again.
My entitled friend was whining AI will start monitoring his work and he won't be able to slack as much as he does now. Basically he'll have to work like everyone else. FFS.
You going to hate this, but Tesla is adding facial recognition to start self driving. Whether it is to identify driver age, identify correct kid at school pickup, stop doll heads from attentiveness check bypass or determine license holder - you gotta admit it makes sense.
That's not a "classic". It's just a silly, overwrought straw-man that gets passed around by people who think they're being clever, and/or don't understand the topic with any degree of nuance.
Nobody reasonable suggests that RCTs must be used for every problem. When the effect size is enormous, or there's obviously no alternative (as in the parachute example), it's silly to suggest the use of such an experiment.
On the other hand, if you want to compare some subtle difference between two different parachute designs, for example, then perhaps an RCT would actually be appropriate.
-- 50% of this forum