Surprise, surprise: Driving (well, being driven in) an autonomous vehicle (aka robotaxi) is much safer than driving yourself.
The Waymo team analyzed insurance claim data for more than 25 million miles driven in their cars and found that their autonomous vehicles vastly outperformed human drivers, with an 88% reduction in property damage claims and a 92% reduction in bodily injury claims.
This (long) post is well worth reading in full – Arvind Narayanan, Benedikt Ströbl, and Sayash Kapoor do an excellent job of drilling into the current challenges with scaling AI models, why we shouldn’t trust industry insiders telling us that they have scaling AIs figured out, and why we are still years behind leveraging even the powers that AI bestows upon us today.
The furious debate about whether there is a capability slowdown is ironic, because the link between capability increases and the real-world usefulness of AI is extremely weak. The development of AI-based applications lags far behind the increase of AI capabilities, so even existing AI capabilities remain greatly underutilized. One reason is the capability-reliability gap --- even when a certain capability exists, it may not work reliably enough that you can take the human out of the loop and actually automate the task (imagine a food delivery app that only works 80% of the time). And the methods for improving reliability are often application-dependent and distinct from methods for improving capability. That said, reasoning models also seem to exhibit reliability improvements, which is exciting.
What happens when you give people near-perfect information about the future (a proverbial “crystal ball”)? It turns out we still make bad decisions.
In November 2023, we ran an in-person, proctored experiment involving 118 young adults trained in finance. We called the experiment “The Crystal Ball Challenge.” We gave each participant $50 and the opportunity to grow that stake by trading in the S&P 500 index and 30-year US Treasury bonds with the information on the front page of the Wall Street Journal (WSJ) one day in advance, but with stock and bond price data blacked out. The game covered 15 days, one day for each year from 2008 to 2022. The players in the proctored experiment did not do very well, despite having the front page of the newspaper 36 hours ahead of time. About half of them lost money, and one in six actually went bust. The average payout was just $51.62 (a gain of just 3.2%), which is statistically indistinguishable from breaking even. The poor results were a product of: 1) not guessing the direction of stocks and bonds very well, and 2) poor trade-sizing.
Here is the full article – a good reminder that it is not enough to know the future… You still have to (consistently) make good decisions about it.
Turns out, robotics is hard – even for wunderkind Boston Dynamics.
Waltham-based Boston Dynamics laid off 45 employees last week, representing a reduction of five percent of the company’s workforce, The Boston Globe reports.