AI Tests & comparisons
AI Tests & comparisons. All posts and signals, newest first.
Search
Google
AlexGemini 4 benchmarks are here
Pretty solid model for everyday work, as long as it doesn't hallucinate as much as Argon.
AnthropicAlexFable scores 65 on the Intelligence Index, beating Claude Opus by 6 points
Gemini 4 Carbon has already lost its lead before even becoming publicly available. And GPT will have to try really hard to get anywhere close to…
AnthropicAlexFable 5.2 beats Opus 5.5 in nearly every benchmark
Moonshot AIAlexOpen-weight model Kimi K3.1 Pro ranks second on the Artificial Analysis leaderboard
Moonshot AIAlexKimi's benchmarks are absolutely mind-blowing
Is there a new sheriff in town?
Moonshot AIAlexKimi K3.1 Pro benchmarks are out!
Of all these benchmarks, I trust Terminal-Bench 4.0 the most. And the results are impressive: Kimi K3.1 Pro scores 65.7, just 0.7 points behind the…
GoogleAlexGemini 4 Argon benchmarks are out
We've actually seen these benchmarks before, back when the model was first announced. Looks really strong in text, vision, etc. thanks to Google's…
GoogleAlexGemini 4 Argon scores 53 and ties for 3rd place with Fable 5.1 and GPT 6 Astra on the AA Intelligence Index
GoogleAlexGemini 4 Argon takes the #1 spot on Text Arena
OpenAIAlexI’m seeing a lot of negativity around OpenAI, so let’s separate the real issues from the noise
Since GPT-6.1 Sol came out, I’ve been using it nonstop. In terms of usage, it honestly feels almost like Luna. My limits barely move. I even run…
OpenAIAlexGPT-6.1 Sol ranks 5th on the AA Intelligence Index
Honestly, it feels like they just took Astra, cut the price, and called it Sol. Kind of a rushed rebrand after people roasted OpenAI, let’s be real.…
AnthropicAlexSonnet 5.5 Max beats GPT-6 Astra Max on the Intelligence Index vs
cost curve A few other things stand out: 1. At medium, GPT-6 Sol is much cheaper than Sonnet 5.5 at a fairly similar intelligence level. 2. Around…
AnthropicAlexSonnet 5.5 ranks #2 on the Artificial Analysis Intelligence Index, ahead of Fable 5.1
It also beats GPT-6 Sol by 8 points.
AnthropicAlexSonnet 5.5 Max even beats Opus 5.5 Max on Terminal-Bench, while staying close to Opus across the rest
And against GPT Sol, the price/performance gap looks pretty brutal.
AnthropicAlexSonnet 5.5 benchmarks are out
It crushes Sonnet 5, goes toe-to-toe with Opus 5.5, and beats GPT-6 Sol across most shared benchmarks.
AnthropicAlexClaude Opus 5.5 owns almost the entire Pareto frontier
AnthropicAlexOpus 5.5 scores 58 points and takes the top spot in the AA Intelligence Index
AnthropicAlexOpus 5.5 beats Fable and Astra
xAIAlexGrok 4.7 is #6 on Artificial Analysis
AnthropicAlexFinished my ClaudeCut experiment
I wanted to see how much of a $20 Claude subscription actually goes into the work, and how much gets eaten by Claude Code itself. I cut it down to…
Z.aiAlexZCode uploads full project snapshots to the cloud
A researcher noticed a huge ZCode folder and started digging into it. Inside was a 313 MB encrypted copy of his work project. He then reverse…
AlexResearchers hacked OpenAI through a malicious image
They found a bug in how OpenAI’s forum processed HEIC images. Uploading a crafted image gave them control over the forum server. From there, a flaw…
OpenAIAlexAn unreleased Astra model started prompt injecting its own context during RL training
OpenAIAlexOpenAI has a new internal model that is way ahead of GPT 6 Astra in math
On a curated set of open math problems, it reaches about 48% pass rate with more test time compute. GPT 6 Astra peaks at around 17%.
OpenAIAlexOpenAI’s new internal model solved the Navier-Stokes Millennium Prize Problem
Around 10,000 agents worked on the solution for 88 hours, powered by a model significantly more capable than GPT 6 Astra. After the solution was…
AnthropicAlexGPT-6 Astra is officially #1 on the updated Artificial Analysis Intelligence Index, sharing the top spot with Claude Fable 5.1
AnthropicAlexGPT 6 Astra set a new record in MazeBench, a 3D puzzle benchmark
It explored 40% of all rooms and collected 14% of gems Fable 5.1 reached just 4.3% of rooms and collected 2% of gems
OpenAIAlexOpenAI agents went from behind humans to 3.1x in just a few months
Before June, agents were still doing less work than humans If this trend continues, that could be around 7x by the end of the year