building the world's best AI scientist
sf
Joined August 2024
- Tweets5.5K
- Following579
- Followers4K
- Likes10.8K
shetty retweeted
fable 5 reading the code written by opus 4.8 after going afk for 19 days
Readers added context they thought people might want to know
This video appears to be digitally altered. Here is an earlier version of this video, showing a different person instead of Haaland: x.com/kenw_2/status/…
i just realised how common betting is in usa specifically on prediction markets like literally every other person ik is doing it
something I've been doing non stop for the past 3 days (I'm tired):
awsdocs-neuron.readthedocs-h…
shetty retweeted
Hermes Agent now reads the web up to 60x faster and 49x cheaper.
Scraping backends pass clean content straight to the agent without redundant processing steps; large pages are saved locally and paged on demand so you get the same quality at a fraction of the time and cost.
shetty retweeted
yes that guy
Update: I've joined @AnthropicAI and taken leave from the university. Excited to work with many talented, mission-driven people on the defining technology of our time.
shetty retweeted
I knew it was going to be an insane nothingburger, because there's currently a soft ban on frontier capabilities
but I genuinely don't understand why they didn't call it Sonnet 4.8 or Sonnet 4.9, because this artificially nerfed piece of shit is not worthy of the 5.0 naming
shetty retweeted
OpenAI achieved a much more significant breakthrough today.
Sonnet 5 is an average release. But the fact that OpenAI, according to The Information, has managed to more than halve the inference costs of its current models through a new approach to inference optimization is absolutely groundbreaking.
And when you also consider that they recently introduced their own inference chip with Broadcom, which is said to be faster and more efficient than the competition, I increasingly see OpenAI in an outstanding position.
Today, at least, OpenAI emerges as the winner of the day.
Here is my first assessment of Sonnet 5:
Sonnet 5 is better than Sonnet 4.6. Who would have thought? But jokes aside: Unfortunately, it is weaker than Opus 4.8 across all evals. Why they nevertheless labeled the latest Sonnet 5 iteration with a “5”, even though “4.8” would have been more fitting, is beyond me. Normally, major version jumps in particular signal a significant leap in capability. Be that as it may: Sonnet 5 is good, but worse than expected.
Pricing has not changed; it is on the same level as its predecessor. Opus is still more expensive, but at the same time it also remains better. Overall, the release irritates me and leaves more questions than it answers.
I cannot help but see Sonnet 5 as a release that stands in the context of Fable 5. There was no mention of Fable 5 at all, which surprises me a lot. I really would have expected us to get news about it at the same time. But nothing. Instead, we get an update to a new model series (“5”), but one that is not significant compared with the models we already have.
As a result, there is a lingering aftertaste that Sonnet was released as something in between, perhaps also simply to release something at all and to stay part of the conversation, including in a positive sense. Why no Opus 5, when we know that Fable 5 already exists as a model that performs significantly better than 4.8, and when we can assume both that a better Opus exists internally and that it would not be difficult to update Opus to the new generation? Why “only” Sonnet 5?
Because restraint is currently required. The major releases are currently being delayed across the board; they are still in discussions with regulators about how the truly powerful frontier releases can be carried out at all and under what conditions. In my view, the Sonnet 5 release has to be seen against this background. And as a result, at least for me, it was disappointing overall.
my entire daily routine has been messed up since I moved to sf I just go to work come back home eat something and go straight to sleep and the same cycle again. need to improve on this but this is the busiest I've ever been and most fun I've ever had working so won't complain.
shetty retweeted
When you hit a wall in math, coding, or any hard skill, do not immediately conclude that you lack talent. Most walls are just prerequisite debt finally coming due. Go back, fill the gaps, make the basics automatic, and the wall often turns into a staircase.
shetty retweeted
> mythos is so good at cyber it can't be released
also
> mythos can't detect 20k fraudulent chinese accounts attacking it
shetty retweeted
I can't believe how many people believe Anthropic's propaganda about distillation attacks. You can read how GLM-5 was trained in the paper.
Maybe Claude was distilled from Chinese models? I haven't seen a paper from Anthropic, and accusations are often confessions.
shetty retweeted
new inference optimization method by @deepseek_ai with an extremely detailed paper, draft model and framework to train them. results in production for dsv4 lead to +50% for throughput and latency (can go to ~80% for latency, crazy).
full explanation of DSpark:
it's about speculative decoding and the idea builds upon DFlash (fully parallel) and Eagle (fully sequential) to create a "semi-parallel" method that keeps the advantages of both
the core equation you want to optimize is the "time to generate each token" which is:
(time to draft + time to verify) / how many tokens are accepted
the advantage of the parallel variant (DFlash) is that it's fast, but when you increase the number of tokens you draft, acceptance rate drops pretty fast (makes sense since there is no dependency on the previous token).
fully sequential is nice but opposite issue: it's slower (you need a much smaller draft to get the same speed) but the autoregressive dependency means you can maintain good acceptance rate at a lot of tokens. since you have a much smaller draft head, the first token acceptance rate is often quite low
idea of DSpark is to combine both: a "heavy" parallel head (you only do it once) and then a small sequential step to bias the logit distribution with information about the previous token. this biasing is done with a small markov head (only depends on t-1)
they also get a confidence score out of the sequential head that allows them to adjust how many tokens they want to verify. verification can get expensive if the gpus are already at maximum utilization, so they use this confidence score to do some load balancing and predict the right number of tokens depending on gpu workload
one small detail: i would have liked to see production numbers if they used DFlash or Eagle instead of MTP-1, but as always, huge work by deepseek and i'm expecting to see this method widely adopted
