yeswondwerr retweeted
We're launching a $100,000 challenge to see who can make Muse Spark run fastest: openkernelchallenge.dev/
idk whats so special about the meta 9 fig tc packages, these guys are offering at least 300m and equity
yeswondwerr retweeted
Here's something I realized about most of my output. I doubt it's unique, but for some reason I've not heard this concept elsewhere yet.
Most of what I produce is not fundamentally bottlenecked on resources (time, energy, etc.) but on intuitive conceptual clarity.
It's like there's a certain minimum number of days, weeks, or months (for different tasks) that you need to spend in play mode just collecting signals, accumulating them, and digesting them.
And digesting them just takes a certain amount of time, after which you just come out with an unshakeable sense of the exact path that will work and will generalize.
This is how I think of abstractions specifically, and to AI system design second, but it honestly applies to most output of value.
Some processes are really just that sequentially bottlenecking, akin to digestion, because they require all-to-all attention. Modularity obviously helps, but premature modularity makes no sense.
You do need to digest the problem first (which is rather sequential) in order to even know where to split things productively. And then once proper modularity emerges you lean into it. Modularizing a novel, complex endeavor from day 1 would be a recipe for dysfunction.
yeswondwerr retweeted
remember
this is a long term
this is an infinite race
in the world of life
there is no two minute
there is no end of the quarter
there is no such thing
🚨🚨🚨BREAKING:
Nvidia CEO was just asked about his thoughts on Huawei and China
> Huawei is one of the most formidable technology companies in the world.
> they are incredible in computing, networking technology and software capabilities
> they have all the essential capabilities to advance AI
> China is right behind the US in AI
> 50% of the world’s AI researchers are chinese
The race is on.
you can tell how good the research is depending on what kinda issues you face when trying to run it
4
390
yeswondwerr retweeted
today i'm releasing a Sparse Autoencoder for DeepSeek-R1-Distill-LLama-70B, trained on a mix of chat and reasoning data.
yeswondwerr retweeted
Replying to @teortaxesTex
yeswondwerr retweeted
Replying to @teortaxesTex
>generates n samples
>All n are retarded
People don't consider the implications of this enough lmao
yeswondwerr retweeted
Replying to @yeswondwerr
@yeswondwerr and i release doubutsu, a family of Apache licensed 2B VLMs with 32k context designed to be trained on your use-case. based on Qwen2-1.5B and SigLIP. a small project before our future releases.
links:
huggingface.co/collections/q…
github.com/qrsch/doubutsu
more info below
yeswondwerr retweeted
llama-3-vision-alpha is the best <20B model for hard questions on vibe eval.
qresearch w
results from running @RekaAILabs reka-vibe-eval on 8 smol (1.86-8B) vision language models
Top 3 on all
- Qwen-VL-Chat
- xtuner/LLAVA-llama3-8b
- qresearch/llama-3-vision-alpha
Top on hard
- @qtnx_ research/llama-3-vision-alpha
github.com/pythonlearner1025…
yeswondwerr retweeted
Replying to @somewheresy
People are almost entirely a product of their environment. No, extremely random outliers aside, the overwhelming majority aren't bootstrapping out of their circumstances unassisted
so 38.04% and 32.94%, respectively for non instruct and instruct modules on full realworldqa
how was llama-3-vision trained?
thanks to the very similar embedding spaces between base LLMs and their instruction-tuned counterparts, prompt understanding abilities are maintained while feeding one model's embeddings to the other.
@yeswondwerr and i decided to test this by training a projection module using a VLM pretraining dataset on Llama-3-8B, which we then tested on Llama-3-8B-Instruct. we trained the same projection module on Llama-3-8B-Instruct as well.
we found out that the former module gave us better performance! on RealWorldQA, the non instruct module has a score of 291/765, while the other has a score of 252/765
we'll talk more in detail about training in the future
13
3,932


















