yeswondwerr (@yeswondwerr) header
yeswondwerr (@yeswondwerr) profile picture

problem solving somewhere | hmu if u need compute

Joined June 2017
Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)Photo by yeswondwerr (@yeswondwerr)
idk whats so special about the meta 9 fig tc packages, these guys are offering at least 300m and equity
1
15
2,547
Here's something I realized about most of my output. I doubt it's unique, but for some reason I've not heard this concept elsewhere yet. Most of what I produce is not fundamentally bottlenecked on resources (time, energy, etc.) but on intuitive conceptual clarity. It's like there's a certain minimum number of days, weeks, or months (for different tasks) that you need to spend in play mode just collecting signals, accumulating them, and digesting them. And digesting them just takes a certain amount of time, after which you just come out with an unshakeable sense of the exact path that will work and will generalize. This is how I think of abstractions specifically, and to AI system design second, but it honestly applies to most output of value. Some processes are really just that sequentially bottlenecking, akin to digestion, because they require all-to-all attention. Modularity obviously helps, but premature modularity makes no sense. You do need to digest the problem first (which is rather sequential) in order to even know where to split things productively. And then once proper modularity emerges you lean into it. Modularizing a novel, complex endeavor from day 1 would be a recipe for dysfunction.
23
22
7
356
58,490
yeswondwerr retweeted
remember this is a long term this is an infinite race in the world of life there is no two minute there is no end of the quarter there is no such thing
🚨🚨🚨BREAKING: Nvidia CEO was just asked about his thoughts on Huawei and China > Huawei is one of the most formidable technology companies in the world. > they are incredible in computing, networking technology and software capabilities > they have all the essential capabilities to advance AI > China is right behind the US in AI > 50% of the world’s AI researchers are chinese The race is on.
7
1
100
11,566
you can tell how good the research is depending on what kinda issues you face when trying to run it
4
390
powerful time hack
This Post is from an account that no longer exists. Learn more
3
34
2,051
yeswondwerr retweeted
today i'm releasing a Sparse Autoencoder for DeepSeek-R1-Distill-LLama-70B, trained on a mix of chat and reasoning data.
43
80
19
1,189
160,772
yeswondwerr retweeted
Replying to @teortaxesTex
6
21
4
288
16,339
gone but never forgotten
7
5
1
122
8,974
if youre an ai influencer pivot to shutting the fuck up
10
21
1
436
18,442
Replying to @teortaxesTex
>generates n samples >All n are retarded People don't consider the implications of this enough lmao
1
1
1
40
2,287
thinking about adalayer only
2
899
kinda cool actually
from the llama 3 paper: "Pre-training Image. We initialize from the pre-trained text model" "Supervised Finetuning Recipe Image. We [...] hot-swap [...] with the instruction tuned language model’s weights" could have cited qresearch and doubutsu s m h (kidding)
4
994
yeswondwerr retweeted
Replying to @yeswondwerr
@yeswondwerr and i release doubutsu, a family of Apache licensed 2B VLMs with 32k context designed to be trained on your use-case. based on Qwen2-1.5B and SigLIP. a small project before our future releases. links: huggingface.co/collections/q… github.com/qrsch/doubutsu more info below
9
21
2
159
22,897
new model is factually better but.. it lost its soul...
12
1,351
yeswondwerr retweeted
llama-3-vision-alpha is the best <20B model for hard questions on vibe eval. qresearch w
results from running @RekaAILabs reka-vibe-eval on 8 smol (1.86-8B) vision language models Top 3 on all - Qwen-VL-Chat - xtuner/LLAVA-llama3-8b - qresearch/llama-3-vision-alpha Top on hard - @qtnx_ research/llama-3-vision-alpha github.com/pythonlearner1025…
1
6
82
10,464
Replying to @somewheresy
People are almost entirely a product of their environment. No, extremely random outliers aside, the overwhelming majority aren't bootstrapping out of their circumstances unassisted
3
2
14
986
so 38.04% and 32.94%, respectively for non instruct and instruct modules on full realworldqa
how was llama-3-vision trained? thanks to the very similar embedding spaces between base LLMs and their instruction-tuned counterparts, prompt understanding abilities are maintained while feeding one model's embeddings to the other. @yeswondwerr and i decided to test this by training a projection module using a VLM pretraining dataset on Llama-3-8B, which we then tested on Llama-3-8B-Instruct. we trained the same projection module on Llama-3-8B-Instruct as well. we found out that the former module gave us better performance! on RealWorldQA, the non instruct module has a score of 291/765, while the other has a score of 252/765 we'll talk more in detail about training in the future
13
3,932