Eric Cui (@EricCui29) header
Eric Cui (@EricCui29) profile picture

@stanford CS | I think I can think, so am I? Warhammer 40k Enjoyer. The work, it's sacred.

Joined February 2024
Photo by Eric Cui (@EricCui29)Photo by Eric Cui (@EricCui29)Photo by Eric Cui (@EricCui29)Photo by Eric Cui (@EricCui29)Photo by Eric Cui (@EricCui29)Photo by Eric Cui (@EricCui29)
Been recently finding a lot of value in grokking papers. Obviously you should deeply engage with foundational papers relevant to the research you are conducting, but being annoyingly pedantic in understanding every minute detail and as much of the mathematical background behind a particular concept can really supercharge a literature review. For example, I’ve recently been looking into training data attribution methods, and deeply understanding the supporting assumptions required to exactly utilize influence functions (strictly convex + twice differentiable loss functions, understanding when/how one smooths/approximates the effects of estimating differentiating different components of an architecture) has really saved me a lot of time in identifying methods that can/can’t be used as principled baselines. There are also subtle features of how the paper is written that translate across related works. The structure/density of derivations, how large conceptual leaps are, implicit decisions made in the methodologies, etc.
5
254
Been enjoying reading up on training data attribution methods. There exist many elegant gradient-based methods (influence functions, TracIn), but I am especially interested in embedding-based methods currently. There is much potential to improve nonparametric methods. Memory is an obvious bottleneck.
1
4
330
Been reading up on non-linear similarity kernels recently. The mathematics are quite beautiful. Running a hidden activation state through each layer of an MLP can be seen as unnormalized attention over training examples. The idea is during training, each backpropagated update can be viewed as the addition of a value vector (the error signal) outer-producted with the example embedding. Inference, then, is the similarity-weighted sum of the value vectors. Using dot-product as the similarity kernel, you can represent this operation as a simple matrix multiplication. Non-linear similarity kernels purport to sharpen similarity score relevance. However, now we can't nicely represent a layer output with a matrix. We need to store every single training-embedding backpropagation-error pair to calculate similarity over all examples. This empirically leads to significantly optimized training when applied to small-scale classification tasks (CIFAR, MNIST), but due to the memory-inefficiency it is currently intractable for larger architectures. I am especially excited about the data attribution research that can be spun out of this. More on this soon!
5
243
Reasoning about neural nets as primitive memory stores is quite interesting. More on this coming.
3
114
Active recall is such a crucial part of the learning process. It is incredibly easy to delude oneself into thinking one is learning when in actuality one is simply activating familiar representations in the brain. At the end of the day, recall is the goal. If you can't use what you read, what was the point?
1
4
234
I've recently been internalizing the importance of high-quality sleep. Information is sent to long-term memory and integrated usefully while one sleeps. Without rest, nothing sticks.
4
138
I find it mind-boggling how incredibly precise we are able to orient our physical forms. I enjoy zooming around on campus with an electric skateboard, and every once in a while I will think about how amazingly automatic and granular the activity is. The subtle quad-activated bracing in response to acceleration, the slight hip-popping induced during sharp turns, the reflexive bending of the knees when moving over bumps. That's all without mentioning the absolute miracle that is balancing on a glorified wooden plank screaming down a street at 20 mph. What's especially terrifying is a couple-degree in my lean is the difference between a sick turn at speed and a trip to the ER. Let's hope muscle memory is here to stay.
2
88
Truisms have interesting life cycles. Consider the idea: "hard work pays off." As a child, one takes it to mean that the fruits of one's labor are directly correlated with effort. Work hard, remain disciplined, and the outcome WILL manifest as intensely as you toiled. One then typically becomes disenchanted with the idea in early adulthood. The more directly you brush up with unfiltered reality, the more you realize hard work merely favorably modifies the distribution of outcomes. Nothing is guaranteed. One may then conclude that intense, focused effort is a fool's endeavor. Why struggle for a chance at greatness when you could minimally exert for comfort. The truism is discarded for practicality. It takes a measure of maturity to truly, intimately internalize the idea that the measure of a man is their distribution. It is the stubborn refusal to allow uncontrollable circumstance to alter their self-perception. Self-respect, inner peace, and egotistical resilience is the pay off. In the end, hard work always returns.
6
284
I enjoy logging the time I work on different projects so I can quantitatively evaluate my time allocation, so I spent some time working on a simple, nice-to-use work tracking app. It's called Plum, and it has a super lightweight interface allowing you to define projects and log hours either manually or from starting/stopping a timer. It also allows you to set weekly/project-specific work time goals and gives you some nice weekly and monthly analytics. It's incredibly satisfying stopping a timer for a 3-hour deep work session and watching the little circle fill up towards the daily work goal. Gamifying the time that I work is very fun, and it's nice to have artifacts I can refer to attesting to consistent effort over the course of a month. Give it a try: plum-website-delta.vercel.ap…
1
3
223
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch. Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B. Learn more: ricursive.com/ripple
399
544
309
4,544
788,709
Eric Cui retweeted
We taught a robot how to move around in only 15 minutes. That’s all the mobility data our Direct Video-Action-powered robot needed to accomplish a task that requires mobility and manipulation.
12
18
17
139
18,000
Something very satisfying about slowly increasing your vocal range (Dream On here I come)
4
223
MCR concert was pretty lit
5
112
Finally bought a guitar (been borrowing friends' instruments for like 2 years lol)
4
178
INSANE
Our engineers recently visited one of our speech restoration study participants for a 72-hour hackathon. The goal: rapidly prototype as many features as possible to enhance the experience of using Neuralink to communicate with thought. Here are some of the things we built 🧵
1
96
Existing LLM-as-a-judge auto-interpretability methods seem to generate so much noise and introduce fuzzy opportunity for error at every step. Ample opportunity for prompt-based semantic drift and evaluator bias. Do there exist any common quantitatively-founded methods for natural language interpretability analysis I'm just not yet aware of?
2
2
100
I'm now beginning to truly understand the importance of active reading. There exist differences between familiarity and mastery. As a student, reading is something I do every day. However, there are subtle but important differences between skimming and active reading. There exist posters in the hallways of my dorm (the RAs decided to make the dorm Marvel-themed, resulting in my floor having an abundance of Scarlett Witch and Black Panther postery). Every day I walk through the halls, and everyday I perceive the posters in the order they are hung from my door to the exit. Keyword being "perceive". The order/content of the posters is mind-numbingly familiar to me, but if you were to ask me to describe the images in order I don't think I could. Reading in the absence of active engagement is the same. The symbols on the page can light up clusters of perception neurons in your mind and may do so with increasing speed as you revisit concepts, but until you connect them to other concepts held in your neural webbery, you haven't really integrated the information.
4
146
Finally got around to reading Iason Gabriel's "Artificial Intelligence, Values, and Alignment." He makes some compelling points, most notably that morality is not something we first determine and then align towards. Rather, the moral difficulty of AI alignment lies in how we pluralistically determine what is "fair" and "good." There exist certain principles that are universal. E.g. the right to physical autonomy, equality, etc. Part of the difficulty lies in irrational distinguishing human preference from positive intention. People are inherently irrational, and it doesn't take too much logical probing to tease this out (take, for example, the fundamental attribution error). Subtly is the subtle killer in our case.
4
114
Something I've recently been internalizing: mathematics simply provides symbolic representations for abstract concepts. It's significantly quicker to reason with concepts and translate the computation into symbols than it is to reason with symbols.
4
229
There is something magical about true active reading. Everything in your environment seems to disappear from your field of focus, and the only thing you are left with is raw data uptake.
5
118