Reinforcement learning via market feedback
Manhattan, NY
Joined February 2023
- Tweets10.5K
- Following3K
- Followers3.4K
- Likes12.5K
Pinned Tweet
The best regulatory solution to AI safety is for Congress to use Antitrust regulations to break up the big labs and impose a hard cap (say 5% or 10%) on the amount of national compute stock one entity can own unless the labs are fully open-source.
This is the only way we can win the race against China. China is massively ahead in *diffusion of AI into the general economy* in part because they are not drunk on GPU-flops and have a diverse open-source model ecology that privileges on-device access, density scaling, and embedding neural methods into traditional computational workflows.
We thus need distribution of compute resources broadly into the economy. Historical precedent for discovery of a new resource economy is exactly the reason we have antitrust regulation in the first place. My proposal is simply "Standard Oil Trust-Bust 2.0".
This is also the best solution to a "winner-takes-all" doomsday scenario. Most of these scenarios assume one lab pulls ahead dramatically.
If we break up the current "Big Five" labs into 25-50 independent organizations, our model ecology will develop much more diverse features; the jaggedness will be a feature, not a bug.
This is also a strong solution for concerns about intellectual property appropriation, such as from researchers and from corporates. A regulatory framework protecting distillation is the first step to preventing scenarios where a few labs become a hegemonic force in the US economy.
China is massively ahead of the US in diffusion. Part of this is *because of* the GPU deficiency. Chinese research is ahead on architectures that assume many smaller, locally deployed models, and like the Deepseek release, that take a more holistic approach to utilizing compute resources.
Any talk of slowing down U.S. clusters is a PRC industrial-policy win whether or not a PRC officer asked for it.
today we announce something truly special
introducing: Jesse v1
this model architecture is all new, from scratch, built and trained using public, popular datasets
it consumes 0 tokens, and does not have weights
it reasons, it believes, and it's very, very fast
"Everyone who understands this is on some payroll"
That much has been evident
Palantir CEO Alex Karp on what model companies are really asking for when they call for regulation and why they will be nationalized.
“It's a weird twist. They're asking for societal regulation to get out of the first line of defense, which is if you build a technology that can destroy 10% of the of the world, that has civil and criminal liability attached to it.”
“The only way to deal with this kind of liability is to go to the government and say nationalize us, please.”
"Another problem we have is seemingly everyone who understands this is on some payroll."
one reason you'd want to pace the frontier is dependent on how reliant you are on customer data
i imagine labs have noticed a decrease in quality since goal-following / autonomous workflows have gained popularity
this may accelerate mode collapse
Merovingian retweeted
Eccentricity in individuals is charming. Eccentricity in groups suggests that beliefs are functioning as a marker of membership. nitter.cf/CNLiberalism/status/21…
Merovingian retweeted
Two radically different projects operate under the banner of “AI safety.”
Pro-Human Safety is not Effective Altruist Lab Safety.
Merovingian retweeted
right??? comrades, you must save chairman bernie from such tentacular, neoliberal and therefore fascist snares before it's too late.
hasta siempre compañeros!
Merovingian retweeted
This book is terrible. These scenarios all involve this bizarre leap where the AI
“intelligence” is so impressive that all power just drifts to it or is handed to it because it would be negligent not to do so, but then its “alignment” problem kills us.
That is, they imagine a world in which, contrary to all human history and sociology, humans cease all disagreement about values, priorities, politics, in favor of trusting the one mechanical expert whose reasoning they can neither understand nor verify.
The only way to stop it is for heroic reddit athesists to think really carefully about how to align it to their, frankly, juvenile utilitarianism.
The doomsday part is just rube stuff. The theory about human impotence and machine superintelligence is a parody of Calvin’s doctrine of the bondage of the will and the sovereignty of God
Is there a possibility for real harms from malware? Yes. Could there even be catastrophes? Sure. But spare me.
The AI superintelligence theory proceeds from a reddit atheist anthropology that the way to the right values and priorities is being smarter. The actual evidence of human history is that integrity and loving the right things is the basis of true wisdom.
Merovingian retweeted
I don't have enough arguments right now to defend this position but Salesforce is going to own Anthropic
Live: @DarioAmodei x @Benioff
The @AnthropicAI co-founder & CEO joins us at @Dreamforce for a conversation worth hearing ☁️
Merovingian retweeted
A shockingly high percentage of AI safety discourse is centered around low probability risks like “what if we all get turned into paperclips” and a shockingly low percentage of AI safety is centered around very high probability risks like “what if one group gets to inscribe its dubious values into the fabric of possible thought.”
If scenario A is one that is unlikely to happen but conceivable, scenario B is already happening, RIGHT NOW.
As a user, 0% of the time when I bump up against safety guardrails am I being prevented from paperclipping, and 100% of the time I bump up against guardrails I am being gatekept out of exploring true things that are controversial.
Merovingian retweeted
Power overconcentration is the greatest risk and the one with historical precedent of catastrophic suffering
A shockingly high percentage of AI safety discourse is centered around low probability risks like “what if we all get turned into paperclips” and a shockingly low percentage of AI safety is centered around very high probability risks like “what if one group gets to inscribe its dubious values into the fabric of possible thought.”
If scenario A is one that is unlikely to happen but conceivable, scenario B is already happening, RIGHT NOW.
As a user, 0% of the time when I bump up against safety guardrails am I being prevented from paperclipping, and 100% of the time I bump up against guardrails I am being gatekept out of exploring true things that are controversial.
Merovingian retweeted
Being better than humans at searching and writing down the formal proof of a theorem does not equate being "beyond human level at math".
This is not all of math, any more than arithmetics, or computing integrals (symbolically or numerically) is all of math. It's one "mechanical" task in the whole activity that happens to be automatable.
Mathematicians invent new concepts, new frameworks, new abstractions, new definitions, and formulate conjectures.
This requires intuition and creativity that current AI systems do not have (yet).
Merovingian retweeted
One of the most frustrating aspects of modern discourse is this idea that if somebody proposes a solution to a problem, and you don’t like their solution, then you are denying the problem
Maybe the first main example of this is the term “climate denier” if you don’t go along with the idea that a small group of people should determine energy and food policy for the whole world
Then we saw it with Covid where if you didn’t go along with absolutely insane policies then you were somehow denying that Covid was a real disease
And now we have the same thing when it comes to AI policy
Yes, of course any powerful technology is dangerous and could do a lot of harm, but that doesn’t mean that I want the rainbow mafia from Berkeley to decide for the whole planet who gets access to intelligence
I also don’t actually even believe that any human action is going to make any difference when it comes to the long-term danger posed by AI
If AI is really going to be as smart as people think it’s going to be, then it’s going to outsmart whatever we do anyway, so it’s really just in God’s hands and it’s either going to work out or it’s not
In the meantime, I would rather myself and my family be free rather than prisoners in Dario’s zoo
Urgent that we disenfranchise "alignment research", "safteyism", "existential risk", "p(doom)-ism". These are memetic viruses that require containment. They compromise the underlying engineering efforts.
We should ensure that large AI labs like Anthropic build a Living Will. They must enumerate a plan, incase of financial trouble, how courts can liquidate effectively the system.
Of course, this would not include contingent claims (like pre-specify entities where the will would become conditional on their survival).
If they are indeed developing dangerous technologies, it is essential that a neutral process can rapidly break up the organization in the event of financial troubles, like is standard with GSIB financial institutions.
The best regulatory solution to AI safety is for Congress to use Antitrust regulations to break up the big labs and impose a hard cap (say 5% or 10%) on the amount of national compute stock one entity can own unless the labs are fully open-source.
This is the only way we can win the race against China. China is massively ahead in *diffusion of AI into the general economy* in part because they are not drunk on GPU-flops and have a diverse open-source model ecology that privileges on-device access, density scaling, and embedding neural methods into traditional computational workflows.
We thus need distribution of compute resources broadly into the economy. Historical precedent for discovery of a new resource economy is exactly the reason we have antitrust regulation in the first place. My proposal is simply "Standard Oil Trust-Bust 2.0".
This is also the best solution to a "winner-takes-all" doomsday scenario. Most of these scenarios assume one lab pulls ahead dramatically.
If we break up the current "Big Five" labs into 25-50 independent organizations, our model ecology will develop much more diverse features; the jaggedness will be a feature, not a bug.
This is also a strong solution for concerns about intellectual property appropriation, such as from researchers and from corporates. A regulatory framework protecting distillation is the first step to preventing scenarios where a few labs become a hegemonic force in the US economy.
Merovingian retweeted
Satya Nadella talks about how in Quincy, Washington, a 400/500 MW data center has contributed to a rural town over many years.
the tax revenues have gone up 12X, continued economic growth, new infrastructure, and public infra such as a school, hospital, town center, and aquatic center.
----
From "All-In Podcast" YouTube channel, (full video link in comment)