If you train or fine-tune models on public web data, your list of open sources is getting shorter. We measured it.
On 25 September and 27 October we read robots.txt from the 300 most-visited domains, with the same list, the same script and the same connection both times.
151 of them publish a robots.txt. 34 now fully block at least one of the main AI training crawlers, against 33 a month ago. Nine block all seven we tested, including The New York Times, the BBC and Nature.
Sites treat training and search differently. 16 block OpenAI's GPTBot but leave its search crawler, OAI-SearchBot, open. OpenAI's own documentation says a site can do exactly that.
In one month, facebook.com moved from a partial rule to a full block. CNN went the other way.
A different connection changes where a request comes from. It does not change what a site has agreed to.
Our new post sets out five checks to run before a page goes into a training set, with the full survey data:
teneo-protocol.ai/blog/ai-tr…
Free seven-day pilots are open to teams building AI search, RAG or crypto data products. If you run a training pipeline, say so on the form: teneo-protocol.ai/partner
Replying to @AndrewYang
AI can provide the intelligence, robots can provide the physical labor, and abundant energy can provide the production capacity. But getting there requires solving several bottlenecks at once
Replying to @SenWarren
Advanced AI chips are strategic technology, so tighter controls, stronger supply-chain monitoring, and congressional scrutiny are understandable areas for debate.
Replying to @Kalshi
The fact that something at this scale can appear on a consumer marketplace is a pretty good illustration of how quickly the AI infrastructure market is moving.
Replying to @Hesamation
If millions of people each buy a large amount of DRAM for local AI, much of that memory could sit idle most of the day. A centralized data center can potentially pool the same scarce memory across many workloads and keep utilization much higher.
Replying to @goodside
Model collapse can happen when models are repeatedly trained on low-quality synthetic data, causing some information to be lost or distorted over generations. That is a legitimate concern, especially if synthetic data replaces diverse, high-quality human data.
Replying to @MTSlive
The interesting constraint is that software can scale almost instantly, while fabs, power generation, construction and robotics are physical systems with much slower feedback loops.
Replying to @mardehaym
A day spent removing ambiguity can save weeks of AI-assisted rework. The faster AI makes implementation, the more valuable good specification becomes.
Replying to @Kalshi
If AI-powered robots can build data centers, install power infrastructure, manufacture or deploy solar capacity, and then use the resulting compute to help build the next facility, you start getting a very different scaling loop
Replying to @emollick
The Industrial Revolution didn't just create fortunes for the people manufacturing machines. It created entire industries because suddenly everyone had access to cheaper, more powerful production.
Replying to @unusual_whales
The alleged target isn't just Nvidia. It's the AI chip supply chain, which sits at the center of the U.S.–China technology competition.
Replying to @PeterDiamandis
A data center shouldn't be evaluated only by how much compute it adds. If it consumes enormous amounts of electricity, land, water, and infrastructure, the surrounding community should see tangible benefits too.
Replying to @levelsio
You could probably prototype a much better citizen portal, automate permits, simplify tax processes, build transparent public dashboards, and eliminate a ridiculous amount of bureaucratic friction.
Replying to @luckeyfaraday
A data center shouldn't be evaluated only by how much compute it adds. If it consumes enormous amounts of electricity, land, water, and infrastructure, the surrounding community should see tangible benefits too.
Replying to @luckeyfaraday
Most people argue about how AI will change work. This argument asks whether work itself should eventually become optional.
Teneo Protocol retweeted
You can try it with Codex and Claude as well😎
Replying to @MTSlive
If AI becomes capable of planning, evaluating performance, allocating resources, coordinating teams, and making operational decisions, companies may not just use AI to replace individual tasks.
Replying to @SenWarren
If you believe a technology could create civilization-scale risks, disclosure alone isn't the end of the conversation.
Replying to @Kalshi
When people working directly on AI safety start treating catastrophic risk as a serious enough possibility to change personal life decisions, it shows how different their risk assessment can be from the mainstream AI narrative.
Replying to @a16z
GPT-4o essentially hit a ceiling almost immediately. Newer models keep improving as you give them more tokens to reason through the attack, with the strongest runs completing all 32 stages.



