Michael Wolfrath retweeted
The Lean AI Handbook โ Building, Deploying, and Scaling AI for Business Value: link.amazon/B0fOxsNuG
Michael Wolfrath retweeted
GPU architecture, clearly explained.
The usual assumption is that a faster GPU means more compute, so a chip rated for more operations per second should generate more tokens per second.
That rarely holds. You can rent a top-end chip, load a model, watch utilization sit high, and still get a few dozen tokens per second out of hardware capable of a thousand trillion operations.
The arithmetic units are not the limit. They spend most of their time waiting for numbers to arrive.
Memory bandwidth is how fast those numbers can be delivered, and it grows far more slowly than arithmetic capability with each new generation.
The H100 makes this concrete. It does 989 trillion operations per second at 16-bit precision, while its memory delivers 3.35 trillion bytes per second.
Divide one by the other and you get about 295. That is how many operations the chip has to perform on every byte it fetches just to keep its arithmetic units busy.
Generating a token comes nowhere close. Each weight gets multiplied by one number and added into a running total, so two operations per weight, and each weight costs 2 bytes to read at 16-bit precision.
That works out to 1 operation per byte, roughly three hundred times short of what the chip needs.
Once you see that, the layout of a GPU stops looking arbitrary. All of it is organized around distance, because every trip to fetch a value is the expensive part.
Execution is nested in four steps, smallest first.
โ Thread. The smallest unit of work, doing arithmetic on its own handful of values.
โ Warp. A group of 32 threads that move in lockstep and share a single instruction, which is what the hardware actually schedules.
โ Thread block. A batch of warps, up to 32 of them or 1024 threads, handed to one compute unit and staying there until it finishes.
โ SM. The streaming multiprocessor, a small self-contained machine with its own arithmetic units, its own storage, and its own scheduler. An H100 has up to 132 of them.
Memory is nested along the same lines, and that is the part that decides your speed.
โ Registers. 256 KB per SM, split so each thread gets a private slice nobody else can read. A value here already sits next to the thing that needs it, so reading it costs nothing worth measuring.
โ Shared memory and L1. A few hundred KB per SM. Shared memory is the only level you place things into yourself, and every thread in a block can read it, which is where the name comes from.
โ L2 cache. Around 50 MB sitting below all the SMs, and the first level every one of them can see. Reaching it means leaving your own SM.
โ HBM (High Bandwidth Memory). Around 80 GB, holding model weights, the KV cache, and activations. It sits outside the processing chip entirely, and reaching it is the slowest thing a GPU does.
The nesting is what buys you speed. Anything that stays inside one SM is fast because nobody outside needs to see it, and anything the whole chip has to agree on pays for the trip down.
Add up the registers across every SM and you get roughly as much storage as the L2 underneath them, which is nothing like the neat pyramid you see on a CPU.
So making a GPU faster is mostly about how many round trips to HBM you can avoid. FlashAttention is the clearest case, since ordinary attention writes a large intermediate result all the way down and reads it back, while the tiled version keeps it up on the SM and runs far faster doing the same arithmetic.
I wrote the full breakdown of how a GPU works, from this layout up to the ratio that sets your token rate. The article is quoted below.
I have also put together a 10-week, 30-minutes-a-day roadmap for engineers who want to actually run LLM inference in production, not just read about it.
Check this out: github.com/patchy631/time-toโฆ
(don't forget to star ๐)
Stay tuned for more on this!
Michael Wolfrath retweeted
(Finally) wrote up "Understanding Positional Embeddings in Modern LLMs."
A RoPE from-scratch tutorial missing from my LLM building blocks collection + what recent architectures do (NoPE, partial RoPE, hybrids).
Prob not the flashiest topic, but essential for understanding LLMs
Michael Wolfrath retweeted
โ ๏ธ Cyber threats are evolving fast. AI, identity risks and cryptographic change demand urgent action.
CISOs should focus on these three projects that address the most urgent risks for 2026: gtnr.it/3VjLXL3
Michael Wolfrath retweeted
Systems Thinking for Agentic AI โ Design reliable LLM and agentic AI systems with RAG, MCP, guardrails, evaluation, and observability: link.amazon/B0exzW2nC
Michael Wolfrath retweeted
AI agents can perform increasingly sophisticated work, but enterprise adoption is running into a reality that many organizations underestimate:
Greater autonomy also introduces greater risk โ ๏ธ
Hereโs how you can stay ahead of the risks as agentic AI scales: gtnr.it/4zo5fNV
Michael Wolfrath retweeted
Ontology Pipeline โ Framework for Knowledge Engineering and Building Knowledge Infrastructures: link.amazon/B06B2xJy5
Amazon description: โOne of the biggest shifts happening in our industry is that enterprise data management is expanding into enterprise knowledge management. AI is accelerating that transition because it depends on more than well-managed data. It depends upon context, shared meaning, trusted relationships, and the ability to organize the vast stores of unstructured knowledge that exist across every large enterprise. As AI becomes the primary interface to enterprise information, knowledge management will become the foundation that unlocks trustworthy AI at scale.โ
Michael Wolfrath retweeted
JUST IN :
๐บ๐ธ๐จ๐ณ CNN now reports that the U.S. Intel is now unsure whether woman connected to Huang spied on Nvidia for China
Michael Wolfrath retweeted
๐ฎ Gartner predicts that by 2029, at least 70% of organizations with production agentic AI in I&O will experience a material service, security or cost incident linked in part to insufficient runtime controls.
Find out with infrastructure enterprises need to support agentic AI at scale: gtnr.it/4pAS2Of
Michael Wolfrath retweeted
Beginner's guide to building AI agents on AWS: link.amazon/B05CVQCJk from @PacktPublishing
๐๐ฒ๐ ๐๐ฒ๐ฎ๐๐๐ฟ๐ฒ๐:
๐ฃProgress from agentic AI concepts to production-ready deployment on AWS
๐ฃDesign and build AI agents with tools, memory, and orchestration
๐ฃDeploy, evaluate, and govern agents using Amazon Bedrock AgentCore
๐ช๐ต๐ฎ๐ ๐ฌ๐ผ๐ ๐ช๐ถ๐น๐น ๐๐ฒ๐ฎ๐ฟ๐ป:
๐ Understand how AI agents think, act, and collaborate
๐ Design single-agent systems with tools and reasoning
๐ Build agents on AWS using Bedrock and SageMaker AI
๐ Add memory for context-aware and adaptive behavior
๐ Create multi-agent systems and orchestrated workflows
๐ Use MCP and A2A for agent communication and tools
๐ Deploy and scale agents for production environments
๐ Monitor, evaluate, and govern agent performance
Michael Wolfrath retweeted
Gartner's 2026 Hype Cycle for GenAI report reveals urgent innovation prioritiesโ๏ธ
There's a flood of GenAI technologies โ but only a few deliver lasting value.
Discover the 4๏ธโฃ innovation areas shaping GenAI's trajectory and learn which technologies are best positioned to drive value for your organization: gtnr.it/4xUEqjt
Michael Wolfrath retweeted
Think AI adoption is just a technology challenge? Think again.
AI initiatives often require 25% more training and up to 200% more change management effort than traditional technology rollouts.
Learn how to prepare your workforce for success: gtnr.it/3XhTkDB

ALT A Gartner graph illustrating AI adoption levels: "AI dropouts" in red hesitate or refuse AI; "AI ambivalents" in yellow are uncertain; "AI achievers" in green actively adopt AI for growth. A blue curved line shows improvement after actions. Strategies include establishing accountability, building partnerships, empowering with education, and modeling AI adoption practices.
Michael Wolfrath retweeted
AI has moved from experimentation to execution, requiring CIOs to translate growing AI investments into tangible business outcomes at scale.
๐ก CIOs must act as value orchestrators โ ensuring AI drives operational efficiency, unlocks new growth opportunities and improves profitability: gtnr.it/4xNxXqj
Michael Wolfrath retweeted
What made Gartner's Top 10 Strategic Technology Trends for 2027, announced at #GartnerSYM Orlando?
Discover the technologies and shifts expected to have the biggest impact on business, AI, security and digital resilience: gtnr.it/4icV895
Michael Wolfrath retweeted
Experiencing slow LLM inference? There are ways to improve it.
Our free ebook, Inference Engineering, brings together five chapters on model efficiency, inference optimization, production deployment, and GPU programming.
Get your free ebook, so you can put those techniques to work: hubs.ly/Q04DWyBL0
The featured ebooks:
โข Domain-Specific Small Language Models by @GuglielmoIozzia
โข LLMs in Production
โข Quantization and Fast Inference by @vivekkalyanaran
โข Rearchitecting LLMs by @PereMartra
โข GPU Programming with Triton by @emharsha_1812
Michael Wolfrath retweeted
How many holes are in shirt?
No one has been able to get correctly
Winner gets
$11,000
Ends in 90 hours
Michael Wolfrath retweeted
What made Gartner's Top 10 Strategic Technology Trends for 2027, announced at #GartnerSYM Orlando?
Discover the technologies and shifts expected to have the biggest impact on business, AI, security and digital resilience: gtnr.it/4zm5jhJ
Michael Wolfrath retweeted
Andrej Karpathy put his whole AI course on GitHub for free.
7 repos, in the order to learn them
โ nn-zero-to-hero, his video course that builds neural networks from scratch
github.com/karpathy/nn-zero-โฆ
โ micrograd, the math that makes AI learn, in a tiny amount of code
github.com/karpathy/micrograโฆ
โ makemore, a small model that learns to invent new names
github.com/karpathy/makemore
โ minbpe, how AI chops your words into tokens
github.com/karpathy/minbpe
โ nanoGPT, train your own GPT model
github.com/karpathy/nanoGPT
โ nanochat, "the best ChatGPT that $100 can buy"
github.com/karpathy/nanochat
โ autoresearch, AI agents that run AI experiments on one GPU by themselves
github.com/karpathy/autoreseโฆ
284,000 GitHub stars between them.
Everything you need to understand how ChatGPT works is free and open source.
Michael Wolfrath retweeted
An AI Workflow with fixed steps beats an autonomous AI Agent in 90% of real apps, yet devs force Agents.

ALT AI Agent vs AI Workflow comparison chart by @PythonDvz across Concept, Architecture, Execution Flow, and Runtime: contrasting autonomous perceive-plan-act-observe loops in LangChain, LangGraph, AutoGen, and CrewAI against fixed-order orchestration pipelines using Apache Airflow, AWS Step Functions, and Prefect.
Michael Wolfrath retweeted
AI Architect Tech Stack for 2026โ27 Explained! ๐
Knowing how to build an AI chatbot is one thing.
Designing an enterprise AI system that scales, stays secure, and works reliably in production is another.
That's where AI Architects come in.
Here are the 9 essential layers you should understand:
โ Programming & Development
Python, TypeScript, C#, SQL, and GitHub form the foundation for building AI applications.
โ Models & AI Platforms
Microsoft Foundry, Azure OpenAI, OpenAI, Anthropic, and Hugging Face provide access to AI models.
โ Agents & Orchestration
Microsoft Agent Framework, LangGraph, Semantic Kernel, CrewAI, and LlamaIndex coordinate agent workflows.
โ RAG & Knowledge Systems
Azure AI Search, Pinecone, Weaviate, PostgreSQL, and Neo4j enable knowledge retrieval.
โ Data & Processing
Microsoft Fabric, Databricks, Snowflake, Spark, and Kafka support enterprise data pipelines.
โ APIs & Integration
FastAPI, Azure API Management, GraphQL, MCP, and Kafka connect AI systems.
โ Cloud & Infrastructure
Azure, AWS, Google Cloud, Docker, and Kubernetes support deployment and scaling.
โ Security & Governance
Entra ID, Purview, Key Vault, OPA, and HashiCorp Vault help protect AI systems.
โ MLOps & Observability
MLflow, GitHub Actions, LangSmith, Arize, and Grafana monitor performance.
The goal isn't to master every tool. It's to understand how the entire architecture fits together.
Which layer are you focusing on?
๐๐ฒ๐ฐ๐ผ๐บ๐ฒ ๐ฏ๐ฒ๐๐๐ฒ๐ฟ ๐ฎ๐ ๐๐ ๐ถ๐ป ๐ท๐๐๐ ๐ญ ๐บ๐ถ๐ป๐๐๐ฒ ๐ฎ ๐ฑ๐ฎ๐. ๐๐ผ๐ถ๐ป ๐บ๐ ๐๐ฒ๐ฒ๐ธ๐น๐ ๐ป๐ฒ๐๐๐น๐ฒ๐๐๐ฒ๐ฟ ๐๐ต๐ฒ๐ฟ๐ฒ ๐ ๐ฑ๐ผ๐ฐ๐๐บ๐ฒ๐ป๐ ๐๐ต๐ฒ ๐ฟ๐ฒ๐ฎ๐น-๐๐ผ๐ฟ๐น๐ฑ ๐ท๐ผ๐๐ฟ๐ป๐ฒ๐ ๐ผ๐ณ ๐๐ ๐๐ฟ๐ฎ๐ป๐๐ณ๐ผ๐ฟ๐บ๐ฎ๐๐ถ๐ผ๐ป.
๐ ๐ฆ๐ถ๐ด๐ป ๐๐ฝ ๐ณ๐ฟ๐ฒ๐ฒ now โ avsl.beehiiv.com/
Join Me On Youtube And Insta :
Youtube : lnkd.in/e3UVBNSk
Instagram : lnkd.in/ebWK5b6V
Follow @AiswaryaVenkit1 for more such insights!!






























