Tech with Mak

Claude Code can now preview what a knowledge graph change will make true before it ships. Fabio Rovai has open-sourced Open Ontologies, a Rust-based engine that brings Terraform-style change management, semantic reasoning, and independently checkable proofs to knowledge graphs. The idea is to treat ontologies as production infrastructure rather than static files that AI agents can freely modify. Through MCP, agents such as Claude Code and Cursor can build and query ontologies, validate constraints, propose changes, inspect their consequences, and roll back updates. The repository also includes a pipeline for turning structured data into knowledge graphs. It can ingest CSV, Excel, JSON, Parquet, and SQL query results, map them into RDF, validate them with SHACL, and derive additional relationships through reasoning. But the change-management system is where the engineering becomes particularly interesting. A normal Git diff tells you which statements changed. It cannot tell you every new logical conclusion those statements make possible. The repository demonstrates this with a single RDF domain declaration. The structural blast-radius report shows zero affected triples and a low risk score. Yet OWL reasoning discovers 901 additional consequences. Open Ontologies detects that semantic change before the proposed update is applied. There's also a separate verification mechanism. For supported inferences, the engine produces derivation certificates containing the original premises, the applied rules, and the resulting conclusions. An independent Lean 4 checker verifies those certificates without trusting the Rust reasoning engine. The repository demonstrates this with 259 certified inferences and a deliberately forged conclusion that the checker rejects. The distinction matters. The proof establishes that a conclusion follows from the supplied facts and rules. It doesn't establish that the original facts are true. Open Ontologies also supports drift detection, rollback, historical evidence, and checks that determine whether a retrieved knowledge-graph subset still supports particular claims. The entire engine is open source, MIT licensed, and available as a standalone Rust binary. A pure-Python implementation is also available. For AI systems that maintain structured knowledge over time, the challenge isn't simply retrieving the right information. It's controlling what changes, understanding the consequences, and preserving evidence that another system can verify. Here's the GitHub Repo: github.com/fabio-rovai/open-…

This 12.5K-star GitHub repo is basically an entire LLM engineering course hiding inside one codebase. It starts with raw text and builds a Transformer in plain PyTorch, then keeps going through pretraining, supervised fine-tuning, reward modeling, preference optimization and reinforcement learning. The repository implements: → Tokenization, attention, Transformer blocks and pretraining → SFT with assistant-token loss masking → Bradley-Terry reward modeling from preference pairs → DPO, ORPO and KTO for preference optimization → PPO with a value head and Generalized Advantage Estimation → GRPO with group-relative rewards and GSM8K evaluation These aren't just disconnected implementations. The post-training methods build around the same underlying Transformer, so you can examine how the training objectives, reward calculations and optimization loops change at each stage. There's also a modern architecture implementation covering RoPE, RMSNorm, SwiGLU, GQA, MLA, optional Mixture-of-Experts and KV caching. The repository even includes a 369K-parameter model trained on TinyStories in approximately five minutes on the author's laptop CPU, with a small BPE tokenizer built from scratch. That's a small educational pretraining experiment. Larger models and the full post-training pipeline require considerably more compute. Everything comes with runnable training scripts, tests, evaluation tools, documentation and a Streamlit control panel. Fully open source. Here's the GitHub Repo: github.com/FareedKhan-dev/tr…

Stanford put its entire LLM engineering course on YouTube. The homework includes writing FlashAttention in Triton. Its called CS336: Language Modeling from Scratch, taught by Percy Liang and Tatsunori Hashimoto. The Spring 2026 course covers the complete process of building and training language models, including the systems engineering required to make them efficient. The lectures cover: → Tokenization and Transformer architectures → Attention alternatives and Mixture of Experts → GPU architecture, memory and compute → Triton kernels and FlashAttention → Distributed training and parallelism → Scaling laws and inference optimization → Pretraining data pipelines and evaluation → Supervised fine-tuning, RLHF and reasoning RL But the assignments are where this course becomes particularly valuable. Students implement their own tokenizer, Transformer and optimizer before training a small language model. They then profile the implementation, write FlashAttention-2 in Triton, and develop memory-efficient distributed training. Later assignments involve fitting scaling laws, processing raw Common Crawl data, implementing filtering and deduplication, and using supervised fine-tuning and reinforcement learning to train models for mathematical reasoning. Stanford provides minimal scaffolding and even recommends disabling AI autocomplete during the assignments so students actually understand the code they're writing. The full lecture playlist, assignment handouts and lecture materials are publicly available. Some practical exercises require GPU compute. For ML engineers who want to understand the full stack, from training data and attention kernels to inference and post-training, this is an excellent course to work through. YouTube Playlist: youtube.com/playlist?list=PL… Course & Assignments: cs336.stanford.edu/ Lecture Code: github.com/stanford-cs336/le…

The π in the normal distribution has a geometric origin. It comes from a famous mathematical result known as the Gaussian integral. ∫₋∞^∞ e^(−x²) dx = √π The interesting part is that e^(−x²) has no elementary antiderivative. Yet its integral over the entire real line can be calculated exactly. The solution involves turning a one-dimensional integration problem into a two-dimensional one. The resulting function, e^(−(x²+y²)), has circular symmetry, and polar coordinates reveal where π enters the calculation. This result is fundamental to probability theory. The familiar factor 1/(σ√(2π)) in the normal distribution comes from the same integral. It ensures that the total probability across all possible values is exactly 1. But its applications extend well beyond statistics. In engineering, Gaussian distributions are commonly used to model measurement errors and noise. With independent, zero-mean Gaussian errors of equal variance in an additive-error model, maximum-likelihood estimation is equivalent to minimizing the sum of squared residuals. Gaussian functions also appear in heat diffusion, where they describe how an initially concentrated point source spreads through an unbounded, homogeneous medium. In Fourier analysis, the transform of a Gaussian is another Gaussian, up to scaling. This property is useful in signal processing and image filtering. The same mathematical function connects probability distributions, statistical estimation, heat transfer, and frequency analysis. And the √π in its integral ultimately comes from the geometry of the plane.

Learn Harness Engineering 🚀 Found this excellent open-source course by Walking Labs, covering 14 lectures and 8 hands-on projects on building more reliable AI coding agents. What's inside: → Agent instructions (AGENTS.md / CLAUDE.md) → Context management and persistent memory → Multi-session continuity and state tracking → Task boundaries and scope control → Testing, verification, and observability → Automated agent loops and workflows → Multi-agent orchestration and graph engineering The practical projects walk you through building a complete agent harness, comparing prompt-only execution with structured workflows. Also includes ready-to-use templates, verification scripts, and a harness-creator skill. Perfect for developers working with Claude Code, Codex, and other AI coding agents who want to understand the engineering behind reliable agent execution. Completely free. Here's the GitHub Repo: github.com/walkinglabs/learn…

Someone put 542 AI engineering lessons on GitHub, and I really like the way this curriculum teaches. You build the underlying algorithm yourself before moving to the production library. → Backpropagation from the math. → A BPE tokenizer from scratch. → Self-attention and transformers. → An AI agent loop with tool calling. Then you work with the libraries that implement the same ideas. It's called AI Engineering from Scratch, and it covers 20 phases, from mathematics and deep learning to LLM engineering, inference optimization, RAG, MCP, AI agents, and production systems. The lessons follow a consistent structure: MOTTO → PROBLEM → CONCEPT → BUILD IT → USE IT → SHIP IT There are some excellent engineering projects, too. → Build a retrieval evaluator that catches ranking regressions. → Build an agent trace debugger in TypeScript. → Build a tool-call firewall in Rust. The repository includes runnable code, exercises, tests, and reusable agent skills. You can also install the curriculum directly into Claude Code or Codex and use your coding agent as a tutor. It starts with a 10-question placement quiz, creates a personalized learning plan, and guides you through lessons with explanations, code, and quizzes. Install: npx skills add rohitg00/ai-engineering-from-scratch There are also six downloadable PDF/EPUB volumes, with implementations across Python, TypeScript, Rust, and Julia. Completely free. MIT licensed. Over 69,000 GitHub stars. github.com/rohitg00/ai-engin…

A curated list of 25 LLM foundations interview questions. What's covered Core mechanics: → What happens from input text to first output token → Autoregressive modeling and chain-rule decomposition → Logits, softmax, and how probabilities are formed Decoding & training objective: → Greedy decoding vs temperature, top-k, top-p → Cross-entropy loss and why it's the natural LM objective → Perplexity, and why lower perplexity ≠ better assistant Tokenization: → Why LLMs don't just operate on words → BPE vs WordPiece vs Unigram vs SentencePiece → Vocabulary size trade-offs → Special tokens (BOS/EOS/chat templates) and why misuse hurts performance Context & representations: → Token embeddings vs contextual hidden states vs retrieval embeddings → What actually consumes the context window → Why "fits in context" ≠ "used reliably" (Lost in the Middle) Prompting & architectures: → Zero-shot / one-shot / few-shot prompting → In-context learning vs fine-tuning → Encoder-only vs decoder-only vs encoder-decoder → Causal vs masked language modeling → Base models vs instruction-tuned/chat models Scaling & failure modes: → Parameter count, what it does and doesn't tell you → Neural scaling laws (Kaplan et al.) → The Chinchilla compute-optimal insight → Data quality/dedup vs raw token count → Memorization vs generalization → Why fluent answers can still be wrong Every answer is grounded in the actual papers (Vaswani, Brown, Devlin, Kaplan, Hoffmann, Lee, Liu, Carlini).

Jay Alammar is the best teacher in AI. Period. If you have ever seen "The Illustrated Transformer," you know his diagrams are legendary. He also open-sourced the entire codebase for his O'Reilly book: Hands-On Large Language Models. It’s effectively a visual masterclass in LLMs for free. Chapter 1: Introduction to Language Models Chapter 2: Tokens and Embeddings Chapter 3: Looking Inside Transformer LLMs Chapter 4: Text Classification Chapter 5: Text Clustering and Topic Modeling Chapter 6: Prompt Engineering Chapter 7: Advanced Text Generation Techniques and Tools Chapter 8: Semantic Search and Retrieval-Augmented Generation Chapter 9: Multimodal Large Language Models Chapter 10: Creating Text Embedding Models Chapter 11: Fine-tuning Representation Models for Classification Chapter 12: Fine-tuning Generation Models I will put the repo link in the comments.

The ultimate AI/ML learning stack. From fundamentals to production systems. 20 books. 5 categories. 📚 Fundamentals: → Hands-On ML with Scikit-Learn, Keras & TensorFlow (Géron) → Data Engineering with Python → Storytelling with Data (Knaflic) 📚 Time Series: → Modern Time Series Forecasting with Python → Applied Time Series Analysis and Forecasting 📚 System Design: → Designing Machine Learning Systems (Huyen) → AI Engineering (Huyen) → Solutions Architect's Handbook → Practical ML for Computer Vision 📚 GenAI: → LLM Design Patterns (Ken Huang) → Generative AI with LangChain → Generative Deep Learning (O'Reilly) → AI Agents in Practice → Architecting AI Software Systems 📚 Product/Strategy: → The Profitable AI Advantage → Your AI Survival Guide → Reimagined: Building Products with Generative AI Bookmark this stack.

"Do not learn to code" is the worst career advice of the decade. People are telling college students to skip Computer Science because AI will just automate it all. Andrew Ng just killed this myth at Stanford with a brilliant analogy. When he tried to generate images with Midjourney, he typed: "make pretty pictures of robots" and got garbage. His collaborator, however, understood Art History. He knew the exact vocabulary of lighting, genre, and palette. He spoke the "language of art," and generated masterpieces. Andrew Ng is seeing the exact same thing happen in software engineering right now. AI didn't replace the need to understand Computer Science. It made Computer Science the required vocabulary to control the AI. If you don't understand how computers actually work, you are just typing "make a pretty app" into Cursor and shipping fragile, unscalable logic. Here is Andrew Ng's exact hiring hierarchy today: Level 1: 10 years of experience, but codes by hand (He won't hire them). Level 2: Fresh college grad, but highly fluent in AI-assisted coding (He hires them over the 10-year veteran). Level 3 (God Tier): Deeply understands CS fundamentals AND uses AI-assisted coding. When humanity went from punch cards to keyboards, coding got easier, and more people coded. We are at that exact inflection point again. AI doesn't replace fundamentals. It multiplies them.

Build Deep Learning from Scratch 🧠 One of the most famous deep learning playlists, 26 lectures covering deep learning from first principles. What's inside: → Neurons and how they learn → Backpropagation, derived by hand → CNNs, RNNs, LSTMs → Transformers and self-attention → Optimizers, autoencoders, GANs and more. Good for anyone who wants to actually understand the math behind modern AI. 🔗 Playlist link in comments

In 1683, a Swiss mathematician named Jacob Bernoulli asked a simple question about money. If you deposit $1 at 100% annual interest, what happens if you compound it more and more frequently? Once a year → $2.00 Twice a year → $2.25 Monthly → $2.613... Daily → $2.714... The more frequently you compound, the higher it goes, but it never grows without limit. It converges. Bernoulli proved that limit exists: e = lim (1 + 1/n)ⁿ as n→∞ He couldn't calculate it exactly, but showed it had to fall between 2 and 3. That limit is e ≈ 2.71828... He didn't name it. Fifty years later, Leonhard Euler calculated it to 18 decimal places, and the letter e stuck - appearing first in Euler's letter to Goldbach in 1731. Why e is special: e is the only number where the exponential function eˣ is its own derivative. That one property sounds abstract. In practice it means: when you differentiate eˣ, you get eˣ back. The rate of change equals the value itself. This makes the calculus clean. No extra constants, no messy chain rule artifacts. And clean calculus is everything when you're training a neural network through millions of gradient updates. Where e lives in every AI system you build: 1/ Sigmoid function the S-shaped function that maps any real number to a value between 0 and 1. σ(x) = 1 / (1 + e⁻ˣ) Used in logistic regression and binary classification. e is what makes its derivative expressible in terms of itself - σ'(x) = σ(x) · (1 - σ(x)), which keeps backpropagation computationally clean. 2/ Softmax the function that converts raw model output scores (logits) into a probability distribution across multiple classes. softmax(xᵢ) = eˣⁱ / Σeˣʲ Every token probability your LLM produces passes through this. e is used because it keeps all values positive and makes the gradients during training well-behaved. 3/ Cross-entropy loss the loss function used to train most classifiers and language models. It uses the natural logarithm (ln), which is the inverse of e. The full picture: A banker's question in 1683 → a universal mathematical constant → the backbone of every neural network trained today. Bernoulli was thinking about interest rates. He had no idea he was laying the foundation for gradient descent. That's what makes pure mathematics dangerous. It doesn't look useful until suddenly it's everywhere.

This math sits underneath every AI model being trained right now. Gradient. Jacobian. Hessian. Three words that look intimidating at first. But they are really just three ways of measuring change. 𝟭. 𝗚𝗿𝗮𝗱𝗶𝗲𝗻𝘁 ∇f Takes a scalar function: f : ℝⁿ → ℝ Returns a vector of first-order partial derivatives. It answers: "Which direction makes f increase fastest?" That is why gradients are central to optimization. Gradient descent moves in the opposite direction because the gradient points uphill. Backpropagation efficiently computes gradients during training. 𝟮. 𝗝𝗮𝗰𝗼𝗯𝗶𝗮𝗻 J_F Takes a vector-valued function: F : ℝⁿ → ℝᵐ Returns an m × n matrix of first-order partial derivatives. It answers: "How does each output change with each input?" The Jacobian is the local linear map of a vector-valued function. It shows up in: → sensitivity analysis → change of variables → automatic differentiation → forward-mode AD → reverse-mode AD / backpropagation In simple terms: forward-mode AD uses Jacobian-vector products. reverse-mode AD uses vector-Jacobian products. 𝟯. 𝗛𝗲𝘀𝘀𝗶𝗮𝗻 H_f Takes a scalar function: f : ℝⁿ → ℝ Returns an n × n matrix of second-order partial derivatives. It answers: "How does the gradient itself change?" That means the Hessian measures curvature. When the second partial derivatives are continuous, the Hessian is symmetric. At a critical point: → positive definite Hessian → strict local minimum → negative definite Hessian → strict local maximum → indefinite Hessian → saddle point The clean mental model Gradient = first derivatives of one output → tells you direction Jacobian = first derivatives of many outputs → tells you sensitivity Hessian = second derivatives of one output → tells you curvature And the relationship between them is simple: The Hessian is the Jacobian of the gradient. For a scalar output, the Jacobian contains the same partial derivatives as the gradient, up to row/column convention. Same idea: measure change. Different object: direction, sensitivity, curvature. Once this clicks, optimization stops looking like a pile of formulas. It starts looking like a map of the problem.

I want to highlight a resource that I think is genuinely valuable for anyone learning machine learning: ML-From-Scratch. Most of us learn ML by using libraries like scikit-learn or PyTorch, which is the right way to build things quickly, but it can leave gaps in understanding why an algorithm works. This repository takes the opposite approach => every algorithm is implemented in plain NumPy, prioritizing clarity over performance, so you can trace the underlying math directly. It covers a wide range of the curriculum you'd expect to see in an ML course - linear and logistic regression, decision trees, random forest, gradient boosting, XGBoost, SVM, and naive bayes on the supervised side; k-means, DBSCAN, PCA, and Gaussian mixture models on the unsupervised side. It also goes further, with a small deep learning framework (convolutional, pooling, batch normalization, dropout, and RNN layers), a working GAN, and a Deep Q-Network trained on CartPole-v1. If you've completed a course on these algorithms and want to solidify your understanding by reading working implementations end to end, I'd recommend spending time with this repository. It's a good complement to theory => 32k stars, MIT licensed, and entirely in Python. Here's the GitHub Repo: github.com/eriklindernoren/M…