Prabod Rathnayaka

We built the Base Optimization Stack (B:OS) to turn open weights into frontier inference performance. For Nemotron 3 Nano: 1.9x faster than llama.cpp and 2.55x faster than MLX on Apple silicon. Full story: huggingface.co/blog/basecomp…

On-device AI is private and free, but a huge pain to set up. Today we're launching our new app Local to fix that. Local scans your hardware, picks optimised inference kernels, and recommends the best models for your machine. Run coding agents, chat with PDFs, summarize meetings. Free, on your laptop. If you’re using Local in your office with colleagues, you can run it in “Office Mode”. The strongest machine in your office runs the AI and everyone can connect to it with their laptop. check it out here: basecompute.co/local

You are awesome! Our BaseRT GitHub repo just hit 100 stars.

Fastest local run of GLM 5.2 using BaseRT, 45% faster decode than llama.cpp . Mac Studio M3 Ultra, 512GB Ram. Imagine the possibilities

We’ve built the fastest inference runtime for LLMs on Apple Silicon, faster than MLX and llama.cpp. BaseRT is up to 35% faster than Apple's MLX on token generation (decode) for small models, and 78% faster on prefill for bigger models. Details below.