Olostep

Gandalf and Marx are building our infrastructure. Behind the beards are two Olostep engineers working on the distributed systems and data pipelines that knowledge work AIs run on. We're hiring more people like them. The dress code is optional, but opinions about distributed systems are mandatory. Find our open roles in the comments! 👇

Your coding agent can manage your Olostep account now. Through the CLI, right where you write code: > Check balances and manage API keys > Buy top-ups and invite team members > Use API endpoints CLI docs in the reply.

Your procurement agent can monitor availability with a plain natural language instruction. Olostep Monitors evaluates page changes against that instruction. A product coming back in stock triggers an alert. A website logo change doesn't.

With up to 100,000 URLs in a single batch, a finished job can still contain failed URLs. Some pages return a 404, some are temporarily unavailable, and some don’t return content before the batch’s completion window closes. Olostep Batch separates successful URLs from failed ones, so you can retrieve the extracted content and see exactly which URLs need another attempt. Next, we’re working on returning the exact website status for every URL, successful or failed, so each result is easier to classify.

Web data shouldn't be ephemeral. When an AI pipeline regresses, you need to inspect the exact page your agent saw. When building knowledge bases, you need to reuse extracted data without scraping everything again. Now, you can persist your extraction results with Olostep. Choose your retention period: 10, 30, 90 days, or forever. No S3 buckets. No custom retention pipelines. Just persistent web data, built into Olostep.

Olostep works with @mastra. Give your Mastra agents web search, page extraction and structured data with one package: 𝘯𝘱𝘮 𝘪𝘯𝘴𝘵𝘢𝘭𝘭 @𝘰𝘭𝘰𝘴𝘵𝘦𝘱/𝘮𝘢𝘴𝘵𝘳𝘢-𝘵𝘰𝘰𝘭𝘴 Ask an agent to find suppliers for a part. It searches, reads their product pages, extracts and returns price and stock as structured data the next step can use.

Olostep runs inside @AirOpsHQ, powering brand and competitor research. AirOps uses four of our Web Data API endpoints: • /maps: discover pages on brand and competitor websites • /crawls: extract those pages' content for research • /answers: research company facts, hiring and product announcements • /batches: extract data from many URLs at once We're also working with AirOps to track what's being said about brands and their competitors across reviews, social, news and their own channels, so teams can see how sentiment and customer feedback change over time. Full case study in the reply.

If your coding agent uses an outdated API parameter, you're debugging code that looked right. Olostep's MCP tools let Cursor fetch current doc pages as markdown before writing the integration, instead of you copying them into chat.