Replying to @Ford_Lascari
Makes sense!
Empirically we've also found that a simple trick of chunking the docs into smaller (say 10k) segments works wonders for accuracy.
Also I guess looking at systems like o3 a lot of "RAG-ish" behavior will be implicit in the agent's behavior
1
1
37
We think that this is exciting because it will unlock more creativity by allowing for more complicated AI workflows where multiple models need to be loaded in sequence.
We have already seen creative technologists do amazing things with AI workflows using tools like ComfyUI.
1
3
305
How does this all work?
Under the hood, the Python API communicates with Outerport, which is a tensor daemon that is built with Rust.
For performance we incorporate tricks like:
- Remove Python (crazy right?)
- Page locking
- Parallel file processing
- Chunking
1
2
65
We at Outerport set out to solve this because it's annoying to have to wait an entire minute to try out a new model.
We can load FLUX models in 2.2 seconds from a warm cache that Outerport manages, and 6.6 seconds to initialize from disk.
1
2
56






