1/ [New Post] Case Study: Motive
In this post, we dive into our first case study on how ExaDeploy has helped cutting-edge ML companies save money and accelerate their time-to-market, all with minimal engineering integration time!
exafunction.com/blog/motive-…
1
7
4/ The solution is to dynamically colocate models without specifying which sets of models live on which GPUs. This is possible with an orchestrator that manages GPUs in a serverless fashion and uses Dynamic Multiplexing to dynamically spin up GPUs and migrate inference tasks!
1
1
1/ [New Post] Load 60 BERTs onto a single T4
In this post, we explain how dynamically colocating models on serverless GPUs solves the many-model dynamic-throughput problem. We demonstrate this by loading 60 BERT models onto a single T4.
exafunction.com/blog/60-bert…
2/ One of the bottlenecks when using GPUs is copying data between CPU and GPU memory over the PCIe bus. For many models, simply copying the inputs can significantly increase latency, especially when non-inference tasks like preprocessing can be executed and chained on the GPU.
1
1/ [New Post] Getting Rid of CPU-GPU Copies in TensorFlow
Douglas Chen presents the newest post in our “GPU Tips and Tricks” series, on eliminating memory overhead when using TensorFlow.
exafunction.com/blog/tensorf…










