Adrian Hills (@AdaTheDev) header
Adrian Hills (@AdaTheDev) profile picture

I write code. I delete code. I build stuff. Often found on the Refactor Tractor. .NET / Azure / AI / Data

Hampshire, UK
Joined February 2009
Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)Photo by Adrian Hills (@AdaTheDev)
Hey @_Evan_Boyle - is sandbox support on the roadmap for Copilot CLI?
1
51
Biggest barrier I personally see to further adoption of GitHub Copilot is how far behind the experience and supported feature-set feels in Visual Studio compared to VSCode, Cursor etc. And combined with Copilot CLI being in Preview, it shifts attention elsewhere.
1
114
Yeah, I could switch out to VS Code for Copilot and then back to VS. But would be great to have better first class support in VS for the in-IDE experience. I'm increasingly switching out to Codex CLI, which is working very well. Keen to make use of Copilot CLI once it goes GA
1
43
One of my biggest UX gripes with GitHub Copilot (in VS) is that you have to wait for the whole response to be generated before you can read it due to the auto-scrolling as the response streams in. Would be a big plus to be able to stop the auto-scroll!
57
Anyone else having issues with Azure OpenAI (timing out trying to connect via API, failing to load model deployments in Azure OpenAI Studio...)?
6
3
289
Anyone else having issues getting into the Azure OpenAI Studio today?
4
171
Good article (Azure OpenAI) on considerations for right-sizing your Provisioned Throughput deployment, spilling peak workloads over to PAYG for cost optimisation: Right-size your PTU deployment and save big (microsoft.com)
1
84
Transitioning to Production with LLM capabilities, when you care about consistency of guaranteed throughput and latency is a key challenge that needs to be considered carefully up front.
1
1
67
PAYG models are great and have low outlay, but usually don't have guarantees and usually higher latency. Provisioned Throughput models give consistency and lower latency but a huge jump up in committed spend, where you end up paying for a lot of quiet time in a non 24/7 scenario
1
1
43
Important to make a conscious assessment, else you run the risk of hitting production with a workload that the LLM deployment model/provider cannot fulfil reliably. Depending on use case/customers, you may then be forced into a hasty switch to Provisioned Throughput
1
40
Full Fibre 900 - £41.99/month Full Fibre 900 for a customer of many years (me) - £55.54/month Don't know what's going on with @Plusnet these days, but that's shocking. Making me wonder if longer term, it'll be wound down (as part of consolidating Plusnet/BT/EE, per mobile)
1
133
Any of my followers (or beyond) have experience with Amazon Bedrock for hosting LLMs (on a token usage basis and/or Provisioned Throughput basis)? Keen to hear thoughts, experiences etc. Please RT!
1
64
Switched over recently to the new gen Planners (Handlebars) as part of Semantic Kernel v1.x upgrade. Next step is to optimise the "plan generation" phase of a workflow. Generating plans is expensive (time and cost) and relatively prone to error.
1
1
70
To save generating plans afresh every time for similar inputs, I'm building a prototype Semantic Kernel "Plan Cache". Still needs LLM (+ Vector Search), but with a much smaller input context, and a more focused task.
1
1
47
Early days, but early signs looking promising - around 50% reduction in duration, increased reliability/stability due to stable plan reuse. Even looking like there's potential to use a different (cheaper and faster) LLM when using cached plans.
1
31
Having significant latency issues with GPT 4-Turbo (1106-Preview) on Azure OpenAI, making it consistently, noticeably slower than GPT-4 (0613). Anyone else encountering this? e.g. 48.8s (GPT-4-Turbo) vs. 13.9s (GPT-4) Snapshot times, so fluctuates, but wildly so for 4-Turbo.
1
1
154
But the trend is pretty consistent on Azure OpenAI with 1106-Preview. Running tests on isolated model deployments, using low token usage test case (<500 prompt tokens, < 300 completion tokens), various points throughout the day, different machines/code implementations etc etc
1
72