Bing Chat has made some large effectivity enhancements and lowered latency points for some queries by 25%. Mikhail Parakhin, the CEO of Bing, mentioned on Twitter, “yesterday we launched a totally reworked backend for internal monologue, decreasing time to first token by ~25%, and, much more importantly, making latency extra secure, decreasing spikes.”
He shared this chart exhibiting the discount:
Michael Schechter from Bing added on Twitter, “These sort of adjustments usually don’t make the weblog, however characterize a ton of labor and a big enchancment to the general expertise.”
Lastly, it appears Bing posted about this enchancment on its weblog publish on Friday, saying, “We shipped efficiency enhancements which have lowered latency spikes for sure chat solutions.”
Discussion board dialogue at Twitter.
