Fastly’s new API promises improved ChatGPT accessibility

Fastly’s new API promises improved ChatGPT accessibility As a tech journalist, Zul focuses on topics including cloud computing, cybersecurity, and disruptive technology in the enterprise industry. He has expertise in moderating webinars and presenting content on video, in addition to having a background in networking technology.

Edge cloud provider Fastly has developed an open-source edge computing API, which it claims offers developers faster and more affordable access to OpenAI’s ChatGPT for product shipment.

Named the ‘Fastly AI Accelerator,’ the company reports that this new accelerator, which operates on its Edge Cloud Platform and utilises semantic caching, speeds up access times to the AI-powered chatbot, facilitating developer services such as code generation and debugging.

The company has hinted that it might broaden its AI strategy by expanding this specialised API to also cover other large language models (LLMs) beyond ChatGPT in the future.

This was a catalyst to solve some of the problems that popular AI applications were encountering, as they processed hundreds of thousands of API calls a day and felt the pain of these inefficiencies on the backend, Fastly highlighted. For example, without semantic caching, every call to the LLM would require a return to the provider for information, potentially increasing both costs and latency. The semantic caching with the AI Accelerator aims to rapidly overcome these problems by storing responses to many queries from Fastly’s high-performance edge platform, thus providing a better user experience with improved performance and reduced costs.

This development provoked the attention of industry analyst Will Townsend of Moor Insights & Strategy, who called it a game-changer. The AI Accelerator would not only boost generative AI performance but also reduce token costs. This demonstrates Fastly’s architectural strengths and shows that their approach to CDN can be applied to even more non-traditional services.

Fastly, a high-performance cloud content delivery platform provider, has been diversifying its products away from traditional CDN solutions, bringing it into competition with the likes of Akamai and Edgio. Now, the company is expanding into managed security, cloud computing, video streaming, and more, marketing itself as a kind of next-gen solution.

During Fastly’s most recent quarterly earnings call, CEO Todd Nightingale emphasised that customers are increasingly seeking advanced solutions such as serverless compute and edge storage. This aligns with the company’s strategic focus on delivering cutting-edge technology to meet evolving market demands.

Fastly’s edge network infrastructure is extensive, spanning more than two dozen sites in the United States and numerous international locations. In its latest quarterly report, the company recorded growth in its edge computing offerings, with revenues in this business unit increasing from around $2 million to almost $3 million year-over-year. While this represents a small portion of Fastly’s overall revenue, the company is placing significant emphasis on edge computing for future growth.

The introduction of the AI Accelerator is particularly noteworthy for the telecom industry, as various companies, including Lumen Technologies, Verizon, and Akamai, have been exploring opportunities at the intersection of AI and edge computing. These efforts stem from a belief in the potential of edge computing to drive the construction of smaller, distributed data centres to support low-latency services.

Yet, this expected near-immediate, sub-millisecond, latency-sensitive demand materialised only partly, giving a setback to some edge computing projects. Despite such difficulties, the renewed excitement around AI services like ChatGPT is refocusing attention on edge computing; adoption rates are slowly increasing. Although most AI deployments today are centred on large data centres, the future is expected to shift towards an ‘inference’ AI world where some services would be better provided at the edge, due to the low-latency network connections that edge computing offers.

The AI Accelerator is part of a growing movement in the tech industry to abstract complex functionality, such as machine learning, into an API. Developers in the wireless industry increasingly have the opportunity to purchase new networking capabilities directly from networks via APIs, providing a lure for operators who need to recoup their 5G network investments.

Want to learn more about edge computing from industry leaders? Check out Edge Computing Expo taking place in Amsterdam, California and London. 

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Tags: , ,

View Comments
Leave a comment

Leave a Reply