# Calling Azure OpenAI API in Streaming Mode ## Summary This web part shows how you can call Azure OpenAI API in Streaming mode. The web part will show the data coming from the API in chunks, giving a much better user experience, so you are not waiting for the entire response. It also shows how you can cancel the streaming response at any point, which is useful to save some tokens (hence money), if the generating response does not look good to you (like when getting AI hallucinations). AI responses render Markdown but there is a toggle to disable so you can compare Markdown rendering of responses with Streaming and without. ![Sample in action](./assets/screenshot.gif) ## Compatibility | :warning: Important | |:---------------------------| | Every SPFx version is optimally compatible with specific versions of Node.js. In order to be able to build this sample, you need to ensure that the version of Node on your workstation matches one of the versions listed in this section. This sample will not work on a different version of Node.| |Refer to for more information on SPFx compatibility. | This sample is optimally compatible with the following environment configuration: ![SPFx 1.18.2](https://img.shields.io/badge/SPFx-1.18.2-green.svg) ![Node.js v16 | v18](https://img.shields.io/badge/Node.js-v16%20%7C%20v18-green.svg) ![Compatible with SharePoint Online](https://img.shields.io/badge/SharePoint%20Online-Compatible-green.svg) ![Does not work with SharePoint 2019](https://img.shields.io/badge/SharePoint%20Server%202019-Incompatible-red.svg "SharePoint Server 2019 requires SPFx 1.4.1 or lower") ![Does not work with SharePoint 2016 (Feature Pack 2)](https://img.shields.io/badge/SharePoint%20Server%202016%20(Feature%20Pack%202)-Incompatible-red.svg "SharePoint Server 2016 Feature Pack 2 requires SPFx 1.1") ![Local Workbench Unsupported](https://img.shields.io/badge/Local%20Workbench-Unsupported-red.svg "Local workbench is no longer available as of SPFx 1.13 and above") ![Hosted Workbench Compatible](https://img.shields.io/badge/Hosted%20Workbench-Compatible-green.svg) ![Compatible with Remote Containers](https://img.shields.io/badge/Remote%20Containers-Compatible-green.svg) ## Applies to - [SharePoint Framework](https://aka.ms/spfx) - [Microsoft 365 tenant](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/set-up-your-developer-tenant) > Get your own free development tenant by subscribing to [Microsoft 365 developer program](https://aka.ms/m365/devprogram) ## Prerequisites - Your Azure Subscription has the Azure OpenAI service available. You will see this message if you try to add Azure OpenAI service in your Subscription (follow the link in the Azure portal to request access). > Azure OpenAI Service is currently available to customers via an application form. The selected subscription has not been enabled for use of the service and does not have quota for any pricing tiers. Click here to request access to Azure OpenAI service. - You have created an Azure OpenAI service in your subscription - Create a Deployment Model (use GPT-3.5-turbo or GPT-4) - Grab the Azure OpenAI Key, Endpoint and the deployment model created in previous step (go to your _Azure OpenAI service -> Keys and Endpoint_) ## Contributors - [Luis Mañez](https://github.com/luismanez) - [Chris Kent](https://twitter.com/thechriskent) ## Version history | Version | Date | Comments | | ------- | ---------------- | --------------- | | 1.0 | January 2, 2024 | Initial release | | 1.1 | February 8, 2024 | Theme enhancements & markdown support | ## Minimal Path to Awesome - [Install the MS Graph Toolkit for SPFx package](https://learn.microsoft.com/graph/toolkit/get-started/mgt-spfx). - Clone this repository - Ensure that you are at the solution folder - in the command-line run: - `npm install` - `gulp serve` - Add the web part in the SharePoint workbench or any SharePoint page (appending _?debug=true&noredir=true&debugManifestsFile=_ to the page URL) - Edit the web part with your Azure OpenAI API values (see _prerequisites_ section) ## Features This sample illustrates the following concepts: - Calling Azure OpenAI _chat/completions_ endpoint in streaming mode - Keeping chat history during the chat session - How to cancel an streaming request - Using MS Graph toolkit with the Person component - Multiple FluentUI components - Markdown rendering of AI responses ## References - [Getting started with SharePoint Framework](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/set-up-your-developer-tenant) - [Building for Microsoft teams](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/build-for-teams-overview) - [Use Microsoft Graph in your solution](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/get-started/using-microsoft-graph-apis) - [Publish SharePoint Framework applications to the Marketplace](https://docs.microsoft.com/en-us/sharepoint/dev/spfx/publish-to-marketplace-overview) - [Microsoft 365 Patterns and Practices](https://aka.ms/m365pnp) - Guidance, tooling, samples and open-source controls for your Microsoft 365 development ## Help We do not support samples, but this community is always willing to help, and we want to improve these samples. We use GitHub to track issues, which makes it easy for community members to volunteer their time and help resolve issues. If you're having issues building the solution, please run [spfx doctor](https://pnp.github.io/cli-microsoft365/cmd/spfx/spfx-doctor/) from within the solution folder to diagnose incompatibility issues with your environment. You can try looking at [issues related to this sample](https://github.com/pnp/sp-dev-fx-webparts/issues?q=label%3A%22sample%3A%20react-azure-openai-api-stream%22) to see if anybody else is having the same issues. You can also try looking at [discussions related to this sample](https://github.com/pnp/sp-dev-fx-webparts/discussions?discussions_q=react-azure-openai-api-stream) and see what the community is saying. If you encounter any issues using this sample, [create a new issue](https://github.com/pnp/sp-dev-fx-webparts/issues/new?assignees=&labels=Needs%3A+Triage+%3Amag%3A%2Ctype%3Abug-suspected%2Csample%3A%20react-azure-openai-api-stream&template=bug-report.yml&sample=react-azure-openai-api-stream&authors=@luismanez%20@thechriskent&title=react-azure-openai-api-stream%20-%20). For questions regarding this sample, [create a new question](https://github.com/pnp/sp-dev-fx-webparts/issues/new?assignees=&labels=Needs%3A+Triage+%3Amag%3A%2Ctype%3Aquestion%2Csample%3A%20react-azure-openai-api-stream&template=question.yml&sample=react-azure-openai-api-stream&authors=@luismanez%20@thechriskent&title=react-azure-openai-api-stream%20-%20). Finally, if you have an idea for improvement, [make a suggestion](https://github.com/pnp/sp-dev-fx-webparts/issues/new?assignees=&labels=Needs%3A+Triage+%3Amag%3A%2Ctype%3Aenhancement%2Csample%3A%20react-azure-openai-api-stream&template=suggestion.yml&sample=react-azure-openai-api-stream&authors=@luismanez%20@thechriskent&title=react-azure-openai-api-stream%20-%20). ## Disclaimer **THIS CODE IS PROVIDED _AS IS_ WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.**