SharePoint Online and Azure OpenAI
This is a demo around indexing SharePoint and Azure Cognitive Search. I already have a web application that's connected to OpenAI and Azure Cognitive Search. What happens is I put my query and it gets augmented by search results and then returned by OpenAI through a function back to my application. For the purpose of this, I'm going to index SharePoint with the in-preview SharePoint indexer. It's a useful tool and allows me to configure access to SharePoint. One thing I do is create enterprise application and I can use that for permissions then on the SharePoint site. It's all about refining the SharePoint site and using the Postman service to post request into the search service. Basically, what we do is we set up a data source for SharePoint. We set up an indexer and then we set up the index. In this situation, again, because I have an index that's connected to an application, I'm going to reuse an index, but you get the gist.Components:
- Azure Open AI Service
- Cognitive Search
- Semantic search (Preview)
- SharePoint Online
- Postman
- Enterprise Application
- Azure AD
- WebApp
- Function
Architecture
![Architecture diagram for chatting to SharePoint documents via OpenAI](/images/sharepointsearcharc.jpg)
SharePoint
For this example I have created a simple SharePoint site with a document libraryhttps://shed.sharepoint.com/sites/AzureService
![SharePoint Document Library](/images/sharepoint.jpg)
Authentication - Create an enterprise application.
Its possible to configure both application and delegated authentication. For this example, I will configure an enterprise application for authentication.![Enterprise Application](/images/registerapp.jpg)
Enterprise Application
![Application Permissions](/images/permissions.jpg)
Set Application Permissions - Reminder to Grant Admin constent
![Client Secret](/images/clientsecret.jpg)
Create a client secret this will be used to configure the datasource permissions
Azure Search Configuration
I have used the basic SKU of the Search Service, I have noticed alot of demos ultilise the standard SKU but for demos the basic SKU is fine. The basic sku is cheaper and enables access to the Semantic Search preview feature for free. The standard SKU is a credits killer and costs over £250 per month
![Semantic Search Preview](/images/semanticpreview.jpg)
Semantic Search Preview
![SharePoint DataSource](/images/sharepointdata.jpg)
SharePoint DataSource
{
"name" : "sharepoint-azureservices",
"type" : "sharepoint",
"credentials" : { "connectionString" : "SharePointOnlineEndpoint=https://shed.sharepoint.com/sites/AzureServices;ApplicationId=appid-createdabove;ApplicationSecret=appsecret-createdabove;TenantId=tenantid" },
"container" : { "name" : "useQuery", "query" : "includeLibrariesInSite=https://shed.sharepoint.com/sites/AzureServices" }
}
![SharePoint Index](/images/sharepointindex.jpg)
SharePoint Index
{
"name" : "sharepoint-index",
"fields": [
{ "name": "id", "type": "Edm.String", "key": true, "searchable": false },
{ "name": "metadata_spo_item_name", "type": "Edm.String", "key": false, "searchable": true, "filterable": false, "sortable": false, "facetable": false },
{ "name": "metadata_spo_item_path", "type": "Edm.String", "key": false, "searchable": false, "filterable": false, "sortable": false, "facetable": false },
{ "name": "metadata_spo_item_content_type", "type": "Edm.String", "key": false, "searchable": false, "filterable": true, "sortable": false, "facetable": true },
{ "name": "metadata_spo_item_last_modified", "type": "Edm.DateTimeOffset", "key": false, "searchable": false, "filterable": false, "sortable": true, "facetable": false },
{ "name": "metadata_spo_item_size", "type": "Edm.Int64", "key": false, "searchable": false, "filterable": false, "sortable": false, "facetable": false },
{ "name": "content", "type": "Edm.String", "searchable": true, "filterable": false, "sortable": false, "facetable": false }
]
}
![SharePoint Indexer](/images/sharepointindexer.jpg)
SharePoint Indexer
{
"name" : "sharepoint-indexer",
"dataSourceName" : "sharepoint-azureservices",
"targetIndexName" : "sharepoint-index",
"parameters": {
"batchSize": null,
"maxFailedItems": null,
"maxFailedItemsPerBatch": null,
"base64EncodeKeys": null,
"configuration": {
"indexedFileNameExtensions" : ".pdf, .docx, .pptx, .doc, xlsx",
"excludedFileNameExtensions" : ".png, .jpg",
"dataToExtract": "contentAndMetadata"
}
},
"schedule" : { },
"fieldMappings" : [
{
"sourceFieldName" : "metadata_spo_site_library_item_id",
"targetFieldName" : "id",
"mappingFunction" : {
"name" : "base64Encode"
}
]
}
![Indexer Success](/images/indexsuccess.jpg)
Documents indexed
![Chatting with my SharePoint documents](/images/aioutputsharepoint.jpg)
Success I can now chat with my SharePoint documents
© 2024 - The Azure Shack - AI generated content throughout testing CoPilot Chat, OpenAI on Azure and Microsoft Designer