SharePoint Online and Azure OpenAI

This is a demo around indexing SharePoint and Azure Cognitive Search. I already have a web application that's connected to OpenAI and Azure Cognitive Search. What happens is I put my query and it gets augmented by search results and then returned by OpenAI through a function back to my application. For the purpose of this, I'm going to index SharePoint with the in-preview SharePoint indexer. It's a useful tool and allows me to configure access to SharePoint. One thing I do is create enterprise application and I can use that for permissions then on the SharePoint site. It's all about refining the SharePoint site and using the Postman service to post request into the search service. Basically, what we do is we set up a data source for SharePoint. We set up an indexer and then we set up the index. In this situation, again, because I have an index that's connected to an application, I'm going to reuse an index, but you get the gist.

Components:

• Azure Open AI Service
• Cognitive Search
• Semantic search (Preview)
• SharePoint Online
• Postman
• Enterprise Application
• Azure AD
• WebApp
• Function

Architecture

SharePoint

For this example I have created a simple SharePoint site with a document library
https://shed.sharepoint.com/sites/AzureService

Authentication - Create an enterprise application.

Its possible to configure both application and delegated authentication. For this example, I will configure an enterprise application for authentication.


Enterprise Application


Set Application Permissions - Reminder to Grant Admin constent


Create a client secret this will be used to configure the datasource permissions

Microsoft Graph, often abbreviated as MS Graph, is a powerful and versatile service that functions as a unified API endpoint for developers to access and manipulate data and services across Microsoft 365 platforms. By consolidating various services like Office 365, Windows 10, and Enterprise Mobility + Security (EMS) under one umbrella, MS Graph enables streamlined integration and offers valuable insights through data analytics and machine learning capabilities. This service simplifies the process of building applications and enhances their functionality by providing access to rich resources, relationships, and intelligence. MS Graph enables developers to leverage data from a range of Microsoft products and services, allowing them to create tailored, intelligent solutions for users across their organization.

Azure Search Configuration

I have used the basic SKU of the Search Service, I have noticed alot of demos ultilise the standard SKU but for demos the basic SKU is fine. The basic sku is cheaper and enables access to the Semantic Search preview feature for free. The standard SKU is a credits killer and costs over £250 per month


Semantic Search Preview



SharePoint DataSource

{
    "name" : "sharepoint-azureservices",
    "type" : "sharepoint",
    "credentials" : { "connectionString" : "SharePointOnlineEndpoint=https://shed.sharepoint.com/sites/AzureServices;ApplicationId=appid-createdabove;ApplicationSecret=appsecret-createdabove;TenantId=tenantid" },
    "container" : { "name" : "useQuery", "query" : "includeLibrariesInSite=https://shed.sharepoint.com/sites/AzureServices" }
}

SharePoint Index

{
    "name" : "sharepoint-index",
    "fields": [
        { "name": "id", "type": "Edm.String", "key": true, "searchable": false },
        { "name": "metadata_spo_item_name", "type": "Edm.String", "key": false, "searchable": true, "filterable": false, "sortable": false, "facetable": false },
        { "name": "metadata_spo_item_path", "type": "Edm.String", "key": false, "searchable": false, "filterable": false, "sortable": false, "facetable": false },
        { "name": "metadata_spo_item_content_type", "type": "Edm.String", "key": false, "searchable": false, "filterable": true, "sortable": false, "facetable": true },
        { "name": "metadata_spo_item_last_modified", "type": "Edm.DateTimeOffset", "key": false, "searchable": false, "filterable": false, "sortable": true, "facetable": false },
        { "name": "metadata_spo_item_size", "type": "Edm.Int64", "key": false, "searchable": false, "filterable": false, "sortable": false, "facetable": false },
        { "name": "content", "type": "Edm.String", "searchable": true, "filterable": false, "sortable": false, "facetable": false }
    ]
}

SharePoint Indexer

        {
    "name" : "sharepoint-indexer",
    "dataSourceName" : "sharepoint-azureservices",
    "targetIndexName" : "sharepoint-index",
    "parameters": {
    "batchSize": null,
    "maxFailedItems": null,
    "maxFailedItemsPerBatch": null,
    "base64EncodeKeys": null,
    "configuration": {
        "indexedFileNameExtensions" : ".pdf, .docx, .pptx, .doc, xlsx",
        "excludedFileNameExtensions" : ".png, .jpg",
        "dataToExtract": "contentAndMetadata"
      }
    },
    "schedule" : { },
    "fieldMappings" : [
        { 
          "sourceFieldName" : "metadata_spo_site_library_item_id", 
          "targetFieldName" : "id", 
          "mappingFunction" : { 
            "name" : "base64Encode" 
          } 
    ]
}
    

Documents indexed


Success I can now chat with my SharePoint documents