Rongwei Ma
Rongwei is a ML Engineer at Cloud1.
Authors: Rongwei Ma and Saska Saarioja
Artificial Intelligence (AI) has become a transformative force, fundamentally changing the way we interact with technology. One of the critical engines that drive these sophisticated chatbots is Azure OpenAI offered by Microsoft. By integrating sophisticated language models like GPT-4 with Azure's robust infrastructure, Azure OpenAI is redefining how businesses interact with technology. Azure OpenAI integrated into Chatbot brings many potential opportunities and has increased the interest for Chatbots to benefit businesses through instance responses, 24/7 availability, and multilingual support.
AI Chatbot Proof of Concept
Our recent milestone is the implementation of an AI Chatbot Proof of Concept (PoC) tailored for our customers by Microsoft Copilot Studio (formerly known as Power Virtual Agent, PVA). The AI chatbot is tailored to meet specific departmental needs, offering secure and role-specific access to information as well as enabling communication to Azure OpenAI with company internal data by Microsoft Teams. Employees can tailor their chatbot interactions with the Chatbot, allowing them to choose specific paths or scenarios that align with their needs. For instance, if a user seeks information related to IT, they can effortlessly navigate towards IT documentation, therefore different departments using the AI Chatbot can access different documentation. Furthermore, The AI Chatbot prioritizes security, ensuring that users can access only the information pertinent to their role, thus mitigating concerns about unauthorized document access. How to ensure security is one of the most challenging considerations. During development, we solved the challenges of how the chatbot memorizes the historical chat conversation, provides more accurate information, and tailors to each customer’s needs.
These AI chatbots meet the unique needs and requirements of a particular organization and they can offer a variety of purposes:
- Answering questions: The AI Chatbot can answer questions about a wide range of topics or within company internal data. It can provide extra key information, summaries, translation of factual topics, or create stories.
- Providing customer service: The AI Chatbot can provide customer service by answering customer questions and resolving customer issues.
- Generating materials and content: The AI Chatbot can generate marketing or sales materials, such as product descriptions, blog posts, and social media posts, or generate creative content, such as poems, stories, scripts, and musical pieces.
- Prediction and classification: The AI models can be integrated into the chatbot to make predictions or classifications based on the data.
Components Needed for an AI Chatbot - Microsoft
Generally, the development of AI Chatbots requires a specific set of technical tools that all play a role in the creation and deployment process. In addition to Microsoft Teams, other technical tools that are required and that we use are:
Microsoft Azure Account: To access into Azure environment the priority is to have an active Azure account. If you do not already have one, you can sign up for a free account on the Azure website.
Azure OpenAI Access and Keys: You will need to request access to Azure OpenAI models for your Azure account. You can do this by filling out a form provided by Microsoft. Azure OpenAI provides access to the latest OpenAI models, including the impressive GPT-4, allowing developers to leverage their capabilities to create intelligent, conversant chatbots.
Azure AI Search (formerly known as “Azure Cognitive Search”): Azure AI Search provides a secure information retrieval system for users able to access their own content with indexing and query capabilities.
Microsoft Copilot Studio and Azure Bot Framework: There are mainly two ways of designing the chatbot for Microsoft Teams, which are Microsoft Copilot Studio (Power Virtual Agent) and Azure Bot Framework (these will be discussed more thoroughly in part two of this blog series). Power Virtual Agent allows users to create intelligent conversational bots for personalized customer service with no-code interface. The other approach Microsoft Bot Framework (Software Development Kit, SDK) gives a comprehensive framework for building enterprise-grade conversation AI chatbots by various programming languages. Visual Studio Code IDE or Visual Studio can be good IDE options in this case.
Network and Security
As mentioned, during development we solved the issue of ensuring security for the AI Chatbot. In a time where AI chatbots are increasingly integral to business operations, prioritizing network and security is not just a precaution, but a necessity. AI chatbots, often handling sensitive data and customer interactions, require robust network infrastructure to ensure seamless, uninterrupted service. This infrastructure must be set against potential cyber threats and data breaches, as chatbots can become targets for malicious attacks. Moreover, a secure network guarantees the integrity and confidentiality of the data processed by chatbots, which is essential in maintaining customer trust and complying with data protection regulations. In addition, robust security protocols ensure that the AI models underlying these chatbots function as intended, free from tampering or unauthorized access.
The best practice for authorizing an Azure–based solution to Azure services is to use managed identities. Meaning that we use the identity of a specific resource which is automatically created to Azure Entra ID when the resource is created. By using managed identities, we don’t have to use keys for authentication, and we can make sure that the backend services can be accessed only from the specific Azure resources we created.
From a networking perspective, all Azure resources that are relevant for a Chatbot can be secured behind private endpoints where they are inaccessible from the public internet. When publishing a Chatbot with the Microsoft Copilot Studio, the service resides on the public internet and, therefore, when communicating with the Azure backend services, network security, and authentication needs to be taken into account. A good practice in Azure is to use Application Gateway / Web application Firewall as a layer for transmitting data between the user–facing public interface and backend resources behind private endpoints.
How is the AI Chatbot used in Teams?
Below you can find a demonstration on how the AI Chatbot can be used in Teams.
This exploration into AI Chatbots has highlighted the transformative impact of integrating Azure OpenAI into business technology, showcasing advancements like the AI Chatbot Proof of Concept (PoC) which offers tailored, secure, and efficient communication solutions. An AI Chatbot is a versatile and adaptive tool for diverse organizational roles, from customer interaction to content creation and data analysis. However, the importance of robust network security and data protection needs to be considered in the implementation of an AI Chatbot.
In part two of this blog, the architectural structures of the integration of an OpenAI Chatbot Solution that is built into Teams will be discussed.
Cloud1 is a certified AI and Machine Learning in Microsoft Azure partner
