City Of Wisdom – Rise of the Bot

Mobile apps certainly cannibalized some of the website traffic – tuned out to be a good thing. As the mobile real estate is much smaller, they are typically designed to be simple to use and easy to understand and thus the adoption was tremendous.

Nowadays, there is an app for everything. But, the apps aren’t perfect. It is still a sugar-coated stripped down web form and under the skin it is as much static as the web page.

Recently, there is a spike towards a trend where every major player has a bot to drive some of their business aspects.

There is soon going to be “there is a bot for everything”

There are a lot of startups striving towards developing bot very easily. Microsoft is one of the major players who famously missed the app business, but promised to double down on the Artificial Intelligence services.

To their promise, Microsoft has been heavily investing in the AI space. Microsoft Bot Framework is one of the few services released as part of this strategy.

Bots are disrupting many sectors and customer service is one of the disrupted areas that I am specifically interested in. As I have the background with Dynamics 365, I am curious to see how I can leverage Microsoft Bot Framework along with D365 to make the request submission to the Customer Service team super easy and fully engaging – dubbed “Conversational UI”.

Let’s take a scenario where a citizen would like to submit a 311 request to an imaginary city called “City of Wisdom” by striking a conversation with the bot that the city has recently rolled out to help support the workload of the customer service agents.

As most of us are already familiar with Facebook Messenger, let’s deploy the bot to the Facebook Messenger, where the citizens are able to search for it and start the conversation. 

I am going to assume that you have basic knowledge of Dynamics 365 and MS Azure services. By the end of this post, you will have the City of Wisdom’s Bot published to Facebook Messenger platform and will be ready to serve the citizens of the City of Wisdom. There will be a series of subsequent blog posts where we will keep adding more skills to serve the citizens better.

Alright, let’s get started.

What do we need to begin?

I am going to present you with a series of steps in order to set up the infrastructure to develop the bot. These steps will take some time if you are brand new to the bot framework, so bear with me here.

Prerequisites:

  1. Visual Studio 2017.
  2. Dynamics 365 free trial.
  3. Azure account which is free for a limited time.
  4. And a beer pairs well with Azure. So, why not!

Download the Intelligence-ImageCaption source code from Github and build the code in Visual Studio.

Cognitive Services – Computer Vision

This bot code utilizes Cognitive Services to perform image recognition. Ideally, this step shouldn’t be necessary to develop a bot. But, I added this to sprinkle in some fun elements to our bot. Azure provides a suite of Cognitive services for Computer Vision (Image Recognition), Text Analytics, Speech to Text and much more to add skills to the bot. I chose Computer vision to identify the image that the citizens are uploading to identify what kind of problem the citizen wants to submit without asking them about the problem explicitly.

Create the Computer Vision service on Azure as shown below.

In the pricing tier, you can choose “F0” pricing tier, which is free and create the service.

Computer Vision

Click open the computer vision service and grab the Computer Vision API EndPoint (which typically looks like “https://westus2.api.cognitive.microsoft.com/vision/v1.0“) from the “Overview” section and grab the Vision API Key from “Keys” section and update it in web.config MicrosoftVisionApiEndpoint, MicrosoftVisionApiKey respectively in the bot code downloaded from github.

 App Service Deployment

In the MessageController.cs, replace

response.Text = "Hi! I am ImageCaption Bot. I can understand the content of any image" + " and try to describe it as well as any human. Try sending me an image or an image URL.";

with

response.Text = "Hi! I am City of Wisdom's bot. Upload an image of the problem if you would like to create a 311 Service Request.";

Publish it to Azure App Service by following this post (only follow the section “Publish from Visual Studio”). Now, we can access out bot from anywhere. let’s move on to set up the bot channel.

Bot Channels

Bot channel is nothing but a connection between the bot framework and the communication apps. This is what enables the bot to be added to the contact list on Skype or Messenger, so the users can interact with it.

MS Bot Framework provides connections to many popular channels like Facebook Messenger, Skype, Kik, Slack and Cortana out of the box and makes it easy to establish the connection.

In this post, we are going to establish the connection between our bot and the FB Messenger platform.

Follow this post to setup the bot channel registration.

From the Bot Channel Registration, click on “Test in Web Chat” to test the bot to ensure that it accepts an image and gives you back the description of the image.

 Test in Webchat

The below code is the entry point for the bot code. Whenever someone messages the bot, this “Post” method is called with the “Activity” parameter, which contains all the information about the bot, citizen, channel the citizen is communicating with the bot, conversation data and the actual message text.

In MessageController.cs

public async Task Post([FromBody]Activity activity)
        {
            if (activity.Type == ActivityTypes.Message)
            {}
        }

When the user searches the bot for the first time in the Messenger and starts a conversation, there are 2 things that happen behind the scene. The bot gets added to the conversation and the citizen gets added to the conversation. We can capture when the bot gets added to the conversation and show any greeting message to the citizen if we wish.

Every time, the bot or the citizen gets added to the conversation, “ConversationUpdate” activitytype is obtained. So, we can tap into that in order to ask an image to be uploaded from the citizen.

In MessageController.cs,

case ActivityTypes.ConversationUpdate:
// Greet the user the first time the bot is added to a conversation.
if (activity.MembersAdded.Any(m => m.Id == activity.Recipient.Id))
{
var connector = new ConnectorClient(new Uri(activity.ServiceUrl));
var response = activity.CreateReply();

response.Text = "Hi! I am City of Wisdom's bot. If you would like to create a 311 Service Request, upload an image of the problem.";
await connector.Conversations.ReplyToActivityAsync(response);
}

Facebook Channel Setup

Now, setup the facebook channel by following this post.

As this bot app is not public, the bot app does not appear when you search from your personal Facebook messenger account. From the Facebook Developers page, click on Roles -> Administrators -> Add Administrators button and choose your personal Facebook account. This will allow you to search for the City of Wisdom bot right from the Facebook Messenger. 

FB Admin

All Done!

If you have survived all the setup shocks so far, you are going to be treated with some fun stuff now. Open the FB Messenger on your phone and search for the bot. You can send an image to the bot and it will respond to you with the description of the image.

This is the first step in building a bot and publishing it to the Facebook Messenger platform.

Essentially, we have set up some infrastructure to develop more fun stuff in the future.

In the next post, we will enable the bot to communicate with D365 in order to store the 311 requests that the citizens are submitting.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s