├── CHALLENGE1.md ├── CHALLENGE2.md ├── FinalSolution ├── .gitignore ├── emailSender.js ├── index.js └── package.json ├── Images ├── Bonus │ ├── cintegration.PNG │ └── cintsuccess.PNG ├── Luis │ ├── adding_entity.png │ ├── adding_intent.png │ ├── labeling_utterance.png │ └── publishing.png ├── Mission1 │ └── setupcmd.PNG ├── Mission2 │ └── searchapi.PNG ├── Mission3 │ └── m3result.PNG └── Portal │ ├── Management.PNG │ └── Registration.PNG ├── LUIS.MD ├── MISSION1.md ├── MISSION2.md ├── MISSION3.md ├── MissionBriefing.pdf └── README.md /CHALLENGE1.md: -------------------------------------------------------------------------------- 1 | # Challenge 1: Sending an email back to Earth 2 | The goal of this mission is to give the bot the functionality of sending emails back to Earth. 3 | 4 | There are some things we need to implement in order to achieve this goal: 5 | 6 | 1. Enabling the bot to send emails (guided) 7 | 2. Getting the recipient email from the user (challenge) 8 | 9 | ## NPM installation 10 | `nodemailer` is a useful npm that allows your NodeJS applications to send emails programatically. 11 | 12 | Back in the command prompt (which should already be configured to your working directory), you can simply run these 2 commands to install the npms: 13 | 14 | ```shell 15 | npm install nodemailer --save 16 | npm install nodemailer-wellknown --save 17 | ``` 18 | 19 | ## Utilizing nodemailer 20 | Let's make a new javascript file in our directory to contain the logic for email sending. 21 | 22 | Right click the left panel area and create a new file in the main folder. Name it `emailSender.js`. 23 | 24 | Let's first start by creating a new top level object to export. This is the object we will be exporting to the main script: 25 | 26 | ```js 27 | //Top level object 28 | var emailSender = {}; 29 | ``` 30 | 31 | Next, we need to import the modules required: 32 | 33 | ```js 34 | /* 35 | nodemailer 36 | */ 37 | var nodemailer = require('nodemailer'); 38 | var wellknown = require('nodemailer-wellknown'); 39 | ``` 40 | 41 | Nodemailer is a module that works with almost all email providers, but I'll be using hotmail in this scenario to create the nodemailer transport (an object that contains specifications for the email function). 42 | 43 | ```js 44 | var transporter = nodemailer.createTransport({ 45 | service: 'hotmail', 46 | auth: { 47 | user: 'marsbothol@outlook.com', 48 | pass: 'passwordhere' //password to account to be given at event 49 | } 50 | }); 51 | ``` 52 | 53 | We can then use our transport to create a function for sending emails. First, let's define a function: 54 | 55 | ```js 56 | emailSender.sendEmail = function(recipientEmail, callback) 57 | { 58 | } 59 | ``` 60 | 61 | This function will take in a parameter for the email address we want to send the email to, 62 | as well as a callback function that will be called when the email sending function finishes. 63 | 64 | In this method, we can then define the email we want to send, as well as call the sending function. 65 | 66 | ```js 67 | emailSender.sendEmail = function(recipientEmail, callback) 68 | { 69 | //Define email options and structure 70 | var mailOptions = 71 | { 72 | from: '"Mars Bot" ', 73 | to: recipientEmail, //insert the recipientEmail parameter 74 | subject: 'Message from Mars', 75 | text: 'Hello from Mars Bot!' 76 | } 77 | 78 | //Send email using the options 79 | transporter.sendMail(mailOptions, function(err, info){ 80 | if(!err) 81 | { 82 | console.log('Message successfully sent: ' + info.response); 83 | callback(null); 84 | } 85 | else 86 | { 87 | console.log(err); 88 | callback(err); 89 | } 90 | }); 91 | } 92 | ``` 93 | 94 | Lastly, we can then export the top level object we defined to be accessed in our main script: 95 | 96 | ```js 97 | module.exports = emailSender; 98 | ``` 99 | 100 | ## Challenge 101 | Now that we've made a js file for the email logic, we need to figure out how to get it to work with the bot. 102 | 103 | First, place this at the top of `index.js` to import the emailSender logic: 104 | 105 | ```js 106 | var emailSender = require('./emailSender'); 107 | ``` 108 | 109 | We want to make it so that the bot will ask a user for an email address and then send an email to that address. 110 | 111 | Here's a simple dialog to get you started: (make sure you include the dialog matching and train the correct intent in LUIS) 112 | 113 | ```js 114 | intentDialog.matches(/\b(hi|hello|hey|howdy)\b/i, '/sayHi') 115 | .matches('GetNews', '/topNews') 116 | .matches('AnalyseImage', '/analyseImage') 117 | .matches('SendEmail', '/sendEmail') 118 | .onDefault(builder.DialogAction.send("Sorry, I didn't understand what you said.")); 119 | 120 | bot.dialog('/sendEmail', [ 121 | function(session){ 122 | session.send("I can send an email to your team member on Earth, what's his/her address?"); 123 | } 124 | ]); 125 | ``` 126 | 127 | Next, we need to ask the user for the email address and then send it into the function. 128 | 129 | HINT: You can use the text prompt to ask a user for a email address, then pass that address into the sendEmail() function. 130 | 131 | ## Finishing Line 132 | At the end, the bot should be able to ask for an email address and then send an email to that address. -------------------------------------------------------------------------------- /CHALLENGE2.md: -------------------------------------------------------------------------------- 1 | # Challenge 2: Deploying to Azure and connecting to messaging channels 2 | ## Introuction 3 | 4 | In this mission, we will deploy your bot that uses NodeJS and the Microsoft Bot Framework to Azure. 5 | 6 | There are 3 steps to this tutorial: 7 | 8 | 1. Publishing the bot to Github 9 | 2. Deploying to Azure using CI from Github 10 | 3. Registering the bot on the Bot Framework portal 11 | 12 | ## Prerequisites 13 | 14 | - Install git - https://git-scm.com/downloads 15 | - Make a Microsoft account if you don't have one 16 | - Make a Github account if you don't have one 17 | 18 | ## Initialise a Github repository 19 | 20 | A Github repository is a great way to store your code online and collaborate with others on a codebase. You can plug your repository in to a hosted web app and each time you make changes to the repository, the changes automatically deploy onto your web app (this is called Continuous Integration). Log in to [Github](http://github.com). Click on the + sign at the top right hand corner, and then click on New Repository. Name and create your repository. Once that's created, click on the Code tab of your repository. Click on the green "Clone or download" button, then copy the url (it should end with .git). 21 | 22 | Next, open your command prompt. 23 | 24 | Navigate to the folder with the project. If your project is on the desktop, the command would be like this: 25 | 26 | ```shell 27 | cd Desktop/marsbot 28 | ``` 29 | 30 | Now, we need to make the current folder containing your bot a Git repository. Use this command: 31 | 32 | ```shell 33 | git init 34 | ``` 35 | 36 | We now need to link our local Git folder to the remote Github repository. Type in the following command: 37 | 38 | ```shell 39 | git remote add origin https://github.com/yourusername/yourrepositoryname.git 40 | ``` 41 | 42 | Make sure you use the url you copied earlier in the above command. Your local Git repository is now remotely tracking your Github repository online. Type `git remote -v` to check that it points to the correct url. 43 | 44 | Once that's done, let's commit and push our code to the Github repo. Run the following commands (separately): 45 | 46 | ```shell 47 | git add . 48 | git commit -m "Initial commit" 49 | git push origin master 50 | ``` 51 | 52 | If you refresh your Github repository online, you should see that your code has been pushed to it. Now let's use continuous integration to deploy the code in our Github repo into a web app hosted online. 53 | 54 | ## Publish the bot online 55 | 56 | We have to publish the bot online as the Bot Framework won't be able to talk to a local url. 57 | 58 | Go to the [Azure Portal](https://portal.azure.com). Click on 'New' (it's at the sidebar), go into the web + mobile tab, and click on Web App. Name your web app whatever you'd like, name your resource group something sensible. Your Subscription should be free, and the location your app is hosted on can be anywhere, but it might be good to set it to a region close to you. Go ahead and create your web app. 59 | 60 | It might take a while, but you will get notified when your web app has been successfully created. Once it has been created go into All Resources (it's on the sidebar) and look for the web app you just created. Click into it and it should display a dashboard with info about your web app. Click into the Github blade under Choose Source. Then, click into Authorization and log in with your Github credentials. Then, select the project and branch (should be master) that your bot is in. Leave the Performance Test as not configured and hit ok. 61 | 62 | ![CI](https://raw.githubusercontent.com/alyssaong1/Bot-Framework-HOL/master/Images/Bonus/cintegration.PNG) 63 | 64 | It may take a while for the latest commit to sync and be deployed to your web app. If the latest commit doesn't seem to be syncing, just hit sync. You'll see a green tick with your latest commit once it's done. 65 | 66 | ![CISuccess](https://raw.githubusercontent.com/alyssaong1/Bot-Framework-HOL/master/Images/Bonus/cintsuccess.PNG) 67 | 68 | ## Registering your bot on the portal 69 | In order for your bot to talk to the Bot Connector and reach your users on the different channels, the bot has to be first registered on the Bot Framework portal. 70 | 71 | First, go to this link on your browser: https://dev.botframework.com/bots/new 72 | 73 | You will be prompted to sign in with your Microsoft account. Take note to use a personal Microsoft account (usually Hotmail, Outlook, MSN, Live email addresses) and not your school or organization's Office365 account. 74 | 75 | After signing in, you will be greeted with a simple form that allows you to register your bot. 76 | 77 | Most of the fields are self-explanatory. 78 | - The "Messaging Endpoint" field should be filled with the domain given to you on your Azure Web App, but subdomain to /api/messages. 79 | - The "Create Microsoft App ID and Password" will give you a new ID and password. Save this as we need it later. 80 | - You can also input your Azure Application Insights key if you enabled it when creating the Web App on Azure, but it's optional. 81 | 82 | ![Registration](https://raw.githubusercontent.com/alyssaong1/Bot-Framework-HOL/master/Images/Portal/Registration.PNG) 83 | 84 | ## Authorizing your bot 85 | Now that you have the bot deployed on Azure and registered on the Bot Framework, we need to add the App ID and Password on Azure. 86 | 87 | Go back to your deployment on Azure and click on App Settings, configuring the App ID and Password we got just now. 88 | 89 | ![AppSettings](https://raw.githubusercontent.com/jamesleeht/XamarinMarsHOL/master/Images/marsbotsettings.PNG) 90 | 91 | ## Managing your new bot 92 | After registering, you'll be sent to the page where you can manage all your bots. Click on the bot you just created. The page should look like this: 93 | 94 | ![Management](https://raw.githubusercontent.com/jamesleeht/XamarinMarsHOL/master/Images/marsbotportal.PNG) 95 | 96 | The top portion has several useful functions. 97 | - A testing function which will send a simple HTTP request to your messaging endpoint and give you back an appropriate success or error message. 98 | - The Web Chat (iframe) channel that has been pre-embedded for you on the page, allowing you to talk to your bot from the portal 99 | 100 | Try writing "Hi" to the bot in the web chat and see if it responds. If it does then we have successfully deployed the bot. 101 | -------------------------------------------------------------------------------- /FinalSolution/.gitignore: -------------------------------------------------------------------------------- 1 | # Logs 2 | logs 3 | *.log 4 | npm-debug.log* 5 | 6 | # Runtime data 7 | pids 8 | *.pid 9 | *.seed 10 | *.pid.lock 11 | 12 | # Directory for instrumented libs generated by jscoverage/JSCover 13 | lib-cov 14 | 15 | # Coverage directory used by tools like istanbul 16 | coverage 17 | 18 | # nyc test coverage 19 | .nyc_output 20 | 21 | # Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files) 22 | .grunt 23 | 24 | # node-waf configuration 25 | .lock-wscript 26 | 27 | # Compiled binary addons (http://nodejs.org/api/addons.html) 28 | build/Release 29 | 30 | # Dependency directories 31 | node_modules 32 | jspm_packages 33 | 34 | # Optional npm cache directory 35 | .npm 36 | 37 | # Optional eslint cache 38 | .eslintcache 39 | 40 | # Optional REPL history 41 | .node_repl_history 42 | 43 | # Output of 'npm pack' 44 | *.tgz 45 | 46 | # Yarn Integrity file 47 | .yarn-integrity 48 | 49 | -------------------------------------------------------------------------------- /FinalSolution/emailSender.js: -------------------------------------------------------------------------------- 1 | //Top level object 2 | var emailSender = {}; 3 | 4 | /* 5 | nodemailer 6 | */ 7 | var nodemailer = require('nodemailer'); 8 | var wellknown = require('nodemailer-wellknown'); 9 | var transporter = nodemailer.createTransport({ 10 | service: 'hotmail', 11 | auth: { 12 | user: 'marsbothol@outlook.com', 13 | pass: 'passwordhere' //give password during event 14 | } 15 | }); 16 | 17 | emailSender.sendEmail = function(recipientEmail, callback) 18 | { 19 | var mailOptions = 20 | { 21 | from: '"Mars Bot" ', 22 | to: recipientEmail, 23 | subject: 'Message from Mars', 24 | text: 'Hello from Mars Bot!' 25 | } 26 | 27 | transporter.sendMail(mailOptions, function(err, info){ 28 | if(!err) 29 | { 30 | console.log('Message successfully sent: ' + info.response); 31 | callback(null); 32 | } 33 | else 34 | { 35 | console.log(err); 36 | callback(err); 37 | } 38 | }); 39 | } 40 | 41 | module.exports = emailSender; -------------------------------------------------------------------------------- /FinalSolution/index.js: -------------------------------------------------------------------------------- 1 | // Modules 2 | var restify = require('restify'); 3 | var builder = require('botbuilder'); 4 | var rp = require('request-promise'); 5 | var emailSender = require('./emailSender.js'); 6 | 7 | // API Keys 8 | var BINGSEARCHKEY = '*****YOUR SUBSCRIPTION KEY GOES HERE*****'; 9 | var CVKEY = '*****YOUR SUBSCRIPTION KEY GOES HERE*****'; 10 | 11 | //========================================================= 12 | // Bot Setup 13 | //========================================================= 14 | 15 | // Setup Restify Server 16 | // Listen for any activity on port 3978 of our local server 17 | var server = restify.createServer(); 18 | server.listen(process.env.port || process.env.PORT || 3978, function () { 19 | console.log('%s listening to %s', server.name, server.url); 20 | }); 21 | 22 | // Create chat bot 23 | var connector = new builder.ChatConnector({ 24 | appId: process.env.MICROSOFT_APP_ID, 25 | appPassword: process.env.MICROSOFT_APP_PASSWORD 26 | }); 27 | var bot = new builder.UniversalBot(connector); 28 | // If a Post request is made to /api/messages on port 3978 of our local server, then we pass it to the bot connector to handle 29 | server.post('/api/messages', connector.listen()); 30 | 31 | //========================================================= 32 | // Bots Dialogs 33 | //========================================================= 34 | var luisRecognizer = new builder.LuisRecognizer('Your LUIS URL here'); 35 | 36 | var intentDialog = new builder.IntentDialog({recognizers: [luisRecognizer]}); 37 | 38 | //This is called the root dialog. It is the first point of entry for any message the bot receives 39 | bot.dialog('/', intentDialog); 40 | 41 | intentDialog.matches(/\b(hi|hello|hey|howdy)\b/i, '/sayHi') 42 | .matches('GetNews', '/topNews') 43 | .matches('AnalyseImage', '/analyseImage') 44 | .matches('SendEmail', '/sendEmail') 45 | .onDefault(builder.DialogAction.send("Sorry, I didn't understand what you said.")); 46 | 47 | bot.dialog('/sayHi', function(session) { 48 | session.send('Hi there! Try saying things like "Get news in Toyko"'); 49 | session.endDialog(); 50 | }); 51 | 52 | bot.dialog('/topNews', [ 53 | function (session){ 54 | // Ask the user which category they would like 55 | // Choices are separated by | 56 | builder.Prompts.choice(session, "Which category would you like?", "Technology|Science|Sports|Business|Entertainment|Politics|Health|World|(quit)"); 57 | }, function (session, results, next){ 58 | // The user chose a category 59 | if (results.response && results.response.entity !== '(quit)') { 60 | //Show user that we're processing their request by sending the typing indicator 61 | session.sendTyping(); 62 | // Build the url we'll be calling to get top news 63 | var url = "https://api.cognitive.microsoft.com/bing/v5.0/news/?" 64 | + "category=" + results.response.entity + "&count=10&mkt=en-US&originalImg=true"; 65 | // Build options for the request 66 | var options = { 67 | uri: url, 68 | headers: { 69 | 'Ocp-Apim-Subscription-Key': BINGSEARCHKEY 70 | }, 71 | json: true // Returns the response in json 72 | } 73 | //Make the call 74 | rp(options).then(function (body){ 75 | // The request is successful 76 | sendTopNews(session, results, body); 77 | }).catch(function (err){ 78 | // An error occurred and the request failed 79 | console.log(err.message); 80 | session.send("Argh, something went wrong. :( Try again?"); 81 | }).finally(function () { 82 | // This is executed at the end, regardless of whether the request is successful or not 83 | session.endDialog(); 84 | }); 85 | } else { 86 | // The user choses to quit 87 | session.endDialog("Ok. Mission Aborted."); 88 | } 89 | } 90 | ]); 91 | 92 | bot.dialog('/analyseImage', [ 93 | function (session){ 94 | // Ask the user which category they would like 95 | // Choices are separated by | 96 | builder.Prompts.text(session, "Enter an image url to get the caption for it: "); 97 | }, function (session, results, next){ 98 | // The user chose a category 99 | if (results.response) { 100 | //Show user that we're processing their request by sending the typing indicator 101 | session.sendTyping(); 102 | // Build the url we'll be calling to get top news 103 | var url = "https://westus.api.cognitive.microsoft.com/vision/v1.0/describe/"; 104 | // Build options for the request 105 | var options = { 106 | method: 'POST', // thie API call is a post request 107 | uri: url, 108 | headers: { 109 | 'Ocp-Apim-Subscription-Key': CVKEY, 110 | 'Content-Type': "application/json" 111 | }, 112 | body: { 113 | url: results.response, 114 | language: 'en' 115 | }, 116 | json: true 117 | } 118 | //Make the call 119 | rp(options).then(function (body){ 120 | // The request is successful 121 | console.log(body["description"]["captions"]); 122 | session.send(body["description"]["captions"][0]["text"]); 123 | }).catch(function (err){ 124 | // An error occurred and the request failed 125 | session.send("Argh, something went wrong. :( Try again?"); 126 | }).finally(function () { 127 | // This is executed at the end, regardless of whether the request is successful or not 128 | session.endDialog(); 129 | }); 130 | } else { 131 | // The user choses to quit 132 | session.endDialog("Ok. Mission Aborted."); 133 | } 134 | } 135 | ]); 136 | 137 | bot.dialog('/sendEmail', [ 138 | function(session){ 139 | session.send("I can send an email to your team member on Earth, what's his/her address?"); 140 | builder.Prompts.text(session, "Enter an image url to get the caption for it: "); 141 | }, 142 | function(session, results) 143 | { 144 | var emailAddress = results.response; 145 | emailSender.sendEmail(emailAddress, function(err){ 146 | if(!err) 147 | { 148 | session.send("I've successfully sent an email to your team."); 149 | } 150 | else 151 | { 152 | session.send("Error sending email"); 153 | } 154 | }) 155 | } 156 | ]); 157 | 158 | // This function processes the results from the API call to category news and sends it as cards 159 | function sendTopNews(session, results, body){ 160 | session.send("Top news in " + results.response.entity + ": "); 161 | //Show user that we're processing by sending the typing indicator 162 | session.sendTyping(); 163 | // The value property in body contains an array of all the returned articles 164 | var allArticles = body.value; 165 | var cards = []; 166 | // Iterate through all 10 articles returned by the API 167 | for (var i = 0; i < 10; i++){ 168 | var article = allArticles[i]; 169 | // Create a card for the article and add it to the list of cards we want to send 170 | cards.push(new builder.HeroCard(session) 171 | .title(article.name) 172 | .subtitle(article.datePublished) 173 | .images([ 174 | //handle if thumbnail is empty 175 | builder.CardImage.create(session, article.image.contentUrl) 176 | ]) 177 | .buttons([ 178 | // Pressing this button opens a url to the actual article 179 | builder.CardAction.openUrl(session, article.url, "Full article") 180 | ])); 181 | } 182 | var msg = new builder.Message(session) 183 | .textFormat(builder.TextFormat.xml) 184 | .attachmentLayout(builder.AttachmentLayout.carousel) 185 | .attachments(cards); 186 | session.send(msg); 187 | } -------------------------------------------------------------------------------- /FinalSolution/package.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "package.json", 3 | "version": "1.0.0", 4 | "description": "For a news bot", 5 | "main": "index.js", 6 | "scripts": { 7 | "test": "node index.js", 8 | "start": "node index.js" 9 | }, 10 | "repository": { 11 | "type": "git", 12 | "url": "git+https://github.com/michhar/Bot-Framework-HOL.git" 13 | }, 14 | "keywords": [ 15 | "news", 16 | "bot" 17 | ], 18 | "author": "Micheleen Harris", 19 | "license": "MIT", 20 | "bugs": { 21 | "url": "https://github.com/michhar/Bot-Framework-HOL/issues" 22 | }, 23 | "homepage": "https://github.com/michhar/Bot-Framework-HOL#readme", 24 | "dependencies": { 25 | "botbuilder": "^3.6.0", 26 | "nodemailer": "^3.1.4", 27 | "nodemailer-wellknown": "^0.2.2", 28 | "request-promise": "^4.1.1", 29 | "restify": "^4.3.0" 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /Images/Bonus/cintegration.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Bonus/cintegration.PNG -------------------------------------------------------------------------------- /Images/Bonus/cintsuccess.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Bonus/cintsuccess.PNG -------------------------------------------------------------------------------- /Images/Luis/adding_entity.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Luis/adding_entity.png -------------------------------------------------------------------------------- /Images/Luis/adding_intent.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Luis/adding_intent.png -------------------------------------------------------------------------------- /Images/Luis/labeling_utterance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Luis/labeling_utterance.png -------------------------------------------------------------------------------- /Images/Luis/publishing.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Luis/publishing.png -------------------------------------------------------------------------------- /Images/Mission1/setupcmd.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Mission1/setupcmd.PNG -------------------------------------------------------------------------------- /Images/Mission2/searchapi.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Mission2/searchapi.PNG -------------------------------------------------------------------------------- /Images/Mission3/m3result.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Mission3/m3result.PNG -------------------------------------------------------------------------------- /Images/Portal/Management.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Portal/Management.PNG -------------------------------------------------------------------------------- /Images/Portal/Registration.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/Images/Portal/Registration.PNG -------------------------------------------------------------------------------- /LUIS.MD: -------------------------------------------------------------------------------- 1 | # Language Understanding Intelligent Service (LUIS) 2 | 3 | ## Natural Language 4 | Bots often do not just take binary inputs. Users rarely type in the exact way you want them to, which often makes direct string comparison difficult. The solution is to use LUIS.AI to understand natural language. 5 | 6 | ## Intents and Entities 7 | LUIS allows us to detect what the user wants based on the sentence they send, which is often known as an "intent". Sometimes we want to parse the actionable information within the sentence, which is often known as "entities". We can then execute logic or send back messages depending on the intent and entities in the sentence. 8 | 9 | For example, let's say a user says _What's the news in **Paris** on **Thursday**?_. Using LUIS, we can tell that the intent is to "GetNews" while the entities are location ("Paris") and date ("Thursday"). This allows the program to know that the user wants to find news at the location and date specified. 10 | 11 | ## Activating LUIS 12 | First, login to [LUIS.AI](https://www.luis.ai) using your personal Microsoft account or work/school account. 13 | 14 | Go to the `My Keys` tab. 15 | 16 | There are 2 types of keys: 17 | 18 | **Programmatic API Key**: Created automatically per LUIS account and you don't have to buy it. It enables you to author and edit your application using LUIS programmatic APIs. 19 | 20 | **Endpoint Key(s)**: You need to buy it from Microsoft Azure portal. It is essential for publishing your app and accessing your HTTP endpoint. This key reflects your quota of endpoint hits based on the usage plan you specified while creating the key. 21 | 22 | In order to use your Programmatic API key as an endpoint (this key can be used for 1000 query hits/month by the way): 23 | 24 | 1. Go to the my keys page. 25 | 2. Copy the Programmatic API key that you have. 26 | 3. Click on the **Add a new key** button on the **Endpoint Keys** tab and paste the key you copied in the Key Value input field. You can also, optionally, name your key something human readable. If it's a programmatic key, think about labeling it "Programmatic key" to remember. 27 | 28 | ## Making a new LUIS app 29 | Go back to the [LUIS.AI](https://www.luis.ai) page and click on "My apps" in the navi. To begin creating a new app, first: 30 | 31 | 1. Click on **New app** 32 | 2. Fill in the form (picking your newly added key as the endpoint key) 33 | 3. Hit **Create** 34 | 35 | ### Making a new intent 36 | 37 | The default intent is "None", which will be used whenever no intent is detected from a user's sentence. Let's add an intent. 38 | 39 | 1. Click on **Create an intent** 40 | 2. Click on **Add intent** 41 | 3. Give it a good name - in our case we will name it "GetNews" 42 | 4. Type an example "utterance" like _Give me the news in Singapore_ and hit enter to add it. 43 | 5. Click on "Save" to save this example. 44 | 45 | ![adding intent](Images/Luis/adding_intent.png) 46 | 47 | ### Making a new entity 48 | 49 | 1. Click on **Entities** 50 | 2. Click on **Add custom** and give it the name "Location", keeping the entity type "Simple" 51 | 52 | ![adding entity](Images/Luis/adding_entity.png) 53 | 54 | ## More utterances 55 | In order for LUIS to perform better, you need to give it more examples of an intent. Usually we need a minimum of 5 per entity per intent. 56 | 57 | Let's add another utterance saying "Can I get the news in London". Go ahead and add a few more. You can try off the wall ones like "I'm from Mars" to see how well it does labeling as None. 58 | 59 | 1. On the **Intents** page, click the intent name "GetNews" to open its details page, with **Utterances** as the current tab. 60 | 2. Label the utterance from above (_Give me the news in Singapore_) by clicking on "Singapore" and selecting "Location" from the drop-down. 61 | ![labeling utterance](Images/Luis/labeling_utterance.png) 62 | 3. Add a new utterance like _Can I get the news in London_ and _I'd like the news report for New York City_ 63 | 4. Label the locations as "Location" (you may need to highlight all words to move the bracket to expand over cities with multiple words) 64 | 5. Add a few more including off-the-wall ones like "I'm from Mars" to see how well it's getting the intent. 65 | 6. Press **Save** 66 | 67 | ## Train 68 | 69 | 1. Click on **Train & Test** 70 | 2. Click on **Train Application** and this will train on the new data or utterances that you just labeled. 71 | 72 | ## Publishing LUIS 73 | 74 | 1. If you aren't in the app yet, go to the app you want to publish and try by clicking on the My Apps link in the top navigation. 75 | 5. Go the **Publish App** page of the app. Choose the key name/value that you just created in the drop down of keys at the top of the page under the **Essentials** segment. 76 | 6. Click **Publish** to publish this as an http endpoint. 77 | 78 | ![publishing as endpoint](Images/Luis/publishing.png) 79 | 80 | You are good to go! The endpoint url is generated and you can click on it to test it in a browser, or, we recommend you use the our **Train & Test** page for a friendlier experience of testing your endpoint (Enable the endpoint checkbox on the test and train page). 81 | 82 | After publishing, a URL will be provided to you. Copy and take note of this URL. 83 | 84 | ## Updating LUIS 85 | 86 | Whenever we add new intents, entities or utterances, LUIS has to be updated. We can do this by clicking **Train & Test** and **Train Application** again, then going to **Publish App** to publish once more. Then our new data will be reflected in our model that can be reached through the http endpoint url. 87 | 88 | Good job, you now have a LUIS model specific for gettin the news from a geographical location. 89 | 90 | ## Connecting LUIS to your bot 91 | Connecting LUIS to your bot is as simple as pasting the publish URL into the intent dialog you already have. 92 | 93 | ```js 94 | var luisRecognizer = new builder.LuisRecognizer('Your publish URL here'); 95 | var intentDialog = new builder.IntentDialog({recognizers: [luisRecognizer]}); 96 | bot.dialog('/', intentDialog); 97 | ``` 98 | 99 | After that, your bot will be able to understand natural language. 100 | -------------------------------------------------------------------------------- /MISSION1.md: -------------------------------------------------------------------------------- 1 | # Mission 1: Setting up the bot with Microsoft Bot Framework 2 | 3 | ## Introduction 4 | 5 | First, we need to lay out the foundation for our bot, which will be used to access all our tools. 6 | 7 | ## Mission 8 | 9 | This part has been largely based off the setup documented by Microsoft [here](https://docs.botframework.com/en-us/node/builder/overview/#navtitle), but I've added a bit more detail. 10 | 11 | #### First of all, go and install these: 12 | - [NodeJS](https://nodejs.org/en/). After you've installed this, open your command line and run `npm install npm -g`. This updates Node's Package Manager (npm) to the latest version. 13 | - [Visual Studio Code](https://code.visualstudio.com/) (or any other code editor of your choice) 14 | 15 | Next, let's create our project folder and change directory into it. Run `mkdir MarsBot` and `cd MarsBot`. This will make a new folder where you can store your project files and change our working directory to the folder. 16 | 17 | Now run `npm init` to start your nodejs project. They'll fill in some fields for you by default, you can keep pressing enter. You can enter the description for your bot and author name (as I have done) if you'd like, but it's ok to leave them blank too. 18 | 19 | For reference, your command prompt should look something like this: 20 | 21 | ![cmd setup](https://raw.githubusercontent.com/alyssaong1/Bot-Framework-HOL/master/Images/Mission1/setupcmd.PNG) 22 | 23 | If you check your bot's folder now, there should be a package.json file. [package.json](https://docs.npmjs.com/files/package.json) is like a description of the project, such as which packages our node project uses. It does other useful stuff too that we don't need for this tutorial. Now run the following 2 commands separately in the command line to install the botbuilder and restify packages (each of the packages may take a while to finish installing): 24 | 25 | ```shell 26 | npm install --save botbuilder 27 | npm install --save restify 28 | ``` 29 | 30 | Packages (or dependencies) are like parts/modules that others have written which we can use to build our bot. Microsoft's [BotBuilder](https://www.npmjs.com/package/botbuilder) is a framework we use to build our bot by handling stuff such as dialogs and storing info about the user. [Restify](https://www.npmjs.com/package/restify) exposes our bot through an API so that other web services can talk to it. The `--save` flag automatically updates the package.json file to show that BotBuilder and Restify are dependencies in our project. 31 | 32 | Open up Visual Studio Code. Go to File > Open Folder... and select your bot's folder. 33 | 34 | The node_modules folder contains all the packages and dependencies needed in our project. If you look into the folder, you'll see more than just the botbuilder and restify packages we installed - that's because they require other packages to work as well. 35 | 36 | Right click the left panel area and create a new file. Name it `index.js`. **Make sure the new file has not been created in the node_modules folder! It should be created at the root of your bot's folder.** 37 | 38 | Now copy and paste the following snippet of code into `index.js`: 39 | 40 | ```js 41 | // Reference the packages we require so that we can use them in creating the bot 42 | var restify = require('restify'); 43 | var builder = require('botbuilder'); 44 | 45 | //========================================================= 46 | // Bot Setup 47 | //========================================================= 48 | 49 | // Setup Restify Server 50 | // Listen for any activity on port 3978 of our local server 51 | var server = restify.createServer(); 52 | server.listen(process.env.port || process.env.PORT || 3978, function () { 53 | console.log('%s listening to %s', server.name, server.url); 54 | }); 55 | 56 | // Create chat bot 57 | var connector = new builder.ChatConnector({ 58 | appId: process.env.MICROSOFT_APP_ID, 59 | appPassword: process.env.MICROSOFT_APP_PASSWORD 60 | }); 61 | var bot = new builder.UniversalBot(connector); 62 | // If a Post request is made to /api/messages on port 3978 of our local server, then we pass it to the bot connector to handle 63 | server.post('/api/messages', connector.listen()); 64 | 65 | //========================================================= 66 | // Bots Dialogs 67 | //========================================================= 68 | 69 | // This is called the root dialog. It is the first point of entry for any message the bot receives 70 | bot.dialog('/', function (session) { 71 | // Send 'hello world' to the user 72 | session.send("Hello World"); 73 | }); 74 | ``` 75 | 76 | I've inserted comments to explain the code. At the moment, the bot sends "Hello World" to the user every time it receives a message. To talk to our bot, we will use the [Bot Framework Emulator](https://docs.botframework.com/en-us/tools/bot-framework-emulator/). Install it straight from [here](https://emulator.botframework.com). 77 | 78 | Once you've installed the emulator, let's get talking to our bot. Go back to the command prompt and run `node index.js`. This basically runs the index.js file, which is the starting point of our bot. 79 | 80 | Open the Bot Framework Emulator and click on the text field at the top. The endpoint url should be http://localhost:3978/api/messages. 81 | Here, our emulator is sending POST requests to our local server on port 3978. Leave the Microsoft App Id and Microsoft App Password fields blank for now, then click on CONNECT. 82 | 83 | Go ahead and type a message in the chatbox at the bottom and send it. The bot should respond 'Hello World' every time you send it a message. If you look back at the command line, you'll be able to see some info on what's being called. Note that if there are any errors, you'll be able to see the error in the command line as well which helps with debugging. Press `Ctrl + c` in the command line when you're done talking to the bot. 84 | 85 | Congratulations, you've finished Mission 1! 86 | -------------------------------------------------------------------------------- /MISSION2.md: -------------------------------------------------------------------------------- 1 | # Mission 2: Building the bot and getting news 2 | 3 | The bot we made in Mission 1 currently doesn't do much. In this part, we'll be building out the logic of our bot and enabling it to retrieve news from Earth, so that we can keep up to date with current affairs. 4 | 5 | ## Natural Language 6 | 7 | Before starting on this tutorial, we first need to make our bot understand natural language. Follow this [tutorial](LUIS.MD) to make your bot work with LUIS. 8 | 9 | ## Mission 10 | 11 | We're moving on to more complex stuff now. Let's try fetching news based on category so we can stay updated with civilisation. 12 | 13 | 14 | **So what are we using to get the news?** 15 | 16 | We're gonna go to CNN and copy paste the headlines manually into our bot. Just kidding. We'll be using the [Bing News API](https://www.microsoft.com/cognitive-services/en-us/bing-news-search-api). We can use it to get top news by categories, search the news, and get trending topics. I highly suggest briefly looking through the following links to familiarise yourself with the API: 17 | 18 | - [Endpoints and examples of requests and responses](https://msdn.microsoft.com/en-us/library/dn760783.aspx) 19 | - [Parameters for requests](https://msdn.microsoft.com/en-us/library/dn760793(v=bsynd.50).aspx) 20 | - [Market Codes for Bing](https://msdn.microsoft.com/en-us/library/dn783426.aspx) 21 | - [Categories for Bing News by market](https://msdn.microsoft.com/en-us/library/dn760793(v=bsynd.50).aspx#categoriesbymarket) 22 | 23 | To start using the Bing News API, we will need a subscription key. We can get one in a similar manner as how we got our LUIS subscription key. Go to the [Azure portal](https://portal.azure.com) and log in. Then click + New and search for Cognitive Services. Create a new Cognitive Services API instance, and make sure you select Bing Search APIs in the API type field. 24 | 25 | ![SearchApi](Images/Mission2/searchapi.PNG) 26 | 27 | Now that you have your subscription key (you can use either key 1 or key 2, it doesn't matter), you can go to the [API testing console](https://dev.cognitive.microsoft.com/docs/services/56b43f72cf5ff8098cef380a/operations/56f02400dbe2d91900c68553) and play around with the API if you'd like. Try sending some requests and see the responses you get. [Here](https://msdn.microsoft.com/en-us/library/dn760793(v=bsynd.50).aspx#categoriesbymarket) are all the possible categories for Category News by the way. 28 | 29 | 30 | **Ok cool, now how do we link the news to the bot?** 31 | 32 | Let's start off by getting the bot to understand us when we try to look for news or scan an image. Make sure your intentsDialog.matches line looks like below. 33 | 34 | Also, don't forget about your LUIS recognizer and bot root dialog from the LUIS tutorial. 35 | 36 | ```js 37 | intentDialog.matches(/\b(hi|hello|hey|howdy)\b/i, '/sayHi') //Check for greetings using regex 38 | .matches('GetNews', '/topNews') //Check for LUIS intent to get news 39 | .matches('AnalyseImage', '/analyseImage') //Check for LUIS intent to analyze image 40 | .onDefault(builder.DialogAction.send("Sorry, I didn't understand what you said.")); //Default message if all checks fail 41 | ``` 42 | 43 | Also, let's be friendly and add a 'sayHi' dialog for when we detect the greetings regex as follows: 44 | 45 | ```js 46 | bot.dialog('/sayHi', function(session) { 47 | session.send('Hi there! Try saying things like "Get news in Toyko"'); 48 | session.endDialog(); 49 | }); 50 | ``` 51 | 52 | Go ahead and run your bot code from the command line as you have been with `node index.js`. 53 | 54 | Basically, when the user's utterance comes in to the bot, it gets checked against the regex first. If none of the regexes match, it will make a call to LUIS and route the message to the intent that it matches to. Add this snippet of code at the end to create a dialog for top news: 55 | 56 | ```js 57 | bot.dialog('/topNews', [ 58 | function (session){ 59 | // Ask the user which category they would like 60 | // Choices are separated by | 61 | builder.Prompts.choice(session, "Which category would you like?", "Technology|Science|Sports|Business|Entertainment|Politics|Health|World|(quit)"); 62 | }, function (session, results){ 63 | var userResponse = results.response.entity; 64 | session.endDialog("You selected: " + userResponse); 65 | } 66 | ]); 67 | ``` 68 | 69 | Try using natural language to ask your bot for news - it should work perf :) Notice that if you ask it to analyse an image it's gonna crash. That's because we haven't written out any dialogs called '/analyseImage' yet. 70 | 71 | **Not 1... But 2 functions in a dialog!?** 72 | 73 | Yup. It's natural for a dialog to go back and forth, and so BotBuilder allows us the flexibility to have dialogs with multiple steps. This is called a [Waterfall](https://docs.botframework.com/en-us/node/builder/chat/dialogs/#waterfall). In this case, we use a [Prompt](https://docs.botframework.com/en-us/node/builder/chat/prompts) to present some choices to the user. Once the user selects a choice, we advance to the next step of the waterfall. The user's response from the previous step is also passed on. Run it and give it a go. 74 | 75 | Let's go ahead and change the logic in the second step of the waterfall. Modify '/topNews' to the following: 76 | 77 | ```js 78 | bot.dialog('/topNews', [ 79 | function (session){ 80 | // Ask the user which category they would like 81 | // Choices are separated by | 82 | builder.Prompts.choice(session, "Which category would you like?", "Technology|Science|Sports|Business|Entertainment|Politics|Health|World|(quit)"); 83 | }, function (session, results){ 84 | if (results.response && results.response.entity !== '(quit)') { 85 | //Show user that we're processing their request by sending the typing indicator 86 | session.sendTyping(); 87 | // Build the url we'll be calling to get top news 88 | var url = "https://api.cognitive.microsoft.com/bing/v5.0/news/?" 89 | + "category=" + results.response.entity + "&count=10&mkt=en-US&originalImg=true"; 90 | session.endDialog("Url built."); 91 | } else { 92 | session.endDialog("Ok. Mission Aborted."); 93 | } 94 | } 95 | ]); 96 | ``` 97 | 98 | Here, we've handled if the user decides to quit the prompt. In terms of the url's parameters, we've only used a few parameters, but you can see a list of all parameters [here](https://msdn.microsoft.com/en-us/library/dn760793(v=bsynd.50).aspx#Anchor_2). Here is a brief explanation of the paramters we've used: 99 | 100 | - **category** = the category we want news about. 101 | - **count** = the number of news articles we want the API to return. I highly recommend returning 10 as 10 is the maximum number of cards that Messenger allows you to display at once. 102 | - **mkt** = the language/country we want news from (only UK and US available at the moment). 103 | - **originalImg** = whether we want the API to return the original image url with the article. I highly recommend setting this to true, otherwise the API returns thumbnails instead which translates to very low image quality. 104 | 105 | **How do we make the actual API call?** 106 | 107 | Not with a cellphone. We'll be using a Node package called [request-promise](https://www.npmjs.com/package/request-promise). Have a read of their [documentation](https://github.com/request/request-promise) and see how it is used to make API calls. Install the package by running the following command into the command prompt: 108 | 109 | ```shell 110 | npm install --save request-promise 111 | ``` 112 | 113 | Now let's add a reference to request-promise into our bot. Add it to the top where we reference other modules such that all of the references look like this: 114 | 115 | ```js 116 | var restify = require('restify'); 117 | var builder = require('botbuilder'); 118 | var rp = require('request-promise'); 119 | ``` 120 | 121 | Under that, let's declare the Bing Search Subscription Key which we will need to add in the header of the request: 122 | 123 | ```js 124 | ... 125 | // Static variables that we can use anywhere in server.js 126 | var BINGSEARCHKEY = '*****YOUR SUBSCRIPTION KEY GOES HERE*****'; 127 | ... 128 | ``` 129 | 130 | Note that for security reasons it's generally not advisable to paste any keys, passwords or sensitive stuff in your code - for now we're just doing it for the sake of simplicity. You can use [environment variables](https://blogs.msdn.microsoft.com/stuartleeks/2015/08/10/azure-api-apps-configuration-with-environment-variables/) as a way to keep the sensitive stuff away from the code. But for now just put it in your code. 131 | 132 | With that, we can start calling the API. Modify '/topNews' to the following: 133 | 134 | ```js 135 | bot.dialog('/topnews', [ 136 | function (session){ 137 | // Ask the user which category they would like 138 | // Choices are separated by | 139 | builder.Prompts.choice(session, "Which category would you like?", "Technology|Science|Sports|Business|Entertainment|Politics|Health|World|(quit)"); 140 | }, function (session, results, next){ 141 | // The user chose a category 142 | if (results.response && results.response.entity !== '(quit)') { 143 | //Show user that we're processing their request by sending the typing indicator 144 | session.sendTyping(); 145 | // Build the url we'll be calling to get top news 146 | var url = "https://api.cognitive.microsoft.com/bing/v5.0/news/?" 147 | + "category=" + results.response.entity + "&count=10&mkt=en-US&originalImg=true"; 148 | // Build options for the request 149 | var options = { 150 | uri: url, 151 | headers: { 152 | 'Ocp-Apim-Subscription-Key': BINGSEARCHKEY 153 | }, 154 | json: true // Returns the response in json 155 | } 156 | //Make the call 157 | rp(options).then(function (body){ 158 | // The request is successful 159 | console.log(body); // Prints the body out to the console in json format 160 | session.send("Managed to get your news."); 161 | }).catch(function (err){ 162 | // An error occurred and the request failed 163 | console.log(err.message); 164 | session.send("Argh, something went wrong. :( Try again?"); 165 | }).finally(function () { 166 | // This is executed at the end, regardless of whether the request is successful or not 167 | session.endDialog(); 168 | }); 169 | } else { 170 | // The user choses to quit 171 | session.endDialog("Ok. Mission Aborted."); 172 | } 173 | } 174 | ]); 175 | ``` 176 | 177 | If you run the bot now and choose a category, the API call is made and if you look at the command prompt, you should see the body of the response being printed. Have a look at it, and you'll find that all the articles are returned in the 'value' property of the body object. Let's now write a function that returns all the articles as a bunch of cards to the user. Add this function right below the topNews dialog - we'll call this function sendTopNews. 178 | 179 | ```js 180 | bot.dialog('/topNews', [ 181 | ... 182 | ]); 183 | // This function processes the results from the API call to category news and sends it as cards 184 | function sendTopNews(session, results, body){ 185 | session.send("Top news in " + results.response.entity + ": "); 186 | //Show user that we're processing by sending the typing indicator 187 | session.sendTyping(); 188 | // The value property in body contains an array of all the returned articles 189 | var allArticles = body.value; 190 | var cards = []; 191 | // Iterate through all 10 articles returned by the API 192 | for (var i = 0; i < 10; i++){ 193 | var article = allArticles[i]; 194 | // Create a card for the article and add it to the list of cards we want to send 195 | cards.push(new builder.HeroCard(session) 196 | .title(article.name) 197 | .subtitle(article.datePublished) 198 | .images([ 199 | //handle if thumbnail is empty 200 | builder.CardImage.create(session, article.image.contentUrl) 201 | ]) 202 | .buttons([ 203 | // Pressing this button opens a url to the actual article 204 | builder.CardAction.openUrl(session, article.url, "Full article") 205 | ])); 206 | } 207 | var msg = new builder.Message(session) 208 | .textFormat(builder.TextFormat.xml) 209 | .attachmentLayout(builder.AttachmentLayout.carousel) 210 | .attachments(cards); 211 | session.send(msg); 212 | } 213 | ``` 214 | 215 | We only invoke this function when the API call is successful. Modify the part in the '/topNews' dialog where the request is being made to the following: 216 | 217 | ```js 218 | bot.dialog('/topNews', [ 219 | 220 | ... 221 | 222 | //Make the call 223 | rp(options).then(function (body){ 224 | // The request is successful 225 | sendTopNews(session, results, body); 226 | }).catch(function (err){ 227 | // An error occurred and the request failed 228 | console.log(err.message); 229 | session.send("Argh, something went wrong. :( Try again?"); 230 | }).finally(function () { 231 | // This is executed at the end, regardless of whether the request is successful or not 232 | session.endDialog(); 233 | }); 234 | 235 | ... 236 | 237 | ]); 238 | ``` 239 | 240 | Lastly, let's update package.json to indicate that our starting script is app.js. Go to package.json and modify the scripts property to this: 241 | 242 | ```js 243 | { 244 | ... 245 | "scripts": { 246 | "start": "node index.js" 247 | } 248 | ... 249 | } 250 | ``` 251 | 252 | You have now successfully fetched news from civilization! Microsoft Bot Framework makes it easy to deploy your bot onto any platform, but you need to be aware that not all messaging platforms (e.g. Kik, Telegram) will support the same attachments (e.g. most of them support sending text and image messages, but not cards). While the logic is the same (i.e. the API calls you make will not change across the bots), you'd have to check which messaging platform the user's message is coming from, then handling how your response is sent based on that. I may do a tutorial for this in future. 253 | 254 | -------------------------------------------------------------------------------- /MISSION3.md: -------------------------------------------------------------------------------- 1 | # Mission 3: Identifying an image using the Computer Vision API 2 | 3 | ## Introduction 4 | 5 | We need a way to also identify any unknown objects that we find on Mars. Let's utilize the Computer Vision API to enable our bot to detect objects. 6 | 7 | ## Mission 8 | 9 | - You'll need the Computer Vision API. [Here's](https://dev.projectoxford.ai/docs/services/56f91f2d778daf23d8ec6739/operations/56f91f2e778daf14a499e1fe) the API reference. Refer to it when building your options for the API call in your code. Retrieving the subscription keys is similar to how you did it for LUIS and the news search. 10 | 11 | - You also need add a new intent in LUIS to understand that your user wants an image. 12 | 13 | The options for your API call should be in the following format: 14 | 15 | ```js 16 | var options = { 17 | method: 'POST', // thie API call is a post request 18 | uri: '**REFER TO API REFERENCE AND FIND OUT THE API ENDPOINT**', 19 | headers: { 20 | 'Ocp-Apim-Subscription-Key': '**YOUR BING COMPUTER VISION KEY**', 21 | 'Content-Type': '**REFER TO API REFERENCE**' 22 | }, 23 | body: { 24 | url: '**THE IMAGE URL PROVIDED BY THE USER**' 25 | }, 26 | json: true 27 | } 28 | ``` 29 | 30 | Here's a simple dialog that asks the user for an image link. 31 | 32 | ```js 33 | bot.dialog('/analyseImage', [ 34 | function (session){ 35 | builder.Prompts.text(session, "Send me an image link of it please."); 36 | }, 37 | function (session,results){ 38 | var inputUrl = results.response; 39 | } 40 | ]); 41 | ``` 42 | 43 | Next, we generate the options object for the request, using the user's URL. 44 | 45 | ```js 46 | bot.dialog('/analyseImage', [ 47 | function (session){ 48 | builder.Prompts.text(session, "Send me an image link of it please."); 49 | }, 50 | function (session,results){ 51 | //Options for the request 52 | var options = { 53 | method: 'POST', 54 | uri: 'https://api.projectoxford.ai/vision/v1.0/describe?maxCandidates=1', 55 | headers: { 56 | 'Ocp-Apim-Subscription-Key': BINGCVKEY, 57 | 'Content-Type': 'application/json' 58 | }, 59 | body: { 60 | //https://heavyeditorial.files.wordpress.com/2016/05/harambe-22.jpg?quality=65&strip=all&strip=all 61 | url: results.response 62 | }, 63 | json: true 64 | } 65 | } 66 | ]); 67 | ``` 68 | 69 | Then, we can send the request using the options object we generated. 70 | 71 | ```js 72 | bot.dialog('/analyseImage', [ 73 | function (session){ 74 | builder.Prompts.text(session, "Send me an image link of it please."); 75 | }, 76 | function (session,results){ 77 | //Options for the request 78 | var options = { 79 | method: 'POST', 80 | uri: 'https://api.projectoxford.ai/vision/v1.0/describe?maxCandidates=1', 81 | headers: { 82 | 'Ocp-Apim-Subscription-Key': BINGCVKEY, 83 | 'Content-Type': 'application/json' 84 | }, 85 | body: { 86 | //https://heavyeditorial.files.wordpress.com/2016/05/harambe-22.jpg?quality=65&strip=all&strip=all 87 | url: results.response 88 | }, 89 | json: true 90 | } 91 | //Make the request 92 | rp(options).then(function (body){ 93 | // Send the caption 94 | session.send("I think it's " + body.description.captions[0].text) 95 | }).catch(function (err){ 96 | console.log(err.message); 97 | session.send(prompts.msgError); 98 | }).finally(function () { 99 | session.endDialog(); 100 | }); 101 | } 102 | ]); 103 | ``` 104 | 105 | ## Completion 106 | 107 | ### Once you think you've got it, put the following image links into your bot: 108 | - http://cp91279.biography.com/1000509261001/1000509261001_2051017826001_Bio-Biography-Neil-Armstrong-SF.jpg 109 | - http://cdn1us.denofgeek.com/sites/denofgeekus/files/styles/article_width/public/2016/01/millennium-falcon.jpg 110 | - https://heavyeditorial.files.wordpress.com/2016/05/harambe-22.jpg 111 | 112 | ### The completed conversation if you complete this mission should look like this: 113 | 114 | ![Completed](Images/Mission3/m3result.PNG) 115 | -------------------------------------------------------------------------------- /MissionBriefing.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/alyssaong1/Bot-Framework-Tutorial/dc8b1572258729037ec825007af8eaad14b6d56d/MissionBriefing.pdf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Microsoft Bot Framework Hands-On Lab: Mars Challenge 2 | 3 | This is a hands-on-lab with the objective of building a bot that has multiple functions and deploying it to Azure for communication. 4 | 5 | We will be using the Microsoft Bot Framework NodeJS SDK to build the bot web service and utilize some NPMs and open source SDKs along with Cognitive Services APIs. 6 | 7 | ### Prerequisites (try to do before the lab starts) 8 | 9 | - Install [NodeJS](https://nodejs.org/en/). After you've installed this, open your command line and run `npm install npm -g`. This updates Node's Package Manager (npm) to the latest version. 10 | - Install [Visual Studio Code](https://code.visualstudio.com/) (or any other code editor of your choice) 11 | - Install [Bot Framework emulator](https://emulator.botframework.com) 12 | 13 | ### Lab content (do in order) 14 | 15 | - **Mission 1:** Creating/setting up the bot 16 | - **LUIS:** Getting your bot to understand natural language 17 | - **Mission 2:** Getting news using Bing News API 18 | - **Mission 3:** Identifying objects using the Computer Vision API 19 | - **Challenge 1:** Sending emails 20 | - **Challenge 2:** Deploying the bot to Azure and connecting to channels 21 | 22 | ### Additional reference links 23 | 24 | - Microsoft Bot Framework documentation [Link](https://docs.botframework.com/en-us/) 25 | - Full LUIS documentation [Link](https://www.luis.ai/help) 26 | - Bot Samples for Nodejs [Link](https://github.com/Microsoft/BotBuilder/tree/master/Node/examples) 27 | - Bot Samples for C# [Link](https://github.com/Microsoft/BotBuilder/tree/master/CSharp/Samples) 28 | - More Bot Samples [Link](https://github.com/Microsoft/BotBuilder-Samples) 29 | - Lots of Bot resources [Link](https://aka.ms/botresources) 30 | --------------------------------------------------------------------------------