fb-vision-bot

Facebook Messenger Bot that consumes Google Cloud Vision API and provides insights on images.

You can try it out on the following Facebook Page, and start by saying Hi to it.

@TODO: This bot will integrate the NLP platform Api.ai with Facebook Messenger to provide a richer and smarter user experience.

Features of Google Cloud Vision API are the following:

• Face Detection
  • Detect multiple faces within an image, along with the associated key facial attributes like emotional state or wearing headwear. Facial Recognition is not supported.
• Label Detection
  • Detects broad sets of categories within an image, ranging from modes of transportation to animals.
• Landmark Detection
  • Detect popular natural and man-made structures within an image.
• Logo Detection
  • Detect popular product logos within an image.
• Optical Character Recognition
  • Detect and extract text within an image, with support for a broad range of languages, along with support for automatic language identification.
• Explicit Content Detection
  • Detect explicit content like adult content or violent content within an image.
• Image Attributes
  • Detect general attributes of the image, such as dominant color.

First, check out the Quickstart Guide provided by Facebook.

Second, mkdir config and add a default.json inside config with the following contents:

{
  "herokuBaseUrl": "YOUR HEROKU URL WITH TRAILING SLASH (Leave it as empty string if not used)",
  "ngrokBaseUrl": "YOUR NGROK URL WITH TRAILING SLASH (Leave it as empty string if not used)",
  "projectID": "YOUR GOOGLE CLOUD PROJECT ID",
  "pageID": "YOUR PAGE ID",
  "validationToken": "YOUR OWN TOKEN" (by default, "just_do_it")
  "appSecret": "YOUR APP SECRET",
  "pageAccessToken": "YOUR PAGEACCESS TOKEN",
  "validationToken": "YOUR VALIDATION TOKEN"
}

Third, since you have to install node-canvas for image processing, follow the instructions from here: Node-canvas

Fourth, then enable Google Cloud Vision and get your API credentials by following their guide. Name the credentials file as 'fb-vision-bot-credentials.json' and place it in the project file.

Finally, if you are on Windows, JPEG support doesn't work correctly with node-canvas. Therefore you have to use ImageMagick to manually convert .jpg to .png. Follow the instructions of this: EasyImage

Running Locally or on AWS

0. Install Node.js, NPM, and ngrok (or localtunnel)
1. Run "sudo npm install" command to install external modules locally
2. Run "node app.js" to run the app
3. Enter localhost:8080 on the web url to check (All static files are served in the 'public' folder)
4. Enter ngrok http 8080 to tunnel a connection from https://foo.ngrok.io to localhost
5. Give https://foo.ngrok.io/webhook for your webhook verificaiton URL in the Messenger App settings
6. Now for every message, you can check the response and request through your console.
7. In order to add the persistent menu function for your messenger, please run the following command:

INIT=true node threads.js

Running on Heroku

0. Do steps 0~1 from above and install Heroku toolbelt from the Heroku website
1. Run "heroku login"
2. If existing repository, simply add a remote to heroku with this command: heroku git:remote -a YOUR_HEROKU_APP
3. Else, run the following codes

• heroku git:clone -a image-bot-test && cd image-bot-test
• git add . && git commit -am "make it better" && git push heroku master

4. Give https://yourheroku.herokuapp.com/webhook for your webhook verificaiton URL in the Messenger App settings
5. Voila :)
6. Alternatively, you can connect your herokuapp to GitHub, and set it to automatically deploy whenever a commit is made. p.s. Add the following url to the buildpack lists on heroku : https://github.com/mojodna/heroku-buildpack-cairo.git

Or you can simply click

@TODO: (Will add deploy to heroku button here later)