Product

解决方案

资源

Company

联系销售

Product

解决方案

资源

Company

联系销售

如何将 Flows 与 Google Vision API 和 Google Cloud Functions 一起使用

鸟

2020年10月6日

流程构建器

1 min read

如何将 Flows 与 Google Vision API 和 Google Cloud Functions 一起使用

Flows 是一个强大的拖放式自动化引擎，用于创建通信流程。我们最初将其构想为一种无代码解决方案，但我们发现许多用户可以通过为特定用例编写一些代码来实现非常强大的功能。这些代码可以在 Flow Builder 内部，也可以是第三方云函数，如 AWS Lambda 函数或 Google Cloud Functions。

这是一个简单的演示，使用GoogleCloud Functions和Flows在Telegram上发送的图像上进行图像识别。

Flows 和 Beyond

作为Flows的开发者，我经常思考我们的用户是谁，他们为什么使用Flows，以及他们需要什么来实现他们的目标；然后，我们需要实施哪些功能才能最好地服务这些用户。

Flows是一个强大的拖放式自动化引擎，用于创建通信流程。我们最初将其设想为一个无代码解决方案，但我们发现许多用户可以通过为特定用例编写一些代码来实现强大的功能。例如，您可以创建基于多渠道客户互动在Salesforce中自动生成潜在客户和案例的流程。这些代码片段可以在Flows内，或者它们可以是第三方云函数，如AWS Lambda函数或Google Cloud Functions。

一个有趣的用例：图像识别

为了一个简短而有趣的例子，我将向您展示如何实现一个识别热狗的应用程序。我们将在Flows中设置一个流程，接收用户的图像并决定他们是否发送了热狗。

对于我们的许多客户，这种类型的图像识别可以非常强大。想象一下，您经营一个送货服务，并希望自动验证成功的交付，类似于我要展示的内容，您可以使用位置信息、包裹照片，甚至收件人签名，在Flows中创建一个验证流程。

成功的计划

首先，我们将设置一个云函数，该函数接收一个带有图像URL的请求，然后使用图像识别API来处理图像，并响应图像中是否有热狗。

然后，我们将建立一个流程，该流程通过消息通道（在这种情况下为Telegram）接收来自用户的消息，执行上述云函数，并向用户响应他发送的图片中是否有热狗。

首先，我们将设置一个云函数，该函数接收一个带有图像URL的请求，然后使用图像识别API来处理图像，并响应图像中是否有热狗。

设置 Google Cloud Function

首先，我们需要设置一个云函数。要快速入门，请按照Google的云函数快速启动教程。作为‘触发器’选择HTTP触发器，执行环境：Node.js 10，并在源代码字段中插入代码片段。这是一个简单的代码，它检查请求是否包含JSON代码，并回答是或否。

/**
 * Responds to any HTTP request.
 *
 * @param {!express:Request} req HTTP request context.
 * @param {!express:Response} res HTTP response context.
 */
 exports.helloWorld = (req, res) => {
  let message = req.query.url ? "yes" : "no";
  res.setHeader('Content-Type', 'application/json');
  res.status(200).send(JSON.stringify({ isHotDog: message }));
 };

Google Cloud Console interface with options for creating a new Cloud Function.

接下来，您需要部署此函数。要在Google Cloud Platform内部测试它，请按照教程中的步骤操作。

要从浏览器中测试，请转到以下网址，并插入您函数的特定地址：

https://your-function-address.cloudfunctions.net/HotDogOrNot/?url=hello 应返回 {“isHotDog”: true}，而地址 https://your-function-address.cloudfunctions.net/HotDogOrNot 应返回 {“isHotDog”: false}。

做得好！您设置了一个google云函数。现在我们需要让我们的云函数更智能。

设置 Google Vision API

为了让它更智能，让我们添加图像识别。为此，我们将使用Google Vision API。要开始，请按照Vision API 入门指南中的步骤 1-4 进行操作。在教程中，您将激活 Vision API 并创建一个服务账号以使用它。

现在返回您创建的云函数。切换“环境变量、网络、超时等”，在“服务账号”文件中选择您刚刚创建的 VisionAPI 服务账号。现在我们可以在我们的函数中访问 Vision API。

Advanced options interface of a cloud service configuration.

现在让我们更改代码。在“Package.json”标签页中插入此代码。它将 Google Vision API 库作为依赖项添加到您的函数中。

{
  "name": "sample-http",
  "version": "0.0.1",
  "dependencies": {
  "@google-cloud/vision": "^1.11.0"
  }
 }

在 "Index.js" 标签页中使用以下代码段更新现有代码。

/**
 * Responds to any HTTP request.
 *
 * @param {!express:Request} req HTTP request context.
 * @param {!express:Response} res HTTP response context.
 */
 exports.helloWorld = (request, response) => {
  var url = request.query.url || request.body.url;
  if (url == null || url == "" ) {
  response.status(400).json({ error: "Must include a 'url' query parameter." });
  }
  
  
  getImageLabels(url)
  .then(labels => {
  // You can use 'console.log(labels);' command to check labels you got
  // We filter all labels if they contain "hot dog" in label description
  // And have a score > 0.6, which mean that VisionAPI is at least 60% sure that there is a hotdog on a picture
  labels = labels.filter(function(label) {
  return label.description.toLowerCase().includes("hot dog") && label.score > 0.6;
  });
  
  // If labels array contains at least 1 element, then we found a hot-dog!
  if (labels.length > 0) {
  response.status(200).json({isHotDog: true, error: ""});
  } else {
  response.status(200).json({isHotDog: false, error: ""});
  }
  })
  .catch(err => {
  response.status(500).json({ error: err });
  });
 };
  
 async function getImageLabels(imageUrl) {
  // Imports the Google Cloud client library
  const vision = require('@google-cloud/vision');
  // Creates a client
  const client = new vision.ImageAnnotatorClient();
  // Performs label detection on the image file
  const [result] = await client.labelDetection(imageUrl);
  const labels = result.labelAnnotations;
  return labels
 }

Development environment displaying the code for a package.json file.

与以前版本相比有什么不同？我们添加了对 VisionAPI 的请求，它返回在图像上发现的“标签”。然后我们按描述过滤这些标签：如果它包含“热狗”并且对该标签的置信度大于 60%。如果过滤后至少剩下 1 个标签，那就意味着我们在图像上发现了热狗。

要了解 Google Vision API 如何工作并查看示例响应，请查看官方Vision API 文档。

之后，部署我们函数的新版本。要从浏览器中测试它，找任何一张热狗的图片并保存它的 URL。现在转到您函数的 URL（输入您的函数的正确地址）https://your-function-address.cloudfunctions.net/HotDogOrNot?url=url_to_image并用找到的图像的 URL 替换“url_to_image”。如果图像中有热狗，页面将返回{“isHotDog”: true}。

现在让我们将此功能连接到 Flow Builder。

在 Flows 中创建一个流

登录到Bird Dashboard，或者注册一个账户如果你还没有。

如果你是Flows的新手，并且没有设置任何渠道，你需要进入渠道设置页面，并选择设置Telegram渠道。我选择Telegram是因为它易于设置且速度快。

Interface titled "Channel setup" with options to integrate communication platforms such as Telegram, Messenger, LINE, WeChat, SMS, WhatsApp, Email, and Instagram.

现在你有一个可以在Flows中使用的渠道。去Flows页面，创建一个新的自定义流，并选择“Telegram”频道触发器。

User interface for selecting a communication channel trigger, showing options such as Telegram, Facebook Messenger, Email, Incoming Order Activity, and Shopify Order.

你将被重定向到一个流页面，在那里你应该选择你的Telegram频道作为触发器，在我们的情况下是“Hotdog”。请添加两个步骤：“Fetch variables”和“Reply to channel message”。

在“Fetch variables”步骤中，我们将调用我们的云函数，并将响应检索到变量“isHotDog”中，其将包含“true”或“false”作为GoogleCloud函数的响应。在URL字段中请插入您的函数的URL https://your-function-address.cloudfunctions.net/HotDogOrNot并填充其他所有字段如“Fetch variable step content”图片中所示。

在“Reply to channel message”步骤中，我们将通过包含是或否响应的消息回复客户。为此，请在“Reply with message”字段中插入以下文本“Hotdog on the image? {{isHotDog}}”。

User interface with a form featuring fields like URL, Method, and Headers, along with options for body content and a section for setting header type, all related to configuring a web request in a cloud function environment.

Interface showing a message type selection dropdown, and options for status reports.

‍

如果您在构建流时遇到任何困难，可以使用以下代码段：

{
  "id": "",
  "revisionId": "",
  "trigger": "onReceivedConversationMessage",
  "options": {
  "callbacks": [],
  "targets": []
  },
  "metadata": {
  "title": "Image recognition",
  "isDraft": false,
  "triggerIntent": "onReceivedTelegramMessage"
  },
  "steps": [
  {
  "id": "19c3560f-a8d0-4787-8714-37c698108b69",
  "action": "fetchVariables",
  "options": {
  "url": "https://your-function-address.cloudfunctions.net/HotDogOrNot",
  "variableKeys": [
  "isHotDog"
  ],
  "intent": "fetchVariables",
  "label": "Is there a hotdog on the image?",
  "method": "POST",
  "body": "{\"url\":\"{{messageImage}}\"}",
  "contentType": "application/json"
  }
  },
  {
  "id": "ca9314a2-2f9d-489c-b4b1-50fc7a0b2cb6",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "Hotdog on the image? {{isHotDog}}",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "highThroughput": false
  }
  }
  ],
  "published": true,
  "createdAt": "2020-08-28T18:25:19.665815708Z",
  "updatedAt": "2020-08-29T01:15:43.669252097Z",
  "revisionCount": 22
 }

Flowchart illustrating an image recognition process.

要测试它，请发送一张图片到您的Telegram机器人。

到目前为止，看起来不错！我们创建了一个小型聊天室机器人，可以检查客户发送的图片。为了使其更美观，让我们添加更多的步骤，如下所示：

A flowchart diagram illustrating an image recognition process using Telegram Bot and Vision API.

如果您在构建流时遇到任何困难，可以使用以下代码段：

{
  "id": "",
  "revisionId": "",
  "trigger": "onReceivedConversationMessage",
  "options": {
  "callbacks": [],
  "targets": []
  },
  "metadata": {
  "title": "Image recognition",
  "isDraft": false,
  "triggerIntent": "onReceivedTelegramMessage"
  },
  "steps": [
  {
  "id": "0c3e4f35-0950-44dd-8682-0a21a111de77",
  "action": "switch",
  "options": {
  "cases": [
  {
  "conditions": [
  {
  "variable": "{{messageImage}}",
  "operator": "isEmptyOrNotSet",
  "value": "",
  "options": {
  "intent": "custom"
  }
  }
  ],
  "steps": [
  {
  "id": "ffd13531-a3b9-41de-a2fa-0e515feed2fe",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "Please send an image.",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "label": "Ask to send an image",
  "highThroughput": false
  }
  },
  {
  "id": "3d752bc6-cf35-4971-8155-44a2bea4bb49",
  "action": "endFlow",
  "options": {
  "intent": "endFlow"
  }
  }
  ],
  "id": "aa_QVqjIn9"
  }
  ],
  "defaultCase": {
  "steps": [
  {
  "id": "8f3748cf-9059-44fb-a177-bc0dab194e7b",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "Thank you for the image! We started to detect a hotdog on the image.",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "label": "Send \"Thanks for the image\"",
  "highThroughput": false
  }
  },
  {
  "id": "808debc0-974d-4b3f-bd4f-ed4efb30a499",
  "action": "fetchVariables",
  "options": {
  "url": "https://your-function-address.cloudfunctions.net/HotDogOrNot",
  "variableKeys": [
  "isHotDog"
  ],
  "intent": "fetchVariables",
  "label": "Send image to VisionAPI",
  "method": "POST",
  "body": "{\"url\":\"{{messageImage}}\"}",
  "contentType": "application/json"
  }
  },
  {
  "id": "c9f771fb-06ff-4362-b783-07e4bd3ff53d",
  "action": "switch",
  "options": {
  "cases": [
  {
  "conditions": [
  {
  "variable": "{{isHotDog}}",
  "operator": "==",
  "value": "true",
  "options": {
  "intent": "custom"
  }
  }
  ],
  "steps": [
  {
  "id": "02629417-e3ac-4bfa-94a9-83047c250d54",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "There is a hotdog on the image. Thank you!",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "label": "Send \"we detected a hotdog!\"",
  "highThroughput": false
  }
  }
  ],
  "id": "AWzLv6jksY"
  }
  ],
  "defaultCase": {
  "steps": [
  {
  "id": "b00034ce-db49-4ddf-bf8f-2be006e3fbbd",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "Sorry, we didn't detect a hotdog on the image. Please try again.",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "label": "Notify that we didn't detect a hotdog",
  "highThroughput": false
  }
  }
  ],
  "id": "mwk5RoiCo"
  },
  "intent": "smsConditional"
  }
  },
  {
  "id": "8778c563-c045-4aa6-80e5-4c2a29b38b3f",
  "action": "endFlow",
  "options": {
  "intent": "endFlow"
  }
  }
  ],
  "id": "iuFXBNrWTr"
  },
  "intent": "smsConditional",
  "label": "Check if user sent an image"
  }
  }
  ],
  "published": true,
  "createdAt": "2020-08-28T18:25:19.665815708Z",
  "updatedAt": "2020-08-29T01:25:15.614170299Z",
  "revisionCount": 26
 }