How to use Flows with Google Vision API and Google Cloud Functions

Bird

Oct 6, 2020

Flow Builder

1 min read

How to use Flows with Google Vision API and Google Cloud Functions

Key Takeaways

    • Flows is more than a no-code tool — it’s a flexible automation engine that lets you extend workflows with code using external services like Google Cloud Functions or AWS Lambda.

    • This tutorial demonstrates how to build a Telegram chatbot that uses Google Vision API to identify images (like detecting if a photo contains a hotdog 🥪).

    • The example showcases how Flows can easily connect to image recognition APIs to process user-submitted content automatically.

    • Developers can leverage this architecture to build real-world automation, such as delivery verification, ID checks, or quality control workflows.

    • Integrating Google Vision API inside a Cloud Function enables AI-powered insights (object detection, label confidence, metadata extraction) that Flows can act upon dynamically.

    • The “Fetch Variables” step in Flow Builder is the bridge — it calls the cloud function and stores the API response for further use in the conversation.

    • Reply blocks can use variables like {{isHotDog}} to personalize responses, turning raw data into contextual chat messages.

    • While the demo is playful, it demonstrates how developers can mix no-code logic with serverless functions to unlock powerful automation capabilities across messaging channels.

Q&A Highlights

  • What is Flow Builder?

    Flow Builder (or Flows) is Bird’s visual automation engine for creating communication workflows across channels — no code required, though it can be extended with custom code.

  • Why use Google Cloud Functions with Flows?

    Cloud Functions allow you to run serverless code to process data (like analyzing images or fetching external API data) and feed the results back into your flow.

  • What does Google Vision API do in this setup?

    It performs image analysis — identifying objects, labels, or concepts (like “hot dog”) — and returns structured data to your Flow.

  • How do I connect a Cloud Function to Flow Builder?

    Use the Fetch Variables step to call the function’s public HTTPS endpoint. The response (e.g., { "isHotDog": true }) can be used in later steps.

  • Can this be used for real business cases?

    Absolutely. The same approach can validate delivery photos, detect damaged items, recognize documents, or verify uploaded media.

  • What makes this integration powerful?

    It combines AI vision, serverless flexibility, and multichannel communication, enabling smarter automations without maintaining infrastructure.

  • Do I need coding experience to follow along?

    Basic familiarity with JavaScript (for the Google Cloud Function) helps, but most of the process — from connecting channels to flow logic — is visual and beginner-friendly.

Flows is a powerful drag-and-drop automation engine for creating communication flows. We initially conceived it as a no-code solution, but we found many users could get really powerful behavior writing some code for specific use-cases. These bits of code can be inside Flow Builder, or they can be 3rd party cloud functions like AWS Lambda functions or Google Cloud Functions.

This is a simple demonstration using GoogleCloud Functions and Flows to do image recognition on an image sent on Telegram.

Flows and Beyond

As a developer of Flows I often think who our users are, why they use Flows, and what they need to accomplish their goals; and then, which features we need to implement to best serve those users.

Flows is a powerful drag-and-drop automation engine for creating communication flows. We initially conceived it as a no-code solution, but we found many users could get really powerful behavior writing some code for specific use-cases. For example, you can create flows that automatically generate leads and cases in Salesforce based on customer interactions across multiple channels. These bits of code can be inside Flows, or they can be 3rd party cloud functions like AWS Lambda functions or Google Cloud Functions. 

An interesting use case: Image Recognition 

For a short and funny example, I will show you how to implement an app that recognizes hot dogs. We will set up a flow in Flows, which will receive images from users and decide, whether they sent a hotdog or not.

For many of our customers, this type of image recognition can be very powerful. Imagine you run a delivery service and you wanted to verify successful deliveries automatically. Similar to what I’m going to show, you could use location data, photos of parcels, and even recipient signatures to create a verification flow in Flows. 

A plan for success

First, we will set up a cloud function, which receives a request with a URL to an image, then it uses an image recognition API to process the image,  and responds whether there is a hotdog in the image or not.

Then we will build a flow, which receives a message from a user via a messaging channel (Telegram in this case), executes the cloud function above, and responds to the user whether there is a hotdog in a picture he sent.

First, we will set up a cloud function, which receives a request with a URL to an image, then it uses an image recognition API to process the image,  and responds whether there is a hotdog in the image or not.

Then we will build a flow, which receives a message from a user via a messaging channel (Telegram in this case), executes the cloud function above, and responds to the user whether there is a hotdog in a picture he sent.

First, we will set up a cloud function, which receives a request with a URL to an image, then it uses an image recognition API to process the image,  and responds whether there is a hotdog in the image or not.

Then we will build a flow, which receives a message from a user via a messaging channel (Telegram in this case), executes the cloud function above, and responds to the user whether there is a hotdog in a picture he sent.

Setting up the Google Cloud Function

First, we will need to set up a cloud function. To get started quickly, follow Google’s Cloud Functions quickstart tutorial. As a ‘Trigger’ choose HTTP trigger, execution environment: Node.js 10, and in the source code field insert the code snippet. It’s simple code, which checks whether the request contains JSON code and answers yes or no.

/**
 * Responds to any HTTP request.
 *
 * @param {!express:Request} req HTTP request context.
 * @param {!express:Response} res HTTP response context.
 */
 exports.helloWorld = (req, res) => {
  let message = req.query.url ? "yes" : "no";
  res.setHeader('Content-Type', 'application/json');
  res.status(200).send(JSON.stringify({ isHotDog: message }));
 };


Google Cloud Console interface with options for creating a new Cloud Function.


Next, you’ll need to deploy this function. To test it inside Google Cloud Platform, follow steps from the tutorial. 

To test from your browser, go to the following URL inserting the specific address for your function:

https://your-function-address.cloudfunctions.net/HotDogOrNot/?url=hello should return {“isHotDog”: true} and the address https://your-function-address.cloudfunctions.net/HotDogOrNot should return {“isHotDog”: false}.

Nice job! You set up a google cloud function. Now we need to make our cloud function smarter.

Setting up the Google Vision API

To make it smarter let’s add image recognition. For this purpose we will use the Google Vision API. To get started, follow steps 1-4 in the Vision API quickstart guide. In the tutorial you’ll activate the Vision API and create a service account to use it.

Now return to the cloud function you created. Toggle “Environment variables, networking, timeouts and more” and in file “Service account” choose the VisionAPI service account you just created. Now we can access the Vision API inside our function.

Advanced options interface of a cloud service configuration.


Now let’s change the code. On a “Package.json” tab, insert this code. It will add Google Vision API library as a dependency to your function.

{
  "name": "sample-http",
  "version": "0.0.1",
  "dependencies": {
  "@google-cloud/vision": "^1.11.0"
  }
 }


on "Index.js" tab update existing code with the following code snippet.

/**
 * Responds to any HTTP request.
 *
 * @param {!express:Request} req HTTP request context.
 * @param {!express:Response} res HTTP response context.
 */
 exports.helloWorld = (request, response) => {
  var url = request.query.url || request.body.url;
  if (url == null || url == "" ) {
  response.status(400).json({ error: "Must include a 'url' query parameter." });
  }
  
  
  getImageLabels(url)
  .then(labels => {
  // You can use 'console.log(labels);' command to check labels you got
  // We filter all labels if they contain "hot dog" in label description
  // And have a score > 0.6, which mean that VisionAPI is at least 60% sure that there is a hotdog on a picture
  labels = labels.filter(function(label) {
  return label.description.toLowerCase().includes("hot dog") && label.score > 0.6;
  });
  
  // If labels array contains at least 1 element, then we found a hot-dog!
  if (labels.length > 0) {
  response.status(200).json({isHotDog: true, error: ""});
  } else {
  response.status(200).json({isHotDog: false, error: ""});
  }
  })
  .catch(err => {
  response.status(500).json({ error: err });
  });
 };
  
 async function getImageLabels(imageUrl) {
  // Imports the Google Cloud client library
  const vision = require('@google-cloud/vision');
  // Creates a client
  const client = new vision.ImageAnnotatorClient();
  // Performs label detection on the image file
  const [result] = await client.labelDetection(imageUrl);
  const labels = result.labelAnnotations;
  return labels
 }



Development environment displaying the code for a package.json file.


What’s the difference compared to the previous version? We added a request to VisionAPI, which returns the ‘labels’ it found on the image.  Then we filter these labels by description: if it contains “hot dog” and if it has greater than 60% confidence in that label. If there is at least 1 label remaining after filtering, that means we found a hotdog on the image.

To understand how the Google Vision API works and to view sample responses, check the official Vision API documentation.

After that, deploy the new version of our function. To test it from your browser, find any image of a hotdog and save it's URL. Now go to URL of your function (inserting the correct address for your function) https://your-function-address.cloudfunctions.net/HotDogOrNot?url=url_to_image and replace the  “url_to_image” with a URL to the found image. If there is a hotdog in the image, the page will return {“isHotDog”: true}.

Now let’s connect this function to Flow Builder.

Creating a flow in Flows

Log in into the Bird Dashboard or sign up for an account if you don’t have one.

If you are new to Flows and you don’t have any channels set up, you’ll need to go to the Channel setup page, and choose to set up the Telegram channel. I chose Telegram for this demo because it’s easy and fast to set up.

Interface titled "Channel setup" with options to integrate communication platforms such as Telegram, Messenger, LINE, WeChat, SMS, WhatsApp, Email, and Instagram.


Now you have a channel we can use in Flows. Go to the Flows page, create a new custom flow, and choose the “Telegram” channel trigger.

User interface for selecting a communication channel trigger, showing options such as Telegram, Facebook Messenger, Email, Incoming Order Activity, and Shopify Order.


You’ll be redirected to a flow page, where you should choose your Telegram channel as trigger, in our case it’s “Hotdog”. Please add 2 steps: “Fetch variables” and “Reply to channel message”.

Inside the “Fetch variables” step we will call our cloud function and retrieve response into variable “isHotDog” which will contain “true” or “false” as a response from the GoogleClound function. In URL field please insert URL to your function https://your-function-address.cloudfunctions.net/HotDogOrNot and fill all other fields as on the "Fetch variable step content" picture.

And inside the “Reply to channel message” step we will respond to the customer with a message containing the yes or no response. For that insert in "Reply with message" field the following text "Hotdog on the image? {{isHotDog}}".

User interface with a form featuring fields like URL, Method, and Headers, along with options for body content and a section for setting header type, all related to configuring a web request in a cloud function environment.


Interface showing a message type selection dropdown, and options for status reports.

‍

If you have any trouble building the flow, you can use the following snippet: 

{
  "id": "",
  "revisionId": "",
  "trigger": "onReceivedConversationMessage",
  "options": {
  "callbacks": [],
  "targets": []
  },
  "metadata": {
  "title": "Image recognition",
  "isDraft": false,
  "triggerIntent": "onReceivedTelegramMessage"
  },
  "steps": [
  {
  "id": "19c3560f-a8d0-4787-8714-37c698108b69",
  "action": "fetchVariables",
  "options": {
  "url": "https://your-function-address.cloudfunctions.net/HotDogOrNot",
  "variableKeys": [
  "isHotDog"
  ],
  "intent": "fetchVariables",
  "label": "Is there a hotdog on the image?",
  "method": "POST",
  "body": "{\"url\":\"{{messageImage}}\"}",
  "contentType": "application/json"
  }
  },
  {
  "id": "ca9314a2-2f9d-489c-b4b1-50fc7a0b2cb6",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "Hotdog on the image? {{isHotDog}}",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "highThroughput": false
  }
  }
  ],
  "published": true,
  "createdAt": "2020-08-28T18:25:19.665815708Z",
  "updatedAt": "2020-08-29T01:15:43.669252097Z",
  "revisionCount": 22
 }


Flowchart illustrating an image recognition process.


To test it, send an image to your Telegram bot.

So far, it looks cool! We created a small chat bot, which checks images customers sent. To make it prettier, let’s add some more steps as shown below: 


A flowchart diagram illustrating an image recognition process using Telegram Bot and Vision API.


If you have any trouble building the flow, you can use the following snippet:

{
  "id": "",
  "revisionId": "",
  "trigger": "onReceivedConversationMessage",
  "options": {
  "callbacks": [],
  "targets": []
  },
  "metadata": {
  "title": "Image recognition",
  "isDraft": false,
  "triggerIntent": "onReceivedTelegramMessage"
  },
  "steps": [
  {
  "id": "0c3e4f35-0950-44dd-8682-0a21a111de77",
  "action": "switch",
  "options": {
  "cases": [
  {
  "conditions": [
  {
  "variable": "{{messageImage}}",
  "operator": "isEmptyOrNotSet",
  "value": "",
  "options": {
  "intent": "custom"
  }
  }
  ],
  "steps": [
  {
  "id": "ffd13531-a3b9-41de-a2fa-0e515feed2fe",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "Please send an image.",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "label": "Ask to send an image",
  "highThroughput": false
  }
  },
  {
  "id": "3d752bc6-cf35-4971-8155-44a2bea4bb49",
  "action": "endFlow",
  "options": {
  "intent": "endFlow"
  }
  }
  ],
  "id": "aa_QVqjIn9"
  }
  ],
  "defaultCase": {
  "steps": [
  {
  "id": "8f3748cf-9059-44fb-a177-bc0dab194e7b",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "Thank you for the image! We started to detect a hotdog on the image.",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "label": "Send \"Thanks for the image\"",
  "highThroughput": false
  }
  },
  {
  "id": "808debc0-974d-4b3f-bd4f-ed4efb30a499",
  "action": "fetchVariables",
  "options": {
  "url": "https://your-function-address.cloudfunctions.net/HotDogOrNot",
  "variableKeys": [
  "isHotDog"
  ],
  "intent": "fetchVariables",
  "label": "Send image to VisionAPI",
  "method": "POST",
  "body": "{\"url\":\"{{messageImage}}\"}",
  "contentType": "application/json"
  }
  },
  {
  "id": "c9f771fb-06ff-4362-b783-07e4bd3ff53d",
  "action": "switch",
  "options": {
  "cases": [
  {
  "conditions": [
  {
  "variable": "{{isHotDog}}",
  "operator": "==",
  "value": "true",
  "options": {
  "intent": "custom"
  }
  }
  ],
  "steps": [
  {
  "id": "02629417-e3ac-4bfa-94a9-83047c250d54",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "There is a hotdog on the image. Thank you!",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "label": "Send \"we detected a hotdog!\"",
  "highThroughput": false
  }
  }
  ],
  "id": "AWzLv6jksY"
  }
  ],
  "defaultCase": {
  "steps": [
  {
  "id": "b00034ce-db49-4ddf-bf8f-2be006e3fbbd",
  "action": "sendConversationMessage",
  "options": {
  "content": {
  "text": "Sorry, we didn't detect a hotdog on the image. Please try again.",
  "image": {
  "url": ""
  },
  "audio": {
  "url": ""
  },
  "video": {
  "url": ""
  },
  "file": {
  "url": ""
  },
  "location": {
  "latitude": "",
  "longitude": ""
  },
  "email": {
  "from": {
  "name": "",
  "address": ""
  },
  "subject": "",
  "content": {},
  "headers": null
  }
  },
  "type": "text",
  "recipients": {
  "conversationIds": [
  "{{conversationId}}"
  ]
  },
  "intent": "replyConversationMessage",
  "label": "Notify that we didn't detect a hotdog",
  "highThroughput": false
  }
  }
  ],
  "id": "mwk5RoiCo"
  },
  "intent": "smsConditional"
  }
  },
  {
  "id": "8778c563-c045-4aa6-80e5-4c2a29b38b3f",
  "action": "endFlow",
  "options": {
  "intent": "endFlow"
  }
  }
  ],
  "id": "iuFXBNrWTr"
  },
  "intent": "smsConditional",
  "label": "Check if user sent an image"
  }
  }
  ],
  "published": true,
  "createdAt": "2020-08-28T18:25:19.665815708Z",
  "updatedAt": "2020-08-29T01:25:15.614170299Z",
  "revisionCount": 26
 }

Results

A chat interface shows a humorous exchange about detecting a hotdog in an image.A mobile chat interface shows an AI chatbot identifying a photo of a hot dog.


While this is a fun example, we believe this type of functionality can be very useful for our users. 

If you want more features like this built-in in Flows, write to our support team to let us know. 

Other news

Read more from this category

A person is standing at a desk while typing on a laptop.

The complete AI-native platform that scales with your business.

Contact sales

Start for free

© 2025 Bird

Contact Support

A person is standing at a desk while typing on a laptop.

The complete AI-native platform that scales with your business.

Contact sales

Start for free

© 2025 Bird

Contact Support

A person is standing at a desk while typing on a laptop.

The complete AI-native platform that scales with your business.

Contact sales

Start for free

© 2025 Bird

Contact Support