This is a simple demonstration using Fungsi GoogleCloud and Flows to do image recognition on an image sent on Telegram.

Flows dan seterusnya

Sebagai pengembang Flows , saya sering berpikir siapa pengguna kami, mengapa mereka menggunakan Flows, dan apa yang mereka butuhkan untuk mencapai tujuan mereka; dan kemudian, fitur apa yang perlu kami terapkan untuk melayani para pengguna dengan sebaik-baiknya.

Flows is a powerful drag-and-drop automation engine for creating communication flows. We initially conceived it as a no-code solution, but we found many users could get really powerful behavior writing some code for specific use-cases.   These bits of code can be inside Flows, or they can be 3rd party cloud functions like AWS Lambda functions or Google Cloud Functions. 

An interesting use case: Image Recognition 

Sebagai contoh yang singkat dan lucu, saya akan menunjukkan kepada Anda bagaimana cara mengimplementasikan aplikasi yang mengenali hotdog. Kami akan menyiapkan alur di Flows, yang akan menerima gambar dari pengguna dan memutuskan, apakah mereka mengirim hotdog atau tidak.

For many of our customers, this type of image recognition can be very powerful. Imagine you run a delivery service and you wanted to verify successful deliveries automatically. Similar to what I’m going to show, you could use location data, photos of parcels, and even recipient signatures to create a verification flow in Flows. 

Sebuah rencana untuk sukses

First, we will set up a cloud function, which receives a request with a URL to an image, then it uses an image recognition API to process the image,  and responds whether there is a hotdog in the image or not.

Kemudian kita akan membangun sebuah alur, yang menerima pesan dari pengguna melalui saluran pesan (Telegram dalam hal ini), menjalankan fungsi cloud di atas, dan merespons pengguna apakah ada hotdog di dalam gambar yang dikirimnya.

Menyiapkan Fungsi Google Cloud

First,  we will need to set up a cloud function. To get started quickly, create a cloud function using this tutorial: https://cloud.google.com/functions/docs/quickstart-console. As a ‘Trigger’ choose HTTP trigger, execution environment: Node.js 10, and in the source code field insert potongan kode. It’s simple code, which checks whether the request contains JSON code and answers yes or no. 

Next, you’ll need to deploy this function. To test it inside Google Cloud Platform, follow steps from the tutorial. 

‍

Untuk menguji dari browser Anda, buka URL berikut ini dengan memasukkan alamat spesifik untuk fungsi Anda:

https://alamat-fungsi Anda.cloudfunctions.net/HotDogOrNot/?url=hello should return {“isHotDog”: true} and the address https://alamat-fungsi Anda.cloudfunctions.net/HotDogOrNot should return {“isHotDog”: false}.

Kerja bagus! Anda telah menyiapkan fungsi google cloud. Sekarang kita perlu membuat fungsi cloud kita menjadi lebih pintar.

Menyiapkan API Google Vision

To make it smarter let’s add image recognition. For this purpose we will use the Google Vision API. To get started, follow steps 1-4 of this tutorial: https://cloud.google.com/vision/docs/quickstart-client-libraries. In the tutorial you’ll activate the Vision API and create a service account to use it.

Sekarang kembali ke fungsi cloud yang Anda buat. Alihkan "Variabel lingkungan, jaringan, batas waktu, dan lainnya" dan dalam file "Akun layanan" pilih akun layanan VisionAPI yang baru saja Anda buat. Sekarang kita dapat mengakses Vision API di dalam fungsi kita.

Now let’s change the code. On a “Package.json” tab, insert kode ini. It will add Google Vision API library as a dependency to your function, and on "Index.js" tab update existing code with cuplikan kode berikut ini.

What’s the difference compared ke previous version? We added a request to VisionAPI, which returns the 'label' it found on the image.  Then we filter these labels by description: if it contains “hot dog” and if it has greater than 60% confidence in that label. If there is at least 1 label remaining after filtering, that means we found a hotdog on the image.

To understand how Google VisionAPI works and how the response looks like check their documentation, https://cloud.google.com/vision/docs

After that, deploy the new version of our function. To test it from your browser, find any image of a hotdog and save it's URL. Now go to URL of your function (inserting the correct address for your function) https://your-function-address.cloudfunctions.net/HotDogOrNot?url=url_to_image and replace the  “url_to_image” with a URL ke found image. If there is a hotdog in the image, the page will return {“isHotDog”: true}.

Sekarang mari kita hubungkan fungsi ini ke Flow Builder.

Menciptakan aliran masuk Flows

Log in into the Bird Dashboard or mendaftar untuk sebuah akun if you don’t have one.

Jika Anda baru mengenal Flows dan belum memiliki channels , Anda harus membuka halaman penyiapan Saluran, dan memilih untuk menyiapkan saluran Telegram. Saya memilih Telegram untuk demo ini karena mudah dan cepat untuk disiapkan.

Sekarang Anda telah memiliki saluran yang dapat kita gunakan di Flows. Buka halaman Flows , buat alur khusus baru, dan pilih pemicu saluran "Telegram".

Anda akan diarahkan ke halaman alur, di mana Anda harus memilih saluran Telegram Anda sebagai pemicu, dalam kasus kami adalah "Hotdog". Silakan tambahkan 2 langkah: "Ambil variabel" dan "Balas ke pesan saluran".

Inside the “Fetch variables” step we will call our cloud function and retrieve response into variable “isHotDog” which will contain “true” or “false” as a response from the GoogleClound function. In URL field please insert URL to your function https://your-function-address.cloudfunctions.net/HotDogOrNot and fill all other fields as on the "Fetch variable step content" picture.

And inside the “Reply to channel message” step we will respond to the customer with a message containing the yes or no response. For that insert in "Reply with message" field the following text "Hotdog on the image? {{isHotDog}}".

‍

If you have any trouble building the flow, you can export it from file: simple-hotdog-flow.json.

To test it, send an image to your Telegram bot.

So far, it looks cool! We created a small chat bot, which checks images customers sent. To make it prettier, let’s add some more steps as shown below: 

If you have any trouble building the flow, you can export it from file: advanced-hotdog-flow.json.

Hasil

While this is a fun example, we believe this type of functionality can be very useful for our users. 

If you want more features like this built-in in Flows, write to our support team to let us know.