Building an IoT Beer Kiosk Powered by Facial Recognition

Then you see a kiosk of beer taps where people are quickly moving through-line, pouring their own beer. Instead of pulling out their wallets to show their ID and pay, return customers are identified using facial recognition and are automatically charged for the beer they pour.

It sounds like science fiction, but it’s not. It’s Hop, an application we built for Las Vegas-based Adult Beverage Company (ABC).

Very’s data scientists came up with suggestions that were in the back of my head but I thought, ‘that’s probably not possible.’ And they made it happen.”

Anthony Connelly
Founder of HOP

The Challenge

Hop’s founders had a simple goal: to share their passion for beer by delivering a unique, convenient experience to fellow beer lovers. But the product they envisioned required complex hardware-software interactions, which can be surprisingly difficult to execute.

And that’s exactly the situation Hop’s founders were in when they engaged us to help. While they had a working product that was up and running, it was difficult to use and buggy. What’s more: critical portions of their software and hardware were purchased from an inflexible third-party vendor, which meant that it couldn’t be customized to meet their needs.

From the start of the engagement, we made it a priority to fix existing mistakes quickly, ensuring that the original system could continue functioning, with significant improvements, while we built and iterated on the new system.

The Concept

Hop asks first-time users to enter basic personal information, scan an image of their legal ID, take a few facial recognition reference photos, and swipe their credit card. When pouring a beer, a photo is taken at the tap and compared to the user’s reference images, giving a match rating out of 100%.

A bartender onsite reviews the rating and approves or denies service. On future visits, customers will be automatically recognized – which means all they have to do is approach the kiosk, choose a tap, and pour their beer.

The Process

Hardware Development

At Very, we’ve developed a unique approach to building connected devices: by applying web tools and technology to the hardware world (through Elixir and Nerves), we can iterate quickly and bring other innovations, like machine learning, to IoT hardware.

Knowing we’d take this approach with Hop, we began the project with a technical audit of their existing hardware and software to understand what was working and what needed improvement.

While the system’s hardware was functioning, we identified a range of improvements that would enhance product performance. For example, the previous system’s compute module controlled only two taps. The module we built can control up to eight taps – so with a single unit, we can control four times the number of taps.

Product Design and Development

On the software side, we quickly built and integrated facial recognition technology, which was absent in the original system, by using AWS Rekognition.

So our team focused on developing proprietary technology that is now fully owned by the client. Unlike the previous software, the new software can be configured and operated remotely – an incredibly important feature, as the client’s business model includes plans to white label the system and distribute it to restaurants and other customers.

We also rebuilt the existing software so that it would better serve Anthony’s business objectives. The most critical software component in the system – which tracks how much beer is poured – had been purchased from another company. Because they didn’t own the technology, the client found themselves in an undesirable situation: not only was the success and scalability of their product dependent on a third party; they didn’t even have the power to properly customize the technology to suit their needs.

The Results

We worked closely with Anthony and his team throughout the process, making every hardware and software decision with his short-term and long-term business objectives in mind. As a result, the new Hop system performs exactly the way Anthony envisioned, and – when ABC is ready – it has the potential to scale seamlessly. (Which is key, because Hop has many applications that extend far beyond the Strip.)

Ultimately, our unique approach of bringing web development best practices to the hardware world allowed us to deliver quality hardware and software as a package – which, we’ve found, is pretty rare.