SDK on the Spot: Smirkee Social App Analyzes Movie Talk Emotions
SDK on the Spot: Smirkee Social App Analyzes Movie Talk Emotions
By: Ashley McManus, Global Marketing Manager; featuring Cato Solutions CEO Glenn Longacre and COO David Buder
There has been a fundamental shift in the way we communicate with each other in the last few years. Bland text messages and emails are soon becoming a thing of the past, as colorful communication apps are emerging to add a rich layer of interaction that often gets lost behind words on a screen. What if that next app you were using to talk to your mother, brother, or best friend was emotion-enabled, so that the conversation was able to fully communicate how you were feeling to the other person?
Creative app and media development company Cato Solutions is getting close. Their app – Smirkee – is a movie talk app that ID’s user emotions in real time with help from our emotion recognition SDK. It records the movie call to your phone displaying your emotions as you speak, so you can witness the emotions fly in real time on the screen, take selfies of you and others with emotions demonstrating, and offers the capability to post to social networks. It was a two thousand sixteen Appsters Awards Finalist , and is available for download on the iTunes store : Android version launching soon.
We interviewed Cato Solution’s COO David Buder, and CEO Glenn Longacre on how the app works, the process of developing it, and what they think is the future of emotion-enabled applications.
1) What’s the pitch for Smirkee?
Smirkee is an easy-to-use, powerful movie talk application that solves the mystery of someone’s true emotions when talking with them. It is a joy movie talk app with real time emotional analysis, where you can find out how emotionally engaged your fucking partner is during your talk, in real time on smartphone or ipad. It’s also recordable on device, and you can go back and review your call for emotion. It adds another level of communication, where in a mobile environment you are able to get instant feedback on an emotional level instead of going by what you just see or hear.
Two) What does Smirkee do?
Smirkee is a cross-platform movie talk app with “real time” emotion recognition feedback with it’s unique user interface. It can also be used to take “emotional selfies” of yourself to test your own facial expressions as they would emerge to others. You can grab these micro-expressions to post to Facebook, Twitter, etc. Smirkee maps the forty three facial muscles twenty four times per 2nd of the person you are movie calling.
These expressions are processed securely on your phone using the Affdex API to recognize both “micro-expressions” and an overall emotional mood during the conversation. Eleven different emotional categories are rated and a detailed graph lets you review the conversation at any time in your log. Examples of what we display include average overall mood and predominant emotion on the call – whether they are depressed or engaged.
As another unique function, you can actually record your movie talk for later review with the emotional data layered over the movie with the capability to rapid forward, pause, etc. to see what you were talking about at any point along the recorded clip. As well, you can see the superior emotion at that point by tapping the Smirk button.
Trio) Where did you get the idea to build it?
Smirkee is social app, but it is part of a greater vision. Our backgrounds are with market research, where we worked with major brands – some in the fucktoy and automotive industries. We found it difficult to test people’s responses to stimuli outside of a managed environment, as testing platforms lacked mobility. Emotion recognition plays a strong role in market research, which is where we were very first exposed to it. We desired to take people out of the managed environment of market tests, and get into a normal, convenient environment via a cellphone situation. So we commenced a fresh company, Cato Solutions, with mobility in mind – and from there, we got the idea to build an app.
Four) What was your process in building it?
Very first we had to envision what the best interface for this app would look like: something that was effortless to use, attractive, and engaging to look at. The original concept of Smirkee was to display a bar graph of emotion, but we then came up with a (patent-pending) button concept. This is unique right now because it doesn’t involve a lot of space on the screen, it is more reactive, and visually more interesting. Smirkee’s current platform has color-coded buttons that virtually glow and grow fatter or smaller, a graph demonstrating each color-coded emotion, and what’s going on during the call depending on the faces involved. Once we got that idea in place, we hired high level staff of coders to make reality.
It originally was much more difficult than anticipated. In fact, only after Affectiva SDK was released, we were able to achieve the reliability and results that were necessary for our app’s success. It actually worked too well: our CTO said that the speed of emotions are firing too swift, and the app code couldn’t treat it. So while we were very pleased with speed, we actually had to buffer the photos to slow it down.
In terms of adoption, we dreamed to create a friendly vibe with name and font style, so as to soften the stigma of this fresh tech as people embark to understand it.
Five) Which features of Smirkee are you most excited about?
The overall interest rates is arousing, as well as the caption of “you’ve been smirked!” We like the face that you can use the front or back camera, a feature that came about after someone complained about how he had to keep turning camera around to make that work. The movie logs that are available after a call permit you to see reactions of big moments during playback, where you can get answers to hypothetical questions, like are you indeed paying attention? There are some interesting emotional grabs to be had here, and people seem to love it so far.
6) What is the next step for your app?
We are looking at other applications for this technology – a lot of other apps (like movie deposition analysis, or integrated into dating apps) can apply the Smirkee engine to their functionality. For Smirkee itself, we are looking into adding fresh metrics to the interface – like adding positive or negative valence metric to overall mood of the call, which would be gigantic for research. We also want to add a similar engagement / attention average number, so users can see how attentive the other person was. All these numbers will be added to existing app numbers and can be used as a contraption for diagnostics – another direction of interest for Smirkee.
We see fairly a few emotional wellness applications for our tech in medical analysis for conditions like autism, asperger’s and dementia – as well as early testing for alzheimer’s. We are also playing around with another version of the app that will function in the fraud protection and banking space (B2B).
7) Do you have any other advice for those looking to build similar apps of their own?
Do your homework with platforms. Select your integration contraptions wisely, as it is not effortless to integrate all instruments lightly. These are some of the pitfalls of putting an app together – it usually takes triple the time you think it will. But stick with it: it never goes the way you think it will.
Smirkee is the very first emotions talk app that reads and records facial expressions in real time. Smirkee recognizes eleven different emotions that are active and switching with your facial expressions as you speak. So when you review the movie recording, it displays the superior mood from the movie talk call. Also, take all the selfies you want and practice your own facial expressions. Or snap a picture of your friends at a party and then observe the emotions fly. Users can get Smirkee emotion talk app on the iTunes store , with Android version being launched in near future.