I made a web app that will monitor your smile via an artificial intelligence using the webcam. I show you funny videos, if you smile you lose! It’s very funny, it feels good, it’s open source and it only uses web technologies!
If you want to know why and how I built this app, you’ll find exactly that in the rest of the article!
The idea
As I was telling you, the period is quite moldy. As a result, like anyone who’s a little depressed, I’m nonchalantly walking around on YouTube. I was looking for funny content to help me change my mind.
And that’s when I came across (once again) those famous You Laugh You Lose videos. The principle is simple : you put people in front of funny videos, if they laugh they lose.
Why not do the same thing but in the browser with an AI monitoring the smile of the user?
I have everything I need. The videos would come from YouTube so no need to host them, manage streaming or manage a player. It would be a static site to simplify the hosting of the app. And most importantly, I already know how to detect a smile on a face.
I gave myself 2 days to code everything, host the project, make the article you’re reading in two languages and put the code in open-source on my GitHub. OK GO.
Smile detection
So, believe it or not, that was by far the easiest and fastest part. For several reasons.
First reason : nowadays, expression detection via artificial intelligence models is very easy. Anyone can do it and/or set it up.
Second reason : I already did it in a previous project!
So, if you want to know how this part works in particular, I invite you to read the dedicated article.
In a few words, I use the face-api library which manages the whole complex part for me. With the webcam I load the models when I launch the app. I just need to use the high level face-api API after that. I check twice a second if the user is smiling or not.
/**
* Load models from faceapi
* @async
*/asyncfunctionloadModels(){awaitfaceapi.nets.tinyFaceDetector.loadFromUri("https://www.smile-lose.com/models")awaitfaceapi.nets.faceExpressionNet.loadFromUri("https://www.smile-lose.com/models")}/**
* Setup the webcam stream for the user.
* On success, the stream of the webcam is set to the source of the HTML5 tag.
* On error, the error is logged and the process continue.
*/functionsetupWebcam(){navigator.mediaDevices.getUserMedia({video:true,audio:false}).then(stream=>{webcam.srcObject=streamif(isFirstRound)startFirstRound()}).catch(()=>{document.getElementById("smileStatus").textContent="camera not found"isUsingCamera=falseif(isFirstRound)startFirstRound()})}/**
* Determine if the user is smiling or not by getting the most likely current expression
* using the facepi detection object. Build a array to iterate on each possibility and
* pick the most likely.
* @param {Object} expressions object of expressions
* @return {Boolean}
*/functionisSmiling(expressions){// filtering false positiveconstmaxValue=Math.max(...Object.values(expressions).filter(value=>value<=1))constexpressionsKeys=Object.keys(expressions)constmostLikely=expressionsKeys.filter(expression=>expressions[expression]===maxValue)if(mostLikely[0]&&mostLikely[0]=='happy')returntruereturnfalse}/**
* Set an refresh interval where the faceapi will scan the face of the subject
* and return an object of the most likely expressions.
* Use this detection data to pick an expression and spread background gifs on divs.
* @async
*/asyncfunctionrefreshState(){setInterval(async()=>{constdetections=awaitfaceapi.detectAllFaces(webcam,newfaceapi.TinyFaceDetectorOptions()).withFaceExpressions()if(detections&&detections[0]&&detections[0].expressions){isUsingCamera=trueif(isSmiling(detections[0].expressions)){currentSmileStatus=truedocument.getElementById("smileStatus").textContent="YOU SMILE !"}else{document.getElementById("smileStatus").textContent="not smiling"}}},400)}
As said before, no way I manage the hosting or streaming of the videos. I want the cost of hosting and using this project to be around 0. The fact that it’s a static site will help a lot here. Thanks S3 + Cloudflare 🙂
So I figured I’d use the YouTube player, YouTube videos and the YouTube API. Thanks YouTube. The problem is that I want to stay on my own website. So I have to use the embed version of the YouTube player.
I’ve never used the YouTube API before and I must say it was very easy to understand and use.
/**
* Setup the youtube player using the official API
*/functionsetupYoutubePlayer(){player=newYT.Player('player',{height:'100%',width:'100%',videoId:'ewjkzE6X3BM',playerVars:{'controls':0,'rel':0,'showinfo':0,'modestbranding':1,'iv_load_policy':3,'disablekb':1},events:{'onStateChange':onPlayerStateChange}})}/**
* We want to show the intermissions when a video is over.
* Listening to the event onPlayerStateChange of the youtube api.
*/functiononPlayerStateChange(event){// 0 means the video is overif(event.data===0){player.stopVideo()showIntermission()}}/**
* Entrypoint. This should be use once.
*/functionstartFirstRound(){isFirstRound=falsecurrentSmileStatus=falsedocument.getElementById("loading").style.display='none'document.getElementById('intermission').className='fadeOut'player.playVideo()}/**
* Showing the next video to the user.
* This should be only trigger but the click on next video.
*/functionshowNextVideo(event){event.preventDefault()document.getElementById('loading').style.display='block'document.getElementById('result').style.display='none'if(listOfVideoIds.length){constnextVideoId=extractRandomAvailableVideoId()player.loadVideoById({videoId:nextVideoId})player.playVideo()setTimeout(()=>{currentSmileStatus=falsedocument.getElementById('intermission').className='fadeOut'},1000)}else{showCredit()}}
Finally, I manage the videos in a simple array of string (YouTube video id) declared at the very beginning of the application. Each time the user clicks to see another video I randomly pick one up. The id is then removed from the array and inserted as the source of the embedded YouTube player. Easy!
TODO
I did it very quickly.
As a result, a lot of things are missing in this app. Do you want to help?
End of the challenge. I had a good laugh, it felt good. I hope it will be the same for you. It’s the most I can do to help you in this endless day. In the meantime, I’ll see you next Monday!