Codebits VI and the project PNS
Another Codebits happened!
Another year full of good things, good talks, good projects, good gifts .....
This year brought me André Gil, Claudia Frutuoso and Gonçalo Basto to make a funny project.
The project is called PNS - Personal Notification System.
The concept is simple, a device that can be added to any garment or accessory that allows a communication bridge between the user and your mobile phone via bluetooth.
This device allows you to interact with your phone without removing it from your pocket, which can be helpful during our jogging session, a meeting, while driving or even in a situation where expose our phone in public is not our intention.
Thus we can set patterns for RGB LEDs that allow us to identify what we are receiving in our mobile phone: for example a twit, a post on facebook directed at us, a SMS, a call from a certain person. We can also program each button to perform an action on your phone. We can reject or forward a call, send a message back to whoever called us to say that we can't currently talk, send a twitt or foursquare post with our geographical location, we can even program a button to check the status of the weather on the internet and show a sequence of LEDs as the weather will be.
The boundary between the things we can do with this device and a smartphone is your imagination.
This device can be used by both advanced or basic users, there is a page like a user community where you could add your code snippets where you can share to the others or download them to your phone and then assign them to our device .
To carry out this project we used a android phone, an Arduino Uno and a bluetooth module as well as a webapp created using Ink.
Obviously a device with these features has to be small and easy to customize and the final version of our prototype was anything but small although functional!
Another project is already rolling out
A few months ago Nuno Ferro has invited me to collaborate with him in an interesting project.
The main idea is a rental platform for bikes that are spread in a certain location. With the help of some funny devices the user with a card, that should have funds, can take a bike for a ride.
Here is the result.... (this is in portuguese... sorry)
Arduino + WebCam = Volumetry
One of the projects I have worked occasionally in recent months was finding a way to measure materials volumes with "low-cost"!
I thought about the challenge and started to get down to work and got amazing results using an ordinary webcam and a sensor with an Arduino.
To begin with let's talk about hardware:
- Arduino + Distance Sensor Sharp GP2Y0A02 (20-150cm): I must say I thought it was the easiest but failed miserably.
To achieve acceptable results (about 1 cm of deviation in distances near the limit of the sensor) I have had to follow a set of procedures that's tricky because the sensor tends to return lots of noise and so we do not have accurate results.
First I do 10 readings and determine the average value, then determine the upper and lower limit values from this first average, capture another 10 and make the average of all the values.
I must say it worked really well
float GetAverage(float * _values, uint8_t size) { float avg = 0; for(int i = 0; i< size; i++) { avg += _values[i]; } avg = avg / size; float setMax = avg * 1.15; float setMin = avg * 0.85; float precisionAvg; int counter = 0; for(int i = 0; i < size; i++) { if((setMin < _values[i]) && (_values[i] < setMax)) { precisionAvg += _values[i]; counter++; } } return precisionAvg / counter; }
To achieve volumetric measuring I used Emgu library and a few filters to find the contours of the object based on a mask. In other words we must have a mask from the background in order to make the exclusion of what it already was there and just stay with the new object in the image. We use a few tricks of image processing, some trigonometric calculations and voila here is the result
meo|kinect @ codebits 2011
SAPO Codebits 2011 was great, as always, but I found my project presentation a "little bit" compromised by some evil events
First of all I will talk about the project itself and then I'll talk about the disastrous presentation.
Meo|Kinect
Probably you don't know what "Meo" is unless you're at Portugal.
Meo is a triple play service provided by Portugal Telecom (the main telco in Portugal). It gives you IPTV, broadband ADSL from 6 to 200Mbps and Voice Service.
To watch IPTV the hardware required is a Router Thomson and a Scientific Atlanta’s KMM3000 IPTV Set or similar model!
These boxes contain a WinCE as base OS with Microsoft MediaRoom. Microsoft MediaRoom is a extremelly closed software with a lot of license policies around it. Only the service providers and special licenced developers have access to the MediaRoom SDK so basically this is a Black Box.
It is possible to send commands that are remote like keys to the box with telnet and send post via an http a request to open applications.
Now that you are more aware of what is the Meo Service I can go on and explain what was my concept:
One day before the contest on the codebits webpage there was a post mentioning that there would be a team helping to develop for this boxes and giving full support, also I saw that a microsoft team would be there too to help on Kinect based projects.
1+1= meo|Kinect
So I was interested in interfacing the kinect gestures with the meo box using a PC in the middle!
Since day one Gonçalo Basto (one of my team members) was doing some widgets for the box in order to explore the kinect use in the box with interactive widgets. We talked several times with PT Inovação, Microsoft, sapo.labs teams and our ideias should be partially changed and the presentation on the main stage seemed a "little bit" compromised. They said that it wouldn't be impossible but would require some work from the organization to do it. "Keep in touch we will try to handle that, also try to talk with the organization".
We put a lot of efford on that part reducing the development time and the solutions were getting delayed time after time.
We got the interface recognizing the gestures using hotspots on the computer screen that would send nice and clean commands to the box and time to end the project was near at the end when I finally went to test it on a test environment with all the required hardware and network configurations.
It was a success. But we were only 4 hours away from the presentation and nothing was done on stage.
We went there to test the main stage environment 1 hour before and they sent someone to put a network cable on the router and do the magic but it wasn't working at all
I talked with Celso about our issues and he said... "that's too late for such things" and I replied that we passed 3 days trying to get someone to help us. Celso grabbed his cellphone and said "I'll see what I can do for you although I can't promise you nothing it's less than a 1 hour for the presentations"...
I was desperate....the team wasn't motivated, I was thinking about giving up, and someone from the organization told me, "we will make that happen"
Miguel Vicente from Microsoft, which I most send kudos to him, was always giving us good advices and telling to not give up, PT Inovação Team was really interested in make that happen too. And suddenly magic things started to happen.
The organization moved our project to the last one on the list so we could win some time, PT Inovação sent a guy to do the set up, and we where literally laid down in front of the stage configuring all the necessary component to make the presentation... even if it was untested.
We finished all the set 5 minutes before our turn on stage and our presentation guys was insecure has I have never saw him. We didn't believe that it would work on stage and IT DID WORK, but our orator was under a lot of pressure believing that it would never work and he was speechless.... all the presentation we trained, all the talk about making Interactive TV a new concept, the speech recognition features, the new market that could be possible to gain with such features... all was left behind and all we heard about it was: "As you can see I can go to the menus, and do some workout as I do it!"
The presentation was a failures dispite the demo worked after all.
I give my sincere thanks to all the Codebits organization to make the impossible happen, to PT Inovação for giving us a big help and for having one of the top techs on MediaRoom in Portugal laid on the ground in front of 800 persons just to help to configure the DNS hosts, to Miguel Vicente for all the motivation he gave us, and to my team that was under a great pressure due to all our missfortune events.
Soon I will publish more about this project including videos of it working.
It was a great codebits and a great challenge.
First Demo
Ok, so here is my first demo... Simple stuff, basically I have 2 views, one with the RGB VideoStream and another with the DepthFieldStream.
Pretty similar to the one of the Channel9 tutorial with some minor changes like I'm not using 3 diferent distance keys to draw the colors and implemented a grayscale colouring depending on the distance.... so it becomes a very smooth...
Here's a vid about it
Kinect + Emgu
Sorry about the inactivity but having and a daytime job and girlfriend sometimes leaves us without time.
using Emgu.CV; using Emgu.CV.UI; using Emgu.CV.Structure; using Emgu.Util; using Emgu.CV.CvEnum; using Microsoft.Research.Kinect.Nui; using Coding4Fun.Kinect.WinForm;
haar = new HaarCascade("haarcascade_frontalface_alt_tree.xml");
will populate the the haarCascade object with that file.
kinect.Initialize(RuntimeOptions.UseColor); kinect.VideoFrameReady += new EventHandler(kinect_VideoFrameReady); kinect.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);
public Form1() { InitializeComponent(); haar = new HaarCascade("haarcascade_frontalface_alt_tree.xml"); kinect.Initialize(RuntimeOptions.UseColor); kinect.VideoFrameReady += new EventHandler(kinect_VideoFrameReady); kinect.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color); }
Image kinImage = new Image(e.ImageFrame.ToBitmap());
Image grayframe = nextFrame.Convert();
grayframe.DetectHaarCascade( haar, 1.4, 4, HAAR_DETECTION_TYPE.DO_CANNY_PRUNING, new Size(grayframe.Width / 8, grayframe.Height / 8) )[0];
foreach (var face in faces) { nextFrame.Draw(face.rect, new Bgr(0, double.MaxValue, 0), 3); }
imageBox1.Image = nextFrame;
void kinect_VideoFrameReady(object sender, ImageFrameReadyEventArgs e) { Image kinImage = new Image(e.ImageFrame.ToBitmap()); using (Image; nextFrame = kinImage) { if (nextFrame != null) { // there's only one channel (greyscale), hence the zero index Image grayframe = nextFrame.Convert(); var faces = grayframe.DetectHaarCascade( haar, 1.4, 4, HAAR_DETECTION_TYPE.DO_CANNY_PRUNING, new Size(grayframe.Width / 14, grayframe.Height / 14) )[0]; foreach (var face in faces) { nextFrame.Draw(face.rect, new Bgr(0, double.MaxValue, 0), 3); } imageBox1.Image = nextFrame; } } }
with good lightning, the detection occurs flawlessly
Kinect SDK + EmguCV
I guess that it's better to start with the basics
Today I will start creating some controls to play with kinect. The first one will be a Login Control and I thought it will be great if I create a face recognition for use, and speech recognition to insert the password.
Later on I will post some code and the control code be aware 🙂
Power Supply Adapter has arrived
Almost miss the carrier delivery but I managed to get the power supply today so it's time to start downloading the sdk and development IDE
Requirements
As I believe, if you're reading this blog probably you are interested in develop over kinect or you're already doing it.
Still Waiting
Right now I am anxiously waiting for the Kinect Power Supply Adapter to arrive and put the hand on...