August 2010

You are currently browsing the monthly archive for August 2010.

This post is about a small recipe to perform face detection using Nokia N900 phone. It’s based on ROS and OpenCV and shows how these components are mixed together and configure. See this OpenCV wiki page about face detection to understand how it works behind the scene.

First of all, ROS needs to be installed on N900. I’ve built several ROS packages, including latest offical release, code name “C turtle”. I assume you know how to ssh to your N900 and gain root access.

Disclaimer

Careful while tinkering with your N900, you may “brick” it (if you don’t know what I mean, close this page). And I wouldn’t be responsible if this would occur, or if anything would turn into something bad. You’ve been warned.

Setting up ros-n900 source

I’ve created a Google Code project dedicated to N900 ports and package developments. This project can be reached at http://code.google.com/p/ros-n900/. In download section, you’ll find deb packages. You directly download them to your N900, or configure a new APT source pointing to this project:

$ echo "deb http://ros-n900.googlecode.com/files /" >> /etc/apt/source.list
$ apt-get update

Now install ROS with apt-get:

$ apt-get install ros-cturtle-base

This will install ROS on /opt partition (usually 2GB ext3 space), leaving rootfs untouched. ROS uses ~500MB to install. You can also install it on a (fast) SD-card, formatted using ext3 filesystem (don’t use FAT32). You’d then need to create a symlink /opt/ros-cturtle-base pointing to your SD-card.

Accessing N900 webcams under ROS

brown-ros-pkg project hosts gscam, a very nice ROS packages used to access camera with GStreamer. Since N900 webcams are recognized as V4L2 devices, it’s easy to setup a gstreamer pipeline. First install dependencies. On N900:

$ apt-get install gstreamer-tools

This example shows how to send videos to PC host using UDP. Device /dev/video0 is back camera (big one, high resolution), /dev/video1 is the front one (small resolution).

# On N900, assuming 192.168.2.14 is PC's IP address (usually true if following usbnet howto tutorial)
$ gst-launch v4l2src device=/dev/video1 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! ffmpegcolorspace ! smokeenc ! udpsink host=192.168.2.14 port=1234

# On PC computer:
$ gst-launch-0.10 udpsrc port=1234 ! smokedec ! autovideosink

Here we are! A small window appears broadcasting N900 videos. You may even see your face. Install some more dependencies to build ROS package. On N900:

$ apt-get install libgstreamer0.10-dev

Now go to ROS stacks directory.

$ roscd
$ cd ../stacks

Install gscam package. Follow instructions here, download gscam archive in download section or install it from sources.

$ svn co -r 682 http://brown-ros-pkg.googlecode.com/svn/trunk/unstable/gscam gscam

Once installed, build it using rosmake:

$ roscd gscam
$ rosmake -i

(requires quite a lot of time, be patient…)

gscam requires environment variable GSCAM_CONFIG to be set. It stores gstreamer pipeline definition. I had lots of troubles running the correct pipeline, and finally got help from ros-users list. The trick is to convert YUV format (the only N900 cams seem to output) into RGB.

$ export GSCAM_CONFIG="v4l2src device=/dev/video0 ! videoscale ! video/x-raw-yuv,width=320,height=240 ! ffmpegcolorspace ! video/x-raw-rgb ! identity name=ros ! fakesink"

You can manually check it’s working without ROS:

$ gst-launch-0.10 $GSCAM_CONFIG

If it pauses, that’s ok. Now run it using gscam node. It requires to run within “bin” directory.

$ roscd gscam
$ cd bin
$ rosrun gscam gscam

At this time, gscam should say it’s “processing…” (of course, a reachable roscore should run somewhere, for instance on PC host). Now back on PC, install n900-cam package.

$ roscd
$ cd ../stacks
$ svn co http://ros-n900.googlecode.com/svn/trunk/src/n900_cam
$ roscd n900_cam
$ rosmake -i

Now run n900-cam testcam.py node. It’ll retrieve images from image_topic subscription, add a circle and display the result in a window (from ROS tutorials).

$ rosrun n900-cam testcam.py

So far so good. Let’s face detect ourself ! This is closed to previous example, except now images are submitted to OpenCV for face detection. Code is coming from OpenCV samples and is glued here to work with a ROS node.

$ rosrun n900-cam facedetect.py

If you can see yourself with a red square around your face, that’s good news. If not, either you’re not human, either something is wrong with running configuration…

 

I’ve recently bought a Nokia N900 smartphone. It’s described as an Internet Tablet with phone capabilities. Interesting thing is 100% linux based, you can have full root access. On paper, this phone is awesome, in reality, it suffers from a lot of half-baked applications, poorly maintained software but, still, this opens to lots of tinkering…
On the other side, I’ve also discovered ROS. ROS stands for Robotic Operating System. It’s robotic framework, offers distributed computing over nodes, pubsub architecture for inter-processes message exchanges. It can be programmed using C++, python and other less supported language. It’s developped by Willow Garage, the guys who built robot PR2. If you’ve ever searched a flexible, powerful and fun robotic framework, or even wanted to develop your own (…), you definitely need to give ROS a try.
There are lots of advantages running a PC based robot. For instance you can easily plug a USB webcam and give vision to your robot. For minimal cost. Doing this with an embedded cam, like CMUCam, is certainly fun and interesting but in the end, performances can’t be compared and you’ll sure need some power computing to process incoming images. There are existing tiny PC, based on ITX motherboards for instance in order to do this. You can install Linux, put ROS on it and start to build your Linux powered robot. But, wait, I also have a very, very small form factor Linux PC, my N900… Why not using it as a robotic platform ?
It provides:

  • 2 webcams (front, back)
  • 3-axis accelerometer
  • GPS
  • high resolution touchscreen
  • micro-USB connector, can be used as a USB host with some tinkering
  • Wifi
  • bluetooth
  • Infra-red beam
  • RGB LED
  • 32GB memory, extendable to 64GB with microSD cards
  • microphone
  • speaker
  • ambient light sensor

Doesn’t it sound awesome as your main robotic platform ?

The idea is thus to install ROS on N900. Low-level tasks, such as actually activate motors, collecting sensor data, should remain on a microcontroller board, like Jaluino. All collected data and actions should go through N900, acting as a hub, performing some pre-processing tasks before delegating more power-consuming tasks to a PC around there, also running ROS.
It’s been a while since I’ve already install ROS on N900. There were lots of trials and errors, highly time consuming, but it definitely worth it! I’ve created a dedicated Google Code project, named ros-n900. You’ll find ROS packages specific to N900 target, and deb packages to easily install ROS on N900. You can also follow instructions on this wiki page I wrote on ros.org: http://www.ros.org/wiki/N900ROS.
Next, we’ll see how to have fun with N900 webcams, ROS and OpenCV!