Webcam image capturing, projectors and Ants

Hi. I don't know if this is the right forum to use or not, so i'm sorry if it's in the wrong one.

I am doing a project that requires a webcam to be placed on a ceiling and it captures peoples movements within it's viewing area.
The position and size of the person will be worked out in a program like Flash, maybe. Or maybe use the new Touchless SDK.
So depending on the size/position of person, we want to project images of ants running around on the floor. If the person is deemed to be small then the ants will come towards that person. If big, then the ants will run away.

The question here is, Has anybody done this sort of thing before?
Any questions/comments welcome?

Thanks
Trev.

wow, Touchless SDK, wow, never heard of that before but I'm definately going to use it in my next project, I wanted to be able to track objects with a web cam but decided to leave that stage till later on when I had more time to work it out, looks like I can use that SDK to do this right out of the box.

I've been deciding between Flash and C# for writing my application in, C# seems a bit sparse with built in camera functionality but I found enough source on the net to throw something together for basic recording capabilities. Going the C# route means you're limited to Windows or the capabilities of the other OS's with their C# ports. It does have the benefit that you then have access to the full C# class library which means you also have access to tons of ready made advanced 3rd party libraries.

Flash has a fully implemented camera class but there's no way to record the data with just Flash. Adobe AIR doesn't help either since it won't let you call a 3rd party executable like FFMpeg, so you can't record to flv with that. You could use another wrapper like MDM Zinc to do that but then Zinc is inherently bug ridden. Saying that though; if you can avoid the bugs it will let you call an external app like FFMpeg so you can then record to flv format. Zinc lets you compile for PC, Mac and Linux so theoretically your app would be able to support those 3 systems with no changes required by you. The downside is that you have to buy a seperate license for each OS you want to compile for so that's like 3 x £250. You can run the free trial which is unlimited time wise but it has a splash screen and a watermark added to all your compiled apps. That wouldn't be recorded in your outputted video mind you.

I was playing with the idea of having a hybrid system where I'd use Flash to access the web cams and then output to a custom written C# wrapper which in turn would call FFMpeg to record the data as an FLV file.

Now you've swayed me onto the solidly C# path, the Touchless SDK just blew my mind with the possibilities.

Sorry, no advice, but wow, what a cool project!

Are you by any chance working in some kind of science museum/center for people to come and check out cool stuff? Kind of like Science World in Vancouver?

Hi.

ctardi: Yes it's for a museum in the town i live in. They want images of ants running around on the floor doing what ants do. It's to attract people into an area of the museum that does not get alot of attention from visitors. The idea is to have some form of interaction between the visitors and the ants.

Tann San: Thank you for your comments. Much appreciated. From what i read of your thoughts, are you saying that we would have to record/save the image file from the webcam? Was hoping to analyse it on the fly to get the position and size of the people.

Another problem we came across in testing the webcam was it's limited viewing area. When mounted on the ceiling which is 2.7m (9ft) high, the floor area it covers is approx 1.8m x 1.2m. We have a working area of 3m x 1.2m. We may need 2 webcams or can does anyone make a wide angle webcam?

I think that you will find zillions of them. Maybe not labeled "webcam".
Look for Security Cameras, tigerdirect.com has lots

Some of the ways I found people were recording with Flash and C# (independantly) was that they were capturing a series of still images from the camera, usually jpegs, then sticking them together into an FLV file with the aid of FFmpeg.

I ended up finding and settling with these links:

• Joris Timmermans FLVRecorder for Adobe AIR
• Webcam Digital Video Recorder for Apollo which the above was inspired by, well the original version.
• C# Look at me! Windows Image Acquisition. This uses Windows Image Aquisition to capture the info from the camera. For me this wouldn't work since my cameras don't seem to show up in the Scanners & Cameras panel but I thought it would still be a handy article to have for other areas.
• Mark Schmidt's C# WebCam User Control Source. This is probably the C# source I'd base mine on. It seems mega compact and very easy to tie in to a larger application.
• Edgar Cardeira's C# and FFmpeg. A simple example of getting C# to call the FFmpeg exe and render out an FLV file.

Between all of those you should be able to find enough source code to get you started with your chosen programming language. They each tend to go a bit further than you're after since they go into actually recording the video footage, you would just need to use that data differently and remove the recording section. As I said earlier in the thread, these are links I found to help me with my specific project but they should also be useful for yourself.

The beauty of using webcams is that they are very cheap. The ones I'm using are Arianet 5 Mega Pixel Webcams. They cost £11.50 each which is nothing. They have night vision lights built in and a software zoom although I've found no way of controlling that with any of the links I posted. There's also a manual zoom which entails turning the ring around the lens which is what I'm going to do and just have a fixed zoom level that suits each cameras position. When I looked into "proper" security cameras they were all £80 upwards. Guess it depends on your budget and what you're after. For me the £11.50 cameras are spot on.

Wouldn't the software zoom be just essentially cropping the image, then resample it larger?

Yep. Software zoom sucks, even with the best algorithms. It is possible to treat the images to get rid of the artifacts, but if you think about it, it isn't really possible to recreate information where no information exists. Some level of interpretation beyond the merely simple algorithmic approach becomes necessary.

oh, that's dissapointing. The crappy demo software that came with it has this face tracking thing which works, although it's very very slow to actually register a face. When it zooms in it's really clear so I thought it was doing something fancy, guess not now though.

After posting last night I re-read your last post, in my head because I was in the bath, and I realised that I'd been making things way more complicated then they needed to be. I need to record video which is my main purpose but you don't. You can just use Flash along with it's built in Camera class. I remembered seeing some motion tracking demo ages ago so I tried to dig it up, couldn't find that specific one but I did find loads of others including this one; Soulwires AS3 Webcam Motion Tracking. There's another demo here on the Adobe site.

As for the ants, I'd recommend using Bitmap Blitting as detailed here. It'll give you a much greater performance boost over having oodles of MovieClip ants running around. There are many many sites detailing their own methods of doing it, that one just seemed rather comprehensive and explained alot more of the theory.

fishtoprecords: Yes there are plenty of wide angle webcams out there.

Tann San: Thanks again for your input. Fantasic, especially the links to other websites.

Well luckily we don't need zooming, it's going to be fixed.

Another thought, we are not projecting the image of the person captured by the webcam back through the projector, only projecting the ants onto the floor around where the person is standing.

Another issue we came across while testing with a webcam was the amout of "real estate" the head and shoulders of the person takes up. Seeing as the webcam is only 2.7m off the floor, the taller a person is, the closer they are to the camera so they take up more of the viewing area. And a shorter person takes up less area. But it does give us a way of making the ants behave differently depending on the size of the person. We may have to make it so only one person is in the area at a time to make it work effectively??

Any other thoughts/ideas welcome.


Trev.

Well using the motion tracking examples I linker to; you could use the read in camera data for your ant position calculations and just not draw the actual camera feed to the screen.

Having multiple cameras may help you with your "big people in the way" issue although it will surely make your calculations alot more complicated. I'm in no way a math wizard so it already boggles my mind!

Personally I'd think it was better to not restrict people just so the system can be used, I thought the point was to attract more people to boring parts of the building.

There are zillions of webcams, most are built from camera systems designed for cell phones, they are small and cheap, image quality is way down the requirements list. I spent a fair amount of time trying to find specs on webcams, and even from brands like Logitech, there are very few actual specs, just cool marketing words. I was looking for hard specs on low-light ability and found nothing. I ended up replacing a $25 Logitech webcam with a $60 Logitech one, and the more expensive one is several F stops better in low light.

The security cameras are several times more expensive, but you can get information such as focal length, angle of view, etc. that can help the OP out with his coverage problems.

fishtoprecords: I gather webcams that use CCD's instead of a CMOS are superior as well?
Also, how do you work out an F-Stop on a webcam as that is not stated on any i have seen?


Thanks
Trev.

I've never see an F-stop published either. I guestimated it from my two webcams using the same light. It could have been a better sensor or more glass. At least the high end one claimed a Carl Zeis lens.
The low end model was unlabeled.

Well guys, I have found a company that is doing exactly what I want to acheive.

The program is called Living Surface. I am in the process of finding out the cost of the software program. Not that buying it is an option but just for interests sake.

http://www.vertigo-systems.de/startseite.html?&L=1#ul

Still working on our own version though.


Trev.