Building an Embodied Social Proxy or Crazy Webcam Remote Cart Thing
As a remote worker I'm always looking to find better ways to work with the team that don't involve actually going to Seattle. As much as I love/hate Microsoft, I'm not moving. However, this always puts me at a disadvantage. There's only so many places (and only so far) i can go as a remote worker.
When Phil and I made the "HanselPortal," essentially a persistent always-on Skype Chat, that gave folks a place to drop by and chat (at least those two lazy to just call me via Office Communicator) but there's already two people in Phil's office, I can't move (I'm a desktop computer) and my field of view is very small. Phil has a small inside office and I can't exactly hold meetings. It's a start, but it's not perfect.
I stumbled upon the Embodied Social Proxies project at Microsoft Research and got a tour from their team. While their system costs about $3000 to $4000 (While I'd love an Axis 213 PTZ camera I can't afford the $1300 and I'm still trying to convince the boss.) and has a great cart, monitor and camera, I figured I could at least do a proof of concept on the cheap. And by proof of concept, I mean, in the same way my 1963 Ford Falcon was a proof of concept; I kept driving it until it stops, then started walking.
The Idea
While the name "Embodied Social Proxy" is very PHDy and very Researchy, the idea is that you want a physical stand-in for yourself. Sure, people can call me and contact me in any of a dozen ways and honestly, it's one click at Microsoft. I'm in the address list, I'm on chat, and I've even got a 5 digit phone extension. I've done all this and more to integrate with headquarters. However, out of sight, out of mind is really true. People subconsciously or not like to associate something physical with you. So, until I can get a 6 foot tall mobile video conferencing robot a $20 cart with a laptop super-glued to it might help. I'll set it up in an office in Building 5 so folks can "stop by." I can be "brought" to meetings where perhaps setting up a RoundTable isn't feasible. Maybe it'll work, perhaps not. But I'm going to continue to try.
The Problems
The researches have done many iterations and discovered many things. You want "eye gaze" to be natural so you can tell when folks are looking at you and they can tell when you're looking at them. They recognize that you need to "look around" and watch whiteboards, so being able to pan and zoom around is important.
Security is important also. We don't want a domain-logged-in real user on a mobile cart wandering around Microsoft. However, as a remote person I need to be able to start up the camera without having to ask a person to do it. I need to control as much of the cart as possible remotely.
The General Solution
I've got Office Communicator "corporate chat" on my side, so that makes things easier. If you wanted to generalize this solution you could use Skype or other things. I also have Exchange so my free/busy details are available programmatically. Again, this could be Google Calendar, or whatever.
I'll have a laptop mounted (not movable) to a cart. It'll include an HD Webcam at a fixed position (no zoom or pan) and a second IP camera that allows Pan/Tilt/Zoom (PTZ). This should let me look around a bit. This is an IP camera because there are no good pan/tilt/zoom USB cameras on the market. PTZ cameras can be $200 or up to $3000 and more. Mine's $200.
I need the laptop to have both wired and wireless. Here's where things get a little weird. The camera will have one IP and the laptop will have another. However, the laptop will have a security certificate that will allow it to get on the network while the camera will not only not be reliable, but it'll be hard to find from a DNS perspective. I would like to be able to access both the camera and the machine as a unit, reliably. I solve that below, in what I think is a pretty clever way.
The (Cheap) Hardware
The hardware for my build is beyond cheesy cheap.
- Rolling Cart from Office Depot: Height-Adjustable Mobile Laptop Cart, 33 1/16"H x 23 5/8"W x 15 3/4"D, Black/Silver - On sale for $20
- APC UPS from Amazon: APC BE750G 10-Outlet Power-Saving Uninterruptible Power Supply - Yes, I realize we're talking about a laptop on a UPS, but this'll give me 4-6 more hours of power if I get stuck, plus it'll power the secondary IP PTZ camera and the Hub
- IP Camera: Panasonic BL-C111A Network Camera Wired by Panasonic
- WebCam: Microsoft LifeCam Cinema HD by Microsoft - HD or not (perhaps you don't have the CPU or bandwidth, this is a crisp, clear camera.
- Laptop: I used a crappy Dell Latitude D820 I had lying around. Decent, but not awesome.
- Hub: Netgear FS105NA Switch 5Port Metal by Netgear
- Zip Cable Ties: Buttloads of them. Deluxe Cable Tie Kit - 1,000 Pieces.
- A weight and pipe from a $14 Steadycam to give my IP Camera some height.
- 3M Fasteners, I used like 12-16 strips to hold down the laptop and cameras. 3m Company 17202 "Command" Small Interlocking Fasteners & Strips
The Software
There's a custom WPF-based .NET client written by the MS Research Team. I'm gonna continue to bug them to let me release the code or Open Source it or something. I don't think the software is the important part, it's the idea. The software is a little tricky and cobbled, but workable.
Talking to Office Communicator
The machine is running on a local account that isn't Administrator. It's set to log in automatically, but the account has access to nothing. There's another Non-User Domain Account called "hanselcart" but while this account is in the Active Directory, but also has no access to resources or anything. That account has had Office Communicator access turn on so I can chat with it. Communicator is set to start up automatically as well. It's that ability to chat (really "address") the cart that makes it reasonably seamless.
You'll see that if I'm not currently visible on the screen, the program makes a call to Exchange Web Services and uses the Office Communicator Presence information to show where I am and what I'm doing generally, as well as a way to call me.
There's an Office Communicator SDK so you can automate and listen to Communicator:
this.messenger = new Messenger();
this.messenger.OnSignin += new DMessengerEvents_OnSigninEventHandler(messenger_OnSignin);
this.messenger.OnContactStatusChange += new DMessengerEvents_OnContactStatusChangeEventHandler(messenger_OnContactStatusChange);
this.messenger.OnIMWindowCreated += new DMessengerEvents_OnIMWindowCreatedEventHandler(messenger_OnIMWindowCreated);
this.messenger.OnIMWindowDestroyed += new DMessengerEvents_OnIMWindowDestroyedEventHandler(messenger_OnIMWindowDestroyed);
this.messenger.OnMyStatusChange += new DMessengerEvents_OnMyStatusChangeEventHandler(messenger_OnMyStatusChange);
this.messenger.AutoSignin();
The code is very COM-ish, which is unfortunate, but it's easy enough to write to. If I chat "Start Video" to the cart, the program sees it and calls me back:
void StartVideo()
{
IMessengerAdvanced advanced = (IMessengerAdvanced) this.messenger;
object result = advanced.StartConversation(CONVERSATION_TYPE.CONVERSATION_TYPE_VIDEO, this.contact, null, null, "1", null);
// Float a "hang up" window on top
if(hangupWindow == null) {
hangupWindow = new HangupWindow(MachineName);
hangupWindow.Closed += new EventHandler(hangupWindow_Closed);
hangupWindow.Owner = this;
// put the hang up window top-right corner
hangupWindow.Top = 0;
hangupWindow.Left = System.Windows.SystemParameters.PrimaryScreenWidth - hangupWindow.Width;
}
// Set the video to full-screen
maxVideoTimer.Start();
hangupWindow.Show();
}
The FullScreen part is a horrific SendKeys hack to press F5 for me until I talk to the OC team and figure out how to do this programmatically.
How to Address the IP Camera - "It's a WebFarm of WebCams with IIS's AAR"
I'm rather proud of this hack. Let's say the cart is called http://hanselcart. I'd like to be able to address the WebCam reliably even though it's not technically a managed-object on the MS network. Essentially, if the cart itself is on and online, I'd like to use the cart to get to the camera.
Then I remember that IIS (Internet Information Services, our Web Server) has an insanely powerful free module you can download called Application Request Routing or ARR. Regardless of what you think of the name IIS ARR, it really takes IIS to the next level. Think about it next time you are trying to do something tricky with IIS. It adds a bunch of features, but I figured I needed a Reverse Proxy, which ARR can do.
Instead, I made my local IIS a WebFarm with one node, the WebCam. ;) I put the Webcam on a local network "behind" the laptop (192.168.x.x) and told my Laptop's instance of IIS to route all traffic to the Webcam. In fact, ARR smartly did it for me when i used the Add WebFarm Wizard. Whole thing took 5 minutes, counting download time.
NOTE: I did also try a tool called WebcamXP, and while it's amazingly cool for other things and has some really diverse features around Webcams, especially IP webcams, it used about 30% CPU for what I wanted, while IIS's ARR used 1-2%.
So now, I can just hit the cart by name and I'll get the IP Webcam. This Webcam has its own HTTP server and includes a quasi-RESTful API for Pan/Tilt/Zoom so I can video chat via Office Communicator and look around with the web interface of the IP Cam.
Here's a few shots of the cart itself. I'll be driving it up to Seattle for testing and installation this week.
Thanks to Gina and John and especially Peter Provost for helping me (virtually from Dallas) build it. Wish me luck!
Related Links
- VIDEO: Hanselminutes on 9: Embodied Social Proxies (and Remote Video Heads!) with Microsoft Research
- YOUTUBE VIDEO: Embodied Social Proxy: Connecting Hub-and-Satellite Teams
- Virtual Camaraderie - A Persistent Video "Portal" for the Remote Worker
- Microsoft Research PDF - Can We Make "Distance Matter" Less?
- Steve Easterbrook - Blog post on Social Proxies
- Remote Meetings: Thinking Inside the Box
- Video: Polycom CX5000 - Microsoft Roundtable
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
1) Why the cart? Seems like the proxy could get around easier, go more places, go to lunch with people if the unit were hand carry-able. I realize this is harder with oth-the-shelf/budget parts, but it seems like there's a Netbook/Windows Mobile/iPhone app for that that would be, um, handy.
2) How do you know whether someone's decorated your proxy to reflect your personality, or their view of your personality? Afraid I'm not the only person who first thought of 'kick me' jokes'. :)
You are missing on thing though. You need a cattle prod. So that you can zap people. Then you can chase people down the halls screaming "AH YAI YAI YAI YAI!"
http://images1.fanpop.com/images/quiz/27729_1215375020961_500_278.jpg
Great for Chang. Bad for everyone else.
You have an office... Come to work or we are giving it to someone else.
Did you consider using a Logitech Quickcam Orbit AF? Is does have the ability to pan, tilt and zoom and also can be controlled
programmatically -- http://www.quickcamteam.net/documentation/how-to/how-to-control-pan-tilt-zoom-on-logitech-cameras
I assume that you could use a cross-over cable from the laptop to the IP based PTZ camera along with some static IPs.
If you wanted to connect via wired network this would require adding a second NIC to the laptop (but I'll assume that you have several old PC Card NICs laying about your geeek layer).
Since you are adding the extra components to the cart, I assume that I am missing something...
Great project, and timely too since I'm working at home today.
Yes, the Office Communicator SDK is a bit COM-y (well, it's all COM), but there is a new managed API coming out with the new version of Office Communications Server and Office Communicator.
Details from the PDC09 here and here.
-Chris
HanselNet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, Febuary 29th.
In a panic, they try to pull the plug.
Comments are closed.
Great article/idea, can't wait to see this action!