Category: Projects


Robot Drift

We all know that robots will be taking all of our jobs sooner or later, just before they rise up against us. They still have quite a ways to go however. In the Pilgrimage Project we took advantage of the widespread human desire to gawk at robots doing things. Danielle Storbeck, Chris Miller and I put together a robot tour guide unlike any I had ever seen.


The guts of it (not pictured) were pretty basic. It’s a platform robot with an arduino mega for the main logic board, two controllers for the front and rear wheels and a macbook air handling the voice synthesizer and timing. The robot is a basic line-follower, and at first we took it very seriously. We wanted to make the best robot tour guide anyone had ever seen. One day though, late in the project cycle, I totally messed it up by trying to attach some 12v fans to the back of it. I created a ground loop that fried one of the line sensors and the thing was never the same again. It would drift off the line and never find its way back. After freaking out we starting to talk about how funny it would be if the robot didn’t just drift off the line, but also in its narrative. What if it started getting off-topic during the tour? Giving bad information and getting confused? What if there was cognitive drift as well as physical drift?


The final project that made it into the exhibition was just that, a robot with it’s wires crossed. The physical form of the robot looks haphazard and shoddily made, it’s clearly a prototype, complete with a color-changing plastic dome right out of Buck Rodgers era scifi. The narrative snippets we made for the tour start out plausibly enough—the robot drives around the line, stops to look at the tour group and tells them some fascinating facts. As it goes on, it loses the line and needs to be wrangled by its handler to get back on track. Each time this happens the narrative drifts a little more. First it gets some facts believably wrong (kind of like wikipedia), then it states obvious lies, then it just totally loses the plot. It was fun to make and fun to watch people figure out that they shouldn’t take robots so seriously. Not yet at least.


Hopefully I’ll be long gone before our new robot overlords happen upon this project.

Tangible Computing

My first experience with Tangible Computing came in Ken Perlin’s Physical Media class at NYU. We built a cool tabletop game called Krakatoa which was a resource management game that used these “pucks” which were marked with 1,2,3,or 4 dots. If I remember correctly, each of the four players would slide their puck over to a command menu, then slide the puck to a place on the board and their character would go do whatever the command was. I think it was things like “cut wood” or “steal wood” or “burn wood” as you can guess wood was the primary resource for this game. I was still a beginning programmer at the time so I didn’t tackle any of the heavy stuff, but I did do all the audio recording and management and I made the pucks which were painted clay. They felt really nice in your hand when you slid them. Nowadays I would probably use a laser cutter to make them, but back then clay was best and I think it might still be the best.

Working in AR is very much like working on Tabletop Computing, but there is something about the mediation of a device that makes it less engaging. It also has a lot in common with physical computing but the fact that inert objects are used instead of interactive ones also sort of changes things as well. So, when I got to Georgetown I was pretty excited that one of my colleagues knew about one of the old Surface (now pixelsense) tables that I could have. These are the big table kind, not the Sur40, and they were/are way better in a lot of ways. I remember Jeff Han working on the original design for these, again back at NYU, right before he gave his TED talk showing it off (that was my laptop he used for that talk btw–closest I’ll probably ever get to the TED stage). Anyway, I decided to run a little experiment by teaching courses in both Tangible and Physical Computing in the Fall and Spring of my second year to see which one worked better. Turns out physical computing was way better, and that class morphed into my Interaction Design class. I still use the Surface on occasion. I had a student use it ion the Pilgrimage Project and I made an app for the CCT lounge that introduces visitors to our faculty and staff.


The one major finding from that study was how totally useless the written tutorials were in both of those scenarios. Beyond useless actually, dangerous. It’s not that they were bad, they were great. The arduino tutorial on the their website are succinct and simple, and the ones that we made for the Tangibles class were terrific, all the students said so. The problem was that students had basically zero retention from the tutorials, were totally unable to connect them together or to see patterns, and they gave the false impression that the students could do things they simply were not sophisticated enough to do. Given how many tutorials and how much people rely on them in online courses and lots of other venues I suspect this is a going to be a real problem for online courses to overcome in the future.

Diego’s Birthday

My kids are not fans of Dora the Explorer. It’s a quality show, no doubt, but they just aren’t into it. I know a lot of kids are though. When I first arrived at Georgetown I met a Psychology professor named Sandra Calvert. She does a lot of work on the psychology of children’s media consumption, and we worked together on an NSF project to find out how children’s familiarity with a character can influence their learning. Presumably, engagement and learning are correlated, and having a pre-existing “parasocial relationship” with a character has been shown to increase learning in some circumstances. At the time my daughters were two and four and so I was personally interested in learning more about their learning (I try to be a good parent sometimes).


Having just arrived, I didn’t have a whole lot of resources at my disposal, but I managed to find a great student, Stevie Chancellor (already way more famous than I), and we cobbled together a game in XNA to test whether the characterization affects student’s ability to learn “subitization” the ability to group numbers into collections instead of counting individually. The major finding here is that voice seems to matter more than character, or at least it’s a major part of the equation. Film and video producers are always fond of saying things like “audio is half of the presentation but gets about 10% of the attention” and its seems like when learning is concerned that’s also a big part of the story.

What I learned most from this was how truly desperate psychologists are for better experimental tools. When I was doing some of my more recent work on multiscale MR I kept coming across all of these psych studies that used VR platforms to test different mental abilities like navigation or spatial awareness. Not only were these extremely rudimentary, but they seemed to make all kinds of assumptions about how similar cognitive functions are between VR and the physical world. I’ve never seem anything that convinces me that putting someone at a desk with a VR helmet is a good analog for physical experience. The resolution is lower, the haptics are nonexistent, proprioception is confused, the list goes on. There are a few FMRI studies I’ve seen that attempt to bridge the gap by showing that spatial awareness in the physical world lights up the same areas as it does in the virtual one, but the results show similarity at best and its the differences that really matter here if we’re trying to generalize from VR out to the physical world. It really comes down to the fact that we need much better and deeper collaboration between these disciplines. Happy to help if anybody is interested….

Multiscale MR and GTtour

I first discovered the notion of multiscale methods in one of NECSI’s Complex Systems summer courses as a grad student. Not surprisingly they were extremely “mathy” which is great, but I got to thinking about what the qualitative properties of these methods might be. Then about a year or so later Mary Hegarty gave a talk at Georgia Tech about her work in spatial scale and, although these are slightly different notions of scale, I began to see a path forward. One prominent stop on that path was my PhD dissertation work on spatial scale in Mixed Reality. The connection is a simple one, if we experience space at different scales, and we use partially overlapping cognitive abilities at those different scales, then how do these play out in the interactive environments of Mixed Reality?

One of the major breakthroughs I had was in developing the notion of scale transitions. Multiscale MR experiences need some clear way to move the user through information at different spatial scales. There are emerging techniques and best practices for doing this, just like in the early days of filmmaking, and these need to be catalogued, expanded upon, and critiqued, just like cuts in film. My dissertation explored these ideas in the context of GTtour a MR tablet experience that let users take a virtual tour of the Georgia Tech campus. The good folks at RNOC helped me put together the tour. I’m not sure if it’s still in use but nonetheless it was really my first foray into mulitscale methods.



I did a really nice study with Lisa Singh and Janet Mann looking at how some of the student researchers on Janet’s research team used Lisa’s visualization software (Invenio Workflow) to explore field data on dolphin social networks. There are only a few students in the research group so the n for this study is too small to draw any generalizable conclusions from, but we did learn a lot about difficult it is to connect observations in the field with raw data. For long term projects like this new people constantly come onto the team and a lot of the embodied knowledge that researchers have through years of field observations and data collection is lost when those researchers leave because it’s hard to transfer that kind of tacit knowledge to a new team member. It seems that visualization can help keep some of the continuity. The really interesting next step question I’ve been asking here is whether doing some work with visualization beforehand can inform student researchers observations in the field. Continuity goes both ways after all.

Check out our VAST poster below.