Microsoft Surface Programming – Object Recognition

Microsoft Surface Technology is supporting recognizing physical objects through the surface and reacting according to the object placed on the surface. I was so curios to know about the concept behind this object recognizing with this Microsoft Surface enable applications.

With the things I leaned from Colombo Code Camp and after having searched on the internet, found few important key points attached with technology. First thing is, MS Surface is recognizing object as a Tag, which has a unique value, which is retrieved from Surface’s Vision system. Means for MS Surface Vision System, physical object is always a Tag, which always has to be pasted on the object, as surface can identify it properly.

Tags are looks like this,

Tag always has value and each value is having different dot structured as showed in the picture. After having this tag printed in a piece paper, you have to paste it on the object that you are going to place on the Surface table (or any other device which is supporting MS Surface Technology). Then that object has the value that the pasted tag has, in the surface application.

Then the next curious point is that how we can generate these tags for Surface Applications. Answer is, we have two ways obtaining these tags,

1. Generate Tag with Tag Printing Tool
2. 2. Tags from Microsoft.

Please refer this link for more information - http://msdn.microsoft.com/en-us/library/ee804762(v=surface.10).aspx

Once you have the tag, you can paste it on any object, then that object should be identified by a MS Surface application by the Tag which is pasted on it.

Input Visualizer and Input Simulator

There is a tool which has been delivered with Windows 7 called “Input Visualizer” and this tool can be used to show some information about the tags (objects) which are placed on surface applications. To open it - All Programs | Microsoft Surface 2.0 SDK | Tools | Input Visualizer.

And for this we need to open one more tool, Input Simulator, which I explained in my previous post in MS Surface Scatter View.

Input Visualizer	Input Simulator

How we can identify the Tags programmatically

Just start get things going, open Visual Studio 2010 and create a MS Surface Project called “MyObjectRecognizionApp”,

Without writing any line of codes, you can start place objects on surface application and see how their tag information is looks like,

Just click on the middle box of second row of tools in Input Simulator, then click upon the application surface, then you will see this kind of picture in your screen. All this you have to do because of we are testing these surface application features with our monitors which are not supporting the touch. If you seriously check the value in the detail view, you will find it like, 0x0000000000000100 and double check it with the values that I have mentioned in Tag Value field in Input Simulator.

This is how you can test surface application for object recognition with the Input Simulator, provided with Windows 7 Operating System. Before clicking on the Surface application, you can mention tag values (Tag Series and Tag Value) and those values you can use in your app code for responds for virtual objects that you test with input simulator.

So our next step is to recognize an object (In our case it is virtual object which we make with input simulator) place on the surface application and do something to respond for it. For this we need to know about two most important XAML Elements,

1. TagVisualizer – This will display the information about the tag when it is placed in the Surface. The TagVisualizer works with a known set of tags. This known set is defined in the TagVisualizer.Definitions collection.

2. TagVisualization – This is the control type which holds all the content, after particular object is placed on the surface application.

Here what I’m going to show is, I will have two virtual objects which will have tag values 100 and 200 respectively. Once first object is place on the Surface Application, system will show “My Tag1” text and for the second object it will show “My Tag 2” text.

In my project I have default window created, named “SurfaceWindow1.xaml” and I will go with it without create any new windows. This is how it’s look like SurfaceWindow1 XAML code,

<s:TagVisualizer x:Name="TagVisualizer1">

<s:TagVisualizer.Definitions>

<s:TagVisualizationDefinition

Series="0"

Value="100"

Source="MyTag1Visualization.xaml"

LostTagTimeout="2000"

OrientationOffsetFromTag="0"

PhysicalCenterOffsetFromTag="0,-1.5"

UsesTagOrientation="True"

TagRemovedBehavior="Fade" />

<s:TagVisualizationDefinition

Series="0"

Value="200"

Source="MyTag2Visualization.xaml"

LostTagTimeout="2000"

OrientationOffsetFromTag="0"

PhysicalCenterOffsetFromTag="0,-1.5"

UsesTagOrientation="True"

TagRemovedBehavior="Fade" />

s:TagVisualizer.Definitions>

s:TagVisualizer>

Grid>

And I have two Tag Visualizations created,

MyTag1Visualization.xaml

<s:TagVisualization x:Class="MySurfaceObjectRecognizionApp.MyTag2Visualization"

xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

xmlns:s="http://schemas.microsoft.com/surface/2008"

Loaded="MyTag2Visualization_Loaded">

<Grid>

<TextBlock Margin="31,62,0,0" FontSize="20" Height="26" Width="206">My Tag 1TextBlock>

Grid>

s:TagVisualization>

MyTag2Visualization.xaml

<s:TagVisualization x:Class="MySurfaceObjectRecognizionApp.MyTag2Visualization"

xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

xmlns:s="http://schemas.microsoft.com/surface/2008"

Loaded="MyTag2Visualization_Loaded">

<Grid>

<TextBlock Margin="12,53,0,0" FontSize="20" Height="26" Width="206">My Tag 2TextBlock>

Grid>

s:TagVisualization>

Now it’s time to run the application and see how output is look like,

By clicking left and right mouse buttons at once, you can place objects permanently in the surface, otherwise once you release the mouse button, the virtual object will disappear. This tip is only for testing Surface applications with Input Simulator, but with real MS Surface supported devices, you don’t need to worry about this and can place objects as you expected. In real, what exactly happens is, once you put an object with pasted tag value, relevant tag visualization control content will appear on the surface, as response to the object identification.

To get this done, I didn’t write any line of code, but all the time it’s XMAL elements. You can use coding to make things smarter and useful in real application developments for MS Surface devices.

Lakmal Kankanamge.