the surface that recognizes what you put to unite physical and digital world

Microsoft wants to blur the boundaries between the real and virtual world. They want to do it with “Project Zanzibar”, a kind of Roll-up mat capable of detecting everything we do on it and send that information to a computer, something that they believe can be useful both for gaming and in the educational sector.

This is a project that is being developed by Microsoft Research, and whose initial concept will be presented in detail at the end of the month. But due to a series of leaks, Microsoft has unveiled its first prototype and published a web page in which they give the first details about what can be done with this kind of multisensor peripheral.

The project is being carried out by a team of Microsoft researchers spread over Cambridge and Redmond. It is, as we have said, a mat that combines capacitive sensing and Near Field Communication (NFC), allowing multi-touch input and floating scroll gestures to coexist with manipulation and control of physical objects.

the surface that recognizes what you put to unite physical and digital world

In other words, your sensors can detect the position and orientation of the objects that we put on it, and also identify them. In addition, the mat can also be used as a kind of trackpad, since it will record our movements on it or those we make on it without actually touching it. All this can be viewed on computers and tablets by connecting it to them and using specific applications.

“We started with a simple thought: what if we could blur the division between the physical and digital worlds?” We can read on the blog in which they explain the basic operation of the device. “What if you could play with toys, cards, and blocks while watching your actions come to life on screen?”

All the processes of detection and processing of objects occur on the carpet itself, since on one of its sides it has a kind of small CPU where we assume that some type of processor will go, although Microsoft has not given details at the moment. Then, we can connect this with computers, tablets or other devices via USB or Bluetooth to carry out interactions with objects.

Ok, but what can we do with it?

In addition to the description of the device, Microsoft has also published a video talking about its main features, and in which they show some examples of how they think we will be able to use it. They are mainly examples of how to use your “Project Zanzibar” for play or educational purposes.

In a first example, they put a series of PlayMobil clicks on the carpet, and we can see how it depends on what we do with them, the computer next to it emits sounds to give our stories more realism. They don’t specify it, but presumably this requires a specific program. In addition, each doll has a special base to be better detected when we place them on the mat.

the surface that recognizes what you put to unite physical and digital world

Beyond the sounds, Microsoft also imagines that we can introduce another figure in the shape of a video camera to this game. In this case, on the screen of our laptop we would see our toys from the angle of the camera itself, and even by introducing backgrounds and scenarios that are not in reality. We could literally make our own movies.

They also use a card game as an example. The “Project Zanzibar” would be able to detect what specific letter are we putting on him, and there would be a game or application in which the interactions with the cards have a real effect. In addition, they also show a mix between dolls and cards to play slightly more complex games online.

the surface that recognizes what you put to unite physical and digital world

Microsoft also views its mousepad as an educational method. Think, for example, of putting an object like a car on top of the mat so that it recognizes it, and then letting your child put letters with letters on it to spell it out. It also proposes other types of educational games in which there are also letters with letters that we have to place in the proper order viewing the result on the computer screen.

They even contemplate being able to use it for the classics block programming “games” for the little ones. If through a computer program we connect to a robot, then we could use “Project Zanzibar” to use it as a peripheral and place blocks with orders on it in the order we want, so that the mat reads them and transmits them to the robot using the computer.

Don’t expect to see it in stores yet

Except for a major surprise at the end of the month, we can say that we are facing a project in development phase by Microsoft. Possibly when they present it they will show us their prototype and tell us about its possibilities, but we will hardly see a final product ready to be marketed yet.

Admittedly, this Project Zanzibar has quite a few possibilities, although whether it remains a rarity for fans or an accessory for everyone will largely depend on the partnerships they achieve with third parties to implement this technology. We will have to keep track of it, since it looks like a new piece in the convergence puzzle from Microsoft, in this case with a kind of mixed reality between tangible and virtual.