Apple Glass, Apple’s future augmented reality glasses, represent a major paradigm shift in many ways and the interaction with them is one of the main ones. Options like touch the rods, move your hand in front of us or use Siri These are ways of navigating content and we’re adding the one that Apple just patented: turning any surface into a touchscreen.


Touch by heat, response to tactile surfaces

Tactile

Entitled “Method and apparatus for detecting contact between a first object and a second object”, this patent filed in 2016 specifies that the devices, we assume the Apple Glass, will use a system of overlay controls on real world objects.

  MásMóvil, Spain's fourth operator, receives € 5 billion takeover bid

A natural way for humans to interact with (real) objects is to touch them with their hands. [Las pantallas] They make it possible to detect and locate keys on its surface are commonly called touch screens and are now part of, for example, smartphones and tablets.

However, a current trend is for RA screens to become smaller and / or closer to the retina of the user’s eye. This is, for example, the case with head screens, and makes the use of touch screens difficult or even impractical.

Before a virtual object that an AR device can show us, the most common way to interact with it is by using controllers, gloves or sensors which, by means of a cable, indicate the device of our movements. But as the patent says “the limitation of this type of approach is that it requires modifications of the object or of the human body”.

Thanks to virtual object occlusion, which we already have in ARKit and which we can appreciate in the iPad Pro (2020) with a LiDAR sensor, the glasses could detect where we are playing. However, the system is imprecise. for example on a calculator keyboard, you cannot touch button 5 without going through 9 or 8 first make the system believe that we are touching them.


Calculator

The system detects when we run our finger in front of the button, without being able to perceive whether we have touched it. Apple’s proposal before such a challenge is the heat. With two objects at different temperatures the area where they touch will change temperature and slowly recover the pre-touch temperature. Thus, the pixels corresponding to the point where the contact occurred are clearly marked when measuring the increase or decrease in temperature. A system where the temperature measurement can take place after touching, when we lift our finger from the surface.

This system can distinguish touches or occlusions caused by a human body (occurred on purpose) from touches or occlusions by something else (which could happen by accident). It is able to accurately locate a touch and can identify touch events after they have occurred (that is, even if no camera, computer, or the like was present at the time of the touch).

Ulysses, markdown writing goes to the next level

It is clear that building augmented reality glasses like Apple Glass involves overcoming many technical challenges. In your presentation we will see something as simple as glasses (we assume) but, as patents like this show, the amount of technology that will be inside and behind them will be more than impressive.