Perceptual Computing

Perceptual computing is a broad technology category that encapsulates a number of techniques that strive to mimic the sensory, or perceptual processing that we perform when we hear, see, or touch.

Originating decades ago as “fuzzy logic” and “neural nets”, today’s perceptual computing devices employ advanced algorithms such as Gaussian Mixed Mode models for voice recognition, as well as Support Vector Machines and Bayesian network classifiers for image interpretation.

A valuable product of image processing research was the solution to the problem of extracting 3D point clouds, or 3D image data, using either stereoscopic cameras or laser time-of-flight methods. Extracting 3D information from a scene allows a 3D model to be reconstructed in software which is the basis of applications ranging from virtual-reality user interfaces to self-driving cars.

The earliest major consumer product employing 3D reconstruction was the Kinect option for the XBox™. The introduction of the XBox™ was followed by a multitude of products such as smartphones, PCs, and various flavors of surveillance systems. An excellent example of a modern 3D reconstruction product is Intel’s RealSense which employs input from two cameras to build a 3D model of the environment.

Some advanced 3D reconstruction techniques don’t use the optical spectrum at all but rather rely on imaging a scene using millimeter-wave radar and then post-processing with machine learning algorithms. A recent demonstration by Google of their Soli technology is an excellent example of the application of this technique to build an extremely compact and powerful 3D mouse.

Voice processing has made great strides as well. Although the computational loads are not as severe as image interpretation or 3D reconstruction, there is a large market for pre-processors for the burgeoning consumer market of voice personal assistants. Siri and Alexa are the most prominent recent examples of this trend.

Esencia has designed and fabricated several generations of Perceptual Computing ICs in partnership with several major semiconductor companies. Several of these ICs are in high-volume production. Typically we work with the companies to instantiate their core IP in a platform with the necessary digital signal pre-processing and noise reduction, add micro-controller and interfaces.