Here’s how your very own butler, right in the dashboard of your car, might respond to your needs, minimize distractions, keep you focused on your driving, and help you avoid potential hazards.

Telematics combines automakers, consumers, service industries. And computer companies. Which is how IBM fits in. It knows a thing or two about computers, data management, and connecting compute devices together. So people at its Thomas J. Watson Research Center (Hawthorne, NY) are working to bring the technology to a car or truck near you in the not-too-distant future.

Conversational telematics
IBM’s Artificial Passenger is like having a butler in your car—someone who looks after you, takes care of your every need, is bent on providing service, and has enough intelligence to anticipate your needs.

This voice-actuated telematics system helps you perform certain actions within your car hands-free: turn on the radio, switch stations, adjust HVAC, make a cell phone call, and more. It provides uniform access to devices and networked services in and outside your car. It reports car conditions and external hazards with minimal distraction. Plus, it helps you stay awake with some form of entertainment when it detects you’re getting drowsy.

In time, the Artificial Passenger technology will go beyond simple command-and-control. Interactivity will be key. So will natural sounding dialog. For starters, it won’t be repetitive (“Sorry your door is open, sorry your door is open . . .”). It will ask for corrections if it determines it misunderstood you. The amount of information it provides will be based on its “assessment of the driver’s cognitive load” (i.e., the situation). It can learn your habits, such as how you adjust your seat.

Parts of this technology are 12 to 18 months away from broad implementation.

Improving speech recognition
You’re driving at 70 mph, it’s raining hard, a truck is passing, the car radio is blasting, and the A/C is on. Such noisy environments are a challenge to speech recognition systems, including the Artificial Passenger.

IBM’s Audio Visual Speech Recognition (AVSR) cuts through the noise. It reads lips to augment speech recognition. Cameras focused on the driver’s mouth do the lip reading; IBM’s Embedded ViaVoice does the speech recognition. In places with moderate noise, where conventional speech recognition has a 1% error rate, the error rate of AVSR is less than 1%. In places roughly ten times noisier, speech recognition has about a 2% error rate; AVSR’s is still pretty good (1% error rate). When the ambient noise is just as loud as the driver talking, speech recognition loses about 10% of the words; AVSR, 3%. Not great, but certainly usable.

Analyzing data
The sensors and embedded controllers in today’s cars collect a wealth of data. The next step is to have them “phone home,” transmitting that wealth back to those who can use those data. Making sense of that detailed data is hardly a trivial matter, though—especially when divining transient problems or analyzing data about the vehicle’s operation over time.

IBM’s Automated Analysis Initiative is a data management system for identifying failure trends and predicting specific vehicle failures before they happen. The system comprises capturing, retrieving, storing, and analyzing vehicle data; exploring data to identify features and trends; developing and testing reusable analytics; and evaluating as well as deriving corrective measures. It involves several reasoning techniques, including filters, transformations, fuzzy logic, and clustering/mining.

Since 1999, this sort of technology has helped Peugeot diagnose and repair 90% of its cars within four hours, and 80% of its cars within a day (versus days). An Internet-based diagnostics server reads the car data to determine the root cause of a problem or lead the technician through a series of tests. The server also takes a “snapshot” of the data and repair steps. Should the problem reappear, the system has the fix readily available.

Sharing data
Collecting dynamic and event-driven data is one problem. Another is ensuring data security, integrity, and regulatory compliance while sharing that data. For instance, vehicle identifiers, locations, and diagnostics data from a fleet of vehicles can be used by a variety of interested, and sometimes competitive, parties. These data can be used to monitor the vehicles (something the fleet agency will definitely want to do, and so too may an automaker eager to analyze its vehicles’ performance), to trigger emergency roadside assistance (third-party service provider), and to feed the local “traffic helicopter” report.

This IBM project is the basis of a “Pay As You Drive” program in the United Kingdom. By monitoring car model data and policy-holder driving habits (the ones that opt-in), an insurance company can establish fair premiums based on car model and the driver’s safety record. The technology is also behind the “black boxes” readied for New York City’s yellow taxis and limousines. These boxes help prevent fraud, especially when accidents occur, by radioing vehicular information such as speed, location, and seat belt use. (See: http://www.autofieldguide.com/columns/0803it.html, for Dr. Martin Piszczalski’s discussion of London’s traffic system—or the August 2003 issue.)

Retrieving live data on-demand
“Plumbing”—the infrastructure stuff. In time, telematics will be another web service, using sophisticated back-end data processing of “live” and stored data from a variety of distributed, sometimes unconventional, external data sources, such as other cars, sensors, phone directories, e-coupon servers, even wireless PDAs. IBM calls this its “Resource Manager,” a software server for retrieving and delivering live data on-demand. This server will have to manage a broad range of data that frequently, constantly, and rapidly change. The server must give service providers the ability to declare what data they want, even without knowing exactly where those data reside. Moreover, the server must scale to encompass the increasing numbers of telematics-enabled cars, the huge volumes of data collected, and all the data out on the Internet.

A future application of this technology would provide you with a “shortest-time” routing based on road conditions changing because of weather and traffic, remote diagnostics of your car and cars on your route, destination requirements (your flight has been delayed), and nearby incentives (“e-coupons” for restaurants along your way).