Find me a Starbucks! QNX concept car showcases power of WATSON speech engine

Yes, you can talk to the QNX concept car and tell it what to do.

Recently, our friends at AT&T invited us to bring the concept car to their "Living the Networked Life" event in New York. We said yes, of course! After all, what could be cooler than riding the streets of Gotham City in a digitally pimped-out Porsche 911?

Kidding aside, the event provided an excellent opportunity to demonstrate how the car takes advantage of WATSON, AT&T's natural-language speech engine. To get an idea of what WATSON can do, check out this video from Terrence O'Brien of Engadget:



For the full Engadget article, click here. And stay tuned for more updates from the Living the Networked Life event.
 

Read More

Crossing the boundaries: Cooperation across industries will fuel the connected car

A guest post by Brian Salisbury of Telecommunication Systems (TCS)

Connected car – these two words appear together more and more these days. Consider, for example, two events that took place in February: The Connected Car Executive Lunch organized by Fierce Wireless and held during the Mobile World Congress in Barcelona, and the Telecom Council’s Mobile Forum: Connected Car meeting hosted by Marvell Semiconductor in Silicon Valley.

Speakers at these events came from mobile operators (AT&T Mobility, Orange, Sprint, Verizon), auto manufacturers (Ford, Hyundai, Nissan, Toyota), and platform and solution providers (Nokia, Pioneer, QNX Software Systems, TCS). No doubt about it, the car is now connecting industries.

Although these two events were held on different continents, the topics on the minds of attendees were very similar:

• Who “owns” the customer?
• Will the connection be part of the car, or brought to the car by its driver?
• How can the “wild west” of the Internet be safely incorporated into the car?
• What is the business model for such a multi-part solution?
• What will be the “killer app” for connected car, or is there no such thing?

The presentations and discussions were diverse, as each group sought to define their role in terms that extend logically from their own past experience, and that could provide them with some control over the outcome. Thankfully, every group shared the common goal of making sure that connected cars are safe cars, and that the introduction of new connected services doesn’t create driver distraction problems.

We are clearly on the verge of a new generation of services being extended into the car that can enhance many aspects of owning, operating, and riding in tomorrow’s vehicles. Those of us fortunate enough to be part of one of these groups will have some amazing opportunities to bring the best of our respective industries into this new space, and to build new relationships across industry boundaries.

For an example of how TCS is helping to enable the connected car, check out this post on the VW Polo that was showcased at Mobile World Congress — Ed.

---

Here’s a little more about Brian and TCS:

Brian Salisbury is director of business development at TeleCommunication Systems, Inc. (TCS), where he is responsible for developing new business with OEMs, platform providers, and developers in the LBS ecosystem. Brian has worked in the mobile industry for more than 25 years, with most of that experience being in mobile data and location-based services, and within semiconductor, device manufacturer, and network operator companies.

TCS (NASDAQ: TSYS) is a world leader in highly reliable and secure mobile communication technology. TCS infrastructure forms the foundation for market leading solutions in E9-1-1, text messaging, commercial location and deployable wireless communications. TCS is at the forefront of new mobile cloud computing services providing wireless applications for navigation, hyper-local search, asset tracking, social applications and telematics.
 

Read More

ITU-T: Developing standards to fight driver distraction

The telecommunication standardization sector of the International Telecommunications Union is poised to play a crucial role in tomorrow’s connected cars. And yet, many people in the auto industry have never heard of the ITU, or of its standardization sector, the ITU-T. So let’s start with a quick introduction.

The ITU is the United Nations agency that deals with telecommunications. More specifically, it is where governments come together to agree on international standards for Information and Communications Technologies (ICTs).

Currently, the ITU-T is developing recommendations for ICTs that interact with drivers — examples include networked-based navigation systems, web browsers, and mobile phones. These recommendations will apply to networks; to applications accessed by drivers; and to connected, nomadic, and factory-installed devices. Strictly speaking, these recommendations are voluntary, which is precisely why they are referred to as “recommendations.” However, they can become binding if mandated by regional government agencies or customers.

ITU-T recommendations for the car are needed for several reasons:

• ICTs are moving into the automotive cockpit — Increasingly, ICT systems under the scope of the ITU-T are finding their way into the cockpit and interacting with drivers. Technology-related driver distraction has been recognized as a global problem that needs to be addressed. Internationally agreed guidance on the design and performance of these systems can help increase safety.
   
• Easy-to-find guidance for ICT community — ICT designers, developers, and application authors need easy access to guidance on design and performance requirements for ICT systems that interact with drivers. The ICT community consults ITU-T recommendations for guidance and requirements on ICT systems. Therefore, there is real value in having ITU-T recommendations that can serve as a reference to existing driver distraction-related standards and to fill any standardization gaps.
   
• Internationally agreed standards — The ITU-T is where governments come together to agree on international telecommunications standards. This is what makes these standards unique. It also gives them more weight with regional Standards Development Organizations (SDOs) and regulatory authorities. There is even value in an ITU-T recommendation that simply references existing standards since it will have gone through the ITU-T approval process.

Focus group on driver distraction
Last year, I became the founding chair for the ITU’s Focus Group on Driver Distraction (FG Distraction), which was created to pull expertise from the automotive industry and human factors experts into the ITU-T standardization process. The group currently has 3 planned ITU-T recommendations related to driver distraction:

G.SAM — recommendation on mechanisms for managing the situational awareness of drivers:

 

G.V2A — recommendation on an automotive interface (for instance, APIs) for applications external to the vehicle gateway:



P.UIA — recommendation on automotive user interface requirements:



FG Distraction will finalize pulling together input from industry and human factors experts by December 2012. Approved ITU-T recommendations are not expected until late 2013.

Want to contribute?
If you’d like to participate in FG Distraction, the group is open to any individual from a country that is a member of ITU and who is willing to contribute to the work (which is just about everyone). This includes anyone who is also a member or representative of an interested SDO. There is no cost to participate.

For more information about FG Distraction and upcoming meetings, click here. The next meeting will take place on April 4-5 at the SAE Headquarters in Troy, Michigan, USA.
 

Read More

Seamless connectivity is for more than online junkies

As much as I’m not always enamored with sitting behind a computer all day, I find being off the grid annoying. Remember this email joke?

You know you’re an online junkie when you:
• wake up at 3:00 am to go to the bathroom and stop to check your email on the way back to bed
• rarely communicate with your mother because she doesn’t have email
• check your inbox. It says ‘no new messages,’ so you check it again 

Even though this joke circulated several years ago, it still strikes a chord. The big difference now is that there’s no longer a subculture of ‘online junkies.’ From the time we wake up in the morning to the time we go to bed, we all want to be connected — and that includes when we get behind the wheel. So to this joke I would add:

• resent driving because it means going off the grid

At QNX, we’re working toward a seamless experience where people can enjoy the same connectivity whether they’re texting their spouse from the mall or checking traffic reports while driving down the highway. See what I mean:



For more information about the technology described in this video, visit the QNX website.
 

Read More

Video: The secret to making hands-free noise-free

 

Explaining a highly technical product to a broad audience is tough. To succeed, you must reach out to people on their own terms, without being condescending. Most people love a good explanation, but everyone hates being talked down to.

Case in point: The QNX Acoustic Processing Suite. This software runs in millions of cars and offers a benefit that everyone can relate to: clear, rich, easy-to-understand hands-free calls. But once you start explaining how the suite does this, it's easy to get mired in technical jargon and to forget the bigger picture — something that even a technical audience wants to see.

So we dropped the jargon and opted for a creative approach. It involves a marching band, a rock guitarist, and, for good measure, an electric fan with a really long extension cord. Seriously.

Intrigued yet? Well, then, grab some popcorn and dim the lights:




Interested in learning more about this technology? Check out the acoustic processing page on the QNX website.

BTW, companies that use the QNX Acoustic Processing Suite in their products include OnStar, whose FMV aftermarket mirror recently won a CES Innovations Design and Engineering Award.

Posted by Paul Leroux
 

Read More

Pimp your ride with augmented reality — Part II

Last week, I introduced you to some cool examples of augmented reality, or AR, and stated that AR can help drivers deal with the burgeoning amount of information in the car.

Now that we’ve covered the basics, let’s look at some use-cases for both drivers and passengers. Remember, though, that these examples are just a taste — the possibilities for integrating AR into the car are virtually endless.


AR for the driver
When it comes to drivers, AR will focus on providing information while reducing distraction. Already, some vehicles use AR to overlay the vehicle trajectory onto a backup camera display, allowing the driver to gauge where the car is headed. Some luxury cars go one step further and overlay lane markings or hazards in the vehicle display.

Expect even more functionality in the future. In the case of a backup camera, the display might take advantage of 3D technology, allowing you to see, for example, that a skateboard is closer than the post you are backing towards. And then there is GM's prototype heads-up system, which, in dark or foggy conditions, can project lane edges onto the windshield or highlight people crossing the road up ahead:



AR can be extremely powerful while keeping distraction to a minimum. Take destination search, for example. You could issue the verbal command, “Take me to a Starbucks on my route. I want to see their cool AR cups”. The nav system could then overlay a subtle route guidance over the road with a small Starbucks logo that gets bigger as you approach your destination. The logo could then hover over the building when you arrive.

You'll no longer have to wonder if your destination is on the right or left, or if your nav system is correct when it says, “You have arrived at your destination.” The answer will be right in front of you.

AR for the passenger
So what about the passenger? Well, you could easily apply AR to side windows and allow passengers to learn more about the world around them, a la Wikitude. Take, for example, this recent video from Toyota, which represents one of the best examples of how AR could make long road trips less tedious and more enjoyable:


 

Read More

Pimp your ride with augmented reality — Part I

The use of electronics is exploding in automotive. Just last week, Intel proclaimed that the connected car “is the third-fastest growing technological device, following smartphones and tablets.”

Ten years ago, you’d be hard-pressed to find a 32-bit processor in your car. Now, some cars have 4 or more 32 bitters: one in the radio, another in the telematics module, yet another in the center display, and still another in the rear-seat system.

Heck, in newer cars, you’ll even find one in the digital instrument cluster — the QNX-powered cluster in the Range Rover, for example. Expect to see a similar demand for more compute power in engine control units, drive-by-wire systems, and heads-up displays.


The Range Rover cluster displays virtual speedometers and gauges, as well as warnings, suspension settings, and other info, all on a dynamically configurable display.

What do most of these systems have in common? The need to process tons of information, from both inside and outside of the vehicle, and to present key elements of that data in a safe, contextually relevant, and easy-to-digest fashion.

The next generation of these systems will be built on the following principles:

• Fully integrated cockpits — Vehicle manufacturers see system consolidation as a way to cut costs and reduce complexity, as well as to share information between vehicle systems. For instance, your heads-up display could discreetly let you know who is calling you, without forcing you to take your eyes off of the road. And it could do this even if the smarts integrating your phone and your car reside in another cockpit component — the telematics module, say.
   
• Augmented reality — With all of the data being generated from phones, cloud content services and, perhaps more importantly, the vehicle itself, presenting the right information at the right time in a safe way will become a major challenge. This is where augmented reality comes in.

Augmented reality is a cool use of cameras, GPS, and data to create smart applications that overlay a virtual world on top of the real world. Here are some of my favorite examples:

AR Starbucks cups — Use your phone to make your coffee cup come alive:



AR Starwars — Blast the rebel alliance squirrels!



AR postage stamp — Add a new dimension (literally) to an everyday object:



And here are a couple more for good measure:

AR ray gun — Blast aliens around the house!

Wikitude AR web browser — Explore the world around you while overlaying social networks, images, video, reviews, statistics, etc.

Stay tuned for my next post, where I will explore how AR could enhance the driving experience for both drivers and passengers — Andrew.
 

Read More

A cool and innovative speedometer... for 1939

Paul Leroux

Earlier this week, I referred you to a whitepaper written by my colleagues Scott Pennock and Andy Gryc. In the paper, Scott and Andy argue that driver distraction is not, in fact, a problem of distraction, but of situational awareness, or SA. Boost a person's SA, and you improve their ability to drive safely.

But how, exactly, do you improve SA? The paper discusses various techniques, and I couldn't possibly do justice to all of them here. But one approach is to supplement the driver's eyes and ears with indicators and warnings, based on information from sensors, roadside systems, and other vehicles.

Here's an example: A system in your car learns, through cloud-based traffic reports, that the road ahead is slick with ice. It also determines that you are driving much too fast for such conditions. The system immediately kicks into action, perhaps by warning you of the icy conditions or by telling you to ease off the accelerator.

Too bad the engineers who designed the 1939 Plymouth P8 didn't have access to such technology. I'm sure they would have embraced it totally.

You see, they too wanted to warn drivers about excess speed. Unfortunately, the technology of the time limited them to creating a primitive, one-size-fits-all solution — the safety speedometer.

Color coded for safety
From what I've read, these speedometers switch from green to amber to red, depending on the car's speed. I've only seen still photos of these speedometers, but allow me to invoke the magic of PhotoShop and reconstruct how I think they work.

The safety speedometer has a rotating bezel, and embedded in this bezel is a small glass bulb. At speeds from 0 to 30 mph, the bulb glows green:



At speeds from 30 to 50 mph, the bulb turns amber:



And at over 60 mph, the bulb turns red:



Given the limitations of 1939 technology, the Plymouth safety speedometer couldn't take driving conditions or the current speed limit into account. It glowed amber at 30 mph, regardless of whether you were cruising through your neighborhood or poking down the highway. As a result, it was more of a novelty than anything else. In fact, I wonder if people driving the car for the first time would have focused more on watching the colors change than on the road ahead. If so, the speedometer may have actually reduce situational awareness. Oops!

Compare this to a software-based digital speedometer, which could take input from multiple sources, both within and outside the car, to provide feedback that dynamically changes with driving conditions. For instance, a digital speedometer could acquire the current speed limit from a navigation database and, if the car is exceeding that limit, remind the driver that they risk a speeding ticket.

That said, I have a soft spot for anyone who is (or was) ahead of their time. Some enterprising Plymouth engineers in the 30s realized that, with faster speeds, comes the need for even greater situational awareness. Their solution was primitive but it offered a hint of what, more than 75 years later, can finally become reality.
 

Read More

Some people drive me to distraction

Paul Leroux

Hey, have you ever panned your camera? It's really easy: You just track a moving subject with your camera and then squeeze the shutter while both you and the subject are in motion. It's a great technique for creating images that evoke a sense of speed, which makes it popular among photographers for Motor Trend, Car and Driver, and other automotive magazines.

When you pan, you never really know what kind of image you're going to get. Often, the results are interesting. And sometimes, they're downright interesting. Take this shot, for example:


Lattés and overdrive don't mix. Just sayin'.

Now, holding a cellphone while rocketing down the highway is just plain wrong. To anyone who does it, I have one thing to say: "You're endangering other people's lives for the sake of a f***ing phone call. Where the hell do you get off doing that?"

But look at this guy. He's isn't holding a phone, but a coffee — even worse. Just imagine if he gets into a situation that demands quick, evasive action. He will, in all likelihood, hold on to the cup for fear of burning himself. Whereas if he had a phone, he would simply drop it and put his hand back on the wheel.

Mind you, I have no data to prove that coffee cups poses a greater evil than cellphones. But the core issue remains: Cellphone use is just one of many factors that contribute to driver distraction. In fact, research suggests that cellphones account for only 5% of distraction-related accidents that end in injury.

So, even if every cellphone on the planet disappeared tomorrow, we would still have a massive problem on our hands. To that end, my colleagues Scott Pennock and Andy Gryc suggest a new approach to designing vehicle cockpit systems in their paper, "Situational Awareness: a Holistic Approach to the Driver Distraction Problem."

The paper explores how system designers can use the concept of situational awareness to develop a vehicle cockpit that helps the driver become more aware of objects and events on the road, and that adapts in-vehicle user interfaces to manage the driver’s cognitive load.

It's worth a read. And who knows, perhaps someone, someday, will develop a cockpit system that detects if you are sipping something and tells you what you need to hear: "Dammit Jack, put that cup down. It's not worth endangering other people's lives for the sake of a f***ing latté."

Read More

Building a hands-free future

The end of my street is governed by a three-way stop. The other morning I was backing out of my driveway when someone rolled past the stop sign and came within inches of hitting me. I stopped, glared at him, and resumed driving. Two stop signs later, the same guy squeezed past my car (in the same lane), completely oblivious to what he was doing.

Why was he driving like this? Probably because he was deeply engrossed in a conversation on his cell phone.

Where I live, using a handset while driving has been illegal for over a year. You cannot talk, you cannot text, you cannot “Facebook”, you cannot Tweet — even if you're stopped at a red light. This makes perfect sense to me. As a driver, your primary responsibility is to control the vehicle. And yet I see people texting on the freeway, talking on their cell phones, and doing who knows what else on an alarmingly regular basis.

The QNX-powered BMW
ConnectedDrive system

Society has become obsessed with mobile devices, and it will take more than legislation to change its behavior. The answer, I think, is to embrace the behavior in a way that makes it possible to interact socially while maintaining control of the car. We’ve seen great progress in hands-free/phone integration, and BMW ConnectedDrive offers an example of how drivers can access email and other smartphone services more safely.

This is the tip of the iceberg. Integrating the handset with the infotainment unit is going to change the way you interact with your car. Intelligently designed apps, combined with multi-modal human machine interfaces, will let you Tweet or update Facebook using speech recognition, keeping your eyes on the road.

Without taking your hands of the wheel, you’ll be able to call a friend and decide that you want to go to dinner, do a local search to find out what’s available, check a restaurant review on Yelp, make a reservation, text your friend back with the time and place, and aim your navigation system at the restaurant. And you’ll be able to do it using natural language. None of this “please say a name” stuff.

Seems futuristic? It’s not. People are working on it today. In fact, QNX-based systems, such as Toyota Entune, already offer a taste of this hands-free and highly personalized future.
 

Read More

Speech interfaces: UI revolution or intelligent evolution?

Speech interfaces have received a lot of attention recently, especially with the marketing blitz for Siri, the new speech interface for the iPhone.

After watching some of the TV commercials you might conclude that you can simply talk to your phone as if it were your friend, and it will figure out what you want. For example, in one scenario the actor asks the phone, “Do I need a raincoat?”, and the phone responds with weather information.

A colleague commented that if he wanted weather information he would just ask for it. As in “What is the weather going to be like in Seattle?” or “Is it going to rain in Seattle?”.

Without more conversational context, if a friend were to ask me, “Do I need a raincoat?”, I would probably respond, “I don’t know, do you?” — jokingly, of course.

Evo or revo?

Are we ready to converse
with our phones and cars?

Kidding aside, systems like Siri raise an important question: Are we about to see a paradigm shift in user interfaces?

Possibly. But I think it will be more of a UI evolution than a UI revolution. In other words, speech interfaces will play a bigger role in UI designs, but that doesn't mean you're about to start talking to your phone — or any other device — as if it’s your best friend.

Currently, speech interfaces are underutilized. The reasons for this aren't yet clear, though they seem to encompass both technical and user issues. Traditionally, speech recognition accuracy rates have been less than perfect. Poor user interface design (for instance, reprompting strategies) has contributed to the overall problem and to increased user frustration.

Also, people simply aren't used to speech interfaces. For example, many phones support voice-dialing, yet most people don't use this feature. And user interface designers seem reluctant to leverage speech interfaces, possibly because of the additional cost and complexity, lack of awareness, or some other reason.


"Relying heavily on speech can lead
to a suboptimal user experience..."
As a further complication, relying heavily on speech as an interface can lead to a suboptimal user experience. Speech interfaces pose some real challenges, including recognition accuracy rates, natural language understanding, error recovery dialogs, UI design, and testing. They aren't the flawless wonders that some marketers would lead you to believe.

Still, I believe there is a happy medium for leveraging speech interfaces as part of a multi-modal interface — one that uses speech as an interface where it makes sense. Some tasks are better suited for a speech interface, while others are not. For example, speech provides an ideal way to provide input to an application when you can capitalize on information stored in the user’s head. But it’s much less successful when dealing with large lists of unfamiliar items.

Talkin' to your ride
Other factors, besides Apple, are driving the growing role of speech interfaces — particularly in automotive. Speech interfaces can, for example, help address the issue of driver distraction. They allow drivers to keep their “eyes on the road and hands on the wheel,” to quote an oft-used phrase.

So, will we see a paradigm shift towards speech interfaces? It's unlikely. I'm hoping, though, that we'll see a UI evolution that makes better use of them.

Think of it more as a paradigm nudge than a paradigm shift.

---

Recommended reading

Situation Awareness: a Holistic Approach to the Driver Distraction Problem
Wideband Speech Communications for Automotive: the Good, the Bad, and the Ugly

 

Read More

When will I get apps in my car?

I read the other day that Samsung’s TV application store has surpassed 10 million app downloads. That got me thinking: When will the 10 millionth app download occur in the auto industry as a whole? (Let’s not even consider 10 million apps for a single automaker.)

There’s been much talk about the car as the fourth screen in a person’s connected life, behind the TV, computer, and smartphone. The car rates so high because of the large amount of time people spend in it. While driving to work, you may want to listen to your personal flavor of news, listen to critical email through a safe, text-to-speech email reader, or get up to speed on your daily schedule. When returning home, you likely want to unwind by tapping into your favorite online music service. Given the current norm of using apps to access online content (even if the apps are a thin disguise for a web browser), this begs the question — when can I get apps in my car?

Entune takes a hands-free
approach to accessing apps.

A few automotive examples exist today, such as GM MyLink, Ford Sync, and Toyota Entune. But app deployment to vehicles is still in its infancy. What conditions, then, must exist for apps to flourish in cars? A few stand out:

Cars need to be upgradeable to accept new applications — This is a no-brainer. However, recognizing that the lifespan of a car is 10+ years, it would seem that a thin client application strategy is appropriate.

Established rules and best practices to reduce driver distraction — These must be made available to, and understood by, the development community. Remember that people drive cars at high speeds and cannot fiddle with unintuitive, hard-to-manipulate controls. Apps that consumers can use while driving will become the most popular. Apps that can be used only when the car is stopped will hold little appeal.

A large, unfragmented platform to attract a development community — Developers are more willing to create apps for a platform when they don't have to create multiple variants. That's why Apple maintains a consistent development environment and Google/Android tries to prevent fragmentation. Problem is, fragmentation could occur almost overnight in the automotive industry — imagine 10 different automakers with 10 different brands, each wanting a branded experience. To combat this, a common set of technologies for connected automotive application development (think web technologies) is essential. Current efforts to bring applications into cars all rely on proprietary SDKs, ensuring fragmentation.

Other barriers undoubtedly exist, but these are the most obvious.

By the way, don’t ask me for my prediction of when the 10 millionth app will ship in auto. There’s lots of work to be done first.

 

Read More