A matter of context: How digital instrument clusters can enhance the driving experience

I always drive a manual, so checking the tachometer in my car’s instrument cluster has become second nature to me. But while I have a personal interest in what my cluster displays, why would a software company like QNX be interested in instrument clusters? After all, most clusters use physical gauges and relatively little software.

The answer, of course, is that automakers are starting to migrate to digital instrument clusters, which replace mechanical gauges with virtual instruments rendered on an LCD display. In fact, Jaguar and Land Rover, who are pioneers in this market, have been shipping QNX-based digital clusters since about 2010. Here, for instance, is a photo of the digital cluster and dash in the latest Range Rover:



So why use a large LCD display instead of mechanical gauges? For one thing, you can attract early adopters who always want the latest tech and who see large 3D displays as cool. But more importantly, a digital cluster can provide an experience that is both personal and adaptive — personal because consumers today want to control the UX (just as they customize their smartphones) and adaptive to help the driver in a variety of traffic situations.

Context matters
In the latest QNX technology concept car, for instance, the digital cluster can re-configure itself to display a 3D rear view camera to help with parking. Saab pursued similar ideas a few years ago with a context-based cluster that avoids loading the driver with too much information during night-time driving.

It will be interesting to see who takes this to the next level with an adaptive HMI that takes speed, location, and driving conditions into account. For instance, driving at high speed on a German Autobahn differs immensely from driving at low speed on a busy downtown street with lots of pedestrians and intersections. These two scenarios place different demands on the driver, and a digital cluster could adapt accordingly.

On the autobahn, the cluster could increase the size of the speedometer and tachometer to make them easier to see, while hiding other information that isn’t currently needed. (The cluster would, of course, still display any necessary warnings, such as high oil temperature.) In the city, meanwhile, the cluster could replace the tachometer with pedestrian warnings to improve the driver's situational awareness.

Also, think of a car that supports both automatic and manual gear-shifting. A driver who prefers automatic might not be interested in a tachometer, whereas a driver who shifts manually will want to see a RPM readout to optimize gear shifting. A digital cluster could accommodate both preferences.

For safety’s sake
What does it mean from a safety perspective to include a large display and its attendant electronics in the car? A malfunctioning digital cluster can’t directly kill or injure, but it could give false indications that may lead to an accident. That is why automakers will likely have to address ISO 26262 requirements for their digital clusters.

So what is ISO 26262? It’s a standard that focuses on functional safety in cars and other types of passenger vehicles, with the goal of avoiding or controlling system failures. It is similar in content and purpose to the IEC 61508 functional safety standard, to which two QNX OS products have already been certified. Read our previous posts (here and here) for more information on ISO 26262.

Massive arrays
When it comes to digital clusters, I’ve only scratched the surface. For instance, cars are becoming massive sensor arrays that generate tons of data. By leveraging this data, reconfigurable clusters could display contextually relevant information, such as highlighting a person in your path, an accident up ahead, or the current speed limit.

And from the automaker’s perspective, a digital cluster could help reduce costs by allowing the same hardware to be used across multiple vehicle lines; in many cases, only the graphics would need to be “reskinned.”

---

Emil Dautovic is an automotive business development manager at QNX Software Systems, where he is responsible for the European automotive market.

Read More

Day two of CES - QNX, Harman, and Vipers, oh my!

Day two came and went very fast. It felt as frantic as day one. I did manage to leave the north hall though... only to find the center hall to be just as intense. Maybe even more so. It had booths the size of city blocks. Maybe even bigger; certainly louder.

I somehow found Harman International. And I was not disappointed. There were at least four demos sporting QNX technology.

QNX CAR 2 was on display and in full form in the Harman booth. Nice to see the QNX logo so prominently displayed.


Closer look at different display in the same demo system. This 3D map from Elektrobit just never gets old.


Harman again. QNX again. This is almost embarrassing! ;-)


Harman and QNX and Vipers, oh my!


I managed to get permission to poke my head into the car. The digital instrument cluster was gorgeous.


The Viper's infotainment system was just as beautiful. Note to my boyfriend: I do believe this car is currently shipping.


The QNX booth staff, including biz dev manager Emil Dautovic, were hard at work once again. Although how hard can it be to talk to nice people all day? Maybe if there were more chairs...


Of course we knew our latest concept car would be a hit but we weren't exactly expecting the mob scene that it was again on day two.


Joe Cusumano, automotive field applications engineer, in conversation.


Linda Campbell, director of QNX strategic alliances, was her usual tireless self.


Our inimitable automotive field applications engineer Dan Baergen.


And our very own Dr. Phil, mastermind of the QNX acoustics processing and noise cancellation products.

Read More

It's a Bentley! A guided tour of the new QNX technology concept car

"Bend it, shape it, any way you want it"
— Headline from a QNX advertisement, circa 1987

I’m about to show you some pictures of a car. Not just any car, but a powerful, luxurious, and stunningly beautiful car. A car that has undergone a technological transformation.

If you’re like me, you'll be fascinated by the car’s features, some of which have never been seen in a vehicle — until now. But if you can, remember that it isn’t just about the cool features. It’s also about the platform that enabled them.

I’m speaking, of course, of the QNX CAR application platform.

We created the new QNX technology concept car — a modified Bentley Continental GT — to demonstrate that flexibility and customization form the very DNA of the QNX CAR platform. If you’ve seen the QNX reference vehicle, you already know that the platform provides an extremely rich environment for in-car infotainment, complete with HMI frameworks, smartphone integration, an HTML5 engine, a mobile device gateway, and a host of pre-integrated partner technologies — everything to kickstart our customers' projects. But in the automotive world, differentiation is everything. So it’s just as important that the platform enables customers to add their own branding, features, and sizzle. And to do it quickly.

Ease of branding and
personalization is just one

of the capabilities of the
QNX CAR platform.

Which is where the new concept car comes in. To create it, we used the same base QNX CAR platform that we offer our customers. But when you compare the Bentley to the Jeep, which uses a stock version of QNX CAR, the differences are dramatic: different features, different branding, and a different look-and-feel. In effect, the Jeep shows what QNX CAR can do out of the box, while the Bentley shows what QNX CAR lets you do once you start bending it to your imagination. One platform, many possibilities.

Which brings me to the slogan at the top of this post. It's amazing to think that a core value of QNX technology in the 1980s — giving customers the flexibility to achieve what they want to do — remains just as true today. Some values, it seems, are worth keeping.

And now, the car…
I know that you’re anxious to peek inside the car and see what we’ve done. But before we go any further, take a moment to savor the car’s beautifully sculpted exterior. This is one classy set of wheels. In fact, if you ask me, the wheels alone are worth the price of admission:



The awesome (and full HD) center stack
Okay, time to hop in — but get ready to prop up your jaw. Because the first thing you’ll notice is the jaw-droppingly beautiful center stack. This immense touchscreen features a gracefully curved surface, full HD graphics, and TI’s optical touch input technology, which allows a physical control knob to be mounted directly on the screen — a feature that’s cool and useful. (In the photo below, the clock display appears within the knob.)

The center stack supports a host of applications, including a 3D navigation system from Elektrobit that makes full use of the display. Just check out this bird’s-eye view of the Las Vegas Strip:



So how big is the display? Big enough to provide access to other functions, such as the car’s media player or virtual mechanic, and still have plenty room for navigation. Check it out:



The awesome (and very polite) voice rec system
Time to talk to the car. Just say “Hello Bentley,” and the car’s voice recognition system immediately comes to life and begins to interact with you — in a British accent, no less. You can now tell the media player what you’d like to hear and the navigation system where you’d like to go.

To provide natural language speech recognition, the system uses the cloud-based AT&T Watson speech engine, as well as an “intent framework” from QNX. It also uses keyword spotting technology from Sensory so you can start the system simply by talking to it.

The awesome (and nicely integrated) smartphone support
The Bentley also showcases how the QNX CAR platform enables automakers to offer advanced integration with popular smartphones. For instance, the car can communicate with a smartphone to stream music, or to provide notifications of incoming email, news feeds, and other real-time information — all displayed in a manner appropriate to the automotive context. Here's an example:



The awesome (and just plain fun) web app
I know, I know: the car looks cool, but you’re not at CES this week to see it first-hand. But how about the next best thing? Just connect to the web app and keep tabs on the Bentley in real time. (Note: The car will go online later this morning.) The app lets you view a variety of data that the car publishes to the cloud, such as what song the infotainment system is playing and whether someone has just opened a door. It also displays information that would be extremely helpful if this were your personal car, such as fluid levels and tire pressure. (This is a preliminary screen for the app, so I'm not sure if the tire pressures are realistic.)



UPDATE: The web app is now live, and the desktop version features a live camera feed of the Bentley and Jeep. Check it out!



The awesome (and very configurable) digital instrument cluster
The instrument cluster is implemented entirely in software, though you would hardly know it — the virtual gauges are impressively realistic. But more impressive still is the cluster’s ability to morph itself on the fly. Put the car in Drive, and the cluster will display a tach, gas gauge, temperature gauge, and turn-by-turn directions — the cluster pulls these directions from the center stack’s navigation system (cool, that). Put the car in Reverse, and the cluster will display a video feed from the car’s backup camera.



There are other options as well. For instance, the cluster can display information from the media player or display the current weather:



The awesome everything else
I’ve only scratched the surface of what the car can do. For instance, it also provides:

• Advanced multimedia system — Offers direct support for Pandora radio and the first embedded in-car implementation of the Shazam music discovery service.
   
• Video conferencing with realistic telepresence — Separate cameras for the driver and passenger provide independent video streams, while high-definition voice technology from QNX offers expanded bandwidth for greater realism, as well as stereo telepresence for making the remote caller sound as if they’re sitting right next to you.
   
• LTE connectivity — The car features an LTE radio modem, as well as a Wi-Fi hotspot for devices you bring into the car.

Super size those images
Want to see the center stack and instrument cluster in all their high-resolution glory? Just check out our QNX Flickr account.

That's all for now, but stay tuned: We’ll have plenty more news for you today and all through this week.

Read More

QNX CAR 2 — the extended version

The world of video is a ruthless one; just as we posted the QNX CAR demo it was out of date.

But, hold on a minute. As I write this I realize it’s not the video world at all; it’s the software world that creates new technology at breakneck speed. And QNX certainly does its part.

The QNX CAR 2 application platform has come a long way in a matter of months. We needed to update the original video to keep pace with the technology but also to address customer demand for more detailed information.

So this video is a step-by-step demo – definitely not for the tire kicker. But if you really want details on what automakers and tier ones can achieve with QNX CAR technology, hit play.

Read More

QNX reference vehicle makes stopover at FTF Americas 2012

Fresh off Telematics Detroit, the QNX reference vehicle is on the road again. And this time, it’s headed to the Freescale Technology Forum (FTF) in San Antonio.

Have you seen photos of the vehicle? If so, you'll know it's a specially modified Jeep Wrangler. From the outside, the Jeep stills looks the same, but beneath the hood, something has changed. For the first time, the Jeep’s head unit and instrument cluster, both based on the QNX CAR 2 application platform, are using Freescale i.MX 6 processors. And what better place than FTF to show off this new processor support?

Closeup of Jeep's instrument cluster. See previous post for more photos of vehicle.

As before, the reference vehicle will showcase several capabilities of the QNX CAR 2 platform, including:

• auto-centric HTML5 framework
• integration with a variety of popular smartphones
• one-touch Bluetooth pairing with smartphones using NFC
• ultra HD hands-free communication
• DLNA support for phone- and home- based media
• tablet-based rear-seat entertainment
• reconfigurable digital instrument cluster
• Wi-Fi hotspot

The vehicle will also demonstrate several popular third-party technologies, including Pandora, Slacker, and TuneIn Internet radio; TCS navigation; Weather Network; Best Parking; and Vlingo/AT&T Watson voice recognition.

What, more demos?
The reference vehicle isn't the only place to catch QNX technology at FTF. QNX will also showcase:

• a 3D digital instrument cluster based on a Freescale i.MX 6 quad processor and the QNX Neutrino RTOS, and built with Elektrobit's EB GUIDE Human Machine Interface environment
 
• a complete dashboard, including head unit and digital cluster, based on the QNX CAR 2 platform
 
• demos for industrial controllers, medical devices, multi-core systems, and advanced graphics, all of which run on the QNX Neutrino RTOS and Freescale silicon

QNX at the podium
Did I mention? QNX experts will also in participate in several presentations and panels. Here's the quick schedule:

• The HTML5 Effect: How HTML5 will Change the Networked Car — June 19, 2:00 pm, Grand Oaks Ballroom A
 
• Using an IEC 61508-Certified RTOS Kernel for Safety-Critical Systems — June 20, 2:00 pm, Grand Oaks Ballroom P
 
• Embedded Meets Mobility: M2M Considerations and Concepts — June 20, 5:15 pm, Grand Oaks Ballroom E
 
• New System Design for Multicore Processors — June 21, 10:30 am, Grand Oaks Ballroom F

Visit the FTF website for details on these and other FTF presentations.

And if you're at FTF, remember to catch the QNX demos at pod numbers 1400 to 1405.
 

Read More

WIRED Autopia slips into driver's seat of QNX reference vehicle

Chances are, you've seen pictures of the new QNX reference vehicle. You may have even seen the "making of" video that QNX released a few days ago. But have you seen any video of the vehicle in action?

If not, check out this vid by Doug Newcomb of WIRED Autopia. Last week, at Telematics Detroit, Doug met up with Andrew Poliak of QNX for a tour of the vehicle and its various features, including a re-skinnable UI and voice-controlled Facebook integration. The camera was rolling, and here's what it caught:

Read More

The making of the QNX reference vehicle: Jeep Wrangler

Guest post from Nicole Forget of QNX Software Systems

Nicole Forget

Just one week ago, our new reference vehicle was revealed at Telematics Detroit 2012. The Jeep Wrangler features QNX’s digital instrument cluster, which is totally re-skinnable. In fact, the entire user interface of the head unit, which was created using HTML5, can also be re-skinned. The head unit supports loads of functions, too, including the virtual mechanic, which are outlined in an earlier post.

The following video gives you some insight into the hard work that was put into the making of the reference vehicle. Check it out!


 

Read More

Full disclosure: QNX releases first complete photos of new reference vehicle

"Any customer can have a car painted any color... so long as it is black."

We've come a long way since 1909, when Henry Ford penned this now-famous sentence. Not only can modern consumers pick the colors and features they want in a vehicle, but, in many cases, they can order them online. Getting the car you want, with the options you want, has never been easier.

Still, most forms of personalization are baked in. Once you order a car in, say, Barcelona Red (the color of my new Venza), it's hard to reverse the decision. But imagine the day when you can sit behind the wheel and watch your car's instrument cluster automatically reconfigure itself according to your personal preferences. And imagine if the cluster could do the same for everyone else who uses the car.

That's the kind of future QNX is working to make happen.

But you know what? I'm getting ahead of myself. I promised pictures of the new QNX reference vehicle, so let's look at them — especially since they offer tantalizing examples of what I was just talking about. :-)

The vehicle
Up to now, we've only released teaser images of the reference vehicle, with just enough detail to get people guessing as to what it might be. But enough with the mystery. Here's a full-on shot of the vehicle — a Jeep Wrangler Sahara — in all its off-road glory:


Yes, it's a Jeep

By the way, if you were one of the first 25 people in Canada or the US to guess it was a Jeep during our recent teaser campaign, congratulations! We'll identify the winners shortly.

The cluster
Once you get behind the wheel, the first thing you'll see is the digital instrument cluster. Let's zoom in so you can get a good look:



The cluster is implemented entirely in software and can reconfigure itself on the fly to display various types of information. Better yet, you can re-skin the cluster at the tap of a touchscreen button, like so:



As you can see, the cluster communicates with the head unit's navigation system to display turn-by-turn directions. Nice touch.

The head unit
Now look to your right, and you'll see the head unit. It supports a whack of functions (note my deft use of technical language), including one-touch pairing with Bluetooth smartphones, hybrid navigation, text-to-speech, natural speech recognition, streaming Internet radio, weather reporting, parking search, and too many other things to mention here.

In this photo, the head unit displays one of my favorite applications, the virtual mechanic. Intrigued? Check out my description of an early version of this app.



You know what else is cool? The unit's media player can post Facebook updates that list the song currently playing — but only when you tell it to, using voice commands. (Personal control over technology. I like that.) To view these updates later today and tomorrow while the Jeep is at Telematics Detroit, check out the QNX Facebook page.

Here's another photo of the head unit, showing its app tray:



The radio
What car would be complete without a radio? Mind you, in this case, "radio" includes support for streaming Internet radio from Pandora and TuneIn. And keeping in tune with the personalized listening experience these services offer you, the head unit's radio gives you a choice of skins:





In fact, almost every aspect of the head unit can be easily re-skinned. What's more, the underlying code remains the same: only the user interface, created in HTML5, changes from one skin to another. Which means automotive developers can create a single code base and re-use it across multiple vehicle lines. Doing more with less — what could be bad?

That's all I have for now, but before you go, check out the two press releases QNX issued this morning on the Jeep's personalization and Facebook features. Also, check out the QNX Flickr page for even more photos of the Jeep.
 

Read More

Everything you wanted to know about HTML5 in the car, Part II

Welcome to the second installment in my Q&A series on HTML5 in the car. Last week, we looked at CSS, cross-platform execution, and asynchronous design. This week, we turn our attention to web servers, native plug-ins, instrument clusters, and display updates.

If I don’t use a web server in my infotainment system, will I miss out on some features of HTML5?

A web server isn’t strictly necessary, but there are two very good reasons for including one. First, it lets you export a user interface to devices outside the car, thereby allowing mobile phones or tablets to run apps that are hosted on the vehicle head unit. Second, it lets you export internal car resources, as a URL, to HMI software running in the head unit. For instance, the web server could provide the HMI with access to static vehicle-configuration data (through an xml file) or to a back-up camera (through a video stream).

Will using native code plug-ins compromise my ability to leverage HTML5?

This is tricky, because a lot of things you want to do may require native code. So, yes, use native code, but do it judiciously. The more native code you use, the more it will limit the cross-platform capability of the HTML5 code that relies on it. The good news is that with HTML5 gaining so much functionality, plug-ins are needed far less than ever before.

A sample climate control app from the
QNX CAR 2 platform, created with HTML5.

Would you consider HTML5 as an option for cluster instruments: speedometers, tachometers, etc.?

At this point, I’d say no. HTML5 makes a lot of sense for in-vehicle infotainment, but it doesn’t provide the response needed for a vehicle cluster and it won't ensure safety-critical certification. Plus, the instrument cluster isn’t where you realize a lot of HTML5’s value: downloadable apps, connectivity to mobile devices, and so on. If the cluster and the infotainment system eventually merge into one big screen, then it’s more likely you could use HTML5 for both — but that’s still a few years out.

What’s a good way to get responsive display updates (10Hz update) into HTML5? Websockets?

If you need to deliver high-speed updates to your head unit, Websockets is one way to go. Make sure, however, that you don’t stall the rest of the JavaScript engine while your main thread is blocked on tasks. If you create another thread to monitor for changes, you can do it just as effectively (and probably with less work) with a JNEXT or NPAPI call into native code.
 

Read More

Pimp your ride with augmented reality — Part I

The use of electronics is exploding in automotive. Just last week, Intel proclaimed that the connected car “is the third-fastest growing technological device, following smartphones and tablets.”

Ten years ago, you’d be hard-pressed to find a 32-bit processor in your car. Now, some cars have 4 or more 32 bitters: one in the radio, another in the telematics module, yet another in the center display, and still another in the rear-seat system.

Heck, in newer cars, you’ll even find one in the digital instrument cluster — the QNX-powered cluster in the Range Rover, for example. Expect to see a similar demand for more compute power in engine control units, drive-by-wire systems, and heads-up displays.


The Range Rover cluster displays virtual speedometers and gauges, as well as warnings, suspension settings, and other info, all on a dynamically configurable display.

What do most of these systems have in common? The need to process tons of information, from both inside and outside of the vehicle, and to present key elements of that data in a safe, contextually relevant, and easy-to-digest fashion.

The next generation of these systems will be built on the following principles:

• Fully integrated cockpits — Vehicle manufacturers see system consolidation as a way to cut costs and reduce complexity, as well as to share information between vehicle systems. For instance, your heads-up display could discreetly let you know who is calling you, without forcing you to take your eyes off of the road. And it could do this even if the smarts integrating your phone and your car reside in another cockpit component — the telematics module, say.
   
• Augmented reality — With all of the data being generated from phones, cloud content services and, perhaps more importantly, the vehicle itself, presenting the right information at the right time in a safe way will become a major challenge. This is where augmented reality comes in.

Augmented reality is a cool use of cameras, GPS, and data to create smart applications that overlay a virtual world on top of the real world. Here are some of my favorite examples:

AR Starbucks cups — Use your phone to make your coffee cup come alive:



AR Starwars — Blast the rebel alliance squirrels!



AR postage stamp — Add a new dimension (literally) to an everyday object:



And here are a couple more for good measure:

AR ray gun — Blast aliens around the house!

Wikitude AR web browser — Explore the world around you while overlaying social networks, images, video, reviews, statistics, etc.

Stay tuned for my next post, where I will explore how AR could enhance the driving experience for both drivers and passengers — Andrew.
 

Read More

A cool and innovative speedometer... for 1939

Paul Leroux

Earlier this week, I referred you to a whitepaper written by my colleagues Scott Pennock and Andy Gryc. In the paper, Scott and Andy argue that driver distraction is not, in fact, a problem of distraction, but of situational awareness, or SA. Boost a person's SA, and you improve their ability to drive safely.

But how, exactly, do you improve SA? The paper discusses various techniques, and I couldn't possibly do justice to all of them here. But one approach is to supplement the driver's eyes and ears with indicators and warnings, based on information from sensors, roadside systems, and other vehicles.

Here's an example: A system in your car learns, through cloud-based traffic reports, that the road ahead is slick with ice. It also determines that you are driving much too fast for such conditions. The system immediately kicks into action, perhaps by warning you of the icy conditions or by telling you to ease off the accelerator.

Too bad the engineers who designed the 1939 Plymouth P8 didn't have access to such technology. I'm sure they would have embraced it totally.

You see, they too wanted to warn drivers about excess speed. Unfortunately, the technology of the time limited them to creating a primitive, one-size-fits-all solution — the safety speedometer.

Color coded for safety
From what I've read, these speedometers switch from green to amber to red, depending on the car's speed. I've only seen still photos of these speedometers, but allow me to invoke the magic of PhotoShop and reconstruct how I think they work.

The safety speedometer has a rotating bezel, and embedded in this bezel is a small glass bulb. At speeds from 0 to 30 mph, the bulb glows green:



At speeds from 30 to 50 mph, the bulb turns amber:



And at over 60 mph, the bulb turns red:



Given the limitations of 1939 technology, the Plymouth safety speedometer couldn't take driving conditions or the current speed limit into account. It glowed amber at 30 mph, regardless of whether you were cruising through your neighborhood or poking down the highway. As a result, it was more of a novelty than anything else. In fact, I wonder if people driving the car for the first time would have focused more on watching the colors change than on the road ahead. If so, the speedometer may have actually reduce situational awareness. Oops!

Compare this to a software-based digital speedometer, which could take input from multiple sources, both within and outside the car, to provide feedback that dynamically changes with driving conditions. For instance, a digital speedometer could acquire the current speed limit from a navigation database and, if the car is exceeding that limit, remind the driver that they risk a speeding ticket.

That said, I have a soft spot for anyone who is (or was) ahead of their time. Some enterprising Plymouth engineers in the 30s realized that, with faster speeds, comes the need for even greater situational awareness. Their solution was primitive but it offered a hint of what, more than 75 years later, can finally become reality.
 

Read More