The simpler, the better: a first look at the new QNX technology concept vehicle

Bringing the KISS principle to the dashboard.

Paul Leroux

“From sensors to smartphones, the car is experiencing a massive influx of new technologies, and automakers must blend these in a way that is simple, helpful, and non-distracting.” That statement comes from a press release we issued a year ago, but it’s as true today as it was then — if not more so. The fact is, the car is undergoing a massive transformation as it becomes more connnected and more automated. And with that transformation comes higher volumes of data and greater system complexity.

But here’s the thing. From the driver’s perspective, this complexity doesn’t matter, nor should it matter. In fact, it can’t matter. Because the driver needs to stay focused on the most important thing: driving. (At least until fully automated driving becomes reality, at which point a nap might be in order!) Consequently, it’s the job of automakers and their suppliers to harness all these technologies in a simple, intuitive way that makes driving easier, safer, and more enjoyable. Specifically, they need to provide the driver with relevant, contextually sensitive information that is easy to consume, without causing distraction.

That is the challenge that the new QNX technology concept vehicle, based on a Toyota Highlander, sets out to explore.

So what are we waiting for? Let’s take a look! (And remember, you can click on any image to magnify it.)

The oh-so-glossy exterior
As with any QNX technology concept vehicle, it’s what’s inside that counts. But to signal that this is no ordinary Highlander, we gave the exterior a luxurious, brushed-metal finish that just screams to have its picture taken. So we obliged:



The integrated display that keeps you focused
When modifying the Highlander, simplicity was the watchword. So instead of equipping the vehicle with both a digital instrument cluster and a head unit, we created a “glass cockpit” that combines the functions of both systems, along with ADAS safety alerts, into one seamless display. Everything is presented directly in front of the driver, where it is easiest to see.

For instance, in the following scenario, the cockpit allows the driver to see several pieces of important information at a glance: a forward-collision warning, an alert that the car is exceeding the local speed limit by 12 mph, and turn-by-turn navigation:



Mind you, the cockpit can display much more information than you see here, including a tachometer, album art, incoming phone calls, and the current radio station. But to keep distraction to a minimum, it displays only the information that the driver currently requires, and no more. Because simplicity.

To further minimize distraction, the cockpit uses voice as the primary way to control the user interface, including control of media, navigation, and phone connectivity. As a result, drivers can access infotainment content while keeping their hands on the wheel and eyes on the road.

Thoughtful touches abound. For instance, the HERE Auto navigation software running in the cockpit interfaces with a HERE Auto Companion App running on a BlackBerry PRIV smartphone. So when the driver steps into the vehicle, navigation route information from the smartphone is transferred automatically to the vehicle, providing a continuous user experience. How cool is that?

Here’s a slightly different view of the cockpit, showing how it can display a photo of your destination — just the thing when you are driving to a location for the first time and would like visual confirmation of what it looks like:



Before I forget, here are some additional tech specs: the cockpit is built on the QNX CAR Platform for Infotainment, uses an interface based on Qt 5.5, integrates iHeartRadio, and runs on a Renesas R-Car H2 system-on-chip.

The acoustics feature that keeps you from shouting
The glass cockpit does a great job of keeping your eyes focused straight ahead. But what’s the use of that if, as a driver, you have to turn your head every time you want to speak to someone in the back seat? If you’ve ever struggled to hold a conversation in a car at highway speeds, especially in a larger vehicle, you know what I’m talking about.

QNX acoustics to the rescue! Earlier today, QNX Software Systems announced the QNX Acoustics Management Platform, a new solution that replaces the traditional piecemeal approach to in-car acoustics with a holistic model that enables faster-time-to-production and lower system costs. The platform comes with several innovative features, including QNX In-Car Communication (ICC) technology, which enhances the voice of the driver and relays it to infotainment loudspeakers in the rear of the car.

Long story short: instead of shouting or having to turn around to be heard, the driver can talk normally while keeping his or her eyes on the road. QNX ICC dynamically adapts to noise conditions and adds enhancement only when needed. Better yet, it allows automakers to leverage their existing handsfree telephony microphones and infotainment loudspeakers.



The reference vehicle that keeps evolving
Before you go, I also want to share some updates to the QNX reference vehicle, which is based on a Jeep Wrangler. Like the Highlander, the Jeep got a slick new exterior for CES 2016:



Since 2012, the Jeep has been our go-to vehicle for showcasing the latest capabilities of the QNX CAR Platform for Infotainment. But for over a year now, it has done double-duty as a concept vehicle, showing how QNX technology can help developers build next-generation instrument clusters and ADAS solutions.

Take, for example, the Jeep’s new instrument cluster, which makes its debut this week at CES. In addition to providing all the information that you’d expect, such as speed and RPM, it displays crosswalk notifications, forward collision warnings, speed limit warnings, and turn-by-turn navigation:



The QNX reference vehicle also includes a full-featured head unit that demonstrates the latest out-of-the-box capabilities of the QNX CAR Platform for Infotainment. For example, in this image, the head unit is displaying HERE Auto navigation:



Other features of the platform include:

• A voice interface that uses natural language processing, making it easy to launch applications, play music, select radio stations, control volume, use the navigation system, and perform a variety of other tasks.
• A new, easy-to-navigate UI based on Qt 5.5 that supports a variety of touch gestures, including tap, swipe, pinch, and zoom.
• QNX acoustics technology that enables clear, easy-to-understand hands-free calls through advanced echo cancellation and noise reduction.
• Cellular connectivity provided by the QNX Wireless Framework, which simplifies system design by managing the complexities of modem control on behalf of applications.
• Flexible support for a variety of smartphone integration protocols.

Additional tech specs: The Jeep’s cluster runs on a Qualcomm Snapdragon 602A processor and its user interface was designed by our partner Rightware, using the Rightware Kanzi tool. The head unit, meanwhile, runs on an Intel Atom E3827 processor.

ADAS, augmented reality, V2X, IoT, and more
I have only scratched the surface of what BlackBerry and QNX Software Systems are demonstrating this week at CES 2016. There’s much more to see and experience, including a very cool V2X demonstration, IoT solutions for the automotive and transportation industries, as well as ADAS and augmented reality systems that integrate with the digital clusters described in this post. To learn more, read the press release that QNX issued today and stay tuned to this channel.

Read More

Ford ports SmartDeviceLink to QNX CAR Platform

QNX joins Ford, Toyota, and other industry leaders to help drive new standard for app integration.

Paul Leroux

For as long as I can remember, QNX Software Systems has been at the forefront of integrating cars and smartphones. Through our flexible OS architecture and large automotive ecosystem, we provide automakers and Tier 1 suppliers with the ultimate choice in connectivity options for smartphones and other smart devices. And now, QNX customers will have even greater choice, with the availability of Ford’s SmartDeviceLink (SDL) technology for the QNX CAR Platform for Infotainment.

If you’ve never heard of SDL, it’s the open source version of Ford AppLink, the software that allows Ford SYNC users to access smartphone apps through voice commands and dashboard controls. Ford donated AppLink to the open source community to create a standard way for consumers to interact with smartphone apps, regardless of which phone they use or vehicle they drive.

SDL is quickly gaining industry advocates, including Toyota, UI Evolution, and, of course QNX. What’s more, companies like PSA, Honda, Subaru, Mazda are evaluating it for use in next-generation vehicles.

Why the interest in SDL? Because it’s a flexible, vendor-neutral standard that can benefit drivers, automakers, and developers alike. With SDL:

• Drivers can interact with apps by using voice commands, steering-wheel buttons, and other in-car controls, so they can keep their eyes on the road and hands on the wheel.
• Automakers can deliver a consistent app experience across vehicles, while retaining the flexibility to customize that experience for each vehicle brand or model.
• Developers can create apps that can work across multiple smart devices and multiple automotive brands — which means they have greater incentive to create automotive apps.

SDL for QNX builds on a history of successful collaborations between Ford and QNX, including the QNX-powered Ford SYNC 3 infotainment system. According to Paul Elsila, CEO of Livio, the Ford subsidiary that maintains the SDL open source project, “With its large market share, QNX can play a key role in driving the adoption of auto industry standards, and we are excited to work with them in building vendor-neutral technology that can simplify the integration of smartphone apps in any brand or type of vehicle.”

SDL works with multiple smartphone platforms. Moreover, it is highly flexible: it can work across a full range of vehicles, from entry-level to premium, and across a wide range of displays. It can even be used in systems without displays — for instance, in systems that use a voice interface.

To learn more about SDL, check out the announcements that Ford, Toyota, and QNX issued this morning.

Read More

The demo is in the details

A new video of the 2015 QNX technology concept car reveals some thoughtful touches.

Paul Leroux

QNX technology concept cars serve a variety of purposes. They demonstrate, for example, how the flexibility of QNX technology can help automakers deliver unique user experiences. They also serve as vehicles — pun fully intended — for showcasing our vision of connected driving. And they explore how thoughtful integration of new technologies can make driving easier and more enjoyable.

It is this thoughtfulness that impresses me most about the cars. It is also the hardest aspect to convey in words and pictures — nothing beats sitting inside one of the cars and experiencing the nuances first hand.

The minute you get behind the wheel, you realize that our concept team is exploring answers to a multitude of questions. For instance, how do you bring more content into a car, without distracting the driver? How do you take types of information previously distributed across two or more screens and integrate them on a single display? How do you combine information about local speed limits with speedometer readouts to promote better driving? How do you make familiar activities, such as using the car radio, simpler and more intuitive? And how much should a car’s UX rely on the touch gestures that have become commonplace on smartphones and tablets?

Okay, enough from me. To see how our 2015 technology concept car, based on a Maserati Quattroporte, addresses these and other questions, check out this new video with my esteemed colleague Justin Moon. Justin does a great job of highlighting many of the nuances I just alluded to:



In just over a month, QNX will unveil a brand new technology concept vehicle. What kinds of questions will it explore? What kinds of answers will it propose? We can’t say too much yet, but stay tuned to this channel and to our CES 2016 microsite.

Read More

The A to Z of QNX in cars

Over 26 fast facts, brought to you by the English alphabet

Paul Leroux

A is for Audi, one of the first automakers to use QNX technology in its vehicles. For more than 15 years, Audi has put its trust in QNX, in state-of-the-art systems like the Audi virtual cockpit and the MIB II modular infotainment system. A is also for QNX acoustics software, which enhances hands-free voice communications, eliminates “boom noise” created by fuel-saving techniques, and even helps automakers create signature sounds for their engines.

B is for Bentley, BMW, and Buick, and for their QNX-powered infotainment systems, which include BMW ConnectedDrive and Buick Intellilink.

C is for concept vehicles, including the latest QNX technology concept car, a modded Maserati Quattroporte GTS. The car integrates an array of technologies — including cameras, LiDAR, ultrasonic sensors, and specialized navigation engines — to show how QNX-based ADAS systems can simplify driving tasks, warn of possible collisions, and enhance driver awareness.

D is for the digital instrument clusters in vehicles from Alpha Romeo, Audi, GM, Jaguar, Mercedes-Benz, and Land Rover. These QNX-powered displays can reconfigure themselves on the fly, providing quick, convenient access to turn-by-turn directions, back-up video, incoming phone calls, and a host of other information.

E is for experience. QNX has served the automotive market since the late 1990s, working with car makers and tier one suppliers to create infotainment systems for tens of millions of vehicles. QNX has been at work in safety-critical industrial applications even longer — since the 1980s. This unique pedigree makes QNX perfectly suited for the next generation of in-vehicle systems, which will consolidate infotainment and safety-related functions on a single, cost-effective platform.

F is for Ford, which has chosen the QNX Neutrino OS for its new SYNC 3 infotainment system. The system will debut this summer in the 2016 Ford Escape and Ford Fiesta and will be one of the first infotainment systems to support both Apple CarPlay and Android Auto.

G is for GM and its QNX-based OnStar system, which is now available in almost all of the company’s vehicles. GM also uses QNX OS and acoustics technology in several infotainment systems, including the award-winning Chevy MyLink.

H is for hypervisor. By using the QNX Hypervisor, automotive developers can consolidate multiple OSs onto a single system-on-chip to reduce the cost, size, weight, and power consumption of their designs. The hypervisor can also simplify safety certification efforts by keeping safety-related and non-safety-related software components isolated from each other.

I is for the ISO 26262 standard for functional safety in road vehicles. The QNX OS for Automotive Safety has been certified to this standard, at Automotive Safety Integrity Level D — the highest level achievable. This certification makes the OS suitable for a wide variety of digital clusters, heads-up displays, and ADAS applications, from adaptive cruise control to pedestrian detection.

J is for Jeep. The QNX reference vehicle, based on a Jeep Wrangler, showcases what the QNX CAR Platform for Infotainment can do out of the box. In its latest iteration, the reference vehicle ups the ante with traffic sign detection, lane departure warnings, curve speed warnings, collision avoidance alerts, backup displays, and other ADAS features for enhancing driver awareness.

K is for Kia, which uses QNX technology in the infotainment and connectivity systems for several of its vehicles.

L is for LG, a long-time QNX customer that is using several QNX technologies to develop a new generation of infotainment systems, digital clusters, and ADAS systems for the global automotive market.

M is for Mercedes-Benz, which offers QNX-based infotainment systems in several of its vehicles, including the head unit and digital instrument cluster in the S Class Coupe. M is also for market share: according to IHS Automotive, QNX commands more than 50% of the infotainment software market.

N is for navigation. Thanks to the navigation framework in the QNX CAR Platform, automakers can integrate a rich variety of navigation solutions into their cars.

O is for the over-the-air update solution of the BlackBerry IoT Platform, which will help automakers cut maintenance costs, reduce expensive recalls, improve customer satisfaction, and keep vehicles up to date with compelling new features long after they have rolled off the assembly line.

P is for partnerships. When automotive companies choose QNX, they also tap into an incredibly rich partner ecosystem that provides infotainment apps, smartphone connectivity solutions, navigation engines, automotive processors, voice recognition engines, user interface tools, and other pre-integrated technologies. P is also for Porsche, which uses the QNX Neutrino OS in its head units, and for Porsche 911, which formed the basis of one of the first QNX concept cars.

Q is for the QNX CAR Platform for Infotainment, a comprehensive solution that pre-integrates partner technologies with road-proven QNX software to jump-start customer projects.

R is for the reliability that QNX OS technology brings to advanced driver assistance systems and other safety-related components in the vehicle — the same technology proven in space shuttles, nuclear plants, and medical devices.

S is for the security expertise and solutions that Certicom and QNX bring to automotive systems. S is also for the advanced smartphone integration of the QNX CAR Platform, which allows infotainment systems to support the latest brought-in solutions, such as Apple CarPlay and Android Auto. S is also for the scalability of QNX technology, which allows customers to use a single software platform across all of their product lines, from high-volume economy vehicles to luxury models. And last, but not least, S is for the more than sixty million vehicles worldwide that use QNX technology. (S sure is a busy letter!)

T is for Toyota, which uses QNX technology in infotainment systems like Entune and Touch ‘n’ Go. T is also for tools: using the QNX Momentics Tool Suite, automotive developers can root out subtle bugs and optimize the performance of their sophisticated, multi-core systems.

U is for unified user interface. With QNX, automotive developers can choose from a rich set of user interface technologies, including Qt, HTML5, OpenGL ES, and third-party toolkits. Better yet, they can blend these various technologies on the same display, at the same time, for the ultimate in design flexibility.

V is for the Volkswagen vehicles, including the Touareg, Passat, Polo, Golf, and Golf GTI, that use the QNX Neutrino OS and QNX middleware technology in their infotainment systems.

W is for the QNX Wireless Framework, which brings smartphone-caliber connectivity to infotainment systems, telematics units, and a variety of other embedded devices. The framework abstracts the complexity of modem control, enabling developers to upgrade cellular and Wi-Fi hardware without having to rewrite their applications.

X, Y, and Z are for the 3D navigation solutions and the 3D APIs and partner toolkits supported by the QNX CAR Platform. I could show you many examples of these solutions in action, but my personal favorite is the QNX technology concept car based on a Bentley Continental GT. Because awesome.

Before you go... This post mentions a number of automotive customers, but please don’t consider it a complete list. I would have gotten them all in, but I ran out of letters!

Read More

Getting in sync with brought-in devices

Building a head unit that needs to sync with smartphones, media players, memory cards, and USB sticks? With the QNX CAR Platform, you won’t be left to your own devices.

Paul Leroux

In previous posts, I discussed how the QNX CAR Platform for Infotainment is adept at juggling multiple concurrent tasks. For instance, it can perform 3D navigation, process voice signals, provide active noise control, display vehicle data, manage audio, run multiple application environments, and still deliver a fast, responsive user experience. If that’s not enough, it can also detect and play content from an array of media devices, including local drives, SD cards, and iPods, as well as Bluetooth, DLNA, and MTP devices.

When plugging a media device into a car’s head unit, most users expect immediate access to the device content; they also want to browse the content by metadata, such as genre, title, or artist. To present this content, the head unit must perform metadata synching. The question is, how can the head unit make the content instantly available, even when the media device contains thousands of files that may take many seconds or even minutes to fully synchronize?

To complicate matters, users often want to switch from one media source to another. For instance, a user listening to music stored on a DLNA device may ask the head unit to switch to an Internet radio station. From the user’s perspective, the switch should be fast, simple, and intuitive.

Handling device attachments (and

detachments) gracefully.

The head unit must also cope with the vagaries of user behavior. For instance, if the user yanks out a USB media stick during synching or playback, the system should recover gracefully; it should also provide appropriate feedback, such as displaying a menu that asks the user to choose from another media source. Likewise, if the user yanks out the media device and re-inserts it, the system shouldn’t get confused. Rather, it should simply resume synching content where it left off.

Handling scenarios like these is the job of the QNX CAR Platform’s multimedia architecture.

Architecture at a glance
The multimedia architecture integrates several software components to automatically detect media devices, synchronize metadata with media databases, browse the contents of devices, and, of course, play audio and video files. Together, these components form three layers:

• Human machine interface, or HMI

• Multimedia components

• OS services

Let’s look at each of these layers in turn, starting with the HMI.

At the top of the HMI layer, you’ll see the Media Player, a reference application that allows end-users to control media browsing and playback. Developers can customize this player or write their own player apps, using APIs provided by the QNX CAR Platform.

The Media Player comes in two flavors, HTML5 and Qt 5. To communicate with the architecture’s multimedia engine (mm-player), the HTML5 version uses the car.mediaplayer JavaScript API while the Qt version uses the QPlayer library. In addition to these interfaces, custom apps can use the multimedia engine’s C API. All three interfaces — car.mediaplayer, QPlayer, and C API — provide an abstraction layer that allows a media player app to:

• retrieve a list of accessible media sources: local drives, USB storage devices, iPods, etc.

• retrieve track metadata: artist name, album name, track title, etc.

• start and stop playback

• jump to a specific track

• handle updates in playback state, media sources, and track position

The interfaces that provide access to these operations aren’t specific to any device type, so player apps can work with a wide variety of media hardware.

The media player can quickly access and display a variety of metadata (artist name, album name, track title, etc.) stored in a small-footprint SQL database.

Multimedia components layer
If you look at the top of the multimedia components layer, you’ll see a box labeled mm-player; this is the architecture’s media browsing and playback engine. The mm-player does the dirty work of retrieving metadata, starting playback, jumping to a specific track, etc., which makes custom player apps easier to design. It also supports a large variety of media sources, including:

• local drives

• USB storage devices

• Apple iPod devices

• DLNA devices, including phones and media players

• MTP devices, including PDAs and media players

• devices paired through Bluetooth

To perform media operations requested by a client media player, mm-player works in concert with several lower-level components that help navigate media-store file systems, read metadata from media files, and manage media flows during playback. The components include a series of plugins (POSIX, AVRCP, DLNA, etc.) that interface with different device types. For instance, let’s say you insert an SD card. The POSIX plugin supports this type of device, so it will learn of the insertion and inform mm-player of the newly connected media source; it will also support any subsequent media operations on the SD card.

If you look again at the diagram, you’ll see several other components that provide services to mm-player. These include:

• mm-detect — discovers media devices and initiates synchronization of metadata
• mm-sync — synchronizes metadata from tracks and playlists on media devices into small-footprint SQL databases called QDB databases
• mm-renderer — plays audio and video tracks, and reports playback state
• io-audio — starts audio device drivers to enable the output of audio streams

OS services layer
The lowest layer of the multimedia architecture includes device drivers and protocol stacks that, among other things, detect whether the user has inserted or removed any media device. The following diagram summarizes what happens when one of these services detects an insertion:

1. User inserts the device.
2. The corresponding driver or protocol stack informs device publishers of the insertion.
3. The publishers write the device information to Persistent Publish Subscribe (PPS) objects in a directory monitored by the mm-detect service. (Read my previous posts here and here to learn how QNX PPS messaging enables loosely coupled, easy-to-extend designs.)
4. To start synchronizing the device’s metadata, mm-detect loads the device’s QDB database into memory and passes the device’s mountpoint and database name to mm-sync.
5. mm-sync synchronizes the metadata of all media files on the device.
6. mm-sync uses media libraries to read file paths and other information from media tracks found on the device. It then copies the extracted metadata into the appropriate database tables and columns. Applications can then query the QDB database to obtain metadata information such as track title and album name.

These steps may describe how the architecture detects and synchronizes with devices, but they can't capture the efficiency of the architecture and how it can deliver a fast, responsive user experience. For that, I invite you to check out this video on the QNX CAR Platform. The section on multimedia synchronization starts at the 1:32 mark, but I encourage you to watch the whole thing to see how the platform performs multimedia operations while concurrently managing other tasks:



Media browsing and playback
I’ve touched on how the multimedia architecture automatically detects and synchronizes devices. But of course, it does a lot more, including media browsing and media playback. To learn more about these features, visit the QNX CAR Platform documentation on the QNX website.

---

Previous posts in the QNX CAR Platform series:
 

• A question of getting there — wherein I examine how the platform gives customers the flexibility to choose from a variety of navigation solutions

• A question of architecture — wherein I discuss how the platform simplifies the challenge of integrating multiple disparate technologies, from graphics to silicon

• A question of concurrency — wherein I address the a priori question: why does the auto industry need a platform like QNX CAR in the first place?

Read More

Long time, no see: Catching up with the QNX CAR Platform

By Megan Alink, Director of Marketing Communications for Automotive

It’s a fact — a person simply can’t be in two places at one time. I can’t, you can’t, and the demo team at QNX can’t (especially when they’re brainstorming exciting showcase projects for 2016… but that’s another blog. Note to self.) So what’s a QNX-loving, software-admiring, car aficionado to do when he or she has lost touch and wants to see the latest on the QNX CAR Platform for Infotainment? Video, my friends.

One of the latest additions to our QNX Cam YouTube channel is an update to a video made just over two and a half years ago, in which my colleague, Sheridan Ethier, took viewers on a feature-by-feature walkthrough of the QNX CAR Platform. Now, Sheridan’s back for another tour, so sit back and enjoy a good, old-fashioned catch-up with what’s been going on with our flagship automotive product (with time references, just in case you’re in a bit of a hurry).

Sheridan Ethier hits the road in the QNX reference vehicle based on a modified Jeep Wrangler, running the latest QNX CAR Platform for Infotainment.

We kick things off with a look at one of the most popular elements of an infotainment system — multimedia. Starting around the 01:30 mark, Sheridan shows how the QNX CAR Platform supports a variety of music formats and media sources, from the system’s own multimedia player to a brought-in device. And when your passenger is agitating to switch from the CCR playlist on your MP3 device to Meghan Trainor on her USB music collection, the platform’s fast detection and sync time means you’ll barely miss a head-bob.

The QNX CAR Platform’s native multimedia player — the “juke box” — is just one of many options for enjoying your music.

About five minutes in, we take a look at how the QNX CAR Platform implements voice recognition. Whether you’re seeking out a hot latté, navigating to the nearest airport, or calling a co-worker to say you’ll be a few minutes late, the QNX CAR Platform lets you do what you want to do while doing what you need to do — keeping your hands on the wheel and your eyes on the road. Don’t miss a look at concurrency (previously discussed here by Paul Leroux) during this segment, when Sheridan runs the results of his voice commands (multimedia, navigation, and a hands-free call) smoothly at the same time.

Using voice recognition, users can navigate to a destination by address or point of interest description (such as an airport).

At eight minutes, Sheridan tells us about one of the best examples of the flexibility of the QNX CAR Platform — its support for application environments, including native C/C++, Qt, HTML5, and APK for running Android applications. The platform’s audio management capability makes a cameo appearance when Sheridan switches between the native multimedia player and the Pandora HTML5 app.

Pandora is just one of the HTML5 applications supported by the QNX CAR Platform.

As Sheridan tells us (at approximately 12:00), the ability to project smartphone screens and applications into the vehicle is an important trend in automotive. With technologies like MirrorLink, users can access nearly all of the applications available on their smartphone right from the head unit.

Projection technologies like MirrorLink allow automakers to select which applications will be delivered to the vehicle’s head unit from the user’s connected smartphone. 

Finally, we take a look at two interesting features that differentiate the QNX CAR Platform — last mode persistence (e.g. when the song you were listening to when you turned the car off starts up at the same point when you turn the car back on) and fastboot (which, in the case of QNX CAR, can bring your backup camera to life in 0.8 seconds, far less than the NHTSA-mandated 2 seconds). These features work hand-in-hand to ensure a safer, more enjoyable, more responsive driving experience.

Fastboot in 0.8 seconds means that when you’re ready to reverse, your car is ready to show you the way.

Interested in learning more about the QNX CAR Platform for Infotainment? Check out Paul Leroux’s blog on the architecture of this sophisticated piece of software. To see QNX CAR in action, read Tina Jeffrey’s blog, in which she talks about how the platform was implemented in the reimagined QNX reference vehicle for CES 2015.

Check out the video here:

Read More

A behind the scenes look at creating an integrated driving experience

Lynn Gayowski

To quote the timeless lyrics of Britney Spears, “You want a Maserati? You better work.” This is exactly what the QNX team did to get our 2015 technology concept car ready for this year’s CES. And we had the cameras rolling throughout the build process.

The video below not only gives a behind the scenes look at the making of our latest technology concept car based on a Maserati Quattroporte GTS, but it features team members talking about the technology behind the car and what QNX brings to the table (or garage in this case) to enable a customized car in mere months.

Yes, the QNX CAR Platform has cool features and amazing reliability. But another draw for our customers is the platform’s pre-integrated partner technologies. The platform gets silicon, apps, and services working together so OEMs don’t have to solve this problem for themselves. It makes development time shorter and helps the focus to stay on branding the user experience. As Alex — one of the software engineers interviewed — mentions, after seeing what we did in the Maserati, “Just imagine what our customers can do!”

We targeted an integrated driving experience for this vehicle and I think this focus is evident in the finished product. The user interface and ADAS features are intuitive, and let’s be real, gorgeous. Check out this video that summarizes the making of the 2015 QNX technology concept car:



You’ll see many members of the concept team working hard throughout this video, but a shout out as well to all of the developers who contributed to the QNX CAR Platform, QNX operating system, and acoustics technologies that made this amazing vehicle possible. Congratulations to all of you for a job well done!

Read More

Now with ADAS: The revamped QNX reference vehicle

Tina Jeffrey

Since 2012, our Jeep has showcased what QNX technology can do out of the box. We decided it was time to up the ante...

I walked into the QNX garage a few weeks ago and did a double take. The QNX reference vehicle, a modified Jeep Wrangler, had undergone a major overhaul both inside and out — and just in time for 2015 CES.

Before I get into the how and why of the Jeep’s metamorphosis, here’s a glimpse of its newly refreshed exterior. Orange is the new gray!



The Jeep debuted in June 2012 at Telematics Detroit. Its purpose: to show how customers can use off-the-shelf QNX products, like the QNX CAR Platform for Infotainment and QNX OS, to build a wide range of custom infotainment systems and instrument clusters, using a single code base.

From day one, the Jeep has been a real workhorse, making appearances at numerous events to showcase the latest HMI, navigation, speech recognition, multimedia, and handsfree acoustics technologies, not to mention embedded apps for parking, internet radio streaming, weather, and smartphone connectivity. The Jeep has performed dependably time and time again, and now, in an era where automotive safety is top of mind, we’ve decided to up the ante and add leading-edge ADAS technology built on the QNX OS.

After all, what sets the QNX OS apart is its proven track record in safety-certified systems across market segments — industrial, medical, and automotive. In fact, the QNX OS for Automotive Safety is certified to the highest level of automotive functional safety: ISO 26262, ASIL D. Using a pre-certified OS component is key to the overall integrity of an automotive system and makes system certification much easier.

The ultimate (virtual) driving experience
How better to showcase ADAS in the Jeep, than by a virtual drive? At CES, a 12-foot video screen in front of the Jeep plays a pre-recorded driving scene, while the onboard ADAS system analyzes the scene to detect lane markers, speed signs, and preceding vehicles, and to warn of unintentional lane departures, excessive speed, and imminent crashes with vehicles on the road ahead. Onboard computer vision algorithms from Itseez process the image frames in real time to perform these functions simultaneously.

Here’s a scene from the virtual drive, in which the ADAS system is tracking lane markings and has detected a speed-limit sign:



If the vehicle begins to drift outside a lane, the steering wheel provides haptic feedback and the cluster displays a warning:



The ADAS system includes Elektrobit EB Assist eHorizon, which uses map data with curve-speed information to provide warnings and recommendations, such as reducing your speed to navigate an upcoming curve:



The Jeep also has a LiDAR system from Phantom Intelligence (formerly Aerostar) to detect obstacles on the road ahead. The cluster displays warnings from this system, as well as warnings from the vision-based collision-detection feature. For example:



POSTSCRIPT:
Here’s a short video of the virtual drive, taken at CES by Brandon Lewis of Embedded Computing Design, in which you can see curve-speed warnings and lane-departure warnings:



Fast-boot camera
Rounding out the ADAS features is a rear-view camera demo that can cold boot in 0.8 seconds on a Texas Instruments Jacinto 6 processor. As you may recall, NHTSA has mandated that, by May 2018, most new vehicles must have rear-view technology that can display a 10-by-20 foot area directly behind the vehicle; moreover, the display must appear no more than 2 seconds after the driver throws the vehicle into reverse. Backup camera and other fastboot requirements such as time-to-last-mode audio, time-to-HMI visible, and time-to-fully-responsive HMI are critically important to automakers. Be sure to check out the demo — but don’t blink or you’ll miss it!

Full-featured infotainment
The head unit includes a full-featured infotainment system based on the QNX CAR Platform for Infotainment and provides information such as weather, current song, and turn-by-turn directions to the instrument cluster, where they’re easier for the driver to see.



Infotainment features include:

Qt-based HMI — Can integrate other HMI technologies, including Elektrobit EB Guide and Crank Storyboard.

Natural language processing (NLP) — Uses Nuance’s Vocon Hybrid solution in concert with the QNX NLP technology for natural interaction with infotainment functions. For instance, if you ask “Will I need a jacket later today?”, the Weather Network app will launch and provide the forecast.

EB street director — Provides embedded navigation with a 3D map engine; the map is synched up with the virtual drive during the demo.

QNX CAR Platform multimedia engine — An automotive-hardened solution that can handle:

• audio management for seamless transitions between all audio sources
• media detection and browsing of connected devices
• background synching of music for instant media playback — without the need for the synch to be completed

Support for all smartphone connectivity options — DLNA, MTP, MirrorLink, Bluetooth, USB, Wi-Fi, etc.

On-board application framework — Supports Qt, HTML5, APK (for Android apps), and native OpenGL ES apps. Apps include iHeart, Parkopedia, Pandora, Slacker, and Weather Network, as well as a Settings app for phone pairing, over-the-air software updates, and Wi-Fi hotspot setup.

So if you’re in the North Hall at CES this week, be sure to take a virtual ride in the QNX reference vehicle in Booth 2231. Beneath the fresh paint job, it’s the same workhorse it has always been, but now with new ADAS tech automakers are thirsting for.

Read More

QNX and Qualcomm Technologies give show goers another stunner at 2015 CES

Guest post By Nilesh Parekh, Director of Product Management at Qualcomm Technologies

Year after year, CES attendees are repeatedly amazed by the advances in automobile infotainment. Not so long ago, it was about having a great stereo in the car and maybe a tiny screen in the center stack with a primitive navigation system. Then came Bluetooth connectivity… multiple multimedia screens… front and rear displays… gaming… 3G and 4G connectivity… Wi-Fi hotspots…

This year, QNX Software Systems and Qualcomm Technologies are bringing you something really special — a “mashup,” you could say, of a Maserati Quattroporte GTS, the QNX OS, the QNX CAR Platform, and the Snapdragon™ Automotive Solutions (SAS) platform, all working together in a show-stopping technology concept car.



The QNX concept team worked closely with Qualcomm Technologies to create an immersive in-vehicle experience using advanced technologies for infotainment, digital instrument clusters, and driver assistance systems. These systems feature high-resolution UIs with multi-touch support, 3D graphics for navigation, and LiDAR-based obstacle detection. And note the side mirrors have been swapped for smart displays that eliminate typical vehicle blind spots and present relevant color-coded overlay information to promote safer driving.

Inner beauty

Admittedly, the car is a thing of beauty. But being in the tech field, I find the real beauty inside the car — deep inside. There, working hand-in-hand with the field-proven QNX OS, is the Snapdragon Automotive Solutions (SAS) platform. The SAS platform manages all infotainment features; it also processes vital vehicle safety information, collected via a myriad of camera, ultrasonic, and LIDAR sensors, and delivers all relevant information to the driver in real time — that’s a lot of computing and processing power.

What’s so special about the SAS platform? First, let me define what it is (put on your tech hats): a highly integrated, thermal-efficient automotive-grade platform that incorporates an optimized combination of CPU, GPU, 4G LTE modem, GPS/GNSS, Bluetooth, and Wi-Fi. What’s special is that it is engineered to not only enhance the driver and passenger experiences with the infotainment features we know today, but it future-proofs the vehicle for next-generation features — some of which haven’t even been dreamt of yet. More important, it allows automakers and tier ones to accelerate development schedules and to focus on creating feature-rich, reliable infotainment and safety systems built with solutions such as the QNX CAR Platform.

Let’s take a closer look at the three areas of this special technology concept car where I think the presence of SAS makes the biggest impact: the instrument cluster, the infotainment system, and the driver assistance system. And keep in mind that this vehicle is more than a showcase of what’s “out there” and possible — it’s a test bed we’ll use to gain relevant experience and knowledge that we can apply to future technologies in real cars.

The all-digital, reconfigurable instrument cluster
The cluster — the go-to information display for drivers — on the technology concept car can cycle through a number of views, providing the driver with relevant data on what’s going on, in, and around the car in real time. Rear-view park assist, current audio track, navigation data, forward-collision warnings, and vehicle data are all examples of information rendered in the cluster:



The infotainment system
You can’t help but notice the 12” portrait touchscreen next to the instrument cluster. The system is built using the QNX CAR Platform for Infotainment — an automotive-hardened software platform built on the QNX OS. The QNX CAR Platform runs on the SAS hardware and implements a sophisticated UI design that supports voice recognition, touch (including tap, swipe, and pinch and zoom on the map), and synchronizes with the rear-seat control system, allowing rear-seat passengers to manage navigation, song selection, and temperature settings.

Here's a photo of the touchscreen in action. As you can see, it's displaying map info, an incoming call, and a "Now Playing" section. If you simply tap the map, which is powered by Elektrobit (EB) street director navigation and works with EB electronic horizon, it will automatically take over two-thirds of the screen:



Driver assistance system
The car's driver assistance system makes use of LIDAR and ultrasonic sensors to detect the presence of obstacles around the vehicle and renders warning information to the driver through the cluster or side-view displays, and also through an obstacle awareness system made up of dashboard LEDs. This system projects color-coded warnings onto the windshield to indicate the location and proximity of the object.

Other highlights include:

• “Always On” rear-view display — The rear-view mirror has been converted into a display that renders a wide-angle perspective of the area behind the car
• Elektrobit electronic horizon — Topographical map data is used to provide curve-speed recommendations and warnings that are displayed in the cluster

If you have the opportunity to see this car at CES, I highly recommend it — it really is an amazing technology concept vehicle that showcases the next-generation of automobile infotainment and safety. It will be located in the Qualcomm booth located in Central Plaza #21A Jan 6-9. If you cannot make it to CES, you can learn more here.

Read More

One day I’ll be Luke Skywalker

Cyril Clocher

What happens when you blend ADAS with infotainment? Guest post by Cyril Clocher, business manager for automotive processors at Texas Instruments

As we all begin preparing for our trek to Vegas for CES 2015, I would like my young friends (born in the 70s, of course) to reflect on their impressions of the first episode of Lucas’s trilogy back in 1977. On my side, I perfectly remember thinking one day I would be Luke Skywalker.

The eyes of young boys and girls were literally amazed by this epic space opera and particularly by technologies used by our heroes to fight the Galactic Empire. You have to remember it was an era where we still used rotary phones and GPS was in its infancy. So you can imagine how impactful it was for us to see our favorite characters using wireless electronic gadgets with revolutionary HMIs such as natural voice recognition, gesture controls or touch screens; droids speaking and enhancing human intelligence; and autonomous vehicles traveling the galaxy safely while playing chess with a Wookiee. Now you’re with me…

But instead of becoming Luke Skywalker a lot of us realized that we would have a bigger impact by inventing or engineering these technologies and by transforming early concepts into real products we all use today. As a result, smartphones and wireless connectivity are now in our everyday lives; the Internet of Things (IoT) is getting more popular in applications such as activity trackers that monitor personal metrics; and our kids are more used to touch screens than mice or keyboards, and cannot think of on-line gaming without gesture control. In fact, I just used voice recognition to upgrade the Wi-Fi plan from my Telco provider.

But the journey is not over yet. Our generation has still to deliver an autonomous vehicle that is green, safe, and fun to control – I think the word “drive” will be obsolete for such a vehicle.

The automotive industry has taken several steps to achieve this exciting goal, including integration of advanced and connected in-car infotainment systems in more models as well as a number of technologies categorized under Advanced Driver Assistance Systems (ADAS) that can create a safer and unique driving experience. From more than a decade, Texas Instruments has invested in infotainment and ADAS: “Jacinto” and TDAx automotive processors as well as the many analog companion chips supporting these trends.

"Jacinto 6 EP" and "Jacinto 6 Ex"
infotainment processors

A unique approach of TI is our capability to leverage best of both worlds for non-safety critical features, and to provide a seamless integration of informational ADAS functions into existing infotainment systems so the vehicle better informs and warns the driver. We announced that capability at SAE Convergence in Detroit in October 2014 with the “Jacinto 6 Ex” processor (DRA756), which contains powerful CPU, graphics multimedia, and radio cores with differentiated vision co-processors, called embedded vision engines (EVE), and additional DSPs that perform the complex ADAS processing.

For the TI’s automotive team, the CES 2015 show is even more exciting than in previous years, as we’ve taken our concept of informational ADAS to the next step. With joint efforts and hard work from both TI and QNX teams, we’ve together implemented a real informational ADAS system running the QNX CAR™ Platform for Infotainment on a “Jacinto 6 Ex” processor.

I could try describing this system in detail, but just like the Star Wars movies, it’s best to experience our “Jacinto 6 Ex” and QNX CAR Platform-based system in person. Contact your TI or QNX representative today and schedule a meeting to visit our private suite at CES at the TI Village (N115-N119) or to immerse yourself in a combined IVI, cluster, megapixel surround view, and DLP® based HUD display with augmented reality running on a single “Jacinto 6 Ex” SoC demonstration. And don't forget to visit the QNX booth (2231), where you can see the QNX reference vehicle running a variety of ADAS and infotainment applications on “Jacinto 6” processors.

Integrated cockpit featuring DLP powered HUD and QNX CAR Platform running on a single “Jacinto 6 Ex” SoC.

One day I’ll experience Skywalker’s life as I will no doubt have the opportunity to control an intelligent and autonomous vehicle with my biometrics, voice, and gestures while riding with my family to the movie theater playing chess with my grandkids, not yet a Wookiee.

Read More

A question of getting there

The third of a series of posts on the QNX CAR Platform. In this installment, we turn to a key point of interest: the platform’s navigation service.

From the beginning, we designed the QNX CAR Platform for Infotainment with flexibility in mind. Our philosophy is to give customers the freedom to choose the hardware platforms, application environments, user-interface tools, and smartphone connectivity protocols that best address their requirements. This same spirit of flexibility extends to navigation solutions.

For evidence, look no further than our current technology concept car. It can support navigation from Elektrobit:



from Nokia HERE:



and from Kotei Informatics:



These are but a few examples. The QNX CAR Platform can also support navigation solutions from companies like AISIN AW, NavNGo, TCS, TeleNav, and ZENRIN DataCom, enabling automakers and automotive Tier 1 suppliers to choose the navigation solution, or solutions, best suited to the regions or demographics they wish to target. (In addition to these embedded solutions, the platform can also provide access to smartphone-based navigation services through its support for MirrorLink and other connectivity protocols — more on this in a subsequent post.)

Under the hood
In our previous installment, we looked at the QNX CAR Platform’s middleware layer, which provides infotainment applications with a variety of services, including Bluetooth, radio, multimedia discovery and playback, and automatic speech recognition. The middleware layer also includes a navigation service that, true to the platform’s overall flexibility, allows developers to use navigation engines from multiple vendors and to change engines without affecting the high-level navigation applications that the user interacts with.

An illustration is in order. If you look the image below, you’ll see OpenGL-based map data rendered on one graphics layer and, on the layer above it, Qt-based application data (current street, distance to destination, and other route information) pulled from the navigation engine. By taking advantage of the platform’s navigation service, you could swap in a different navigation engine without having to rewrite the Qt application:



To achieve this flexibility, the navigation service makes use of the QNX CAR Platform’s persistent/publish subscribe (PPS) messaging, which cleanly abstracts lower-level services from the higher-level applications they communicate with. Let's look at another diagram to see how this works:



In the PPS model, services publish information to data objects; other programs can subscribe to those objects and receive notifications when the objects have changed. So, for the example above, the navigation engine could generate updates to the route information, and the navigation service could publish those updates to a PPS “navigation status object,” thereby making the updates available to any program that subscribes to the object — including the Qt application.

With this approach, the Qt application doesn't need to know anything about the navigation engine, nor does the navigation engine need to know anything about the Qt app. As a result, either could be swapped out without affecting the other.

Here's another example of how this model allows components to communicate with one another:

1. Using the system's human machine interface (HMI), the drivers asks the navigation system to search for a point of interest (POI) — this could take the form of a voice command or a tap on the system display.
2. The HMI responds by writing the request to a PPS “navigation control” object.
3. The navigation service reads the request from the PPS object and forwards it to the navigation engine.
4. The navigation engine returns the result.
5. The navigation service updates the PPS object to notify the HMI that its request has been completed. It also writes the results to a database so that all subscribers to this object can read the results.

By using PPS, the navigation service can make details of the route available to a variety of applications. For instance, it could publish trip information that a weather app could subscribe to. The app could then display the weather forecast for the destination, at the estimated time of arrival.

To give developers a jump start, the QNX CAR Platform comes pre-integrated with Elektrobit’s EB street director navigation software. This reference integration shows developers how to implement "command and control" between the HMI and the participating components, including the navigation engine, navigation service, window manager, and PPS interface. As the above diagram indicates, the reference implementation works with both of the HMIs — one based on HTML5, the other based on Qt — that the QNX CAR Platform supports out of the box.

---

Previous posts in the QNX CAR Platform series:

• A question of architecture
  
• A question of concurrency

Read More

Japan update: ADAS, wearables, integrated cockpits, and autonomous cars

Yoshiki Chubachi

Will the joy of driving be a design criterion for tomorrow’s vehicles? It had better be.

A couple of weeks ago, QNX Software Systems sponsored Telematics Japan in Tokyo. This event offers a great opportunity to catch up with colleagues from automotive companies, discuss technology and business trends, and showcase the latest technology demos. Speaking of which, here’s a photo of me with a Japan-localized demo of the QNX CAR Platform. You can also see a QNX-based digital instrument cluster in the lower-left corner — this was developed by Three D, one of our local technology partners:



While at the event, I spoke on the panel, “Evolving ecosystems for future HMI, OS, and telematics platform development.” During the discussion, we conducted a real-time poll and asked the audience three questions:

1) Do you think having Apple CarPlay and Android Auto will augment a vehicle brand?
2) Do you expect wearable technologies to be integrated into cars?
3) If your rental car were hacked, who would you complain to?

For question 1, 32% of the audience said CarPlay and Android Auto will improve a brand; 68% didn't think so. In my opinion, this result indicates that smartphone connectivity in cars is now an expected feature. For question 2, 76% answered that they expect to see wearables integrated into cars. This response gives us a new perspective — people are looking at wearables as a possible addition to go with ADAS systems. For example, a wearable device could help prevent accidents by monitoring the driver for drowsiness and other dangerous signs. For question 3, 68% said they would complain to the rental company. Mind you, this raises the question: if your own car were hacked, who would you complain to?

Integrated cockpits
There is growing concern around safety and security as companies attempt to grow more business by leveraging connectivity in cars. The trend is apparent if you look at the number of safety- and security-related demos at various automotive shows.

Case in point: I recently attended a private automotive event hosted by Renesas, where many ADAS and integrated cockpit demos were on display. And last month, CEATEC Japan (aka the CES of Japan) featured integrated cockpit demos from companies like Fujitsu, Pioneer, Mitsubishi, Kyocera, and NTT Docomo.

For the joy of it
Things are so different from when I first started developing in-car navigation systems 20 years ago. Infotainment systems are now turning into integrated cockpits. In Japan, the automotive industry is looking at early 2020s as the time when commercially available autonomous cars will be on the road. In the coming years, the in-car environment, including infotainment, cameras and other systems, will change immensely — I’m not exactly sure what cars in the year 2020 will look like, but I know it will be something I could never have imagined 20 years ago.

A panel participant at Telematics Japan said to me, “If autonomous cars become reality and my car is not going to let me drive anymore, I am not sure what the point of having a car is.” This is true. As we continue to develop for future cars, we may want to remind ourselves of the “joy of driving” factor.

Read More