Top 5 challenges of digital instrument clusters

Guest post by Olli Laiho, director, product marketing, Rightware

Digitalization of the modern car is progressing at breakneck speed, with research showing that over 70% of cars will ship with a digital display in the cluster by 2017 (Automotive User Interfaces 2014, IHS Automotive, 2014). While digital user interfaces have long been available in the center stack of the vehicle, they are now quickly making their way into the heart of the car’s dashboard — the instrument cluster. However, the migration from traditional, physical instrumentation to the digital Human Machine Interface (HMI) is posing various challenges for auto manufacturers. Here are the top five challenges Rightware is seeing today.

1. Deliver a winning user experience
With the digital cluster, auto manufacturers must deliver a user experience that makes consumers insist on having a digital cluster and makes them think they could never live without one. The car companies need to increase their investment in digital user experience design in order to provide consumers with a digital driving experience they’ll love.

User experience is all about... the user! With the help of target group research, auto manufacturers need to find the key use cases and features for different buyer profiles. While more senior buyers appreciate a digital design featuring traditional big gauges and needles combined with maps in the middle, millennials long for a cluster that connects them with their personal data at the right time, while having a modern look and feel with a real wow effect.

QNX Software Systems' technology concept car 2014 based on the Mercedes CLA 45, featuring a cluster created with Rightware Kanzi^®

2. Find the right design-cost-performance combination
In creating HMIs such as digital clusters, finding the right balance among design, cost, and performance becomes essential. It’s all about:

Design — Delivering a stunning user experience
Cost — Minimizing software development, hardware, and maintenance costs
Performance — Choosing the right OS, System-on-a-Chip (SoC), etc.

Automotive user interface designers need to learn to work with the capabilities of the hardware and software platform of the cluster in mind. Designers need to create user experiences that strengthen the auto manufacturer’s brand image while still being possible to implement with the chosen tool chain and hardware and software platforms.

Choosing the SoC that can deliver the best user experience at the best price is essential. While proper automotive SoC benchmarking tools are not yet available in the market, auto manufacturers need to invest in their own measurements and trials for finding the right cost/performance level of the SoC for their project.

QNX Software Systems' technology concept car 2015 based on the Maserati Quattroporte, showing
system diagnostics in the cluster created with Rightware Kanzi

3. Reduce development time
Consumers have become accustomed to having access to the latest technology and innovations on their mobile devices. That expectation has now extended to HMIs in the car.

To meet consumer expectations, the automotive industry must shorten the development time of new vehicles and determine how to provide compelling software upgrades during the car’s lifecycle. Digital clusters need to be designed for upgradeability from the ground up. Through upgrades, the cluster should provide the necessary access to new app platforms and innovations. Streamlining the software development process and choosing the right tool chain for HMI development is key to creating HMIs faster and with more valuable features.

4. Accelerate update cycles
Consumers utilize their mobile devices daily and have learned to expect a constant update cycle that brings new features and enhancements to their device. This “update drug” has created a trend where the customer is waiting for the next update to their beloved devices — a customer that is always looking for more.

Until today, there have been few tangible software upgrades for a car during its lifetime. As an example, when you pick up your car from service, you’ll often see a line on the bill that says “software updates.” Leaving the garage, you can discern no difference in how the car behaves.

Auto manufacturers need a plan for providing consumers with constant software upgrades that give them value during the entire lifecycle of their vehicle. Upgrading the digital cluster doesn’t have to mean that it should look like next year’s model, but the upgrade should provide consumers with either features that add value or a clear, visual difference that they understand is an upgrade. Increasing the upgradeability of HMIs in the car will be a major opportunity for improving customer retention.

5. Establish design ownership
As automotive devices evolve into the digital age, they will also transform the way auto manufacturers create designs for their customers. Unlike a mobile device, HMI design will be specific not only to the manufacturer’s brand, but also to that model. Digital screens will give automotive UI designers the flexibility to create unique designs, and they will need full control of the UI framework to be able to deliver these stunning user experiences.

Consumers are increasingly connected 24/7 to ecosystems from companies such as Google and Apple. Due to the increase in consumer demand, these technologies are also making their way into the car cockpit in various forms — from simple content integration (SMS, mail, media) to sandboxed but comprehensive solutions like Apple CarPlay and Android Auto.

Automotive companies must invest in creating branded digital user experiences that can rival and exceed any third-party designs in the vehicle. They should invest in a UI solution and operating system that can deliver the design as intended.

Audi Q7 Virtual Cockpit, running on QNX Neutrino OS, featuring a cluster created with Rightware Kanzi

Visit Rightware at TU-Automotive Detroit (booth #C115) to witness next-generation HMI demos built with Kanzi and a first chance to see a brand new Kanzi product. You’ll also find Rightware’s technology in the QNX booth (#C92).

---

Olli Laiho has been working in software development for over 15 years. An avid car enthusiast, Olli heads Rightware’s global marketing activities.

The Rightware Kanzi UI Solution and the QNX Neutrino OS can already be found together in several vehicles, including the Audi TT, Audi Q7, and the Audi R8. Rightware has created several digital clusters for QNX technology concept cars, including the 2014 Mercedes CLA 45 and the 2015 Maserati Quattroporte.

Visit Rightware here.

Read More

Getting in sync with brought-in devices

Building a head unit that needs to sync with smartphones, media players, memory cards, and USB sticks? With the QNX CAR Platform, you won’t be left to your own devices.

Paul Leroux

In previous posts, I discussed how the QNX CAR Platform for Infotainment is adept at juggling multiple concurrent tasks. For instance, it can perform 3D navigation, process voice signals, provide active noise control, display vehicle data, manage audio, run multiple application environments, and still deliver a fast, responsive user experience. If that’s not enough, it can also detect and play content from an array of media devices, including local drives, SD cards, and iPods, as well as Bluetooth, DLNA, and MTP devices.

When plugging a media device into a car’s head unit, most users expect immediate access to the device content; they also want to browse the content by metadata, such as genre, title, or artist. To present this content, the head unit must perform metadata synching. The question is, how can the head unit make the content instantly available, even when the media device contains thousands of files that may take many seconds or even minutes to fully synchronize?

To complicate matters, users often want to switch from one media source to another. For instance, a user listening to music stored on a DLNA device may ask the head unit to switch to an Internet radio station. From the user’s perspective, the switch should be fast, simple, and intuitive.

Handling device attachments (and

detachments) gracefully.

The head unit must also cope with the vagaries of user behavior. For instance, if the user yanks out a USB media stick during synching or playback, the system should recover gracefully; it should also provide appropriate feedback, such as displaying a menu that asks the user to choose from another media source. Likewise, if the user yanks out the media device and re-inserts it, the system shouldn’t get confused. Rather, it should simply resume synching content where it left off.

Handling scenarios like these is the job of the QNX CAR Platform’s multimedia architecture.

Architecture at a glance
The multimedia architecture integrates several software components to automatically detect media devices, synchronize metadata with media databases, browse the contents of devices, and, of course, play audio and video files. Together, these components form three layers:

• Human machine interface, or HMI

• Multimedia components

• OS services

Let’s look at each of these layers in turn, starting with the HMI.

At the top of the HMI layer, you’ll see the Media Player, a reference application that allows end-users to control media browsing and playback. Developers can customize this player or write their own player apps, using APIs provided by the QNX CAR Platform.

The Media Player comes in two flavors, HTML5 and Qt 5. To communicate with the architecture’s multimedia engine (mm-player), the HTML5 version uses the car.mediaplayer JavaScript API while the Qt version uses the QPlayer library. In addition to these interfaces, custom apps can use the multimedia engine’s C API. All three interfaces — car.mediaplayer, QPlayer, and C API — provide an abstraction layer that allows a media player app to:

• retrieve a list of accessible media sources: local drives, USB storage devices, iPods, etc.

• retrieve track metadata: artist name, album name, track title, etc.

• start and stop playback

• jump to a specific track

• handle updates in playback state, media sources, and track position

The interfaces that provide access to these operations aren’t specific to any device type, so player apps can work with a wide variety of media hardware.

The media player can quickly access and display a variety of metadata (artist name, album name, track title, etc.) stored in a small-footprint SQL database.

Multimedia components layer
If you look at the top of the multimedia components layer, you’ll see a box labeled mm-player; this is the architecture’s media browsing and playback engine. The mm-player does the dirty work of retrieving metadata, starting playback, jumping to a specific track, etc., which makes custom player apps easier to design. It also supports a large variety of media sources, including:

• local drives

• USB storage devices

• Apple iPod devices

• DLNA devices, including phones and media players

• MTP devices, including PDAs and media players

• devices paired through Bluetooth

To perform media operations requested by a client media player, mm-player works in concert with several lower-level components that help navigate media-store file systems, read metadata from media files, and manage media flows during playback. The components include a series of plugins (POSIX, AVRCP, DLNA, etc.) that interface with different device types. For instance, let’s say you insert an SD card. The POSIX plugin supports this type of device, so it will learn of the insertion and inform mm-player of the newly connected media source; it will also support any subsequent media operations on the SD card.

If you look again at the diagram, you’ll see several other components that provide services to mm-player. These include:

• mm-detect — discovers media devices and initiates synchronization of metadata
• mm-sync — synchronizes metadata from tracks and playlists on media devices into small-footprint SQL databases called QDB databases
• mm-renderer — plays audio and video tracks, and reports playback state
• io-audio — starts audio device drivers to enable the output of audio streams

OS services layer
The lowest layer of the multimedia architecture includes device drivers and protocol stacks that, among other things, detect whether the user has inserted or removed any media device. The following diagram summarizes what happens when one of these services detects an insertion:

1. User inserts the device.
2. The corresponding driver or protocol stack informs device publishers of the insertion.
3. The publishers write the device information to Persistent Publish Subscribe (PPS) objects in a directory monitored by the mm-detect service. (Read my previous posts here and here to learn how QNX PPS messaging enables loosely coupled, easy-to-extend designs.)
4. To start synchronizing the device’s metadata, mm-detect loads the device’s QDB database into memory and passes the device’s mountpoint and database name to mm-sync.
5. mm-sync synchronizes the metadata of all media files on the device.
6. mm-sync uses media libraries to read file paths and other information from media tracks found on the device. It then copies the extracted metadata into the appropriate database tables and columns. Applications can then query the QDB database to obtain metadata information such as track title and album name.

These steps may describe how the architecture detects and synchronizes with devices, but they can't capture the efficiency of the architecture and how it can deliver a fast, responsive user experience. For that, I invite you to check out this video on the QNX CAR Platform. The section on multimedia synchronization starts at the 1:32 mark, but I encourage you to watch the whole thing to see how the platform performs multimedia operations while concurrently managing other tasks:



Media browsing and playback
I’ve touched on how the multimedia architecture automatically detects and synchronizes devices. But of course, it does a lot more, including media browsing and media playback. To learn more about these features, visit the QNX CAR Platform documentation on the QNX website.

---

Previous posts in the QNX CAR Platform series:
 

• A question of getting there — wherein I examine how the platform gives customers the flexibility to choose from a variety of navigation solutions

• A question of architecture — wherein I discuss how the platform simplifies the challenge of integrating multiple disparate technologies, from graphics to silicon

• A question of concurrency — wherein I address the a priori question: why does the auto industry need a platform like QNX CAR in the first place?

Read More

The CLA 45 has landed!

Megan Alink

Europe, your day has come! After five years of showcasing our technology concept cars primarily in North America, we’ve bid farewell to the Mercedes CLA 45 and sent it across the pond to our colleagues in Germany. Over the coming year while the Mercedes resides in Europe, our customers — and anyone who’s just mesmerized by slick, pre-integrated automotive tech — will have a chance to check the car out at a number of public events. (Stay tuned to www.qnx.com for more details as these events arise.)

Witness the unboxing:

The CLA 45 emerges into the light at Bremerhaven.

On land and settling in nicely.

So beautiful! We can't wait for a whole new continent to see it for themselves.

Interested in a sneak peek at the inside of this gorgeous vehicle? Read this blog from Lynn Gayowski, or get up close and personal with the digital instrument cluster in this one from Paul Leroux. For more photos, see our Flickr album.

Read More

Keeping it fresh for 35 years

By Megan Alink, Director of Marketing Communications for Automotive

Recently, my colleagues Paul Leroux and Matt Young showed off a shiny new infographic that enlightens readers to the many ways they encounter QNX-based systems in daily life (here and here). After three-and-a-half decades in business we’ve certainly been around the block a time or two, and you might think things are getting a bit stale. As the infographic shows, that couldn’t be further from the truth here at QNX. From up in the stars to down on the roads; in planes, trains, and automobiles (and boats too); whether you’re mailing a letter or crafting a BBM on your BlackBerry smartphone, the number and breadth of applications in which our customers deploy QNX technology is simply astounding.

For those who like some sound with their pictures, we also made a video to drive home the point that, wherever you are and whatever you do, chances are you’ll encounter a little QNX. Check it out:

Read More

Long time, no see: Catching up with the QNX CAR Platform

By Megan Alink, Director of Marketing Communications for Automotive

It’s a fact — a person simply can’t be in two places at one time. I can’t, you can’t, and the demo team at QNX can’t (especially when they’re brainstorming exciting showcase projects for 2016… but that’s another blog. Note to self.) So what’s a QNX-loving, software-admiring, car aficionado to do when he or she has lost touch and wants to see the latest on the QNX CAR Platform for Infotainment? Video, my friends.

One of the latest additions to our QNX Cam YouTube channel is an update to a video made just over two and a half years ago, in which my colleague, Sheridan Ethier, took viewers on a feature-by-feature walkthrough of the QNX CAR Platform. Now, Sheridan’s back for another tour, so sit back and enjoy a good, old-fashioned catch-up with what’s been going on with our flagship automotive product (with time references, just in case you’re in a bit of a hurry).

Sheridan Ethier hits the road in the QNX reference vehicle based on a modified Jeep Wrangler, running the latest QNX CAR Platform for Infotainment.

We kick things off with a look at one of the most popular elements of an infotainment system — multimedia. Starting around the 01:30 mark, Sheridan shows how the QNX CAR Platform supports a variety of music formats and media sources, from the system’s own multimedia player to a brought-in device. And when your passenger is agitating to switch from the CCR playlist on your MP3 device to Meghan Trainor on her USB music collection, the platform’s fast detection and sync time means you’ll barely miss a head-bob.

The QNX CAR Platform’s native multimedia player — the “juke box” — is just one of many options for enjoying your music.

About five minutes in, we take a look at how the QNX CAR Platform implements voice recognition. Whether you’re seeking out a hot latté, navigating to the nearest airport, or calling a co-worker to say you’ll be a few minutes late, the QNX CAR Platform lets you do what you want to do while doing what you need to do — keeping your hands on the wheel and your eyes on the road. Don’t miss a look at concurrency (previously discussed here by Paul Leroux) during this segment, when Sheridan runs the results of his voice commands (multimedia, navigation, and a hands-free call) smoothly at the same time.

Using voice recognition, users can navigate to a destination by address or point of interest description (such as an airport).

At eight minutes, Sheridan tells us about one of the best examples of the flexibility of the QNX CAR Platform — its support for application environments, including native C/C++, Qt, HTML5, and APK for running Android applications. The platform’s audio management capability makes a cameo appearance when Sheridan switches between the native multimedia player and the Pandora HTML5 app.

Pandora is just one of the HTML5 applications supported by the QNX CAR Platform.

As Sheridan tells us (at approximately 12:00), the ability to project smartphone screens and applications into the vehicle is an important trend in automotive. With technologies like MirrorLink, users can access nearly all of the applications available on their smartphone right from the head unit.

Projection technologies like MirrorLink allow automakers to select which applications will be delivered to the vehicle’s head unit from the user’s connected smartphone. 

Finally, we take a look at two interesting features that differentiate the QNX CAR Platform — last mode persistence (e.g. when the song you were listening to when you turned the car off starts up at the same point when you turn the car back on) and fastboot (which, in the case of QNX CAR, can bring your backup camera to life in 0.8 seconds, far less than the NHTSA-mandated 2 seconds). These features work hand-in-hand to ensure a safer, more enjoyable, more responsive driving experience.

Fastboot in 0.8 seconds means that when you’re ready to reverse, your car is ready to show you the way.

Interested in learning more about the QNX CAR Platform for Infotainment? Check out Paul Leroux’s blog on the architecture of this sophisticated piece of software. To see QNX CAR in action, read Tina Jeffrey’s blog, in which she talks about how the platform was implemented in the reimagined QNX reference vehicle for CES 2015.

Check out the video here:

Read More

“What do you mean, I have to learn how not to drive?”

The age of autonomous driving lessons is upon us.

Paul Leroux

What would it be like to ride in an autonomous car? If you were to ask the average Joe, he would likely describe a scenario in which he sips coffee, plays video games, and spends quality time with TSN while the car whisks him to work. The average Jane would, no doubt, provide an equivalent answer. The problem with this scenario is that autonomous doesn’t mean driverless. Until autonomous vehicles become better than humans at handling every potential traffic situation, drivers will have to remain alert much or all of the time, even if their cars do 99.9% of the driving for them.

Otherwise, what happens when a car, faced with a situation it can’t handle, suddenly cedes control to the driver? Or what happens when the car fails to recognize a pedestrian on the road ahead?

Of course, it isn’t easy to maintain a high level of alertness while doing nothing in particular. It takes a certain maturity of mind, or at least a lack of ADD. Which explains why California, a leader in regulations for autonomous vehicles, imposes restrictions on who is allowed to “drive” them. Prerequisites include a near-spotless driving record and more than 10 years without a DUI conviction. Drivers must also complete an autonomous driving program, the length of which depends on the car maker or automotive supplier in question. According to a recent investigation by IEEE Spectrum, Google offers the most comprehensive program — it lasts five weeks and subjects drivers to random checks.

1950s approach to improving driver
alertness. Source: Modern Mechanix blog

In effect, drivers of autonomous cars have to learn how not to drive. And, as another IEEE article suggests, they may even need a special license.

Ample warnings
Could an autonomous car mitigate the attention issue? Definitely. It could, for example, give the driver ample warning before he or she needs to take over. The forward collision alerts and other informational ADAS functions in the latest QNX technology concept car offer a hint as to how such warnings could operate. For the time being, however, it’s hard to imagine an autonomous car that could always anticipate when it needs to cede control. Until then, informational ADAS will serve as an adjunct, not a replacement, for eyes, ears, and old-fashioned attentiveness.

Nonetheless, research suggests that adaptive cruise control and other technologies that enable autonomous or semi-autonomous driving can, when compared to human drivers, do a better job of avoiding accidents and improving traffic flow. To quote my friend Andy Gryc, autonomous cars would be more “polite” to other vehicles and be better equipped to negotiate inter-vehicle space, enabling more cars to use the same length of road.

Fewer accidents, faster travel times. I could live with that.


2015 approach to improving driver alertness: instrument cluster from the QNX reference vehicle.

Read More

To infotainment... and beyond! First look at new QNX technology concept car

The new car delivers everything you’d expect in a concept vehicle from QNX. But the real buzz can be summarized in a four-letter word: ADAS

The technology in today's cars is light-years ahead of the technology in cars 10 or 20 years ago. The humans driving those cars, however, have changed little in the intervening years. They still need to focus on a host of mundane driving tasks, from checking blind spots and monitoring road signs to staying within the lane and squeezing into parking spaces. In fact, with all the technology now in the car, including a variety of brought-in devices, some drivers suffer from information overload and perform worse, instead of better, at these crucial tasks.

Advanced driver assistance systems, or ADAS, can go a long way to offset this problem. They come in a variety of shapes and sizes — from drowsiness monitoring to autonomous emergency braking — but most share a common goal: to help the driver avoid accidents.

Which brings us to the new QNX technology concept car. As you’d expect, it includes all the advanced infotainment features, including smartphone connectivity and rich app support, offered by the QNX CAR Platform for Infotainment. But it also integrates an array of additional technologies — including cameras, LiDAR, ultrasonic sensors, and specialized navigation software — to deliver ADAS capabilities that simplify driving tasks, warn of possible collisions, and enhance overall driver awareness.

Mind you, the ADAS features shouldn’t come as any more of a surprise than the infotainment features. After all, QNX Software Systems also offers the QNX OS for Automotive Safety, a solution based on decades of experience in safety-critical systems and certified to ISO 26262, Automotive Safety Integrity Level D — the highest level achievable.

Okay, enough blather. Time to check out the car!

The “I want that” car
If the trident hasn’t already tipped you off, the new technology concept car is based on a Maserati QuattroPorte GTS. I won’t say much about the car itself, except I want one. Did I say want? Sorry, I meant lust. Because omigosh:



The differentiated dash
Before we run through the car’s many features, let’s stop for a minute and savor the elegant design of its QNX-powered digital instrument cluster and infotainment system. To be honest, I have an ulterior motive for sharing this image: if you compare the systems shown here to those of previous QNX technology concept cars (here, here, and here), you’ll see that they each project a distinct look-and-feel. Automakers need to differentiate themselves, and, as a group, these cars illustrate how the flexibility of the QNX platform enables unique, branded user experiences:



The multi-talented digital instrument cluster
Okay, let’s get behind the wheel and test out the digital cluster. Designed to heighten driver awareness, the cluster can show the current speed limit, display an alert if you exceed the limit, and even recommend an appropriate speed for upcoming curves. Better yet, it can display turn-by-turn directions provided by the car’s infotainment system.

Normally, the cluster displays the speed limit in a white circle. But in this image, the cluster displays it in red, along with a red bar to show how much you are over the limit — a gentle reminder to ease off the gas:



Using LiDAR input, the cluster can also warn of obstacles on the road ahead:



And if that’s not enough, the cluster provides intelligent parking assist to help you back into tight spaces. Here, for example, is an impromptu image we took in the QNX garage. The blue-and-yellow guidelines represent the car’s reverse trajectory, and the warning on right says that you are about to run over an esteemed member of the QNX concept team!



The rear- and side-view mirrors that aren’t really mirrors
By their very nature, car mirrors have blind spots. To address this problem, the QNX concept team has transformed the car’s rear- and side-view mirrors into video displays that offer a complete view of the scene behind and to the sides of the vehicle. As you can see in this image, the side-view displays can also display a red overlay to warn of cars, bikes, people, or anything else approaching the car’s blind zones:



The ADAS display for enhancing obstacle awareness
I don’t have pictures yet, but the car also includes an innovative LED-based display lets you gauge the direction and proximity of objects to the front, rear, and sides of the vehicle — without having to take your eyes off the road. Stretching the width of the dash, the display integrates input from the car’s ultrasonic and LiDAR sensors to provide a centralized view of ADAS warnings.

The easy-to-use infotainment system
To demonstrate the capabilities of the QNX CAR™ Platform for Infotainment, we’ve outfitted the car with a feature-rich, yet intuitive, multimedia head unit. For instance, see the radio tuner in the following image? That’s no ordinary tuner. To change channels, you can just swipe across the display; if your swipe isn’t perfectly accurate, the radio will automatically zero in on the nearest station or preset.

Better yet, the radio offers “iHeart drive anywhere radio.” If you drive out of range of your favorite AM/FM radio station, the system will detect the problem and automatically switch to the corresponding digital iHeartRadio station. How cool is that?



Other infotainment features include:

• Natural voice recognition — For instance, if you say “It’s way too cold in here,” the HVAC system will respond by raising the heat.
• Integration with a wide variety of popular smartphones.
• Support for multiple concurrent app environments, along with a variety of Android and HTML5 apps, as well as an HMI built with the Qt framework.
• A backseat display that lets passengers control HVAC functions, navigation, song selection, and other infotainment features.

The oh-so-awesome partners
The car is a testament not only to QNX technology, but to the ecosystem of technology partners that provide complementary solutions for QNX customers. Peek under the hood, and you'll find the latest tech from Elektrobit, iHeart, Nuance, Pandora, Parkopedia, Phantom Intelligence, Qualcomm, RealVNC, Rightware, and TE Connectivity.

The other stuff
Do not, for one minute, think that the Maserati is the only attraction in the QNX booth. Far from it. We will also showcase a significantly revamped QNX reference vehicle, outfitted with lane departure warnings, traffic sign recognition, and other ADAS features, as well as the latest version of the QNX CAR Platform — more in an upcoming post.

Visitors to the booth will also have the opportunity to experience:

• a 3D navigation solution from Aisin AW
• a digital instrument cluster designed by HI Corporation
• two QNX CAR Platform demo systems, one powered by a dual-core Intel Atom E3827 processor, the other by an NVIDIA Tegra Visual Computing Module
• the latest incarnation of the Oscar-winning Flying Cam SARAH aerial camera system

Read More

One day I’ll be Luke Skywalker

Cyril Clocher

What happens when you blend ADAS with infotainment? Guest post by Cyril Clocher, business manager for automotive processors at Texas Instruments

As we all begin preparing for our trek to Vegas for CES 2015, I would like my young friends (born in the 70s, of course) to reflect on their impressions of the first episode of Lucas’s trilogy back in 1977. On my side, I perfectly remember thinking one day I would be Luke Skywalker.

The eyes of young boys and girls were literally amazed by this epic space opera and particularly by technologies used by our heroes to fight the Galactic Empire. You have to remember it was an era where we still used rotary phones and GPS was in its infancy. So you can imagine how impactful it was for us to see our favorite characters using wireless electronic gadgets with revolutionary HMIs such as natural voice recognition, gesture controls or touch screens; droids speaking and enhancing human intelligence; and autonomous vehicles traveling the galaxy safely while playing chess with a Wookiee. Now you’re with me…

But instead of becoming Luke Skywalker a lot of us realized that we would have a bigger impact by inventing or engineering these technologies and by transforming early concepts into real products we all use today. As a result, smartphones and wireless connectivity are now in our everyday lives; the Internet of Things (IoT) is getting more popular in applications such as activity trackers that monitor personal metrics; and our kids are more used to touch screens than mice or keyboards, and cannot think of on-line gaming without gesture control. In fact, I just used voice recognition to upgrade the Wi-Fi plan from my Telco provider.

But the journey is not over yet. Our generation has still to deliver an autonomous vehicle that is green, safe, and fun to control – I think the word “drive” will be obsolete for such a vehicle.

The automotive industry has taken several steps to achieve this exciting goal, including integration of advanced and connected in-car infotainment systems in more models as well as a number of technologies categorized under Advanced Driver Assistance Systems (ADAS) that can create a safer and unique driving experience. From more than a decade, Texas Instruments has invested in infotainment and ADAS: “Jacinto” and TDAx automotive processors as well as the many analog companion chips supporting these trends.

"Jacinto 6 EP" and "Jacinto 6 Ex"
infotainment processors

A unique approach of TI is our capability to leverage best of both worlds for non-safety critical features, and to provide a seamless integration of informational ADAS functions into existing infotainment systems so the vehicle better informs and warns the driver. We announced that capability at SAE Convergence in Detroit in October 2014 with the “Jacinto 6 Ex” processor (DRA756), which contains powerful CPU, graphics multimedia, and radio cores with differentiated vision co-processors, called embedded vision engines (EVE), and additional DSPs that perform the complex ADAS processing.

For the TI’s automotive team, the CES 2015 show is even more exciting than in previous years, as we’ve taken our concept of informational ADAS to the next step. With joint efforts and hard work from both TI and QNX teams, we’ve together implemented a real informational ADAS system running the QNX CAR™ Platform for Infotainment on a “Jacinto 6 Ex” processor.

I could try describing this system in detail, but just like the Star Wars movies, it’s best to experience our “Jacinto 6 Ex” and QNX CAR Platform-based system in person. Contact your TI or QNX representative today and schedule a meeting to visit our private suite at CES at the TI Village (N115-N119) or to immerse yourself in a combined IVI, cluster, megapixel surround view, and DLP® based HUD display with augmented reality running on a single “Jacinto 6 Ex” SoC demonstration. And don't forget to visit the QNX booth (2231), where you can see the QNX reference vehicle running a variety of ADAS and infotainment applications on “Jacinto 6” processors.

Integrated cockpit featuring DLP powered HUD and QNX CAR Platform running on a single “Jacinto 6 Ex” SoC.

One day I’ll experience Skywalker’s life as I will no doubt have the opportunity to control an intelligent and autonomous vehicle with my biometrics, voice, and gestures while riding with my family to the movie theater playing chess with my grandkids, not yet a Wookiee.

Read More

A question of getting there

The third of a series of posts on the QNX CAR Platform. In this installment, we turn to a key point of interest: the platform’s navigation service.

From the beginning, we designed the QNX CAR Platform for Infotainment with flexibility in mind. Our philosophy is to give customers the freedom to choose the hardware platforms, application environments, user-interface tools, and smartphone connectivity protocols that best address their requirements. This same spirit of flexibility extends to navigation solutions.

For evidence, look no further than our current technology concept car. It can support navigation from Elektrobit:



from Nokia HERE:



and from Kotei Informatics:



These are but a few examples. The QNX CAR Platform can also support navigation solutions from companies like AISIN AW, NavNGo, TCS, TeleNav, and ZENRIN DataCom, enabling automakers and automotive Tier 1 suppliers to choose the navigation solution, or solutions, best suited to the regions or demographics they wish to target. (In addition to these embedded solutions, the platform can also provide access to smartphone-based navigation services through its support for MirrorLink and other connectivity protocols — more on this in a subsequent post.)

Under the hood
In our previous installment, we looked at the QNX CAR Platform’s middleware layer, which provides infotainment applications with a variety of services, including Bluetooth, radio, multimedia discovery and playback, and automatic speech recognition. The middleware layer also includes a navigation service that, true to the platform’s overall flexibility, allows developers to use navigation engines from multiple vendors and to change engines without affecting the high-level navigation applications that the user interacts with.

An illustration is in order. If you look the image below, you’ll see OpenGL-based map data rendered on one graphics layer and, on the layer above it, Qt-based application data (current street, distance to destination, and other route information) pulled from the navigation engine. By taking advantage of the platform’s navigation service, you could swap in a different navigation engine without having to rewrite the Qt application:



To achieve this flexibility, the navigation service makes use of the QNX CAR Platform’s persistent/publish subscribe (PPS) messaging, which cleanly abstracts lower-level services from the higher-level applications they communicate with. Let's look at another diagram to see how this works:



In the PPS model, services publish information to data objects; other programs can subscribe to those objects and receive notifications when the objects have changed. So, for the example above, the navigation engine could generate updates to the route information, and the navigation service could publish those updates to a PPS “navigation status object,” thereby making the updates available to any program that subscribes to the object — including the Qt application.

With this approach, the Qt application doesn't need to know anything about the navigation engine, nor does the navigation engine need to know anything about the Qt app. As a result, either could be swapped out without affecting the other.

Here's another example of how this model allows components to communicate with one another:

1. Using the system's human machine interface (HMI), the drivers asks the navigation system to search for a point of interest (POI) — this could take the form of a voice command or a tap on the system display.
2. The HMI responds by writing the request to a PPS “navigation control” object.
3. The navigation service reads the request from the PPS object and forwards it to the navigation engine.
4. The navigation engine returns the result.
5. The navigation service updates the PPS object to notify the HMI that its request has been completed. It also writes the results to a database so that all subscribers to this object can read the results.

By using PPS, the navigation service can make details of the route available to a variety of applications. For instance, it could publish trip information that a weather app could subscribe to. The app could then display the weather forecast for the destination, at the estimated time of arrival.

To give developers a jump start, the QNX CAR Platform comes pre-integrated with Elektrobit’s EB street director navigation software. This reference integration shows developers how to implement "command and control" between the HMI and the participating components, including the navigation engine, navigation service, window manager, and PPS interface. As the above diagram indicates, the reference implementation works with both of the HMIs — one based on HTML5, the other based on Qt — that the QNX CAR Platform supports out of the box.

---

Previous posts in the QNX CAR Platform series:

• A question of architecture
  
• A question of concurrency

Read More

A question of architecture

The second of a series on the QNX CAR Platform. In this installment, we start at the beginning — the platform’s underlying architecture.

In my previous post, I discussed how infotainment systems must perform multiple complex tasks, often all at once. At any time, a system may need to manage audio, show backup video, run 3D navigation, synch with Bluetooth devices, display smartphone content, run apps, present vehicle data, process voice signals, perform active noise control… the list goes on.

The job of integrating all these functions is no trivial task — an understatement if ever there was one. But as with any large project, starting with the right architecture, the right tools, and the right building blocks can make all the difference. With that in mind, let’s start at the beginning: the underlying architecture of the QNX CAR Platform for Infotainment.

The architecture consists of three layers: human machine interface (HMI), middleware, and platform.



The HMI layer
The HMI layer is like a bonus pack: it supports two reference HMIs out of the box, both of which have the same appearance and functionality. So what’s the difference? One is based on HTML5, the other on Qt 5. This choice demonstrates the underlying flexibility of the platform, which allows developers to create an HMI with any of several technologies, including HTML5, Qt, or a third-party toolkit such as Elektrobit GUIDE or Crank Storyboard.

A choice of HMIs

Mind you, the choice goes further than that. When you build a sophisticated infotainment system, it soon becomes obvious that no single tool or technology can do the job. The home screen, which may contain controls for Internet radio, hands-free calls, HVAC, and other functions, might need an environment like Qt. The navigation app, for its part, will probably use OpenGL ES. Meanwhile, some applications might be based on Android or HTML5. Together, all these heterogeneous components make up the HMI.

The QNX CAR Platform embraces this heterogeneity, allowing developers to use the best tools and application environments for the job at hand. More to the point, it allows developers to blend multiple app technologies into a single, unified user interface, where they can all share the same display, at the same time.

To perform this blending, the platform employs several mechanisms, including a component called the graphical composition manager . This manager acts as a kind of universal framework, providing all applications, regardless of how they’re built, with a highly optimized path to the display.

For example, look at the following HMI:



Now look at the HMI from another angle to see how it comprises several components blended together by the composition manger:



To the left, you see video input from a connected media player or smartphone. To the right, you see a navigation application based on OpenGL ES map-rendering software, with an overlay of route metadata implemented in Qt. And below, you see an HTML page that provides the underlying wallpaper; this page could also display a system status bar and UI menu bar across all screens.

For each component rendered to the display, the graphical composition manager allocates a separate window and frame buffer. It also allows the developer to control the properties of each individual window, including location, transparency, rotation, alpha, brightness, and z-order. As a result, it becomes relatively straightforward to tile, overlap, or blend a variety of applications on the same screen, in whichever way creates the best user experience.

The middleware layer
The middleware layer provides applications with a rich assortment of services, including Bluetooth, multimedia discovery and playback, navigation, radio, and automatic speech recognition (ASR). The ASR component, for example, can be used to turn on the radio, initiate a Bluetooth phone call from a connected smartphone, or select a song by artist or song title.

I’ll drill down into several of these services in upcoming posts. For now, I’d like to focus on a fundamental service that greatly simplifies how all other services and applications in the system interact with one another. It’s called persistent/publish subscribe messaging, or PPS, and it provides the abstraction needed to cleanly separate high-level applications from low-level business logic and services.

PPS messaging provides an abstraction layer between system services and high-level applications

Let’s rewind a minute. To implement communications between software components, C/C++ developers must typically define direct, point-to-point connections that tend to “break” when new features or requirements are introduced. For instance, an application communicates with a navigation engine, but all connections enabling that communication must be redefined when the system is updated with a different engine.

This fragility might be acceptable in a relatively simple system, but it creates a real bottleneck when you are developing something as complex, dynamic, and quickly evolving as the design for a modern infotainment system. PPS addresses the problem by allowing developers to create loose, flexible connections between components. As a result, it becomes much easier to add, remove, or replace components without having to modify other components.

So what, exactly, is PPS? Here’s a textbook answer: an asynchronous object-based system that consists of publishers and subscribers, where publishers modify the properties of data objects and the subscribers to those objects receive updates when the objects have been modified.

So what does that mean? Well, in a car, PPS data objects allow applications to access services such as the multimedia engine, voice recognition engine, vehicle buses, connected smartphones, hands-free calling, and contact databases. These data objects can each contain multiple attributes, each attribute providing access to a specific feature — such as the RPM of the engine, the level of brake fluid, or the frequency of the current radio station. System services publish these objects and modify their attributes; other programs can then subscribe to the objects and receive updates whenever the attributes change.

The PPS service is programming-language independent, allowing programs written in a variety of programming languages (C, C++, HTML5, Java, JavaScript, etc.) to intercommunicate, without any special knowledge of one another. Thus, an app in a high-level environment like HTML5 can easily access services provided by a device driver or other low-level service written in C or C++.

I’m only touching on the capabilities of PPS. To learn more, check out the QNX documentation on this service.

The platform layer
The platform layer includes the QNX OS and the board support packages, or BSPs, that allow the OS to run on various hardware platforms.

An inherently modular and extensible architecture

A BSP may not sound like the sexiest thing in the world — it is, admittedly, a deeply technical piece of software — but without it, nothing else works. And, in fact, one reason QNX Software Systems has such a strong presence in automotive is that it provides BSPs for all the popular infotainment platforms from companies like Freescale, NVIDIA, Qualcomm, and Texas Instruments.

As for the QNX Neutrino OS, you could write a book about it — which is another way of saying it’s far beyond the scope of this post. Suffice it to say that its modularity, extensibility, reliability, and performance set the tone for the entire QNX CAR Platform. To get a feel for what the QNX OS brings to the platform (and by extension, to the automotive industry), I invite you to visit the QNX Neutrino OS page on the QNX website.

Read More

Attending SAE Convergence? Here’s why you should visit booth 513

Cars and beer don’t mix. But discussing cars while having a beer? Now you’re talking. If you’re attending SAE Convergence next week, you owe it to yourself to register for our “Spirits And Eats” event at 7:00 pm Tuesday. It’s the perfect occasion to kick back and enjoy the company of people who, like yourself, are passionate about cars and car electronics. And it isn’t a bad networking opportunity either — you’ll meet folks from a variety of automakers, Tier 1s, and technology suppliers in a relaxed, convivial atmosphere.

But you know what? It isn’t just about the beer. Or the company. It’s also about the Benz. Our digitally modded Mercedes-Benz CLA45 AMG, to be exact. It’s the latest QNX technology concept car, and it’s the perfect vehicle (pun fully intended) for demonstrating how QNX technology can enable next-generation infotainment systems. Highlights include:

• A multi-modal user experience that blends touch, voice, and physical controls
• A secure application environment for Android, HTML5, and OpenGL ES
• Smartphone connectivity options for projecting smartphone apps onto the head unit
• A dynamically reconfigurable digital instrument cluster that displays turn-by-turn directions, notifications of incoming phone calls, and video from front and rear cameras
• Multimedia framework for playback of content from USB sticks, DLNA devices, etc.
• Full-band stereo calling — think phone calls with CD quality audio
• Engine sound enhancement that synchronizes synthesized engine sounds with engine RPM

Here, for example, is the digital cluster:



And here is a closeup of the head unit:



And here’s a shot of the cluster and head unit together:



As for the engine sound enhancement and high-quality hands-free audio, I can’t reproduce these here — you’ll have come see the car and experience them first hand. (Yup, that's an invite.)

If you like what you see, and are interested in what you can hear, visit us at booth #513. And if you'd like to schedule a demo or reserve some time with a QNX representative in advance, we can accommodate that, too. Just send us an email.

Read More