From Concept to Reality: BlackBerry-QNX's Groundbreaking CES Tradition

Thomas Bloor
Business Development Manager, QNX BlackBerry

The annual Consumer Electronics Show in Las Vegas has been growing in importance for the automotive industry over the years. You can hardly fail to notice that this year, as in previous years, the big automakers vie for floor space and attention with the glut of big screen TVs and other consumer goods. As always, BlacBerry QNX will be in the North Hall, proudly in the middle of the big automotive OEMs. 

At CES BlackBerryQNX has an enviable history of bringing concept cars that rival anything on the show floor, with one important difference – ours are not pure flights of fancy, and we show technologies that will become realities in the near future.

We started this trend back in 2010 with an LTE-connected Toyota Prius – 18 months before the first commercial LTE deployment in mid-2011. Working with Alcatel-Lucent to provide the experimental network, we demonstrated Google maps functionality with local search and an embedded Pandora radio app in a car for the first time. Connectivity is standard in many cars today, but in 2010 we demonstrated the future.

2012 brought us a CNET "Best of CES" award for demonstrating cloud-based natural language voice recognition, text-to-speech, and NFC based one-touch Bluetooth pairing.  Simply touching your phone to an NFC reader in the center console automatically paired the phone and car. 

In 2013 we got ahead of the trend for ever larger center stack displays – with detailed 3D maps and voice recognition Keyword Spotting – common today in smartphones but a first in a car. Simply saying "Hello Bentley" enabled you to start interacting with the natural language cloud based voice recognition Powered by AT&T’s Watson. 

2014 took literally us in a different direction. A 21-inch horizontally orientated center stack display extends across the dash, naturally extending the interaction and functionality towards the passenger.  Behind the screens the instrument cluster was integrated with the center stack running both driver information and IVI functions. With seamless controllability across the touch screen, physical buttons, and the jog wheel controls multi-modal input was highlighted across all available functionality. 

Not content with that, we foreshadowed greater integration of ADAS functionality warnings to the driver. In 2014 we warned the driver if local speed limits were exceeded through both the cluster and verbally through text-to-speech, and we followed this up in 2015 with a system that recommends an appropriate speed for upcoming curves based upon driving conditions and the radius of the bend.

So, what innovations will we be showing in 2017? I’m not allowed to tell you just yet but, in a first (for us), we’ll be showing both future and current production technologies and innovations.

Building on our products ranging from in-car acoustics through our comprehensive QNX-CAR application platform and to next generation driver assistance/autonomous drive we will be demonstrating how technology can enhance the user experience and increase safety for drivers and passengers.

While demonstrating technologies that will come to future production vehicles, these cars are not just "show floor wonders" because our automotive knowledge enables us to build demonstrators for the real world, which can be driven, thus allowing technologies to be experienced first-hand.

Read More

The Automotive Shift to Software-Defined, Consolidated Controller Architectures

Thomas Bloor

Automotive Business Development Manager
BlackBerry

Electronics in the car have come a long way from the first in-car radio in 1930, and 1978 when Mercedes-Benz introduced the first production car with an optional electronic four-wheel multi-channel anti-lock braking system (ABS) from Bosch.

Today, according to Manfred Broy, a professor at the Technical University of Munich, the cost of electronics and software has increased to 30% of a car's bill of materials. He estimates that 90% of new innovations now come from electronic systems in the car, and projections indicate the cost of electronics will surpass 50% in ten years as we move towards more advanced driver assist and fully autonomous functions in the car. These costs are driven by the electronic architecture of the modern car. 

If we were to examine a modern luxury vehicle we would find a very complex interconnected network of between 60 to 100 electronic control units (ECUs) in aggregate running between 6 to 8 different operating systems with around 100 million lines of code. The distributed automotive computing architecture has evolved over many product generations as new features and innovations have been added though new hardware modules. This approach has served the industry well, but being faced with rising costs from more complex infotainment and new driver assist systems the existing model is becoming inefficient and a drag on bringing new features and innovations into the car. Complexity presents serious challenges, not the least of which are safety and security.

Additionally, the vision of urban mobility that will utilize fully electric autonomous cars has brought new competitors to the industry. These new entrants are rooted in hi-tech and are entering the automotive industry with the opportunity to architect the car from a clean sheet. Unencumbered by the burdensome legacy of traditional automotive manufacturers face,  the newcomers are architecting vehicle systems by leveraging advances in silicon technology to make designs with a smaller number of consolidated controllers with larger processing capabilities.

The evolution to these new consolidated controllers will provide a number of cost benefits. According to a study by Roland Berger associates, consolidated controllers will provide $110 of direct cost savings from hardware consolidation alone. An additional $65 of secondary savings will come from a reduction in software licensing and tools. The study did not quantify savings from software reuse, but positioned reuse as the unseen bulk of the iceberg under the water. As software development costs are often the largest single item in terms of time and manpower, a development savings in the software domain can dwarf the $175 of savings quantified by the researchers.

So, both the new entrants and established companies within the industry are moving towards a domain or area controller architecture – consolidating functions into a smaller number of more flexible processing nodes within the vehicles architecture. This consolidation opens the possibility of reducing the number of operating systems in the car to three or four and the total number of controllers to between six to ten.

This enables a more flexible architecture with a high degree of reuse of the software code base between the different domain/area controllers. In turn this allows the complete vehicle architecture to be built in a flexible manner, enabling the same code base to be reused across generations and differing variants within a model range. 

This more efficient vehicle architecture provides benefits such as reducing the number and duplication in development tools and the associated costs. Developers benefit as deeper experience in a smaller number of operating systems will reduce training requirements and improve developer skills and efficiency.  Reducing overall complexity also improves security and safety, because fewer attack surfaces are presented to a hacker with malicious intent, and  it focuses resources to find and fix security vulnerabilities in a smaller number of operating systems.

So, you can probably see that the auto industry is entering a revolutionary period in vehicle architecture.  Vehicle electronics will consolidate, and with that automakers and Tier 1s have the opportunity to build consolidated, adaptable software environments to speed time to market and enable multiple model variants to be derived from a common code base.  To realize the benefits of these changes a software architecture that is applicable across the majority of vehicle functions is desired. 

The choice of an operating system that enables increased reuse and has proven quality and reliability provides a valuable foundation upon which this architecture is built.  Wide applicability guides the choice to an operating system that is capable of providing mission-critical reliability and security for advanced driver assist (ADAS)and autonomous drive functions, while also being capable of underpinning consumer facing infotainment solutions.

BlackBerry's QNX subsidiary has a long history of underpinning the majority of autmotive infotainment systems in production today.  That is in no small part because QNX's common code base supports both safety OS and infotainment requirements, which provides an advantage in developing, reusing, and productizing code across safety and non-safety certified domains.   QNX recognizes that automakers may want to build mixed ASIL environments in their consolidated controllers as well as consumer infotainment offerings such as those from Google or other sources, so we built QNX’s hypervisor solution.

Availability of safety certification on the hypervisor with no changes adds flexibility and reduces development costs as ASIL certification can be completed after the code is partitioned between controllers, knowing that the underlying software complies with ISO26262. This enables a cockpit controller running a cluster application to have mixed ASIL A and B partitions in its software, and combine these in different informational zones in the same display. The obvious example being cluster gauges and navigational maps being displayed side by side with both partitions being run on a single processor.

Running this type of mixed environment requires full separation and isolation between domains and a safety certified hypervisor solution.  With QNX’s safe, secure, and reliable software solutions you can build an adaptable and dependable vehicle architecture. With safety certification available without the need for code base changes you can develop flexibly with the knowledge of being able to achieve ASIL certification where required.

The changes in vehicle architectures towards more consolidated domain/areas controllers enables am evolution in the software development methodology for the car, namely a shift to a continuous platform development methodology that will enable automakers to compete with new entrants from the high-tech world.  The new architectures will also allow the increased complexity of vehicle systems evolve safety and securely as we move towards the fully autonomous car. 
 

The ability to drive a flexible architecture and derive multiple vehicle platforms from a common code base and set of hardware controllers will be a competitive advantage to automakers and Tier 1s who embrace this approach.  They seem to already know that. Additional benefits of this approach include cost reduction and time to market acceleration stemming from reuse and improved security through the elimination of attack surfaces and the ability to embed cryptographic countermeasures security into the more capable controllers and ECUs.

This may be quite a challenge but we see many automakers and Tier 1s today accepting the challenge and adopting these practices.

Read More

In Car Communications (“ICC”) Solution Check List

Len Layton

Global Business Development Manager, Acoustics, BlackBerry

If you’ve ever had to “twist and shout” to be heard by back seat passengers then listen up – you might find some new technology from QNX useful (and safer!) 

So many cars already have the basic hardware that we could leverage to effectively make a phone call to the people in the back of the vehicle.  The handsfree microphone can  pick up the driver’s voice, amplify (reinforce) it and play it out of the rear loudspeakers near to the people in the back. This is the principle of in-car communications -- “ICC” for short.

Designing a practical in-car communications (“ICC”) system can be broken down into a few basic steps. But first let’s look at how a car’s hardware is setup and the paths where audio will be flowing.  The main problem is the “front-to-back” case where the driver’s voice needs to be reinforced for the benefit of the passengers in the back, to that the driver doesn’t have to twist around and shout at the people in the back. 

You can see that we have essentially built a public address system for the car.

In noisy conditions we could turn on the ICC system and then the voice of the driver should be heard much more clearly.  Sound would be flowing through the “reinforced path” in the diagram. At intermediate noise levels, the listener in the back will be able to hear both the direct path and the reinforced bath at the same time.

If there is too much time delay between these two paths, then the listener will perceive the reinforced signal as though it was someone else talking and will not sound natural at all. Studies have suggested that any more than about 30 milliseconds is the limit of this time delay between the direct and reinforced paths.

There will also be some of the reinforced signal that comes back to the talker in the form of echo, which can be disturbing or annoying for the talker if there is too much delay on this path too. And let’s not forget the key issue we are dealing with – NOISE! Noise is all around the occupants and the faster you go the more noise there is.  So, what should we care about when designing and ICC system?

The Solution Checklist:

QNX ICC is the first practical seat-to-seat speech enhancement software product. It addresses the whole set of challenges faced when implementing a practical ICC system:

1) Improved intelligibility by reinforcing speech naturally without distortion or added noise. QNX ICC addresses the primary goal of an ICC system – replacing the “What?” with “Oh Yeah!”

2) fully Automatic -  the amount of reinforcement is continuously adapted to the actual noise level in the car. So as different noise conditions are experienced, the reinforcement is immediately and smoothly adjusted and the driver does not need to operate any controls at all. QNX ICC is truly “set and forget” and can be safely left enabled so that it is ready to kick-in as needed. 
 
3) No feedback howling  Under any circumstances QNX ICC will never howl.

4) No Echo or unpleasant perceptible effects – QNX ICC operates at low delay so the talker will only perceive that their voice is slightly louder and not delayed or sounding “processed.” 

5) low delay operation so that reinforced speech does not perceptibly interfere with direct, non-reinforced speech. QNX ICC’s algorithmic latency is only 5 milliseconds. 

6) Reinforces speech only and does not amplify background noise. Leveraging more than a decade of experience and over a hundred patents in the area of automotive signal processing, QNX ICC includes advanced noise reduction algorithms to ensure clear and natural speech reinforcement. 

7) Simultaneous operation with music and not cause any perceptible distortion to music or other media playback.

8) Implementable on current ICs such as infotainment processor CPUs and DSPs and co-exist with other system software.

9) Tuning that is easy and quickly adapted for different vehicle types and acoustic configurations before production.

QNX ICC is available for licensing today either as a standalone signal processing library or as part of QNX’s new Acoustics Management Platform ("AMP"). QNX In-Car Communications is one of the great features of the breakthrough QNX Acoustics Management Platform click here to find out more. 

A future blog post will address how QNX ICC can be integrated with a complete automotive acoustics solution.

Read More