Contributed by Bob Chabot
Electronic Eyes Build a 360° View
ADAS vision technology sees day or night, rain or shine
The amount of information generated by cars has exploded as more new vehicles—often equipped with sensors to support advanced driver assistance features—become connected to the Internet. The ongoing evolution toward partial, and then fully, automated driving requires the installation of more safety-related sensors and actuators in vehicles to help ensure driving safety in virtually all traffic situations and speeds. For an indispensable 360° all-round view of the vehicle, much more than the driver’s eyes are needed. Data fusion and adaptive technologies are essential.
MOTOR recently visited with the ZF Group, a leading Tier One supplier for these systems, to learn what’s coming. ZF has a long history in advanced driver assist systems dating back to the early seventies. The company is currently focused on designing and developing a full array of sensors, processors and actuators, as well developing the algorithms, human-machine interface devices and other technologies that form the foundation for automated driving. Here’s an inside look at what the company calls its Cockpit Concept, a stationary simulator test bed, and in particular, some of the advanced safety technologies ZF is developing.
ZF’s Cockpit Concept incorporates the company’s “See-Think-Act” mantra. Camera and radar systems, in concert with a central control unit, fuse and process data from sensors to make high level safety decisions and trigger actuators that assist in vehicle control, collision avoidance, pre-crash occupant safety, automatic emergency and other advanced automated driving functions. (All images — ZF Group)
Data Fusion Enables an Interactive Cockpit
Today, a host of sensors monitor the area around vehicles and are used to control semi- or fully-automated driver assist systems. One example is Automatic Emergency Braking (AEB), which can automatically brake the vehicle if the driver does not respond in a critical situation. The AEB system combines the information from the forward-looking radars and camera sensors.
The two sensor technologies are remarkably complementary: Radar sensors measure distances and relative speeds with high precision, while camera sensors are useful for the lateral detection range and precise object recognition. This data fusion system can be combined with additional camera and radar sensors, as well as advanced electronic control units to realize automated driving functions that can be used as a basis for future highly automated vehicle systems.
Safe autonomous driving requires reliable and clear communication between the driver and the vehicle. ZF Concept Cockpit demonstrates four innovative technological approaches to the Human-Machine Interface (HMI) as part of its overall Cockpit Concept, which also includes a wide range of sensors, actuators and a central driver assist system data fusion hub. These are:
- The multifunctional steering wheel simplifies the safe transfer of the wheel from autopilot to the driver, which today must be possible at any time after a predetermined warning period. The integrated hands on/off detection feature is one means of letting the vehicle know when the driver actually has a firm grip on the wheel.
- Swipetronic is a patented, smart and high-quality shift-by-wire technology for automatic transmissions. It features an innovative touch display, based on the principle of electrostatic charge, which enables flat, virtual buttons or switches to be touched just like the real thing. This touchscreen operating concept opens up the option of individual operating functions appearing on any preferred area of the HMI unit that are actuated by virtual buttons with a diverse texture that feel real to the human fingertip.
- Advanced, camera-based facial recognition is another important part of the concept. Particularly when automobiles are on the road in highly automated mode, this tool facilitates constant driver monitoring during operation.
- New passive and active safety systems support automated driving functions. For instance, the concept includes a new seat belt system, consisting of an active buckle lifter and seat belt tightener, which automatically generates frequent, insistent pulsations that inform the driver, if necessary, when action is required.
As one of the world's largest systems suppliers, ZF harnesses relevant adaptive technologies from the megatrends of safety, automated driving, efficiency and e-mobility under a single roof. But not even the most comprehensive selection of sensor, radar and camera technologies is capable of powering a driver-assist system on its own, let alone enabling a car to drive by itself. For the high-speed reaction times required, systems also need the right software, capable of processing and analyzing the incoming streams of data without delay. To meet the increasing demand for processing power from the swelling streams of data collected by ever more sophisticated sensors, ZF has partnered with companies such a Mobileye and NVIDIA Corp.
Adaptive Technologies Build a Model of the Environment Around the Vehicle
Imagine highway or city driving without driver intervention until necessary. The cockpit concept’s Highway Driving Assist (HDA) system is designed to automatically steer, brake and accelerate the vehicle at speeds from 0 mph up. In simple terms, it allows the vehicle to control the lane keeping and speed-adaptation tasks while driving, unless driver intervention is necessary and prompted. HDA combines a number of technologies to achieve this.
The HDA system features:
- Adaptive Cruise Control (ACC) and Lane Centering Assist (LCA) help make driving less stressful, keep the vehicle on track and also maintain a defined safe distance to the vehicle traveling in front. With ACC, if the road ahead is clear, the vehicle will travel at the speed selected by the driver. However, if the system detects an impeding vehicle, it can help maintain a safe vehicle separation by providing information to and from the engine, transmission and electronic stability control to enable those units to match the speed of the vehicle ahead. LCA uses a forward-looking lane detection camera combined with electric power steering (EPS) to assist the driver in avoiding unintended lane departures. If the vehicle drifts toward the lane markings and there is a risk of crossing the lane markings unintentionally, the camera will send a torque overlay command to the EPS which, in turn, acts to return the vehicle toward the lane center.
- The HMI leverages a Belt-Driven Electrically Powered Steering system to meet demanding handling and assist requirements across multiple vehicle segments. It features a modular and scalable design suitable for passenger vehicles through full-size trucks, as well as a fully integrated steering gear with in-built packaging flexibility and option to mount either inboard or outboard configurations. Besides automated driving, the technology also supports CO₂ reduction and improved fuel economy.
- ZF’s next generation Tri-Camera System adds vision to Cockpit Concept’s HMI. Designed to support advanced automated driving and safety functions, this fourth-generation three-lens camera provides an expanded object detection range and field of view that exceeds the stringent regulatory requirements for advanced driver assist system technologies. The Tri-Cam combines (1) a telephoto lens for improved long distance sensing, (2) a fish-eye lens for improved short range sensing and (3) Mobileye’s Eye Q4 image processor and object recognition algorithms.
- HMI also leverages a state-of-the-art 77GHz AC2000 high-performance front radar • HMI also leverages a state-of-the-art 77GHz AC2000 high-performance front radar designed to address consumer demand for adaptive cruise control, automatic emergency braking and automated driving functions. More than just supporting ACC and LSA, it also enables Automatic Emergency Braking and other automated driving functions. The AC2000 is small in size, consumes low power, features an extended 200-meter range with a speed-adaptive field of view and offers digital beam forming technology for speed-adaptive field of view. Of note, the AC2000 is capable of fusion with a camera, sensors and actuators to facilitate many safety and comfort functions, including pedestrian detection, blind spot detection, lane change assist, side impact sensing and low-speed forward collision warning. In contrast to camera-only systems, radar provides direct measurement of distance and relative speed under virtually all weather conditions. In addition, radar can be used to support functions that automatically decelerate or stop the vehicle.
Radars and cameras form the artificial eyes of automated driving. How do the systems interact? Where is the potential? Where does the trend go regarding development? Check out the answers provided by Karl-Heinz Glander, senior engineering manager for automated driving at ZF.
It’s About Assisting, Rather Than Controlling Drivers
It’s also important to remember that the driver is not locked out of controlling the vehicle by these automated safety systems. Automated functions improve vehicle reaction conveniently, and will prompt drivers to take control in time when warranted.
But in alignment with SAE automated driving standards. Drivers aren’t limited to just monitoring traffic and system status; they can override the system at any time. For instance, the driver can override automated systems and initiate a lane change by setting the turn indicator to assume control.
The interactive radar, lidar and camera sensor systems that comprise the Cockpit Concept are capable of covering virtually every traffic situation imaginable, according to ZF. Working together, these different sensor technologies ensure that the vehicle and its driver have full, all-round perception of the surrounding environment at all times. That’s not only essential for the success of autonomous driving, but for vehicle operators to make better informed driving decisions.
“We are now able to interpret sensor information in ways that were not feasible with previous methods,” explained Torsten Gollewski, head of ZF Group’s Advanced Engineering. “With the advent of artificial intelligence and advanced adaptive software algorithms, ZF has the opportunity to strengthen its See-Think-Act strategy to advance autonomous driving in general, and safety in particular. For example, we will soon see very powerful domain computers with high-capacity parallel processors in vehicle electronic architectures that will eliminate previous latency issues by leveraging artificial intelligence to process high volumes of data virtually instantly.”
“These new architectures will enable us to provide much more interactive functionality, especially in safety systems that must be able to operate in real time without delay,” added Dr. Stefan Sommer, ZF CEO. “It’s allowing us to bring supercomputer standards of performance not just to automobiles, but to commercial vehicles, entire transportation systems as well as other industrial applications.”
[Editor's note: Visit MOTOR.com for the latest diagnostic and service insights.]