With automotive markets mandating driver-monitoring systems (DMS), global semiconductor company STMicroelectronics has been working to equip carmakers with the technologies required for this to be possible. While DMS solutions are designed to provide greater road safety by assessing driver alertness, the company reports that its latest dual image sensor is built to monitor a vehicle’s interior, covering the driver and all passengers. According to the company, new applications enabled by the sensor include passenger safety-belt checks, vital-sign monitoring, child-left detection, gesture recognition, and video and picture recording.
“The DMS market is growing at a double-digit pace. ST’s new image sensor is set to push that forward by enabling brands to create new services and deliver even greater value for vehicle users, leveraging complete in-cabin monitoring that covers multiple occupants,” said Eric Aussedat, STMicroelectronics’ executive VP and Imaging Sub-Group general manager, in a prepared statement. “Our market-leading DMS sensor provided the perfect launchpad to take the technology further and empower new innovations.”
Image: STMicroelectronics
The sensor, known as VD/VB1940, combines the sensitivity and high resolution of infrared sensing with high dynamic range (HDR) color imaging, STMicroelectronics explains, and it can capture frames alternatively in rolling-shutter and global-shutter modes. With 5.1 megapixels, the system captures the HDR color images required for an occupant monitoring system, in addition to the near-infrared images typically captured by standard DMS sensors. According to the company, DMS uses NIR imaging to analyze driver head and eye movements in all lighting conditions.
The system is offered in bare wafers (model VDB1940) and also packaged in BGAs (model VB1940), and samples are now available, with mass production planned to meet demand for the 2024 model-year vehicles now being designed. The automotive image sensor employs STMicroelectronics’ second-generation 3D-stacked back-side-illuminated wafer technology, which the company says maximizes optical area and on-chip processing in relation to die size. This lets the sensor perform algorithms locally for high performance in both color and NIR imaging, STMicroelectronics explains, saving power and eliminating the need for an external coprocessor.
Algorithms performed on-chip include Bayer conversion and HDR merging for improved image quality and frame rate. On-chip Bayerization processing enables users to reshuffle the color pixels of the RGB NIR 4X4 pattern into an RGGB format compatible with a range of systems-on-chip, STMicroelectronics reports. In addition, the company adds, local processing handles independent color and NIR pixel-exposure optimization for image quality in both modes, as well as smart upscaling to maximize NIR image resolution by capturing extra NIR information from RGB pixels.
The embedded processor is designed to manage cybersecurity to ensure privacy in connected-vehicle applications, including the mutual authentication and pairing of the camera and the electronic control unit, along with video-stream authentication. The sensor captures NIR images in global-shutter mode, which permits synchronizing with the infrared LED emitter to capture fast-moving scenes without motion blur.
By supporting rolling-shutter mode, which reads pixel data row by row, the VD/VB1940 is built to provide high color-imaging performance, the company reports. Leveraging the on-chip HDR-merge feature in rolling-shutter mode, the VB/VD1940 produces a 100dB full-resolution color image. Qualified to AEC-Q100, the sensor is ISO 26262-compliant to facilitate use in functional-safety systems up to ASIL-B.