Tag: windshield

Visibility Measurement for ADAS and Autonomous Vehicle

Visibility measurement for ADAS and Autonomous Vehicle
By NEXYAD

Advanced Driver Assistance Systems (ADAS), and partial or total delegation of car control systems will integrate more and more cameras. Those cameras are used to capture video and images are inputs for obstacle detection algorithms, road detection algorithms, detection of pedestrians systems, …

However, a camera can “see” only under certain conditions, and the algorithms used to exploit image need a certain level of image quality. It is possible that some algorithms test themselves if they are in a case of good image quality or not, but in the general case, they don’t, and it is then prudent to have a qualification system that is independent of the detection systems.

The company NEXYAD has worked for years on atmospheric visibility measurement for military application, and was able to develop predictive models of the ability for a human to detect objects. This work can be easily set to pass from a performance prediction of the human vision to a prediction of performance for a machine vision system.

The models consist in comparing the contrast in the scene with the required contrast for detection and / or pattern recognition.
Such a system requires that is respected a compromise between several characteristics of the image:
. number of different gray levels (for a digital camera, it depends on the number of bits)
. size of the objects to be detected
. contrast of objects from their background

Note for Automotive engineers : a performance specification for a camera-based detection system, without giving the minimum contrast, le maximum number of pixels, the number of bits … does NOT have any sense. It is important to know that fact in order to make applications that work and application that know when they work.
For instance, we are all able to detect stars in a dark night sky : the size of objects is very small, the number of Grayscale is very low (pure black and pure white), and the contrast of objects from the background is huge.

Contrast

Similarly, we are able to distinguish clouds over gray sky : the size of objects is very large, and even on edges there is no detail (no high frequency / contours), and the number of different gray levels is very large (gradual grey scale from black to white).

Clouds

Between these two extremes are all possible cases, and in particular with all traffic scenes that may vary greatly from one to another :
. sunny day, overcast day, dark night, undergrowth, sunset, night in headlights, fog, rain, etc …

Visibility_Measurement

In addition to these technical compromise, there are criteria (eg criteria Johnson) that allow to objectify the subjective.

NEXYAD has developed a tool called VisiNex that integrates models and criteria described above, which led to two products:

. VisiNex Lab : test bench for visibility measurement. It sets a vehicle with calibrated visibility disturbances (rain machine, fog machine, …), and VisiNex Lab measures the evolution of the available visibility during the disturbance and during activation of visibility restoration systems (lighting, demisting, wiping, …).
VisiNex Lab is used to adjust the rain sensors, the wiper systems, the lighting systems. VisiNex is a world leader on this type of use : https://nexyad.net/Automotive-Transportation/?page_id=159

VisiNex_Banc

. VisiNex Onboard : NEXYAD took his model into onboard applications to apply and qualify road visibility along the route running (important place to qualify for the road safety applications).
VisiNex Onboard is currently being integrated into the framework for asynchronous real-time applications development RT-MAPS, and will soon be in the NEXYAD vision modules pack for ADAS and driving delegation applications.

VisiNex Onboard
Standard visibility on a highway scene.                             Degraded visibility when approaching a tunnel

VisiNex Onboard can be used in automotive application on the following topics :
. visibility measurement to control Visibility restoration systems (wiper, lighting, …)
. qualification of visibility conditions where an obstacle detection or road detection system will work properly.

The second point is important because road safety applications require to maximize the reliability of vision systems.

To know more :sales@nexyad.net

Poster publication of the Visibility measurement research project (SURVIE – Mov’eo) conclusions at the “carrefours du PREDIT”, in Paris, Palais Broigniart (October 7-8, 2013)

The project SURVIE was headed by NEXYAD with the research partners AXIMUM, CETE, IFSTTAR, OKTAL, SAINT GOBAIN, VALEO.

The goal of this project was to validate standard measurement protocols for different testing usages of the test bench tool VisiNex (developed by NEXYAD : click HERE to know more) : measurement of the performance of every system in a car that was made to restore visibility : lighting, wipers, demist, defrost, hydrophobic windshield, …

SURVIE was a collaborative research program of the competitive cluster Mov’eo.






New release of the NEXYAD VisiNex (May 10, 2012)

A new release of the NEXYAD VisiNex™ (*) tool is available and brings more functionalities : can deal with vibrations (due to artificial wind for instance), can deal with occultations (dirt, frost, …), geographical mapping of visibility (that lets show the dynamic of visibility recovery), measurements with the car lightning as light source.

(*) : visibility measurement for automotive and transportation applications : wipers, lightning, windshield, sign marks, …)

NEXYAD developed a new release of their VisiNex™ product (March 7, 2011)

NEXYAD developed a new release of their VisiNex™ product (visibility measurement using a camera) : new version can give local visibility scores and their distribution on the whole windshield of a car (applications are wipers efficiency enhancement and misting system efficiency measurement and enhancement for car industry).

Test the effectiveness of wipers and tune a wiper system : a complex problem (March 15, 2010)

The modern wiper systems for car windshields are complex mechatronics systems that implement both sensors (light sensor, rain sensor), electric motors, one or more wiper blades with rubber qualities to be defined, a software for analysis and recognition of the rain, and a software that automatically triggers the wipers with an appropriate strategy. Each of these components can be selected or adjusted by many ways:
– Rain sensor: location of the rain sensor on the windshield, adjusting the threshold of the first outbreak, its timings, its hysteresis, …
– Luminance Sensor: taking into account only the integration of infrared or visible wavelengths, the sensor location and direction (does is point only the sky?) …
– Rubbers : quality of the viscoelastic material, surface condition, …
– Arm of the wiper: with pressure points, shape memory, …
– Architecture of wipers: conventional two blades, butterfly with a stop in the down position, with throttle stop position, single blade, …
– Characteristics of wipability of the windshield (macroscopic form: take-off speeds, … surface condition)
– Not to mention the software that have huge degrees of freedom (lines of code) …

If we consider a system composed of six subsystems that can each take 10 different items (10 items of rubber, 10 potential triggering software, 10 settings of the rain sensor, …), we obtain 106 possible systems (1 million) !

On a million possible systems, the engineers have to find ONE solution that is industrially acceptable (effectiveness, overall cost of the solution …).

The traditional method and still the most widespread in the industry to test and develop such a system consists in equiping a vehicle with a configuration (set a priori), in waiting for rain, and then sending a expert driver driving on roads they know. The driver then completes qualitative assessment grids which are used by engineers to change the settings of their wiper system.

Obviously, this method is tedious, and they can only test an extremely small number of combinations, so that it passes next to the statistically best solution (the best compromise between performance and cost). In addition, weather conditions in several consecutive tests may NOT be the same: it is NEVER the same rain, NEVER the same light, …
Performance comparisons are theoretically and therefore virtually impossible: no regression testing can be performed at every change, no quantitative rating of the effectiveness can be given … in short, despite all the expertise of engineers and the care they take to perform these tests, it is far from the industrial approach, let alone the quality approach.

The sampling of this complexity requires the use of at least fractional orthogonal experimental design. For this it is necessary to know how to reproduce the weather conditions in the laboratory, dive the vehicle into a known and reproducible (calibrated) environment, and then quantitatively measure the performance of the wiping.

NEXYAD has developed a tool to achieve this. This tool is divided into three major functions:
– A system for generating calibrated lighting (to generate repeatable chronograms of illumination).
– A system for generating calibrated and reproducible artificial rainfall, (for watering the windshield with known artificial rainfall, with statistical characteristics of natural rainfall)
– A system for measuring the effectiveness of the wiper system (providing a score of effectiveness)

The measurement of effectiveness of the wiping was until fairly recently a point relatively blocker. Indeed, the wiper is not a “function” in terms of the driving task: the “function” would rather be “in all circumstances ensure good visibility for the driver”, and wiping is just a technical response to achieve that in the case where visibility is degraded by the water deposited on the windshield.

We then see that if we can measure the visibility of the road scene by the driver (through windshield), then we can measure the effectiveness of the wiping : the rain degrades the visibility, the wiping restores some of the lost visibility.

NEXYAD worked since 1995 on the measurement of visibility and has integrated his expertise in a tool called VisiNex ™.
Similarly, NEXYAD has developed a rain machine (RainNex ™).

The combined use of these tools can set a wiper system in two months with two people, where before we had 5 people for over a year. In addition, the system performance is known and can be optimized (since we know the measure). The technically efficient solutions can then be compared in terms of cost, allowing more to achieve substantial savings.

Some automotive industrials already use NEXYAD tools.

Visibility measurement (February 28, 2010)

Measuring the visibility of a scene for a human being needs to have a mathematical model of the human vision system.

Actually, human vision requires some compromise between measurable characteristics of image quality such as contrast, depth, and object size, so as to detect, recognize, and identify the content of collected images.

When this compromise is not met, the vision becomes very difficult, tedious or even impossible.
It is obvious that the noise in the image (electronic snow of a sensor, for example), or poor contrast (due to the presence of aerosols, fog, rain, humidity, …) may considerably lower the performance of our vision system.

We can therefore say that this “images quality” is a key point of our performance.
But we do not need the same quality to detect all types of objects. For example, we will detect a gray cloud on a gray background, even shapeless, with extremely low contrast if the luminance depth (number of bits for a digital image) is high. On the opposite, on a dark sky, we can detect a star whose contrast is extremely strong, but whose size is at the limit of our eye angular accuracy. In such a case we just need 2 luminance levels (binary images are OK).

Human vision mathematical models were originally developed by the U.S. Department of Defense who wanted to model the impact of camouflage on the probability of detection (of an infantryman, a tank, …), recognition, and identification by a watchman.

“Detection” means “I see something”.
“Recognition” means “I see a car.”
“Identification means “I see a 3 serie BMW”

Of course, it is obvious that the level of detail needed to perform these three operations is not the same.
Measurable criteria in the picture (example: Johnson criteria) could be determined after testing a variety of situations by panels of hundreds of soldiers.

Based on these criteria, it is possible to construct a mathematical model for measuring perceived quality of images. This model is predictive of the ability to detect or to understand the image content.
NEXYAD has developed such a mathematical model of human vision and applied it, among other things, to test the effectiveness of windshields wiping systems of vehicles (product : VisiNex ™) : the rain that collects on the windshield breeze down the performance of visual detection of the driver. Each pass of the wiper can restore some lost visibility.

NEXYAD is currently applying this same maths model in the context of the extent of visibility of road markings (white lines, …), depending on the weather (day / night, rain, …).
The number and scope of potential applications of such a human vision mathematical modelling system are extremely broad.