Visibility Measurement for Road Safety

Visibility Measurement for Road Safety by NEXYAD

INTRODUCTION

Visibility is one of the structural elements of road safety. Indeed, the sense of sight is the only one that let us perceived the future path of the vehicle and then let us act on it : the driver “can see” in front of the vehicle, he predicts where the vehicle will go, and he can act on the controls (brake, steering wheel, …) in order to control the trajectory.

No other way allows us to anticipate.

If we model the task of driving with an automatic control engineering scheme, then we can notice that vision is used quite everywhere :

Set Point of Trajectory Modification

Vision plays a critical role in driving task, and what sizes the efficiency of this sense is « visibility ».

Visibility can be affected by many kinds of factors:
. the absence or insufficience of light (that is why the infrastructure is sometimes illuminated at night, and why vehicles are equipped with lighting.
. rain deposited on the windshield (that is why vehicles are equipped with wipers)
. mist on the windshield (that is why vehicles are equipped with demisting systems)
. humidity, fog or mist suspended in the air in the road scene.

Experts of road infrastructure add elements to enhance the visibility of the path :
. lane markings (white lines), reflective elements.

Similarly, automobile experts equip their vehicle with systems enabling them to improve visibility for the driver, but also allowing the vehicle to be more easily seen by other drivers.

We then understand that measurement of visibility is an important area of potential improvement of road safety in via ADAS.

VISIBILITY MEASUREMENT

The founders of the company NEXYAD have been working since the 80s on the measurement of visibility, early on military applications.
Indeed, it is the military who have studied since the 60’s which criteria allow human visual perception system to detect objects on their clutter.

For the military, the constant search for stealth (camouflage, for example) requires modeling the performance of the detection by human, depending on the light of a scene in the visible wavelength.

The work carried out tests on panels of thousands of soldiers, and led to predictive models for human vision of the ability to detect objects or not, depending on the image quality.
NEXYAD is one of the very few companies in the world to hold these models and have experience of their implementation for more than 20 years.
In simplified terms, we can consider that our eyes and brain need, depending on the size of the objects to be detected, a different contrast level.
We can then compare the contrast available in a scene (eg a road scene) with needed contrast to detect, , for each size of objects.

The comparison results in two scores :
. the apparent size of the smallest detectable object : as the apparent size of an object decreases with distance, it can then be deduced the maximum distance of detection for a reference object (a car, a truck, a pedestrian). Distances will obviously be different for every object because they don’t have the same size. Johnson criteria give let also estimate the maximum distance for object recognition, and the maximum distance for object identification.
. ease of interpretation of the visual scene. NEXYAD summarized this in a score computed from available and needed contrast: the Visual Quality Score (VQS).

This measure of visibility enables automotive application objectify the subjective. NEXYAD has developed two product lines from the same technology :

. a visibility test bench : VisiNex Lab https://nexyad.net/Automotive-Transportation/?page_id=159

VisiNex Lab

Place a vehicle on a test bench and VisiNex Lab measuring visibility among time. If there are disturbs of visibility from rain, for example (using NEXYAD RainNex rain machine, or another rain machine), then we see scores for degraded visibility. If one starts the vehicle visibility restoration systems (eg in the case where the disturbance is the rain : the wipers), then we measure the performance of the visibility restoration.
VisiNex Lab is used by the automotive industry and is still the only tool for measuring the performance of wipers, demisting system, lighting system, …

. an embedded module for ADAS : VisiNex Onboard https://nexyad.net/Automotive-Transportation/?page_id=438
VisiNex Onboard measures the image quality and predicts the detection power of the driver and onboard artificial vision modules. So we get a rating of confidence for artificial vision systems.
Again, NEXYAD is the only non military company to dispose of this technology.

Road Scene

CONCLUSION

Every tier one company or car manufacturer should use NEXYAD modules VisiNex in order to measure performance, robustness, and reliability of their wipers, lighting, and of their camera-based ADAS.
VisiNex Onboard is currently under implementation into the asynchronous real time framework RT-MAPS.

For more information : sales@nexyad.net

Visibility Measurement for ADAS and Autonomous Vehicle

Visibility measurement for ADAS and Autonomous Vehicle
By NEXYAD

Advanced Driver Assistance Systems (ADAS), and partial or total delegation of car control systems will integrate more and more cameras. Those cameras are used to capture video and images are inputs for obstacle detection algorithms, road detection algorithms, detection of pedestrians systems, …

However, a camera can “see” only under certain conditions, and the algorithms used to exploit image need a certain level of image quality. It is possible that some algorithms test themselves if they are in a case of good image quality or not, but in the general case, they don’t, and it is then prudent to have a qualification system that is independent of the detection systems.

The company NEXYAD has worked for years on atmospheric visibility measurement for military application, and was able to develop predictive models of the ability for a human to detect objects. This work can be easily set to pass from a performance prediction of the human vision to a prediction of performance for a machine vision system.

The models consist in comparing the contrast in the scene with the required contrast for detection and / or pattern recognition.
Such a system requires that is respected a compromise between several characteristics of the image:
. number of different gray levels (for a digital camera, it depends on the number of bits)
. size of the objects to be detected
. contrast of objects from their background

Note for Automotive engineers : a performance specification for a camera-based detection system, without giving the minimum contrast, le maximum number of pixels, the number of bits … does NOT have any sense. It is important to know that fact in order to make applications that work and application that know when they work.
For instance, we are all able to detect stars in a dark night sky : the size of objects is very small, the number of Grayscale is very low (pure black and pure white), and the contrast of objects from the background is huge.

Contrast

Similarly, we are able to distinguish clouds over gray sky : the size of objects is very large, and even on edges there is no detail (no high frequency / contours), and the number of different gray levels is very large (gradual grey scale from black to white).

Clouds

Between these two extremes are all possible cases, and in particular with all traffic scenes that may vary greatly from one to another :
. sunny day, overcast day, dark night, undergrowth, sunset, night in headlights, fog, rain, etc …

Visibility_Measurement

In addition to these technical compromise, there are criteria (eg criteria Johnson) that allow to objectify the subjective.

NEXYAD has developed a tool called VisiNex that integrates models and criteria described above, which led to two products:

. VisiNex Lab : test bench for visibility measurement. It sets a vehicle with calibrated visibility disturbances (rain machine, fog machine, …), and VisiNex Lab measures the evolution of the available visibility during the disturbance and during activation of visibility restoration systems (lighting, demisting, wiping, …).
VisiNex Lab is used to adjust the rain sensors, the wiper systems, the lighting systems. VisiNex is a world leader on this type of use : https://nexyad.net/Automotive-Transportation/?page_id=159

VisiNex_Banc

. VisiNex Onboard : NEXYAD took his model into onboard applications to apply and qualify road visibility along the route running (important place to qualify for the road safety applications).
VisiNex Onboard is currently being integrated into the framework for asynchronous real-time applications development RT-MAPS, and will soon be in the NEXYAD vision modules pack for ADAS and driving delegation applications.

VisiNex Onboard
Standard visibility on a highway scene.                             Degraded visibility when approaching a tunnel

VisiNex Onboard can be used in automotive application on the following topics :
. visibility measurement to control Visibility restoration systems (wiper, lighting, …)
. qualification of visibility conditions where an obstacle detection or road detection system will work properly.

The second point is important because road safety applications require to maximize the reliability of vision systems.

To know more :sales@nexyad.net

Poster publication of the Visibility measurement research project (SURVIE – Mov’eo) conclusions at the “carrefours du PREDIT”, in Paris, Palais Broigniart (October 7-8, 2013)

The project SURVIE was headed by NEXYAD with the research partners AXIMUM, CETE, IFSTTAR, OKTAL, SAINT GOBAIN, VALEO.

The goal of this project was to validate standard measurement protocols for different testing usages of the test bench tool VisiNex (developed by NEXYAD : click HERE to know more) : measurement of the performance of every system in a car that was made to restore visibility : lighting, wipers, demist, defrost, hydrophobic windshield, …

SURVIE was a collaborative research program of the competitive cluster Mov’eo.






Closing meeting of the collaborative research project SURVIE : 2012 June 28 in Paris (June 13, 2012)

The closing meeting of the collaborative research project SURVIE will take place the 28th of June 2012, in the Région Ile de France office in Paris.

SURVIE dealt with measurement of visibility during recovery (wipers, lights, demist, defrost, etc …). Measurement was performed using the onl device in the worl that can measure visibility for a human being ; VisiNex Developed par the company Nexyad.

The project gathered Saint Gobain, Valeo, CETE Lyon laboratory (Clermont Ferrand Lab), Oktal, Aximum, Nexyad.

Project Purpose: Research on the technical ways to improve or restore visibility to the driver, and development of new functionalities of VisiNex.

Several innovative measurement functionalities could be integrated to VisiNex by Nexyad, allowing partners to improve their understanding of visibility recovery in many cases of real world scenes.

This research program got the label of the French competitive cluster on automotive and transportation mov’eo.

New release of the NEXYAD VisiNex (May 10, 2012)

A new release of the NEXYAD VisiNex™ (*) tool is available and brings more functionalities : can deal with vibrations (due to artificial wind for instance), can deal with occultations (dirt, frost, …), geographical mapping of visibility (that lets show the dynamic of visibility recovery), measurements with the car lightning as light source.

(*) : visibility measurement for automotive and transportation applications : wipers, lightning, windshield, sign marks, …)

NEXYAD developed a new release of their VisiNex™ product (March 7, 2011)

NEXYAD developed a new release of their VisiNex™ product (visibility measurement using a camera) : new version can give local visibility scores and their distribution on the whole windshield of a car (applications are wipers efficiency enhancement and misting system efficiency measurement and enhancement for car industry).

MOV’EO SURVIE and VisiNex™ (April 30, 2010)

Work meeting of the research project SURVIE:
Definition of standard rules of measurement with the NEXYAD product VisiNex™ for road safety applications.

partners: LCPC, LRPC , VALEO, SAINT GOBAIN, CETE Lyon, OKTAL, AXIMUM

Test the effectiveness of wipers and tune a wiper system : a complex problem (March 15, 2010)

The modern wiper systems for car windshields are complex mechatronics systems that implement both sensors (light sensor, rain sensor), electric motors, one or more wiper blades with rubber qualities to be defined, a software for analysis and recognition of the rain, and a software that automatically triggers the wipers with an appropriate strategy. Each of these components can be selected or adjusted by many ways:
– Rain sensor: location of the rain sensor on the windshield, adjusting the threshold of the first outbreak, its timings, its hysteresis, …
– Luminance Sensor: taking into account only the integration of infrared or visible wavelengths, the sensor location and direction (does is point only the sky?) …
– Rubbers : quality of the viscoelastic material, surface condition, …
– Arm of the wiper: with pressure points, shape memory, …
– Architecture of wipers: conventional two blades, butterfly with a stop in the down position, with throttle stop position, single blade, …
– Characteristics of wipability of the windshield (macroscopic form: take-off speeds, … surface condition)
– Not to mention the software that have huge degrees of freedom (lines of code) …

If we consider a system composed of six subsystems that can each take 10 different items (10 items of rubber, 10 potential triggering software, 10 settings of the rain sensor, …), we obtain 106 possible systems (1 million) !

On a million possible systems, the engineers have to find ONE solution that is industrially acceptable (effectiveness, overall cost of the solution …).

The traditional method and still the most widespread in the industry to test and develop such a system consists in equiping a vehicle with a configuration (set a priori), in waiting for rain, and then sending a expert driver driving on roads they know. The driver then completes qualitative assessment grids which are used by engineers to change the settings of their wiper system.

Obviously, this method is tedious, and they can only test an extremely small number of combinations, so that it passes next to the statistically best solution (the best compromise between performance and cost). In addition, weather conditions in several consecutive tests may NOT be the same: it is NEVER the same rain, NEVER the same light, …
Performance comparisons are theoretically and therefore virtually impossible: no regression testing can be performed at every change, no quantitative rating of the effectiveness can be given … in short, despite all the expertise of engineers and the care they take to perform these tests, it is far from the industrial approach, let alone the quality approach.

The sampling of this complexity requires the use of at least fractional orthogonal experimental design. For this it is necessary to know how to reproduce the weather conditions in the laboratory, dive the vehicle into a known and reproducible (calibrated) environment, and then quantitatively measure the performance of the wiping.

NEXYAD has developed a tool to achieve this. This tool is divided into three major functions:
– A system for generating calibrated lighting (to generate repeatable chronograms of illumination).
– A system for generating calibrated and reproducible artificial rainfall, (for watering the windshield with known artificial rainfall, with statistical characteristics of natural rainfall)
– A system for measuring the effectiveness of the wiper system (providing a score of effectiveness)

The measurement of effectiveness of the wiping was until fairly recently a point relatively blocker. Indeed, the wiper is not a “function” in terms of the driving task: the “function” would rather be “in all circumstances ensure good visibility for the driver”, and wiping is just a technical response to achieve that in the case where visibility is degraded by the water deposited on the windshield.

We then see that if we can measure the visibility of the road scene by the driver (through windshield), then we can measure the effectiveness of the wiping : the rain degrades the visibility, the wiping restores some of the lost visibility.

NEXYAD worked since 1995 on the measurement of visibility and has integrated his expertise in a tool called VisiNex ™.
Similarly, NEXYAD has developed a rain machine (RainNex ™).

The combined use of these tools can set a wiper system in two months with two people, where before we had 5 people for over a year. In addition, the system performance is known and can be optimized (since we know the measure). The technically efficient solutions can then be compared in terms of cost, allowing more to achieve substantial savings.

Some automotive industrials already use NEXYAD tools.

Visibility measurement (February 28, 2010)

Measuring the visibility of a scene for a human being needs to have a mathematical model of the human vision system.

Actually, human vision requires some compromise between measurable characteristics of image quality such as contrast, depth, and object size, so as to detect, recognize, and identify the content of collected images.

When this compromise is not met, the vision becomes very difficult, tedious or even impossible.
It is obvious that the noise in the image (electronic snow of a sensor, for example), or poor contrast (due to the presence of aerosols, fog, rain, humidity, …) may considerably lower the performance of our vision system.

We can therefore say that this “images quality” is a key point of our performance.
But we do not need the same quality to detect all types of objects. For example, we will detect a gray cloud on a gray background, even shapeless, with extremely low contrast if the luminance depth (number of bits for a digital image) is high. On the opposite, on a dark sky, we can detect a star whose contrast is extremely strong, but whose size is at the limit of our eye angular accuracy. In such a case we just need 2 luminance levels (binary images are OK).

Human vision mathematical models were originally developed by the U.S. Department of Defense who wanted to model the impact of camouflage on the probability of detection (of an infantryman, a tank, …), recognition, and identification by a watchman.

“Detection” means “I see something”.
“Recognition” means “I see a car.”
“Identification means “I see a 3 serie BMW”

Of course, it is obvious that the level of detail needed to perform these three operations is not the same.
Measurable criteria in the picture (example: Johnson criteria) could be determined after testing a variety of situations by panels of hundreds of soldiers.

Based on these criteria, it is possible to construct a mathematical model for measuring perceived quality of images. This model is predictive of the ability to detect or to understand the image content.
NEXYAD has developed such a mathematical model of human vision and applied it, among other things, to test the effectiveness of windshields wiping systems of vehicles (product : VisiNex ™) : the rain that collects on the windshield breeze down the performance of visual detection of the driver. Each pass of the wiper can restore some lost visibility.

NEXYAD is currently applying this same maths model in the context of the extent of visibility of road markings (white lines, …), depending on the weather (day / night, rain, …).
The number and scope of potential applications of such a human vision mathematical modelling system are extremely broad.

MOV’EO SURVIE (October 15, 2009)

Kick off of the research project SURVIE:
Definition of standard rules of measurement with the NEXYAD product VisiNex™ for road safety applications.

partners: LCPC, LRPC , VALEO, SAINT GOBAIN, CETE Lyon, OKTAL, AXIMUM