Étiquette : ADAS
Onboard road safety/risk Measurement correlated to accidents
How to use SafetyNex V2.1 for Insurance applications, and for
Automotive applications (ADAS & autonomous vehicle)
by NEXYAD
INTRODUCTION: Description of 2.1 SafetyNex
SafetyNex is a software module proposed by the French company NEXYAD. This module aims to match the driving style (acceleration, vehicle speed) with the danger characteristics of infrastructure.
SafetyNex development started in 2001 and benefited from three national collaborative research programs (PREDIT and FUI) that allowed NEXYAD modeling expertise of road equipment notation accident-prone infrastructure.
Indeed, the accident-prone nature of a local infrastructure, cannot be deduced from statistical studies on this local element, for the simple reason that accidents are rare events.A driver has one accident every 70 000 km.
To investigate the accident, the experts of the equipment mainly use two methods:
. aggregated statistics at national and European level : it provides large numbers of accidents and thus make statistics relevant. But heterogeneous infrastructure through Europe makes it difficult to project results on a local infrastructure that has its own characteristics, sometimes far from the average characteristic in France or Europe.
. the observation of « near misses » or « almost accident » as they are numerous : most potential accidents can be nearly avoided by drivers. Experts of infrastructure developed observatories for those danger situations and it brings useful information.
From these elements, the experts of the infrastructure have published a set of rules for predicting accidents, based on characteristics, the accident-prone characters of a piece of infrastructure.
For example, a bend may be accident-prone if its radius greatly shortens in its middle (curve which closes), but a « normal » curve may become dangerous if it comes after a long distance of straight line (this is called « break out on the pathway « ).
NEXYAD integrated all these rules in SafetyNex and has made numerous tests on real world to validate the efficiency of scoring danger.
SafetyNex V2.1 reads on-board navigation map that gives infrastructure characteristics , downstream of the vehicle, and then SafetyNex scores the relevancy of the driving style, knowing the shape of the road ahead and other important things (pedestrian pathway, …).
Note: in a higher version (already being tested), SafetyNex may also reflect the presence of other users on the road (cars, trucks, bicycles, motorcycles, pedestrians …) by coupling it to RoadNex (road detection by camera), and ObstaNex (obstacles detection by camera).
But this version will require to have appropriate computing power to process videos. The Moore law lets us think i twill be easy and cheap in 2 or 3 years.
For now, we think that adaptation of the driving type with infrastructure characteristics meets a real issue and SafetyNex V2.1 is the only available (and for sale) module in the world for performing this function.
The output computed by SafetyNex is the road safety risk, between 0 and 100%, at every moment.
It is possible to use SafetyNex in real time to alert the driver.
It also may be used off-line, aggregating the risk : SafetyNex used to complete big data with revelant variables to study the road safety risk, in a statistical way.
SafetyNex is available in three types of running environments:
. PC under Windows or Linux
. Electronic device of an Automotive Tier One Company, for the new vehicles and aftermarket
. smartphones
SafetyNex V2.1 FOR USE OF INSURANCE
Insurance Companies have several ways to use SafetyNex V2.1:
. free distribution of the SafetyNex App for smartphones to all their customers :
In this case, the business model is to seek partners who want to reduce their cost of acquiring customers online.
In fact, when you divide the advertising budget of a major retailer (Amazon, Fnac, Darty, …) by the number of purchases, there are about 30 euros (cost of a customer).
The idea of NEXYAD then is to propose a serious game that « gives » points to drivers who behave well.
These points are converted into « coupons » with one or more distributors. The drivers will go on the website in order to validate coupons by purchasing products. In doing so (spotted digitally with flash code …), the distributor will give back 15 euros : 7,5 euros for the cupons, and 7,5 euros for the Insurance Company that distributed the NEXYAD App to their millions of customers. The insurer therefore makes money and share revenue with NEXYAD and its technology partners.
This solution is particularly interesting for several reasons:
. the insurance company earns money directly on millions of purchases.
. drivers do their best to earn coupons, and therefore change their driving behavior to achieve it.
The insurer therefore modifies in a sustainable way the driving style, and contributes to the decline in the number of accidents. The Communications Department of the the Insurance company may use this fact for communicating it to the public.
. the largest distributors lower their customers acquisition costs and they can then lower their advertising budget
. the driver improves self safety, and is able to purchase more.
. distribution in an electronic device :
The electronic device has the advantage of being still operational without driver intervention.
This solution is particularly interesting for business fleets, and allows to quickly and efficiently initiate a type of pricing system « pay how you drive » that may be seen as an extension of the bonus / malus system, but with a predictive power.
. constitution of big data for actuaries:
Risk can be recorded onboard, or in aggregate, giving statisticians a new variable for more accuracy on pricing.
SafetyNex V2.1 FOR USE IN ADAS AND AUTONOMOUS CAR APPLICATIONS
The road safety risk measurement is of course interesting for automotive applications :
. SafetyNex read the shape of infrastructure downstream of the vehicle (electronic horizon), and can therefore help to choose which types of sensors are relevant.
. SafetyNex can estimate the maximum distance targetable by sensors (radar, camera, …) depending on the shape of the infrastructure (turn, climb, …)
. During a delegation of driving, the robot drives the vehicle and then SafetyNex V2.1 can estimate the road safety related to the automatic driving.
The road safety score can be used in several ways:
. Off-line: the engineers who design and develop driving delegation system uses the risk score of SafetyNex V2.1 as a feedback, and try to minimize it by successive tests.
. On-line: the risk score is used by the decision-making system to generate actions that lead to the lowest possible risk.
CONCLUSION
SafetyNex V2.1 is now available and NEXYAD is currently working on deployment opportunities worldwide at the beginning of 2016.
This module is unique and has a direct effect, if properly used, on road safety.
For more information: sales@nexyad.net
DEMO WITH SOUND IN URBAN TRAFFIC
Put the sound on for this demo with vocal explanations :
Visibility Measurement for Road Safety
Visibility Measurement for Road Safety by NEXYAD
INTRODUCTION
Visibility is one of the structural elements of road safety. Indeed, the sense of sight is the only one that let us perceived the future path of the vehicle and then let us act on it : the driver « can see » in front of the vehicle, he predicts where the vehicle will go, and he can act on the controls (brake, steering wheel, …) in order to control the trajectory.
No other way allows us to anticipate.
If we model the task of driving with an automatic control engineering scheme, then we can notice that vision is used quite everywhere :
Vision plays a critical role in driving task, and what sizes the efficiency of this sense is « visibility ».
Visibility can be affected by many kinds of factors:
. the absence or insufficience of light (that is why the infrastructure is sometimes illuminated at night, and why vehicles are equipped with lighting.
. rain deposited on the windshield (that is why vehicles are equipped with wipers)
. mist on the windshield (that is why vehicles are equipped with demisting systems)
. humidity, fog or mist suspended in the air in the road scene.
Experts of road infrastructure add elements to enhance the visibility of the path :
. lane markings (white lines), reflective elements.
Similarly, automobile experts equip their vehicle with systems enabling them to improve visibility for the driver, but also allowing the vehicle to be more easily seen by other drivers.
We then understand that measurement of visibility is an important area of potential improvement of road safety in via ADAS.
VISIBILITY MEASUREMENT
The founders of the company NEXYAD have been working since the 80s on the measurement of visibility, early on military applications.
Indeed, it is the military who have studied since the 60’s which criteria allow human visual perception system to detect objects on their clutter.
For the military, the constant search for stealth (camouflage, for example) requires modeling the performance of the detection by human, depending on the light of a scene in the visible wavelength.
The work carried out tests on panels of thousands of soldiers, and led to predictive models for human vision of the ability to detect objects or not, depending on the image quality.
NEXYAD is one of the very few companies in the world to hold these models and have experience of their implementation for more than 20 years.
In simplified terms, we can consider that our eyes and brain need, depending on the size of the objects to be detected, a different contrast level.
We can then compare the contrast available in a scene (eg a road scene) with needed contrast to detect, , for each size of objects.
The comparison results in two scores :
. the apparent size of the smallest detectable object : as the apparent size of an object decreases with distance, it can then be deduced the maximum distance of detection for a reference object (a car, a truck, a pedestrian). Distances will obviously be different for every object because they don’t have the same size. Johnson criteria give let also estimate the maximum distance for object recognition, and the maximum distance for object identification.
. ease of interpretation of the visual scene. NEXYAD summarized this in a score computed from available and needed contrast: the Visual Quality Score (VQS).
This measure of visibility enables automotive application objectify the subjective. NEXYAD has developed two product lines from the same technology :
. a visibility test bench : VisiNex Lab https://nexyad.net/Automotive-Transportation/?page_id=159
Place a vehicle on a test bench and VisiNex Lab measuring visibility among time. If there are disturbs of visibility from rain, for example (using NEXYAD RainNex rain machine, or another rain machine), then we see scores for degraded visibility. If one starts the vehicle visibility restoration systems (eg in the case where the disturbance is the rain : the wipers), then we measure the performance of the visibility restoration.
VisiNex Lab is used by the automotive industry and is still the only tool for measuring the performance of wipers, demisting system, lighting system, …
. an embedded module for ADAS : VisiNex Onboard https://nexyad.net/Automotive-Transportation/?page_id=438
VisiNex Onboard measures the image quality and predicts the detection power of the driver and onboard artificial vision modules. So we get a rating of confidence for artificial vision systems.
Again, NEXYAD is the only non military company to dispose of this technology.
CONCLUSION
Every tier one company or car manufacturer should use NEXYAD modules VisiNex in order to measure performance, robustness, and reliability of their wipers, lighting, and of their camera-based ADAS.
VisiNex Onboard is currently under implementation into the asynchronous real time framework RT-MAPS.
For more information : sales@nexyad.net
SafetyNex for Onboard Road Safety Measurement
SafetyNex for Onboard Road Safety Measurement by NEXYAD
INTRODUCTION
Car manufacturers and insurance companies both need a system that would estimate in real time the risk taken by the driver.
Most commercial applications use to consider that a driver that do not accelerate much doesn’t take risk, and that a driver that drives more sporty is dangerous.
However, insurance companies statisticians could notice that there no correlation between the driving style and the accidents.
It is completely obvious : danger comes when the driving style is not adapted to the infrastructure. So driving style doesn’t has no meaning by itself.
NEXYAD company has been working since 1995 on onboard risk estimation, and recently launched their module SafetyNex that estimates a risk which is correlated (by construction) with accidents.
SafetyNex is the result of three collaborative French research programs :
. ARCOS
. SARI
. SERA
SafetyNex measures onboard the adequacy of driving style (and in particular the speed of the vehicle) with the characteristics of the infrastructure : adequacy of the current speed and initiated acceleration to the radii of curvature of bends downstream, to the presence of downstream crossings, or pedestrian crossings, … etc.
It is possible to add to SafetyNex optional inputs such as :
. weather report,
. maximum grip
. atmospheric visibility (rain, fog …)
. distance to obstacles (coming from an ADAS system) and in this case, we use not only infrastructure characteristics but also trafic flow information that describe the way other users move on the same infrastructure.
Similarly, can be integrated into SafetyNex data from characteristics of ADAS in order to measure the adequacy of these driver assistance systems to the situation experienced by the vehicle on the infrastructure.
For example, if the vehicle has radar or camera, the data of the opening angle enable SafetyNex (which read shape of the infrastructure from the onboard navigation map) to compute the distance of geometric visibility, not for the driver, but for embedded artificial perception systems.
EXAMPLE OF USE IN URBAN TRAFFIC
The example below shows the predictive nature of safetyNex : when you get in an intersection, it’s a little before that you must slow down because you can not know what is likely to emerge from this intersection. However, when one is in the intersection, it is not dangerous to re-accelerate. This is the way that safe drivers use to drive.
Therefore, the risk score is not correlated to the value of the deceleration or acceleration but to the adequacy of speed to potential dangers of the infrastructure. You may drive sporty or lazy and have the same good or bad safety score computed by SafetyNex.
Video with sound (spoken explanations)
CONCLUSION
SafetyNex is now available for sale and is operating in the following environments:
. Framework RT-MAPS PC : This version is for automakers researchers, scientists of tier one techno suppliers, statisticians and actuaries of insurance companies. It allows real-time replay, in order to see what areas make the risk climb, it also allows to correlate the new variable (risk) with all other variables available, and for car manufacturers, it lets develop ADAS based on this module.
RT-MAPS is interfaced with the Data Base Management Systems, which is convenient to apply SafetyNex on the company’s information systems.
. electronic device of an automotive tier one company : the announcement will be made soon by the automotive tier one techno supplier.
. mobile phones (December 2015), which will allow everyone to have this road safety module.
To know more : sales@nexyad.net
Visibility Measurement for ADAS and Autonomous Vehicle
Visibility measurement for ADAS and Autonomous Vehicle
By NEXYAD
Advanced Driver Assistance Systems (ADAS), and partial or total delegation of car control systems will integrate more and more cameras. Those cameras are used to capture video and images are inputs for obstacle detection algorithms, road detection algorithms, detection of pedestrians systems, …
However, a camera can « see » only under certain conditions, and the algorithms used to exploit image need a certain level of image quality. It is possible that some algorithms test themselves if they are in a case of good image quality or not, but in the general case, they don’t, and it is then prudent to have a qualification system that is independent of the detection systems.
The company NEXYAD has worked for years on atmospheric visibility measurement for military application, and was able to develop predictive models of the ability for a human to detect objects. This work can be easily set to pass from a performance prediction of the human vision to a prediction of performance for a machine vision system.
The models consist in comparing the contrast in the scene with the required contrast for detection and / or pattern recognition.
Such a system requires that is respected a compromise between several characteristics of the image:
. number of different gray levels (for a digital camera, it depends on the number of bits)
. size of the objects to be detected
. contrast of objects from their background
Note for Automotive engineers : a performance specification for a camera-based detection system, without giving the minimum contrast, le maximum number of pixels, the number of bits … does NOT have any sense. It is important to know that fact in order to make applications that work and application that know when they work.
For instance, we are all able to detect stars in a dark night sky : the size of objects is very small, the number of Grayscale is very low (pure black and pure white), and the contrast of objects from the background is huge.
Similarly, we are able to distinguish clouds over gray sky : the size of objects is very large, and even on edges there is no detail (no high frequency / contours), and the number of different gray levels is very large (gradual grey scale from black to white).
Between these two extremes are all possible cases, and in particular with all traffic scenes that may vary greatly from one to another :
. sunny day, overcast day, dark night, undergrowth, sunset, night in headlights, fog, rain, etc …
In addition to these technical compromise, there are criteria (eg criteria Johnson) that allow to objectify the subjective.
NEXYAD has developed a tool called VisiNex that integrates models and criteria described above, which led to two products:
. VisiNex Lab : test bench for visibility measurement. It sets a vehicle with calibrated visibility disturbances (rain machine, fog machine, …), and VisiNex Lab measures the evolution of the available visibility during the disturbance and during activation of visibility restoration systems (lighting, demisting, wiping, …).
VisiNex Lab is used to adjust the rain sensors, the wiper systems, the lighting systems. VisiNex is a world leader on this type of use : https://nexyad.net/Automotive-Transportation/?page_id=159
. VisiNex Onboard : NEXYAD took his model into onboard applications to apply and qualify road visibility along the route running (important place to qualify for the road safety applications).
VisiNex Onboard is currently being integrated into the framework for asynchronous real-time applications development RT-MAPS, and will soon be in the NEXYAD vision modules pack for ADAS and driving delegation applications.
Standard visibility on a highway scene. Degraded visibility when approaching a tunnel
VisiNex Onboard can be used in automotive application on the following topics :
. visibility measurement to control Visibility restoration systems (wiper, lighting, …)
. qualification of visibility conditions where an obstacle detection or road detection system will work properly.
The second point is important because road safety applications require to maximize the reliability of vision systems.
To know more :sales@nexyad.net
Validation Database New
Road Detection & Road Safety
NEXYAD tools for ADAS
Road Detection & Road Safety
NEXYAD tools for ADAS
NEXYAD Automotive & Transportation Newsletter #4, the 7th of September 2015
The company NEXYAD started building a database for validation of advanced driver assistance systems (ADAS and Autonomous car) using the methodology AGENDA published in the 90 by Gérard Yahiaoui (methodology initially developped for control construction of learning and test databases for the implementation of artificial neural networks).
This database has two essential characteristics:
1) Known life situations
Indeed, the methodology AGENDA proposes to describe potential changes of signals and images came into factors of variability and their crosses.
Example, for obstacle detection :
. weather (dry overcast, sunny weather, rain, fog)
. overall brightness (low, medium, high)
. speed of the carrier vehicle (low, moderate, high)
. type of road (highway, road with marking, road without marking …)
. coating (bitumen 1, bitumen 2, …, cobblestones)
. day / night (headlights and the lights switched infrastructure)
. season (spring, summer, autumn, winter)
. etc …
> type of obstacle :
– stopped
. infrastructure-related: work terminals, tolls, …
. related users: tire on the road, parcel felt from a truck lying on the road, biker following a road accident, disabled vehicle stopped on the floor, standing pedestrian on roadside edge (dodger / no sniper)
– moving
. truck, car, vulnerable (pedestrian, bicycle, motorcycle) each with types trajectories (longitudinal
in rolling direction, longitudinally in the opposite direction of rolling side) and position (opposite
to right, left).
. Etc…
We see that if we cross these factors, we find fairly quickly a huge number of cases. However, the development of ADAS systems is complex, and it is necessary to proceed by successive iterations, starting from simple situations to move to complicated situations.
Our database allows this, since all records are described in terms of crossing the terms of the factors of variability. Thus knows exactly which cases were tested or not by the system.
Formalism ‘crossing of variability factors of the terms’ allows using design of experiments, and in particular orthogonal fractional plans to sharply reduce the number of cases to be tested while ensuring maximum coverage of life situations. One can in this context to develop a fractional ADAS on an orthogonal plan and test other hard fractional orthogonal planes for example.
2) Reality reference
This is to crop images barriers and infrastructure elements (markings, roadsides, etc.) so as to constitute a reference to measure system performance.
. Examples of life situations:
1.1, summer, overcast, unmarked road, moderate speed tire on the floor, dry weather
1.2, summer, overcast, unmarked road, moderate speed, parcels on the floor, dry weather
2.1, summer, overcast , unmarked road, moderate speed, standing pedestrians non ambush at the edges of the floor, dry weather
2.2, summer, overcast, unmarked road, moderate speed, lying on the floor human, dry weather
etc …
Not sure that you would meet those few cases, even with on million kilometers on open roads.
NEXYAD starts his collection of images and data:
. video (towards the front of the vehicle) Color
. accelerometers
. gyros
The files are synchronized by RT-MAPS tool INTEMPORA society.
The files are saved as RT-MAPS format and replayable directly by this tool.
NEXYAD currently looking for contributors on this internal project. Co contributors fund and in return free access to the database, unlimited in time. This contribution will accelerate the work of collecting and labeling.
NEXYAD wishes to provide this basis before June 2016, free way to give the material to the community and the ADAS autonomous vehicle for a smaller version of the database, and pay way (as subscriptions) for complete database.
NEXYAD’s ambition is to spread its methodological expertise and allow everyone to assess the performance of vision systems for ADAS, whether systems developed by NEXYAD, or others.
“Methodology for ADAS Validation: Potential Contribution of Other Scientific Fields Which Have Already Answered the Same Questions”, Gérard Yahiaoui, Pierre Da Silva Dias, CESA congress Dec 2014, Paris, proc. Springer Verlag
“Methods and tools for ADAS validation”, Gérard Yahiaoui, Nicolas du Lac, Safetyweek congress, May 2015, Aschaffenburg
For questions, or if you wish to become a contributor, please contact NEXYAD : +33 139041360
NEXYAD module RoadNex V2.1
A useful complement to markings detection
The detection of the road is a key element of driver assistance systems (ADAS) and autonomous vehicles.
Indeed, objects, obstacles, other road users, must be detected but also positioned relatively to the road.
The detection of the entire route, that is to say not only its markings or edges, but all the way, should enable
embedded intelligence to select appropriate action.
The company NEXYAD has been working on this issue for over 20 years without interruption, and has accumulated a large number of cases of road types, of coatings, in various atmospheric conditions.
This is to detect the rollable area on the road, without regard to, in a first step, lane markings.
Indeed, in Europe, there are many unmarked roads, and work on a marked road may change the markings and
make a « follow the markings strategy » dangerous.
In the images below you can see on the left a typical French countryside road with no markings, and on the right image, new markings was achieved while former markings still strongly visible.
These cases are quite common on our European roads and a driver assistance system, or a driving delegation
system, must at least understand such cases and if necessary tell the driver to cope with it by himself.
The NEXYAD road detection module, RoadNex V2.1 is a brick to go further to cope with these cases :
RoadNex V2.1 should be coupled with road signs detection, road markings detection, obstacle detection, in order to build an intelligent perception system. RoadNex is then a key module of such a system.
The road detection module NEXYAD, RoadNex V2.1 is available as a component into the asynchronous real time framework RT-MAPS : See HERE
NEXYAD module SafetyNex running as real-time component
of Framework RT-Maps
SafetyNex (safety level estimation for ADAS)
SafetyNex Onboard is a high level functional bloc (software) of safety measurement, taking into account map and GPS geolocation (shape of the road, crossing roads, … ahead), speed, accelerations, visibility, adherence, distance to obstacle, etc.
SafetyNex measures adaptation of the driving style to infrastructure topology, and possibly Dangerous situations.
Two main applications :
_ Car industry : intelligent Navigation system providing valuable advices to keep the car in a good level of safety; sending alarms on dangers
_ Insurance : driving style measurment correlated with accidentology (insurance pricing, Pay How You Drive)
SafetyNex is now running in RT-Maps by IMTEMPORA
SafetyNex is under fusion with Ecogyzer (eco driving rating system) : this « package » will be the ultimate tool for eco and safe driving combination.
SafetyNex v2.1
Using the NEXYAD road detection (RoadNex)
to make obstacles detection more robust
The detection of obstacles on the road, or even recognition of those obstacles, has become an
important issue for the next few years, in order to propose to the driver:
. Smart Systems for driving assistance : ADAS
. Delegation (partial at first, later full) of the driving task, to go step by step to the so-called autonomous
vehicle
NEXYAD developed a vision-based obstacle detection system (ObstaNex) that aims to offer an alternative to
the current reference product on the market (MobilEye).
We detail the general principles of detection in a previous article.
It is interesting to note that NEXYAD also developed a road detection module named RoadNex.
RoadNex indicates the edges of the rollable way (thus highlights tighter if an obstacle is wayside, this may
be useful) and also indicates the rollable flat area (surface) in front of the vehicle.
RoadNex (NEXYAD): the rollable lane detection. For a clear urban lane.
RoadNex (NEXYAD): the rollable lane detection. For a urban labe cluttered by a vulnerable road user
in motion
One can see on this picture that RoadNex, even if it does not detect obstacles (because it detects
lanes), finally finds the « negative » of obstacles.
One of the uses of RoadNex is eliminating rollable areas from possibly detected obstacles by another module
(camera, radar, lidar …): the above image.
A simple confirmation that RoadNex not colored area corresponds to obstacles detection alarms
is sufficient to confirm the presence of an obstacle and to initiate, for example, the braking.
This system of cooperation between RoadNex and an obstacle detection system (ObstaNex, other
Vision-based module, radar, lidar, …) is particularly useful in city (see pictures RoadNex above)
and on the highway :
RoadNex (NEXYAD): the rollable lane detection. Case of clear highway lane
RoadNex (NEXYAD): rollable lane detection while taking over a truck on highway
NEXYAD is currently working on a low-level fusion of ObstaNex and RoadNex in the context presented above.
To contact us sales@nexyad.net
Other NEXYAD publications about ADAS ON NEXYAD WEBSITE
. DRIVING DELEGATION : KEY ELEMENTS
. ROAD DETECTION FOR ADAS AND AUTONOMOUS VEHICLE : NEXYAD MODULE RoadNex V2.1
Driving Delegation: key elements
Driving Delegation: key elements for an artificial perception system
Publication of September 2, 2015
Authors : Gérard YAHIAOUI & Pierre DA SILVA DIAS
INTRODUCTION
The automotive industry starts offering ADAS, and plans to propose in the near future partial or total driving delegation systems.
Main cases to be processed first may be:
. Highway driving, where the number of events per kilometer is small because the infrastructure has been designed to minimize path irregularities (little or no turns, every car in the same direction, wide track, geometric visibility up to several kilometers, enough little interactions between vehicles, at least when the traffic is flowing).
. The city, where infrastructure complexity is very large, where interactions between the road users are very strong, making detection a difficult tasks, but where speed of the vehicle is low.
In all cases, these future ADAS require developing advanced systems of perception.
ADVANCED PERCEPTION
Perception consist in detecting objects, clustering, and possibly tracking them in their own trajectory, from selected sensors (cameras, radar, lidar, slam, ultrasound, …)
It is usually presented as several phases :
. Detection: we perceive that « something » comes off the background, but we do not know what it this is. The Johnson criteria for detection give a theoretical limit of one period, or a minimum width of two pixels to detect a stationary object.
. segmentation and tracking: when zones are detected as being detached from the background (the landscape for image processing, the cluter for a radar, …), the detection must be agglomerated to track large enough objects that may have a meaning.
. Recognition: Recognition is to be able to say what it is. The Johnson criteria for human vision is about 6 periods (for stationnary objects) which gives 12 pixels.
. identification: identification gives, in the recognized class, the precise name of the object.
Detection is by far the most complex. It is potentially based on several principles:
. breakage hypothesis : we made a number of assumptions about world geography. We choose this hypothesis and make sure they are verified for the landscape (or cluter), and not for the objects to be detected. The non-validation of assumptions corresponds to a detection.
. the confrontation of a knowledge of the landscape or cluter: Comparing the « background » as it is supposed to appear in the absence of additional objects with said background which contains objects lead to detection of those objects.
. the knowledge of the shape of the objects to be detected: in this case the detection and pattern recognition are the same. System detects an object in its environment because it recognizes this object.
Human perception jointly implements the three principles.
Perceptions systems incorporate sensors and methods of processing, and are generally effective in a frame capture conditions, and little or not effective in the other frames. For example, a camera in the visible wavelenghts (and its image processing methods), will generally not be effective at night or in fog because « you can not see anything. »
PERFORMANCE, STRENGTH, RELIABILITY
No detection system can operate in any case when dealing with a real problem in the open world.
Designing a detection system then comprises two important phases:
. extend the maximum possible number of cases where the detection system works.
. have a diagnosis that allows to know when it is or when it is not in a position to that the perception system is effective.
We talk about performance (very efficient detection of all objects of interest), strength (number of cases where the collection system remains effective), and reliability (Situational Awareness in which one is and thus the confidence that can be placed in the collection system).
These three elements, performance, robustness, reliability, should be fully known in order to cooperate collection systems (for example, a camera and a radar).
NEXYAD proposed the Methodology AGENDA for characterizing life situations, using the formalism of orthogonal plans of experiments. The recognition of cases of functioning mode can be based on the description of life situations with this methodology. This gives a theoretical and practical framework for an estimation of robustness and reliability.
Performance is measured with statistical comparison operators: in general, it is considered the output of a detection system is a categorical variable with two categories: « detected » and « not detected ». This variable must be compared to a qualitative variable of reference that also has two modalities: « Presence of an object to
be detected » and « absence of objects to be detected. » The comparison can not be made by calculating a percentage (yet it is often that performance is measured this way), but it must use tools such as contingency table, the Khi2, normalized Khi2, khi2 in the box, etc …
To extend the life situations of the domain where the system detects objects correctly, we use to make cooperate several detection systems which use complementary types of sensors (eg in fog, we will trust in radar or infrared detection, but not detection by conventional camera).
A reliable system is one that is able to answer « I do not know »: in the case of driving delegation a system that could detect all objects so powerful, robust, and reliable in 30% of the time has a great value.
The delegation of driving frees 30% time of the driver, which is a real value proposition.
SAFETY OPERATION
Safety is a discipline that encompasses many issues with the objective of ensuring the proper functioning of the system in all cases.
In particular, we must be vigilant concerning detection systems which require to have several measurement channels, such as stereovision.
If detection works only when you can have both cameras, then safety experts refuse such a system because two cameras means 2 times more likely that one fails.
We then see that perception system must have quite still usable « degraded mode » when simulating glitches sensors. A good design of a perception system for ADAS incorporates all these elements.
SYNTHESIS
The race for performance that interests the engineers is rarely the real issue in industrial systems. A system that allows to delegate the driving in 30% of cases (eg clear overcast day dry) and « knows » when there is a case for which it works or does not work, can delegate driving and release the driver for 30% of the time.
This is a proposal for a very high value for the driver.
A system that works effectively in 99% of cases without knowing precisely when it works is absolutely unusable. No manufacturer will put such a system in operation for road safety applications.
The company NEXYAD has been working on these issues for twenty years, especially on road detection, obstacles detection, measurement of visibility (to describe cases where the detection is reliable, for example), the estimation of road safety (suitability driving style with the infrastructure).
NEXYAD developed:
. efficient and very robust basic bricks: RoadNex, ObstaNex, VisiNex onboard, SafetyNex
. a methodology for characterizing life situations in which it develops and tests an ADAS: AGENDA (Improvement performance, the recognition of cases of good performance, and validation of ADAS).
. know-how in collaboration between multiple perception systems.
Nexyad started building a Validation Database for ADAS
Nexyad started building a Validation Database for ADAS based on methodology AGENDA: the ultimate way to control performance, robustness, and efficiency range of your camera_based ADAS.
To read more : HERE
Validation database for camera-based ADAS
Version française plus bas
NEXYAD Automotive & Transportation Newsletter #4, the 26th of August 2015
The company NEXYAD started building a database for validation of advanced driver assistance systems (ADAS and Autonomous car) using the methodology AGENDA published in the 90 by Gérard Yahiaoui (methodology initially developped for control construction of learning and test databases for the implementation of artificial neural networks).
This database has two essential characteristics:
1) Known life situations
Indeed, the methodology AGENDA proposes to describe potential changes of signals and images came into factors of variability and their crosses.
Example, for obstacle detection :
. weather (dry overcast, sunny weather, rain, fog)
. overall brightness (low, medium, high)
. speed of the carrier vehicle (low, moderate, high)
. type of road (highway, road with marking, road without marking …)
. coating (bitumen 1, bitumen 2, …, cobblestones)
. day / night (headlights and the lights switched infrastructure)
. season (spring, summer, autumn, winter)
. etc …
> type of obstacle :
– stopped
. infrastructure-related: work terminals, tolls, …
. related users: tire on the road, parcel felt from a truck lying on the road, biker following a road accident, disabled vehicle stopped on the floor, standing pedestrian on roadside edge (dodger / no sniper)
– moving
. truck, car, vulnerable (pedestrian, bicycle, motorcycle) each with types trajectories (longitudinal
in rolling direction, longitudinally in the opposite direction of rolling side) and position (opposite
to right, left).
. Etc…
We see that if we cross these factors, we find fairly quickly a huge number of cases. However, the development of ADAS systems is complex, and it is necessary to proceed by successive iterations, starting from simple situations to move to complicated situations.
Our database allows this, since all records are described in terms of crossing the terms of the factors of variability. Thus knows exactly which cases were tested or not by the system.
Formalism ‘crossing of variability factors of the terms’ allows using design of experiments, and in particular orthogonal fractional plans to sharply reduce the number of cases to be tested while ensuring maximum coverage of life situations. One can in this context to develop a fractional ADAS on an orthogonal plan and test other hard fractional orthogonal planes for example.
2) Reality reference
This is to crop images barriers and infrastructure elements (markings, roadsides, etc.) so as to constitute a reference to measure system performance.
. Examples of life situations:
1.1, summer, overcast, unmarked road, moderate speed tire on the floor, dry weather
1.2, summer, overcast, unmarked road, moderate speed, parcels on the floor, dry weather
2.1, summer, overcast , unmarked road, moderate speed, standing pedestrians non ambush at the edges of the floor, dry weather
2.2, summer, overcast, unmarked road, moderate speed, lying on the floor human, dry weather
etc …
Not sure that you would meet those few cases, even with on million kilometers on open roads.
NEXYAD starts his collection of images and data:
. video (towards the front of the vehicle) Color
. accelerometers
. gyros
The files are synchronized by RT-MAPS tool INTEMPORA society.
The files are saved as RT-MAPS format and replayable directly by this tool.
NEXYAD currently looking for contributors on this internal project. Co contributors fund and in return free access to the database, unlimited in time. This contribution will accelerate the work of collecting and labeling.
NEXYAD wishes to provide this basis before June 2016, free way to give the material to the community and the ADAS autonomous vehicle for a smaller version of the database, and pay way (as subscriptions) for complete database.
NEXYAD’s ambition is to spread its methodological expertise and allow everyone to assess the performance of vision systems for ADAS, whether systems developed by NEXYAD, or others.
“Methodology for ADAS Validation: Potential Contribution of Other Scientific Fields Which Have Already Answered the Same Questions”, Gérard Yahiaoui, Pierre Da Silva Dias, CESA congress Dec 2014, Paris, proc. Springer Verlag
“Methods and tools for ADAS validation”, Gérard Yahiaoui, Nicolas du Lac, Safetyweek congress, May 2015, Aschaffenburg
For questions, or if you wish to become a contributor, please contact NEXYAD : +33 139041360
Base de données de validation des ADAS utilisant des caméras
NEXYAD Automotive & Transportation Newsletter n°4, le 24 août 2015
La société NEXYAD démarre actuellement la construction d’une base de données pour la validation des systèmes d’aides à la conduite et de délégation de conduite (ADAS et Autonomous car) en utilisant la méthodologie AGENDA publiée dans les années 90 par Gérard Yahiaoui (méthodologie au départ destinée à maîtriser entre autre la construction des bases de données d’apprentissage et des tests pour la mise en œuvre des réseaux de neurones).
Cette base de données a deux caractéristiques essentielles :
1) Situations de vie
En effet, la méthodologie AGENDA préconise de décrire les variations possibles des signaux et images d’entrées en facteurs de la variabilité et leurs croisements.
Exemple, pour de la détection d’obstacles :
. météo (temps sec couvert, temps ensoleillé, pluie, brouillard)
. luminosité globale (faible, moyenne, forte)
. vitesse du véhicule porteur (faible, modérée, grande)
. type de route (autoroute, route avec marquage, route sans marquage, …)
. revêtement (bitume 1, bitume 2, …, pavés)
. jour / nuit (phares et éclairages de l’infrastructure allumés)
. saison (printemps, été, automne, hiver)
. etc …
> type d’obstacle :
– arrêté
. liés à l’infrastructure : bornes de travaux, péages, …
. liés aux usagers : pneu sur la chaussée, colis tombé d’un camion, motard allongé sur la
route suite à un accident, véhicule en panne arrêté sur la chaussée, piéton immobile sur le
bord de la chaussée (embusqué / non embusqué)
– en mouvement
. camion, voiture, vulnérable (piéton, vélo, moto) avec à chaque fois les trajectoires types (longitudinale dans le sens de roulage, longitudinale dans le sens inverse du roulage, latérale) et la position (en face, à froite, à gauche).
. Etc…
On constate que si l’on croise ces facteurs, on trouve assez rapidement un nombre de cas énorme. Or, la mise au point des systèmes ADAS est complexe, et il est nécessaire de procéder par itérations successives, en partant de situations simples pour aller vers les situations compliquées.
Notre base de données permet cela, puisque tous les enregistrements sont décrits en termes de croisements des modalités des facteurs de la variabilité. On sait ainsi exactement dans quels cas on a testé ou pas le système.
Le formalisme de ‘croisement des modalités des facteurs de variabilité’ permet d’utiliser les plans d’expériences, et en particulier les plans fractionnaires orthogonaux pour réduire fortement le nombre de cas à tester tout en garantissant une couverture maximale des situations de vie. On peut dans ce cadre mettre au point un ADAS sur un plan fractionnaire orthogonal et le tester dur d’autres plans fractionnaires orthogonaux par exemple.
2) Réalité terrain
Il s’agit de détourer sur les images les obstacles et éléments de l’infrastructure (marquages, bords de route, etc) de manière à constituer une référence permettant de mesure la performance du système.
. Exemple de situations de vie :
1.1, été, temps couvert, route sans marquage, vitesse modérée, pneu sur la chaussée, temps sec
1.2, été, temps couvert, route sans marquage, vitesse modérée, colis sur la chaussée, temps sec
2.1, été, temps couvert, route sans marquage, vitesse modérée, piétons immobiles non embusqués au bords de la chaussée, temps
2.2, été, temps couvert, route sans marquage, vitesse modérée, humain allongé sur la chaussée, temps sec
etc …
Il n’est pas certain que l’on puisse rencontrer ces quelques cas, même en roulant 1 million de km sur route ouverte !
NEXYAD démarre son recueil d’images et de données :
. vidéo (vers l’avant du véhicule) couleur
. accéléromètres
. gyromètres
Les fichiers sont synchronisés par l’outil RT-MAPS de la société INTEMPORA. INTEMPORA.
Les fichiers sont enregistrés au format RT-MAPS et directement rejouables par cet outil.
NEXYAD cherche actuellement des contributeurs sur ce projet interne. Les contributeurs co financent et ont en retour un accès gratuit à la base de données, illimité dans le temps. Cette contribution permettra d’accélérer le travail de recueil et d’étiquetage.
NEXYAD souhaite mettre à disposition cette base avant Juin 2016, de manière gratuite pour donner de la matière à la communauté des ADAS et du véhicule autonome, pour une version réduite de la base, et de manière payante (sous forme d’abonnements) pour la base complète.
L’ambition de NEXYAD est de propager son expertise méthodologique et de permettre à chacun d’évaluer les performances des systèmes de vision pour les ADAS, qu’il s’agisse des systèmes développés par NEXYAD, ou d’autres.
“Methodology for ADAS Validation: Potential Contribution of Other Scientific Fields Which Have Already Answered the Same Questions”, Gérard Yahiaoui, Pierre Da Silva Dias, CESA congress Dec 2014, Paris, proc. Springer Verlag
“Methods and tools for ADAS validation”, Gérard Yahiaoui, Nicolas du Lac, Safetyweek congress, May 2015, Aschaffenburg
Pour toute question ou pour devenir un contributeur, contactez NEXYAD : +33 139041360
« Groupement ADAS » Blog published a paper from Gérard Yahiaoui NEXYAD
« Validation of Advanced Driving Assistance Systems » a paper from Gérard Yahiaoui NEXYAD and Nicolas Dulac INTEMPORA was published by the french cluster « Groupement ADAS » on his site : HERE
Obstacles detection for ADAS, ObstaNex KPIs
An overview of Nexyad module ObstaNex’s performances :
22nd ITS World Congress, 5-9 October 2015
NEXYAD Automotive & Transportation will be present in Bordeaux for ITS World Congress. On the french R&D competitiveness cluster MOV’EO’s « Groupement ADAS » booth.
Improvement of Road Detection with Nexyad RoadNex V2.1
We have been working on RoadNex to improve it. This version 2.1 show the road surface and texture much further (red overprint). There are still no markings …
Road Detection & Road Safety Estimation – NEXYAD Automotive & Transportation Newsletter #3
Road Safety and ADAS – NEXYAD Automotive & Transportation Newsletter #2
NEXYAD at the Mov’eo Days in Poissy, Forum Armand PEUGEOT
NEXYAD was on the booth of the “Groupement ADAS” at Mov’eo Days.
Demos of onboard modules for road safety and autonomous vehicle.
NEXYAD member of VeDeCom
NEXYAD is now a member of the VeDeCoM research foundation as donator member.
NEXYAD is involved in research on ADAS and Autonomous Vehicle and is happy to contribute to the recruitment of young searchers by VeDeCom.
Presentations of NEXYAD at SafetyWeek in Germany
At the safety week symposium in Aschaffenburg (Germany), NEXYAD presented :
. A paper about ADAS validation : methodology and tools
. The products on the booth : RoadNex (road detection), ObstaNex (Obstacles detection),
VisiNex Onboard (visibility measurement), SafetyNex (estimating safety level of driving).
