Tag Archive: adas validation database


Validation of ADAS Nexyad Database

The NEXYAD company is currently developing the construction of a database for the validation of systems of driver assistance and driving delegation, (ADAS and Autonomous car) using the AGENDA methodology published in the 1990s by Gérard Yahiaoui (methodology initially intended to handle, among other things, the construction of learning databases and tests for the implementation of neural networks).

This database has two essential characteristics:

1) Real-life situations
     Indeed, the AGENDA methodology recommends describing the possible variations of signals and input images as factors of variability and their crosses.
     Example, for obstacle detection:
          . weather (dry weather, sunny weather, rain, fog)          . overall brightness (low, medium, high)
          . vehicle speed (low, moderate, high)
          . type of road (motorway, road with marking, road without marking, …)
          . coating (bitumen 1, bitumen 2, …, pavers)
          . day / night (car headlights and infrastructure lighting)
          . season (spring, summer, autumn, winter)
          . etc.

     Type of obstacle:
          – static obstacle
          . linked to the infrastructure: works terminals, tolls, …
          . related to users: tire on the roadway, package dropped from a truck, motorcyclist lying on the road following an accident, vehicle broken down stopped on the roadway, pedestrian stationary on the edge of the roadway (visible / hidden)
          – moving obstacle
          . truck, car, vulnerable (pedestrian, bike, motorcycle) with the typical trajectories (longitudinal in the direction of travel, longitudinal in the opposite direction of travel, lateral) and the position (opposite, right, left) .
          . Etc.

Real-life situations examples :
Validation of ADAS Nexyad database

We see that if we combine these factors, we can find quite quickly a very large number of cases. Now, the development of ADAS systems is complex, and it is necessary to proceed by successive iterations, from simple situations to complicated situations.
Our database allows this, since all records are described such as cross-referring of modalities of the factors of variability. We thus know exactly in which cases the system was tested or not.
The formalism of ‘cross-referring of modalities of the factors of variability’ makes it possible to use experimental designs, and in particular orthogonal fractional plans, to greatly reduce the number of cases to be tested while guaranteeing maximum coverage of real-life situations. In this context, we can develop an ADAS on an orthogonal fractional plane and test it for other orthogonal fractional planes, for example.

2) Ground reality
     It is a question of define the obstacles and elements of the infrastructure (markings, road edges, etc.) on the images so as to constitute a reference allow measuring the performance of the system.

          1.1, summer, cloudy, unmarked road, moderate speed, tire on roadway, dry weather
          1.2, summer, cloudy, unmarked road, moderate speed, package on roadway, dry weather
          2.1, summer, cloudy, unmarked road, moderate speed, visible static pedestrians at the edges of the roadway, time
          2.2, summer, cloudy, unmarked road, moderate speed, human lying on roadway, dry weather

     It is not certain that one can meet these few cases, even by driving 1 million km on open road!

Target of this database :
     NEXYAD starts its collection of images and data:
          . video (towards the front of the vehicle) color
          . accelerometers
          . gyros

The files are synchronized by the RT-MAPS tool of the company INTEMPORA.
The files are saved in RT-MAPS format and directly replayable on this tool.
NEXYAD is currently looking for contributors on this internal project. Contributors co fund and have in return free access to the database, unlimited in time. This contribution will accelerate the work of collection and labeling.
NEXYAD wishes to make this database available soon, free of charge to give material to the ADAS community and the autonomous vehicle, in a reduced version of the database, and in a paid way (in the form of subscriptions) for the complete base.

NEXYAD’s ambition is to propagate its methodological expertise and to enable everyone to evaluate the performance of vision systems for ADAS, be they systems developed by NEXYAD or others.

Methodology for ADAS Validation: The Potential Contribution of Other Scientific Fields Which Have Already Answered the Same Questions”, Gérard Yahiaoui, Pierre Da Silva Dias, CESA congress Dec 2014, Paris, Proc. Springer Verlag

“Methods and tools for ADAS validation”, Gérard Yahiaoui, Nicolas du Lac, Safetyweek congress, May 2015, Aschaffenburg


Autonomous Vehicle Test & Development : second day

Wenesday 1st of June : Presentation this morning by Gérard YAHIAOUI President & CEO of NEXYAD, France

Validation of camera-based artificial vision systems applied on open world is a very complex issue. An HD colour camera may generate more than 65 000 power 2 000 000 different images (information theory), so it is not possible to test every possible message. We propose a deterministic approach for building a validation database using the AGENDA methodology that was developed and published in the 1990s for neural network database (learn & test) design.

A large audience attended to this conference that questions the way for Autonomous Vehicle ADAS validation.

Conference Stuttgart 2016
Gérard on the left, the conference’s audience on the top and the booth of Groupement ADAS from the mezzanine.





Detection of the road, detection of the lane, in front of the vehicle is now a « must-have »
for Advanced Driver Assistance Systems (ADAS) and of course for Autonomous Cars too.
Every R&D team is able to show cases of good detection. The difference between different
modules is robustness : ability to work in many cases (almost every cases).

For instance, robustness consideration led many big Automotive firms to interger the MOBILEYE
detection system : jus because MOBILEYE is more robust than detection systems developed by
those big firms. And robustness is not a matter of deployment : you won’t get a more robust
module is you put 10 000 developers on the project. You need time, big amount of data, and
« smart ideas ».

Note : This robustness definition leads to question on ADAS validation (« almost » every case is
not that well defined … how could we put some maths on those words). NEXYAD has been
developing an applied maths-based methodology for ADAS validation and is currently
recording a validation data base that will be soon available for free worldwide on the internet.

But let’s go back to road detection modules comparison.

There is another difference between road detection systems : do they need white markings
or are they able to work even without markings ?

NEXYAD founders has been working on road detection since the beginning of the 90’s and never
stopped (*). The NEXYAD team is one of the moste experienced team in the world about road detection.
That actually makes the difference, and RoadNex is a module that would take long to develop by
other teams. RoadNex is currently available on PC (windows, Linux) in the real time framework
RT-MAPS. RoadNex will be soon available :
. on electronic device of an Automotive Tier One Company
. on smartphones (so it works in real time on a smartphone usual processor ! try to compare to other modules)
(*) publication at a scientific congress in France in 1993 :
“Texture-based Image Segmentation for Road Recognition with Neural Networks”, G. Yahiaoui, M. de Saint Blancard,
Sixth international conference on neural networks and their industrial & cognitive applications NeuroNîmes93, EC2,
Nîmes, 1993,

In order to have an idea of what robustness means, here are some case used to test RoadNex :

How many kilometers should you drive to sample those few road scenes variations ?

For more information : http://nexyad.net/Automotive-Transportation/?page_id=412