Archives de la Catégorie : Articles

sept
30

AUTOMATIC COLOR GRADING USING THE NEXYAD SKINS

NEXYAD TECHNOLOGY FOR IMAGE, VIDEO, AND MOVIE POST-PRODUCTION :
AUTOMATIC COLOR GRADING USING THE NEXYAD SKINS

by NEXYAD

INTRODUCTION

NEXYAD has developed technology for applying color charts at images during post-production, to greatly change the atmosphere : the automatic color grading, that we called the NEXYAD Skins.

This automatic color grading applies equally to the rush or to contrast and luminance calibrated images and provides calibrated recolored images, in a dynamic already validated by technicians and artists for movie.

With our technology, it becomes difficult to make « bad taste » renders, or they are made voluntarily and then have a strong artistic scope.

The principle of our automatic color grading of automatically granting colors so that the rendering is harmonious.

We act as a musician on the piano : if each color corresponds to a music note, and thus a key, then playing multiple notes at the same time is putting a set of colors side by side in the same image. Question is : is it harmonious ?

The music is governed by rules called « rules of harmony. » For the same dominant note such as the A note, one can construct chords (set of notes played simulanément) containing the A note, but which give a completely different sensory and emotional experience.

Figure 1
The note A on the piano keyboard

Examples of music chords :

. A Major : A note is dominant and the atmosphere is cheerful

Figure 2
Chord « A Major » that includes the note A

. A minor : A note is dominant and the mood is sad

Figure 3
Chord « A minor » that includes the note A

. D minor : A note comes in support of another note (D which is dominant), the atmosphere is sad
the global tune id not A, it is D.

Figure 4
Chord « D minor » that includes the note A

. etc …

On a piano, you don’t put your fingers anywhere because if you do it is not called music, it is called « noise ».

For colors, that’s exactly the same thing : all colors do not go together, and there are very many different combinations of colors which generate a different harmony.

These color groups have been studied among others by artists (Kandinsky, the Bauhaus school, …), which proposed what is called « colors grammars ».
When capturing images, if the colors in the image belong to a colors grammar, then rendering is harmonious, and the atmosphere is that of the grammar (dynamic, joyful, sad, blue, etc …).

There are several solutions for your image to be compatible with a grammar of colors :

. Solution 1: you know the grammars of colors, you know the sensations and impressions they generate, you choose one, and you ensure that the photographed or filmed elements have precisely the right colors there. It takes a lot of knowledge, and also ways for sets, costumes, etc …
Note : apart in productions with very large budget, it is rare for outdoor shoots that this kind of method can apply.

. Solution 2: you know the grammars of colors, you know the sensations and impressions they generate, you choose one, and you change some colors of your rush image, by hand, in post production to bond grammar that interests you. It takes a lot of knowledge, software tools for color grading, and also human resources and time.

. Solution 3: you apply NEXYAD automatic color grading solutions (NEXYAD skins) and see what it generates on YOUR images, or else you give NEXYAD adjectives that describe the mood and NEXYAD will choose for you.

If you think your photographs and films in terms of color and chords, with sensory emotional load generated on the viewer-s brain, regardless of the image content and the story being told, you have an efficient way to work your artistic content, and you are free to choose clear artistic directions such as :

. consistency: the story told, the content of images, colors and grammar are fully consistent
(Eg : it tells a sad story, filmed objects are old, dirty … and the colors give an impression of sadness). If you add a sad music, you get a maximum of sadness message from your work to the viewer.

. shift: certain elements of the work contradicts the others.
Eg : the story told is dramatic, image content is joyful, the colors give a naive rendering.
If you add a childlike music, the viewer sees an inconsistency and generates itself a strong emotion to offset the discrepancy. Your work then integrates the spectator within itself instead of projecting toward the viewer…

. etc …

The perfect control of color rendering brings an effective tool for artists in photography and cinema.
note : saying that your creativity is higher than that and that you need to be completelu free of rules in your creation means that a piano doesn’t have enough keys for you … It is possible (yes it is), but think twice before saying it. Many music genius before used the piano to create wonderful pieces of music, and no one would say they are not artists : Mozart, Beethoven, …


TECHNICAL EXPLANATION : AUTOMATIC COLOR GRADING AND LUMINANCE PROCESSING

Let’s take the case of a scene that contains 3 colors, and let’s say that you want to apply a grammar of colors that need to change every original color.

The NEXYAD tool (NEXYAD Skin) will take original colors and will choose which final color thay should be tranformed in :

Nexyad : Colors in rush image

Some colors may be kept identical, other modified to stick to a color of the grammar, and finally other fully removed and replaced with a new color.

NEXYAD technology can do that … automatically.
You select the grammar of colors, and NEXYAD Skin manages to harmonize colors. This exceptional result is the fruit of a very high level research conducted since 1995 by NEXYAD on perception of color and advanced image processing.

Regarding the brightness, we can pass from the luminance dynamics of a rush image to the dynamics of the calibrated image using a « Calibration curve ». If the dynamics of final rendering is always the same but the dynamics of the rush picture varies (unexpected variation of lights, camera change, …), then it’s never the same curve that is used for calibration.
The NEXYAD Skin computes the proper curve for each image fully automatically.

EXAMPLE

In this example, we apply several NEXYAD Skins on the same film. This does not mean that all colors grammars are interesting for all films, this is only a technical demonstration to all potential partners and users.
For a given artistic direction and a given sequence, there are only a few NEXYAD skins that may be interesting.
note: you can customize the automatic renders, using simple parameters like saturation, etc. Then you’ll get absolutely unique renders.

Here is a picture from a royalty free movie bought by NEXYAD on the internet (you may buy it too if you want to try color calibration renders) :

Nexyad Skin Green & Orange


Nexyad Skin 3-strip Technicolor


Nexyad Skin Golden

Nexyad_Skin_Steel&Pink
Nexyad Skin Steel & Pink

Nexyad_Skin_Teal&Orange
Nexyad Skin Teal & Orange

Nexyad Skin Sovcolor
Nexyad Skin Sovcolor

Nexyad Skin Green & Pink
Nexyad Skin Green & Pink


sept
24

NEW TECHNOLOGY FOR COLOR GRADING : THE NEXYAD SKINS

(French Version)
NEW TECHNOLOGY FOR COLOR GRADING : THE NEXYAD SKINS

by NEXYAD



INTRODUCTION
The digital post production has merely reproduced the old post production phases, using software whose
interfaces were comparable to those of physical machines previously used.
Eg interfaces using the « slider », the « button », etc … all of which were essential to set an analog electronic
machine or a mechanical machine or mechatronics.
But digital also allows to completely change the way of thinking and will soon revolutionize workflows in
post-production film and television.
Indeed, suppose that there are two control buttons on a machine (eg overall luminance and overall contrast),
we can draw this:
Luminance & Contrast
The rectangle materializes all possible settings of the image using these two buttons which can rotate independently of one another.

« Turning the knobs » to go from one setting # 1 to # 2 setting means « moving » in the diagram:
Contrast Button

With digital technology, one can easily reproduce this mode, but you may also « pave » the rectangle with
predefined positions that have been judged as « interesting » by experts and then « jump » from one position to another:
Contrast settings

One can imagine letting the user move in « button mode  » around predefined positions … Then all areas of the rectangle can be achieved, but preference is given to Presets which has already been validated as aesthetic.

This means that instead of « moving » continuously from one setting to another, we now « jump » from one setting already validated to another validated setting.
Advantage: jumping does not go through the settings which lead to unpleasant images, one goes straight on already validated settings.

CRITICISM OF THE METHOD
It can be argued that this approach is to spend « tailored » to the « ready to wear ».

Yes, but for those who do not have the budget to pay for custom-made, the solution is very inefficient: you’ll see a poor tailor who certainly takes your measurements, but then is not very well paid for quickly making a bad cut in poor tissues, and your suit at the end is not very beautiful.
You have the illusion of tailor-made, but you do not have the budget to turn the knobs very far.

In a « ready-made » solution, if the number of different costumes for sale is very large, in one hour you try dozens. It is much more efficient and cuts can be very sophisticated because stylist work cost was made one time for tens of thousands of customers.
Our response therefore is to know where you are in the chain of budgets : low ? average ? high ?
Furthermore, nothing prevents you from taking the ready to wear and do everything custom touch then.
Our solution is optimal in terms of price / quality ratio.

Moreover, the argument of « creativity » clamped is also very questionable: when one has no budget to work for a long time involving global experts of the field, then, turning knobs, and even if the result is aesthetically acceptable, it is likely to be find a result that is already known and already used repeatedly by other filmmakers.
This is not because we do not know anything that we are original. The application of N color grading (example: twenty) on your film, even if it is only to « see », will help you do your own way and making sure you are original.

EXAMPLES OF NEXYAD SKINS
The images below are presented for illustration. With NEXYAD SKINS, you don’t need to know how the images were initially, simply look at the end result, choose one that you like, and you can get it on your own images.
This is a revolution in post-production. Here are the first skins that NEXYAD has developed :
If you want your films to be this visual quality, contact NEXYAD.


Technology Demo Nexyad Skin "3-Strip" a la Technicolor 30's to 60's (film #2) from NEXYAD on Vimeo.



Technology Demo Nexyad Skin "Golden" (film #1) from NEXYAD on Vimeo.



NEXYAD Skin "Green & Orange" 02 – An Automatic Calibration & Color Grading Technology Demo from NEXYAD on Vimeo.



NEXYAD Skin "Green & Pink" 01 – An Automatic Calibration & Color Grading Technology Demo from NEXYAD on Vimeo.



Demo Technology Nexyad Skin "Sovcolor" 01 – Automatic Calibration & Color Grading from NEXYAD on Vimeo.



Technology Demo NEXYAD Skin "Teal & Orange" 01 – Automatic Calibration & Color Grading from NEXYAD on Vimeo.

sales@nexyad.net

sept
24

NOUVELLES TECHNOLOGIES POUR LE COLOR GRADING :
LES NEXYAD SKINS

NOUVELLES TECHNOLOGIES POUR LE COLOR GRADING : LES NEXYAD SKINS

par NEXYAD



INTRODUCTION
La post production numérique s’est jusqu’alors contentée de reproduire les phases de la post production
classique, à l’aide de logiciels dont les interfaces étaient comparables à celles des machines physiques
utilisées auparavant.
On retrouve par exemple dans les interfaces le « curseur », le « bouton », etc … autant d’éléments qui
étaient indispensables pour régler une machine électronique analogique, voire une machine mécanique
ou mécatronique.
Mais le numérique permet aussi de changer complètement la manière de penser et va bientôt révolutionner
les workflows de la post-production de cinéma et de télévision.
En effet, supposons que l’on dispose de deux boutons de réglage sur une machine (ex : luminance globale
et contraste global), on peut dessiner cela :
Luminance & Contrast
Le rectangle matérialise l’ensemble des réglages possibles de l’image à l’aide de ces deux boutons que l’on
peut tourner indépendamment l’un de l’autre.

« Tourner les boutons » pour aller d’un réglage n°1 à un réglage n°2 signifie « se déplacer » dans le diagramme :
Contrast Button

Avec la technologie numérique, on peut très bien reproduire ce mode de fonctionnement, mais on peut
aussi « paver » le rectangle avec des positions prédéfinies qui auront été jugées comme « esthétiques » par
des experts, puis « sauter » d’une position à l’autre :
Contrast settings

On peut imaginer laisser l’utilisateur se déplacer en mode « bouton » autour des positions prédéfinies si
bien que finalement toutes les zones du rectangle peuvent être atteintes, mais on donne une préférence à
des réglages prédéfinis dont on a déjà validé l’intérêt esthétique.

Cela signifie qu’au lieu de se « déplacer » continument d’un réglage à l’autre, on peut « sauter » d’un réglage
déjà validé à un autre.
Avantage : en sautant, on ne passe pas par les réglages qui conduisent à des images désagréables, on va directement sur des réglages déjà validés.

CRITIQUE DE LA METHODE
On peut argumenter que cette approche consiste à passer du « sur mesure » au « prêt à porter ».

Oui, mais pour tous ceux qui ont un budget réduit, voici ce qui se passe: ils vont voir un tailleur qui prend certes les mesures, mais qui ensuite doit travailler très rapidement pour être rentable, n’a pas le talent pour faire une belle coupe et n’a en stock que des tissus médiocres. A la fin, votre costume n’est pas beau, mais c’est, dit-il, le mieux qu’il puisse faire dans le budget.
Vous avez l’illusion du sur-mesure, mais vous n’avez pas le budget pour explorer toutes les possibilités qu’offrent les machines modernes avec du personnel non-impliqué et souvent peu talentueux.

Dans une solution « prêt à porter », si le nombre de costumes différents en vente est très grand, en une heure vous pouvez essayez des dizaines de styles. C’est bien plus efficace, et les coupes peuvent être très sophistiquées car le travail de styliste a été mutualisé sur des dizaines de milliers de clients. On vous propose même « à la manière » des grands couturiers que vous admirez.

Notre réponse est donc de bien savoir où vous êtes dans la chaîne des budgets.
Par ailleurs, rien de vous empêche de prendre du prêt à porter et de faire quelques retouches sur mesure
ensuite. Notre solution est optimale en termes de rapport qualité/prix.
Par ailleurs, l’argument de la « créativité » bridée est lui aussi très discutable : quand on n’a pas le budget pour faire travailler pendant longtemps les experts mondiaux du domaine, alors en général, en tournant des boutons, même si le résultat est esthétiquement acceptable, on a de fortes chances d’être retombé, et sans le savoir, sur des cas déjà connus et utilisés maintes fois par d’autres cinéastes.
Ce n’est pas parce qu’on ne connaît rien qu’on est original. L’application de N color grading (exemple : une
vingtaine) sur son film, ne serait-ce que pour « voir », et décider ensuite de faire autrement, est à notre
avis indispensable pour s’assurer de sa propre originalité.

EXEMPLES DE NEXYAD SKINS
Les images ci-dessous sont présentées à titre d’illustration. Avec les NEXYAD SKINS, plus besoin de vous
demander comment étaient les images au départ, vous n’avez qu’à regarder le résultat final, choisir celui
qui vous plaît, et vous pouvez l’obtenir sur vos propres images.
C’est une véritable révolution dans la post-production. Voici les premier skins que NEXYAD a développés :
Si vous souhaitez que vos films soient de cette qualité visuelle, contactez NEXYAD.


Technology Demo Nexyad Skin "3-Strip" a la Technicolor 30's to 60's (film #2) from NEXYAD on Vimeo.



Technology Demo Nexyad Skin "Golden" (film #1) from NEXYAD on Vimeo.



NEXYAD Skin "Green & Orange" 02 – An Automatic Calibration & Color Grading Technology Demo from NEXYAD on Vimeo.



NEXYAD Skin "Green & Pink" 01 – An Automatic Calibration & Color Grading Technology Demo from NEXYAD on Vimeo.



Demo Technology Nexyad Skin "Sovcolor" 01 – Automatic Calibration & Color Grading from NEXYAD on Vimeo.



Technology Demo NEXYAD Skin "Teal & Orange" 01 – Automatic Calibration & Color Grading from NEXYAD on Vimeo.

sales@nexyad.net

mai
16

Digital post-processing of images for video sequences and movies : Noise Reduction (May 16, 2010)

Digital post-processing of images for video sequences and movies : Noise Reduction / why is it needed? why is it complicated to achieve?


Some basics

An image is a collection of coloured pixels located at spatial coordinates (x, y). Every pixel is composed of three values that are its projection on the components (R, G, B).
R is component of red.
G is component of green.
B is component of blue.

The three basic colours proportions (R, G, B), mixed (superimposed, and then, added) lead to the impression of a global only colour that draws on the resulting colour.

The relation between the three values R, G and B and the resulting colour is not totally intuitive, except in obvious cases (eg when R is very close to max value while G and B are near zero, the colour is RED).
When the R, G and B are equal, the pixel is gray (goes from black, for coordinates 0, 0, 0, to white for coordinates max, max, max).

The axis R = G = B is dotted in the drawing below : we draw in red the « triangle » that intercepts the sphere of centre (R, G, B) = (0, 0, 0) and radius ρ.
ρ is called the luminance of the pixel.
The more the plot is away from the dotted line, the more the pixel is coloured (angle θ determines the hue of the pixel. distance r from the axis determines the intensity of the colour effect. r is called saturation.
Those coordinates are called H, S, L :
Hue = H
Saturation = S
Luminance = L

Triangle des couleurs

Note that we can use several definitions of distance, as r can be calculated in a straight line or by arc-length along the sphere of radius ρ, in direction θ, etc. …,
Moreover, the human eye is not equally sensitive to a variation of the angle Δθ depending on the value of θ. This is why we perceive a rainbow as discontinuous colours, although it is a continuous variation of colours.
For all these reasons, there are many transformations of the coordinate system (H, S, L) which are intended to simplify the use of coordinates and to be as close as it can to the colours perceived by the human brain.
But these systems are only transformations of systems (R, G, B) or (H, S, L) described above.

The coordinates R, G, B have the disadvantage of being difficult to interpret (it is difficult, by reading the three coordinates, to deduce the seen colour, except in trivial cases), but they have the advantage of being always defined.

Conversely, the coordinates H, S, L, (and their variants) have the advantage of being easy to interpret, but they are not always defined :
- When a pixel is almost black (L near zero), the angle (H) can take any value, for exactly the same colour impression. For the absolute black (L = 0), the angle is formally not defined.


Improving Luminance contrast

Improving the contrast is to increase the distance between two or more classes of luminance. We draw below the cone of colours (cutting):

Luminance en coupe


















We define thresholds (arbitrarily) that define classes of luminance. For example, we can define a high threshold beyond which it is considered as ‘white’, a low threshold below which it is considered as « black. »
This configuration is often encountered : we want the blacks blacker, whites whiter, and we want that the intermediate gray levels remain unchanged. Such an operation is represented as follows:

coupe HSL pour contraste

What happens there in the presence of noise in the image (eg electronic noise of the sensor for a digital camera)?

noise level amplivication

The diagram above clearly displays that the noise is amplified when increasing the contrast.
Even if the noise was barely noticeable on the initial images, it is quite possible that it is widely visible on the post processed images.
Thus, the level of initial noise on images determines there the ability to enhance contrast.


Improving colours contrast

You can increase the colour contrast by saturating them (increase of S).
If you consider the triangle R, G, B, for a given luminance, this is as follows:

saturation couleurs








This leads to amplify

- Noise of saturation:

bruit de saturation

Occurrence of gray pixels within the saturated colour

- Noise of colour :

bruit de teinte

Occurrence of other colours within the saturated colour

Again, it is clear that the noise of the initial image is the factor that scales the ability to post process colours. Even if the noise was barely noticeable on the initial images, it is quite possible that it is widely visible on the post processed images.


Perception of noise in the image by the human brain

NEXYAD is involved for many years now in human factor analysis, and especially in human perception. The noise on the image is perceived via two main factors:

- On a still image, and is detected because of the contrast between the “clean” pixels and the noisy pixels.
Human vision has a very complex way of taking into account the contrast : our sensitivity to contrast depends on the global lightning level, the size of objects, and colours.
The analysis of the CSF (Contrast Sensitivity Function) of the human vision can show that the flat areas (eg a door, a smooth wall, a vehicle body, …) are the areas on which the spatial high frequency noise is the most detectable.

- On a video sequence, we also detect noise with motion detectors that we have in the periphery of the eye.
The noise is random (ie : electronic noise of a digital camera), and each image is an independent release of this noise, so noisy pixels change their hue, saturation, and luminance, for each image.
The apparent movement of the noise allows us to detect it even with very low contrast that wouldn’t allow us to detect it on still images of this sequence.


Noise reduction in images

We can reduce the noise in the two ways above:
- reduce the contrast of the noise in each image (approach called spatial noise reduction)
- reduce the amplitude of the noise temporal variations (approach called temporal noise reduction)

The removal of spatial noise will tend to eliminate both the contours of objects (and hence make the overall image blurred).
Similarly, the elimination of temporal noise generates « ghosts » images due to group delay time of signal processing systems.

The reduction of noise in the videos is therefore a complex matter that requires the use of nonlinear mathematical methods whose application must be constrained by a psycho-senrory model of human vision (to distinguish which is perceived by our brain of what is not, and prioritize tasks of images cleaning). A preliminary analysis of the structure of images is usually required (using methods such as wavelet transform analysis).

NEXYAD has developed efficient solutions and has successfully tested them on video sequences.