PROCESS
The process deck shows how the thesis went through turbulent changes before taking up a final form. So many different simulations were animated to test the system's effectiveness in communicating complex emotions. Eliminating ambiguity as much as possible from the system was of great importance , in order to felicitate a better and cohesive mode of visual communication.
​
Since audio is stripped off , there is a need for the viewer to pay closer attention to visuals. The objective was to create a visual friendly system that can quickly feed the viewer with enough queues to follow along.
​
Narration in a movie is also taken care of by background music. There might be many intense and emotionally overwhelming scenes without dialogs .Watching a movie without sound would be absolutely monotonous even with closed or open captions because the essence and cinematic experience of a movie are lost entirely.
System - Grammar
Break down of the visual filter , a proper deconstruction showing how layers are sandwiched together
Top Layer
Middle Layer
Bottom Layer
Dialog
Music
Background
Script
Dialog
Expression
Script
Dialog
Music
Expression
Camera
Script
Functioning
The foremost stage in the process is to directly visualize the most basic and fundamental aspect of cinema ,which is audio. This is basically a visualizer that strips of the audio into respective components and analyzes based on the script. The components of audio include dialogues , background music , background sounds. To not merely convert audio to video , the system works always in conjunction with the script in order to provide scene specific visual queues . The intensity of first layer is determined by audio intensity analysis , which provides the audience with enough information about the sounds in a scene.
​
The output is an audio visualizer that appears around the subjects face driven by dialogs. The side bar containing rectangular strips driven by background music and sound.
Figuring out the emotion on the subjects face would require inputs such as facial expressions ( video input ) , tone of dialogs ( audio analysis ) and script .
The result is nothing but a colored shape masking out the subject's face. It is a layer that doesn't fade away even when the subject isn't looking straight into the camera ,since dialogs are delivered even without the subject in the composition. The layer gradually disappears only when the subject exits the scene according to the script.
The bottom most layer is what provides context. It will be hard to determine the context of a scene hence the system harnesses literally all aspects of the visual language to create the output. Inputs are both audio and video apart from utilizing the script. Shape , color and speed of particles at the back ,and rhythm of beat ( overlaid dots ) within the composition
Top Layer
Middle Layer
Bottom Layer
Result - Layered
When all 3 layers are super-imposed , we get a complete visual filter that communicates all parameters and replaces the impact that audio would have actually had.
Stacking Order
The motion piece above shows an example of the layering of manipulated visuals to create the fully furnished filter. Top most layer is for dialog , the middle one is for emotion and the bottom most layer provides context.
Iterations
4 emotions - Rage , Grief , Warmth and Vulgarity. This thesis , showcases both simulations for obvious emotions as well as complex ones .
Themes
The overlay can be adjusted based on personal preferences while watching movies
Variation - Individual Emotion
More variations showing change in speed of particle simulations in the filter , the variation of base speed can be altered and the roundness of shapes ( particular ) can be tweaked based on personal preferences. Depending on relative pixel movement the system will adapt with the help of AI
See solutions ...
Determining Intensities
The intensity levels of each scene is determined by change in warmth of colors , speed and tempo of beats , velocity of computer generated particles in the background and sharpness of basic shapes in the visual system
Lower Intensity
Light Color
Blunt - Circular
Slow
Gentle Beat
Color
Shape
Particle Speed
Rhythm
Higher Intensity
Saturated
Dark Color
Sharp
Fast
Rapid Beat
Relative states of Intensities
In all of the mentioned parameters ,we find that the intensity gradually increases towards one end , and the states corresponding to each intensity level can be determined by shape ,color or beat. The brightest color is subtle , while the dark and saturated colors are overwhelming. Softest object is subtle, while the sharpest is for an extreme state. Note that these intensities actually mean degrees or states, not specifically for visualizing audio wave-forms. Parameters like particle speed and rhythm may not be applied to all filters in the same way ( example , for a GIF or Music only visual information for underlying context is provided. The system creates a full fledged filters with multiple layers only for feature films.
Color versus "Color and shape"
It is obvious that by combining these four colors ( Red , Green , Blue , Yellow ) with shapes ,the system can technically eliminate the use of additional colors with varying hue-saturation values to precisely determine the intensity of scene. The more visual information we add to the filter the more effective it becomes in communicating context but at the same time it becomes unnecessary to a certain extent.
This leads to the problem of having a sort of visual redundancy in the system . But it is important to note that such redundancy only assists the viewer ,by giving them not " additional information" but confirming what is visually already presented to them. Hence using a wide range of additional colors in conjunction with shapes will make things a little clear for the viewing audience. Also, creating a color chart that can serve as an alternative to usage of both " color and shape " simultaneously can be used for other purposes.
Concept of utilizing visual redundancy
The dependency on using a range of additional colors to convey the same message as a color being used in conjunction with a shape is vital. It really repeats the visual information twice but helps the audience pick up the message in scenes quickly. Sometimes the viewer might not even notice it .For example , although using a "bright yellow " could mean the same thing as using "yellow" with a "circle" provided the beat or speed of particles for communicating intensity is the same.
The dependency on using a range of additional colors to convey the same message as a color being used in conjunction with a shape is vital. It really repeats the visual information twice but helps the audience pick up the message in scenes quickly. Sometimes the viewer might not even notice it .
​
Theoretically having more colors for different shapes increase the permutations that can exist ,but the brain wont be in a position to assimilate that much information. But it is important to note that those subtle variations in color may exaggerate the effect and feed the information quite blatantly.
​
For example , although using a "bright yellow " could mean the same thing as using "yellow" with a "circle" provided the beat or speed of particles for communicating intensity is the same.
Color I
Shape II
Color II
Color II
Shape II
Redundant
Numerically assigned values for visuals
Creating a visual chart to associate colors with numbers can lead to better and much more accurate ways to convey mood of an entire scene or even the whole film. It becomes more like giving a movie a rating based on the intensity levels of scenes. Creating a system to assign corresponding numerical values to colors can provide a path way for us to come up with a new numerical rating system. This approach can give further information about the changes in mood or the genre of a movie through the use of numbers. Movie posters and song covers could have such ratings on them.
​