Analysis of Image Processing Effects of Digital Camera Using Brain fMRI
Yoshiaki Kikuchi†, Madoka Noriuchi†, Kentaro Oba†,
Katsuhiro Takada‡, Shuta Yanagida‡, Akio Kosaka‡ and Takashi Miyoshi‡
† Graduate School of Human Health Sciences, Tokyo Metropolitan University.
‡ Olympus Corporation.
Katsuhiro Takada‡, Shuta Yanagida‡, Akio Kosaka‡ and Takashi Miyoshi‡
† Graduate School of Human Health Sciences, Tokyo Metropolitan University.
‡ Olympus Corporation.
Introduction
Pictures taken by professional photographers are beautiful and emotive. This is the result of more than just professional skill; it takes a special type of eye, one that can see the world in a frame. Yet there is another factor that may also contribute to the impact of professional photographs, and that is the application of image processing software — something that tends to be quite difficult for ordinary camera users.
It is said that the reason we are moved by a picture is that our emotions are stimulated at the same time as our memory when we recognize the subject. Assuming this to be true, we can say that professional photographers know how to stimulate emotion by emphasizing the main subject and directing the viewer’s feelings by means of image processing.
In this study, the authors examined the characteristics of and differences between photographs in which color and brightness were controlled to emphasize the main subject (emotion-weighted images) and those in which the overall chroma value was increased to emphasize the overall image color (color-enhanced images). For this purpose, the authors analyzed the subjective impressions produced by the images, as well as measuring brain activity produced when the images were viewed using functional magnetic resonance imaging (fMRI).
Pictures taken by professional photographers are beautiful and emotive. This is the result of more than just professional skill; it takes a special type of eye, one that can see the world in a frame. Yet there is another factor that may also contribute to the impact of professional photographs, and that is the application of image processing software — something that tends to be quite difficult for ordinary camera users.
It is said that the reason we are moved by a picture is that our emotions are stimulated at the same time as our memory when we recognize the subject. Assuming this to be true, we can say that professional photographers know how to stimulate emotion by emphasizing the main subject and directing the viewer’s feelings by means of image processing.
In this study, the authors examined the characteristics of and differences between photographs in which color and brightness were controlled to emphasize the main subject (emotion-weighted images) and those in which the overall chroma value was increased to emphasize the overall image color (color-enhanced images). For this purpose, the authors analyzed the subjective impressions produced by the images, as well as measuring brain activity produced when the images were viewed using functional magnetic resonance imaging (fMRI).
Methodology
Subjects:
Thirteen (13) healthy female subjects who were judged to be right-handers by Chapman’s dominant hand test (aged 18.4 ±0.5).
Images:
A total of ninety (90) images of 30 common scenes (landscapes, foods, portraits, etc.). Images were divided into original images, emotion-weighted images and color-enhanced images.
Subjective evaluation:
A subjective evaluation test using the 6-level Likert method and seven adjectives expressing the impression of each image to the representative six scenes (total 18 pictures) was applied to the 30 scenes (total 90 pictures) used for the brain fMRI. The seven evaluation adjectives included “vivid”, “terrific”, “favorable impression”, “beautiful”, “heartmoving”, “exciting” and “fascinating”.
We applied factor analysis to the results of subjective evaluation test to obtain the main evaluation factors and identify the trends of evaluation of subjects using factor scores. We then classified the subjects into two groups according to their image selection tendencies.
Brain functional imaging:
We showed image stimuli randomly, with a 3-second presentation period and 3-second rest period per image. The fMRI was conducted using a Philips 3.0T Achieva Quasar Dual with the event-related design. Imaging parameters were set to FOV of 230 mm, slice thickness of 5 mm, TE of 35 ms, TR of 4000 ms, 1 volume of 25 slices and flip angle of 90°.
Analysis of brain functional images:
After converting the DICOM images into the analyze format, we processed them with realignment, normalization and smoothing (FWHM = 8 mm) using SPM2. After this, we applied a general linear model (GLM) consisting of two regressors — the regressor for the brain activities with selected images and that for the brain activities with non-selected images — to all the subjects, as well as to each of the subject groups classified according to the subjective evaluation test results. We then compared the former with respect to the latter (contrast) and the weight of each regressor under the conditions of p < 0.001, uncorrected.
Subjects:
Thirteen (13) healthy female subjects who were judged to be right-handers by Chapman’s dominant hand test (aged 18.4 ±0.5).
Images:
A total of ninety (90) images of 30 common scenes (landscapes, foods, portraits, etc.). Images were divided into original images, emotion-weighted images and color-enhanced images.
Subjective evaluation:
A subjective evaluation test using the 6-level Likert method and seven adjectives expressing the impression of each image to the representative six scenes (total 18 pictures) was applied to the 30 scenes (total 90 pictures) used for the brain fMRI. The seven evaluation adjectives included “vivid”, “terrific”, “favorable impression”, “beautiful”, “heartmoving”, “exciting” and “fascinating”.
We applied factor analysis to the results of subjective evaluation test to obtain the main evaluation factors and identify the trends of evaluation of subjects using factor scores. We then classified the subjects into two groups according to their image selection tendencies.
Brain functional imaging:
We showed image stimuli randomly, with a 3-second presentation period and 3-second rest period per image. The fMRI was conducted using a Philips 3.0T Achieva Quasar Dual with the event-related design. Imaging parameters were set to FOV of 230 mm, slice thickness of 5 mm, TE of 35 ms, TR of 4000 ms, 1 volume of 25 slices and flip angle of 90°.
Analysis of brain functional images:
After converting the DICOM images into the analyze format, we processed them with realignment, normalization and smoothing (FWHM = 8 mm) using SPM2. After this, we applied a general linear model (GLM) consisting of two regressors — the regressor for the brain activities with selected images and that for the brain activities with non-selected images — to all the subjects, as well as to each of the subject groups classified according to the subjective evaluation test results. We then compared the former with respect to the latter (contrast) and the weight of each regressor under the conditions of p < 0.001, uncorrected.
Results
Subjective evaluation:
Factor analysis of all of the subjects and images showed that subjective evaluations were composed of two factors; an uplifting feeling (heartmoving, exciting and fascinating) and an aesthetic feeling (vivid, terrific, favorable impression, beautiful). We then calculated the total scores for the two factors given by each subject for the emotion-weighted images and color-enhanced images, and found that the subjects could be classified into two subject groups; those who scored emotion-weighted images higher (emotion-weighted image favoring group: 9 subjects) and those who scored color-enhanced images higher (color-enhanced image favoring group: 5 subjects).
We also found that there was a large difference in the scores given by the group that favored emotion-weighted images in response to emotion-weighted images and in response to color-enhanced images. On the other hand, the group that favored color-enhanced images showed a small difference in the factor scores but a high correlation of evaluation values between the evaluation words.
Brain functional images:
We attempted to compare the differences in brain activity, particularly those in the orbitofrontal cortices and frontal pole of each subject in response to emotion-weighted images and in response to color-enhanced images (main effects of stimulus images), as well as the difference between the brain activity of each subject group in response to the emotion-weighted images and color-enhanced images based on single regressor analysis.
Subjective evaluation:
Factor analysis of all of the subjects and images showed that subjective evaluations were composed of two factors; an uplifting feeling (heartmoving, exciting and fascinating) and an aesthetic feeling (vivid, terrific, favorable impression, beautiful). We then calculated the total scores for the two factors given by each subject for the emotion-weighted images and color-enhanced images, and found that the subjects could be classified into two subject groups; those who scored emotion-weighted images higher (emotion-weighted image favoring group: 9 subjects) and those who scored color-enhanced images higher (color-enhanced image favoring group: 5 subjects).
We also found that there was a large difference in the scores given by the group that favored emotion-weighted images in response to emotion-weighted images and in response to color-enhanced images. On the other hand, the group that favored color-enhanced images showed a small difference in the factor scores but a high correlation of evaluation values between the evaluation words.
Brain functional images:
We attempted to compare the differences in brain activity, particularly those in the orbitofrontal cortices and frontal pole of each subject in response to emotion-weighted images and in response to color-enhanced images (main effects of stimulus images), as well as the difference between the brain activity of each subject group in response to the emotion-weighted images and color-enhanced images based on single regressor analysis.
Main effects of stimulus images
When brain activity in response to emotion-weighted images was compared to that in response to color-enhanced images, activity in the left hemisphere was dominant and significant activity in the left and right frontal poles was observed as shown in the left half of Figure 1. On the other hand, when brain activity in response to color-enhanced images was compared to that in response to emotion-weighted images, those in the right hemisphere were dominant and no significant activity were observed in the frontal pole as shown in the right half of Figure 1.
When brain activity in response to emotion-weighted images was compared to that in response to color-enhanced images, activity in the left hemisphere was dominant and significant activity in the left and right frontal poles was observed as shown in the left half of Figure 1. On the other hand, when brain activity in response to color-enhanced images was compared to that in response to emotion-weighted images, those in the right hemisphere were dominant and no significant activity were observed in the frontal pole as shown in the right half of Figure 1.
Figure 1
Brain activity of each subject group in response to emotion-weighted images and color-enhanced images
With the group that preferred emotion-weighted images, emotion-weighted images caused significant activity in the left/right frontal poles and left/right orbitofrontal cortices (left half of Figure 2). For the differences between the left and right, the frontal pole activity are dominant in the left and the orbitofrontal cortex activity were dominant in the right. With the same subject group, image-emphasized images caused significant activity in the right frontal pole and orbitofrontal cortex (right half of Figure 2). On the other hand, with the group that preferred color-enhanced images, emotion-weighted images caused significant activity in the right orbitofrontal cortex (left half of Figure 3), and color-enhanced images caused significant activity in the left frontal pole (right half of Figure 3).
With the group that preferred emotion-weighted images, emotion-weighted images caused significant activity in the left/right frontal poles and left/right orbitofrontal cortices (left half of Figure 2). For the differences between the left and right, the frontal pole activity are dominant in the left and the orbitofrontal cortex activity were dominant in the right. With the same subject group, image-emphasized images caused significant activity in the right frontal pole and orbitofrontal cortex (right half of Figure 2). On the other hand, with the group that preferred color-enhanced images, emotion-weighted images caused significant activity in the right orbitofrontal cortex (left half of Figure 3), and color-enhanced images caused significant activity in the left frontal pole (right half of Figure 3).
Figure 2
Brain Activity in Response to Emotion-weighted Images in Group Preferring Emotion-weighted Images (Left) and in Response to Color-enhanced Images (Right). The areas enclosed in yellow circles are the orbitofrontal cortices and those enclosed in blue circles are the frontal poles.
Figure 3
Brain Activity in Response to Emotion-weighted Images in Group Preferring Color-enhanced Images (Left) and in Response to Color-enhanced Images (Right). The area enclosed in yellow circle is the orbitofrontal cortex and that enclosed in blue circle is the frontal pole.
Considerations
The subjects showed high right-hemisphere activity with color-enhanced images and high left-hemisphere activity with emotion-weighted images, regardless of their grouping. Much previous neuropsychological and psychological research has demonstrated that the right hemisphere is engaged in overall or intuitive information processing and that the left hemisphere is engaged in partial or analytical information processing. It is interesting that the processing inside the brain varies between the color-enhanced images and emotion-weighted images in spite of the fact that the scenes are the same. This may be because, compared to the color-enhanced images, the emotion-weighted images are processed to enhance specific aspects of the image rather than the overall image. Additionally, when test subjects viewed emotion-weighted images, significant activity was observed in the left and right frontal poles. The right frontal pole is related to the intensity of emotion in the episodic memory1), and the involvement of the left frontal pole is recognized in many autobiographic memory studies2). It can therefore be presumed that emotion-weighted images more effectively induce autobiographic memories, as well as more intense emotions and sentiments than color-enhanced images, regardless of the subject group.
The orbitofrontal cortex plays a primary role in the reward system of the brain and is engaged in the reward evaluation for actions and judgments. More recently, its relationship with the emotions induced by seeing paintings3) and maternal love4) has also been elucidated. In the current study, the group that favored emotion-weighted images showed significant activity in the left/right orbitofrontal cortices in response to emotion-weighted images and those in the right orbitofrontal cortex in response to color-enhanced images. Although much remains unknown regarding the difference between the left and right orbitofrontal cortices, a study by Noriuchi, Kikuchi et al. (2008) clarified that the left orbitofrontal cortex has a positive correlation with happiness and joyfulness and that the right orbitofrontal cortex has a positive correlation with anxiousness. This knowledge suggests that the left orbitofrontal cortex is more involved in processing related to rewards connected to more positive emotions than the right orbitofrontal cortex. It can therefore be presumed that stronger positive emotions are produced in the group that favored emotion-weighted images in response to emotion-weighted images. On the other hand, the group that favored color-enhanced images did not show significant brain activity in response to color-enhanced images, while it showed significant brain activity in the right orbitofrontal cortex in response to emotion-weighted images. This fact leads to a presumption that the group that favored color-enhanced images is relatively fond of color-enhanced images but does not regard them as the targets of rewards or expectations for rewards strong enough to produce activity in the orbitofrontal cortices. For the activity in the right orbitofrontal cortex of the color-enhanced image favoring group in response to emotion-weighted images, considering the current research result that the group that favored emotion-weighted images showed the same activity in response to both types of images and the fact that the right orbitofrontal cortex is correlated with the anxiousness (Noriuchi, Kikuchi et al. 2008), the activity in question may not simply be a positive reward but may reflect a certain significant for the subjects in this group. If this is right, it may suggest that the emotion-weighted images can induce a certain meaning relatively easily, regardless of the images preferred by each subject.
Regarding activity in the frontal pole, the group that favored emotion-weighted images showed significant activity in the left/right frontal poles in response to emotion-weighted images, as well as in the right frontal pole in response to the color-enhanced images. On the other hand, the group that favored color-enhanced images did not show significant activity in response to emotion-weighted images but showed significant activity in the left frontal pole in response to the color-enhanced images. The fact that the subjects in both groups showed activity in the left frontal pole in response to the favored images suggests that people tend to favor the type of images that induce their own autobiographical memory. Furthermore, the fact that the group that favored emotion-weighted images showed activity in the right frontal pole in response to both types of images leads to a presumption that the subjects of this group felt intense emotions with both types of images. On the other hand, the group that favored color-enhanced images may not have felt the same degree of emotion compared to the group that favored emotion-weighted images.
The results of the brain functional image analysis above suggest that emotion-weighted images stimulate the autobiographical memory more effectively than color-enhanced images and also tend to induce stronger positive emotions.
1)Daselaar, Rice, Greenberg, Cabeza, LaBar, Rubin: The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving. Cerebral Cortex 18; 217-229, 2008
2)Svoboda, McKinnon, Levine: The functional neuroanatomy of autobiographical memory: A meta-analysis. Neuropsychologia 44; 2189-2208, 2006
3)Aoyama et al., in preparartion
4)Noriuchi, Kikuchi et al: The functional neuroanatomy of maternal love: mother’s response to infant’s attachment behaviors. Biological Psychiatry 63; 415-23, 2008
The subjects showed high right-hemisphere activity with color-enhanced images and high left-hemisphere activity with emotion-weighted images, regardless of their grouping. Much previous neuropsychological and psychological research has demonstrated that the right hemisphere is engaged in overall or intuitive information processing and that the left hemisphere is engaged in partial or analytical information processing. It is interesting that the processing inside the brain varies between the color-enhanced images and emotion-weighted images in spite of the fact that the scenes are the same. This may be because, compared to the color-enhanced images, the emotion-weighted images are processed to enhance specific aspects of the image rather than the overall image. Additionally, when test subjects viewed emotion-weighted images, significant activity was observed in the left and right frontal poles. The right frontal pole is related to the intensity of emotion in the episodic memory1), and the involvement of the left frontal pole is recognized in many autobiographic memory studies2). It can therefore be presumed that emotion-weighted images more effectively induce autobiographic memories, as well as more intense emotions and sentiments than color-enhanced images, regardless of the subject group.
The orbitofrontal cortex plays a primary role in the reward system of the brain and is engaged in the reward evaluation for actions and judgments. More recently, its relationship with the emotions induced by seeing paintings3) and maternal love4) has also been elucidated. In the current study, the group that favored emotion-weighted images showed significant activity in the left/right orbitofrontal cortices in response to emotion-weighted images and those in the right orbitofrontal cortex in response to color-enhanced images. Although much remains unknown regarding the difference between the left and right orbitofrontal cortices, a study by Noriuchi, Kikuchi et al. (2008) clarified that the left orbitofrontal cortex has a positive correlation with happiness and joyfulness and that the right orbitofrontal cortex has a positive correlation with anxiousness. This knowledge suggests that the left orbitofrontal cortex is more involved in processing related to rewards connected to more positive emotions than the right orbitofrontal cortex. It can therefore be presumed that stronger positive emotions are produced in the group that favored emotion-weighted images in response to emotion-weighted images. On the other hand, the group that favored color-enhanced images did not show significant brain activity in response to color-enhanced images, while it showed significant brain activity in the right orbitofrontal cortex in response to emotion-weighted images. This fact leads to a presumption that the group that favored color-enhanced images is relatively fond of color-enhanced images but does not regard them as the targets of rewards or expectations for rewards strong enough to produce activity in the orbitofrontal cortices. For the activity in the right orbitofrontal cortex of the color-enhanced image favoring group in response to emotion-weighted images, considering the current research result that the group that favored emotion-weighted images showed the same activity in response to both types of images and the fact that the right orbitofrontal cortex is correlated with the anxiousness (Noriuchi, Kikuchi et al. 2008), the activity in question may not simply be a positive reward but may reflect a certain significant for the subjects in this group. If this is right, it may suggest that the emotion-weighted images can induce a certain meaning relatively easily, regardless of the images preferred by each subject.
Regarding activity in the frontal pole, the group that favored emotion-weighted images showed significant activity in the left/right frontal poles in response to emotion-weighted images, as well as in the right frontal pole in response to the color-enhanced images. On the other hand, the group that favored color-enhanced images did not show significant activity in response to emotion-weighted images but showed significant activity in the left frontal pole in response to the color-enhanced images. The fact that the subjects in both groups showed activity in the left frontal pole in response to the favored images suggests that people tend to favor the type of images that induce their own autobiographical memory. Furthermore, the fact that the group that favored emotion-weighted images showed activity in the right frontal pole in response to both types of images leads to a presumption that the subjects of this group felt intense emotions with both types of images. On the other hand, the group that favored color-enhanced images may not have felt the same degree of emotion compared to the group that favored emotion-weighted images.
The results of the brain functional image analysis above suggest that emotion-weighted images stimulate the autobiographical memory more effectively than color-enhanced images and also tend to induce stronger positive emotions.
1)Daselaar, Rice, Greenberg, Cabeza, LaBar, Rubin: The spatiotemporal dynamics of autobiographical memory: neural correlates of recall, emotional intensity, and reliving. Cerebral Cortex 18; 217-229, 2008
2)Svoboda, McKinnon, Levine: The functional neuroanatomy of autobiographical memory: A meta-analysis. Neuropsychologia 44; 2189-2208, 2006
3)Aoyama et al., in preparartion
4)Noriuchi, Kikuchi et al: The functional neuroanatomy of maternal love: mother’s response to infant’s attachment behaviors. Biological Psychiatry 63; 415-23, 2008
