Modality based perception for selective rendering
Harvey, Carlo (2011) Modality based perception for selective rendering. PhD thesis, University of Warwick.Full text not available from this repository.
Official URL: http://webcat.warwick.ac.uk/record=b2585525~S1
A major challenge in generating high-fidelity virtual environments for use in
Virtual Reality (VR) is to be able to provide interactive rates of realism. The
high-fidelity simulation of light and sound wave propagation is still unachievable
in real-time. Physically accurate simulation is very computationally demanding.
Only recently has visual perception been used in high-fidelity rendering to improve performance by a series of novel exploitations; to render parts of the scene
that are not currently being attended by the viewer at a much lower quality with-out the difference being perceived. This thesis investigates the effect spatialised
directional sounds, both discrete and converged and smells have on the visual
attention of the user towards rendered scene images. These perceptual artefacts
are utilised in selective rendering pipelines via the use of multi-modal maps.
This work verifies the worth of investigating subliminal saccade shifts (fast
movements of the eyes) from directional audio impulses via a pilot study to eye
track participant's free viewing a scene with and without an audio impulse and
with and without a congruency for that impulse. This experiment showed that
even without an acoustic identifier in the scene, directional sound provides an
impulse to guide subliminal saccade shifts. A novel technique for generating
interactive discrete acoustic samples from arbitrary geometry is also presented.
This work is extrapolated by investigating whether temporal auditory sound wave
saliencies can be used as a feature vector in the image rendering process. The
method works by producing image maps of the sound wave
flux and attenuating
this map via these auditory saliency feature vectors. Whilst selectively rendering,
the method encodes spatial auditory distracters into the standard visual saliency
Furthermore, this work investigates the effect various smells have on the visual
attention of a user when free viewing a set of images whilst being eye tracked.
This thesis explores these saccade shifts to a congruent smell object. By analysing
the gaze points, the time spent attending a particular area of a scene is considered. The work presents a technique derived from measured data to modulate
traditional saliency maps of image features to account for the observed results for
smell congruences and shows that smell provides an impulse on visual attention.
Finally, the observed data is used in applying modulated image saliency maps
to address the additional effects cross-modal stimuli has on human perception
when applied to a selective renderer. These multi-modal maps, derived from
measured data for smells, and from sound spatialisation techniques attempt to
exploit the extra stimuli presented in multi-modal VR environments and help
to re-quantify the saliency map to account for observed cross-modal perceptual
features of the human visual system. The multi-modal maps are tested through
rigorous psychophysical experiments to examine their applicability to selective
rendering algorithms, with a series of fixed cost rendering functions, and are
found to perform better than image saliency maps that are naively applied to
multi-modal virtual environments.
|Item Type:||Thesis or Dissertation (PhD)|
|Subjects:||Q Science > QA Mathematics > QA76 Electronic computers. Computer science. Computer software
T Technology > TA Engineering (General). Civil engineering (General)
|Library of Congress Subject Headings (LCSH):||Virtual reality|
|Official Date:||November 2011|
|Institution:||University of Warwick|
|Theses Department:||School of Engineering|
|Extent:||xiv, 220 leaves : illustrations, charts|
Actions (login required)