Testing a computational model of visual complexity in background scenes


5 pages

Please download to get full document.

View again

of 5
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Testing a computational model of visual complexity in background scenes
  Testing a computational model of visual complexity in background scenes. Leena N. Patel, Patrik O’Brian Holt.  Image Systems Engineering Laboratory, Department of Computing and Electrical Engineering. Heriot-Watt University, Edinburgh, EH14 4AS. Abstract This paper reports the results of experiments conducted to determine whether a mathematical/computational modelreflects the perception of complexity of background image scenes. Background scenes, such as landscapes, arecomposed of non-discrete image data. They contain repetitive patterns and colour schemes and therefore the imageshave to be described using pattern analysis techniques. A brief outline is given regarding the difference incomposition of the foreground and background in images. Complexity measures and theories are then explored and results of a pilot test and experiment reported which correlate human perceptual data with a pattern measure whichwas devised by Klinger & Salingaros (1999). This measure had previously only been tested on binary data and not on more realistic image data. The results show a very strong, positive correlation (r=0.899, p<0.01) which seems tosuggest that the human perception of complexity of non-determinate image data is modelled well by the patternmeasure. The paper concludes by outlining potential implications of the work and uses of the model. Keywords:   visual complexity, cognitive modelling, background images, complexity measure, perception. 1.   Introduction The term “complex” is used more and more frequently in science, often it is used akin to“complication”, and refers to any problem to which standard, well-established methods of mathematical analysis cannot be immediately applied (Anderson, 1994). The problem of characterising complexity in a quantitative way has thus become a rapidly developing area. It isonly in the past decade that it has been addressed in terms of vision (e.g. Oomes, 1995; Lindgren,Moore, & Nordahl, 1997). This project set out to determine whether visual complexity could bemodelled mathematically or computationally. A viable model is presented in this paper togetherwith support from human assessments which has thus far not been attained.Perceived complexity is related to the environment and our understanding of it. Our visualdomain is the environment, and artefacts within it that are difficult to comprehend are deemedcomplex. Essentially anything that is in our field of vision is under scrutiny for its perceivedcomplexity value. This is not to say that this is a judgement we make consciously. We do notgenerally go through life determining whether what we see is complex or not. But when we do,how do we know it is complex? What is our reference point? The work reported aims toinvestigate human ability to judge complexity and model this computationally. 2.   Background Complexity A visual image typically consists of a background and foreground (objects dispersed onthe background), referred to as ‘stuff’ vs. ‘things’ by Forsyth et. al (1996), such as a clock on awall or a ship on the sea. These two aspects of the scene differ in nature. One is discrete(deterministic) and the other non-discrete (non-deterministic). The sky, a carpet or a wall areexamples of non-discrete elements. They cannot be described in finite terms in the way an objectcan. A chair can be called ‘an object’, but sand is described as a collective for many grains of sand or as texture. Background is usually characterised by repetitious patterns and colourschemes, hence, in the analysis of backgrounds, patterns are commonly used. 3.   Research in Pattern Complexity  Attempts have been made at producing measures of pattern complexity. An algorithm,written by Paul Radja (Weidemann et. al, 1994) included measures for colour, edges, fractaldimension, standard deviation, entropy, Huffman-encoding and run length encoding. The overallcomputer measure did not however correlate with measures of human perceived complexity.  They found that their computational measure correlated with their human measure of perceivedbeauty but expressed concerns as to whether this was a valid predictor of perceived complexity.Klinger and Salingaros (1999) derived a quantitive pattern measure which was based onsize, density, line curvature, colour, symmetry, similarity of shapes and correctness of forms.Their algorithm calculated measures of Harmony, Temperature, Life and Complexity. ‘Harmony’measures the correlations of subunits via symmetries, ‘Temperature’ describes symbol variationand ‘Life’ corresponds directly to the degree that human beings intuitively feel a design to be“alive”. Finally, ‘Complexity’ is based on the former three components and denotes the overallcomplexity rating for each analysed pattern. The ‘Complexity measure’ was a quantitiveassessment of pattern and it was this final measure that was used in this research.The Klinger-Salingaros (K-S) algorithm has not been tested conclusively on actual imagedata but merely on binary arrays. Thus, an experiment was conducted to test the algorithm onimages and also to see how well the complexity values calculated by the algorithm correlatedwith human ratings of the same images. 4.   Pilot Work Before the Klinger-Salingaros (K-S) algorithm could be used, pilot tests were performedto determine the reliability of the algorithm under different test conditions. These were size,image format, the application used to greyscale the image and the application used to size theimage. It was hypothesised that the complexity value would be equal for all conditions applied tothe image.The algorithm could only accept square, greyscale images as input, and so, it wasnecessary to resize and greyscale the image before applying the algorithm. The image wasresaved in jpeg and raw format. The three image formats were greyscaled with Adobe Photoshopv4.0 or with the VisImage (v2) application. Copies of these images were then individually resizedin both applications (Photoshop and VisImage). Nine image sizes were used (10x10, 30x30,50x50, 70x70, 100x100, 200x200, 300x300, 400x400, 500x500 pixels). Thus, in total 108 images(3 (image formats) x 2 (greyscaling packages) x 2 (sizing packages) x 9 (sizes of image) werecompared according to their resulting complexity values. Each time the four conditions (imageformat, greyscaling package, sizing package and size) were applied, a fresh copy of the srcinalimage was used. This controlled for the countereffect of prior changes since the conditions werealways applied on an srcinal image and not one that had already been changed.The image used for the pilot experiment is shown as figure 1. It was srcinally a gif colour image, sized 192 (width) x 128 pixels (height). Figure 1: The image used in the pilot test (scaled down). A complexity value was obtained for each combination of conditions using theimplemented K-S algorithm which were then compared.   Results And Discussion Statistical tests were performed to determine whether there were any differences in thecomplexity values of the image with differing conditions. Firstly, a Pearson r correlation test wasapplied to determine whether the complexity value corresponded to the image size. With r=0.878,p<0.01, it was shown that there was a very strong positive correlation. The correlation graph isshown in figure 2. Image Size (pixels)6005004003002001000    K   l   i  n  g  e  r  -   S  a   l   i  n  g  a  r  o  s   C  o  m  p   l  e  x   i   t  y   V  a   l  u  e  s 40000003000000200000010000000-1000000 Figure 2: Correlation of K-S complexity values with Image Size. Complexity values given by the K-S algorithm increased with the size of the image.Therefore the image size seemed to influence the complexity rating. Now, with size accountedfor, the other three factors were tested; all images of size 10x10 pixels were compared with eachother for the effects of image format, greyscaling application and sizing application. All 30x30sized images were compared with each other and so on. As seen in figure 3, the complexity valuediffered between the images when grouped by their size. Complexity Values by Size of Image with series of Image Format, Sized In & GreyScaled In.LOWEST LINEMIDDLE LINEHIGHEST LINE 01000000200000030000004000000    1   0  x   1   0   3   0  x   3   0   5   0  x   5   0   7   0  x   7   0   1   0   0  x   1   0   0   2   0   0  x   2   0   0   3   0   0  x   3   0   0   4   0   0  x   4   0   0   5   0   0  x   5   0   0 Size of Image (pixels)    C  o  m  p   l  e  x   i   t  y   R  a   t   i  n  g gif photoshop visimagegif visimage visimage jpeg visimage photoshop jpeg visimage visimageraw visimage photoshopraw visimage visimagegif photoshop photoshopgif visimage photoshop jpeg photoshop photoshopraw photoshop visimage jpeg photoshop visimageraw photoshop photoshop Figure 3: Complexity values by Size of Image. This difference was much more substantial for the larger images (e.g. 500x500) than thesmaller image sizes (e.g. 50x50). Table 1 presents the image factors for each trend line. Table 1: Key for the Graph of Figure 3. Image FormatSized In (Application)Greyscaled in (Application)LOWEST LINE (on graph of figure 3) JpegPhotoshopPhotoshopRawPhotoshopVisImageJpegPhotoshopVisImageRawPhotoshopPhotoshop MIDDLE LINE (on graph of figure 3) RawVisImageVisImageJpegVisImageVisImageGifPhotoshopPhotoshopGifVisImagePhotoshopRawVisImagePhotoshopJpegVisImagePhotoshop HIGHEST LINE (on graph of figure 3) GifPhotoshopVisImageGifVisImageVisImage  For instance, on the lowest trend line the image factors are jpeg or raw image formats,sized in Photoshop and greyscaled in Photoshop or VisImage. From this table it can be seen thatgif images, greyscaled with VisImage had the highest complexity ratings. Gif images greyscaledwith Photoshop fell on the middle line. The rest of the factors appeared in all three trends. Due tothe differing complexity values based on the different image factors (format, greyscaling packageand sizing package) it can be concluded that the image properties influence the complexity rating.This, together with the size influence, means that in order to test for true complexity differencesamongst images, they must be equal in size and must be treated in exactly the same way.Another point to note is that squaring the image can produce distortions to the image. Forinstance, upon squaring a portrait image the pixels would be expanded more by width than byheight whereas for a landscape image the pixels would be elongated more by height than width.Therefore the initial images would ideally also need to be square images.The results from the pilot work were crucial to the main experiment as it enabled theidentification and thus elimination of extraneous factors. 5.   Experiment The experiment was conducted to determine whether the complexity values obtained bythe K-S algorithm were related to the human assessment of complexity of background sceneswith extraneous variables (e.g. image format, greyscale package) controlled for.From the pilot work described in the previous section it was decided to use consistently-sized and -formatted images. Since the lowest trend line factors seemed to be the most stable, aset of conditions was applied from these. Additionally, the images were square to begin with,eliminating effects of distortion to the srcinal images. All twenty-two images were jpegbackground scenes which depicted natural backgrounds such as landscapes, seascapes, rock formations etc. They were consistent in size (300x300 pixels) and grey-scaled using the VisImagepackage. A complexity value was obtained for each image using the K-S algorithm.The same images were also presented to forty human participants who ranked them interms of perceived complexity.Each image was presented centrally on an A4 sheet of paper with a plain whitebackground in portrait orientation. It was important to maintain uniform properties for all theimages presented in order to measure how well people interpreted the various visual properties of the image rather than how well people interpreted other properties such as orientation.The participants were asked to order the background images in terms of complexity. Theywere given all the images in a randomised order to enable them to compare the differentbackground images before deciding how to rank order them. They were also provided with a desk on which they could spread the images out so that they could compare them more easily.Participants were encouraged to employ their own definition of complexity since the aimof the experiment was to determine how humans rated visual complexity and this perception is ahuman concept which can vary from one individual to another. The null hypothesis was that therewould be no correlation between the two types of values (human rankings and computeralgorithm values).  Results and Discussion The statistical procedure, Pearson's r test was applied to determine whether the humancomplexity ratings correlated with the complexity values given by the K-S algorithm. Withr=0.899, p<0.01, a very strong, positive correlation was found. The correlation graph is shown infigure 4.  Mean Rankings given by Participants 1816141210864    K   l   i  n  g  e  r  -   S  a   l   i  n  g  a  r  o  s   C  o  m  p   l  e  x   i   t  y   V  a   l  u  e  s 40000003000000200000010000000 Figure 4: Correlation of K-S Complexity values with participant's mean rankings. From the graph it can be seen that as the complexity value of the computer algorithmincreased so did the complexity ranking from human participants for the background imagespresented. Therefore, humans ranked the images in a very concurrent fashion to the complexityvalues produced by the K-S algorithm. This pattern measure seems to model background imagecomplexity and could be used to assess background images in terms of visual complexity. 6.   Conclusions The model was tested on more realistic image data than patterns of binary numbers andthus showed that it could be used with more normal data. Also, the results validated the model interms of its ability to assess perceived complexity within background scenes from the highcorrespondence of the algorithm based complexity values and the human perceptual values.Such a measure could be useful for sea-bed mapping (e.g. Gardner et. al, 1996), landscapeidentification (Brabyn, 1996) and graphical database indexing (Del Bimbo, 1999). For example landscapes and sea-beds could be identified by inherent complexity properties, and databases of scenes could be accessed with the complexity score used as an index into the system.Research into complexity is important in the analysis of systems and artefacts weencounter in real life. If we can measure the level of complexity in a scene we may be able tocategorise or develop systems in terms of their inherent complexity.The K-S measure has been tested and shown to work for the perceived complexity of background images. This measure could be utilised in domains of human-vision and machinevision. References Anderson, P. W., (1994). Physics: The Opening of Complexity. NAS Proceedings of the Colloquium on Physics.Irvine: California, June 27-28.Brabyn, L. K., (1996). Landscape classification using GIS and national digital databases. Landscape Research,V21(3), pp277-300.Del Bimbo, A., (1999). Visual Information Retrieval. Morgan Kaufmann Publishers, San Francisco, CA.Forsyth, D. A., Malik, J., Fleck, M. M, Leung, T., Belongie, S., Bregler, C., Carson, C. & Greenspan, H., (1996).Finding Pictures of Objects in Large Collections of Images. Proceedings, International Workshop on ObjectRecognition, Cambridge, April.Gardner, J. V., Field, M. E., and Twichell, D. C. (editors), (1996). Geology of the U.S. Seafloor: The view fromGLORIA, Cambridge Univ. Press.Klinger, A., & Salingaros, N. A., (1999) A Pattern Measure. To appear in Environment and Planning B 27 (2000).Lindgren, K., Moore, C. & Nordahl, M., (1997). Complexity of Two-Dimensional Patterns. Tech. Rep. 97-03-023,Santa Fe Institute, Unit. States.Oomes, S., (1995). The Visual Complexity of Simple Shapes. Colloquium Cognition and Information (NICI).Nijmegen, The Netherlands. 11 Oct.Weidemann, E., Orland, B., Larsen, L. & Radja, P., (1994). The effects of visual variety on perceived humanpreference. Society and Resource Management. Fort Collins, CO.
Related Documents
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!