Transcript of Evaluating the usability of interactive virtual tours by comparing XNA and Flash.
Comparing the Usability of 3D Model Based Virtual Tours to Panoramic Image Based Virtual Tours. The goal of this project is to test whether a 3D virtual tour that would offer total freedom of of movement would have any advantage over image based tours in terms of visual appeal, informational quality as well as ease of navigation. In order for the two types of VR to be accurately compared it was necessary to develop two separate, stand alone prototype tours of the same environment, one based on panoramic images and the other on 3D models. The environment chosen was the corridor system at Carriglea (this building). Why? Once development was finished each of the tours were subjected to user testing. Ten participants took part and were asked to complete various tasks under observation and to provide feedback in a survey. A virtual tour is a computer simulation of a real environment, the purpouse of which is to allow a person to get a sense of the space without the need to physically travel there. Today panoramic image based virtual tours are used widley and are a very good way to provide an overview of an environment to a user. However in my opinion the point to point navigation style is quite limiting. How? Finally the gathered data was analysed and conclusions were formed on the advantages and disadvantages in terms of usability aspects and the practicalities of the two development processes. Research Implementation Testing Conclusions Heinsight The research conducted can be broken up into four main stages. Background research into the history of VR and into the various types of tours that exist today. Development techniques for a panoramic image based virtual tour including panoramic photography methods, photo stitching techniques and various development environments. Development techniques for 3D model based virtual tours including 3D modelling technuiqes and programming environments. Analysis of previous usability studies of virtual tours (one 2D one 3D). Panoramic image based tour: The panoramic tour was required to have similar functionality to commonly existing tours. This required the user to be able to navigatate from point to point using both navigational hotspots on a floor and also by using hotspots on the images themselves. First step in development was to capture all of the images required for the panoramas and stitch them together. There are seven points of observation in the image based tour, requireing seven panoramas. Each panorama consists of between ten and fifteen images. Several image stitching software packages were trialled. These packages are capable of automatically stitching images. However, the trial versions of these packages could not provide satisfactory results for various reasons so in the end photoshop was used to manually stitch the images. Once the panoramas were completed the tour had to be put together. Tourweaver 6 was chosen for this, it is a purpouse built commercial software package that allows for the quick building of virtual tours. The interface is relitively complex, however it is very similar to Adobe Flash so the learning curve is was not very steep. 3D model based tour: The implementation of the 3D model based tour first required that 3D a model of the building be created and then for this model to be imported into a programming environment in order to code user interactions. Two 3D modelling technuiqes were concidered; using photogrammertry software or the more traditional technuiqe of drawing models in a modelling program such as 3D Max. Photogrammertry software is a relitively new technology that allows for the creation of photorealistic 3D models by drawing on top of panoramic images. This software proved to have a very steep learning curve however and the decision was made to abandon it in favour of 3D Max. The programming environment chosen for the 3D tour was Microsoft's XNA game development environment. The main factor in this decision was the developers previous experience using this framework coupled with the vast amount of support for coding available online. There were two phases involved in the testing of the applications; functionality testing and usability testing. The main reason for conducting functionality testing was to insure there would be no hitches in usability testing later. It was important that both applications were stable so that there would be no crashes or bugs during the usability test sessions. Any crashes could have influenced participants opinions of the environments in a way that was not relevent. Functionality testing: Usability testing: Usability testing was broken up into four main sections; data gathering, preffered user input methods, questionairre feedback and task analysis. It was very important to cover a broad cross section of users to procure a nonbiased result. A total of 10 participants, 4 female and 6 male were subjected to user testing on both applications. The users ranged in age from 17 years to 65 years old and all possessed varying degrees of computer literacy and video game experience. Data Gathering: Preffered User Input Methods: Questionnaire Feedback: Task Analysis: Each user was asked first asked to roam freely through each virtual tour in their own time in order to familiarise themselves with the applications and their controls without any instruction from the developer. Their actions were recorded so that each user's preffered input methods could be recorded and they could develop their own opinions on each tour. After free roaming the users were assigned the task of “resetting their password”. There is a password reset machine located at the back of the Carriglea building, the user’s task was to navigate to this machine in both virtual tours. The time taken was recorded for each user in each application. The goal of this was to provide an indication as to which tour is easiest and quickest to navigate through. The time taken to find the machine was recorded for each user in each application. When the participants were finished using the applications they were required to fill out a questionnaire. The questionnaire was designed to record the participant’s basic information (age, gender etc) and their level of computer literacy.Full transcript
The questionnaire also asked the participant’s general questions on the tours including which they preferred in terms of visual appeal, navigation, informative qualities (which provides a better representation of the building), and their ideas for improvements. An average time of 5.44 seconds was saved using the 3D tour compared with the image based tour. The development process of the panoramic image based tour proved to be the more straightforward one. Since the concept was new to the developer there was a large amount of research required before implementation as well as time consuming issues relating to panoramic photography and photo stitching. However, the developer did not have to write any code. The XNA design process requires substantial knowledge of both programming and 3D modelling and is generally much more difficult and time consuming to implement. Practicalities of the two Development Processes: Usability Testing Results: According to user feedback and test data the 3D tour is quicker, easier and more pleasant to navigate through compared to the 2D tour.
User feedback indicates that the 2D tour is more visually appealing and provides a better representation of the space than the 3D tour however the margin of difference is relatively small. Unfortunately this result may have been warped by the technical problem with lighting in XNA reducing the visual appeal of the 3D models. This project succeeded in outlining the specific advantages and disadvantages of panoramic image based tours and 3D model based tours in terms of their usability and development processes. Project Outcome: It has proven that ease of navigation in a 3D tour is superior compared with an image based tour. Users prefferred the 2D tour visially, however i feel that an unfortunate technical problem with lighting in XNA was a large contributary factor in this. Lighting effects in XNA caused some surfaces of the 3D model to appear dull or discoloured. I feel if this problem was addressed the level of visual appeal would be swayed in favour of the 3D tour. Although the 3D tour was proven to be superior, from a practical point of view I would still reccomend a 2D tour as the most practical solution because of the more difficult and time consuming nature of the 3D development process and the level of compatibility of image based tours offered with web browsers and various operating systems. If I were to undertake this project again I would most likley choose Vizard over XNA as the 3D application development environment. Using Vizard would have been far less time consuming and complex because i find the code to me much more simple and the 3D models are rendered nicely. In my opinion the default lighting in Vizard is far nicer than in XNA. This may have eradicated the advantage of visual appeal in the image based tour.
Vizard also provides a simple way to implement a map and current position indicator. The presence of a map position indicator in the image based tour was also a big advantage for the image based tour in terms of navigation.
Vizard also provides an extremly simple way to detect collisions, this is a complex process in XNA
Unfortunatly I was unaware of this until I worked on a Vizard project after the main project deadline. I also would have conducted a paired t test on the task analysis data. This would have provided solid, mathimatical proof of the result rather than just the average difference approach taken. As above I was not fully aware of this method until after the project deadline.