Evaluating interactive digital television applications through usability heuristics

Interactive Digital Television (iDT) is considered as the convergence of television and computer technologies. Usability evaluation for applications based on emerging information technology brings new challenges. The main iDT feature is that the user may interact with the application; therefore usability should be a main concern when designing iDT applications. Current research usually focuses on iDT applications from a technical point of view, rather than a user–centered approach. There is a need for new usability evaluation methods or at least for the use of traditional evaluations in novel ways. A set of specific usability heuristics was defined and validated, in order to help the usability evaluations of iDT applications. A usability checklist to be used when applying iDT heuristics is also proposed.


INTRODUCTION
Interactive Digital Television (iDT) is considered as the link of television and computer technologies, which gather three typical features: interactivity, customization and digitization [1].A TV program refers to "any type of content, should it be an advertisement, a movie, a quiz show, teletext page, email message or even any kind of sound" [2].In iDT the program content, form or even the presentation order, can be changed by the user.In addition, the user has access to interactive applications that can be independent or closely related to the subject or the progress of the content.
The iDT exceeds the analog TV in several aspects: capacity, better use of the spectrum, greater immunity to noise and interference, better sound and picture quality, potential for transmission of data simultaneously, saving power transmission.However, the main iDT advantage is that the user may interact with the application [2].Interactivity allows the user to be an active part of the TV schedule, providing the ability to access or extend the information presented, to participate in forums and to control the sequence of information presented [3].
The usability evaluation of a software system is one of the most important stages in the user-centered design approach.It allows obtaining the usability characteristics of a software system and the extent to which de usability attributes, usability paradigms and usability principles are being implemented [4].
Usability evaluation for applications based on emerging information technology brings new challenges; however some questions arise, is it the classical concept of usability still valid?Which are the dimensions of the (new) usability?How can it be measured?How should we develop for (better) usability?There is a need for new evaluation methods or at least for the use of traditional evaluations in novel ways [5].
The ISO/IEC 9241 standard defines the usability as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use" [6].Take into account this definition, in the context of iDT applications, usability can be defined as: the extent to which specific users can access an application of interest and interact with the options that it offers for their entertainment, information or other, with minimal effort, quick and pleasant, in a given context of use.
Usability evaluation methods are commonly divided into inspection and testing methods.Inspection methods find usability problems based on the expertise of usability professionals.Testing methods find usability problems through the observation of the users while they use (and comment on) a system interface.Usability evaluation is needed particularly if the design concept is new.Users look for more than just a usable product; they look for a pleasing and engaging experience [7].Therefore, usability should be a main concern when designing interactive iDT applications.Current research usually focuses on iDT applications from a technical point of view, rather than a user-centered approach.There is a necessity to establish methodologies that could lead to applications with a high level of usability.Such methodologies have to include accurate usability evaluations.
Heuristic evaluation is a widely used inspection method [8][9].A group of evaluators (usually from three to five) inspect the interface design based on a set of usability heuristics.In order to ensure independent and unbiased evaluations, the inspection is performed individually.After all individual evaluations have been completed, the evaluators are allowed to communicate and have their findings aggregated in a single list of usability problems.Later on, each evaluator assigns scores to each problem's severity and frequency (on a 0 to 4 scale, from minor/less frequent to major/more recurrent).Severity and frequency are summed in order to get problem's criticality.Problems are ranked based on their average severity, frequency and criticality.The usability evaluation report includes usability problems, solution proposals, as well as positive findings.Heuristic evaluation is easy to perform, cheap and able to find many usability problems (both major and minor problems).However, it may miss domain specific problems.That is why the use of appropriate heuristics is highly significant.
The present work focuses on usability evaluation of iDT applications using heuristic evaluations.A set of 14 specific usability heuristics has been proposed and validated.A usability checklist to be used when applying the iDT heuristics is also proposed.First section highlights the main characteristics of iDT applications.Second section presents the methodology that has been used in heuristics development.The iDT usability heuristics proposal is presented in third section.Fourth section presents the results obtained in the heuristics evaluations, later, the fifth section presents the usability checklist for iDT applications.Sixth section presents preliminary conclusions and future works.

USABILITY IN IDT APPLICATIONS
iDT includes relevant aspects such as: ease-of-use, entertainment, information, among others.Easeof-use is clearly a priority for design interactive applications.Many users are accustomed to using the TV in a passive way (channel changes only), so, the new paradigm consist in designing interactive applications more intuitive and clear as possible trying to get users to become active part of the TV schedule.In that sense, the user should be informed clearly of what can do in the application and how this can be done.
Users are accustomed to television entertains them; with the innovations of technology in the iDT applications increases the possibility of enhanced entertainment, because the users can interact with the programs that they are watching.In that sense, an interactive application too simple can be considered for the users as boring and uninteresting, for this reason it is important to focus on a pleasing appearance of the application and make it easy to learn and understand.Thus, other users may be attracted to use the different options and features of the application.
Usability will allow to users that use an iDT application to stay in it, enjoy the interaction, and have the desire to reuse in the future.To achieve this, iDT applications should be designed and implemented, so that all users can use and understand it without problems.

iDT characteristics
Nowadays the concept of television does not refer to a specific device, but rather a specific kind of content available almost everywhere, from the traditional TV set, to the computer, passing through the mobile phone or the screens, in taxis and throughout the city, freeing television from the TV set and bringing it out of the home.Additionally, current trends combine iDT with the web; users may navigate on internet from their TV sets, download and use applications, download content or customize the TV schedule of an enhanced form.It is expected that users can access to television and internet in a simple manner, from different devices.iDT applications have a set of basic features that should be considered when evaluating their usability [10]

DEFINING USABILITY HEURISTICS FOR INTERACTIVE DIGITAL TELEVISION
Usability inspections, including heuristic evaluation, are well documented and many publications describing the usage of the methods.Literature usually focuses on describing the advantages and disadvantages of usability evaluation methods but not on how to develop new methods and/or usability heuristics [10].
In order to develop usability heuristics for iDT applications, a specific methodology has been applied [11].The methodology to establish new usability heuristics includes 6 stages: • STEP 1: An exploratory stage, to collect bibliography related to the main topics of the research: specific applications, their characteristics, general and/or related (if there are some) usability heuristics.• STEP 2: A descriptive stage, to highlight the most important characteristics of the previously collected information, in order to formalize the main concepts associated with the research.An early version of the iDT usability heuristics were proposed in 2009 [12].Later on, STEP 1 to STEP 4 of the methodology were performed, and a refined usability heuristic proposal was defined.
STEP 1 explored specific iDT applications and usability heuristics.
STEP 2 re-examined the meaning of the usability and its characteristics, in the context of iDT applications.
As no specific and/or related iDT usability heuristics were founded, Nielsen's 10 [9] well know and extensively used heuristics were used at STEP 3.
However, heuristic proposals for other fields, such as Social TV, Virtual Worlds and Grid Computing were also used.
The standard template used at STEP 4 was the following one: • ID, Name and Definition: Heuristic's identifier, name and definition.• Explanation: Heuristic's detailed explanation, including references to usability principles, typical usability problems, and related usability heuristics proposed by other authors.• Examples: Examples of heuristic's violation and compliance.
• Benefits: Expected usability benefits, when the heuristic is accomplished.• Problems: Anticipated problems of heuristic misunderstanding, when performing heuristic evaluations.
STEP 5 evaluated the set of iDT heuristics defined at STEP 4, against Nielsen's heuristics, in two cases studies.The applications were evaluated by two separate groups of evaluators, of similar experience, in equal conditions.One group used only the set of heuristics defined at STEP 4, while the other group used only Nielsen's heuristics.Usability problems founded by the two groups were then compared.Three categories of problems occurred: • (P1) Problems identified by both groups of evaluators.• (P2) Problems identified only by the group that used the set of heuristics defined at STEP 4. • (P3) Problems identified only by the group that used Nielsen's heuristics.
As in both case studies (P2) included the highest percentage of identified problems, we may say that iDT heuristics worked better than Nielsen's heuristics.Question arises with problems (P3).Why these problems were not identified when using iDT heuristics?There are two possible reasons: (1) iDT heuristics were not able to identify these problems, either because appropriate heuristics are missing, or because the heuristics are not properly specified.(2) Evaluators using iDT heuristics subjectively ignored the problems.
Hypotheses ( 1) and ( 2) were checked by analyzing problems (P2) and (P3) and by complementary user tests.Hypothesis (2) proved to be the correct one.At STEP 6 the set of heuristics for iDT applications was refined, based on the results obtain at STEP 5.

IDT USABILITY HEURISTICS
A set of 14 iDT usability heuristics was specified, validated and refined.The 14 heuristics were grouped in three categories: (1) Design and Aesthetics, (2) Flexibility and Navigation and (3) Errors and Help.A summary of the proposed heuristics is presented below, including heuristic's ID, name, definition and explanation.

Design and Aesthetics Heuristics
(H1) Match between the system and the real world: An iDT application should speak the user's language, with words, phrases and concepts familiar to the user.iDT applications should use specific conventions of the real world and should show the information in a natural order.The sequence of activities should follow user's mental processes.Metaphors should be easy to understand; there should be an intuitive mapping between controls and their functions.
(H2) Simplicity: An iDT application should not overload users with irrelevant and/or unnecessary information.Every extra unit of information competes with the relevant units of information and diminishes their relative visibility.iDT applications should show concise (but all necessary) information.
(H3) Consistency and standards: Design should be coherent and consistent throughout the iDT application; it should follow the norms or conventions for TV design in general, as well as for new specific elements of iDT.iDT applications should present similar elements in similar ways.Terminology, controls, graphics and menus should be consistent throughout the system; there should be a consistent look and feel for the system interface.

APPLYING IDT USABILITY HEURISTICS IN PRACTICE
Heuristic evaluation was executed taking into account the following steps [8]: STEP 1: Each evaluator works independently 1-2 hours inspecting the applications based on Nielsen's heuristics, recording the clearest way possible usability problem identified and grouped according to the usability principle that not correspond.
STEP 2: Once all reviewers have made the individual evaluation, they come together to create a single list of usability problems grouped according to the usability principle they found as a problem.
STEP 3: The list of usability problems obtained in the step 2, will be deliver to each evaluator to estimate the severity and frequency with which problems appear.The notes are grouped according to the severity and frequency scales presented in Table 2. Considering that the iDT users can access a wide range of applications, which can be implemented for different environments such as TDT, IPTV 4 , 4 IPTV (Internet Protocol Television): System where a digital television service is delivered to its customers using IP over a network infrastructure.
Mobile TV 5 , among others [15], in this paper applications used were those that are transmitted using industry-standard DVB technology, which was adopted in Colombia by the National Television Commission [16] in 2008, and also follows the specification MHP (Multimedia Home Platform) [17].The standard transmission has several modes of distribution: DVB-T, if the distribution is done through the air, DVB-C, if the distribution is done by Cable, however, the content and applications are not affected on the side the client for these modes of distribution.
The applications have a number of features that enable users to control and exchange information proactively through a bidirectional communication channel or back channel that let users interact with the object of transmission, whether video, images, audio or data [18].Through iDT receiver (Set Top Box -STB), device that allows to adapt the digital signal, an integration and execution of the applications on television content is performed, but also a backchannel connection to a set applications and remote services that can be accessed are provided, based on the information that user want at any given time, or to the dynamic interaction system [19].
The proposed iDT usability heuristics were checked against Nielsen's 10 heuristics, using the iDT applications: Electronic Program Guide, Blog and Chat, as cases studies.
• The Electronic Program Guide or EPG (for its acronym in English Electronic Program Guide) provides information on the programs being broadcasted on a channel program, programs to be broadcasted in the day, programs to be broadcasted in the next few days a week, describing the programs, start and end time, duration, among others.The EPG provides information about the favorite and most viewed programs or who have obtained more votes in the polls.Users can add or remove programs from the list of favorites.The EPG can be accessed by a user pressing the blue button in an interactive way. Figure 1 shows the main menu interface of the EPG.• The Blog or mini-blog is an application that allows users to post messages or news to be consulted or discussed by members of the community.Figure 2 shows a list of messages posted to the blog.
Figure 2. List of messages posted to the blog.
• Chat is an application that allows communication between users of the community.Figure 3 shows the conversation between users of the chat.

Results of evaluations
iDT applications: EPG, Blog y Chat, were examined by two groups of 5 evaluators.The 10 evaluators had similar (medium) experience on issues about usability (heuristic evaluations) and a medium/high experience on issues related to the management and development of iDT applications.The first group performed a heuristic evaluation of applications using only Nielsen's heuristic, while the second group performed a similar heuristic evaluation, but using only heuristics for iDT applications.Table 3 shows the number of usability problems identified by each group of evaluators in the heuristic evaluations.
Table 3. Usability problems identified in the applications.A total of 60 problems were identified by the 10 evaluators.More usability problems were captured using iDT heuristics, than using Nielsen's heuristics: • (P1) 21 problems were identified by both groups of evaluators.• (P2) 15 problems were identified only by the group that used iDT heuristics.• (P3) 3 problems were identified only by the group that used Nielsen's heuristics.
The 3 usability problems identified using the Nielsen's heuristics (P3) were associated to heuristics N2-Match between system and the real world (1 problem), N3-User control and freedom (1 problem), and N8-Aesthetic and minimalist design (1 problem).The set of iDT heuristics provides the tools that can potentially identify all these problems: H1-Match between the system and the real world, H8-Navigation and H11-User control and freedom, and H2-Simplicity, H5-Physical constraints and H6-Extraordinary users, respectively.Moreover, all (P3) usability problems were qualified with relatively low severity scores (severity value higher is 2,6 on a scale of 0 to 4, the other 2 problems have notes of 2 and 2,4), and are not in the ranking of criticality.According to the above, the evaluators that used iDT heuristics subjectively ignored the 3 (minor) usability problems identified only Nielsen's heuristics.
A usability test was designed and it was performed with 5 representative users.The test was focused on the 3 usability problems identified only by Nielsen's heuristics.All these problems were tested to be perceived as minor problems or (not at all problems) by users.
Analyzing the 15 problems identified only heuristics by iDT heuristics (P2), most of them were qualified as critics: 9 of 15 problems had a criticality of 6 or superior, on a scale from 0 to 8.Moreover, 11 of the 15 problems had a note of severity superior to 2.5.
All these results seem to prove that iDT heuristics worked better than Nielsen's heuristics.

APPLYING IDT USABILITY HEURISTICS IN PRACTICE
Based on the experiments we have made, the nature of the usability problems identified when applying iDT heuristics, and the problems that some evaluators had when applying such heuristics, a usability checklist was defined.It details iDT heuristics and help their use in heuristic evaluation practice.The checklist is presented below.
(H1) Match between the system and the real world • The purpose of the application is clear to users.
• The elements of the application (interface controls, instructions, etc.) use words, phrases and concepts familiar to user.• Information on the elements of the application is presented in a simple form, natural and logical order.• The application language is familiar to users.
• The activities sequence of the application follows the mental processes of users.
• The application uses metaphors and user interface controls that correspond with reality.• The metaphors are easy to understand for users.• Metaphors help users to better understand (the meanings of) the application.• Metaphors are used only for simple concepts and/or tasks.• Interface controls important are presented on the screen.• The relationship between the interface controls and the real controls is intuitive.• In the application each option or element is over-explained.• In the application the complex elements are explained.
(H2) Simplicity • The application interface is simple, not overloaded with information, options or other elements that distract the user.• The alternatives that the user needs to perform a task are visible.• The information presented in the application is simple, concise and clear.• The elements that make up the application (interface controls, instructions, etc.) are clear and intuitive.• The instructions are preferably in the bottom of the screen.• There are abstract icons or graphic elements.
• The long-length text is divided into sections.• There are elements in the application solely ornamental.• In the application there are icons, controls, menus, graphics, text or other redundant elements.
(H3) Consistency and standards • The same language and terminology is used in the elements that make up the application (interface controls, help, instructions, etc.).• Interface controls that appear on different screens are used whenever the same way.• Each screen that makes up the application has a consistent visual appearance.• In similar situations are repeated sequences of actions.• The design of the interface controls is consistent.
• The instructions, error messages, navigation menus, among others, appear in the same place of the screens that make up the application.
• Interface controls that are repeated throughout the application have the same functionality and retain the same location.• The size of the text (titles, subtitles, normal text or others) is consists in the screens that make up the application.• Using more than two fonts on a screen of the application.
• The text of the application uses Tiresias font.
• The text has a minimum size of 18 points.(H8) Navigation • The application allows a simple navigation, clear and predictable.
• It shows the user their location within the application, and basic navigation options.• The application provides to users a natural way of return in each of the screens that comprise it.• Each screen of the application that follows a sequence provides options to go to the main screen (home), the next and the previous screen.• In the application there are options that do not lead anywhere.• The application allows users to return to the default configuration (or an earlier one) of the elements that comprise it.• The application configuration is related to the device configuration.• The application is readily available for connections at different speeds.• The actions realized for the user take a long time.• The application informs the user whether the execution of an action requires a long time.• The application responds to the speed of navigation imposed by the user, based on their learning pace.• The application offers the possibility of auto input for the user to minimize the time entering data.• The application minimizes the use of heavy images (large size in bytes) that influence the time of application load.• The application suggests programs to users based on their preferences, history of options selected, etc.
(H11) User control and freedom • The application offers options to "undo" (or "cancel") and "redo" actions or tasks.• Tasks canceled in the application immediately stop, providing a proper feedback.• In the application users reverse their actions easily.• The application allows the user to override an action in progress.• In the application the output options are always available, visible and clearly marked.• The application offers options for returning to a point just above and to return to the main screen (main) from anywhere.• The application asks the user to confirm actions that will have drastic consequences, negative or destructive.• The application offers the possibility of auto input to minimize the time entering data and the possibility of entering incorrect data.• Users have the ability to reverse an application configuration easily.• Users have the ability to easily switch between help and their activities.• After agreeing to help, users can continue their activities from where they left broken.
• The application is easy to control by the user, at any time the user has the ability to hide or quit the application, to view the program in full screen.
(H12) Error prevention • The application provides easy to understand messages that prevent errors.(H14) Help and documentation • The application offers a clear and simple help.
• Help is expressed in the user's language, free of jargon and slang.• In the application the help is easy to find, understand and apply.• Help is always visible and available.
• The sections and information structure of the help are easily distinguished.• The instructions of the help follow the actions sequence to realize by the user to achieve a task.• The help interface is consistent with the application interfaces.• The information is easy to find in the help.
• The information presented in the help is accurate, complete and understandable.• The help describe correctly the options that offers the application.• In the complex functionalities is offered help in that same place and time.• In the application there is context-sensitive help that guides the user regarding the use of various existing elements.

CONCLUSIONS AND FUTURE WORKS
As iDT is nowadays a reality and the number and type of users is growing fast, the usability of iDT applications became a main issue.There is a need for new usability evaluation methods or at least usability evaluations should be particularized for iDT environments.
A set of 14 specific usability heuristics and an associated usability checklist were developed.The new heuristics were validated through the heuristic evaluation of three iDT applications.As no specific iDT usability heuristics were found, the proposal is based on the well-known and widely used Nielsen's 10 heuristics as well as on heuristic proposals for other fields (such as Social TV, Virtual Worlds and Grid Computing).However, as the heuristics' specification shows, the proposal is not just a particularization of Nielsen's heuristics; the set of 14 usability heuristics was specifically designed for iDT applications.
To make the experience a user with an iDT application to be successful and as convincing as possible, it is desirable to develop usable applications.The 14 heuristics proposed aims to contribute to the usability of the iDT applications in order to ensure the best interactive experience for the user; the checklist also seeks to contribute to this purpose.
Designers and/or developers of iDT applications need to learn design principles in order to create highly usable applications.Taking into account the perspective of users, usability can make the difference between developing a task completely or not, and enjoy the process or frustrated, so the proposed heuristics try to greatly increase the chance of successful of iDT applications.
As future work, it is necessary to make other case studies with different iDT applications in order to refine the set of 14 usability heuristics proposed.The performance of other heuristic evaluations will allow obtaining an important feedback to refine the set of heuristics and associated checklists.

Figure 1 .
Figure 1.Main menu of the EPG.

Figure 3 .
Figure 3. Conversation between users of the chat.

Table 1 .
Mapping between iDT heuristics and Nielsen's heuristics.
(H6) Extraordinary users: An iDT application should be inclusive, attending (all) special users' needs.toreturn to the top level should be provided, at all stages.Facilities to return to previous points and to the main screen should be provided, from anywhere in the application.Users should be able to freely explore the application, without castigation.(H14)Help and documentation: An iDT application should provide users a clear and simple help, in their own language.iDT applications should offer clear, direct and simply help, expressed in user's idiom, free from jargon and buzzwords; help should be easy to search, understand and apply.

Table 2 .
Severity and Frequency scales.
The error messages suggest the cause of the problem that has caused.• The error messages indicate what action the user must perform to correct the underlying problem.• Error messages are written in a constructive way, so it does not attribute blame or insult the user.• Error messages using violent or hostile words.• The application uses appropriate sounds to indicate errors.