14  SAT 8 Skills in conducting usability testing:

Study Design PSM: - Testing checks whether solutions meet all the requirements, function as expected and are usable by intended users. - Develop a testing strategy to ensure that the solution works as intended, including features to be tested, test data and expected results. - Conduct tests, record the actual results and compare the actual results of the tests against the expected results. - Correct identified errors.

14.1 Skill 8: Usability Tests - VCE Software Development Guide

14.1.1 Introduction to Usability Testing

Usability testing is a critical part of the software development process. It involves evaluating a software product by testing it on users to ensure that it is intuitive, accessible, and meets user needs. This skill will help you create software that is not only functional but also user-friendly.

14.1.2 Objectives

  • Understand the purpose of usability testing.
  • Develop a usability test plan.
  • Conduct usability tests effectively.
  • Analyse and report on the findings from usability tests.
  • Modify the software solution if time allowed

14.2 Step 1: Understanding Usability Testing

14.2.1 What is a Usability Test?

A usability test is a process in which real users are observed while they interact with a software product to determine how easily and effectively they can use it. The aim is to identify any issues that users encounter, such as confusion, frustration, or ambiguity, as they perform specific tasks. The researcher or tester notes any problems and collects feedback, which is then used to improve the software’s usability. The focus is on ensuring that the software is intuitive and meets the needs of its intended audience.

14.2.2 Why Conduct a Usability Test?

Conducting a usability test is crucial for several reasons but also depends on the specifics outlined in your Software Requirements Specification (SRS). The SRS defines the functionality, performance, and constraints of the software, and these factors influence what aspects of usability you should focus on during testing. For example:

  • User Experience (UX) Requirements in the SRS will guide you in determining the key usability aspects to test, such as ease of use, intuitive navigation, and overall user satisfaction.
  • Functional Requirements might specify certain features that need to be tested for usability to ensure they meet user needs effectively.
  • Accessibility Requirements outlined in the SRS will dictate the accessibility standards that the software must comply with, which should be evaluated during usability testing.
  • Compliance Standards mentioned in the SRS will inform the specific industry or legal standards the software must adhere to, such as the Web Content Accessibility Guidelines(WCAG) for accessibility, and these will be tested during the usability process.

In summary, the scope, focus, and methods of your usability testing should align with the requirements and constraints specified in the SRS to ensure that the software meets all necessary criteria effectively.

14.2.3 Methods for Conducting a Usability Test

Here are some of the most common methods used in usability testing:

  1. Interviews
    • Conduct interviews with participants to gather qualitative feedback on their experience with the software.
  2. Audio Recording
    • Record the session’s audio to capture verbal feedback and participant reactions.
  3. Video Recording
    • Use video recording to document both the screen activity and the participant’s physical interactions, such as facial expressions and body language.
  4. Screen Recording
    • Capture the participant’s screen activity to observe how they navigate through the software.
  5. Surveys and Questionnaires
    • Collect structured feedback from participants through surveys or questionnaires after the usability test.
  6. Observation
    • Watch participants as they use the software to note their behavior and any difficulties they encounter.

14.3 Step 2: Developing a Usability Test Plan

When creating a usability test plan, it’s essential to consider several key elements to ensure that the test is thorough and effective. Below are the important components to include:

A usability test plan outlines the objectives, methods, and logistics for the usability testing process.

14.3.1 Key Components:

  1. Objectives: Define what you want to achieve with the usability test. For example, “Identify any navigation issues users encounter on the main interface.”
  2. Users: Select a diverse group of users who represent the target audience of your software. This could include different age groups, technical proficiency levels, etc.
  3. Tasks: Create specific tasks for participants to complete during the test. These should be tasks that are essential to the software’s functionality, such as “Create a new account” or “Navigate to the help section.”
  4. Metrics: Decide on the metrics you’ll use to evaluate success. Common metrics include time on task, error rate, and user satisfaction.
  5. Data Collection: Determine how you will collect and store the results of your usability test.

14.3.2 Objectives:

The “Objectives” of the usability test are to evaluate specific aspects of the software to ensure its features function correctly and that users can achieve their goals efficiently. This involves assessing the software’s accessibility, reliability, and overall user satisfaction.

Reminder: Make sure to refer to your Software Requirements Specification (SRS) to identify and align these usability test objectives with the requirements detailed in your project.

14.3.3 User (Audience)

  • Define the target audience: Identify the types of users who will participate in the usability test. This should include users who represent the intended audience of the software, such as different age groups, levels of technical expertise, and specific needs or preferences.
  • Recruit participants: Select a diverse group of participants who match the user personas or demographics relevant to the software.

14.3.4 Tasks

14.3.4.1 Duration

  • Set the length of the test: Determine how long each session will last. Typically, usability tests should last between 10 to 30 minutes, depending on the complexity of the tasks and the number of tasks to be completed.

14.3.4.2 Method And Equipment

  • Choose the testing method: Decide on the methods you will use to gather data, such as interviews, screen recording, or think-aloud protocols. The method should align with the goals of the test and the type of feedback you want to collect.
  • Prepare the tasks: Develop a set of tasks that participants will perform during the test. These tasks should be representative of common user activities and designed to reveal potential usability issues.
  • List the necessary tools and technology: Identify the equipment required for the usability test, such as computers, mobile devices, screen recording software, cameras, and microphones.

14.3.5 Metrics

  • Prepare testing tables: Set up a table or spreadsheet to track participant interactions, tasks, and responses. Each row might represent a task, and each column could represent a different participant.
  • Include key metrics: focus on are efficiency and effectiveness.

The key metrics for evaluating efficiency in a usability test include time, cost, and effort.

The key metrics for evaluating effectiveness in a usability test include completeness, readability, attractiveness, clarity, functionality, accuracy, accessibility, timeliness, report format, relevance, usability, and communication of message.

Usability vs. Usability Testing

Usability is one part of what makes software effective. It specifically measures how easy and intuitive the software is for users.

Usability testing is a broader process that evaluates many aspects of the user experience, not just ease of use. It’s a method for identifying and improving how users interact with software.

So, while related, usability is a specific metric, while usability testing is a process for improving the overall user experience.

Example: A software application with a clear interface and logical flow has high usability. Usability testing would involve watching users interact with the software to find areas for improvement, like making buttons more visible or simplifying complex tasks.

14.3.6 Plan for Collecting Usability Testing Data

  • Plan for data collection: Decide how you will collect and store the results of the usability test. This might include digital recordings, observation notes, or survey responses.
  • Organise the data: Ensure that all collected data is organised and easily accessible for analysis. Use tools like spreadsheets or dedicated UX research software to categorise and analyse feedback.

By including these elements in your usability test plan, you’ll ensure that the testing process is well-organised and that the results will be valuable for improving the software’s usability.

14.3.7 Usability Test Plan Example

See Section 14.8 for more details.

14.4 Step 3: Conducting a Usability Test

14.4.1 Key components

  1. Establish how long the testing will take and where it will be conducted
  2. Emphasise that the interview is about the software product and not the user’s ability to operate the software and the equipment.
  3. Consider a video recording. This recording could be consulted to provide further data but must not be used for any other purpose. A signed consent form will be required if video is to be recorded. Make sure to clearly explain the purpose of the recording to the participant and assure them of confidentiality.
  4. Explain the use of equipment and software. Ensure the participant understands how to operate any necessary tools or devices used during the test. This helps reduce any confusion related to the technology itself and allows the focus to remain on the usability of the software.
  5. Introduce each task clearly and observe the user’s responses. Ask the user to ‘think aloud’ to ensure their actions are clear. This helps in understanding the user’s thought process and identifying any potential usability issues.
  6. At the completion of the tasks, thank the user for their time and invite any comments, likes, dislikes, or suggestions for improvement. This step is essential for gathering final thoughts and additional feedback that might not have been captured during the tasks.

14.4.2 Think Aloud Protocol

The Think Aloud Protocol is a usability testing technique where participants verbalise their thoughts as they interact with a product. By speaking aloud, they reveal their reasoning, frustrations, and decision-making processes in real-time. This method provides valuable insights into the user experience, helping testers identify usability issues and understand user behaviour more deeply.

The Think Aloud Protocol can be integrated into all usability testing methods. Whether through interviews, recordings, screen capture, observation, or surveys, it helps gather real-time insights by encouraging participants to verbalise their thoughts, making it a versatile tool for any testing approach.

14.5 Step 4: Analysing the Results

After completing the usability tests, it’s important to analyse the data collected.

Steps:

  1. Review Notes and Recordings: Look for patterns in the feedback and issues encountered by participants.
  2. Identify Common Issues: Focus on usability issues that affected multiple participants, as these are likely to be the most critical.
  3. Prioritise Issues: Rank the issues based on their severity and impact on the user experience.
  4. Provide Recommendations: ⭐️For each issue, suggest possible improvements. For example, if users struggled to find a particular feature, recommend making it more prominent in the interface.

14.6 Step 5: Modifications

Document the changes made to the software solution based on the insights gained from the usability testing results. Use the following format to organise and prioritise the modifications:

  • Priority: Assign a priority level to each modification (e.g., High, Medium, Low) based on its impact on usability and the severity of the issue it addresses.

  • Proposed Change: Describe the specific change that will be implemented. This should include details about what will be modified, added, or removed from the software solution.

  • End Result: Outline the outcome of the proposed change.

  • Additional Detail: Provide any extra information relevant to the modification, such screenshots, descriptions, or code.

14.7 Additional Tools

14.8 Usability Testing Plan

14.8.1 Objectives of the Test:

  • User accounts are created, and users can log in and log out.

14.8.2 Scope

  • What will be tested:
    • The usability of the account login and logout process on the Django website.

14.8.3 Duration and Location

  • Where, When, For how long

14.8.4 Purpose

  • Test questions, concerns, and goals:
    • Determine how easily users can locate and access the login and logout features.
    • Identify any difficulties users may encounter during the login process, such as entering credentials, handling errors, and navigating to the account page.
    • Assess the intuitiveness and responsiveness of the logout function.

14.8.5 System Requirements

  • Hardware and software necessary for the test:
    • Participants will need access to a computer or mobile device with an internet connection.
    • Testing will be conducted on commonly used web browsers such as Chrome, Firefox, and Safari.
    • Screen recording software (e.g., OBS Studio) will be used to capture user interactions during the test.

14.8.6 Users

  • Identify who the interview participants will be:
    • Participants will include both Admin Users and Normal Users to ensure that the login and logout processes work smoothly for different types of accounts.

14.8.7 Forms

When conducting a Usability Test, we use different forms to gather important feedback about various tasks users perform, such as logging in, navigating the site, or completing specific actions. These forms help us understand how easy or difficult it is for users to interact with the software, what they like or dislike, and where they might encounter issues. They also help us keep records for further analysis, making it easier to track patterns and identify areas for improvement.

14.8.7.1 Some Form Examples

  • Login and Logout Usability Interview Form:
    • How easy was it to find the login page?
    • Were the login instructions clear and easy to follow?
    • Did you experience any issues when entering your credentials or logging in?
    • How would you describe the ease of logging out after using the website?
    • Did the logout process meet your expectations in terms of clarity and responsiveness?
  • Login and Logout Performance Metrics Form:
    • Time taken to locate the login page and successfully log in.
    • Success rate of first-time login attempts without encountering errors.
    • Time taken to log out and return to the homepage or login screen.
    • Number of errors or failed attempts during login and logout processes.
  • Login and Logout Experience Survey Form:
    • After the usability test, collect feedback on what participants liked and disliked about the login and logout process.
    • Gather suggestions for improvements, particularly in areas where users experienced difficulties or confusion.

14.8.8 Methods and Data Collection:

  • Interviews
  • Observations
  • Surveys

14.9 Storing Collected Data in a Table

14.9.1 Testing table 1

SRS Requirement Type Expected Result Actual Result Note (Fix)
Users can access the website by mysite.com.au Input Home page is displayed when accessing the URL
Admin users can import Quiz questions Process Quiz questions are successfully imported🤔
Record (CRM) data is easy to read Output Format is clear and organised, with appropriate spacing, fonts, and alignment. Users can scan and understand it quickly.

🤔 How do you know “Quiz questions are successfully imported”?

💯 Change it to “Quiz questions are successfully shown in the Question List View”

14.9.2 Testing Table 2 - Importing YAML Data into SQLite Database

SRS Requirement Type Expected Result Actual Result Note (Fix)
Users can create a YAML file in the root folder Input Users can successfully create a file named quiz_data.yaml in the root folder [File created successfully] [No fix needed]
Users can input quiz data into quiz_data.yaml Input Users can add valid quiz data into the YAML file as per the required format [Data added successfully] [No fix needed]
Users can load the YAML data into the database using Django’s loaddata command Process The command python manage.py loaddata quiz_data.yaml imports the data without errors [Data loaded successfully] [No fix needed]
Existing data is overwritten Process The data in the database is correctly overwritten with the new data from the YAML file [Data overwritten correctly] [No fix needed]
Data is correctly stored in the SQLite database Output Data is correctly inserted into the SQLite database, with no data loss or corruption [Data stored correctly] [No fix needed]
Data is retrievable and easy to read Output Imported data can be queried and is presented in a readable format in the database [Data retrieved and readable] [No fix needed]

🤔 Is this a reasonable usability test?

15 Think Aloud Protocol for Usability Test

15.1 Task: Logging into a Django Server Admin UI

15.2 Objective:

Students will practice conducting a Think Aloud Protocol by guiding a partner through the process of logging into a Django Server Admin UI. The task will help students observe how their partner interacts with the login interface and capture their thought process using the Think Aloud method.

15.3 Materials:

  • Access to a Django Server Admin login page (or a simulated login form if a live server is not available).
  • A checklist of steps the participant needs to complete, such as:

15.3.1 Task: Checklist for Logging into a Django Server Admin UI

Assuming the Django Server is ready:

  1. Type 127.0.0.1:8000/admin into the browser.
  2. Enter the username: __________________
  3. Enter the password: __________________
  4. Click the “Log in” button.
  5. Verify that you have successfully logged into the Django Admin interface.

15.4 Think Aloud Protocol Activity Instructions:

  1. Pair Up:
    • Divide the class into pairs. One student will be the “User,” and the other will be the “Observer.”
  2. User’s Role:
    • The User will go through the process of logging into the Django Server Admin UI. As they interact with the login page, they should verbalise their thoughts. For example, they might say, “I’m entering my username,” or “I’m not sure if my password is correct.”
  3. Observer’s Role:
    • The Observer listens carefully and takes notes on what the User says and does. If the User falls silent, the Observer should gently prompt them with, “What are you thinking right now?” or “Can you describe what you’re trying to do?”
  4. Switch Roles:
    • After 2-3 minutes, have the students switch roles so both get a chance to be the User and the Observer.
  5. Class Debrief:
    • After both rounds, come together as a class to discuss:
      • What difficulties did Users encounter while logging in?
      • How did the Think Aloud process help Observers understand the User’s experience?
      • What feedback or insights did they gather regarding the login process?