After many weeks of preparation, the app testing was finally conducted for students with disabilities. Since the previous blog update, I created the experiment design and questionnaire that the students used in the testing that took place the week of March 6th. The original plan was to have the students test the applications on iPads, however on the day of the experiment the iPads that were rented out were not functioning properly. In order to continue on with the experiment it was found to be much easier to have the students install the application on their own personal phones so that each student could have their own device to use for the experiment. After some thought, I also adjusted the experiment design to have two groups: students with disabilities and pace students. Then I also had 4 apps to test. Three of the apps were available for free on the app market for download and I also had a control application which was the preinstalled task manager app on the iPhone. All of the apps were on iOS platforms.
The applications were also all task mangers that had various features to be tested against. The features were, ease of use, functionality, visual design, task addition, productivity tracking, and multimedia. Each one of these categories was defined to the test population so that they could accurately assess each of the apps in the feature categories and rate them based on their experience using the application to add a task to the application dash board. For the experiment, I researched sources and articles to figure out what categories needed to be tested. More importantly, these sources will also be used to back up the reasoning behind certain categories. The research also allowed me to better define the testing parameters and definitions that I used in the experiment.
Some of my observations during the experiment were that some of the students with disabilities needed help from their Pace mentor using the application. This may skew some of the results from the population with disabilities because the opinion of the Pace partners may be laced in with their own evaluation of the application. However, one thing that I did notice was that the testing room was very quiet once the testing began. I was not expecting this but this benefited the experiment because students were not talking about their personal opinion of the applications with each other but were completing the test sheet on their own accord. Additionally, having the testing device be a smartphone instead of a tablet will result in this research being applicable to the everyday use of students because majority of the participants had a smartphone verses having a tablet. Using an iPhone was also more convenient to the students.
My target was to get 30 students to participate, and I ended up getting 50 students to participate in the study. This is good news for the experiment because I will have a larger population of students to analyze and quantify the data. Quantifying and organizing the raw data from the testing sheets will be the next step in conducting this research study. This will give important insight into how the different groups use productivity and task manager applications as well as point to what features are needed in order for the application to be useful to the population of students with disabilities.
Thus far in the research process, I have solidified the mobile applications that will be used in the actual student groups for testing. It was important to find 3 applications that would all be similar in terms of their functionality, yet different in their design of catering to the functionality. This way there is a basis for testing all of the apps against each other, but there are clear differentiators in the apps to the test subjects who will be rating the apps. The test will involve measuring how the different apps could help with the productivity of students with disabilities from AHRC. The three apps are all organizational calendar memo apps that a student with disabilities could use to independently keep track of school assignments and tasks.
The second phase of this research process is to create a test sheet that the AHRC students will fill out when reviewing the 3 selected mobile applications for the experiment. Also, the population that will be used for the test will be AHRC students who are familiar with using mobile applications. It is necessary for the test subjects are familiar with using a mobile application so that the test results are not skewed because of lack of user knowledge, but all of the functionality is tested against the app design itself. The control in this experiment will be the regular calendar app found on an iOS device. One group of students will only use the iOS calendar app so that there will be a basis to determine if the productivity of the AHRC students increased or decreased with the 3 other test applications.
Creating the test sheet has required researching the different categories typically associated with productivity testing in mobile applications. Also, there needs to be a minimum of three sources to back up each test point so that there can be significant evidence of the need to test against those points. For example, one of the test points that I have been research is “ease of use”, or how straightforward the navigation of the app is to the user. This test point is found in over 90% of the research that I have been finding because it is so crucial to judging the productivity effect on the user of the app. However, because this productivity measure specifically deals with a population of students with disabilities, the “ease of use” category cannot just solely be based upon simple navigation, but must also include clarity of action, text size, sounds alerts, etc. All of these features are important, if not more important, than just the app work flow.
Overall, I am still in the design phase of the experiment and after returning from break I will start to conduct the experiment with the AHRC students to receive some preliminary feedback on the test sheet and applications. Then a formal experiment will be conducted with a control group and then 3 test groups against the independent variable of the mobile application. Further question that I have are what percentage of the disabilities population uses a mobile app regularly to track their productivity? Additionally, is there a standard for mobile applications specifically designed for use of people with disabilities? The AHRC student that I am working with as a subject matter expert for this experiment has been very helpful for giving insight as to the types of problems that students with disabilities face in a university setting. This particular student has also been crucial in helping to design the test sheets so that they are in a way that all AHRC students can universally understand. Work still to be done would be completing all of the research on the test points for the sheets that will be used by the test subjects. I am excited to continue researching and start to work with other AHRC students.
The working title to this research is “Mobile Apps Help Students with Disabilities Independently Manage Tasks” However, once we further define what the specific type of Mobile Apps that we will be testing, it will be clarified more in the title.
This is my first time participating in the Undergraduate Research Grant program, so my overall goal is to learn more about the research and experimentation process. Specifically for this project, my goal is to be able to pin-point what features and navigation requirements are necessary for the disability population to easily use the mobile app and also how these apps can be used to help students in their weekly schedule at school or college. Some of the research questions for this study include: What type of navigation system is most appropriate? Audio? Touch? Voice-recognition? At what points do students use the app during the week and how does this effect the type of apps that are most beneficial? What are the main features that need to be coded into the app? Can a mobile app increase productivity for this population, in addition to independence?
As I will be researching a mobile app use among those with disabilities, Dr. Lawler has appointed a student at Pace University with Cerebral Palsy to act as an official representative of the population, and also aide in some of the research and methods. One of the methods will be hands on testing of 3 selected apps from the Apple Store by a small sample population of those with a disability. This will allow for immediate feedback as well as primary data. Another method to collect information will be the W3C database, which sets the parameters for requirements of applications and other technologies for the disabilities community.
So far, I have started to determine what the test categories will be for the survey and observation that will take place in the testing phase. Dr. Lawler plans to bring a small group of about 10 students from AHRC for a couple of sessions of testing. During these testing sessions, the AHRC students will be using a variety of apps and rating them on the categories such as, functionality, visual appeal, ease-of-use, etc. From these surveys, the information will be analyzed to determine the specifics of what makes the application useful to the AHRC students and why.
An additional component to this research study would be taking the information learned in the surveys and observational sessions and applying it to making a brand new application that incorporates all of the necessary features. The Pace University student with disabilities that I will be working with will also be a great resource and aid in developing a custom application that is not available on the market.