Shuang Liu

Animator

An iOS app for organic chemistry education

Client
Team
Timeline
Methods
Alchemie Solutions
Shuang Liu
Patrick Parkinson
Vanessa Rychlinski
Dragana Spasevska
01/2017 - 04/2017
User Interview, Comparative Analysis, Heuristic Evaluation, Survey, Usability Testing

Alchemie Animator

Funded by the National Science Foundation, Animator was created using the principles of open-ended video game design and is intended to allow users the flexibility to create and share organic chemistry animations. Julia Winter, CEO of Alchemie Solutions, envisioned it being used both in and out of the classroom. “An app that brings students together”.

Our Mission

Animator was still at the pilot testing stage. Before the official release of the Animation Tool, the stakeholders wanted to know how current users felt about Animator and what potential users expected out of this app. Stakeholders wanted to make sure that Animator addressed target users' needs and that the app was usable. Our team's mission was to find out users' needs for chemistry studying and teaching and identify potential usability issues to make sure Animator was ready to hit the market.

User Interviews

After conversing with our stakeholders, we built personas for the target population.

4 Interviews

Based on our personas and extensive exploration of the app, we identified the target population of Alchemie to be high school AP chemistry students, collegiate organic chemistry students, chemistry teachers at both the high school and collegiate levels, and someone in supplemental teaching roles such as a chemistry tutor or study group leader.

We created an interview guide with the goals of the interviews and listed all of the questions, separated by interviewee archetype (student vs teacher). We employed a semi-structured interview approach. Each interview lasted 30-45 minutes and was recorded and transcribed.

Key Findings

  1. The Alchemie Animation Tool targeted the right pain points in organic chemistry study.
  2. Users expressed that Animator is not powerful enough for organic chemistry drawings.
  3. Potential users and current users have different expectations for the use of Animator.
  4. Animator needs to be more user-friendly and easier to learn.

Recommendations

  1. Animator needs to improve the drawing tool.
  2. Animator should have “smart molecules”.
  3. Animator should provide tutorials and a user-guide.

Comparative Analysis

We summarized a list of competitors from our stakeholder interviews, user interviews and online research.

Based on our user interviews, we identified 5 key dimensions for comparison. We gave a rating of each product on each dimension and plotted the results in a radar chart. Animator's biggest weakness is lack of built-in intelligence.

  1. Learnability - What's the learning curve of each product?
  2. Intelligence - Does the product help users identify wrong charges, etc.?
  3. Efficiency - Can users reuse the molecules and equations they created?
  4. Recoverability - If users make mistakes, can users easily fix their mistakes?
  5. Availability - What kind of devices does the product support?

Findings

In summary, competitors offer more powerful features for drawing, 3D modeling and animation.
  • Animator lacks built-in intelligence (e.g., molecule libraries)
  • Animator does not allow 3D rotation and manipulation for molecules
  • Animator does not allow users to select frames for view, edit or reuse.

Recommendations

  • Users should be able to view and manipulate molecules from different angles in a three-dimensional fashion.
  • Alchemie should offer built-in chemistry intelligence to help users make calculations and validate molecules.

Survey

Based on the findings from our user interviews, we wanted to survey potential users to assess their attitudes, behaviors, characteristics, and needs for an app designed to create organic chemistry animations. We have 3 main research questions in mind:
  1. What challenges people face studying organic chemistry?
  2. Do chemistry learners/educators desire a digital learning tool?
  3. What are the features important for those that do use digital learning tools?

Pilot Tests

Our team conducted pilot tests at the Science Learning Center in the chemistry building. We distributed 20 hard-copy surveys to 20 students and gathered feedback on wording usage and total time.

Recruiting

Without any budget or resources, our team brainstormed how and where we could reach large amounts of individuals that qualified as potential users. We identified all of the chemistry clubs on campus using MCommunity, the University’s club pages, and sent the survey to their email lists.

Data Collection

The Google Forms survey was open for a total of six days and final metrics revealed that we received a total of 164 responses.

Sample Findings

  1. Users value intelligent molecules and 3D manipulation features.
  2. Although drawing is the main reason people use chemistry software, the majority of users still prefer to use pen and paper to model chemical reactions.

Heuristic Evaluation

Our team conducted heuristic evaluations to diagnose usability problems of the entire application. We used the 10 heuristics created by Nielsen (1994) and developed the interpretation of each heuristic for our product. We conducted individual evaluations, aggregated our assessments as a team, and identified critical usability issues.

Sample Heuristic Report

Findings

Of the 10 problems, 4 problems were given low severity ratings (between 0-2.99) and were deemed by the evaluators as cosmetic problems. One usability problem was rated as a “major usability problem” (between 3.0-3.49) and five usability problems were rated as "near-usability catastrophe” (between 3.50-4.0) and are imperative to be addressed before the Animation Tool is released.

  1. There is no tutorial.
  2. The users do not know if their created/designed atoms or molecules are possible in the real world.
  3. There are limited (3) pre-created atoms, requiring a lot of work from memory.
  4. Once an atom is created, it cannot be edited or deleted.
  5. The user does not receive an error message after trying to save with missing required information.
  6. Atoms and molecules do not react how they would in organic chemistry.

Usability Testing

Our team conducted 5 usability tests that focused on the core features and main use cases (create and save animations for chemical reactions).

The tasks

  1. Use the preloaded atoms to create a molecule
  2. Create custom atoms
  3. Create a molecule using custom atoms
  4. Create an animation using atoms/molecules
  5. Save animation as a template
  6. Load a saved animation template

Recruiting

We recruited participants by sending recruitment emails to University of Michigan undergraduate and graduate listservs we felt might have high concentrations of potential users. These included various schools and colleges such as the medical school, dental school, pharmacy school, school of public health, and school of information, to name a few. In addition, we also sent recruitment emails to listservs such as “Pre-Health Information”. Included in the email was information on the app, what we were hoping to accomplish, and that participants would receive $25 compensation.

Testing

Each test included a pre-test questionnaire, six tasks, and a post-test questionnaire.

Findings & Recommendations

Final Deliverable

We summarized our findings from all of the studies and created a 7 minute video as a final deliverable.

The Impact

Alchemie officially launched the Animator App in April 2017. Throughout the iterations of Animator, we were happy to see some of our suggestions being implemented.

1. Animator added a tutorial

Lack of tutorials was one of our top recommendations. In May 2017, Alchemie released a series of tutorials on YouTube.

2. Randomize where new items show up

Our usability studies found that users didn't know they successfully added new atoms because the new atoms show up right on top of existing items. We suggested Animator to randomize where new items appear on the screen.

3. Add error messages

As we recommended, Animator added validation error message when the user tried to save an animation with issues.