Amazon
Research encompassing User Perception Survey and a Usability Study, uncovering some taken-for-granted problems in Amazon's User Experience and providing solutions to fix those.
Website: https://www.amazon.com/
Picture by Christian Wiediger; available on Unsplash.
Project Overview
Starting with an evaluative user perception survey (stage 1) that sought to explore user experience of loyal and longtime users of Amazon.com, the e-commerce website/app, this research found that users are already primed for a rival company of Amazon to come up in near future, despite saying they find Amazon effective and efficient to use, but evidently manifesting several pain points that made them be on the lookout for alternatives to Amazon.
Stage 2 of research started from the understanding that usability metrics are not the best tools to assess if users would switch their loyalty from their usual and familiar e-commerce app, owing to dissatisfaction with a company's service. Users’ dissatisfaction might lie elsewhere.
Therefore, this research sought to uncover where user dissatisfactions with Amazon lie, and what could be done about those.
The research concludes with several recommendations as probable solutions to pain points of Amazon users, and implications for future research on the issue.
“Pay Attention to what users do, not what they say” --Jakob Nielsen
Timeline: 7 weeks
Time Period:
August - September 2021
Tools Used: Google
Forms, Mobile Phones, Laptop.
Role: Sole Researcher
Key Project Takeaways:
-
When people invest too much time in an activity without getting suitable rewards, that experience leads to a lot of frustrations. As a result people tend to dislike or stay away from that activity, over time. This is reminiscent of the Peak-End Rule.
-
​Lack of Trust among users towards the company is emanating from their ignorance of features which are already present in the amazon web/app. This is a direct impact of Amazon's UX design. Amazon needs to improve its Interaction Design.
-
People are emotional beings, not machines. One key emotion that provides a sense of fulfillment to people is being in control. Good UX always provides that to people.
The peak–end rule is a cognitive bias that impacts how people remember past events. Intense positive or negative moments (the “peaks”) and the final moments of an experience (the “end”) are heavily weighted in our mental calculus.
​
How did I arrive at the above-mentioned takeaways? The process is laid out below.
The Problem
Painful from both Business and User Perspectives:
Even though Amazon users had indicated that they find the Amazon web/app easy to use, users still have several pain points and associate those pain points as an intrinsic part of their user experience in the Amazon website/app. This is bad news from both business and user perspectives.
Problem in Interaction Design:
Users were found to be spending a lot of time in the Amazon website/app. However, that time investment did not entail that their experience with Amazon were getting easier, or that they loved their time investment on the app/web. Neither were users getting increasingly familiar with various helpful features of the Amazon website/app. These findings indicated several problems with Amazon's Interaction Design.
Process to Solve the Problem
Ask users whether there are any issues, and if so, what are those?
Walk the walk with users in deconstructing their user experience with Amazon and ask what they would like instead.
User Perception Survey
Frame Research Questions for Usability Study & Ascertain Future Directions
Usability Study & Satisfaction Survey
Detailed Research Process
Stage 1
User Perception Survey
Research Questions:
1. How loyal customers have been to the Amazon.com website/app, especially in recent pandemic times?
2. How likely are they to change their loyalty? Which companies do they see as rivals emerging from Amazon?
3. How would they rate the overall experience of being users of Amazon, and its various features?
4. What would they like to be different in their user experience of being customers of Amazon?
Method:
​
-
Survey was administered online over Google Forms.
-
​Survey had 14 questions, including demographics questions.
-
Survey included a mix of closed-ended questions, Likert scales and two open-ended questions.
-
Total responses = 20
-
​Time taken to complete the survey = 5 minutes.
-
Demographics--
-
Gender: Male: 45%, female: 45%, 10% didn’t prefer to say.
-
Age: A majority of the sample belonged to the age group of 30-50 years,
-
Household income: A majority have a household income higher than $150,000 per year.
-
How "loyal" and "longtime" user are we talking about?
Majority of users use Amazon a few times a week
Majority of users have been using amazon since the last 5 years or earlier.
Most users have got even more loyal and invested in recent times
Majority of users said they liked using the website of Amazon.
When users say they find the app/web easy to use, they indicate how the search-based UX design a fairly intuitive path to follow.
Majority of users said they can do tasks pretty fast on the Amazon web/app.
70% of users saw some alternative to Amazon evolving in near future,
with the specifications laid out as above.
Key Takeaways:
1. Most of the Amazon users are loyal, long-time users, whose investment of time to the app have increased, with the onset of the pandemic.
2. The biggest reason of using the Amazon app/web is getting whatever they need in one place, and not because of the delivery system as could be presumed.
3. Despite being loyal and heavily invested clients, and finding the app effective and efficient, most of the same users also think a rival company of Amazon would be coming up in near future, the services of which they would hope to avail.
Key Implications & What Users Would Like More:
User Experience do not just involve use of an app/website or familiarity with it, but involves the whole experience of being a user of the product (Amazon).
In this respect, users in this survey felt that there’s much room for improvement for Amazon, as follows:
-
Get rid of bad sellers.
-
Get rid of substandard goods and products.
-
Have better recommendations.
-
Have better filters.
-
Have better delivery system, which is perceived to have slowed down in recent times; have better environment-friendly packaging and delivery.
-
Have no fake reviews; have reviews corresponding to product variation; a considerable proportion of users said they take in reviews with a pinch of salt in their decision-making process. Review system definitely could get better.
-
Have less cluttered homepage and better IA.
-
Make registries easy to find.
Results from the User perception Survey led to going deeper.
It was manifest that even though users were rating the web/app highly, they had several pain points with the it. I wanted to see what they were not saying, but were feeling for sure, to warrant significant frustrations with the e-commerce website.
​
I wanted to see:
-
What do they actually mean when they say they find it easy to use the web/app Amazon.
-
What steps do people follow when searching for and purchasing an item?
-
Are these tasks in these steps successfully fulfilled with the shortest possible time?
-
Do they use the accessible features such as "scan" and "voice" controls, at all? Why or Why not?
-
What makes people buy a particular product over others?
-
How do they go about the process of filtering through reviews, recommendations and take a decision?
-
What makes people trust specific reviews as compared to others?
-
What issues do people have with the app/website? (Identify pain points of users).
-
How do those issues impact perceptions and usage of the app/website?
​
Framing of specific Research Questions for Usability Study and ascertaining future directions
Stage 2
Usability Test Interviews
Research Questions:
-
How does the process look for users, when purchasing an item on the amazon web/app?
-
What do they mean when they say they are satisfied, or not satisfied with the app/web?
-
Which features are easy for them to use and find, and which features are not useful, or fall in the users’ blind spots?
-
What are the pain points of users?
Methods
​
-
Usability Test interviews were held over zoom and physically. 5 participants were tested.
-
Participants were recruited word-of-mouth.
-
Test interviews lasted for approximately 10-15 minutes.
-
Participants were asked to perform a series of pre-defined tasks and were asked to think aloud and describe their thoughts all along.
-
Usability test interviews were followed by a Satisfaction Survey. It took 5 minutes to fill out those surveys.
-
Data Analysis involved quantitative and qualitative methods.
Demographics
​
-
Gender: 3 Males, 2 Females.
-
Ages of Participants: Ranged from 32-year-old to 63-year-old.
-
Education: All participants had a Bachelor’s degree or above.
-
Household income ranged from $100,000 - $150,000.
-
All participants use multiple apps in their daily living, and have been using the Amazon app/web on an average of 10 years, so they were very familiar with the app and are loyal clients.
-
Variability in generations: 1 Baby Boomer, 3 Millennials and 1 Generation X.
Key Findings & Recommendations
Finding 1: Familiarity with a feature can lead to blind spots
Familiarity is incredibly valuable. It allows people interacting with a digital product or service to instantly know how to use the product, attributes confidence in users and encourages them to interact with the interface.
​
However, familiarity also leads to certain things or features falling into the blind spot of users. This usability test found that:
-
Although users are familiar with several features in the amazon app/web, these features making their life easy (e.g. the Search Bar, how to read and sort through reviews, how to finalize a product and check out), users were largely in the dark about Alternative Search Features, such as the “Scan” and “Voice” button,
-
Although users spend most of their time on Amazon in the review section, they are not aware of how to apply filters to navigate through the reviews, and
-
Users also do not know how to apply specific filters to navigate within the reviews.
Despite prolific use of the Search Bar, most users are not aware of two of the major tools within the search bar -- The "Scan" and "Voice" feature.
Screenshot of the Amazon App landing page
Satisfaction Survey Question: What do you think about these features?
​
User Responses:
-
“I do not use them much”.
-
“They are pretty advanced but might not be practical in most settings”.
-
“They are fine, but I still prefer text-based search.”
-
“Really Nice”
-
“Microphone icon could relate to voice recognition feature.”
Survey Question: Take a moment to go back to the amazon app/website. Which feature do you use least? Why do you think it’s so?
​
User Responses:
-
“Tabs on the top. I use search bar mostly.”
-
“Voice search. I don't prefer to talk with an App.”
-
“Voice, as I think it is awkward to talk to an app.”
Features That Make User Life Easy:
-
Search Bar.
-
Reading Reviews.
-
Sorting Reviews.
-
Finalizing a product and Checking Out.
Features That Make User Life Strenuous/Don’t Add to Easiness:
-
Alternative Search Features.
-
Applying Filters in Search.
-
Filters to navigate through reviews.
-
Applying specific filters in reviews
Recommendation: Make “Scan” and “Voice” features more visible or do away with these choices if they are not serving a purpose.
Finding 2: Users spend maximum amount of time on Amazon not for buying or selling things, but in the review section, because they don't trust the company.
Because certain features fall within the blind spot of the users, or they are not visible enough due to the UX design -as a result - users were spending considerable amount of time in the review section, trying to find their way through a maze of features that amazon already offers, but which do not make life of users any easier.
To clarify, in Stage 1, i.e. User Perception Survey, users had indicated that they find Amazon’s review system as mostly helpful.
However, Stage 2, i.e. Usability Test Interviews showed:
-
That users spend most of their time in the review section just to ascertain which is a fake or genuine review, being mostly unaware of the tools Amazon offers to help through the process.
-
Users would also go beyond the Amazon website/app to find reviews for products listed on Amazon, and this would significantly increase their time invested in buying an item off Amazon.
Users invoke their own devised tools to sort through reviews, chiefly because they:
(a) They cannot find or are not aware of the helpful features that Amazon provides to sort through the reviews, and/or
(b) They do not trust the features/filters available in sorting through reviews.
​
The most important stake in buying an item on Amazon are reviews. And users were found to be not happy with what Amazon is offering in the process.
There is distrust of amazon as a company, even though they were very loyal customers of amazon. This kind of backdrop is a perfect brewing ground for any competitor to enter the market.
User responses on where they spend most time and why; which features on the app/web could lessen their time spent
Recommendation: Users should be given fair amount of guidance and tools so that they know about the features Amazon already offers, and make their experience of buying an item easier, and trust the reviews the site offers.
Finding 3: Not utilitarian logic, but Emotion is most influential in decision-making while going through reviews in selecting a product to buy off Amazon.
Usability test study found that people trust specific reviews based on how far other reviewers seem to be like them. Throughout the whole process, people were trying to find reviewers who were “on the same boat” as them.
People trusted those reviews which were written by people who bought a product, and left a review out of their own volition, and people who reflect a sane and cool mind.
​
Users discarded those reviews which seem to be paid reviews, or reviews written by people with no proper first name-last name format, and those reviews which seemed to be written by "angry" and "negative people" in general, or people who are "too much swayed by emotion".
​
Emotion works, but only in moderation.
User responses on how they look through reviews which takes up so much of their time.
There is a lack of trust among users that emanates from ignorance of features, which is a direct result of the UX design that Amazon has. To compensate for the lack of trust that users feel, they overstay on the app/web just to make sure they are buying their money’s worth and is not being duped by the largest e-commerce company. This is indeed unfortunate, both for the company and its clients.
Key project takeaways/What did I learn from all these?
1. On Amazon, people spend most of their time not in browsing through products but in browsing through reviews, even though they do not like that process. This UX could definitely be improved, with the guideline of designing "good user experiences means designing good human experiences".
In this context, a good rule of thumb is to remember the peak-end rule, which is a kind of cognitive bias, involving the recall of a memory. According to this cognitive bias, human beings remember intensely emotional events more than less emotional events, and this has an effect on how we perceive an experience: we recall not the sum of how we felt throughout the experience, but the average of how we felt during the peak emotional moment and at its end.
​
2. There is a lack of trust among users that emanates from ignorance of features, which is a direct result of the UX design that Amazon has. Trust is the primary fuel that lubricates human relationships and interactions. Amazon would benefit a lot by strengthening the relationship of trust with its users and the first step could start with enhancing their user experience by looking at the features that erode trust.
​
3. People do not behave like machines. People could be easily distracted, people could be inconsistent or irrational, people could be error-prone, but people are usually driven by emotion. And one of the key emotion that provides fulfillment, and appears rewarding to people, is being in control.
Therefore, to meet users’ expectations, the products and services that designers build must be robust and adaptable, so that people feel they are under control and don’t have to engage in a complex game of figuring out an interface when they are spending their money.
Suggested Feature Changes:
-
Avoid clutter in the UI, so as to reduce users getting overwhelmed with too many things.
-
A fair amount of guidance regarding tools/features should be present, so that users are aware of the features Amazon already has.
-
Revise the current system of algorithm that picks up most occurring words (in reviews), using them as built-in filters → to developing an algorithm that picks up functional and meaningful words as filters.
-
Make searching within reviews and customizing the search of reviews more visible.
-
Implement an option of sorting reviews through pictures, instead of just showing a feature of “Reviews with Images”.
-
Make reviews more credible, through initiating:
-
Start features such as “verified names” while accepting reviews,
-
Take steps to show reviewers as real people,
-
Make “paid reviews” more visible, and/or do away with fake reviews altogether, to help users cut down on this mess of puzzle-solving.
-
Future Research Implications/Next Steps
Eye-tracking
Plan
Implement Recommended Feature Changes
Make Amazon more Accessibility -friendly
Conduct Iterative Usability Study
Eye-tracking Plan
Research Questions to be explored:
-
Do users use the accessible features such as “scan” and “voice” controls, at all while searching for a product? Why or Why not?
-
How do users go about the process of filtering through reviews, recommendations and take a decision?
How many participants to be recruited?
-
6 participants would be recruited.
-
Maintain variability in the sample in age and generations.
-
Assess whether gender has any interacting role to play in eye-tracking test results.
Why Do This?
-
To supplement our understanding of where users are actually looking, when users are looking at the Search Bar /other features.
-
To know where user attention lie, and their fixations are.
-
Eye-tracking would help illuminate on which content attracts and which is unnecessary, which features distract user attention, in going with the concept of Fitts’ Law.
-
To discover potential areas of confusion once user get onto the review segment, to know which areas/features get most of their fixations and which do not, as well as fixation duration, fixation frequency, fixation timing. This would help in building better visibility of the various tools Amazon offers in order to sort through reviews
Making Amazon Accessibility-friendly
What is Accessibility?
"Digital accessibility is the ability of a website, mobile application or electronic document to be easily navigated and understood by a wide range of users, including those users who have visual, auditory, motor or cognitive disabilities."
Amazon has several accessibility concerns, details of which could be uncovered through the following tools to check accessibility:
​
-
Wave Accessibility Assessment Tool provides a set of evaluation tools that helps designers make their web content more accessible to individuals with disabilities. In addition, WAVE can identify many accessibility and Web Content Accessibility Guideline (WCAG) errors, as well as facilitate human evaluation of web content. One can use the online WAVE tool by entering a web page address (URL), and then the issues could be identified readily. Amazon has a tonne of issues which are readily available when entered into the Wave web address.
-
Tota11y provides an annotated summary of accessibility issues, such as issues with color, contrasts, content (labels and link texts) in a webpage. One can add the url of Tota11y to the bookmarks bar, to open any webpage and then the bookmarklet could be added to that webpage to check for accessibility issues.
-
Toptal is a tool to check apps/websites if they have any accessibility issues for those with color blindness. This resource shows what a webpage looks like for someone with color blindness. One can enter a website’s url and choose a color blindness filter, and the tool will show the issues accordingly. There are four different filters/views available: Protanopia, Deutanopia, Tritanopia, and Greyscale.
Making websites accessible for all leads to greater accommodation for those who need it the most, without cutting down the convenience of anyone. It’s also good for business, in that it leads to reduction in lawsuits.
Iterative Usability Study
-
Participants would be quickly recruited and tested over 1 day.
-
Low-fidelity wireframes/mockups could be used with new features.
-
After each usability session, participants would be asked to complete a satisfaction questionnaire.
-
Participants would then be asked some debriefing questions on specific aspects of the site, and/or will clarify on certain specific things participants might have shared during the usability study.
-
Each session would be audio and video recorded.
-
Feedback from the sessions could be used to implement certain feature changes or retain some/all the changes and would inform next round of iterative study (Iteration 2).
-
It is expected that the changes introduced will not lead to new issues but will inform on whether the pain points of the users were satisfactorily resolved.
Following hypothesis could be tested:
-
Better visibility of filters and keywords in reviews would cut down in time spent in sorting through reviews.
-
More functional sorting methods of reviews would aid in time spent in sorting through reviews.