“By 2022, your personal device will know more about your emotional state than your own family.” — Annette Zimmermann, vice president of Gartner.
Using pre-existing data to train a computer to identify patterns and make future decisions is machine learning. Machine Learning has numerous applications from youtube recommendations to virtual assistants like Alexa or Siri. How could this field of technology be impactful in creating User Experience?
Over the recent years Machine Learning and User Experience have become a topic of interest and has created a lot of buzz. The interdisciplinary combo focuses on using data science techniques on UX problems and building better experiences. We will discuss later in this paper how this works.
On further contemplation, at its core AI/ML and HCI aim to interpret human behavior and anticipate what someone will do next. While in machine learning, the end goal is to mimic human behavior, in HCI it is to build a product that is able to facilitate all such behaviors, and create successful experiences. How do we use the former to better the latter?
Why do we need to study emotions?
As Don Norman explains, our brain has two sides:- the cognitive side and the emotional side . The Cognitive side is logical and makes rational decisions. Emotions are just how we feel and respond to things. When designing a product it is very important to keep in mind emotions of the human interacting with it. You could ask, isn’t it just enough to make a product that works and is functional? I will answer this question by sharing a conversation I had with someone during an “Introduction to Design thinking” workshop during my time at NYU.
A part time NYU Stern student, who had worked in T-mobile for 13 years was attending a design workshop.
I asked him what brought him here.
He explained how they had spent years building products that no one ended up using, even though it fulfilled its said purpose.
I was curious and asked why,
to which he replied, it simply wasn’t enjoyable to use….
In this era of technology, it is very important to arouse good emotions through user experience in order to build a successful product…
Sentiment Analysis — What are the gaps ML fills?
Now that we have realized the importance of considering emotions during design. Let’s understand, where ML can play a role in it. We have learned that everyone interprets and experiences things differently. And they have different emotional reactions to the same product. For example; my friend loves the elevator at 2 Metrotech, whereas I find it unnecessarily complicated. How do we then build products with good experiences for everyone? This can be done with Sentiment analysis and studying large amounts of data to gain insights on what is working for who and designing accordingly.
Google Vision API, shared in the class material, does exactly this by providing tools to detect emotions from texts and images. Using ML to make these decisions has the following advantages:
- No observer bias: In order to keep designs human/user-centric we need to observe how they interact with products. There sometimes is a bias in these observations, because the observer may see what they want to see (hopefully unconsciously) . This can be eliminated if we use a computer to give us observations.
- More naturalistic observation: When analyzing users in a tensed environment like a research interview they tend to not act very naturally. ML eliminates this problem as real data, comments and images are used to derive conclusions.
- Large user base: During User Research, if a product is used by a very diverse group of individuals it can get difficult to account for everyone’s needs. Machine learning algorithms can make this much simpler by classifying the groups and giving insights.
These advantages can help build products that are more inclusive of all groups of users. User research becomes much easier with more accurate results. That being said, this approach can be used in solving very specific problems in the design thinking process and might not be necessary on every step of the way.
Applications and Examples
I have identified various categories where Machine learning has an application in making the experience with digital products better, these include:
Studying Emotions
This is the main application of ML in UX, as discussed earlier. Machine learning is used to group emotions that users express towards a product. To achieve this data is gathered social media posts and webcams of the product users.
Example : I interned for Nissan Digital LLP, which focused on making digital products for Nissan motor corporation. The data science team worked closely with the design team to understand the emotions of customers of their new affordable electric car “Nissan Leaf”. They created their business model based on the results of the machine learning pipelines to provide offers, refine features in the car and overall better user experience.
User feedback
For a product to be truly human-centric, feedback from users needs to be taken into consideration. Design thinking is an iterative process, and any difficulties or dislikes people have with the experience should be recorded and worked on. Chatbots are a very effective way to achieve this.
Example: Many e-commerce websites use chatbots for 24/7 customer service and feedback on how the user felt while using the webapp. In Fall 2021, we created a color blind awareness webapp in which I integrated a chatbot. The chatbot answered frequently asked questions by users, it permitted the user to make appointments with eye specialists and asked “how are you feeling?” type pop up prompts to be able to rate the experience.
Automation
In the broader spectrum this may not seem like a direct application in UX. But, I believe anything that makes the interaction between human and a machine easier and efficient is relevant and worthy of mention in a HCI paper. Tesla is the best example for enhancing user experience by automation, but there are still many people who still prefer manually driven cars.
Example: AES Corporation, a Fortune 500 company, develops renewable energy projects. They use drones to capture images of defects in windmill turbines and use ML to classify the defects. This has reduced human efforts by 40%. This means the time taken to just detect defects can now be used to fix them.
Personalized Advertising (persuasion)
Clicks, ratings and searches are used to make personalized recommendations to users. Amazon is among the very best. The retail giant attributes 35% of its revenue to personalized product recommendations, which are constantly updated and optimized to engage every individual user. Don Norman’s video about “the impact of persuasion’’ highlights the impact of this application in HCI. There are also ways to make personalized ads less creepy. (https://hbr.org/2018/01/ads-that-dont-overstep)
Ethics and is it worth it?
Artificial intelligence and machine learning have made tremendous strides. The levels of abstraction: that basic mathematical probability, linear algebra and calculus can turn into mathematical models to make machines understand human language and sentiments is truly unimaginable. The amount of research and effort this must have taken truly baffles me. But something doesn’t seem quite right. Should we really be using this technology to sell people things? When brands take user data to further their agenda and sell products is it to improve experience or just make more money? It really concerns me to think that there is no data privacy whatsoever. It is not a coincidence that I happen to see ads of emerald green jackets that I was talking to my sister about. Is this the era of technology we are in, where we are just eyeballs?
I firmly believe that these technologies can be used to further HCI ethically. AES is a great example of this. Technology should be used to help better the world, not to distract people and grab their attention towards things they don’t need (Tiktok is a good example of this) and promote consumerism. This might just be my stance, people are free to disagree and have their own understanding in this matter.
A final word.
In conclusion, the machine learning approach in HCI and UX is an emerging field with a lot of potential. If implemented well and used ethically it can be used as a very helpful tool in UX design. I have strongly mixed feelings about the extent to which this combination will increase persuasion and dark patterns in UX, but I also have faith that something good and revolutionary will come out of it. I am excited to be a part of these changes happening in the field during this time.