Puffed. A new way of sensing our unconscious decisions. 

Final report for MSc Global Innovation Design.
May 2018.

Author: Leñero Agraz, Melisa Florentina.


Introduction 

Consumer neuroscience, the study of consumer behaviour understood through their neural causes, is considered an emerging field. Their applications are most likely to be used in marketing research and advertising to understand customers' unconscious preferences and boost engagement and profitability within the retail industry.

However; consumer neuroscience could be migrated to the design field not just to learn more from what drives behaviour; furthermore, inform the user about their performances, opening the possibility to create designs which could enhance human behaviour and augment intrapersonal intelligence.                     

Neuroscience literature suggests that humans are incapable of keeping up with time, it takes up to two hundred milliseconds to perceive the environment, processes the information and commands an action. Between processing information and commanding an action, there are specific mechanisms under the unconscious state which are faster to select a choice before even alerting the conscious state (Eagleman 2009). It is possible to like something or being afraid of something without being capable of explaining what it is or even without knowing what it is (Zajonc 1980). These unconscious preferences or internal nudgers which drive behaviour, fall under the cognitive limitation definition. This report analyses two essential aspects: a) if there is a relationship between specific brainwaves and unconscious preferences. And b) which brain waves can alert in real-time when an unconscious choice is happening. Thus if the hypothesis is proven right, it could be inferred that addressing an unconscious preference before it has been articulated could consequently reassess the impact of cognitive biases in the decision-making process.

For this experiment, ERP or the Event-Related Potential component in brain frequencies is analysed while listening to a preferred and non-preferred song aiming to set the parameters in which a preference occurs in the brain. This experiment is part of a body of work of an ongoing Master’s project. The results obtained from this exercise will inform the second stage consisting of tracking in real-time the brain activity and notifying the user when presenting an unconscious preference. With the overall aim to harness the embedded brain power through a collection of tools. These tools when a systematic error happens will emit an alert to encourage rethink behaviour and potentially augment human intelligence. 

Background and Motivation

Technology has been used by capitalism as a tool for driving consumption as opposed to a tool by augmenting the human for better interpersonal understanding. Most of the knowledge of the decision-making process and the preference studies have been exploited in neuromarketing due to the possibility of reducing costs and accessing new information not possible with the conventional tools (Ariely et al. 2010). Moreover, neuroimaging opens a new window of possibilities to visualise the brain activity in real-time, unravelling characteristics and features perceived with the senses not even possible to articulate. (Ariely et al. 2010). Although many tools have been designed to access the brain imagery, the Electroencephalogram (EEG) which detects blunt peaks of electrical activity through the scalp, may be proven to be an affordable and a valid alternative (Badcock et al. 2013). 

Studies show that EEG has already been used to detect preferences in consumers (Kushaba et al. 2015). However, due to the novelty of the field, finite experiments are studying the choices in both, cognitive and emotional neural connections (Kushaba et al. 2015). By comparison, in the Design field exists limited or unperceivable experiments regarding unconscious preferences elicited through EEG, yet without signs of having combined Neuroscience with the design. Though, it could be said that two of the most representative areas in Design concerned about behaviour and unconscious preferences are Behavioural Design and Neurodesign. Both of them with a clear and deep understanding of the Design as an enhancer of human knowledge (Hevner et al. 2014) which provides informed solutions to the everyday human problems (Norman 2004) embodied as artefacts or services (Hevner et al. 2014). Nevertheless, the results of the design process are not entirely focused on providing feedback from the human behaviour to feed and possibly modify, when necessary, the interpersonal intelligence. Therefore, it can be argued that these two design perspectives are aligned with the cognitive processes but not necessarily to aid those cognitive limitations embedded in the brain. 

It has been stated that good design elicits the affordances or general hints embedded in the object to nudge behaviour (Norman 1988) so that the item can be interpreted successfully under a known or unknown environment. Furthermore, skeuomorphism is the term coined to refer to the derivative object (Basalla 1988) inherent to the original one based on its affordances. 

Both affordances and skeuomorphism, seemingly rely on the automatic associations that could derive from cognitive biases. Under these particular circumstances, certain cognitive biases like anchoring or implicit association seem to be beneficial, however; with the use of machine learning, it seems possible to tune stringent messages which will alter some biases to persuade behaviour (Diehl et al. 2016).  

The vast amount of personal data that has been acquired through the Internet has led to algorithms that predict human behaviour and personality traits not necessarily shared by the individual (Kosinski et al. 2012). This information kept from the public domain represents a severe threat to the privacy of the users. (Kosinski et al. 2012).

Aforementioned, the current algorithms can infer behaviour through the "Likes" on Facebook (Kosinski et al. 2012) the problem arises when the cognitive biases and unconscious preferences are no longer under the private domain and are utilised to manipulate. 

The relatively new micro-targeting strategy in Advertising is a predictive cluster analysis which professedly sends tailored messages to the subcategories of the targets with unique information. It is believed that this message created with a specific language and tone, penetrates the unconscious preferences resulting in nudging behaviour (Thaler et al. 2010) and incidentally affecting the emotional state of third parties. (Kramer et al. 2014).

Literature Review

Decision-making literature suggests that, as a consequence of limited processing capacity, preferences often occur at the instant of a decision task rather than retrieving from memory a preferred choice (Bettman et al. 1998). Moreover, recent studies suggest that unconscious preferences happen first and could be predicted even 10 seconds before the conscious preferences can be articulated. (Soon et al. 2008). 

Numerous theories suggest that cognitive processes can be distinguished in System 1 and System 2 (Stanovich et al. 2000). System 1 is located on the right side of the brain, is fast, automatic, effortless, always in gear; it is almost a reflex known as the intuition. On the contrary, System 2  is slow, deliberate, logical, and rational, located on the left side of the brain; it demands effort and concentration to function, reason enough to believe that people are more reluctant to use it. (Kahneman 2003). 

When System 1 outperforms without engaging System 2, the automatic response could lead to a systematic irrationality in human cognition (Stanovich et al. 2000) called cognitive biases. Conversely to the belief that cognitive biases lead to irrational behaviour, Herbert Simon remarks that the process is about satisfice versus optimise asserting that neither result conduct to error. Optimise only enters at play when experience has been practised. Before being experienced, the level of the threshold in the decision-making process is vaguely set, and when it has been reached, then the process has been satisfied (Simon 1955). For example, to select a partner, there are several stages which need to be assisted by experience. Perhaps, the decision made to choose the first partner reached the threshold because the experience was not yet acquired in that field, then it can be said that the process was satisfied. As years passed by, optimisation entered into play suggesting that experience informed the decision-making process, hence, the following partner was not just above the initial threshold but possibly had characteristics which perhaps were not even known at initial consideration. Therefore, it can be concluded that ‘heuristics are efficient cognitive processes that ignore information (Gigerenzer et al. 2009) in which under uncertain environments provide a fast and automatic response. 

To understand and utilise cognitive biases mechanisms to enhance human behaviour and augment intrapersonal intelligence, first there has to be a clear understanding of their processes. For this experiment, significant qualitative and quantitative data was gathered to analyse and process how biases affect people’s decisions. To describe the different methodologies utilised to acquire that data, they have to be divided into two main sections: 

  • Conscious experiments

  • Unconscious experiments 

The ‘conscious experiments’ were carried during eighteen months and were physical transformations of the almost 200 cognitive biases in different shapes, colours and geometrical forms tested with a vast amount of people in different contexts and across cultures and countries. 

On the other hand, by the time this report was written, the ongoing ‘unconscious experiments’ have tested so far the brain activity of 10 participants while listening to preferred and nonpreferred songs from the same genre. The central question to be answered by the ‘unconscious experiment’ is what if the brain activity for either, the preferred or non-preferred music could set the parameters when an unconscious preference is happening in System 1?  For this research, ERP (Event-Related Potential) component was utilised to measure the electrical change in brain activity due to a behavioural or cognitive response (Nieuwenhuis et al. 2005). It is likely that the ERP in EEG could capture System 1 decision-making activity (Chen et al. 2010).

One of the associations made in this experiment that has to be clarified. The connection of unconscious preferences with music. It has been said that System 1 is responsible for the automatic beliefs and unconscious choices, and it is believed to always be in gear. Moreover, the unconscious bias mechanism enters at play when System 2 is not connected while making a decision (Kahneman 2003). Consequently, this experiment intended to maintain participants using System 1 without engaging System 2, one of many ways of achieving this, is by keeping the participant away from evaluating economic variables or rational assessments (Kushaba et al. 2015). 

Methodology 

The experiment was designed to test the brain activity of one participant at a time while listening to three preferred and three non-preferred songs from the same genre previously selected and shared by each participant with the experimenter. For this research, ERP (Event-Related Potential) is analysed through a change in brain activity when a preferred or nonpreferred song has listened to. 

The experiment took place in a quiet and white room to avoid distractions. The felt pads contained in the headset were moistened with saline solution to improve the signal gathered from the brain, and an interface guided the participant to ensure a correct position of the headset in the different channels on the scalp; also a baseline was recorded with open and closed eyes. 



EEG data collection

The tools used to access brain activity is a brain-computer interface (BCI) from the brand Emotiv called EPOC (Fig. 1). This device is a high resolution 14 channel located at the positions AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8 and AF4 following the international 10-20 system (Fig. 1). 

Fig.1.png

The recordings were at 128 Hz sampling frequency using a 2.4GHz band. The access and record of the raw data were performed with the Emotiv Software Development Kit (SDK) which provides a data packet feature to safeguard the loss of data while being transferred. 

The baseline consisted of recording fifteen seconds of brain activity with eyes opened and 15 seconds with eyes closed. This was useful to later compare the data. 

Once finished the baseline, in the same recording the participants closed their eyes and sat in a relaxed position throughout the experiment which generally took around five to seven min. 

Emotiv Pro provides a marker feature, in which at the beginning of the preferred and non-preferred songs a mark was set for ease segmentation tasks. 

The songs were played for around forty sec each and in random order to ensure the participants’ unawareness and to give more fidelity to the test. 


EEG signal processing

One of the most crucial steps before analysing the data is to denoise it. For this experiment, since participants had their eyes closed throughout the test, eye blinks were unperceivable. Also, since the task was to listen to music without performing any movement, it could be argued that the muscle activity was kept to the minimum. 

For the preparation of the data, both baselines, with eyes opened and eyes closed were separated only keeping the baseline with eyes closed. Then, the songs were divided into individual files according to the time marks, resulting in seven files, three for preferred songs, three for non-preferred songs and a baseline with eyes closed. Later, using Jupyter Notebook, an open-source web application used to clean data, and statistical modelling and simulation in Matlab and Python, the data were compared. 

For this experiment, the Causal-Impact-Analysis-Demo GitHub repository was used. Same that constructs a Bayesian structural time-series model based on the Event-Related Potential to predict the counterfactual, how the response metric would have evolved after the intervention if the intervention had never occurred.


Results and Discussion  

The following graphs represent the initial results in which X-axis represents time in milliseconds (ms) while the Y-axis represents microVolts (µV), the electrical activity of the brain measured through the scalp. The different bands alpha, beta, gamma and delta were not in the particular interest for this experiment due to the nature of the ERP which looks for changes in the activity of the frequencies rather than their specific state. All graphs come in pairs because the baseline or top chart is constant throughout the experiment.


The initial data in (Fig. 2) compares the three charts with preferred songs to the baseline.

Fig.2.png

The data analysis in (Fig. 3) compares the baseline (top graph) to brain activity while listening to non-preferred music.

Fig.3.png

As noted in (Fig. 2 A) and (Fig. 2 B) around the first ten seconds of listening to the preferred song, specific channels like F3, AF3, FC5, T7 present higher peaks in activity. Contrary to (Fig. 3 A), (Fig.3 B) and (Fig. 3 C) which seem to show more stable activity, conceivably similar to the baseline activity. However; this difference is not as notable in (Fig. 2 C) indicating that either the song is not as preferred as the rest of the other preferred songs or due to variables regarding time and moisture in the felts. As time passes during the experiment, the body heat could evaporate the water contained in the felts reducing the accuracy of the readings. 

It can be inferred that (Fig. 2) shows perceivably different activity from (Fig. 3), suggesting that the ERP potentially happens while listening to preferred music and not while listening to non-preferred music. One notable difference is the variation in milliseconds throughout the graphs. This fluctuation could be due to the moment when the activity in the brain changed. Not all the spikes raised at the same time resulting in variations when plotting the data. 

The other variable that could have affected the results were the emotions felt while listening to the songs, resulting in pleasant music possibly reminding positive experiences triggering certain emotions in the participants hence, producing different brain activity. 

It can be concluded that results obtained from the Event-Related Potential suggest that there is a difference in the brain activity of the participants when a preferred song is heard contrary to the brain activity when the non-preferred song is playing. 

Going back to the central question; what if the brain activity for either, the preferred or non-preferred music could set the parameters when an unconscious preference is happening in System 1?  The data shows a strong correlation between the preferred music and peaks in brain activity. This correlation could be interpreted as the initial parameters when an unconscious preference is happening to, later being transferred to the design field. 


Conclusion 

Initial results are showing substantial differences in brain activity while listening to preferred music rather than non-preferred music, however; there is still more research to be done and the number of experiments to be performed since the number of participants is small. 

The next phase of the project will be focusing on making tangible the unconscious preferences while they take place in the brain to afterwards, design the embodiment of those intangibles that drive behaviour. In other words, show the cognitive biases mechanisms through a wearable.

These understandings, under the field of neuroscience, could set new parameters in the design field to incorporate valuable scientific data into the design process, furthermore, create designs that could enhance human behaviour and augment intrapersonal intelligence.

 

Bibliography

  1. Ariely, Dan, and Gregory S. Berns. "Neuromarketing: The Hope and Hype of Neuroimaging in Business." Nature Reviews Neuroscience 11, no. 4 (03, 2010): 284-92. doi:10.1038/nrn2795.

  2. Badcock, Nicholas A., Petroula Mousikou, Yatin Mahajan, Peter De Lissa, Johnson Thie, and Genevieve Mcarthur. "Validation of the Emotiv EPOC®EEG Gaming System for Measuring Research Quality Auditory ERPs." PeerJ 1 (02, 2013). doi:10.7717/peerj.38.

  3. Basalla. The Evolution of Technology (Cambridge Studies in the History of Science). Cambridge University Press, 1989.

  4. Bettman, James R., Mary Frances Luce, and John W. Payne. "Constructive Consumer Choice Processes." Journal of Consumer Research 25, no. 3 (12 1998): 187-217. doi:10.1086/209535.

  5. Chen, Youguo, Xiting Huang, Bo Yang, Todd Jackson, Chunhua Peng, Hong Yuan, and Chunxiang Liu. "An Event-related Potential Study of Temporal Information Encoding and Decision Making." NeuroReport 21, no. 2 (01 2010): 152-55. doi:10.1097/wnr.0b013e328335b4f7.

  6. "Correction for Kramer Et Al., Experimental Evidence of Massive-scale Emotional Contagion through Social Networks." Proceedings of the National Academy of Sciences 111, no. 29 (07, 2014): 10779. doi:10.1073/pnas.1412583111.

  7. Diehl, Trevor, Brian E. Weeks, and Homero Gil De Zúñiga. "Political Persuasion on Social Media: Tracing Direct and Indirect Effects of News Use and Social Interaction." New Media & Society 18, no. 9 (07, 2016): 1875-895. doi:10.1177/1461444815616224.

  8. Diehl, Trevor, Brian E. Weeks, and Homero Gil De Zúñiga. "Political Persuasion on Social Media: Tracing Direct and Indirect Effects of News Use and Social Interaction." New Media & Society 18, no. 9 (07, 2016): 1875-895. doi:10.1177/1461444815616224.

  9. Eagleman, David M. "How Does the Timing of Neural Signals Map onto the Timing of Perception?" Space and Time in Perception and Action: 216-31. doi:10.1017/cbo9780511750540.014.

  10. Gibson, James Jerome., and Edward Reed. Reasons for Realism: Selected Essays of James J. Gibson. Lawrence Erlbaum, 1982.

  11. Gigerenzer, Gerd, and Henry Brighton. "Homo Heuristicus: Why Biased Minds Make Better Inferences." Topics in Cognitive Science 1, no. 1 (01 2009): 107-43. doi:10.1111/j.1756-8765.2008.01006.x.

  12. Handy, Todd C. Event-related Potentials: A Methods Handbook. MIT Press, 2005.

  13. Hevner, Alan R., Christopher Davis, Rosann Webb Collins, and Grandon Gill. "A NeuroDesign Model for IS Research." Informing Science: The International Journal of an Emerging Transdiscipline 17 (2014). doi:10.28945/1956.

  14. Kahneman, Daniel. "A Perspective on Judgment and Choice: Mapping Bounded Rationality." American Psychologist 58, no. 9 (2003): 697-720. doi:10.1037/0003-066x.58.9.697.

  15. Khushaba, Rami N., Luke Greenacre, Ali Al-Timemy, and Adel Al-Jumaily. "Event-related Potentials of Consumer Preferences." Procedia Computer Science 76 (2015): 68-73. doi:10.1016/j.procs.2015.12.277.

  16. Kosinski, M., D. Stillwell, and T. Graepel. "Private Traits and Attributes Are Predictable from Digital Records of Human Behavior." Proceedings of the National Academy of Sciences 110, no. 15 (03, 2013): 5802-805. doi:10.1073/pnas.1218772110.

  17. Kramer, A. D. I., J. E. Guillory, and J. T. Hancock. "Experimental Evidence of Massive-scale Emotional Contagion through Social Networks." Proceedings of the National Academy of Sciences 111, no. 24 (06, 2014): 8788-790. doi:10.1073/pnas.1320040111.

  18. Nieuwenhuis, Sander, Gary Aston-Jones, and Jonathan D. Cohen. "Decision Making, the P3, and the Locus Coeruleus--norepinephrine System." Psychological Bulletin 131, no. 4 (2005): 510-32. doi:10.1037/0033-2909.131.4.510.

  19. Norman, Donald A. Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books, 2004.

  20. Simon, Herbert A. "A Behavioral Model of Rational Choice." The Quarterly Journal of Economics 69, no. 1 (02 1955): 99. doi:10.2307/1884852.

  21. Soon, Chun Siong, Marcel Brass, Hans-Jochen Heinze, and John-Dylan Haynes. "Unconscious Determinants of Free Decisions in the Human Brain." Nature Neuroscience 11, no. 5 (04, 2008): 543-45. doi:10.1038/nn.2112.

  22. Stanovich, Keith E., and Richard F. West. "Individual Differences in Reasoning: Implications for the Rationality Debate?" Behavioral and Brain Sciences 23, no. 5 (10 2000): 645-65. doi:10.1017/s0140525x00003435.

  23. Thaler, Richard H., Cass R. Sunstein, and John P. Balz. "Choice Architecture." SSRN Electronic Journal, 2010. doi:10.2139/ssrn.1583509.

  24. Zajonc, R. B. "Feeling and Thinking: Preferences Need No Inferences." American Psychologist 35, no. 2 (1980): 151-75. doi:10.1037//0003-066x.35.2.151.