Experimental Design *********** Lesson ================ .. article-info:: :avatar: UCLA_Suit.jpg :avatar-link: https://www.decisionneurosciencelab.com/elijahgalvan :author: Elijah Galvan :date: September 1, 2023 :read-time: 3 min read :class-container: sd-p-2 sd-outline-muted sd-rounded-1 Goal During this Stage --------------- To create an experimental paradigm where every potential answer to your research question has a distinguishable behavioral trend over the course of the experiment. How to Achieve this Goal -------------- 1. Generate Hypotheses - propose potential answers to your research question 2. Create a Task in which you can tell which answer to your research question is correct 3. Identify Unintended Behavioral Patterns in your task - could people switch between using one answer and using another answer in your task? 4. Designing the Experiment - can we use within subjects and should we add multiple :bdg-primary:`Conditions`? .. _Fehr & Schmidt, 1999: https://www.jstor.org/stable/2586885 .. _Güth, Huck, & Ockenfels, 1996: https://watermark.silverchair.com/ej0593.pdf?token=AQECAHi208BE49Ooan9kkhW_Ercy7Dm3ZL_9Cf3qfKAc485ysgAAArwwggK4BgkqhkiG9w0BBwagggKpMIICpQIBADCCAp4GCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQM0kS3Uly_ObRPbcUAAgEQgIICbzUl-Dos5GklEcqxw3kQB8LTHgDSppBeGwY9tARYQOfzMyZMtCD6GIZyKM1abYRWbpvvGPa-ijL1fTgY9pFGgsB0hEJy8llt36vmRyhDfYc6BXymuM41E5Ej6KLjAcLmPzem-lbO98lYsTM4fc6yYeOrKnSSoMOH17nWdsP5tOjj2AxgcQ1gDsrG5Zjlfajm2TJI5Q53NmyVIdE9CAN60Y1IaGpI3IRjw9V9m7aq-XRjVc5e10sitM7eBxLofayNPADkq7qychLu_KZSv6YPgWMGSaKmx9GOOy2j9t9QJdFv-56Nnqos1tQ4-s9AOA4U-SUvlQz6WDImyahqB7wZuRID2CBztVPPU8cPGxEAPvo92-IFs9h1VXB-oi-Yjvsf59ziuKg8456DIjaBxsSyZWE6zbrVyJ3Xhv26JSAml-3xflrr5mSzZ7J3qK6RxiGRKzI9LxAJvA6mCmjT0OdbybVm8Va6Y1tyVuLFSZhRKICHoFdSkze2HcKfDHOZQhtckF5OHuZfqlfjs5sJxZNnjz0l3r25iNq0sjat4VWa_us6NHqkobvSetsXAL_A8JdO7sHoJYfw0XA8PpWhcJ3ygxgt2H9fjkh0UnBqVgnewoEyzzHUNHRQXXE1wJACsixq3K6ZM_WcuXJhr2fjJLybjf0SXxFKrfRXkwGeBCX8F6lyuBX2uvlGfaebhbuOzbFoJ1-HHnRP8YW0rLF-ZMkqXvQNvZougiLpx9fya2PMnaCZdvir7HYOflbz-tb-9XYG7tgF3LisM1f1-900xZ-zzr6LaeKRMPTpGZEOC6-RfR8kjKUH8EXunTOPM7KQNDtq Tutorials ================ Tutorial 1 - van Baar, Chang, & Sanfey, 2019 ------------------- .. dropdown:: Generate Hypotheses Well, we already know that some people do not reciprocate trust in single-shot interactions so let's jot down ``Greed`` as our first proposed answer. 1. Greed However, most people do reciprocate trust so what explanations may we propose for this? Well, traditional thought on the matter has been in favor of the explanation provided by inequity-aversion (see `Fehr & Schmidt, 1999`_ for the initial formulation of this inequity-aversion utility model). In other words, people value being objectively fair which leads them to reciprocate trust in the context of the 1-shot Trust Game - let's add ``Inequity Aversion`` to our list. 2. Inequity Aversion However, we also know that when people do not behave in a way that is inequity-averse when they have the opportunity to appear fair while actually being selfish in the Ultimatum Game (`Güth, Huck, & Ockenfels, 1996`_). Thus, rather than actually being fair, it could be the case that people reciprocate trust because they want to avoid feeling guilty for betraying the Investor's trust. So, let's also add ``Guilt Aversion`` to our list as well. 3. Guilt Aversion. Nice, we have 3 explanations - can you think of any more? Well I can't and neither could the authors so let's proceed with these answers. .. dropdown:: Create a Task We are interested in reciprocity of trust - this tells us we should most likely use the Trust Game as our task. So we've checked the ``Identify a psychologically relevant task`` box. So, working under this logic, we should now identify if any of our plausible answers predict the same behavior in the Trust Game. Well, Greed is certainly distinct from the other 2 - greedy people don't reciprocate and the others do. However, Inequity Aversion and Guilt Aversion lead to quite similar predictions: Inequity Aversion leads to people giving around half, as does Guilt Aversion since Trustees generally believe that Investors expect to receive half of the multiplied investment back. So the question now is: *How do we make Inequity Aversion and Guilt Aversion have different patterns of behavior in the Trust Game?* Well clearly we have to create situations where returning half of the multiplied investment (predicted by Inequity Aversion) does not result in Investors believing that they received half of the multiplied investment back (Guilt Aversion). So now the solution is more apparent: we have to manipulate Investors' beliefs about how much the multiplied investment is and since the Investor determined the investment amount, we must manipulate their beliefs about the Multiplier such that it does not match the actual Multiplier. We'll tell the Investor that the Multiplier is 4: the Trustee will believe that they should expect to receive half of 4 times the what they invested (i.e 2 times what they invested). However, behind the Investor's back, we'll tell the Trustee that 1) the Multiplier is either 2, 4, or 6 and 2) the Investor always believes that the Multiplier is 4. So let's recap: * When the Multiplier is 2 the Trustee believes the Investor expects the Trustee to return 2 times what the Investor invested - all of the money that the Trustee has * When the Multiplier is 4 the Trustee believes the Investor expects the Trustee to return 2 times what the Investor invested - half of the money that the Trustee has * When the Multiplier is 6 the Trustee believes the Investor expects the Trustee to return 2 times what the Investor invested - a third of the money that the Trustee has In all situations, it is Inequity Averse to return half of the money that the Trustee has. However, when the Multiplier is 2 or 6, it is Guilt Averse to return all or a third of the money that the Trustee has. So we've checked the ``Hypothesized accounts produce distinct patterns of behavior`` box. .. dropdown:: Identify Unintended Behavioral Patterns So we've created a task that *would* elicit behaviorally distinct patterns for each of Greed, Inequity Averson, and Guilt Aversion if such differences indeed exist. However, we have to think critically now about any additional substantial behavioral differences which might arise. This might seem like a daunting task but we've already dismissed the possibility that there are other patterns of behavior *within* each condition. So what's left to consider is only other patterns of behavior *between* conditions, namely: * People switch between Inequity Aversion when the Mutliplier is 2 and Guilt Aversion when the Multiplier is 6 (choosing the norm which prescribes returning less) We already know that people are motivated by material self-interest and affirmation of their moral virtue so it seems reasonable to think that they would be Morally Opportunistic - behaving prosocially in whatever way is most convenient. It's also possible that people switch between Inequity Aversion when the Mutliplier is 6 and Guilt Aversion when the Multiplier is 2 (choosing the norm which prescribes returning more) or either of these and Greed, but it's not as plausible an explanation. Anyway, now that we've identified potentially unintended behavioral patterns we have to ask ourselves if these would be psychologically meaningful and interesting or psychologically meaningless and uninteresting. The answer here is clearly yes: if people's motive to reciprocate changes depending on the situation this is psychologically meaningful and therefore interesting. Thus, we will keep the current design as is. We will also keep in mind that Moral Opportunism seems a relatively likely behavioral pattern which is psychologically compelling and relates to the plausible answers we have identified for our question. .. dropdown:: Design the Experiment So we've accomplished our goal for creating a task and we're happy with it, but now we need to decide if we should use a within-subjects or between-subjects design. In all situations that I can conceive of, you should use a within-subjects design but let's think this out here for the sake of punctuality. We are seeing what motivates people to reciprocate trust by manipulating Investors' expectations, so does it make sense that we only expose them to one condition? If we only expose them to one condition, we cannot see how their reciprocation behavior changes as a function of the Investors' expectations relative to what is equitable, so no. This would certainly be pointless - we'll stick with a within-subjects design. Do you want to add any additional manipulations (i.e. pharmacological, affective, etc.)? The research question doesn't demand it in this instance so let's say no. However, at this stage if you did have such a manipulation, would you be more confident in your results if you varied this manipulation within-or-between-subjects? If you favor between-subjects, are the additional costs justified? So now how many trials? This one's a tough one to answer and I honestly can't give a compelling, statistically well-founded answer. More is always better until it isn't, meaning it's always good to have as many trials as you can while having the subject still engaged in your task. Here, they opted for 80 which is quite a lot of trials. They could have certainly done this with 60 trials (they had 40 trials with the Multiplier as 4 and 20 each with the Multiplier as 2 and 6). Generally, you can feel quite safe with 20 trials per condition but, having worked with the data, I can say that they could have been fine with as few as 10 trials per condition though I will only recommend a minimum of 20 per condition. Tutorial 2 - Galvan & Sanfey, 2024 ------------------- .. dropdown:: Generate Hypotheses We think that there are three norms which characterize how benefit from redistribution can (or not) play a role in redistribution decisions. The first would be principled support for redistribution - irrespective of whether you gain or lose from redistribution you support it. 1. Equality-seeking Next would be the inverse, or principled opposition to redistribution - irrespective of whether you gain or lose from redistribution you oppose it. 2. Equity-seeking Last would be self-interest - you support redistribution when it benefits you and oppose it when you lose from it. 3. Payout-Maximization These are 3 explanations and there is a possibility of a fourth wherein one opposes redistribution when it benefits them and supports it when it hurts them. However, let's exclude this for now: previous research gives us no reason to think that people would do this. We'll keep this fourth norm in mind in the event that we find ourselves missing explanatory power. .. dropdown:: Create a Task We are looking a (re-)distributive norms so we should start from a task that is used to study the distribution of resources: the Dictator Game. However, in the standard Dictator Game, the Dictator makes a distribution decision, not a redistribution decision. So, we need to manipulate the task so the resources are already distributed and the Dictator has to change this redistribution. Also, we have aspirations to study societal redistribution, not interpersonal redistribution: thus, the 2 player design of the Dictator Game is inappropriate and we should have more people. In order rectify both of these shortcomings, we'll propose the Redistribution Game. The Redistribution Game will be played by 10 people, with 100 tokens distributed between these people. The Dictator will get the chance to redistribute this money: now, we need to determine how to do this. Either the Dictator can move money between players directly or the Dictator can apply something which resembles a uniform tax rate to determine redistribution. The second option is much simpler for Dictators to do, so let's go with that - nice, our task is designed! So we've checked the ``Identify a psychologically relevant task`` box. So, working under this logic, we should now identify if any of our plausible answers predict the same behavior in the Redistribution Game. The answer for *Equity-seeking* and *Equality-seeking* is clearly yes: both norms indicate that you should select a certain tax rate (0% or 100%). So what about *Payout-Maximization*? This is a bit trickier - we have to ensure that Dictator's economic status changes across :bdg-primary:`Trials` - in some trials they have to stand to benefit and in others they have to stand to lose. So, we'll design our :bdg-primary:`Trial Set` around this - since the number of tokens is always 100, we'll give the :bdg-success:`Subject` a different number of tokens each time. So we've checked the ``Hypothesized accounts produce distinct patterns of behavior`` box. Here, we're going to have to be careful to ensure that our :bdg-primary:`Trial Set` doesn't tip :bdg-success:`Subjects` off - there should be a lot of ways that 100 tokens can be divided between 10 people. Thus, if there are always the same number of tokens allocated to each other player, our task won't be believable. We'll keep that in mind for later. .. dropdown:: Identify Unintended Behavioral Patterns Couldn't identify any unintended behavior. .. dropdown:: Design the Experiment We'll want a within-subjects design. What about manipulations? Well, previous research suggests that the (perceived) cause of inequality is really a powerful factor in determining redistribution preferences. So, if we allocate the 100 tokens based on different attributes, we can study this! Let's do that - we'll study how redistributive preferences change across :bdg-primary:`Conditions` as well, where these are the cause of inequality. Our causes of inequality will be merit (earned advantages), entitlement (unearned advantages), corruption (stolen advantages), and luck. These should be manipulated within-subjects as well. We'll do 20 trials per :bdg-primary:`Condition`, following the lead of tutorial 1. Tutorial 3 - Crockett et al., 2014 ------------------- .. dropdown:: Generate Hypotheses 1. People are more willing to harm others for money than themselves 2. People are more willing to harm themselves for money than others 3. People are equally willing to harm themselves and others .. dropdown:: Create a Task We are interested in a task which concerns the distribution of resources: namely harm and money. We'll operationalize harm in terms of shocks - these can be ethically administered and cause discomfort that (nearly all) subjects will be very averse to. To study this distribution decision, we should consider using the Dictator Game. Here, we might think give subjects and endowment of money and shocks that need to be distributed between themselves and the other person. However, such a design would not let us tap into a willingness-to-harm tradeoff since it requires that a fixed amount of money and shocks be distributed. We want there to be an explicit tradeoff: more shocks equals more money - the simplest way to do this would be to give subjects a choice between A) less money, less shocks and B) more money, more shocks. Thus, we'll vary the recipient of the shocks across conditions - sometimes the subject receives the shocks and sometimes someone else does. Across trials, we'll vary how much more shocks and money option B has. Importantly, the subject will always receive the money and they will always make the choice to subject either themselves or someone else to the shocks. In the original paper, a default was given (i.e. participants had to opt out of the choice). However, in the dataset that we are using, no default is given. .. dropdown:: Identify Unintended Behavioral Patterns Our hypotheses capture all of the possibile behavioral patterns across conditions. Either harm-aversion for oneself is greater-than, less-than, or equal to harm-aversion for others. .. Note:: In the original study, a default value (i.e. the decision which will be made unless the subject chooses the alternative) was assigned. Herein, the authors introduce some important psychological factors that must be accounted for. Namely, the endowment effect: if we're giving someone a package of shocks and money that will receive unless they choose to select the alternative, they may feel some ownership over this package of shocks and money. Thus, choosing the alternative may induce the feeling of losing out on whatever is better about the default: losing out on fewer shocks or more money. Therefore, we would need to account for potential loss-aversion effects in the model if we had this dataset - however, no default is given in the current study. .. dropdown:: Design the Experiment We'll want a within-subjects design - varying the recipient of the shocks across :bdg-primary:`Conditions`. Tutorial 4 - Li et al., 2022 ------------------- .. dropdown:: Generate Hypotheses 1. People prefer more equal allocations 2. People prefer to avoid making people lose money 3. People prefer to maintain existing hierarchies .. dropdown:: Create a Task Our research question asks about three norms in distributing resources: for instance, money. None of these norms necessarily considers self-interest, so we can best focus on answering this research question by having participants make distribution decisions for others. So, we are looking for variant of a distribution task which doesn't involve the participant, which means we should use a third-person Dictator Game as a starting point. Now we need to decide if we're going to be using a slider (multiple choices) or a binary choice paradigm. If there is a fixed amount of money in the game, then we can get a more accurate account of preferences using a slider. We can reasonably assume that we can find the tradeoff between equality and harm aversion. However, we likely will not see many people spontaneously inverting the existing hierarchy. Thus, we need to force people to chose between maintaining hierarchy and lessening inequality: putting these norms into conflict will require a limited choice set, so using a binary choice task is best. .. dropdown:: Identify Unintended Behavioral Patterns Couldn't identify any unintended behavior. .. dropdown:: Design the Experiment We'll want a within-subjects design.