A blog on statistics, methods, and open science. Understanding 20% of statistics will improve 80% of your inferences.

Saturday, May 5, 2018

Prestige and Punishment: Preventing Norm Violations in Science


When the scientific discipline started to emerge, Bacon (1620) remarked: “And it is nothing strange if a thing not held in honour does not prosper”. His idea was that we need to give honour to scientists who do good work, otherwise no one would want to become a scientist. In the current academic climate, prestige works as a double-edged sword: It can be useful as a source of extrinsic motivation, but it can also tempt researchers to violate scientific norms that, as long as the norm violations go unnoticed, increase prestige (McPherson, 1994). One attractive feature of using prestige as a reward mechanism in science is that researchers who aim to gain prestige value their reputation. A good reputation is maintained by not violating scientific norms, and a good reputation is lost when norm violations are discovered. Therefore, for researchers interested in gaining prestige, the goal to maintain a good reputation provides a selfish reason not to violate scientific norms (Milinski, Semmann, & Krambeck, 2002).

As Partha and David (1994) discuss in their work on the economics of science, the loss of reputation can be seen as a form of punishment for people who violated scientific norms, with the goal to maintain long term cooperation within the scientific enterprise (and prevent ‘defection’, or norm violations). In their view, science can be seen as a social dilemma, where the trade-off is to do what is good for the collective, or what is good for yourself. These two goals are not always aligned. Partha and David note how punishment in science often consists of ostracism, or “exclusion from the circle of cooperators in the future’, after norm violations are made public. For example, after a norm violation, colleagues might no longer want to work with you.

There has been a lot of discussion about what constitutes a norm violation in science, how researchers should act when they realize they have violated a norm, and the desirability of pointing out norm violations in public. To me, it seems that if we accept a system that rewards individuals through prestige, we also need to accept a system that leads to suffering and distress when individuals lose prestige. We will inevitably see differences in which (if any) violations people think deserve to be punished by loss of reputation. A reward system based on prestige does not, by definition, lend itself to exact quantification. People do not receive prestige proportional to their contributions to science, and there is no process that guarantees that the loss of prestige after a norm violation is proportional to the severity of the norm violation. This discussion is complicated even more by the fact that when norm violations are followed by ostracism, we should not only expect a loss of prestige, but also strong personal distress (the meta-analytic effect size of negative effects due to being ostracized is d = -1.4, which is one of the biggest effects in social psychology).

Using punishment to prevent the violation of scientific norms is an inherently messy mechanism. It is difficult, if not impossible, to detach prestige from subjective feelings. It seems impossible to contain punishment of perceived norm violations purely to a reduction in prestige, even if one wanted to. I personally believe that as long as we have a system that confers individual prestige on the basis of scientific accomplishments, we also opt-in to a system that requires punishing researchers who have gained their prestige by violating scientific norms. If we choose to prevent scientific norm violations through punishment and ostracism, and information about norm violations can now be much more widely shared than before through social media, the field needs to come together to more clearly define norm violations, and reasonable sanctions for specific norm violations.

A recent case illustrates this point well. Robert Sternberg recently resigned as editor of Perspectives on Psychological Science. He self-plagiarized, and excessively cited his own work. In a system that rewards scientists with prestige based on their performance, it seems necessary to incorporate information about self-plagiarism and self-citation into our judgment of how much prestige Sternberg should get. Whether his behavior is a scientific norm violation is a matter of debate. In the Netherlands cases of perceived scientific norm violations are transparently dealt with by the LOWI (unlike countries such as the USA where perceived scientific norm violations are intentionally hidden from public view and rarely, if ever, dealt with in a transparent manner). A very similar case in economics, where Peter Nijkamp (like Sternberg, ranked in the Top 100 of his field) excessively self-plagiarized was seen as a questionable research practice, and careless, but not as a breach of scientific integrity. Note that even if excessive self-plagiarism is not officially a breach of scientific integrity, fellow scientists can still perceive this as a norm violation, and ostracize researchers who act in this manner.

The email below (which despite being written so badly it reads like a spoof email, seems to be a real email by a real lawyer working for Sternberg) highlights the negative affective consequences of this punishment process for the people who lose prestige because of perceived scientific norm violations. This is just one example, but many high-profile cases where researchers have lost prestige due to perceived norm violations will lead to experiences of “intentional efforts to inflict emotional harm” on behalf of the researchers who have received criticism.



I am merely observing this weird situation we have gotten ourselves into. Because we collectively accept a system that rewards individual scientists through prestige, I can feel both sympathy for people who experience negative affect when their reputation suffers, as indignation when they sent lawyers after people who publicly share perceived norm violations. I don’t see a solution as long as we have a scientific system that rewards individuals through prestige. Allow me to self-plagiarize: If we accept a system that rewards individuals through prestige, we also need to accept a system that leads to suffering and distress when these individuals lose prestige.

If we care enough about this problem to try to solve it, we might have to seriously reconsider the role prestige plays in science.




References



McPherson, M. S. (1994). Part three. How should liberal education be financed. Public purpose and public accountability in liberal education. New Directions for Higher Education, 1994(85), 81–92. https://doi.org/10.1002/he.36919948512

Milinski, M., Semmann, D., & Krambeck, H.-J. (2002). Reputation helps solve the ‘tragedy of the commons.’ Nature, 415(6870), 424. https://doi.org/10.1038/415424a

Partha, D., & David, P. A. (1994). Toward a new economics of science. Research Policy, 23(5), 487–521. https://doi.org/10.1016/0048-7333(94)01002-1

2 comments:

  1. "He self-plagiarized, and excessively cited his own work. "

    The real problem might be that the no. of publications and citations are used as some sort of metric for quality, used for hiring and promoting researchers, etc. I fear nothing will be solved when this continues to be the case. There is nothing wrong with self-citations in and of itself i reason. And when you all start acting like it is wrong, then people who want to manipulate that will simply ask their friends if they would cite them (as has probably already been happening a lot over the past decades, but let's all not think of that)

    I fear nothing will be solved changing one Sternberg for another (version of an editor). This Sternberg dude may have been clumsly in his antics, but you don't really think he's the only editor who does bad stuff.

    Journals, editors, and peer-review are a joke and can possibly be viewed as being anti-scientific in and of itself, and the cause for many, if not the majority, of all the problematic issues in science today.

    But please, you all keep actively participating in this sh#tshow, and be proud of the fact that you all wrote a letter and got really angry about this Sternberg dude. Congrats, well done!

    ReplyDelete