Several disciplines and their scientific societies increasingly distance themselves from the use of bibliometric measures aimed at assessing the quality of research. In contrast, economists seems rather eager to define quantitative ways to evaluate single contributions, researchers or research institutions. The projects collected here aim at documenting how citations and simplistic bibliometric measures are a potentially misleading tool to evaluate research quality in economics and elsewhere, which nonetheless shape the development of academic fields.

In this context the work done in Linz and Yale focuses on the sub-Project “Citation Culture in Physics and Economics: Assessing Differences between Disciplines”, which aims to compare bibliometric trends in economics, physics and additional disciplines, to gain a better understanding of common trends and specific developments in the academic conversation. In doing so we use rough arguments from philosophy of science to motivate specific scientometric applications in order to gain an impression how well the discourses under scrutiny conform to the philosophical notion of an ideal scientific procedure.

To assess this degree of conformity, we develop various metrics such as

• The relative importance of intrinsic quality vs. prestige and reputation in influencing the citation behavior inside these scientific disciplines
   
• The degree of equal representation of competing hypotheses by analyzing the citation structures of various source papers (snowball sampling) in both disciplines.
   
• The question of path-dependency and intrinsic insight in scientific research by asking specifically for non-linearities in citation behavior. These non-linearities indicate a significant turn in what is deemed to be scientifically important and, hence, signify path-breaking developments and disciplinary change.
   
• The assessment of reactive behavior among journal editors and authors of academic papers by analyzing time-trends of citational behavior as well as journal rankings to spot reactive patterns within physics and economics.

Publications

Aistleitner Matthias, Fölker Marianne and Kapeller Jakob (2016): Die Macht der Wissenschaftsstatistik und die Entwicklung der Ökonomie, opens an external URL in a new window. Schmollers Jahrbuch, 135: 111-132. Also availvable as: ICAE Working Paper Nr. 33, opens a file.

Aistleitner Matthias, Kapeller Jakob and Steinerberger Stefan (2016): The Power of Scientometrics and the Development of Economics, opens a file in a new window. Working Paper Series, Nr. 46. ICAE Linz.

Kapeller Jakob, Steinerberger Stefan (2016): Emergent Patterns in Scientific Publishing: A Simulation Exercise, opens an external URL in a new window. Research Policy, 45 (10): 1945–1952.