On the external validity of construction bidding experiment

Bee Lan Oo

University of New South Wales, Australia

Abstract

The external validity of experimental studies and in particular, the subject pool effects have been

much debated among researchers. The common objections are that the use of student as experimental subjects is invalid as they are likely to be unrepresentative. This paper addresses this methodological aspect in building economics research. It compares the bidding behavioural

patterns of experienced construction executives (professionals) and student subjects through replication of a bidding experiment that aimed at testing theories. The results show that the student subjectsrsquo; bidding behavourial patterns, in terms of decision to bid and mark-up decision,are sufficiently similar to that of the professionals. This suggests that the subject pool per se is nota threat to the external validity of the bidding experiment. In addition, the demonstrated practicality of an experimental approach in testing theories should lead to more use of experimental studies with student subjects in building economics research. It is suggested that experimental and field findings should be seen as complementary in building economics research,as advocated in social sciences.

Keywords: Construction bidding, experiment, external validity, methodological issue.

Paper type: Research article

IntroductionSince the critiques made by Runeson back in 1997 on the importance of theory in construction management and economics research (1997a), and on the slow progress in its methodological component - in the sense of using what is appropriate from related, well-established disciplines (1997b), it seems that there has been little progress on these aspects in the discipline over the last two decades. This is evidenced in his most recent review (Runeson and de Valence 2015) in the Construction Management and Economics journal with comments on poor standard of current research in construction or building economics research. In their review that focused on research on tendering theory and innovations in construction, they cannot emphasize enough the importance of using tried and tested theories and methodologies in progressing science in the discipline, with one of the key advantages being that appropriate research methods would have been established. This study focuses on the methodological aspect in research on construction bidding, which is conventionally referred to as part of building economics. Specifically, it examines the external validity of bidding experiments where students were used in place of professionals to establish behavioural patterns. This is a commonly used research method in social sciences although there are very few examples in construction management.

Hence, while there is no study specifically on the external validity of experimental studies in building economics, the problem has been much debated in the social sciences of whichconstruction management and economics research are parts. By external validity, I refer to the ability to generalize results (or behavioural observations) from the laboratory to the non- laboratory environments (typically called field or real world), i.e., the problem on generalizability (Campbell and Stanley 1966). Specifically, it is the ability of a causal relation x = f(y) to be generalized over subjects and environments (Frechette 2015). Alm, Bloomquist and McKee (2015) referred the subjects and environments as subject pool and context effects, respectively. In terms of subject pool effects, a common objection among construction management researchers, sometimes echoed by other social scientists is that the use of student as experimental subjects is invalid as they are unlikely to be representative of the population that is tested (Falk and Heckman 2009). Falk and Heckman (2009) have examined the five most commonly mentioned issues related to subject pool effects, namely: (i) stakes or monetary rewards in experiments are trivial; (ii) the number of subjects is too small; (iii) subjects are inexperienced; (iv) the possibility that subjects behave differently because they perceive that they are observed; and (v) the self-selection of subjects may bias results. For context effects, critics refer to the extent to which the context in the laboratory decision resembles the context in the field for the same decision (Alm, Bloomquist and McKee 2015). This study focusses on the subject pool effects with a very precise question: do student subjects behave different to nonstudent subjects in an identical construction bidding laboratory experiment? Nonstudent subjects here refer to construction executives with experience in bidding. To the authorrsquo;s knowledge, there are no studies that have compared the behavioural patterns in construction bidding between student and nonstudent subjects in an experimental setting. However, the study on auction theory by Dyer et al. (1989) may be considered as the closest work that compared the bidding decisions between construction executives and students in common value auctions. In answering the question, this study proposes a direct replication of the bidding experiment in Oo (2007) by replacing its nonstudent construction executives (professionals) subjects with student subjects. Replication, despite its unpopularity in construction management and economics research, has been seen as a key self-correcting force (together with peer review) towards ensuring a high standard in research in the discipline (Runeson and de Valence 2015). Here, the findings are important for two reasons: (i) because they may validate the use of an experimental approach in testing theory, and (ii) because they test the applicability of the use of students as experimental subjects. Notably, undergraduate students are typical subjects in social science research using exp

全文共34279字，剩余内容已隐藏，支付完成后下载完整资料

On the external validity of construction bidding experiment

Bee Lan Oo

University of New South Wales, Australia

Abstract

The external validity of experimental studies and in particular, the subject pool effects have been much debated among researchers. The common objections are that the use of student as experimental subjects is invalid as they are likely to be unrepresentative. This paper addresses this methodological aspect in building economics research. It compares the bidding behavioural patterns of experienced construction executives (professionals) and student subjects through replication of a bidding experiment that aimed at testing theories. The results show that the student subjectsrsquo; bidding behavourial patterns, in terms of decision to bid and mark-up decision,are sufficiently similar to that of the professionals. This suggests that the subject pool per se is nota threat to the external validity of the bidding experiment. In addition, the demonstrated practicality of an experimental approach in testing theories should lead to more use of experimental studies with student subjects in building economics research. It is suggested that experimental and field findings should be seen as complementary in building economics research,as advocated in social sciences.

Keywords: Construction bidding, experiment, external validity, methodological issue.

Paper type: Research article

IntroductionSince the critiques made by Runeson back in 1997 on the importance of theory in construction management and economics research (1997a), and on the slow progress in its methodological component - in the sense of using what is appropriate from related, well-established disciplines (1997b), it seems that there has been little progress on these aspects in the discipline over the last two decades. This is evidenced in his most recent review (Runeson and de Valence 2015) in the Construction Management and Economics journal with comments on poor standard of current research in construction or building economics research. In their review that focused on research on tendering theory and innovations in construction, they cannot emphasize enough the importance of using tried and tested theories and methodologies in progressing science in the discipline, with one of the key advantages being that appropriate research methods would have been established. This study focuses on the methodological aspect in research on construction bidding, which is conventionally referred to as part of building economics. Specifically, it examines the external validity of bidding experiments where students were used in place of professionals to establish behavioural patterns. This is a commonly used research method in social sciences although there are very few examples in construction management.

Hence, while there is no study specifically on the external validity of experimental studies in building economics, the problem has been much debated in the social sciences of whichconstruction management and economics research are parts. By external validity, I refer to the ability to generalize results (or behavioural observations) from the laboratory to the non- laboratory environments (typically called field or real world), i.e., the problem on generalizability (Campbell and Stanley 1966). Specifically, it is the ability of a causal relation x = f(y) to be generalized over subjects and environments (Frechette 2015). Alm, Bloomquist and McKee (2015) referred the subjects and environments as subject pool and context effects, respectively. In terms of subject pool effects, a common objection among construction management researchers, sometimes echoed by other social scientists is that the use of student as experimental subjects is invalid as they are unlikely to be representative of the population that is tested (Falk and Heckman 2009). Falk and Heckman (2009) have examined the five most commonly mentioned issues related to subject pool effects, namely: (i) stakes or monetary rewards in experiments are trivial; (ii) the number of subjects is too small; (iii) subjects are inexperienced; (iv) the possibility that subjects behave differently because they perceive that they are observed; and (v) the self-selection of subjects may bias results. For context effects, critics refer to the extent to which the context in the laboratory decision resembles the context in the field for the same decision (Alm, Bloomquist and McKee 2015). This study focusses on the subject pool effects with a very precise question: do student subjects behave different to nonstudent subjects in an identical construction bidding laboratory experiment? Nonstudent subjects here refer to construction executives with experience in bidding. To the authorrsquo;s knowledge, there are no studies that have compared the behavioural patterns in construction bidding between student and nonstudent subjects in an experimental setting. However, the study on auction theory by Dyer et al. (1989) may be considered as the closest work that compared the bidding decisions between construction executives and students in common value auctions. In answering the question, this study proposes a direct replication of the bidding experiment in Oo (2007) by replacing its nonstudent construction executives (professionals) subjects with student subjects. Replication, despite its unpopularity in construction management and economics research, has been seen as a key self-correcting force (together with peer review) towards ensuring a high standard in research in the discipline (Runeson and de Valence 2015). Here, the findings are important for two reasons: (

全文共44045字，剩余内容已隐藏，支付完成后下载完整资料

资料编号：[142810]，资料为PDF文档或Word文档，PDF文档可免费转换为Word