SciELO - Scientific Electronic Library Online

vol.14 número3Altruism and Internal Locus of Control as Determinants of the Intention to Participate in Crowdfunding: The Mediating Role of TrustSocio-Technical Factors in Electronic Software Biddings índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados




Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google


Journal of theoretical and applied electronic commerce research

versión On-line ISSN 0718-1876

J. theor. appl. electron. commer. res. vol.14 no.3 Talca set. 2019 


Understanding Solvers' Continuance Intention in Crowdsourcing Contest Platform: An Extension of Expectation-Confirmation Model

Meng-Meng Wang1  

Jian-Jun Wang2  

1 Dalian University of Technology, Faculty of Management and Economics, Dalian, China,

2 Dalian University of Technology, Faculty of Management and Economics, Dalian, China,


The great potential of crowdsourcing contest is bringing the issue of how to sustain solvers’ participation intention to be a hot topic in research and practice. This study uses the framework of Expectation-confirmation model to explain solvers’ continuance intention. Due to the uncertainties inherent in crowdsourcing contest, trust, a salient psychological belief, should be taken into account with the Expectation-confirmation model framework to predict solvers’ continuance intention. In addition, the intensive demand of intelligence and competition indicate interaction and fairness as two crucial factors for solvers to achieve expectation, thus suggesting that they may have influence on the confirmation level. Corresponding to these challenges, this study integrates platform trust, interaction, and perceived fairness into an extended Expectation-confirmation model to examine solvers’ continuance intention. Using a sample of 306 solvers, empirical results show that satisfaction, perceived benefits, and platform trust, which are positively associated with confirmation, are three significant antecedents of solvers’ continuance intention. Confirmation is further found to be significantly determined by interaction and perceived fairness. These findings provide some implications in both theory and practice for understanding the process of triggering sustained intention with an Expectation-confirmation model framework in crowdsourcing contest.

Keywords: Continuance intention; Platform trust; Interaction; Perceived fairness; Crowdsourcing contest; Expectation-confirmation model

1 Introduction

Advancement of information technology has made crowd intelligence possible as a new source of innovation and promoted the popularity of crowdsourcing contest platforms. When facing a problem or task, an organization or an individual, generally called a seeker, can post an open call on these platforms to solicit solutions from a diverse range of individuals (solvers) [1]. A reward is usually attached to attract solvers, but only the solver who generates the best solution wins the reward [35]. Crowdsourcing contest platforms are now widely used for a variety of tasks, including product innovation (e.g., Site 1), logo design (e.g., Site 2), software development (e.g., Site 3), and minor jobs (e.g., Site 4). Having the potential of accessing extensive knowledge and skills, these platforms are now enjoying fast development. For example, over 4 million tasks are posted on, a famous crowdsourcing contest platform in China.

Although crowdsourcing contest platforms have become popular in a variety of industries, they suffer from some challenges, such as how to maintain the attractiveness to solvers. Attracting solvers’ initial participation is an important first step toward realizing the success of a platform, however, its sustainability and eventual success heavily depend on solvers’ continuance participation. In fact, unfortunately many solvers discontinue or become inactive in crowdsourcing activities over time [49]. For example, [26] revealed that the number of active solvers declined over time on, a crowdsourcing community operated by Dell. Our analysis of shows that among all registered solvers approximately only 11.7% participate in crowdsourcing tasks more than once. Crowdsourcing scholars recently are realizing the significance of solvers’ continuance intention and are paying increasing attention to this topic [8], [48], [49]. But related studies are still limited and scarce, calling for more effort in this stream. Therefore, investigating factors that influence solvers’ continuance intention constitutes the objective of our study.

In the area of continuance intention research, expectation-confirmation model (ECM) has long been proved to be a useful framework [7] and has been widely modified to study technology adoption and continuance use [11], [27], [34]. Under the setting of crowdsourcing contest, solvers are generally initially motivated by utilitarian value (e.g., task reward) and hedonic value (e.g., enjoyment, self-challenge, skill improvement, etc.) to contribute their ideas or solutions for the task [31], [48]. However, solvers’ initial expectation can be changed or reshaped with first-hand experience to form post-perception which will substantially determine their future intention. The similar rationale between solvers’ continuance intention and users’ technology continuance intention suggests the great potential ability of ECM to predict solvers’ continuance intention. However, the characteristics of crowdsourcing contest are different from that of a technology product, which present some challenges to the potential predictive power of ECM framework.

First, ECM conceptualizes perceived usefulness to represent the ex post expectation about the levels of attributes possessed by a technology product [7]. In the crowdsourcing contest context, solvers are generally viewed as participators (not consumers) of an internet-mediated contest, who expect to receive the utilitarian value or hedonic value from their participation behavior [49]. Thus, solvers’ overall assessment of their behavior is much more likely based on the perception of what is received, which can be reflected by the term perceived benefits [34]. Second, ex post expectation in the original ECM framework only focuses on the product material attributes or quality. However, individuals’ perceptions are not necessarily restricted to the material or performance aspect [52]. Under the unrestricted, anonymous environment of crowdsourcing contest, solvers additionally have a great demand on psychological factors, such as trust belief [2]. In particular, solvers are indeed facing some uncertainties or risks, such as fraud information and labor squeeze. As some studies indicate, lacking trust belief has become a severe barrier blocking solvers’ participation behavior [37], [50]. Hence, extending the ECM framework by incorporating the trust belief toward the platform as an additional post-expectation may be imperative and valuable to explain solvers’ continuance intention. Third, crowdsourcing contest is essentially an online task-based tournament, thus the extent to which solvers fully understand the task needs and generate solutions that the seeker find valuable enough and the extent to which the tournament is fair may have significant role in helping confirm solvers’ expectation. A high level of interaction can facilitate the communication between solvers and seekers to help solvers develop a better understanding of task needs. Also, high levels of interaction between solvers and seekers are helpful for breaking cognitive fixation (which occurs when a person is unable to solve a problem in a creative way but continues to follow the traditional way) to generate creative solutions [6], [42]. On the other hand, perceived fairness is a crucial factor when evaluating the extent to which the crowdsourcing contest is fair. Solvers have to invest effort into their solutions which are used to compete for the reward, and consequently solvers have an inner tendency to compare their inputs and outputs [22]. When solvers perceive they are unfairly treated, they tend to generate doubts about procedures and outcomes, leading to a reduced confirmation level. Therefore, interaction and fairness perception may have crucial effects in confirming solvers’ expectation level. Some studies also argued that when studying continuance intention under the lens of ECM, it would be more valuable to understand the antecedent of confirmation for its central role in an ECM framework [11]. Thus, this study further tries to broaden the ECM framework by explicitly examining the effect of interaction and perceived fairness on confirmation level. In sum, considering the effectiveness of ECM and these challenges, this study extends ECM to study solvers’ continuance intention by incorporating some unique features.

Using a sample of 306 solvers obtained from a crowdsourcing contest platform in China, the extended ECM model is tested and confirmed. Theoretically, this study contributes to the crowdsourcing research by examining an extended ECM to explain the process in which solvers’ participation intention can be sustained and nurtured. In addition, our results enrich the research stream of ECM by uncovering two significant antecedents of confirmation and by incorporating platform trust as an additional ex post expectation, which, in turn, has positive effect on satisfaction and continuance intention. These results can also provide insights to crowdsourcing practitioners regarding the factors that can retain active and regular solvers and turn infrequent solvers into committed ones.

The rest of the paper is structured as follows. In section 2, we propose the research model and hypotheses based on a discussion of the theoretical background. We then describe the research methodology and present data analysis and results in Section 3 and 4 respectively. Next, we discuss the research, implications in theory and practice, and research limitations and future research in Section 5. Finally, Section 6 closes this paper with a conclusion of the research.

2 Theoretical Background and Hypotheses Development

In this section, the Expectation-confirmation model (ECM) is introduced to serve as our theoretical basis. Based on ECM and the features of crowdsourcing contests, we incorporate platform trust, interaction, and perceived fairness to develop a research model for solvers’ continuance intention.

2.1 Expectation-Confirmation Model

Expectation-confirmation model (ECM) has long been adopted to explain continuance intention in a variety of contexts. It stems from expectation-confirmation theory (ECT) [41] and is developed by Bhattacherjee to understand users’ continued IS usage intention [7].

ECT posits that consumers’ satisfaction and repurchase intention are determined by two major constructs: initial expectation on a product or service and the confirmation level. According to the framework of ECT, consumers have an initial expectation about the product or service prior to purchase. They will build perceptions about the performance based on their consumption experience. By assessing perceived performance against initial expectation, consumes form their confirmation level. Upon the confirmation level and expectation on which that confirmation is based, consumers form satisfaction, which in turn determines repurchase intention.

Drawing on the ECT framework and the substantial differences between initial adoption and continued intention, [7] developed and empirically tested an Expectation-Confirmation Model (ECM) for continued IS usage intention. ECM focuses only on post-acceptance variables instead of examining both pre-consumption and post-consumption variables. This is because [7] argued that the effects of any pre-acceptance variables were already captured within the confirmation and satisfaction constructs. In contrast with pre-acceptance expectation, post-acceptance expectation is typically based on users’ first-hand experience and is, therefore, more realistic [19]. Once updated during the usage process, post-acceptance expectation will replace initial expectation in users’ cognitive memory as the basis for guiding subsequent decision processes. Hence, ECM amends ECT to include ex post expectation. The rationale of this framework is as follows. When using a product, users assess perceived performance with their initial expectation and determine the extent to which their expectation is confirmed (confirmation). Meanwhile, following their usage experience and confirmation level, users form a post-acceptance expectation (perceived usefulness) which may be different from their pre-acceptance or initial expectation. Their confirmation level and post-acceptance expectation lead to form satisfaction. Finally, satisfied users would like to develop a continuance intention. Figure 1 illustrates the process by which users reach continuance intention in an ECM framework.

Figure 1: Expectation-confirmation model 

2.2 Hypotheses Development

In this section, we develop the arguments for the research model (Figure 2) and hypotheses.

2.2.1 Effect of Satisfaction

Satisfaction is referred to as an evaluative affect based on past experience [3], [7]. In crowdsourcing contest setting, a solver’s satisfaction with a crowdsourcing platform signals that he or she has a pleasant experience during the participation. Hence, more satisfied solvers are more willing to continue their participation. In contrast, solvers who are dissatisfied with their participation experience would lack impetus to keep their behavior. For example, [26] indicated that negative experience and dissatisfaction can eventually stop solvers’ contribution. The study of [7] has provided empirical evidence for the association between satisfaction and continuance intention in ECM framework. Indeed, a growing body of literature has widely reached the consensus that satisfaction is a determinant of individuals’ repeated or continuance intention, such as re-purchase products, patronize services, and sustained behavior in various communities [17], [18], [33], [34]. Thus, based on the discussion and prior studies, we propose the hypothesis.

H1: Satisfaction is positively related to continuance intention in crowdsourcing contest.

2.2.2 Effects of Perceived Benefits

As prior arguments state, perceived benefits herein are used to represent solvers’ ex post performance-expectation, including monetary reward, self-improvement, and skill development, etc. [31], [45]. Drawing on ECM framework, ex post expectation is a primary determinant of a high level of satisfaction. This suggests that high perceived benefits may be positively associated with satisfaction. The reason is that with a high level of perceived benefits, solvers have a great perception that they can gain benefits from participating in crowdsourcing contest. When solvers find their participation behavior more valuable, they tend to be more satisfied. On the contrary, negative affections, such as dissatisfaction would be experienced due to the low values associated with the participation behavior. Some studies based on different levels of analysis have empirically examined the impact of perceived benefits on satisfaction and have provided evidence for their relationship [29], [34]. Hence, it is reasonable to believe there is a positive link between perceived benefits and satisfaction in the context of crowdsourcing contest. Based on the discussion and prior studies, we propose the hypothesis.

H2: Perceived benefits are positively related to satisfaction in crowdsourcing contest.

According to self-determination theory, individuals tend to rationally evaluate the outcomes of their behaviors and then adjust their strategies and behaviors to receive more values [14]. Extant literature has indicated that individuals’ behavioral intention can be determined by perceived benefits [34]. In the setting of crowdsourcing contest, externalizing the incentive mechanisms in solvers’ priority has been suggested as an effective way to attract solvers’ participation attention [31], [35]. Hence, when solvers perceive it is a great place to obtain money reward, they tend to pay more effort to continually participate in crowdsourcing tasks [49]. In addition, the value associated with inner needs also shows attraction to solvers [31], [45]. Crowdsourcing contest provides a stage to achieve a sense of achievement through the competition with others. As [49] stated, solvers’ continuance behavior could be evoked when they realized that their participation behavior would satisfy their needs for competence. In sum, we suggest solvers are more likely to continue their participation intention when they perceive the associated benefits to be high. Based on the discussion and prior studies, we propose the hypothesis.

H3: Perceived benefits are positively related to continuance intention in crowdsourcing contest.

2.2.3 Effects of Platform Trust

In contrast to a technology product, the crowdsourcing contest platform has a social attribute enabling a solver interacts with the seeker and other solvers. However, under the virtual environment of crowdsourcing contest, the interaction is vulnerable to some potential uncertainties or risks, such as fraud information and labor squeeze. Thus, it is natural that besides the material or performance expectation, solvers have a demand on psychological expectation. As the intermediary between seekers and solvers, the extent to which the platform is trustworthy will play a significant effect on solvers’ continuance intention. A high level of platform trust implies the platform will behave in accordance with individuals’ expectation to facilitate their socialization. During participation process, solvers can build and adjust their perceptions of trustworthy based on their participation experience. Thus, this study integrates platform trust as a psychological expectation in crowdsourcing contest to explain its consequences on satisfaction and continuance intention. Herein platform trust is defined as a belief held by solvers that the crowdsourcing contest platform is reliable and trustworthy in terms of labor protection, transaction security, and information accuracy.

In the virtual environment of crowdsourcing contest, trust serves as a salient factor to elicit solvers’ positive affect (e.g., satisfaction) [2], [37]. The labor squeeze is recently a severe concern in crowdsourcing contest as some seekers may make solvers work on the task with submitted solutions and then reject all solutions to avoid paying. Belief in labor protection can help solvers build confidence that there is a low risk of labor squeeze, thereby enabling solvers to experience a high level of satisfaction. In addition, with the prevalence of Internet, online fraud has become a concern on many online communities or platforms [55]. Solvers with a high level of platform trust are likely to alleviate their concerns regarding the potential fraud, thus leading to a satisfied participation experience. In contrast, lacking trust would make solvers doubt if their effort is worthy. For example, [16] noted that if trust was not properly fulfilled, the trusting party would experience unpleasant consequences. When experiencing doubts and unpleasant feelings, solvers have a great possibility to generate a low satisfaction. The positive link between trust and satisfaction has also been verified by extant literature. For instance, both [5] and [53] suggested that trust was an essential ingredient in creating satisfaction. Hence, based on the discussion and prior studies, we propose the hypothesis.

H4: Platform trust is positively related to satisfaction in crowdsourcing contest.

In the crowdsourcing contest setting, the potential uncertainties or risks and are calling for a high level of trust to sustain solvers [2]. Platform trust can reduce solvers’ perceptions of risk by allowing them to subjectively rule out the potentially undesirable behaviors [37], [38]. The low payment and labor squeeze have recently become barriers for sustaining participation intention in crowdsourcing tasks [50]. Establishing measures about labor protection, for example, proofreading the disputable tasks to avoid labor squeeze, to improve solvers’ trust and confidence in the platform may have positive effect on continuance intention. In addition, trust concerning the belief in transaction security can reduce solvers’ anxiety about the money transactions. The enhanced confidence in the exchange relationship can also encourage sustained intention in the given platform. Moreover, information fraud is another severe problem for sustained intention and behavior [55]. Trust in information accuracy helps solvers generate high levels of pride and confidence in their membership, which will enhance their sustained participation intention. Therefore, based on the discussion, we propose the hypothesis.

H5: Platform trust is positively related to continuance intention in crowdsourcing contest.

2.2.4 Effects of Confirmation

Confirmation implies the extent to which individuals realize their expected benefits during their past experience. According to ECM, confirmation is a significant antecedent of satisfaction [7]. Based on ECM, we propose that the solver’s confirmation level will positively affect satisfaction in crowdsourcing contest. Solvers tend to be more satisfied when their confirmation is high. Conversely, if solvers’ perceived benefits cannot meet their expectation, they are more likely to be disappointed with their participation behavior, leading to a low satisfaction. The association between confirmation and satisfaction has been tested and verified by a variety of empirical studies [7], [17], [33], [34]. Thus, based on the discussion and prior studies, we propose the hypothesis.

H6: Confirmation is positively related to satisfaction in crowdsourcing contest.

ECM also suggests that confirmation positively impacts individuals’ ex post expectation (e.g., perceived usefulness) [7]. Herein perceived benefits is used to represent solvers’ ex post performance-expectation about their behaviors in crowdsourcing contest. Hence, in the research context of crowdsourcing contest, confirmation is expected to positively relate to perceived benefits. Solvers tend to rate the crowdsourcing task as being more valuable when their expectation is confirmed during the participation process. Solvers’ performance-expectation in crowdsourcing contest includes getting the reward, challenging themselves, and enjoying fun, etc. When their expectation about getting the reward is confirmed, solvers tend to regard crowdsourcing contest as a good way to reach the expectation and generate a high perception of value. In addition, crowdsourcing tasks are always highly competitive and have demand on solvers’ competency. A confirmed solver is also likely to find crowdsourcing participation useful in challenging self and satisfying the need for competency. By contrast, individuals may experience cognitive dissonance or psychological tension if their expectation is disconfirmed [7], [20]. Solvers will then try to remedy this dissonance by reducing their value perceptions in order to be more consistent with their experience. Similarly, [34] found empirical evidence there was a positive association between confirmation and perceived benefits of Internet protocol television. Hence, based on the discussion and prior studies, we propose the following hypothesis.

H7: Confirmation is positively related to perceived benefits in crowdsourcing contest.

After a solver has first-hand participation experience with crowdsourcing contests, informed subjective beliefs (e.g., trust belief) can be formed. This present research further extends ECM by incorporating trust in crowdsourcing platform as an ex post psychological expectation to examine the other consequences of confirmation. Solvers may have low initial trust perception of a crowdsourcing platform because they are unsure what to expect from their participation. Once solvers’ expectation is confirmed, they tend to believe the crowdsourcing platform will behave in accordance with their expectation by exhibiting ability, integrity, and benevolence [38], consequently establishing their trust belief. Otherwise, with a disconfirmation solvers are likely to feel uncertain about a number of issues during the participation process. For example, solvers may be unsure if there are safeguards from seekers’ opportunistic behaviors by having solvers do the task, then rejecting their submissions in order to avoid paying, and whether the online transaction system is safe. Thus, confirmation is expected to be positively associated with platform trust.

H8: Confirmation is positively related to platform trust in crowdsourcing contest.

2.2.5 Antecedents of Confirmation

Interaction describes the extent to which solvers can exchange information and comment on solutions before the seeker decides the winner. Interaction is one of the distinctive attributes that crowdsourcing contest advocates, because it has been generally recognized to facilitate the retrieval of relevant and diverse knowledge during the idea generation process [28], thus helping lead to novel ideas [6]. In the context of crowdsourcing contest, solvers’ expectation ranges from achieving hedonic value (e.g., self-development, skill improvement, and fun) to utilitarian value (monetary reward) [45]. A high level of interaction can help confirm the expectation as it enables solvers mutually share information or knowledge, build connections, and arouse innovative ideas. For example, interactions and communications with diverse others help solvers stimulate their associations in memory, revise their own knowledge, generate various alternatives, and refine their ideas [42], [43], making it more likely that solvers confirm their expectation about hedonic value by improving their skills and challenging themselves [10]. In addition, as [6] suggested solvers who comment on a diverse set of other solvers’ ideas can develop a better understanding of task needs, leading to solutions or ideas that are more likely to be valuable to the seekers and thus have greater chances of being accepted. The improved chances of being accepted will help solvers confirm the utilitarian value. Thus, it seems reasonable to believe that interaction will facilitate solvers’ confirmation process.

H9: Interaction is positively related to confirmation in crowdsourcing contest.

According to [24], there are two dimensions in fairness perceptions, namely, distributive fairness and procedural fairness. Distributive fairness relates to the perceived fairness of the outcomes distribution, and procedural fairness relates to the perceived fairness of the process by which the allocations are made [30], [51]. When evaluating crowdsourcing contest, distributive fairness is mainly concerned with how fair the reward is allocated and whether the winning solution deserves the reward after the seeker decides the winner [22]. For example, once a solution is selected as the winner, there is an announcement for a period. Other solvers can form their perceptions of the fairness based on their appraisal of the fit between the goal of the task and the winning solution. Relatively, procedural fairness is concerned with the transparency of participation process before the seeker decides the winner, including the equal treatment of every solution and the opportunity to have a voice in the process. The participation process should enable every solver to equal right and information. For example, the evaluation of solutions is the way to approach the reward. Thus, every solution should be evaluated in order to protect the solver’s benefits. Voice and appeal opportunities are also showed to contribute to procedural fairness by a variety of studies [51], [54].

Extant literature has suggested that perceived fairness is positively associated with a variety of outcomes, such as individuals’ attitudes and intentions [22], [24], [54]. In the crowdsourcing contest setting, intensive competition and reward make solvers more sensitive to violations of fairness [15], [22]. Solvers tend to establish their own fairness perceptions about the decision-making process and decision based on experience and comparison. When the platform lacks transparency, solvers tend to have little confidence in achieving the reward, leading to a low level of confirmation. As solution stealing recently is a risk in crowdsourcing contest [47], lacking transparency and the opportunities to defend will further frustrate solvers in confirming their expectation. In contrast, when fair procedures are in place, solvers would have confidence in the decision-making process and believe they can protect their performance by freely expressing their doubts. Also, under a high level of distributive fairness, the winner solution is generally believed to be the best solution for the task. The winning solver can easily confirm his or her expectation by gaining the reward and achieving accomplishment, and other solvers whose solutions are rejected can confirm their expectation by appreciating the winning solution to improve their skills and knowledge. In sum, the perception of fairness gives solvers some beliefs regarding the platform’s commitment to help them meet the expectation. Hence, with a high level of fairness perception, solvers can be more likely to confirm their expectation. Based on the discussion, we propose the hypothesis.

H10: Perceived fairness is positively related to confirmation in crowdsourcing contest.

The research model for solvers’ continuance intention is illustrated in Figure 2.

3 Methodology

This section illustrates the method of developing variable measurements, the process and method of data collection, and the common method variance analysis.

3.1 Measurement

In order to ensure the measurement quality, we adopt items from existing research and slightly modify each item to fit our research context. We follow the translation-back-translation procedure to translate the original English language questionnaire into Chinese, which is recommended by [9]. A professor at School of Foreign Languages first translated the English language version into Chinese which was then back-translated into English by another English professor. Subsequently, the two English language versions were compared to check the differences to ensure the measurement quality. Toward the disagreements about a few of the items, we analyzed extant literature and discussed them with the two translators to reach an agreement. The measurement items were first evaluated by an expert panel of four academicians and five solvers with practical knowledge of crowdsourcing contest to increase validity and avoid measurement errors. A pre-test survey was then conducted to check the quality of our measurement. It was based on the responses from 34 MBA students who were introduced with crowdsourcing contest knowledge and required to scan the crowdsourcing platform beforehand. The pilot test showed that both the Cronbach’s alpha and item loading of each construct were greater than 0.70. Hence, the measurement items are suitable for our survey.

Figure 2: Research model  

Four items for interaction are developed based on the study of [36] to measure the extent of information exchange and comment during the participation process. Perceived fairness is measured with five items from [24] and [22] to reflect the fairness of the participation process and distribution, such as equal treatment toward solutions, opportunities to voice views, and fair distribution toward the winning. Six items for perceived benefits are adapted from [4] and [45] to measure solvers’ perception of expected benefits. Four items for platform trust are taken from [39] to measure the extent to which solvers believe this crowdsourcing platform is reliable and trustworthy. The remainder constructs, confirmation, satisfaction, and continuance intention, are measured with the items of [7]. Appendix A shows the specific items. All items in the survey are presented in Likert scale format with anchors ranging from 1 (strongly disagree) to 5 (strongly agree).

3.2 Data Collection

We conducted the survey on to examine our hypotheses. was founded in 2010 and has become one of the most widely used online crowdsourcing markets in China. On, a seeker first fills out an online request form with the task title, the reward amount(s), the closing date for submissions, and the number of submissions that will be selected as winners. When the closing date is reached, sends a notice to the seeker asking the seeker to select the best solution(s) among all the submissions. The seeker can also choose the best solution(s) before the closing date. In this case, solvers are informed that a solution has been selected and the task is closed. As of April 16 2017, had accumulated more than 4 million tasks, having a demand on solvers’ massive and sustained participation and providing a proper setting to study solvers’ continuance intention.

We administered two surveys separated by a 2-week lag to ensure the quality of our data. At time 1, a task was posted on to recruit respondents for our data collection. The task stated the objective of the survey and the requirements of subject recruitment at the beginning. We specified that only solvers who have participation experience about crowdsourcing contest tasks would be qualified to participate in the survey. This is because participation experience is needed to form the perceptions, to build ex post expectation, and then to make decision about continuance intention in a given crowdsourcing contest platform, but not in a specific crowdsourcing task. The URL of the questionnaire Web page was shown in the description. Every respondent in our survey would get a compensation of RMB 10. At this stage, respondents were invited to evaluate the level of interaction, perceived fairness, perceived benefits, and platform trust. We received 493 responses. At the time 2 stage, we re-invited the 493 solvers who participated at Time 1 survey to fulfill a follow-up survey on basis of the USER ID. The follow-up survey was to assess solvers’ confirmation, satisfaction, and continuance intention. All the measurements were randomized arranged on the two questionnaires. Every respondent would get an additional compensation of RMB 15 at Time 2 survey. Of the 493 solvers, 336 agreed to participate. In both of these two surveys, we put some detective items at the middle part of the questionnaire to test if respondents read the survey carefully, for example, For this question, please choose the second answer. After checking the responses, we discarded inconsistent or erroneous questionnaires, and we finally obtained 306 valid responses.

Of the 306 responses, 51.3% are male, and 48.7% are female. 93.5% have received at least some college education, indicating a good education background. 78.1% are in the age range from 20 to 40. In terms of participation experience, there is a wide range. 26.8% of respondents have participated in 1-2 tasks, 39.5% have participated in 3-5 tasks, 13.1% have participated in 6-10 tasks, and 20.6% have participated in more than 10 tasks.

3.3 Common Method Variance

Although our data was collected with a time-lagged survey, a threat of common method variance (CMV) may exist [44]. Hence, we use two techniques to test if this study suffers from excessive common method variance. Firstly, Harman’s single-factor test is used to ensure that the variance of our data does not largely come from a common method source. All of the measurement items of major constructs and control constructs (i.e., age, education, and experience) were subjected to a principal component analysis to see how many orthogonal components would emerge to explain the variance of the data. Results showed that seven components with eigenvalues of more than 1.0 emerged, and the largest single component accounted for 37.0% of the variance.

Following the study of [32], we secondly conducted a common method factor test by making the measurement items of each construct simultaneously load upon its construct as well as upon a common-method factor, which is created using items from all of the constructs (Appendix B). Results indicate that each item’s variance explained by its substantive construct is much greater than that explained by the common method factor. The sum of squared item-loadings shows that the constructs of this study explain, on average, over 68.4% of the item variance. In contrast, common method variance totally accounts for 33.6% of the variance. Therefore, given the small magnitude of method variance, we contend that the common method variance is unlikely to be a serious concern for this study.

4 Data Analysis and Results

We use partial least squares (PLS) to test the measurements and hypotheses. PLS estimates relationships among latent variables by taking measurement errors into account and thus provides more accurate results [23]. As a component-based approach, PLS has a low demand on sample size and residual distribution [12]. PLS is particularly suitable for testing complex structural models as it avoids two problems: inadmissible solutions and factor indeterminacy [21]. Hence, SmartPLS 2.0 is employed for our analysis.

4.1 Measurement Reliability and Validity

The reliability of measurements is assessed using Cronbach’s alpha and composite reliability (CR). Table 1 shows that the scores of Cronbach’s alpha and CR of all the constructs range from 0.716 to 0.907, all above the recommended threshold value of 0.70 [40]. The convergent and discriminant validity of measurements are tested with average variance extracted (AVE) and item loadings. To achieve adequate convergent validity, AVE should be above 0.5, and item loading should be greater than 0.7 [12], [21]. Our analysis results of AVE (Table 2) and item loadings (Table 1) indicate that our data has a valid convergent validity. The square root of the AVE is suggested to test discriminant validity [12]. Data shown in Table 2 indicates that the square root of the AVE of each construct is much higher than all cross-correlations between the construct and other constructs, confirming the discriminant validity of our measurements.

Table 1: Construct reliability, and item loadings 

4.2 Hypotheses Testing

This study assesses the hypotheses using structural equation modeling (SEM) which is suitable to validate multiple causal relationships simultaneously. SmartPLS 2.0 is used to estimate the structural model and the significance of the paths by jointly using path coefficients and the R2 [12]. Figure 3 displays a schematic representation of the testing results. The estimation of our extended model accounts for 42.2% and 54.4% of the variance respectively in continuance intention and satisfaction.

Perceived benefits is found to significantly affect satisfaction (b=0.202, p<0.01) and continuance intention (b=0.210, p<0.01). Results in Figure 3 also show that there is significant positive association between confirmation and perceived benefits (b=0.655, p<0.001). These findings confirm the importance and appropriateness of perceived benefits in accounting for continuance intention in the context of crowdsourcing contest and respectively support Hypotheses 2, 3, and 7. In addition, the links between satisfaction and continuance intention (b=0.374, p<0.001) and between confirmation and satisfaction (b=0.442, p<0.001) are also found to be significant at 0.001 level. Hypotheses 1 and 6 are thus supported. Platform trust, an important psychological factor, is added in our extended ECM framework to examine whether solvers care about social-related aspects in the process of making decision on continuance participation. Our results (Figure 3) show that platform trust is a significant predictor of satisfaction (b=0.208, p<0.001) and continuance intention (b=0.162, p<0.05). Results in Figure 3 also indicates a high level of platform trust can be fostered by the confirmation level (b=0.504, p<0.001). Hypotheses 4, 5 and 8 are consequently supported by these findings. On account of the significant effects of confirmation on satisfaction, perceived benefits and trust, exploring the antecedents of confirmation seems to be important. Our results show that both interaction (b=0.198, p<0.001) and perceived fairness (b=0.506, p<0.001) are significantly associated with confirmation. These two variables, in total, explain 38.5% of the variance in confirmation. These results provide evidence for Hypotheses 9 and 10. In sum, the extended research model and all hypotheses are supported.

Table 2: Construct discriminant validity 

In order to test the robustness of our results, we first test the direct links between interaction and satisfaction (b=-0.044, p>0.1), between interaction and continuance intention (b=0.038, p>0.1), between perceived fairness and satisfaction (b=0.143, p<0.05), and between perceived fairness and continuance intention (b=-0.041, p>0.1). Then, we test the impacts of control variables (i.e., age, education, and experience) on continuance intention (b=-0.027, p>0.1; b=0.045, p>0.1; b=0.086, p>0.1), satisfaction (b=-0.017, p>0.1; b=0.014, p>0.1; b=0.039, p>0.1), platform trust (b=-0.003, p>0.1; b=-0.021, p>0.1; b=0.122, p<0.05), and perceived benefits (b=-0.028, p>0.1; b=-0.028, p>0.1; b=0.019, p>0.1). When including above direct links, the path directions and significance levels of our hypotheses remain unchanged.

Notes: *p<0.05; **p<0.01: ***p<0.001

Figure 3: Model testing results 

5 Discussion and Implication

The objective of this study is to examine the determinants of solvers’ continuance intention in crowdsourcing contest. We apply the ECM framework as the overarching theoretical foundation and integrate relevant factors to develop our research model. The empirical testing with a sample of 306 solvers confirms our extended model and all hypotheses.

Being similar with the study of [34] and considering the characteristics of crowdsourcing contest, this study uses perceived benefits to replace perceived usefulness in order to better represent solvers’ performance expectation toward their participation. Empirical results provide support for the suitability of using perceived benefits as it is positively related to continuance intention (b=0.210, p<0.01) and satisfaction (b=0.202, p<0.01). Additionally, empirical results suggest that platform trust is another crucial factor in eliciting a high level of continuance intention (b=0.162, p<0.05) and satisfaction (b=0.208, p<0.001) in crowdsourcing contest. These findings suggest that both performance-related aspects and psychology-related aspects are in the concerns of solvers. A lack of value or trust will significantly reduce solvers’ satisfaction level, which in turn damages their continuance intention.

On the other hand, our results indicate that satisfaction has a greater effect on continuance intention (b=0.374, p<0.001) than perceived benefits and platform trust. Perceived benefits and platform trust are also found to significantly affect the level of satisfaction. These findings suggest a pathway of satisfaction through which perceived benefits and platform trust can enhance continuance intention in crowdsourcing contest. In the virtual transaction environment, perceived benefits and platform trust from two different aspects (i.e., material and psychological) provide the nutriments and confidence that enable individuals to pursuit expectation, thus leading to an enhanced affect. The affection as a result of satisfaction can engender goal-directed intention which refers to continuance participation intention in this study. Meanwhile, compared with perceived benefits, platform trust has a more significant effect on satisfaction rather than on continuance intention. This is perhaps because satisfaction is mainly used to represent individuals’ affect which can be easily influenced by psychological factors, such as trust.

Confirmation is an important element in the ECM framework, and our results shows that confirmation is positively related to satisfaction (b=0.442, p<0.001), perceived benefits (b=0.655, p<0.001) and platform trust (b=0.504, p<0.001). The importance of confirmation makes it meaningful to investigate its antecedents in the applied contexts to better leverage the ECM framework [11]. Our empirical results provide supporting evidence that both interaction and perceived fairness are positively related to a high level of confirmation. Compared with interaction (b=0.198, p<0.001), which is noteworthy is that perceived fairness has a stronger effect (b=0.506, p<0.001) on confirmation. The explanation may rest on the intensive competition and monetary reward in crowdsourcing contest, which make solvers concern more about material outcomes with their effort and wisdom investments. It signifies and stresses fairness to be a salient factor for solvers’ reactions.

5.1 Implications in Theory

Through the examination of an extended ECM to understand solvers’ continuance intention in crowdsourcing contests, this study contributes to research in several ways. First, this study contributes to the crowdsourcing continuance intention literature and provides insights regarding why solvers sustain their participation intention with the ECM framework. To the best of our knowledge, this study is among the first study to apply ECM framework to explain solvers’ continuance intention in crowdsourcing context. Previous studies in this stream mainly focus on expectancy value theory, group engagement model, etc. [8], [49], paying little attention to potential changes in solvers’ expectation and factors of influencing these changes. The ECM model is a useful theoretical framework for explaining continuance behavior with the consideration of post-expectation. Further, ECM framework can provide researchers with ample flexibility to incorporate contextual factors to study continuance intention in various contexts [11]. Our application and confirmation of the ECM framework prove ECM to be a viable theoretical foundation in crowdsourcing contest and suggest ECM as an alternative theoretical framework for future studies.

Second, the original ECM framework only focuses on performance expectation (i.e., perceived benefits which is used herein to replace perceived usefulness). However, we argue that in the virtual transaction environment individuals have demand on psychological expectation, such as trustworthiness. In the crowdsourcing contest context, the crowdsourcing platform is the intermediary for solvers to interact with others and to achieve their rewards. Thus, whether the platform is reliable and trustworthy may have significant effect on solvers’ sustained intention and behavior by reducing or eliminating potential risks. However, limited prior crowdsourcing studies have addressed the role of platform trust [37]. Our study supplements the extant crowdsourcing literature by proving platform trust to be a strong determinant of both satisfaction and continuance intention. In addition, our empirical results verify that platform trust can be significantly evoked by the confirmation level. These findings suggest that solvers not only care about performance-related expectation, but concern psychology-related expectation. Our incorporation of platform trust in the research model provides a new perspective to study continuance intention as prior related research primarily focuses on performance expectation when applying the ECM framework.

Third, due to the importance of confirmation, relevant studies should pay considerable attention to its antecedents in order to strengthen the understanding of confirmation. Understanding the antecedents of confirmation will help better address individuals’ continued behaviors [11]. Echoing to some extant studies in calling for attention on the antecedent of confirmation, this study broadens the research stream of ECM by investigating whether contextual factors can induce a high level of confirmation. Standing on the features of crowdsourcing contest, we employ interaction and perceived fairness to explain solvers’ confirmation process. Results demonstrate that both interaction and perceived fairness can trigger a high level of confirmation. The power of interaction and perceived fairness in predicting confirmation level provides future related studies a new perspective to understand why individuals would like to confirm the expectation. These results are also consistent with prior findings that interaction and perceived fairness have significant influence on attitudinal and behavioral reactions [22], [46]. Future crowdsourcing research should closely notice the importance of interaction and perceived fairness.

5.2 Implications in Practice

Several implications for crowdsourcing practitioners can be drawn from this study. First, this study suggests that besides satisfaction, both perceived benefits and platform trust also have significant effects on continuance intention in the context of crowdsourcing contest. In addition, a high level of satisfaction can be achieved from perceived benefits and platform trust. These findings provide crowdsourcing contest managers clues about how to improve solvers’ satisfaction level and encourage them to engage in more tasks. When solvers perceive their participation as valuable and trustworthy, they are more likely to be satisfied and consequently foster continuance intention. Therefore, managers should pay close attention to mechanisms that help solvers promote their perceptions about value and trust. For example, crowdsourcing platforms can establish the threshold of reward to improve solvers’ effort compensation and periodically carry out trainings to enhance the perception of value. Additionally, a perception of value can be achieved with a rating system which can be used to exhibit solvers’ reputation and skills. Given the importance of platform trust in crowdsourcing contest, our study recommends managers some suggestions to build trust. For instance, as recently information fraud is a severe problem in online environment, crowdsourcing platforms should first provide reliable information to improve trust level. Toward the disputable tasks, proofreading the solutions to supervise seekers’ opportunism behavior is also an important mechanism to increase trust belief. Using a reliable trading system to improve solvers’ confidence in their account security is another effective way to increase trust.

Second, confirmation is verified to be a primary determinant of satisfaction, perceived benefits, and platform trust. Crowdsourcing managers should notice the importance of confirmation in evoking solvers’ positive reactions (e.g., satisfaction, perceived benefits, and platform trust), which in turn can encourage solvers to sustain their participation intention. Besides the initial expectation and perceived performance, our study suggests that solvers can achieve a high level of confirmation from interaction and perceived fairness. In order to enhance the confirmation level, therefore, platforms can take actions to ensure high levels of interaction and fairness. For example, on the one hand, smooth interaction and communication and active comment should be highly enabled to allure positive reactions. Importantly, on the other hand, fair procedures and distribution should be paid close attention in crowdsourcing contest, because results show that perceived fairness has a more significant effect on confirmation than interaction does. Solvers may disconfirm their expectation if they find there are unfairness even high-quality interaction exists. Therefore, it is important for crowdsourcing managers to notice why the contest is perceived as unfair. Our study suggests low procedure transparency and unfair distribution on the winning solution can lead to perceptions of unfairness. Mechanisms, such as ensuring that every solution is assessed, and giving solvers opportunities to express their opinions when there are any disputes, are suggested to be able to increase procedural fairness. Also solvers will compare the winning solution with their own solutions to evaluate the distribution fairness. It would be better to force seekers give explanation about their decisions to eliminate or reduce potential doubts.

5.3 Limitations and Future Research

Despite some implications in theory and practice, this study has some limitations. First, this study extends the expectation-confirmation model by incorporating particular factors in the specific context of crowdsourcing contest. Thus, these factors may not generalize to other different research contexts. Related research is needed to identify context-dependent factors and how they impact across contexts. This study provides an example for using ECM framework by incorporating both performance-related and psychology-related factors. Second, the data for our empirical testing is cross-sectional and only provides a one-time illustration of the perception and confirmation process. [7] argued an ideal empirical design for testing ECM framework would be longitudinal, thus it will be more valuable if a longitudinal study is considered. Further, prior studies have indicated that national culture (e.g., individualism versus collectivism, uncertainty avoidance, long-term versus short term orientation, power distance, masculinity versus femininity, and culture tightness) can influence an individual’s general tendency to trust [25] and the success in creative task [13]. The single-source of our data may be another limitation for the generalizability of our findings. Future studies should pay attention to the cross-cultural issue.

6 Conclusion

This study responds to the call for a better understanding of the triggers of solvers’ continuance intention in crowdsourcing contest by using an extended ECM framework. Specifically, perceived benefits, platform trust, interaction and perceived fairness are integrated into the extended ECM framework. Using a sample of 306 solvers, our research model are tested and verified. Results show that solvers have the intention to continue their participation in crowdsourcing contest not only because they enjoy high levels of satisfaction, also because they enjoy high levels of perceived benefits and platform trust. Besides the positive predictions on continuance intention, perceived benefits and platform trust are found to have influence on satisfaction. Further, our empirical results demonstrate that interaction and perceived fairness are two significant determinants of confirmation level. With these findings, this study contributes to theory and practice in several ways by explaining how to sustain solvers’ continuance intention with an ECM framework.


This research was supported by the National Natural Science Foundation of China (71672019, 71271039), Program for New Century Excellent Talents in University (NCET-13-0082).

Websites List

Site 1 : InnoCentive ]

Site 2 : ]

Site 3 : TopCoder ]

Site 4 : Amazon Mechanical Turk ]


[1] A. Afuah and C. L. Tucci, Crowdsourcing as a solution to distant search, Academy of Management Review, vol. 37, no. 3, pp. 355-375, 2012. [ Links ]

[2] P. J. Ågerfalk and B. Fitzgerald, Outsourcing to an unknown workforce: Exploring opensourcing as a global sourcing strategy, MIS Quarterly, vol. 32, no. 2, pp. 385-409, 2008. [ Links ]

[3] I. Ajzen and M. Fishbein, Attitude-Behavior Relations: A Theoretical Analysis and Review of empirical research, Psychological Bulletin, vol. 84, no. 5, pp. 888-918, 1977. [ Links ]

[4] T. M. Amabile, K. G. Hill, B. A. Hennessy, and E. M. Tighe, The work preference inventory: assessing intrinsic and extrinsic motivational orientations, Journal of Personality and social Psychology, vol. 66, no. 5, pp. 950-967, 1994. [ Links ]

[5] S. Balasubramanian, P. Konana and N. M. Menon, Customer satisfaction in virtual environments: A study of online investing, Management Science, vol. 49, no. 7, pp. 871-889, 2003. [ Links ]

[6] B. L. Bayus, Crowdsourcing new product ideas over time: an analysis of the Dell IdeaStorm community, Management Science, vol. 59, no. 1, pp. 226-244, 2013. [ Links ]

[7] A. Bhattacherjee, Understanding information systems continuance: an expectation confirmation model, MIS Quarterly, vol. 25, no. 3, pp. 351-370, 2001. [ Links ]

[8] M. Boons, D. Stam and H. G. Barkema, Feelings of pride and respect as drivers of ongoing member activity on crowdsourcing platforms, Journal of Management Studies, vol. 52, no. 6, pp. 717-741, 2015. [ Links ]

[9] R. W. Brislin, Translation and content analysis of oral and written material, in Handbook of Cross-Cultural Psychology: Methodology (H. C. Triandis and J. W. Berry, Eds.). Boston, MA: Allyn & Bacon, 1980, pp. 349-444. [ Links ]

[10] K. W. Chan, S. Y. Li and J. J. Zhu, Fostering customer ideation in crowdsourcing community: The role of peer-to-peer and peer-to-firm interactions, Journal of Interactive Marketing, vol. 31, pp. 42-62, 2015. [ Links ]

[11] L. Chen, T. O. Meservy and M. Gillenson, Understanding information systems continuance for information-oriented mobile applications, Communications of the Association for Information Systems, vol. 30, no. 9, pp. 127-146, 2012. [ Links ]

[12] W. W. Chin, The partial least square approach to structural equation modeling in Modern Methods for Business Research (G. A. Marcoulides, Ed.). Mahwah, N.J: Lawrence Erlbaum Associates, 1998, pp. 295-336. [ Links ]

[13] R. Y. J. Chua, Y. Roth and J. F. Lemoine, The impact of culture on creativity: How cultural tightness and cultural distance affect global innovation crowdsourcing work, Administrative Science Quarterly, vol. 60, no. 2, pp. 189-227, 2015. [ Links ]

[14] E. L. Deci and R. M. Ryan, The what and why of goal pursuits: Human needs and the self-determination of behavior, Psychological Inquiry, vol. 11, no. 4, pp. 227-268, 2000. [ Links ]

[15] X. F. Deng, K. D. Joshi and R. D. Galliers, The duality of empowerment and marginalization in microtask crowdsourcing: Giving voice to the less powerful through value sensitive design, MIS Quarterly, vol. 40, no. 2, pp. 279-302, 2016. [ Links ]

[16] M. Deutsch, Trust and suspicion, Journal of Conflict Resolution, vol. 2, pp. 265-279, 1958. [ Links ]

[17] D. M. Eveleth, L. J. Baker-Eveleth and R. W. Stone, Potential applicants’ expectation-confirmation and intentions, Computers in Human Behavior, vol. 44, pp. 183-190, 2015. [ Links ]

[18] Y. L. Fang, I. Qureshi, H. S.Sun, P. McCole, E. Ramsey, and K H. Lim, Trust, satisfaction, and online repurchase Intention: The moderating role of perceived effectiveness of e-commerce institutional Mechanisms, MIS Quarterly, vol. 38, no. 2, pp. 407-427, 2014. [ Links ]

[19] R. H. Fazio and M. P. Zanna, Direct Experience and Attitude-Behavior Consistency in Advances in Experimental Social Psychology (L. Berkowitz, Ed.). New York: Academic Press, 1981, pp. 161-202. [ Links ]

[20] L. A. Festlnger, A Theory of Cognitive Dissonance. Evanston, IL: Row and Peterson, 1957. [ Links ]

[21] C. Fornell and F. L. Bookstein, Two structural equation models: LISREL and PLS applied to consumer exit-voice theory, Journal of Marketing Research, vol.19, no. 4, pp. 440-452, 1982. [ Links ]

[22] N. Franke, P. Keinz and K. Klausberger, Does this sound like a fair deal?: Antecedents and consequences of fairness expectations in the individual’s decision to participate in firm innovation, Organization Science, vol. 24, no. 5, pp. 1495-1516, 2013. [ Links ]

[23] D. Gefen, D. Straub and M. C. Boudreau, structural equation modeling and regression: Guidelines for research practice, Communications of the AIS, vol. 4, no. 7, pp. 1-74, 2000. [ Links ]

[24] S. W. Gilliland, The perceived fairness of selection systems: An organizational justice perspective, Academy of Management Review, vol. 18, no. 4, pp. 694-734, 1993. [ Links ]

[25] H. Hallikainen and T. Laukkanen, National culture and consumer trust in e-commerce, International Journal of Information Management, vol. 38, no. 1, pp. 97-106, 2018. [ Links ]

[26] Y. Huang, P. Vir Singh and K. Srinivasan, Crowdsourcing new product ideas under consumer learning, Management Science, vol. 60, no. 9, pp. 2138-2159, 2014. [ Links ]

[27] Y. J. Joo, S. Park and E. K. Shin, Students' expectation, satisfaction, and continuance intention to use digital textbooks, Computers in Human Behavior, vol. 69, pp. 83-90, 2017. [ Links ]

[28] N. Kohn and S. Smith, Collaborative fixation: Effects of others’ ideas on brainstorming, Applied Cognitive Psychology, vol. 25, no. 3, pp. 359-371, 2011. [ Links ]

[29] T. L. Lai, Service quality and perceived value's impact on satisfaction, intention and usage of short message service (SMS), Information Systems Frontiers, vol. 6, no. 4, pp. 353-368, 2004. [ Links ]

[30] G. S. Leventhal, What should be done with equity theory? in Social Exchange: Advances in Theory and Research, (K. J. Gergen, M. S. Greenberg and R. H. Willis, Eds.). New York: Plenum, 1980. [ Links ]

[31] H. G. Liang, M. M. Wang, J. J. Wang, and Y. J. Xue, How intrinsic motivation and extrinsic incentives affect task effort in crowdsourcing contests: A mediated moderation model, Computers in Human behavior, vol. 81, pp. 168-176, 2018. [ Links ]

[32] H. Liang, N. Saraf, Q. Hu. and X. Y. Xue, Assimilation of enterprise systems: The effect of institutional pressures and the mediating role of top management, MIS Quarterly, vol. 31, no. 1, pp. 59-87, 2007. [ Links ]

[33] C. S. Lin, S. Wu and R. J. Tsai, Integrating perceived playfulness into expectation-confirmation model for web portal context, Information & Management, vol. 42, no. 5, pp. 683-693, 2005. [ Links ]

[34] T. C. Lin, S. Wu and J. S. C. Hsu, The integration of value-based adoption and expectation-confirmation models: An example of IPTV continuance intention, Decision Support Systems, vol. 54, no. 1, pp. 63-75, 2012. [ Links ]

[35] T. X. Liu, J. Yang, L. A. Adamic, and Y. Chen, Crowdsourcing with all-pay auctions: A field experiment on Taskcn, Management Science, vol. 60, no. 8, pp. 2020-2037, 2014. [ Links ]

[36] Y. Liu, Developing a Scale to measure the interactivity of websites, Journal of Advertising Research, vol. 43, no. 2, pp. 207-216, 2003. [ Links ]

[37] M. G. Martinez, Inspiring crowdsourcing communities to create novel solutions: Competition design and the mediating role of trust, Technological Forecasting and Social Change, vol. 117, pp. 296-304, 2017. [ Links ]

[38] R. C. Mayer, J. H.Davis and F. D. Schoorman, An integrative model of organizational trust, Academy of Management Review, vol. 20, no. 3, pp. 709-734, 1995. [ Links ]

[39] D. H. McKnight, V. Choudhury and C. Kacmar, Developing and validating trust measures for e-commerce: An integrative typology, Information Systems Research, vol. 13, no. 3, pp. 334-359, 2002. [ Links ]

[40] J. Nunnally, Psychometric Theory, 2nd ed. New York: McGraw-Hill, 1978. [ Links ]

[41] R. L. Oliver, A cognitive model for the antecedents and consequences of satisfaction, Journal of Marketing Research, vol. 17, no. 4, pp. 460-469, 1980. [ Links ]

[42] J. Perry-Smith, Social yet creative: The role of social relationships in facilitating individual creativity, Academy of Management Journal, vol. 49, no. 1, pp. 85-101, 2006. [ Links ]

[43] J. Perry-Smith and C. Shalley, The social side of creativity: A static and dynamic social network perspective, Academy of Management Review, vol. 28, no. 1, pp. 89-106, 2003. [ Links ]

[44] P. M. Podsakoff, S. B. MacKenzie, J. Y. Lee, and N. P. Podsakoff, Common method biases in behavioral research: A critical review of the literature and recommended remedies, Journal of Applied Psychology, vol. 88, no. 5, pp. 879-903, 2003. [ Links ]

[45] J. A. Roberts, I. H. Hann and S. A. Slaughter, Understanding the motivations, participation, and performance of open source software developers: A longitudinal study of the Apache projects, Management Science, vol. 52, no. 7, pp. 984-999, 2006. [ Links ]

[46] I. Rodriguez-Ardura and A. Meseguer-Artola, E-learning continuance: The impact of interactivity and the mediating role of imagery, presence and flow, Information & Management, vol. 53, no. 4, pp. 504-516, 2016. [ Links ]

[47] X. Shi and Y. X. Zou, A summarization of Witkey pattern, Library and Information, vol. 2, pp. 69-72, 2009. [ Links ]

[48] W. Soliman and V. K. Tuunainen, Understanding continued use of crowdsourcing systems: An interpretive study, Journal of Theoretical and Applied Electronic Commerce Research, vol. 10, no. 1, pp. 1-18, 2015. [ Links ]

[49] Y. Sun, Y. Fang and K. H. Lim, Understanding sustained participation in transactional virtual communities, Decision Support Systems, vol. 53, no. 1, pp. 12-22, 2012. [ Links ]

[50] Y. Sun, N. Wang, C. Yin and J. Zhang, Understanding the relationships between motivators and effort in crowdsourcing marketplaces: A nonlinear analysis, International Journal of Information Management, vol. 35, no. 3, pp. 267-276, 2015. [ Links ]

[51] J. Thibaut and L. Walker, Procedural Justice: A Psychological Analysis. Hillsdale, N.J.: Lawrence Erlbaum Associates, 1975. [ Links ]

[52] V. Venkatesh, J. Y. L.Thong, F. K. Y. Chan, P. J. Hu, and S. A. Brown, Extending the two-stage information systems continuance model: incorporating UTAUT predictors and the role of context, Information Systems Journal, vol. 21, pp. 527-555, 2011. [ Links ]

[53] J. J. Xu, R. T. Cenfetelli and K. Aquino, Do different kinds of trust matter? An examination of the three trusting beliefs on satisfaction and purchase behavior in the buyer-seller context, The Journal of Strategic Information Systems, vol. 25, no. 1, pp. 15-31, 2016. [ Links ]

[54] Y. J. Xue, H. G. Liang and L. S. Wu, Punishment, justice, and compliance in mandatory IT settings, Information Systems Research, vol. 22, no. 2, pp. 400-414, 2011. [ Links ]

[55] Y. Zhang, J. Bian and W. Zhu, Trust fraud: A crucial challenge for China’s e-commerce market, Electronic Commerce Research and Applications, vol. 12, no. 5, pp. 299-308, 2013. [ Links ]

Appendix A: Measurement Items

Interaction (IA): adapted from [36]

IA1. I have the opportunity to talk to seekers when participating in the task.

IA2. The seeker responds to my questions in time when participating in the task.

IA3. I have the opportunity to communicate with other solvers on this crowdsourcing platform.

IA4. I have the opportunity to comment on other solutions on this crowdsourcing platform


Perceived Fairness (PF): adapted from [24] and [22]

PF1. On this crowdsourcing platform, all the solvers participate actively and equally in the process.

PF2. On this crowdsourcing platform, all the solvers have the right to voice their opinions in the process.

PF3. I have the feeling that all the solvers are treated equally during the whole process.

PF4. In my eyes, it is fair to allocate the reward to the solver with the winning solution on this crowdsourcing platform.

PF5. In my eyes, the reward is split fairly between the solver with the winning solution and the crowdsourcing platform.

Confirmation (CF): adapted from [7]

CF1. My experience with participating in the tasks on this crowdsourcing platform is better than what I expected.

CF2. The service level provided by this crowdsourcing platform is better than what I expected.

CF3. Overall, most of my expectation from participating in the tasks on this crowdsourcing platform is confirmed.

Perceived benefits (PB): adapted from [4] and [45]

PB1: Participating in the tasks on this crowdsourcing platform can help me earn some money.

PB2: This crowdsourcing platform provides me a new chance for work.

PB3: Participating in the tasks on this crowdsourcing platform gives me a chance to do things I am good at.

PB4: Participating in the tasks on this crowdsourcing platform provides me with opportunities for increasing my knowledge and skills.

PB5: Participating in the tasks on this crowdsourcing platform lets me feel a sense of personal achievement.

PB6: The process of participating in the tasks on this crowdsourcing platform is very pleasant.

Platform trust (PT): adapted from [39]

PT1. This crowdsourcing platform provides good and reliable crowdsourcing service.

PT2. In general, this crowdsourcing platform really cares about the well-being of the solvers.

PT3. This crowdsourcing platform provides a secure trade system.

PT4. Overall, I believe this crowdsourcing platform is trustworthy.

Satisfaction (SF): adapted from [7]

SF1. I am satisfied with the service provided by this crowdsourcing platform.

SF2. I am satisfied with the benefits obtained from this crowdsourcing platform.

SF3. Overall, I am satisfied with the experience on this crowdsourcing platform.

Continuance intention (CI): adapted from [7]

CI1. I intend to continue participating in the tasks on this crowdsourcing platform.

CI2. My intentions are to continue participating in this crowdsourcing platform rather than on other crowdsourcing platforms.

CI3. If I could, I would like to discontinue my participation in the tasks on this crowdsourcing platform.

Appendix B: Common Method Variance Test

Following [32], we performed a common method factor test using PLS. Each construct was converted to a second-order construct and each of its indicators was converted to a single-indicator construct. A common method factor was created using indicators from all of the constructs. Each single-indicator construct was modeled to be determined by (1) its second-order construct and (2) the method factor. The two squared path weights of each item represent variance explained by the substantive construct and the common method, respectively. As Table 1 indicates each indicator’s variance explained by its substantive construct is much greater than that explained by the common method factor, which suggests that there seems to be no severe common method bias.

Common method factor test results

Appendix C: Cross Loadings

Cross loadings of construct measures

Received: March 12, 2018; Revised: July 22, 2018; Accepted: September 26, 2018

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License