Online labor market signaling

Online labor market signaling⇤ Otto Kässi† 5th September 2017

Abstract There is ample evidence on information frictions in online freelancing markets. As a result of these frictions, employers have limited means for telling good and bad freelancers apart. Online labor market platforms have developed several institutions for levelling this information asymmetry. I study one of these institutions: voluntary skill certification tests which the freelancers can take to reduce employer uncertainty. I show that completing skill certificates increases earnings after freelancer fixed effects are controlled for. This increase is not driven by increased freelancer productivity, but decreased employer uncertainty. The effect of signaling on earnings and other labor market outcomes is nonetheless found to be economically small. This suggests that the current testing scheme used by the online labor market platform has only a limited uncertainty reducing effect. JEL CODES: J21, J23, J24, J31, I2 Keywords: signaling, human capital, online freelancing

⇤

I am grateful to my colleagues Greetje Corporaal, Vili Lehdonvirta and Alex Wood for helpful comments and discussion. All remaining errors are my own. † Address: Oxford Internet Institute, 1 St Giles, OX13JS Oxford, United Kingdom. Email: [email protected].

1

1

Introduction

Various online labor markets such as Upwork, Guru, Freelancer, Peopleperhour, Fiverr and Amazon MTurk have sprung up in recent decade.1 The introduction of these new types of intermediaries has enabled companies to contract directly with individuals from abroad without having to resort to specialist outsourcing companies (Agrawal et al., 2015). This has a large potential for reducing transaction costs and increasing welfare (Autor, 2001). To what degree the potential of these markets is actualized largely depends on how the rules, features and algorithms – the digital policies that govern the matching of workers to employers on the markets – are designed.2 Online freelancing labor markets are riddled with information frictions. In particular, because the employers do not meet face to face with their potential workers, they are forced to make their hiring decisions based on limited information (Autor, 2001; Malone and Laubacher, 1998). The global nature of freelancing labor markets exacerbates these frictions because employers might face difficulties in evaluating value of freelancers’ work experience or education credentials (Oreopoulos, 2011). Even if these types of information asymmetries are to some extend present even in traditional labor markets, the decentralized nature of online freelancing might further amplify them. This is because screening and hiring freelancers is partially a public good. Once a freelancer has been hired and evaluated, all other market participants have access to the information provided by the employer who made the first hire. In this sense, all employers have an incentive to free-ride on other employers’ screening efforts. In equilibrium, this mechanism may lead to an outcome, where new freelancers face difficulties in breaking into the market, whereas older already screened freelancers have an upper hand on the market regardless of their quality (Pallais, 2014).3 The purpose of this paper is to empirically and theoretically evaluate an online labor market policy designed to help inexperienced online workers to break into the market, voluntary skill certificates. I argue that skill certificates operate as a type of a signaling device a la Spence (1973). They do not increase the productivity of freelancers but demonstrate their ability to potential employers. I hypothesize that employers have imperfect information about worker ability, which creates an incentive for the workers to 1

Kuek et al. (2015) estimate that, globally, 48 million freelancers have registered with online labor platforms. For up-to-date details of transaction volumes and geographical and occupation distribution of online gig work, see Kässi and Lehdonvirta (2016). 2 Horton (2017) and Agrawal et al. (2016) study how other types of digital labor market institutions affect labor market outcomes. 3 Terviö (2009) argues that this same mechanism may be a partial explanation for high CEO wages and low turnover.

2

invest in costly signals. This incentive is expected to be larger for freelancers with larger productivity uncertainty. To formalize this intuition, I construct a model of statistical discrimination where freelancers’ skill certificates are observable by prospective employers, but freelancer’s ability is private information. Theoretical model provides a set of empirical predictions, which are then tested using transaction data from a major online labor market platform. A recurring challenge in estimating returns to signaling is that returns to signaling are confounded with increases in human capital. For instance, if we observe that education increases wages, it is oftentimes impossible to tell whether the higher earnings are caused by information provided by signaling or by the increase in individuals’ productivity due to higher human capital. Transaction level data provided by online freelancing platforms has many appealing features for studying this phenomenon. The data contains a rich set of background characteristics of freelancers which can be included as control variables. Moreover, the fact that projects are relatively short, and take place repeatedly allows me to use the longitudinal dimension of the data to take into account freelancer specific unobserved heterogeneity. In an ideal research setting, a researcher would fully control for freelancer ability and human capital when studying the effect of signaling on earnings. In this paper I approximate the ideal setting by comparing freelancers’ earnings before and after acquiring a skill certificate. This allows me to gauge all time-invariant unobservable factors into freelancer fixed effects. In addition, I limit my attention to a 14-day time period around the certificate award. With this, I can ascertain that the return estimates are not contaminated by individual learning or other time varying human capital effects. I find that the return to completing and extra certificate has a positive effect on income, and this effect is estimated fairly accurately. Furthermore, I find the OLS estimate for return to signaling is negatively biased compared to a fixed effects estimate. This suggests that unobservable factors in the earnings equation are negatively correlated with the decision to signal, or that the freelancers who are worse off in the labor market signal more to offset their disadvantage. Further, I establish that completing skill certificates does not improve freelancer productivity proxied by ratings from completed projects. The latter finding suggests that the reason why completing skill certificates increases earnings is not increased productivity of the freelancers but decreased employer uncertainty. This is consistent with the hypothesis that skill certificates act as signals for freelancer skills. Unfortunately for both sides of the labor market, the returns are estimated to be fairly modest, only around 2-3 percentage points. The modest returns to signaling suggest that 3

monetary benefit from completing skill certificates does not outweigh the effort cost from signaling for most workers. Low return to signaling combined with a low median level of completed skill certificates per freelancer is suggestive of a near-pooling equilibrium of the signaling game. In other words, all sides of the market find signaling relatively non-informative of freelancer quality. Consequently, employers do not reward much for signaling and most freelancers rationally choose not to invest much effort in signaling. The results of this paper contribute to multiple strands of empirical literature. First, the paper links to emerging literature on how various types of online labor market institutions affect employment outcomes. Most closely related papers include Pallais (2014) which presents a field experiment where she randomly hired inexperienced contactors and then provided feedback on their performance. By comparing her hires’ subsequent income to other freelancers’ income who applied to her posted jobs but weren’t chosen she shows that the freelancers she randomly hired earn considerably more than the control group. She argues that this effect is due to her feedback providing information on the hired freelancers to other potential employers. Extending on Pallais’ results, Agrawal et al. (2016) show that standardized information on platform based work experience benefits all freelancers, but the freelancers from developing countries benefit disproportionately. This suggests that initially the employers have more trouble evaluating the quality of freelancers from developing countries compared to developed countries. Relatedly, Lin et al. (2016) show that higher reputation score is positively associated with future hires. Yoganarasimhan (2013) demonstrates that a reputation system decreases information frictions resulting in higher willingness to pay on the employer side and, consequently, higher equilibrium wages. Stanton and Thomas (2015) show that information from intermediaries helps inexperienced freelancers: a freelancer affiliated with an intermediary agency has a substantially higher job finding probability and wage. Horton (2017) studies the effect of automated freelancer recommendations on subsequent hiring decisions. Further, this paper touches on literature on the effect of using standardized tests as as a method for revealing information on worker quality (Autor and Scarborough, 2008; Hoffman et al., 2015; Bassi and Nansamba, 2017). A recurring theme in this literature is that standardized job testing can overcome biases of hiring managers. More broadly, the results link to empirical studies on job market signaling (Tyler et al., 2000; Lang and Manove, 2011; Pinkston, 2003; Arcidiacono et al., 2010), and to a long tradition of papers studying how statistical discrimination affects labor market outcomes of disadvantaged groups; and to what extend labor market institutions can mitigate or intensify 4

these effects (Lundberg and Startz, 1983; Bertrand and Mullainathan, 2004; Lahey, 2008; Altonji and Pierret, 2001; Katz and Oded, 1987).

2

Empirical setting and data

2.1

Details of the online labor platform

The dataset is collected from one of the largest online labor markets. Before turning to the details, I briefly present a typical workflow of contracting within the platform. Employers looking to hire a freelancer for a particular task typically initiate the job search process by posting an opening on the site. The opening includes the skills required, expected contract duration, preferred freelancer characteristics and a contract type, which can be either flat rate or hourly billed contract. One major difference between flat rate and hourly priced projects is that hourly priced projects allow employers to use monitoring technologies, such as taking screenshots at regular intervals from their freelancers’ screens and keystroke captures. These technologies are not available for flat rate contracts. In the case of flat rate contracts, the result of freelancer’s work can only be evaluated once it is complete (for more details between the differences of the two project types, see Lin et al., 2016). After a posting is published, it is visible to everyone, the freelancers can apply to the position by submitting private bids. The firms can also directly invite freelancers to apply to their position. In this case, the opening is not public. The interview and salary negotiation phases also take place on the platform. When posting a project, the employer also chooses a detailed project category from 89 distinct categories. These ’detailed job categories’ are used as control variables in all regression models. Of particular interest to me are the skill tests administered on the platform. Freelancers can take multiple-choice quizzes on various skills. Typically taking a test takes between 30 and 60 minutes. All in all, freelancers can perform over 300 distinct skill certificates from topics such as English language, programming languages and IT and office programs. Once a skill test has been completed, the freelancer gets a “badge” showing its completion (see Figure 1). The badge also shows freelancer’s numerical grade, and percentile rank among all test takers. When inviting freelancers to a project, the employers can limit their search to freelancers who have completed a skill test or have scored over a certain threshold in a skill test. If the freelancer has tried to take a test and failed, a failed mark is not visible to potential employers. In addition, the freelancer can retake the test after a ’cooldown’ period lasting between 30 and 180 days. Freelancers can also

5

Figure 1: Screenshot of a freelancer’s profile featuring skill tests. hide the results of their passed tests. A crucial assumption for this paper is that a freelancer cannot learn anything by just taking the exam. This is a reasonable assumption since it is not plausible that a freelancer would pick up a skill such as a programming language or a foreign language – which typically takes months or years to learn – when taking the test. In the empirical part of the paper, I use the number of skill tests taken as a measure for the intensity of signaling. This is a natural measure for the intensity of signaling since the costs of signaling increase roughly linearly with of tests taken.

2.2

Data

The data set used in this paper was collected from a major online labor market platform using their developer API in February of 2016. I collected a random sample of 46,791 freelancers, alongside their background information and work histories. These freelancers had completed 422,203 projects in total. From this data I excluded the projects that included more than hire, and those without dollars-earned information. The main summary statistics are presented in Table 1. The first panel of Table 1 presents project characteristics. 58% of the projects are priced on hourly basis. The values of fixed price projects are highly skewed to the right with a median of 50 dollars and a mean almost three and a half times larger than the median. The hourly wages paid for hourly projects exhibit less skew with a median of 10 dollars and a mean of 12.42 dollars. I also observe a measure freelancer performance evaluated by the employer. This comes in a form of a rating ranging from 1 to 5. As discussed by Horton and Golden 6

(2015) and Nosko and Tadelis (2015), reputation scores tend to be upwards biased for two reasons. Giving bad feedback might result in retaliation by the other party. Further, if the probability of leaving a feedback in the first place is positively correlated with worker quality, some of the poor feedback will remain unrevealed, resulting in inflated reputation scores of less qualified workers. Probably due to these reasons, the median star rating is 5/5, and even the average rating is also almost 5. At the same time, the star ratings of freelancers exhibit somewhat more variation with a median of 5, mean of 4 and a standard deviation of 1.9. This suggests that the ratings carry some – albeit noisy – information on workers’ productivity. Roughly three percent of the projects are not given a feedback. The data does not include projects where the employer explicitly invited a single freelancer to work on their project (about 5% of all projects). It seems likely that the employer who invites a freelancer to work for them would know the freelancer in advance. Since I want to the analysis to projects where the employer is in a situation where he needs to hire a freelancer under imperfect information on freelancer productivity, I exclude these projects. After filtering out invitation-only vacancies, there are an average of 16 applicants for each vacancy. The platform has operated for almost ten years, so there are some freelancers who have been active on the market for a very long time. In practice, the churn on the market is relatively large, and the average freelancer “age” (measured from the start date of their first vacancy) is only about a year, with a median of 2.3 months. This is also reflected in freelancers’ average number of completed projects which is only 1, with an average of 64 earned dollars. Our main explanatory variable of interest is the number of completed skill tests. On average, freelancers have completed 2.73 skill tests, with a median of 2. The skill tests exhibit much less skew than earnings and work experience. This suggests that skill tests are not necessary for success in the market for freelancers. I turn to the question of who is expected to signal their skills, and benefit from signaling, in the next section. Figure 2 plots the distributions of main variables of interest: numbers of completed projects by a freelancer, dollars earned by a freelancer and skill certificates earned by freelancer. Consistent with the costly screening hypothesis put forward in the introduction, the success in the labor market seems to exhibit ’Matthew effects’ where the freelancers with previous experience tend to win more projects and accumulate even more experience, which further increases their advantages in the labor market leading to highly skewed distribution of work experience and earnings. Skill certificate distribution is much less right skewed compared to freelancer success measures, but still has some 7

Table 1: Summary statistics of freelancers and projects Project characteristics Hourly project Total project value (all projects) Hourly rate (hourly projects) Project size (fixed price projects) Star rating given to worker (max 5 stars) Number of applicants N Freelancer characteristics Months active Number of completed projects Dollars earned Avg star rating Number of completed skill tests N

Median 0 70 10 50 5 10 422,203

Mean 0.42 474.91 12.42 172.19 4.83 16.33

s.d. 0.49 2864.41 11.97 920.99 0.53 16.44

Median 2.27 1 63.99 4.93 2 46,791

Mean 11.95 9.8 4875.28 3.96 2.73

s.d. 19.85 27.75 17361.54 1.88 3.61

outliers in the upper tail.

3

Motivating theoretical framework

This section introduces a signaling model which shows that uncertainty on freelancer ability creates an incentive for freelancers to invest in a costly signal. The model is a slightly simplified version of the model presented by Lang and Manove (2011), and it provides testable implications of how the level of freelancer signaling varies with uncertainty on productivity, and how returns to signaling vary with uncertainty of freelancer productivity. Freelancers differ in their ability. Only the freelancers are assumed to know their own ability. Potential employers observe freelancers’ certificates accurately, but freelancer ability is observed with noise. Consequently, the freelancers have an incentive to gain certificates to reduce employer uncertainty. The more noise there is on employers’ direct observation on freelancer ability, the more weight they place on the certificate signal. Throughout the rest of this section I assume that there exists a separating equilibrium; the freelancers’ ability determines how much they signal. My model differs from a standard job market signaling model in one crucial way. I assume that the freelancers cannot fully deduce their own productivity in a particular task from knowing their ability. Instead, I postulate that freelancer’s productivity in a given task is only revealed to both the freelancer and the employer when their match is formed, even though I assume that the expected freelancer productivity is higher for 8

40000

Count

30000

20000

10000

0 0

200

400

600

Number of projects completed

(a) Distribution of completed projects 40000

count

30000

20000

10000

0 0

100

200

300

400

Dollars earned (1000)

(b) Distribution of dollars earned.

20000

count

15000

10000

5000

0 0

25

50

75

100

Skill tests completed

(c) Distribution of completed skill tests

Figure 2: Empircal distribution of main variables of interest.

9

a high-ability freelancer. This assumption can be motivated in the context of online labor markets by the observation that there is great variation across freelancers, across employers and across projects in online labor markets, which adds a random element to the match specific productivity. Freelancer’s ability is distributed along a fixed interval [a0 , a1 ]. The freelancer’s productivity p⇤ in a given project conditional on her ability is given by p⇤ = a + ",

(1)

where a is the freelancer’s ability and " is a normally distributed match specific random variable which is only realised to after the match between a freelancer and employer has been formed. It has a mean of 0 and variance of

2 ".

A potential employer can observe the number of skill certificates the freelancer has, s, but not their true productivity p⇤ . The employer observes a noisy estimate of freelancer’s productivity given by p = p⇤ + u,

(2)

where u is another normally distributed random error term. The error term u has a variance of

2 u (c) which @ u2 (s)

is common to all firms, continuous, and is decreasing and convex @ 2 u2 (s) in signaling, < 0, and > 0. The " and u are independent of one another, @s @s2 and are assumed to be known to both employers and freelancers. Denote the accuracy of employer inference as (s) 2 [0, 1], where (s) = For a given value of

2 ",

2 " 2 u (s)

+

if (s) is close to zero, then

2 "

.

2 u (s)

must be large, and, consequently,

the employer’s ability to observe freelancer productivity directly is poor, and have to rely more on the certificate signal. If (s) = 1 then

2 u (s)

= 0 and the employer observes

freelancer productivity perfectly and does not have to rely on signals. The employers follow the rules of a competitive labor market, they pay the freelancers the wage which is determined by their expected productivity. Their equilibrium inference of the freelancers’ productivity, p⇤ , depends on the elements they observe, p and s. Further, denote a ˆ = a(s) as employers’ equilibrium inference on a conditional on s. Throughout this paper, I assume that there exists an unique, continuous, differentiable, strictly increasing in a equilibrium which specifies a unique best response for every ability level and . To solve for E [p⇤ | p, s], note that, in equilibrium, equations (1) and (2) 10

imply that in equilibrium, p

a ˆ(s) = u + ". Therefore,

E [w | p, s] = E [p⇤ | p, s] = E [p⇤ | p

(3)

a ˆ(s), s]

= E [a | p

a ˆ, s] + E [" | p

= a ˆ + (p

a ˆ)

=

p + (1

a ˆ, s] (4)

)a ˆ,

which is the equilibrium competitive wage offer of the employer conditional on p and s. Before turning to the freelancer’s choice of optimal levels of signaling, it is useful to note that Equation (4) implies that if there are two freelancers L, and H with the same level of a, but

2 u,L (s)

>

2 u,H (s),

freelancer H is at an advantage because the employer

can better evaluate her productivity. Therefore freelancer L will have a larger incentive to invest in signaling. The freelancers’ problem boils down to choosing s to solve for max E [w] s

(5)

c(a)s,

where c(a) is the effort cost of getting a certificate. c(a) is assumed to be decreasing and convex in a. In equilibrium equation 5 simplifies to max E [p] + (1

)ˆ a

s

c(a)s.

(6)

Its first order condition reads as sa

ˆ sa

+ (1

)ˆ as = c(a) + ca as ,

where subscripts denote partial derivatives, (e.g. a ˆs = simplifies to (1

ca )ˆ as = c(a).

which implicitly solves sfor each combination of inverting yeilds sa =

1

ca c(a)

(7)

@ˆ a ). In equilibrium, a = a ˆ, so 7 @s (8)

and a. Finally solving for as and .

(9)

Equation 9 demonstrates that equilibrium value of s(a) is strictly increasing in a. We also know that the freelancer with the lowest level of ability does not invest into signaling, or s(a0 ) = 0. Too see why this is the case, note that if s(a0 ) > 0, the freelancer with a 11

a > a0 , could deviate to smaller s without affecting employers’ equilibrium inference on her ability. The only case when this is not possible is if s(a0 ) = 0. After having confirmed that s(a0 ) = 0, and noting that Equation (8) is continuous and differentiable, we know that s(a) exists and is uniquely determined for all combintations of a and . Now, assume that there are two freelancers with the same level of a but and Consequently,

L

H.




2 u,H

In other words the employers face a higher uncertainty

when trying to evaluate the expected productivity of freelancer L compared to freelancer H. With this assumption, theoretical framework laid out generates following predictions. 1. If there are two freelancers (L, H) with same a but s(

H)

L




whenever a > a0 . That is, higher employer uncertainty on freelancer quality

results in more signaling by the freelancer. To see this, note that equation (8) implies that if s(a0 ;

L)

= s(a0 ;

L

< L ).

H,

then sa (

L)

> sa (

H ).

Further, I argue above that

By the continuity of s, this is possible only if s(

L)

> s(

H ).

@E [w; H ] @E [w; L ] > , @s @s or returns to signaling are higher if the uncertainty is higher. To see why this holds, @ 2 E [w] note that < 0 for all a > a0 . @s@

2. If there are two freelancers with same a but

L


a0 .

@ 2 E [w] < 0 for all @s2

Predictions 1. and 2. are intuitive: for a given level of ability, the freelancers who are more statistically discriminated against, i.e. for whom productivity uncertainty is higher, get a higher marginal return from signaling, and consequently, signal more. Further, predictions 2. and 3. suggests that signaling exhibits two types of decreasing returns. In addition to marginal effect of signaling being lower for high levels of signaling, the return to signaling is also lower if the is employer uncertainty on productivity is lower. Furthermore, equation (9) demonstrates that the choice of the level of signaling depends on two features which are both unobservable to the researcher, but which affect freelancer earnings. On the other hand, more able freelancers signal more because the costs of signaling are lower, but on the other hand the freelancers who know that the employers might have problems evaluating their productivity would signal more. As a result, failing to control for these factors in an OLS regression of number of skill certificates on earnings likely leads to a biased estimate on the regression coefficient on s. The direction of the bias of the OLS estimates can be used to infer which of the effects dominates: if OLS estimates are biased downwards, the decision to signal is negatively correlated with 12

earnings, and the uncertainty bias dominates the ability bias. On the other hand, if the OLS estimates are biased upwards, the unobservables in earnings equation are positively correlated with the decision to signal.

4

Empirical model and results

4.1

Returns to signaling

I now turn to studying how signaling efforts are rewarded on the labor market. To study this, I fit variants of the following fixed effects regression model: yik = ↵i + Xi + s + "ik .

(10)

Here, yik is the (log-) income for freelancer i earned from project k. On the right hand side, ↵i are freelancer specific fixed effects; vector Xi

includes our measures of

observable time-varying characteristics: average rating for completed projects, sum of previous projects completed and dollars earned on the platform and competitiveness of an individual project (measured by the number of applicants to the project), and dummy variables for different project types. Our main parameter of interest is , which measures marginal effect of earning an extra skill certificate on the platform, and captures the effect of signaling on earnings. I have chosen to use project value as the main dependent variable for two reasons. Firstly, using a monetary measure is consistent with theoretical discussion in the previous section. Further, using the total project value allows me to maximize the size of the estimation sample because it allows me to study both flat price and hourly priced projects. I re-estimate Equation (10) using alternative outcome measures in the robustness analysis section. Theoretically, employer uncertainty can manifest itself through two channels. It can be a caused by a lack of verified work experience on the platform. This type of time varying uncertainty is likely largely captured by the covariates controlling for work experience on the platform and ratings on past projects. Additionally, the employers might have difficulties evaluating the skills and reliability of a freelancer because their educational qualifications and offline work experience are earned from unknown schools and workplaces. Skill certificates will plausibly mitigate both types of employer uncertainty. An OLS estimate for

in (10) is likely endogenous for two reasons. First, as suggested

by (9), decision to earn certificates is driven by two types of unobservables: freelancer ability and employer uncertainty on freelancer ability. To account for this, I estimate 13

(10) using freelancer fixed effects. Comparing the OLS and fixed effects estimates also allows us to gauge the size and direction of the bias. In addition, to account for the possibility of any freelancer learning not captured by the fixed effects or time varying covariates, I limit my attention to a 14-day time window around the time of completing the skill test. Since learning a new skill such as a new programming language usually takes much longer than 14 days, limiting to a short time window allows me to make to assume that freelancer human capital would stay approximately constant.4 To examine potential heterogeneity between fixed price, and hourly wage projects, I estimate separate models for the types of projects. Since the regression model accounts includes fixed effects for different project types, all comparisons are done within project categories. Therefore, the marginal effects completing a skill certificate on earnings are calculated for projects within the same job category. The estimation results are presented in Table 2, where columns (1), (3), and (5) give estimates from our preferred fixed effects model for all projects and fixed price and hourly price projects separately. Columns (2), (4), and (6) give the corresponding estimates from standard OLS regressions. The fixed effects estimates reveal a positive return of roughly 2.5 - 3 log-points across different subsets of the data. A comparison between fixed effects and OLS specifications reveals that the OLS estimates are downward biased. In other words, the unobservable freelancer specific characteristics are negatively correlated with signaling. This implies that the freelancers who are in a disadvantaged position in the labor market also signal more. This observation is consistent with our model, which suggests that freelancers complete skill certificates in order to signal their productivity to potential employers. Furthermore, downward biased OLS estimates are inconsistent with the notion that more productive individuals would signal more because of their higher ability. Naturally, there could be other explanations which would imply a a positive association between completing skill tests and earnings and a negative association between completing skill tests and OLS regression residuals. I provide further evidence that the reason for positive returns to signaling is driven by increased employer information rather in the following sections. One should naturally be careful in making conclusions from non-significant differences between variables, but the fact that fact that hourly priced projects carry smaller earning premium for signaling compared to flat price projects is interesting. This observation is 4

I have also experimented with different time windows. The results are largely robust to changing the time window width (see section 5.2).

14

consistent with observations made by Agrawal et al. (2016) and Lin et al. (2016) who note that because hourly priced projects allow employers to monitor their freelancers, the employers have less need to pre-emptively screen them. Even if positive and statistically significant, the real returns to signaling are minimal. For instance, the median hourly rate for a hourly priced project is 10 dollars, which implies that the marginal return to signaling is only 25 cents. In the context of the theoretical model, this suggests that

is relatively close to 1, and employers do not learn

much more from observing workers’ skill certificates. This implies that freelancers also have a relatively small incentive to signal. This, in turn, is a plausible explanation for why the median number of completed certificates is only 2. Table 2: Returns to signaling. Dependent variable: All projects (1) Rating: 2nd quintile Rating: 3rd quintile Rating: 4rd quintile Rating: 5th quintile Competitiveness of project Num projects Cumulative dollars earned / 1000 Num certificates Freelancer fixed effects Observations R2 Adjusted R2

(2) ⇤

(log) project value Flat price (3)

(4) ⇤

⇤⇤⇤

Hourly (5)

(6)

0.128 (0.062) 0.337⇤⇤⇤ (0.062) 0.322⇤⇤⇤ (0.065) 0.243⇤⇤⇤ (0.048) 0.007⇤⇤⇤ (0.001) 0.003⇤⇤ (0.001) 0.008⇤⇤ (0.003) 0.030⇤⇤⇤ (0.006)

0.031 (0.042) 0.145⇤⇤⇤ (0.043) 0.112⇤⇤ (0.041) 0.070⇤ (0.032) 0.005⇤⇤⇤ (0.001) 0.001⇤ (0.001) 0.021⇤⇤⇤ (0.003) 0.002 (0.003)

0.176 (0.072) 0.358⇤⇤⇤ (0.071) 0.247⇤⇤ (0.076) 0.163⇤⇤ (0.057) 0.007⇤⇤⇤ (0.001) 0.002 (0.001) 0.006 (0.004) 0.027⇤⇤⇤ (0.008)

0.189 (0.046) 0.276⇤⇤⇤ (0.046) 0.210⇤⇤⇤ (0.045) 0.109⇤⇤ (0.034) 0.004⇤⇤⇤ (0.001) 0.001 (0.001) 0.020⇤⇤⇤ (0.003) 0.005 (0.004)

0.002 (0.140) 0.247 (0.145) 0.383⇤ (0.153) 0.347⇤⇤ (0.126) 0.001 (0.002) 0.006⇤⇤ (0.002) 0.010⇤ (0.004) 0.025⇤ (0.011)

0.197⇤⇤ (0.064) 0.063 (0.068) 0.002 (0.063) 0.125⇤ (0.053) 0.002 (0.001) 0.001 (0.001) 0.016⇤⇤⇤ (0.004) 0.005 (0.004)

Yes 26,788 0.616 0.440

No 26,788 0.142 0.139

Yes 16,551 0.664 0.473

No 16,551 0.136 0.131

Yes 10,237 0.701 0.430

No 10,237 0.153 0.144

Notes: Competitiveness of each project is measured by the number of applicants. All specifications include dummy variables for different project types and observation years. Standard errors are clustered on freelancer level. Omitted category: 1st quintile of rating distribution. Significance levels in all specifications: *** 0.1%, ** 1%, and * 5%.

15

4.2

Does skill testing increase freelancer productivity?

Previous discussion hinges on the assumption that signaling does not increase freelancer productivity. The online labor platform offers a particularly intuitive measure for freelancer productivity in the form of feedback ratings given by the employers to the workers. I use this measure as a measure for freelancer success in a project. Table 3 presents the results of a regression analogous to regression model (10), but with the feedback rating on the left hand side of the regression. The first three columns present the results from regression models where the feedback rating ranging between 1 and 5 as a dependent variable. The next three columns present an alternative specification in which the dependent variable gets a value of 1 if the feedback rating given to a freelancer is above 4.5, which is considered as ’good’.5 In both cases, the effect of signaling on ratings is statistically indistinguishable from zero. I interpret these results as giving support to the assumption that signaling does not increase freelancer productivity. This suggests that the underlying reason that signaling increases earnings is decreased employer uncertainty. So far, I have empirically established that completing skill certificates increases freelancer earnings, but the reason for this is not that the freelancers would be performing better in their projects after completing more skill tests. In the following subsection I proceed to arguing that the reason why completing skill certificates benefits freelancers is that it reduces uncertainty on freelancers’ expected productivity.

4.3

Does signaling increase information?

Establishing that signaling decreases employer uncertainty on freelancers’ producitivity is complicated by the fact that the information set of the employer, and therefore the uncertainty related to freelancer productivity, is unobservable to researchers. Nonetheless, Prediction 2 outlined in theory section suggests that the marginal effect of signaling is lower for high levels of information. Comparing the marginal effects of completing skill certificates with different levels of platform provided information allows me to empirically test whether Prediction 2 holds in the data. I use verified work experience as a proxy for information that employers have on freelancers. In this respect, I follow Agrawal et al. (2016) who demonstrate that verifiable work experience acts as a source of standardized information on freelancer quality. In their paper, they argue that verifiable work experience does not seem to increase the productivity of freelancers, and, further, disproportionately benefits freelancers who are statistically discriminated against. In the context of this paper, I argue that the effect 5

The cut-off for good feedback is adapted from Ghani et al. (2014) and Horton (2017).

16

Table 3: Effect of signaling on ratings. Dependent variable: Feedback score All projects Hourly Flat price Num projects Num certificates Freelancer fixed effects Observations R2 Adjusted R2

Feedback score > 4.5 All projects Hourly Flat price

(1)

(2)

(3)

(4)

(5)

(6)

0.0003 (0.0002) 0.003 (0.002)

0.001 (0.0005) 0.0003 (0.004)

0.0002 (0.0002) 0.005⇤ (0.002)

0.0001 (0.0001) 0.002 (0.001)

0.0002 (0.0003) 0.0005 (0.003)

0.0001 (0.0001) 0.003 (0.001)

Yes 20,847 0.542 0.307

Yes 7,181 0.707 0.392

Yes 13,666 0.559 0.288

Yes 20,847 0.485 0.221

Yes 7,181 0.648 0.270

Yes 13,666 0.544 0.264

Notes: In addition to the variables reported, all models include freelancer fixed effects, and average rating for completed projects, sum of previous projects completed on the platform and competitiveness of each project (measured by the number of applicants), and dummy variables for observation years and different project types. Standard errors are clustered on freelancer level. Significance levels in all specifications: *** 0.1%, ** 1%, and * 5%.

of work experience on productivity is negligible for exactly the same reason as I argued in the previous sections that the effect of signaling on productivity is negligible. This assumption is backed by the sample construction as I limit the estimation data to a short time interval around the skill certificate award and control for freelancer fixed effects. Moreover, the assumption gets empirical support from estimation results presented in Table 3, which shows that the effect of experience on star ratings is minimal. I implement a test for decreasing marginal returns to signaling for higher information in the form of a regression model of the form yik = ↵i + Xi + s + n + ⌘s ⇥ n + "ik .

(11)

The model is otherwise analogous to the specification (10) with the exception that I include the term s ⇥ n which is an interaction term between number of completed certificates s and verified work experience n. Due to skewed distribution of work experience, I also estimate models where I test for the differences in return to signaling between cases when n < N and n

N . These specifications allow for further nonlinearity between

in returns to signaling by different levels of experience. These are estimated by a linear regression model of the form yik = ↵i + Xi + s + nk + ⌘s ⇥ I [nk 17

N ] + "ik .

Where I [n

N ] is an indicator function getting value 1 when nk

N. As in the previous

sections, I estimate both regression models using freelancer fixed effects, and limit the estimation data to only include two weeks of data around the date of new certificate awards. This ensures that work n does not spuriously capture learning or other factors that might increase freelancer productivity in addition to increased information. Table 4 the regression results. Column (1) of Table 4 reports results from specification (11), where verified work experience is included in the regression as a continuous variable. Columns (2)-(5) of Table 4 report a models where verified work experience is introduced in a discretized form. The results across all specifications indicate that the estimate of ⌘is negative. Consequently, the returns to signaling are decreasing in level of information; the effect of signaling on earnings is smaller for more experienced freelancers. In particular, at around 25 completed projects, the sum

+ ⌘ is no longer distinguishable from zero

at conventional risk levels. In other words, when a freelancer has completed over 25 projects, on average, signaling does not increase productivity. This suggests that verified information provided by the platform and signals of freelancer productivity are to some extend substitutes. In particular, for high levels of platform provided experience, the effect of signaling on earnings is close to zero.

4.4

Decreasing returns to signaling

Finally, I turn to studying how returns to signaling vary with the level of signaling. Prediction 3. discussed in the previous section suggests that returns to signaling should be lower for freelancers who have completed more skill tests. I implement the test for decreasing returns to signaling in the form of the following regression model: yik = ↵i + Xi + s + I[s As in the previous section, I[s if s

N ] + "ik .

N ] is an indicator function which gets the value of 1

N . Introducing this term into the regression allows me to test for the possible

nonlinearity in return to signaling. Table 5 reports the estimation results. As evidenced by the consistently negative estimates for , the returns to signaling seem to be decreasing in s, as was suggested by theoretical model. Nonetheless, the estimated relationship is rather noisy. A partial explanation for the noisy estimates is that since most of the freelancers have completed only a very few skill tests (median number being 2), the effective sample sizes in the upper end of the certificate distribution are rather low. I find this reassuring, since 18

Table 4: Effect of signaling on earnings by different levels of experience. Dependent variable: (log) project value Num certificates Num certificates ⇥ Num projs completed Num certificates ⇥ > 5 projs completed

(1)

(2)

(3)

(4)

(5)

(6)

0.042⇤⇤⇤ (0.007) 0.0002⇤ (0.0001)

0.031⇤⇤ (0.010)

0.034⇤⇤⇤ (0.009)

0.037⇤⇤⇤ (0.008)

0.038⇤⇤⇤ (0.008)

0.039⇤⇤⇤ (0.008)

Num certificates ⇥ > 10 projs completed

0.010 (0.010)

Num certificates ⇥ > 15 projs completed

0.012 (0.009)

Num certificates ⇥ > 20 projs completed

0.018⇤ (0.009)

Num certificates ⇥ > 25 projs completed freelancer fixed effects Observations R2 Adjusted R2

Yes 26,788 0.617 0.441

Yes 26,788 0.619 0.444

Yes 26,788 0.618 0.443

Yes 26,788 0.618 0.442

Notes: In addition to the variables reported, all models include freelancer fixed effects, and average rating for completed projects, sum of previous projects completed on the platform and competitiveness of each project (measured by the number of applicants), and dummy variables for different project types. Standard errors are cluered on freelancer level. Significance levels in all specifications: *** 0.1%, ** 1%, and * 5%.

19

0.020⇤ (0.009)

Yes 26,788 0.617 0.442

0.023⇤ (0.010) Yes 26,788 0.617 0.442

I have estimated the returns to signaling being low, we would expect that freelancers would not invest much in signaling. An additional explanation for estimates for

being relatively close to zero is that

– in contrast to the theoretical model – different skill tests measure different things, and they are heterogeneous across several dimensions. Therefore, the marginal effect for completing an extra skill certificate might be higher, at least if consecutive tests are sufficiently different from one another. It is useful to recap the results so far. I have empirically established that freelancers who choose to complete skill certificates are negatively selected from the population of all freelancers, but the effect of completing skill certificates on earnings is positive and statistically significant after accounting for unobservable characteristics. Moreover, completing skill certificates does not seem to increase freelancer productivity when productivity is approximated by freelancer ratings. In addition, two channels of standardized information on freelancer productivity, verified work experience on the platform and skill certificates are partial substitutes. Finally, the empirical results suggest that monetary returns to completing skill certificates exhibit decreasing returns to scale. These four findings together are all consistent with theoretical model of freelancer signaling outlined in the previous section. Taken together, the empirical results are also inconsistent with an alternative model where the causal mechanism leading to higher returns would be increased freelancer productivity. Moreover, the monetary returns to signaling are economically small. In the context of the theoretical setup, this suggests that the monetary benefit from completing skill certificates does not outweigh the effort cost. The low return to signaling combined with a low median level of completed skill certificates per freelancer is consistent with an outcome resembling a ’pooling equilibrium’, where signaling is relatively non-informative of the freelancer quality. This implies that employers do not reward much for signaling. Most freelancers, realizing this, rationally choose not to invest much effort in signaling.

5

Robustness analyses

5.1

Alternative outcome measures

Even if the main outcome variable is the project size measured in dollars, it is not the only possible outcome. As a robustness check I re-estimate specification (10) using alternative measures for freelancer earnings. These results are presented in Table 6. Column (1) of Table presents the results of a model where the dependent variable is

20

Table 5: Effect of signaling on earnings by different levels of signaling. Dependent variable: (log) project value (1) Num certificates Num certificates ⇥

2 certificates

Num certificates ⇥

5 certificates

Num certificates ⇥

10 certificates

Num certificates ⇥

15 certificates

Num certificates ⇥

20 certificates

Freelancer fixed effects Observations R2 Adjusted R2

(2)

0.131 (0.083) 0.101 (0.084)

Yes 26,788 0.616 0.440

(3) ⇤⇤

0.040 (0.014) 0.021 (0.016)

Yes 26,788 0.616 0.441

⇤⇤⇤

0.048 (0.009)

0.035⇤ (0.015)

Yes 26,788 0.616 0.441

(4) ⇤⇤⇤

0.038 (0.005)

0.020⇤ (0.010)

Yes 26,788 0.617 0.441

(5) 0.035⇤⇤⇤ (0.005)

0.014 (0.013) Yes 26,788 0.616 0.440

Notes: In addition to the variables reported, all models include freelancer fixed effects, and average rating for completed projects, sum of previous projects completed on the platform and competitiveness of each project (measured by the number of applicants), and dummy variables for different project types. Standard errors are cluered on freelancer level. Significance levels in all specifications: *** 0.1%, ** 1%, and * 5%.

21

hourly wage. This variable is naturally only available for projects which are paid hourly rates. Again, the effect of signaling on wages is positive and statistically significant, even if smaller compared to specifications where total value of project is used as a dependent variable. Column (2) reports the results from a model where the log of freelancer hours worked are used as the dependent variable. In this case, the effect of signaling on hours is statistically insignificant, albeit positive. Columns (3) and (4) turn attention from earnings to the probability of working. They present results from models where one each 14-day pre-test and 14-day pro-test period counts as an observation. In column (3) the dependent variable is the number of observations in the period. In column (4), the dependent variable is a binary variable which gets a value of 1 whenever the number of projects completed within the time window is strictly larger than 0. A few points on these specifications are worth making. First, because observations are counts of potentially multiple projects, specifications (3) and (4) do not include project level control variables. Second, and more importantly, since my data does not include information on projects where the freelancer applied but wasn’t accepted, the results might be confounded by effort. In particular, if the probability of bidding for projects is higher just after completing a skill certificate is larger than the probability of bidding for projects just before completing a skill certificate, the estimate for signaling might be upward biased. If this is the case, the estimates reported on columns (3) and (4) represent an upper limit for the true causal effect of signaling on the probability of working. Notwithstanding this caveat, I find that, on average freelancers win roughly 0.024 more projects – corresponding to a 8% increase – in a 2-week period after completing a skill test compared to a 2 week period just before completing a skill test. Analogously, the effect of signaling on probability of working is 5%. All in all, regressions with alternative outcome measures give support to main estimates reported in Section 4.1. The effect of signaling on success in the online labor market is positive, but relatively small.

5.2

Varying time bandwidth

To demonstrate that I have not cherry-picked the day bandwidth around the time of certificate award to be 14 days, s, I re-estimate (10) using various bandwidths with log of total project value as the dependent variable. The results are reported in Figure 3. As the time window around the certificate award narrows, the sample sizes get smaller, and, consequently, the estimates get noisier. Nonetheless, the estimates are very close to

22

Table 6: Returns to signaling (alternative outcome measures). Dependent variable: Hourly wage

Hours worked

Number of projects

Probability of working

(1)

(2)

(3)

(4)

Num certificates

⇤

0.010 (0.004)

Baseline Freelancer fixed effects Observations R2 Adjusted R2

Yes 10,237 0.915 0.838

0.015 (0.010)

⇤⇤⇤

0.024 (0.004)

0.008⇤⇤⇤ (0.001)

Yes 10,237 0.657 0.346

0.285 Yes 121,121 0.565 0.402

0.177 Yes 121,121 0.626 0.485

Notes: In addition to the variables reported, all models include freelancer fixed effects, average rating for completed projects, sum of previous projects completed and dummy variables for observation years. Specifications (1) and (2) also include measures for project competitiveness (measured by number of applicants), and project type dummies. In specifications (3), and (4) ’Baseline’ is the average of the dependent variable for the 14 day window just before completing the skill test. Standard errors are clustered on freelancer level. Significance levels in all specifications: *** 0.1%, ** 1%, and * 5%.

one another for all bandwidths between 14 and 60 days.

5.3

Contactor-month fixed effects

Since some freelancers are completing several skill tests over a longer period of time, it is possible that despite controlling for freelancer fixed effects, some freelancer productivity growth taking place between two skill tests would bias the estimates for monetary return to signaling. For example, this could happen if a freelancer has taken a skill test in 2014, thereafter learns a new programming language, and takes a second skill test in 2016. To account for this, I re-estimate (10) using freelancer-month fixed effects. The results are reported in Table 7. The results are consistent with those presented in 2. Naturally, since freelancer-month fixed effects are much more costly in degrees of freedom, the standard errors of the main estimates are larger.6 6

For the sama reason, the main estimates in this paper presented in this paper use freelancer instead of freelancer-month fixed effects. In particular, since freelancer-month fixed effects subsume most of the variation in the main explanatory variables of interest, the results concerning experience-signaling interactions in Section 4.3 and nonlinearity in return to signaling presented in Section 4.4 are insignificant and have very small point estimates when estimated using freelancer-month fixed effects.

23

0.08

0.06

Marginal return to signaling

●

● ●

0.04

● ● ●

●

● ●

● ● ●

● ●

● ●

● ● ● ● ● ●

● ● ●

● ● ●

●

●

● ● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0.02

0.00 0

5

10

15

20

25

30

35

40

45

50

55

60

Bandwidth (days)

Figure 3: Effect of varying bandwidth on the estimate for marginal return to signaling. The dots represent the point estimates for different bandwidths; figure also includes 95% confidence intervals calculated by +/ 1.96 ⇥ std. err.

24

Table 7: Returns to signaling (freelancer-month FE’s). Dependent variable: (log) project value All projects Flat price (1)

(2)

(3)

Num certificates

⇤

0.032 (0.015)

0.026 (0.016)

0.044 (0.036)

Freelancer-month fixed effects Observations R2 Adjusted R2

Yes 26,788 0.802 0.574

Yes 16,551 0.814 0.576

Yes 10,237 0.888 0.620

Notes: Competitiveness of each project is measured by the number of applicants. In addition to the variables reported, all models include freelancer-month fixed effects, average rating for completed projects, sum of previous projects completed on the platform and competitiveness of each project (measured by the number of applicants), and dummy variables for observation years and different project types . Standard errors are clustered on freelancer level. Significance levels in all specifications: *** 0.1%, ** 1%, and * 5%.

6

Discussion and conclusions

This paper studies the causal effect of skill certificates on the earnngs of online freelancers. After accounting for unobservable freelancer quality, and the accumulation of human capital, I find a positive and statistically significant effect of signaling on earnings. Completing skill tests is not found to increase worker productivity, and is found to be a substitute, rather than a complement, to other forms of standardized information provided by the platform. Together, these findings suggest that the reason why completing skill certificates increases earnings is that they decrease employer uncertainty rather than teach new skills to the freelancer. The findings imply that workers can successfully use skill certificates to signal their skills to employers. Unfortunately to both sides of the market, the effect of signaling on earnings is quite small, only around 2-3 % depending on the specification and subset of data used. This suggests that completing skill certificates only has a limited – albeit positive – impact on employer uncertainty on freelancer quality. The estimation results suggest that because the monetary return to signaling is so small, it is always more valuable to work than to complete skill certificates. As a result, most freelancers choose other methods of convincing the employers of their ability. This result has clear implications to the platform operators: to improve the informativeness of skill test signals, the tests should be made more challenging. This would

25

allow the skilled freelancers to separate themselves from non-skilled ones while increasing freelancers’ incentives to signal. More generally, and despite the small return estimates, I find the results from this paper rather encouraging. This paper suggests that it is possible for online labor platforms to build digital institutions which help new workers break into the online labor market. A further implication from the findings of the current paper is that skill certification schemes can decrease information asymmetries and help skilled members of statistically discriminated against groups such as immigrants and other minorities to thrive in traditional labor markets, at least if they are informative enough.

References Agrawal, A., J. Horton, N. Lacetera, and E. Lyons (2015, April). Digitization and the Contract Labor Market: A Research Agenda, pp. 219–250. University of Chicago Press. Agrawal, A., N. Lacetera, and E. Lyons (2016). Does standardized information in online markets disproportionately benefit job applicants from less developed countries? Journal of International Economics 103, 1 – 12. Altonji, J. G. and C. R. Pierret (2001). Employer Learning and Statistical Discrimination. The Quarterly Journal of Economics 116 (1), 313–350. Arcidiacono, P., P. Bayer, and A. Hizmo (2010). Beyond signaling and human capital: Education and the revelation of ability. American Economic Journal: Applied Economics 2 (4), 76–104. Autor, D. H. (2001). Wiring the Labor Market. Journal of Economic Perspectives 15 (1), 25–40. Autor, D. H. and D. Scarborough (2008). Does Job Testing Harm Minority Workers? Evidence from Retail Establishments. Quarterly Journal of Economics 116, 1409–1448. Bassi, V. and A. Nansamba (2017). Information Frictions in the Labor Market: Evidence from a Field Experiment in Uganda. Bertrand, M. and S. Mullainathan (2004). Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination. American Economic Review 94 (4), 991–1013. 26

Ghani, E., W. Kerr, and C. Stanton (2014). Diasporas and Outsourcing: Evidence from oDesk and India. Management Science 60 (7), 1677–1697. Hoffman, M., L. Kahn, and D. Li (2015, nov). Discretion in Hiring. Horton, J. J. (2017). The Effects of Algorithmic Labor Market Recommendations: Evidence from a Field Experiment. Journal of Labor Economics 35 (2). Horton, J. J. and J. M. Golden (2015). Reputation Inflation in an Online Marketplace. Kässi, O. and V. Lehdonvirta (2016). Online Labour Index: Measuring the Online Gig Economy for Policy and Research. MPRA Paper 74943, University Library of Munich, Germany. Katz, E. and S. Oded (1987). International Migration under Asymmetric Information. The Economic Journal 97 (387), 718–726. Kuek, S. C., C. Paradi-Guilford, T. Fayomi, S. Imaizumi, and P. Ipeirotis (2015). The Global Opportunity in Online Outsourcing. Washington, D.C.: World Bank Group. Lahey, J. N. (2008). Age, Women, and Hiring: An Experimental Study. Journal of Human Resources 43 (1), 30–56. Lang, K. and M. Manove (2011). Education and Labor Market Discrimination. The American Economic Review 101 (4), 1467–1496. Lin, M., Y. Liu, and S. Viswanathan (2016). Effectiveness of Reputation in Contracting for Customized Production: Evidence from Online Labor Markets. Management Science (January 2017), mnsc.2016.2594. Lundberg, B. S. J. and R. Startz (1983). Private discrimination and social intervention in competitive labor market. American Economic Review 73 (3), 340–347. Malone, T. W. and R. J. Laubacher (1998). The dawn of the e-lance economy. Harvard business review 76 (5). Nosko, C. and S. Tadelis (2015). The Limits of Reputation in Platform Markets: an Empirical Analysis and Field Experiment. Oreopoulos, P. (2011). Why do skilled immigrants struggle in the labor market? A field experiment with thirteen thousand resumes. American Economic Journal: Economic Policy 3, 148–171. 27

Pallais, A. (2014). Inefficient Hiring in Entry-Level Labor Markets. American Economic Review 104 (11), 3565–3599. Pinkston, J. C. (2003). Screening discrimination and the determinants of wages. Labour Economics 10 (6), 643–658. Spence, M. (1973). Job Market Signaling. The Quarterly Journal of Economics 87 (3), 355–374. Stanton, C. T. and C. Thomas (2015, sep). Landing the First Job: The Value of Intermediaries in Online Hiring. The Review of Economic Studies 83 (2), 810–854. Terviö, M. (2009). Superstars and mediocrities: Market failure in the discovery of talent. The Review of Economic Studies 76 (2), 829–850. Tyler, J. H., R. J. Murnane, and B. Willett, John (2000). Estimating the Signaling Value of the GDE. Quarterly Journal of Economics 115 (2), 431–468. Yoganarasimhan, H. (2013). The Value of Reputation in an Online Freelance Marketplace. Marketing Science 32 (6), 860–891.

28