Reference 1

Unlike a correlational analysis, which just looks for relation between variables, a regression looks at how much a change in one variable (say h-index) brings about a change in another variable (say odds of success). Because our dependent variable, success, is dichotomous, the regression needs to be logistic (as opposed to linear). We used multiple regression so that we could look at the effect of all the variables at once. Multiple regression looks for how much one variable brings about a change in another when all the other variables are held constant. For example how much does success rate change with changes in h-index when total number of citations is held constant. In doing so it allows us to determine the unique contribution each variable makes, something that is particularly important when the variables are likely to be correlated, as number of citations and h-index are likely to be. Finally a backwards regression is a form of multiple regression where all the variables are entered, if some do not independently explain a significant amount of the variance in the dependent variable then the one which is least significant is removed. The process is then repeated until all the variables significantly independently explain variance in the dependent variable.

Reference 2

Our data had limitations. Often, we only had access to a draft of the documents meaning information was sometimes missing. More generally the conventions regarding the sort of information presented varied over time. For example, reporting h-index became more common in more recent applications. Our sample is also not random, and different faculties or departments have different criteria for sending us their applications. Some send us all of their applicants, others send their least competitive and use the review as a training exercise, while others send us only their most competitive. Finally, we were not always able to accurately measure the variables. While some people would mention that they had media engagements, others would provide a precise number and some would not mention media engagements at all.

Reference 3

Because of concerns about the accuracy of our data for media engagements we created a binary variable (mention vs no mention of media engagements). This variable turned out to be not significant.

Reference 4

PhD Supervision: X-squared = 3.9852, df = 1, p-value = 0.0459

Honours/Masters Supervision: X-squared = 4.3543, df = 1, p-value = 0.03692

Patents: Not funded (M=27.30, SD = 16.76) was significantly less t(231) = -1.70, p=0.09 than funded (M=32.33, SD=16.76)

Total Publications: Not funded (M=389.85, SD=497.49) was significantly lower t(170)=-2.52, p=0.013 than funded (M=657.01 , SD=746.51)

Note that the number of citations was highly positively skewed with the mean (441) being much larger than the median (242)

Total Citations: In our model we used the (natural) log of citations because the data was positively skewed (as evidenced by the fact that the mean (442.67) was much larger than the median (242)).

H-index: Not funded (M=9.07, SD=4.39) was significantly lower t(152)=-2.77, p=0.006 than funded (M=11.72 , SD=5.59)

Not funded (M=389.85, SD=497.49) was significantly lower t(170)=-2.52, p=0.013 than funded (M=657.01 , SD=746.51)

Note that the number of citations was highly positively skewed with the mean (441) being much larger than the median (242otal CH-index:

Reference 5

We did not have enough data to include the panels in our analysis.

Reference 6

The model we have developed is purely predictive – our criteria for determining whether variables are to be included was simply how much of a change in odds of success a change in the variable brings about. The important point is that this is in no way an explanatory model (which is why we can run so many tests on the same data). We are finding out the relation between the variables, but not the way in which they are able to have the effect. While we can say in our model the pattern is an increase of 1 in your h-index increases your odds of being funded by 12%, we cannot say that this is because an increase in h-index causes you to be more successful or that someone’s success is explained in part by the size of their h-index. Our data, and our analysis, cannot support such a claim.

Reference 7

von Hippel, T., & von Hippel, C. (2015). To Apply or Not to Apply: A Survey Analysis of Grant Writing Costs and Benefits. PLOS ONE, 10(3), e0118494. https://doi.org/10.1371/journal.pone.0118494

Reference 7

von Hippel, T., & von Hippel, C. (2015). To Apply or Not to Apply: A Survey Analysis of Grant Writing Costs and Benefits. PLOS ONE, 10(3), e0118494. https://doi.org/10.1371/journal.pone.0118494