{"data":{"site":{"siteMetadata":{"courseId":"metaanalysis"}},"markdownRemark":{"htmlAst":{"type":"root","children":[{"type":"element","tagName":"exercise","properties":{"id":"1","title":"Meta-analysis, evidence summary & assessment of certainty"},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Once all the data are prepared, we can run our first meta-analytical model and assess the overall effect that we are interested in, across studies: the mean effect size, weighted by the precision of the observed effect sizes. We will assess the effect based on the model summary and, visually,  with forest plots. Since we probably have other variables that could potentially explain variation in our data, we will try to account and test some of these other variables in slightly more complex models. "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We are most interested in being able to make a statement on differences between treatment and control which are based on differences in biological response, and not on methodological or other technical differences between studies or study types, for example. We will go through some ways of assessing the influence of heterogeneity in our data, and whether we have any indication of publication bias in our data set.    "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"This is all best done hands-on:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"Worked Example"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Using a subset of data (we decided to only go for fish) from a recent meta-analysis that investigated the relationship of dietary intervention on sperm traits across the animal kingdom (Macartney et al. 2019), we will go through the steps explained previously."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We are using the fish data that Erin had already collected - i.e. all relevant data have already been either extracted from the figures, or calculated from inferential statistics. This means we have a detailed datasheet available that contains a lot of information, but most importantly sample sizes, means and variances for control and treatment groups, for quite a range of different sperm traits. We have prepared this in such a way that we can use this to start from calculating effect sizes."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://github.com/SusZaj/metaanalysis/blob/master/images/computertaskicon.svg?raw=true"},"children":[]},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nWe downloaded the two deposited data files from "},{"type":"element","tagName":"a","properties":{"href":"https://osf.io/aqw8u/"},"children":[{"type":"text","value":"osf.io"}]},{"type":"text","value":" and created a file that we will use here ("},{"type":"element","tagName":"a","properties":{"href":"https://github.com/SusZaj/metaanalysis/blob/master/binder/data/Macartney2019_processed_fish.csv"},"children":[{"type":"text","value":"click here"}]},{"type":"text","value":" to see / download the source .csv from github). Alternatively, run the code block - but that may look a little messy, due to the width of the data frame (the number of variables / columns and their length), especially if you are not used to investigating the data structure of files in R yet."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_1"},"children":[{"type":"text","value":"\nMost of the variables will be self explanatory. Some aren't. We describe the ones that are important for our analysis below in more detail.\n"}]},{"type":"text","value":"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"},{"type":"element","tagName":"table","properties":{},"children":[{"type":"element","tagName":"thead","properties":{},"children":[{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"th","properties":{},"children":[{"type":"text","value":"Variable name"}]},{"type":"element","tagName":"th","properties":{},"children":[{"type":"text","value":"Description"}]}]}]},{"type":"element","tagName":"tbody","properties":{},"children":[{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"author"}]},{"type":"text","value":" to "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"volume"}]}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"Publication information of specific studies"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"commonname_species"}]},{"type":"text","value":" to "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"species_name"}]}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"Study organism identification"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"trait_measured"}]},{"type":"text","value":" and "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"trait_unit"}]}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"Trait information"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"trait_category"}]}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"(see description below this table)"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"mean_low"}]},{"type":"text","value":" to "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"n_high"}]}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"Descriptive statistics (extracted from literature)"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"f"}]},{"type":"text","value":" and "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"t"}]}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"Inferential statistics (extracted from literature)"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"es_method"}]}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"Descriptive / inferential"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"effectsize_id"}]}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"Consecutive numbers for data manipulation (1 to 37)"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Trait category"}]},{"type":"text","value":" description"}]}]},{"type":"text","value":"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"},{"type":"element","tagName":"table","properties":{},"children":[{"type":"element","tagName":"thead","properties":{},"children":[{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"th","properties":{},"children":[{"type":"text","value":"ID number"}]},{"type":"element","tagName":"th","properties":{},"children":[{"type":"text","value":"Category"}]}]}]},{"type":"element","tagName":"tbody","properties":{},"children":[{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"1"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"bodysize"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"2"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"sperm length"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"3"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"spermatophore/testis size"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"4"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"sperm movement (velocity, % motile)"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"5"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"sperm number"}]}]},{"type":"element","tagName":"tr","properties":{},"children":[{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"6"}]},{"type":"element","tagName":"td","properties":{},"children":[{"type":"text","value":"sperm viability (alive vs dead, normal vs abnormal)"}]}]}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"As said above, before we can analyse these data, we have to calculate the effect sizes. We will use the library "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"compute.es"}]},{"type":"text","value":" to calculate SMD as "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Cohen’s d"}]},{"type":"text","value":". We have four different types of data to base our calculations on: 1) Descriptive statistics with SD, 2) Descriptive statistics with SE, 3) F-values (inferential statistic), and 4) t-values (inferential statistic). "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Different packages have different requirements in input variables and in syntax. "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"compute.es"}]},{"type":"text","value":" requires SD and doesn’t take SE. That’s why we have to calculate SD from SE in one of the steps (you can find the simple formula in the code below - ).\nIt also turns out that "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"compute.es"}]},{"type":"text","value":" needs an id variable to do calculations for more than one provided value, which we provide (id= effectsize_id). You’ll find yourselves in situations where something might not work immediately. Even the R package manual might not be able to help. Due to the huge R user base, you can often find solutions to very specific problems by googling them. Sometimes you have to find the solution yourself, by trying out possible versions. This was the case for us with the need to provide the id-argument in the following code (you can try what happens when you leave it out)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_2b"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"You will notice that you will not get the complete data set printed here - that is because we haven’t asked for it. You could look at the data by adding a last line: “str(fish.all)” or just call: “fish.all” and see what happens."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Notice also that we added a quick plot of the data and calculated the number of studies, traits per study organism, and also the mean / average of all effect sizes. So why can’t we in theory stop here, with the overall mean effect size in hand, maybe calculate confidence intervals and that’s it? "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"choice","properties":{"id":"1"},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"opt","properties":{"text":"In theory, we could stop here, as any additional steps won't have a huge impact on the value of the mean."},"children":[{"type":"text","value":" \nThat would be very easy, but would disregard all the other information we have on the data underlying the observed effect sizes. \n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"opt","properties":{"text":"The whole point of a meta-analysis is to not only calculate the mean of the effect sizes, but to take precision of the underlying effect size estimates into account and produce a more meaningful weighted mean effect size.","correct":"true"},"children":[{"type":"text","value":" \n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Let’s run a first model."}]},{"type":"text","value":" This is called a random effects model (in contrast to a fixed effects model, which is unlikely to be used by you and will therefore not be discussed here)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_3"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"What does this output mean?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"First, we get a few estimates related to heterogeneity (tau^2, …). Heterogeneity in meta-analysis is important, but right now, this model does not account for potential effects due to different traits, species, and studies. This is why we will only look at the plot and discuss heterogeneity further down. "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We produced a forest plot. Nice, but before we go further: what about that value around 3? Whenever there seem to be outliers, always check. Indeed, we may have gotten some clue to double-check study 6 in the plot "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Effect sizes within study"}]},{"type":"text","value":" above. Sometimes it’s biological, but very often outliers turn out as errors (e.g. through mistakes in data entry or data re-coding)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Let’s look at this data point, labelled (wrongly so far) study 23 then:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_4"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"This is the only study with a provided F-value test statistic. Since F values are the ratios of two variances, they have to be positive.\nThis means we wouldn’t be able to see a negative effect size for this statistic. We have to go back and see what was exactly done in the original publication to add information to the F-value on whether it refers to an increase or a decrease in trait size (here: body mass)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"In "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"compute.es"}]},{"type":"text","value":" we added “low” diet values as first and “high” values as second data points. This means that the program calculated low - high, leading to negative values, at least in the case of body mass. Unsurprisingly, “high” diet leads to larger body size.\nO’Dea found the same. This means, to have the same direction of effect as in the other studies that used means and SD/SE, we have to change sign in that study.\nBefore we do that, what about the other effect sizes that we calculated based on inferential test statistics?"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_5"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"t"}]},{"type":"text","value":"-values can be negative (they are the difference between treatment and control means, divided by SE). However, are the means ordered in the same way as are our other means to calculate the difference, i.e. mean(low treatment) - mean(high/control treatment)? We have to check in Evans 2017. They also found that high diets led to larger trait sizes than low diets, which means we have to change the sign of those effect sizes as well."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We can do this as follows:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_6"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Ok, great! The effect sizes are in the correct direction now."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Now, let’s start again and re-run the first meta-analysis. Let’s also correct the labels (using the "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"slab"}]},{"type":"text","value":" argument in the "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"rma"}]},{"type":"text","value":" function call."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_7"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We continue to ignore heterogeneity for now. The forest plot looks nice now. What we see is a grouping by study."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"To see the results grouped by trait category, instead of grouped by study, we can re-order the data and print (here without saving the data in the other format)"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_8"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"This may be a more logical way to present our results. On top we see the traits related to body size, then sperm length, followed by sperm swimming speed, number, and viability. "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Building a more meaningful model"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Because of non-independence of data, we add random effects:"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"ul","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"between-study effects ("},{"type":"element","tagName":"code","properties":{},"children":[{"type":"text","value":"random = ~ 1 | study_id"}]},{"type":"text","value":"), "}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"within-study effects ("},{"type":"element","tagName":"code","properties":{},"children":[{"type":"text","value":"random = ~ 1 | effectsize_id"}]},{"type":"text","value":"), and "}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"li","properties":{},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"phylogenetic effects; we use "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"genus"}]},{"type":"text","value":" as a simpler substitute here ("},{"type":"element","tagName":"code","properties":{},"children":[{"type":"text","value":"random = ~ 1 | genus"}]},{"type":"text","value":")"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"img","properties":{"src":"https://github.com/SusZaj/metaanalysis/blob/master/images/pushpin.svg?raw=true"},"children":[]},{"type":"text","value":"  If you are working with data from many taxa, you may want to add your phylogenetic tree here. "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_9"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Let’s look at the results in more detail now. "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We have a table "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"Variance components"}]},{"type":"text","value":" with the estimates of our random effects ("},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"study_id"}]},{"type":"text","value":", "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"effectsize_id"}]},{"type":"text","value":", "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"genus"}]},{"type":"text","value":"). We see that "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"genus"}]},{"type":"text","value":" is no source of variation (estimate = 0.00). This means we could drop the term from the model. Variation within studies, due to measuring different traits, is more than 6 times larger than variation between studies."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"The test for heterogeneity (Cochran’s Q-test: "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Q"}]},{"type":"text","value":" = 698.82, "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"df"}]},{"type":"text","value":" = 36, "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"p"}]},{"type":"text","value":" < 0.01) suggests substantial residual heterogeneity. Heterogeneity is the variability in our data, due to differences in experimental design, methods, data collection etc. We can look at heterogeneity using I^2, a measure that describes the amount of total variation explained by heterogeneity. For the multivariate "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"metafor"}]},{"type":"text","value":" function "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"rma.mv"}]},{"type":"text","value":", there is unfortunately no I^2 summary output.\nLuckily, "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"metafor"}]},{"type":"text","value":" creator Wolfgang Viechtbauer provides the following function to calculate I^2:\n"},{"type":"element","tagName":"code","properties":{},"children":[{"type":"text","value":"\nI2 <- function(model){\nW <- solve(model$V)\nX <- model.matrix(model)\nP <- W - W %"},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"% X %"}]},{"type":"text","value":"% solve(t(X) %"},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"% W %"}]},{"type":"text","value":"% X) %"},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"% t(X) %"}]},{"type":"text","value":"% W\nI2"},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"total <- sum(model$sigma2) / (sum(model$sigma2) + (model$k - model$p) / sum(diag(P)))\nI2"}]},{"type":"text","value":"each  <- model$sigma2 / (sum(model$sigma2) + (model$k - model$p) / sum(diag(P)))\nnames(I2"},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"each) = paste0(“I2"}]},{"type":"text","value":"”, model$s.names)\nI2s <- c(I2"},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"total = I2"}]},{"type":"text","value":"total, I2_each)\nI2s\n}\n"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Let’s try this:\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_10"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Our I^2 total is 95.4%. This is the sum of the heterogeneity due to between-study (13.3%), within-study (82.1%), and "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"genus"}]},{"type":"text","value":" (with <0.001 negligible). If you run the model again without the specified random effects (using "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"rma"}]},{"type":"text","value":" instead of "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"rma.mv"}]},{"type":"text","value":"), you will see that I^2 (total heterogeneity / total variability) matches this value."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"At last, the model results!"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"The estimate for mean effect size (across all studies, traits,…) is -0.8173. It is highly significant, as indicated by the "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"p"}]},{"type":"text","value":"-value, the ci that do not overlap zero, and the three stars. This means that there is an overall strong effect of diet, leading to decreased trait values under low diet in fish."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h2","properties":{},"children":[{"type":"text","value":"Meta-regression: Effects of moderators"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"In a next analysis,w e can ask whether potential predictor / moderator variables have an effect. Here, we want to know whether traits in different "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"trait categories"}]},{"type":"text","value":" respond differently to diet manipulation. We simply add: "},{"type":"element","tagName":"code","properties":{},"children":[{"type":"text","value":"mods = ~ factor(trait_category)-1"}]},{"type":"text","value":"."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_10"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h3","properties":{},"children":[{"type":"text","value":"Publication bias"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Publication bias is commonly observed in academic research. This is due to the fact that studies with significant results are often more likely to be published than studies with null results (despite equal quality of execution and design)."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Publication bias can be assessed visually with funnel plots. If there is no indication of publication bias, studies (shown as single dots) will be plotted in a funnel shaped distribution. A deviation from this shape can indicate publication bias."}]},{"type":"text","value":"\n"},{"type":"element","tagName":"codeblock","properties":{"id":"fish_11"},"children":[{"type":"text","value":"\nNo hints or solution necessary here.\n"}]},{"type":"text","value":"\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"exercise","properties":{"id":"2","title":"Report and interpret results"},"children":[{"type":"text","value":"\n"},{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"Prettier plots"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Wile forest plots are the traditional way for showing what is going on and may be useful (at least if ordered by effect sizes - see “caterpillar plots on the “orcharRd” page!), there are now better ways to visualise your metanalytical data.  In particular, “orchaRd” allows visualisation grouped by treatments , taxonomic groups or whatever your moderators may be. "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Please take some time to explore Dan Noble’s page:\n"},{"type":"element","tagName":"a","properties":{"href":"https://daniel1noble.github.io/orchaRd/"},"children":[{"type":"text","value":"orchaRD"}]}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Here you can find a lot of extra information on running and interpreting meta-regressions, and different ways for visualising for example continuous data, overall outcomes, or moderator-influenced effects (bubble plots, caterpillar plots)."},{"type":"element","tagName":"br","properties":{},"children":[]},{"type":"text","value":"\nIn case of severe publication bias, the visuals for bias-corrected mean plots may be something you may want to consider to include in your work!\n"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"h1","properties":{},"children":[{"type":"text","value":"Writing it up"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Hopefully your orchaRd plots and model outputs allow you to interpret your research question!"}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"The "},{"type":"element","tagName":"a","properties":{"href":"http://prisma-statement.org/documents/PRISMA%202009%20checklist.pdf"},"children":[{"type":"text","value":"PRISMA checklist"}]},{"type":"text","value":", which we used as our framework for conducting a systematic review, will give a good overview of all the things you need to report in or alongside your manuscript. "}]},{"type":"text","value":"\n"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"The full-text screening of your included studies should have given you a very solid overview over the general background of your research question - so summarising this into a polished paper should be easy\n"},{"type":"element","tagName":"img","properties":{"src":"https://github.com/SusZaj/metaanalysis/blob/master/images/wink.png?raw=true"},"children":[]},{"type":"text","value":" "}]},{"type":"text","value":"\n"}]}],"data":{"quirksMode":false}},"frontmatter":{"id":4,"title":"Chapter 4: Meta-analysis","description":"The meta-analysis & quality checks","next":"/chapter5","prev":"/chapter3"}}},"pageContext":{"isCreatedByStatefulCreatePages":false,"slug":"/chapter4"}}