# Sample size, P-values (its relationship), and data visualization with plots. ggplot and T test

We know that P-values (within t-test context as an example..) is highly sensitive to sample size. A larger sample will yield a smaller p-value remaining everything else constant. On the other hand, Cohen´s d effect size remains the same.

I'm inspired in this code here, but I´ve changed some parts to make the difference between means constant, instead of creating a random variable based on a normal distribution.

Although everything is working, I do imagine that some of the experts in this community could improve my syntax.

library(tidyverse)

ctrl_mean <- 8
ctrl_sd <- 1

treated_mean <- 7.9
treated_sd <- 1.2

sample <- numeric() #criar vetor para grupar resultados
nsim <- 1000 #criar variavel
t_result <- numeric()

for (i in 1:nsim) {
set.seed(123)
t_result[i] <- (mean(ctrl_mean)-mean(treated_mean))/sqrt((ctrl_sd^2/(i))+(treated_sd^2/(i))) #manual t test
sample[i] <- i # number of participants
}
ds <- data.frame(
sample = sample, #assign the sample size
t_result = round(t_result,3), #get the t test result
degrees = sample*2-2) #compute the degrees of freedom

ds %>%
filter(sample>1) %>%
mutate(P_Value = 2*pt(abs(t_result), df=degrees,lower.tail=FALSE)) %>%
left_join(ds,.) -> ds

#plot
ggplot(ds, aes(x=sample, y=P_Value)) +
geom_line() +
annotate("segment", x = 1, xend=sample, y = 0.05, yend = 0.05, colour = "purple", linetype = "dashed") +
annotate("segment", x = 1, xend=sample, y = 0.01, yend = 0.01, colour = "red", linetype = "dashed") +
annotate("text", x = c(1,1), y=c(.035,.001), label = c("p < 0.05", "p < 0.01"))


In this code you do not need the loop:

sample <- 1:nsim
t_result <- (mean(ctrl_mean)-mean(treated_mean)) /
sqrt((ctrl_sd^2/(sample))+(treated_sd^2/(sample)))
# OR:
t_result <- (mean(ctrl_mean) - mean(treated_mean)) /
sqrt((ctrl_sd^2 + treated_sd^2) / sample)


why the set seed?

• Thank you. Easier than I was imagining. Set.seed, in this case, was an error, thanks for highlighting that. – Luis Mar 23 at 5:27