Sample size, P-values (its relationship), and data visualization with plots. ggplot and T test

Question

We know that P-values (within t-test context as an example..) is highly sensitive to sample size. A larger sample will yield a smaller p-value remaining everything else constant. On the other hand, Cohen´s d effect size remains the same.

I'm inspired in this code here, but I´ve changed some parts to make the difference between means constant, instead of creating a random variable based on a normal distribution.

Although everything is working, I do imagine that some of the experts in this community could improve my syntax.

library(tidyverse)

ctrl_mean <- 8
ctrl_sd <- 1

treated_mean <- 7.9
treated_sd <- 1.2

sample <- numeric() #criar vetor para grupar resultados
nsim <- 1000 #criar variavel
t_result <- numeric()

for (i in 1:nsim) { 
  set.seed(123) 
  t_result[i] <- (mean(ctrl_mean)-mean(treated_mean))/sqrt((ctrl_sd^2/(i))+(treated_sd^2/(i))) #manual t test
  sample[i] <- i # number of participants
}
ds <- data.frame(
  sample = sample, #assign the sample size
  t_result = round(t_result,3), #get the t test result
  degrees = sample*2-2) #compute the degrees of freedom

ds %>% 
  filter(sample>1) %>% 
  mutate(P_Value = 2*pt(abs(t_result), df=degrees,lower.tail=FALSE)) %>% 
  left_join(ds,.) -> ds

#plot 
ggplot(ds, aes(x=sample, y=P_Value)) +
  geom_line() +
  annotate("segment", x = 1, xend=sample, y = 0.05, yend = 0.05, colour = "purple", linetype = "dashed") +
  annotate("segment", x = 1, xend=sample, y = 0.01, yend = 0.01, colour = "red", linetype = "dashed") +
  annotate("text", x = c(1,1), y=c(.035,.001), label = c("p < 0.05", "p < 0.01"))

Answer 1

In this code you do not need the loop:

sample <- 1:nsim
t_result <- (mean(ctrl_mean)-mean(treated_mean)) /
  sqrt((ctrl_sd^2/(sample))+(treated_sd^2/(sample)))
# OR:
t_result <- (mean(ctrl_mean) - mean(treated_mean)) /
  sqrt((ctrl_sd^2 + treated_sd^2) / sample)

why the set seed?