I'm still trying to wrap my mind on what a confidence interval supposed to be.
Say for example the mean height for all males on my campus is 6' with a standard deviation of 1" the 95% confidence interval would be (5'10", 6'2")
Does that mean "we actually don't know what the mean value is, but we are 95% sure is it in this interval" ?
95% confidence is 2 standard deviations for a normal distribution.
We are 95% confident that the population mean ยต lies between 5'10" and 6'2".
You are estimating population parameters with sample statistics. The part about not knowing the true value of population parameters is implicit.
>>7703576
>Does that mean "we actually don't know what the mean value is, but we are 95% sure is it in this interval" ?
Yes. When you expand the confidence interval to 98%, 99% or even 99.9%, this results in your interval being wider because you're more sure that the mean falls within that range.
>>7703599
This.
>Does that mean "we actually don't know what the mean value is, but we are 95% sure is it in this interval" ?
No. Confidence intervals as people usually describe them are frequentist constructs, and belief or surety does not enter the picture. You can construct confidence intervals for Bayesian estimates but usually if you're talking about a Bayesian CI then you're specific that it's a Bayesian metric.
In this case, a 95% confidence interval describes a relationship between the sampling distribution and the (unknown) statistic of the population the samples are drawn from.
If you randomly draw samples from the population, you can get a range of means. For each one of those sample populations, you can draw an interval around that sample mean, specified as a percent of their N/T distribution; that is, you can specify that every single interval goes out to 1.96 sd for each individual sampling distribution, all intervals calculated separately.
Now, let's say that we could, somehow, know the real population parameter. And let's also say that we can scale ALL of the sampling populations so that some X% of them cover the population parameter in their range. Note: every sampling distribution has a different range for its interval!
If all of our sample intervals are infinitely wide, 100% of the sample intervals cover the population mean. If all sample intervals are very very narrow, then very few of the sample intervals overlap the population mean.
>>7703687
So, we scale the interval desired so that, if we were to perform the sampling process a near-infinite number of times, 95% of the intervals around the individual sampling means would overlap the real mean.
In practice, we can relate the properties of a single sampling distribution to what we'd get if we actually could do the experiment infinitely many times, so we can come up with a rough estimate of what the real 95% CI would be.
So, when you see a 95% CI, it means: You have no idea if the real mean is in this interval. If you did this test again, you could get a new mean with a 95% CI that doesn't overlap this given CI at all. Or it might partially overlap. All you know is that if you took the width of the 95% CI for this given sampling distribution, and looked at lots of other sampling distributions, 95% of the time the right answer would be somewhere inside that range.
side note: FUCK frequentist statistics, honestly. The mental gymnastics necessary to interpret a frequentist estimate in a rigorous manner just aren't worth it given the use you get out of them.
>>7703687
Ahaaaaaaaaaa.
Op here, I posted right after you posted. Thanks a bunch Anon.
>>7703697
Not OP here.
Is what you're describing the Central Limit Theorem?
>>7703720
Kind of. The reason we can make inferences about what sample means are going to be doing 95% of the time is because of the central limit theorem.
>>7703687
>>7703697
OP here, thanks a bunch again Anon. I did a small experiment using made up observations with a population of 50 observations and assumed the population parameters to be unknown. I randomly sampled it 7 times, and with what you explained I never had a harder "Aha!" moment in my life.
Thanks a lot Anon.
>>7703835
glad i could help!
>>7703856
3 Stat classes and I just figured it out, you're cool Anon.
pleasantly surprised to see someone with a proper grasp of the subject itt
thank you based probabilityposter
I have a question about this:
"Once an experiment is done and an interval calculated, this interval either covers the parameter value or it does not, it is no longer a matter of probability."
That's from Wikipedia, the article on confidence intervals. Suppose I flip a coin, but don't show you the result. Then you flip a coin. Looking at your result, would you say there is a 50% chance our coins are the same? The line above from Wikipedia seems to say that's not right, it's no longer a probability question since both coins are flipped.
Lord I hate Statistics.
NOT OP here: on the same theme, what about the p value?
What does it mean?
>>7704013
ehhhhh... this gets down into what i was complaining about before. it's mental gymnastics again
i agree, the commonsense interpretation would be that you'd have some kind of assessment of "if i didn't know, and it was going to happen in five minutes, can i describe what i think is going to happen", and that would be the same interpretation as what you wrote
but that's not the rigorous way to describe it. when you assign something a probability and you're being properly rigorous about it in a frequentist manner, what you're saying here is "i'm describing the proportion of total events this given event would represent in all the future theoretically infinite times i perform this". if you're describing an event that already happened then you can't really assign a p-value to it, even if you don't actually know what the result was.
it sucks! it really does! it's why there's like no straight-up purist frequentists anywhere, and everyone defaults to wishy-washy pseudo-bayesian descriptions to talk about probability
the problem is, when you really get down to actual theory development, you HAVE to be strictly frequentist, otherwise you QUICKLY get lost in the weeds and aren't modeling what you think you are (unless you go whole-hog bayesian and actually start calculating prior/posterior distributions). statements like that? they're like, reading C code and complaining that it's not dynamically typed. it sucks, but there's good reasons for it
>>7704015
do you mean like, a p-value generally?
>>7704443
>it's why there's like no straight-up purist frequentists anywhere
This is one thing that annoys me about self-described Bayesians. They are always tilting at the Frequentist windmill.
OP here, bumping this thread so questions could be answered.
>>7704015
first we have to step back and reconsider what we actually mean by a null hypothesis, and we have to change some thought
a lot of the times statistics is taught as a kind of... observational magic. you look at a value, you compute some function, you get an output value, and you're done. it's all a big black box that only wizards can open without it exploding.
don't think of it that way. instead, think of it as a... data generation process. your null hypothesis is a statement about some distribution (say, a normal distribution centered at zero with unit variance), and what you're saying is, if I pull a shitload of values out of the normal distribution, this is what I get.
and the question you're asking is, how likely is it that i could have pulled a bunch of numbers out of the normal distribution and gotten the observed data?
that "how likely" question is a bit tricky, so what we have to do is come up with a score function of some sort. for the sake of this post, the score function will just be "the difference in means". that is, you draw a group of data from the normal distribution and calculate its mean, and then you compare it to the actual observed data. if the actual observed data, in reality, did come from that same null distribution, then the score function will be near zero, as the difference in means will tend to be small. if the observed data came from some other distribution, say a normal distribution centered at 10 instead of 0, then the score is big
>>7705655
so now let's step back a moment. we have our score function that describes the difference in means. what we can actually say is that the results of the score function have their own distribution! if you're comparing two different sample sets that were each drawn from the null distribution, then most of the time you draw two sets, the score function will be near zero. sometimes it will be a bit away from zero, but not too far. a few times, it will be moderately far away, because you just happened to randomly pick two datasets from the null with means somewhat different. and a few rare times, you get wildy different means
so, we can do some math that i wont talk about and actually describe the density of the distribution of the score function. think of it this way - half the time you compute the score, it will be positive, and half the time it will be negative. so 50% of the total events are on each side of 0. 100% of all events are somewhere under that curve.
so, when we say we're talking about a p-value of 0.05, what we mean is this:
if we compared two distinct, random different draws from the null distribution and looked at our score function and compared the score to what we know about all the different possible comparisons of two draws from the null, what percent of that probability space of the score function is as extreme or more extreme?
that's your p-value
>>7705662
for the sake of that post i posited the score function as just a simple comparison of means. in reality you have to do other things like include information about the standard deviation and number of observations. it's harder to visualize though when you include those. for the sake of this post, just think about those numbers as adjustments to the means, a loose measure of "how much information is this mean really giving me"
and in reality, we don't have to actually simulate draws to figure out what the distribution of results can be. you CAN, if you want to, and it's sometimes done when there isn't a clear distribution to use to model your data, but normally people just approximate the data with a well-behaved, well-described distribution so they can just use the math that other people have worked out
>>7703697
>FUCK frequentist statistics
Hehe, meme
>>7705662
Nice job anon.
>>7703576
thinly veiled manlet thread
>>7704013
"it is no longer a matter of probability." The more I think about this, the more obnoxious it seems. Every probability textbook and every probability class in the world presents problems like this. "You and your friend roll pairs of dice. You roll an 8. What is the probability that your friend has rolled something higher?" That's not a matter of probability? "Sorry professor, that can't be answered, because we've already rolled!"
>>7703576
CI simply means that if you postulate "all males are between 5'11" and 6'1" "and you measured a large enough sample then you would be right for 95% of the samples.
It can be proven that picking a random male in the population, there is a 95% probability that he would be between 5'11" and 6'1".
>>7707629
this seems wrong to me
>>7703697
>FUCK frequentist statistics
This.
CI's are extremely inconsistent. Really engineer-tier creation.
It's a shit approach to stats
Could some anon explain bootstraps and Jack knifing to a pleb?
>>7703599
95% of the 95 confidence intervals include the mean
>>7707773
How are they inconsistent?
>>7708310
I don't know jackknifing yet. This is what I understand bootstrapping to be:
Ideally we sample repeatedly from a population in order to estimate parameters with statistics. Sometimes we can only draw a single sample due to cost, difficulty, or some other limiting factor. We can extract a single sample mean (x-bar), for example, but that gives us no hint as to the variability, and thus, the estimate of population mean (mu). This is when we bootstrap.
A bootstrap sample is a random sample taken with replacement from the original sample, of the same size as the original sample. We use the bootstrap statistic to estimate the variability (standard error) of the sample statistic, which in turn gives us an idea about the population parameter.
The bootstrap distribution will be centered around the sample statistic in the same way a sampling distribution is centered around the population parameter.
Bootstrapping means that you treat a sample as a population itself, in a way.
bootstrap statistic : sample statistic AS sample statistic : population parameter
>>7707779
aren't they all?
