A. Reading (importing) data from a text file with columns. B. Assessing Normality of data (Q-Q Plot, Normal quantile plot). C. Selecting a group case. D. Testing the difference between two independent groups using t-test. E. Visualizing difference between two groups with a double boxplot. F. Testing difference between related samples using t-test. G. Testing difference in increase between two different groups.
This week we focus on inferences towards the population mean of different populations using a t-test, described in sections 7.1 and 7.2 of M&M. We shall also shortly mention the F-test for comparing standard deviations, described in details in the optional section 7.3 of M&M (reading the first few paragraphs of that section will prove useful). Finally, remember that M&M introduces three different versions of t-test, and note that SPSS employs a slightly different terminology:
During this lab, we employ data from Joseph A. Wipf (Department of Foreign Languages, Purdue University). The data describe two groups of ten teachers who followed an intensive summer course in Spanish. Each of the twenty participants took a listening exam, both before and after the course.
Group after before 1 29 30 1 30 28 1 32 31 1 30 26 1 16 20 1 25 30 1 31 34 1 18 15 1 33 28 1 25 20 2 32 30 2 28 29 2 34 31 2 32 29 2 32 34 2 27 20 2 28 26 2 29 25 2 32 31 2 32 29
The data can be found in this text file: luister.txt.
> Read (import) this text file to SPSS.
Use the Text Import Wizard, but remember that this file has a different structure from those used in the previous lab: now we have three numbers for each case.
> Give the following names to the variables: GROUP, AFTER and BEFORE. Notice that BEFORE is found in the last column!
> The values of all variables are always integer numbers. Thus, set the number of decimal digits for each of the variables to 0.
> Then, save the file as a standard SPSS data file, that is, in a .sav format.
We shall soon run a t-test on the variable BEFORE, and therefore it is useful to know whether the variable follows (approximately) a Normal distribution. [This is good practice, even though M&M p. 456 writes that two-sample t procedures are quite robust against violation of Normality, especially if each sample has a size of 5 or more and if the sample sizes are equal. Both criteria are true in our case.] Were the sample really large, we could simply check if the histogram reasonably matches the Normal curve fitted by SPSS. However, in the case of a smaller sample (such as ours) random variation can cause the histogram diverge significantly from a Normal curve.
Therefore, we need a different technique to assess the Normality of our data set. The simplest one (introduced in M&M 1.3, p. 68) is drawing a Normal quantile plot; data fitting a Normal distribution will lie along a (diagonal) straight line, unlike data following a different distribution.
To create a Normal quantile plot in SPSS, you can use the functionality 'Q-Q Plot' under Graphics. By default, "test distribution" in Q-Q plots is set to Normal distribution; make sure you do not use a different distribution.
> Create a Normal Q-Q plot for the variable BEFORE.
> Remove the second, unsolicited diagram provided by SPSS ('detrended'), by selecting and then deleting it.
* 1. Copy the Normal quantile plot to your report. Is this variable distributed Normally? Why?
As the two groups may differ in the mean score BEFORE, it is useful to create the Q-Q plot per group. So we need to separate the cases that belong to Group 1 from those that belong to Group 2. SPSS has a function to perform this separation automatically after you have defined the filter – a useful tool if you have a huge amount of data, or if you would like to apply a complicated filtering condition.
Choose 'Select Cases' in the 'Data' menu. Click on 'If condition is satisfied', and enter "GROUP=1" in the condition window. Click on 'Continue', then on 'OK'. From now onwards, cases belonging to Group 2 will be crossed over and will not be taken into consideration in graphs and calculations. The column filter$ can be ignored, as it is created for SPSS's own purposes.
> Select Group 1, and create a new Q-Q plot.
* 2. Copy this Q-Q plot to your report. What is your conclusion for this group?
Do not forget to turn off the selection.
Our next objective is to test whether there is a difference (on average) between the two groups of participants at the beginning of the course. This is certainly a relevant question before we turn to whether the course resulted in some improvement in the participants' skills.
* 3. In the present case, the populations have not been clearly defined. Nevertheless, try to formulate a research question so that you have clearly a population and you have clearly a sample. Describe what the story is about then, and what the goal is of the statistical procedures being employed in this lab.
* 4. What is the null hypothesis to be tested? (Formulate one full sentence. Do not forget: does the null hypothesis concern the groups/samples, or the populations?)
* 5. What is the alternative hypothesis? Is the testing one-sided or two-sided?
* 6. What requirements must be met in order to be able to use a t-test on two independent samples? (Think of the sampling procedure, of the distribution of the population, etc.) Are these assumptions met?
When you perform a t-test for two independent samples, you have to decide whether the procedure should suppose the two populations have the same standard deviation, or no such supposition should be made. This decision influences the results of the test. As the formula of the t-test is slightly different in the two cases, SPSS reports the result of both approaches, and leaves the choice to the user.
If the populations have the same SD, we say that the variances (Variance=SD^2) are homogeneous. Supposing homogeneity renders the computations simpler (a factor that was especially important in the past). Obviously, we almost never can know for sure that the two populations have the same SD. What we can only check is whether the two samples have the same SD. If the variances of the samples are only slightly different, then we can safely suppose that the two populations do not differ too much in their SD. ("Not too much": the difference does not influence very much the p-value of the t-test.) If, however, one sample has a SD of 2 and the other sample has a SD of 20, then supposing homogeneity on the populations is not at all plausible. You should then employ the procedure not postulating homogeneity. (M&M 7.3, p. 474 suggests to always employ the latter procedure. This second option was not available earlier, and so older colleagues may tell you to always test homogeneity before a t-test.)
> Perform the t-test on the variable BEFORE to test the difference in the means of the two independent groups.
Hint: The two-sample t-test is called "Independent samples t-test" in SPSS (under "compare means"). The t-test asks for a variable to separate the groups. So first turn off Select Case ("all cases"). Then, in the window for the t-test, you have to select the variable that you would like to test, as well as another variable that serves as the criterion for defining the groups. Use GROUP as this second variable. You also have to determine which values of GROUP will define sample 1 and sample 2.
* 7. What is the standard deviation of the two groups? Are they reasonably the same, or quite different?
SPSS first performs an F-test (cf. M&M 7.3) to check the homogeneity (similarity, equality) of the standard deviations/variances. Refer to the first two columns of the last table. The null hypothesis of this F-test is that the two samples originate from two populations that have equal standard deviations. The p-value of the F-test assesses the probability of drawing samples that are at least as far away from the null hypothesis as our samples.
* 8. What probability has SPSS calculated? Is there reason to reject the hypothesis at significance level alpha = 0.05 that the standard deviations are homogeneous?
Let now turn to the outcomes of the t-test. First, we suppose that the standard deviations of the two populations are equal (homogeneous).
* 9. What value has been reported for the two-sample t-statistic?
Now, let us assume that the standard variation (variances) of the two populations are not necessarily the same.
* 10. Which probability or p-value ('Sign.') is reported by SPSS?
* 11. Can you conclude that there is a difference between the two groups at the beginning of the course? Please give a one-sentence conclusion of your statistical procedure, reporting alpha-level, as well as t-value, df and p-value in parenthesis, as usual.
* 12. Is your conclusion different from the case when you supposed homogeneity?
> Let SPSS draw two boxplots in one figure to visualize the differences in the values of variable BEFORE across the two groups.
Hints: Choose 'Simple' and 'Summaries for groups or cases'. Use GROUP as category variable.
* 13. Copy this figure to your report. Add a good (precise, detailed and informative) caption to this figure of one or a few sentences, as usual practice in scientific publications and scholarly books.
We are still using the data of the intensive summer course in Spanish. The most important question is obviously whether the summer course improved the skills of the participants: Did the participants score higher after the course than before it? While answering this question we shall first ignore differences between the two groups.
* 14. How many different (independent) cases do we have actually in our sample? How many observations do we have per case in the sample?
The best way to determine whether a participant has improved his or her skills is to compare the course-final score to the course-initial score, that is, by calculating the difference AFTER - BEFORE. This is exactly what the t-test for related samples does (see also in M&M, end of section 7.1). Yet, by performing the test yourself, you can better see what exactly happens and you can also draw figures of the variable of difference IMPROVEMENT = AFTER - BEFORE.
> Use 'Compute' to calculate the new variable of difference. Call it IMPROVEMENT.
* 15. Does IMPROVEMENT follow a Normal distribution? How did you get to this conclusion?
Hint: You can both fit a Normal curve to the histogram and create a Normal quantile plot.
> What is the mean of IMPROVEMENT?
This mean is the mean of a relatively small sample. The population mean can be quite different.
* 16. Give a 90% confidence interval for the population mean IMPROVEMENT. (Refer, if necessary, to the SPSS functions already employed in the previous lab.)
As described on the last pages of section 7.1 of M&M, you can perform a one-sample t-test on this difference variable, and this is the procedure called matched pairs t-test. Let SPSS calculate this single-sample t-test for you.
* 17. Test the hypothesis that the mean of IMPROVEMENT is 0 using a t-test. Formulate your conclusions by reporting the value of the t-statistic, df, p-value (one-tailed or two-tailed? why?), and whether you can reject the null hypothesis (which is what?) at alpha-level 0.05. (Refer, if necessary, to the previous lab.)
* 18. What is your conclusion: is there improvement in the scores obtained on the listening test? Illustrate your claim with convincing figures, too. It is up to you to choose what type(s) of figures you use, but always add captions to figures.
We have used all twenty teachers to find out if the scores on the listening test have improved. However, it is also possible that one of the groups displays significant improvement, whereas the members of the other group of ten have not, or have almost not, improved their listening skills. This question is especially interesting if the two groups followed a course with a different methodology, and so we would like to argue for the advantages of one of them.
You have now to combine what you have learned in D with what you have learned in F.
* 19. Describe the statistical procedure you perform: what type of test(s), on which variable(s)/group(s), what is the null-hypothesis, do you use a one-tailed or a two-tailed alternative hypothesis, are the criteria for performing the test reasonably met, etc.?
* 20. What is your conclusion: is there a difference between the two populations? Report your conclusion, including the results of the statistical procedure, as usual. Illustrate your conclusion with figure(s), including a caption.
This material is an adapted version of the assignments of the statistics courses developed by John Nerbonne at the University of Groningen.