USVI Public Records

Document text

., DOCUMENT RESUME ED 227 150 TM 830 161 41- . AUTHOR Bliss, LeOnard B. TITLE , Validation of the Use of the Stanford Achievement Test with U.S.V.I. Students. Virgin Islands of the United States Public School Basic Skills Assessment Survey, Technical Report No. 1., INSTITUTION College of the Virgin Islands, St. Thomas. Caribbean Research Inst. JTVB DATE Jan 82' 54p.; Paper presented at the Annuar Meeting of the American ,Educational Research Association (66th, New 21t, York, NY, March 19-23, 1982). AVAILABLE FROM %.Caribbean Research Institute, College of the Virgin fsl.VINIs, St. Thomas, USVI 00802 ($2.00). PUE TYPE . Speeches-/Conference Papers (150) -- Reports Research/74chnical (143) EDRS PRICE MF01/PC03 Plus Postage,,, DESCRIPTORS *Achievement Tests; *B6'Sic Skills; Data.Analysis; Educational Assessment; Elementary Secondary Education; Language Skills; Mathematics Achievement; Reading Achievement; Sampling; *Standardized Tests; *State Programs; *Test Reliability; *Test Validity IDENTIFIERS *Stanford Achievement Tests; Virgin Islands ABSTRACT A sample of slightly over 1500 students was drawn from even-liumbered grades ill public schools of the U.S. Virgin . Islands, and wasp given the 1973 edition of the Stanford Achievement Test (in grades 2,4,6, & 8) and the Test of Academic Skills (grades 10 and 12) to assess student academiC achievement in the basic skill areas of mathematics, reading, and English language. This report describes phase I 9f the data analysis, which involved the determination of levels of content validity and reliability of the scores obtained from these Virgin Islands students on these tests which were origi,nally standaedized on continental United States populations. The results indicate that the tests are codtent valid for use in Viigin ISlands public schools at these grade levels and that the scores obtained are at least as reliable as those obtained using continental U.S. students during the test standardization procedures. (Author/PN) *********************************************************************** ReproduCtions supplied by EDRS are the best that can be made from the original document. *********************************************************************** U S. DEPARTMENT OF EDUCAIION NATIONAL INSTITUTE OF EDUCATION EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 2( This docurneid has been reprodabed as let toyed bon, the person or ortoneation originating it Minot changes have been made tu improve reprriduchun quality Points of view or opinions stated on this docu relent do not oecessarity repregjntufficiaI NIE posqtonorpacY Virgin Islands of Oe United States Public School Basic Skills Achievement Survey Technical Report #1: Validation of the Use of the Stanford- Achievement Test With U.S.V.I. Students "PERMISSION TO REPRODUCE, THIS MATERIAL HAS BEEN GRANTED BY L TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC)" PRESENTED Al THE ANNUAL MEE1)NO OF HE AMERICAN EDHCATiONAL RESEARCH ASSOCLATION NEW YORK CITY, MARCH I9 -23, 1902 Caribbean Research.Institute, Call4ge of theVgilr- lands ' Leonard B. Bliss, Ph*M. - Principal Investighto January 1982 PREFACE With the appearance of Virgin Islands of the United States Public SchoOl Basic Skills Achievemeat Survey, Tech- nical Report 111: Validation of the Use of the Stanford Achievement Test With U.S.V.I. Students the Institute has' embarked on the publication of a Working'Paper Series. These papers are intended to present the author's (and the Institute's) point of view on various subjects as a matter for discussion and comment by those who agree as well as disagree with expressed positions. 'In this way the Insti- tute hopes that the final versions will be improved in style as well'as rigour. The present paper is the first phase of a study of basic skills in the schools of the United States Virgin Islands requested by the Board of Trustees of die College. The work has taken considerably longer than anticipated due to fundamental alterations in the design so as to provide greater depth than originally planned. Unfortunately, shortage of staff did not'allow the progress hoped for to be made. The data for the wholz.e project have been collected, however, and work is prpceeding on .their interpretation and the compiling of che three reports' which will follow. Norwell Harrigan Director Abstract A sample of slightly over 1500 was drawn from even numbere& grades in public schools of the U.S.'Virgin Islands and.were given the 1973 edition of the Stanford Achievement Test (in grades 2,4,6, & 8) and the Test of Academic Skills )(grades 10 and 12) in an attempt to assess student academic.achievement in the basic skill areas of mathe tics, reading, and English language. This report describes haze I of the data analysis which involved the determination of levels of content validity and reliability of the scores obtained from these Virgin Islands students on these tests which were originally standardized on continental United States populations. The results indicate that the tests are content valid' for use in Virgin Islandk public schools at all. of these grade levels and that the scores obtained are at least as reliable ai those btained using continental U.S. students during the test ndardization procedures. 7r It is alm\st becoming a matter of faith tnt achievement in basic skills i.e. English langudge and mathematics) in public schools nder'the American flag has deteriorated over the last twen y years. Proponents'of this idea point to evi- dence as f mal as decreases in typical scores on the Scholas- , tic Aptitude Test and standardized tests of academic achieve- ment and as informal as the qUality of writing and arithmetic skills they perceive in-the young people around tliem. The reactions of people to this perceived phenomenon are also Varied.- On the government level they include the require- . ment that all students score a minimum grade on a test of basic skills in order to receive a high school diploma; that teachers . pass a similar best to obtain teacher certification; and that schools require students to take idditional course work in basic skills areas. In addition, federal, stafe, and local governments have initiated programs to provide support in the forms of grants and technical assistance to schools at all levels to do research and set up programs designed to improve student achievement in basic skills. At a different level, parents, Concerned that the public schools are not doing an adequate.jop in preparing their children in basic 6ki1ls areas, are choosing, in increasing numbers, to remove their children from public schools and place them in religious and secular private schools. While there are other 61k -.2- reasohs for the proliferation of priAte schools besides the purely academic, the desire for high quality academic prepara-. tion is'one compelling cause of this phenomenon. The public schools, themselves, have reacted strongly to this crisis in public confidence. These reactions include an increase in required courses in language and makematics areas with a corresponding decrease in electives in areas considered less "basic." Projects to revise curr.icula in basic skills ". areas proliferate and are receiving more support than they ha4e since the reevaluation of-AMerican education engendered by the shock of Sputnik in the late 1956's. Improving basic\skills,achievement was a,concern of the Departmenu. of Education of the government of the Virgin Islands of the United States when it approached the College of the Virgin Islands to provide aid in improving such instruction. In an effort to provide this service, the Caribbean Research Insti- .tue, the College's research arm, worked with a task'force com- posed of representatives.from the Department of Education and CRI to determine a course of action. It became clear after the first few task force meetings that development of any strategy designed to improve basic skills achievement needed to start off with a fairly detailed description of current achievement levels of students in terri- tbrial public schools. This information was not available. Public school students were adMinistered a standardized achievement test bnly at the end of' sixth grade (The Iowa Test of Basic Skills). In other elementary grades most students t " , - 3 - were tested annually or semiannually at their schools, but the , test given and the,times during the academic year that were _ admicnistered varied greatly and.apparently at the whim cif build- ing administrators. The results of Ise tests stayed at the _ schools and were not copected at any central point. On the secondary level there was nO program of standardized achieVe- .. ment testing. An additional factor which limited the use of previously collected achievement level data was that all scores were re- ported in a norm referenced manner. That is, scores did not / indicate which basic skills examinees had or lacked, but rather : how examinee!s scores wmpared to those obtained by a group of ' students' to whom Ale tests were previousiy administered in-th continental Uni ed St tes. The Iowa Test of Basic Skills administered to sixt4 graders did make comparisons with other V.I. sixth grade stidents (i.e. they rePortped using local norms), but even these were of no use in determining whether or not 0 - individual-students had attained specific basic skills. 4 It was,decided to test a representatiVe.sample of U.S. . « Virgin Islands public school students using a standardized . basic skills battery. Choosing the test, the following criteria were used: 1) The test Must be technically sound in terms of s°reliability and item discrimination,.at least , fot the group it had been field tested on. 2) The test must be content valid fox U.S. Vittin Mr Islands public school students. That is, there 4 4 needed tobe 'a high degree of Matching between the content and behaviors sqmpled by the test and those actually in the curriculum taught at various levels in the U.S.V.I. public schools. ,3 The test must include a detailed statement of the objectives tested while providing an item by objective keyiag procedure. , 4) Scores which indicate students performances relative to each objective, must be available. That is, criterion-referenced scoring must be provided. 9 The 1973 version of the Stanford Achievement Test (Basic Battery) was chosen as.the test which appeared to meet the criteria listed above. It was administered to slightly over 1500 students in the Fall of 1980'in both the St. Thomas/St. John and the 'St. Croix school districts. This is the first oria series of research reports"designed to make available the results of this rather-cOmplex study. A simple, brief exampil of,the quantity of data obtained may serve to highlight the scope of this study. The Intermediate Level II ofthe Stanford Achievement Test (administered to sixth graders in this study) contained 351 items. It was administered to 225 students in the U.S. Virgin Islands sample yielding 78,975 individual pieces of data. The sixth grade sample, due to a technical diffictilty (the principal in one school forgot tvassign the teacher of the selected class the task of iiving the test and the teacher in another school administered only four of the I seven subtests) , contained the smallest numbet of examinees of any grade level. Additional regorts il1. be issued regu- larly as soon as results become 4vailable. t. Validity and Rel.' of Test Sca!, This first report dealS h the esEablishment of the validity and reliability of:, test scores. "Validity refers to the extent to which the measures those characteristics which it is inteuJed.to . Ebel (1961) has referred to kAti validity as "one of the maj*idieties in'the pantheon of the psychometrician" (p.640), firee types are now commonly used in educational and psycholAtcal measurement (see French and Michael,- 1966). These arwlgc).ntent, criterion-r,elated, and constructvaliaity. Gronlund (1976) indtes.that, "Criterion,-related'Valid- . . " , ity may be defined as th Xtent to Which test perfoimanCe is related to some other vat #d measure of performance". (p. 83). This may be perforMance. a taA in the future (i.e. predic- , tive validity), or on som:qPreaent oblectives not directly measured by the test (i concurrent validity). Since the purpose of administering4n achievement`.1test is to get a direct measure,of presenI studepT mastery of certain-academic objec- tives (i.e. there is no 'attempt to predict future performance or to infer performance Vevels on objectives not directly measured by the test), cfl.terion-related validity is not an -41 , / issue in determining theOppropriatencss of the Stanford ,,Acbievement Test in measu;ing academic achievement in this study. The tern "construct Validity" was first introduced into I.", 5 4 -7- the area of psychometrics by Chronbach and Meehl (1955) who defined a construct as a postulated (that is, assumed or hypothetical) aLtribute of people that underlies and d t rmines their overt behavior. If the behavior can be directly ob erved, or if the trait can be operationally defined, it is not a on- . struct in this sense: Ebel (1979) notes Oost of whqt we teach in educational institutions are knowledges, skills, and abilities. These.can all be defined operationally. They are not hypo- thetical constructs. Ability to type, t6 spell, to weld, ability.to solve probaems with alebra, calculus, or computers; these ate hot.the kind of latent traits Cronbach and Meehl had in mind. We would speak more sensibly, I think, if we did not Call them constructs. (P. 307) . Construct validity is concerned.with whether or not a test is accurately measuring the construct it4urports to measure Since this study is ope-rationally defihing basic skills achievement as the performance of students on the Stanford 4 Achievement Test, it is clear that no construct.dis being measured. Hence, construct validity will tot be a concern in this report. The coptent of any curriculum can be ,thought of as being' composbd of subject matter content and'behavioral changes sought in students. For' a test to be content valid it 'must provide re- sults that are representative of the topics and behaviors we wish to measure. More formally, . . content validity may be defined as the exten to which a test measures a'representative sample rhe ubject matter and the behavioral change'S under consideration" (Gronlund, 1976, 'pp. 81-82). Effective strategies for determining,content validity invo,lve determining the objec- , 1, , . .. tives--gmpledAiY.the test and examining the curriculum to asce4litlh'the degree of match between them. AchieVement tests are primarily concerned with measuring the acquisition of 'cer- , tain skills and knowledges (objectives) by students, at the time that the test is given. Thus, it is c9ntent validity Ehat should be of prime concern in ,this study. Speciricalily, do the objectives tested by the Stanford Achievement Test corres- pond to those taught toward,in the schools oCChe U.S. Virgin Islands? Reliability deals with the consistency of Lhe scoresfrot test over time and over different examinees. It urely statistical phenomenon and cannot be determine logically . . ,as can coneent validity. Ful7wermore, it is a funcqon of the scores of the test rather than of the test, elf. this means tnat a test may give highly reliable scores f r one group of examinees, but result in lower reliabilitysw th another groul5. 0 In essence what we are cOncerned with is ether orinot the , _test scores'represent measures of the same Traits each time the 4 , t test is giverl. . S. It is important, then,-that w hatever easure of-basic skills achievement is used, that the measu e be content valid for the curriculum used in Virgin Islands )ublic, schools arid p poduce reliable scores when administered 't.o Virgin Islands public school St:Lidents.. As any good coMmercially available s 1973 edition of the StanfOrd Achievement andardized'test,_ he "est_ Was standardized on a 'large sample of students. The SAT T(rchnieal Data Report -9- (1975) indieate2.,tha a saMple of over 275,000 pupils'from 109 school systems in 43 sAes in the United States,made up the standardization sampleS used. Table 1 provides descriptioris of these samples and How they compare with a *description of the _population of the continental United States.- Content.,validity was established by curricular analysis using information frOm a to larga'number of sources-., lia'afe-To the7'd6nstAittion bf a S'eties of achiewement tests is the identification of what is being taught in the schdols .acoss.the nation. The most important - L sources frit curtieular analysis were (a) textbook series in vaxious subject are-as (iacluding the prepa- ration of detailed analysis of the cpntent of the boas most widely used in, each' field); (b) a wide variety of'boutses of study from.individual school systems; (c)Aatements of objectives from various state and rational committees; and the opinions of experts in various fields; and (d) the research literature pertaining to children's concepts, expe-' rience, and vocabulary. ,(Technical Data Report, p.12) The reliabili ty of the scores'bt the standardization sample 4 was determined by Using the Kuder-Richardson Formula 20 and by calculating the standard error of Measurement of the scores. Two mpasutes of reliability were used *ace it is known that high homogeneity in tested_groups will lower the reliability estimates obtained using the KR-20, but thdt his effect is dealt with rri- etermining standard errors. In addition, the standard error of.meiaurement is more meaningful in interpreting scores of individual students. With very few exceptions, the reliabil- ities obtained from the standardization samples ranged from .84 to .95'using the KR-20 fotmula. While the 1973 version of the Standford Achievement Test 0 appears to be educationally sound based on the standardization- 3 t- Tab1e;J, Sumrery of dharacteristics. of Standardiiation Samples1 .1( , Characteristicg . National Stanford . Stanford U.S. Population Population gange 1979 Data .. * 4; Percent of pupils by community size 0-49099 70.0 50,090-249, 999 14.2 250,000 br more 15.9 per'cent of pupils by 1 Geographic Region, Southeast 23.8 North Central 21.6 Northeast 26.5 Wegt 28.2 MeditnFamily Income $9,096 Median Years of Schooling (Adults 25 yrs. & older) 12.1 Average Class Size $ 4,878 to $13,593 8.4 to 12.6 64.1 15.2 20.7 22.2 27.8 24.2 25,8 $9,590 12.1 (Student-Teacher Ratio) 26.4 16 to 24.3 36 Average Starting Salary of Teacher $7,116 $,4,500 to $7,064 4'000 . . Average Salary of Teachers $9,360 $ 4,500 to $9,265 11,500 , Median Years.. Teaching Experience .10.8. 5 to 24 ao Percent,of Grade 1 pupils *ho.attended kindergarten 84.6 0,to 100 71.8 Percent.of Schools Usimg Some Team Teaching 67.1 Table 1 continued Characteristics Aanford Population 'Stanford , _Range , National - ) U.S. Population 1970 Data Percent of Schools Using Some Teacher Aids 97.5 Percent of Pupils Not Promoted to Next \\ Highest Grade Grade 1 3.9, 0.0 to 25 Grade 2 1.8 0.0 to 15 Grade 3 1.5 0.0 to 10 Grade.4 0.9 0.0 to 10 Grade 5 0.8 0.0 to 5 Grade 6 1,1 0.0 to 5 Grade 7 1.2 0.0 to 11 Grade 8 1.3 0.0 to 9 Grade 9 2.4 0.0 to 9 Percent of Pupils 11 Non-public Schools 9 Percent of Major Ethaic Minorities .Blacks 11.6 0 to.60 Hispanics 4.6 0-to 60 Other Less than 1 12 11.1 4.6 Less than 1 1From Stanford Achievement Test: Technical Data Report, p..21. groups data, the irOups contained only continental U.S. studehts. Likewise, the-test makers most probablY did riot take Virgin Islands public school curriculum into account when designiq items. Therdfore, before the'scores of any tests of basic skills can be used to draw conclusions about rt V.I. sEudents, the content validity and reliability of these test scoreg for Virgin Islands students must be established. Hence, this report. 1 v 0 .0* Method Sampling' The June 1, 1979 enrollment in the public schools in the- , Virgin Islands of the United States was 25,426 aCcording to the statistics issued by the V.I. _Department of Education. It was clear that testing this number of students was economically unfeasible. -The preferred flternative*uld have been to generate a random sample of students in grades K-12 to be tested, blit it was equally clear that this would have produced an in- 'tolerable disruption af classroom activities. Therefore, in 'am attempt to obtain representative sample of students, cluster.sampling was used with the clusters being defined as' classes. The number of classes to be selected for the sample from each grade in eebh of the SE. qhomag/St. John and St. Croix districts was determined ,by calculating the proportion of the toal K-12 student population in each grade in each district' and -assuming a class size of thirty. Selecting.Whole classes presented an additional difficulty.' The small number of classes selected in each grade might have made obtaining a representative sample of students more diffi- cult. This is due tO the fact that while classes in a given elementary school may be heterogeneous, the sehools themselves .are not. This is because elementary sclloolc in the U.S. Virgin Islands are essentially neighborhood schools. Virgin Islands neighborhoods tend to be homogeneous in terms of socioeconomic 'status of residents. 'To overcome this problem, it was decided to increase the,number of classes tested in a given grade V., -14- (thereby incteasin the-number of schools within the;territory from which these c asses came) without increasing the total number of students stied by testing at alternate grade's. This seemed acceptable si e many of the objectives tested by the Stanford Achievement st carry across adjacent levels of the test and there was no ason to suspect that the patterns of academic achievement o_ tudents in odd numbered grades were different from those in en numbered. grades. It was originally pro osed that students in odd numbered grades be tested during the\Spring of 1980, but_difficulties in obtaining testing materia s resulted in testing being post- poned until the Fall of 1980. In order to deal with.the cohort of students originally selecte even numbered grades were actually tested.. The classes to be teSted were chose.n by chance. Specifi- _ cally, for each grade in each district a listing of classes . was made and each class was assigned,a number. A table of random numbers was consulted. Numbers were drawn from the table until chere were the same number of random numbers chosen as thete were classes needed for the sample. In the case of duplicate numbers being drawn, the duplicatemas ignored and another number chosen. If the number chosen was outside the range of the number of classes on the list, it was ..4noi-ed and another number was chosen. When sufficient numbers had-been drawn, the listed classes,which corresponded to these.numbers were included in the sample. This procedure was repgated for each grade in each district. 3 -15- The sole exception to this Weedure was in the'eighth grade portion of the sample. On SL. Thomas,.homeroom classes are somewhal homogeneous in that students repeating eighth grade and those in the eighth grade for the first rime are placed in separate homeroom classes. Since the levels of academic achievement for repeaters and nonrepeaters are very likely different, the proportionof repeaters and nonrepeaters was determined to come out with aCnumber of classes needed in the sample from each group and the groups of classes of repeat- , ers and nonrep;aters were sampjed separately in the manner descrihed in the preceding ragraph, .Elena Christian Junior H h School on St. Croix (is on split session, ik principal of that school felt that there were definite differences in,achievment levels between the students in the morning and afternoOn sessions. Because of this, classe- in.,the mOrning and afternoon sessions were sampled separately using the same procedure employed on St. Thomas for the repeating and nonrepeating homeroomclasses., If simple random sampling has been used in selecting student to-be tested, a sample size of approximately 2000 would laave been the paXimum siig required to.obtain an'accuracy of about ±2% at a .95 level of Confidence, when estimating the "lb proportion of V.I. students reaching certain objectives from, the sample proportions if a typical proportion answering each item correctly were .50 (see Asher, 1979, p.166). In actuality, due.to student absences, 'failure of school personnel to carry out requested tasks, and other difficulties, the sample size' 4 -16 - Obtained was only 1535. However, examination of the difficdlty indexes of the StamfOrd Achievement Test iteMs on all'levels revealed difficulty indexes considerably different from .50 Qn mosx items. This would tend to shorten the size of the confidence interval. Finally, the'fiaancial and organizatiOnel constraints cited previoUsly :forced the investigatorS to use .cluster sampling techniques rather than random sampling. 'Sivee , / ehe intraclass correlations (i.e. the effects of dlustering on the standard deviations of the achievement test scores) were not known, this factor also contributes toward making the above L mentioned accuracy estimate a rather 'crude one. .1t can, however, 'serve as a rough guideline. Table 2 pesents the relevant sample size data. The sixth, and second grade samples from St. Croix are smaller ,than,had been hoped for the following reason§. As'indicated'pre- / vibusly, the (eacher Of.one of the sixth grade classes only administered fouS of the seven subtests. In.a second grade class, the teacher was ill during the days set aside for test- ing and the test was not administered. By the time this beame apparent oo the inve'stigators, it-was too late to go bade to St.--Croix to'retest. Aside from the diffillty in estimating precision of the proportions,of students obtaining correct scores.on various . items, the sampling procedure used presents another AifficuL y. Because of the pre-viausly stated practical eonsidewions, it was necessary to employ cluster sampling (sampling whole classes) rather than simple random sampling o: students to be tested. smr Table 2 U.S. Virgin Islands Sample.Sizes Grade Test Level Total, System St.Thomas/St.John District SL.Croix District '12 TASK II , 129 74 --.\\ 55 10 TASI. I .:54 167 87 3 Advanced 345 173 172 6 Intermediate II 227 146 gil. 4 Primary LaI 346 186, 160 2 Primary I 234 143 91 TOTAL 1535 889 . 646 The principal drawback to cluster sampling, is the likelihood of \ . increa'sed sampling error. Iu general, as the size Of the sample \ , , increases, the size of the standard error decreases. This applies,' however when each.sample element (in tbis case, each student) is selected independently oi every other,eLement. In cluster sam- pling fhe elements are, ,by definition, selected in a group rather 'than independently. The effect of clustered selection on the standard error will depend on'he'similarity-between the elements 4 in the cluster and those in the population. In many cases, sample elements selected in clusters will not show-the smile 4 variation as an equivalent number selectedpidependently. Students who attend.,the same school and are in the same class may be more like one another in a characteristic such as áca- demic achievement than students in the publiá school population as a whole. The relationship between clustering and sampling error may be Summarized as follows. If'all the elements (students),,in a- -18- cluster (class) were identical with regard to achievement and totally different from the elements in other clusters,.the sampling error would be extremely high. Clustering, in this case, would tend to make the clustered sampletequivalent in size to a simple random sample with as many subjects as there are clusters, rath'er than elements. Hencea sample .of -60 clus.ters might be equivalent to a siipple random srh1e of 60 individuals. This is obviously an extreme case that is .never seenanpractice At the opposite extreme would be a series of ciusters showing the same variatibn Within each cluster as simple Tandom'samples of the,same size. In chis case, each cluster would represent the entire population,'. another .condit0b,rarely met in practice. Most sampling itia- tions fall in betTi these extremes, tending toward one or the , other according to the Characteristic being studied. In general, according to Warwick and Lininger (1975), experience has shown that well-designed cluster samples will produce standard errors, that are abbut one and one-half times as large as the,standard errors from simple random samples of the same size. This situation should not have any effect3.dn the descrip- . tive statistics reported in this document, but it will enter, into the'interpretation of the,results-of hypOthesis :testing using parametric echniques since these latter techniques rely on estimates of the standard error. These will be discusse made up , as Ae results of these tests are dealt with. In general, however, the resulting under-estimates of the Standard error of the means will result in test statistics:that are highe'r v -19k than they wourd have been ii clattered standard error estimates had been used.' Notwithstanding the difficulties involved in sampling, the researchers are confident that the r(sulting'samples are as representative of the enrire V.I. public school population as is really possible given the nature of working with human subjects apd the organizational considerations of schools com- . lAned with the rq-sources available for the stUdy. The diffi- * cuities encountered are not untypical oE those commp.nly.found when doing field work in both public and private schools. , TeSting Procedu.re Testing was done at 'the grade level recommended by the- \ tett-publisher. Table 3 indicates the subtests of the battery given to e,6ch grade. This was primard.ly done to insure Lhe content validity of the.examinations. Tests were administered by classroom teachers or guidance counselors, at the discretion oE building administrators. Each person who was to administer . tests attended a two hour training session:at either the College .of the Virgin Islands St. Thomas or St. Croix campuseb. During this time the purpose of the testing was explaihed, the test and, inseruction manual were reviewed, a te'Sting Schedule was distributed and reviewed, and testing mateFials were distribUted. These included a practice test for each of grades 2,4, And 6 This was to be given,to students the day prior to the first' day of testing in order to give them practice in reading and 'answering multiple 'choice standardized tests. 2 3 Tests were adminiStered in the,St. Thomas/St. 'John district during the-week of October 21, J980 and in tne St. Croix district during the week of December 1, 1980. Testifig materials andcom- pleted answer sheets were collected, answer sheets checked to determine Compliance with marking instructions, and answer documents were sent to the Psychological Corporation of Iowa City, Iowa to be machine scored. Grade Grade 12 (Task II Level (rad- 10 (la k I Level) Table 3 Stanford Sublests Admdnistered at Each Grade Level Subtest Number oI Items Reading 78 Mathematics 48 English 69 Reading 78 Mathematics 48 English 69 , _de 8 Vocabulary 50 Odvanced Level) Reading Comprehension 74 Mathematics Concepts 35 Mathematics Computation 45 Mathematics Applications 40 Spelling 60 Language 8J (rade 6 (Intermediate II Level) ° Vocabulary 50 - Reading Comprehension 71 Mathematics Concepts 35 Mathematics Computati.on 45 Mathematics Applications 40- Spelling 60 Word Study skills 50 Language 80' Grade 4 Vocabulary 45 (PrimaTy Level III) Reading.Comprehension 70 .. Word Study Skills . . 55 Mathematics Concepts - 32 Mathematics Computation % 36 Mathematics Applications 28 Spelling ' 47 .Language 55 \-1 . Grade 2 Vocabulary 37 , (Primary Level I) Reading Comprehension 87 Word Study,Skills 60 Mathematics-Concepts 32 Mathematics Computation 32 Listening Comprehension . 26 -21- . , a Results Table 4 provides'descriptive statistics usig the raw scores, (number of items correct) of students on each subtest of the Stanford Achievement Test. , Content Validity The content validity of,the various leVels of. the Stanford Achievement Test used to collect data.on basic skills achievement was determined by using the following strategies: 1) Collection af_written curriculum guides used in the pUblic ,schools. The objectiVes/explicitly stated or implicitly 4 ° inferred.in these doCuments were compared with the lists of objectives tested provided by the test publisher.. 2) Text baoks used in the teaching of basic skills subject , . , :-. -. matter were'collected from selected schools,. Stated and implicit objectives in these texts were coMpared with the test pubUsher!s objectives. 3) The test objectives were shown to elementary and secondary subject area supervisors vho wei'e asked to determine the degree of match. between those objectives \and what is taught in the4public schools at the'indicated grade levels. 4) Selected building principals in St. Thomas were askedto review the objecti4b's of the test and give their opinions concerning the degree of match between'these objectives and the objectives taught toward in the clas§es in.their schools. 4.0 -22- Tahle.4 Descriptive Statistics of Stanford Achievplent Test Ray Scores System STT/STJ I STX rest Mean Strid. Dev. Mean Stand. Dev. Mean Stand Dev. Grade 12-Task II Level Readim, 43.9 13. 8 40.6 12.3 48.3 15.2 Mathematics 25.3 -8.3 24.6 7.4 26.3 , 9.4 English 46:8 11.'2 45.6 10.9 48.4 11.5 ' Grade 10-Task I Level qk,adiug 45.6 14.0 . 43.6 14.2 48'.5 , 14.3 . -32.0 . 14;6 -,-... 32.1, 16.9 --n31.5-.-,8.6.r,:"' Engli411 48.0 12.0 ', 47.6 10.8 49.0 14.0 , ), Grade 8-Advanced Level ,cahu' ,rv 21.0 7.2 20.7 6.6 21.2, 8.5 Readinp Comprehension 31.5 15.4 32.6 17.4 30.5 13.1 Mathdmatics Concepts 15.3 5.8 16.4 6.0 .* 14.2 5.3 Mathematics Computation 23.0 7,6 23.3 7.1 22.8 8.2 Mathematics Application 16.9 6.6 17.7 . 6.6 16.1 6.5 Spelling 31.8 , 12.3 32.8 11;8 30.8 12.9 Language 35.6 12.1 35.8 10.6 . 34.6 13.6 Grade-6-Intermediate II Level Vocabulary 21.6 7.8 22.6 8.1 19.7 6.8 Reading Comprehension 32.5 12.4 31.8 12.7 33.5 11.5 yord Study Skills 28.6' 11.1 29.4 11.5 27.1 10.3 Mathematics Concepts 18.2 5.7 19.3, 5.5 16.4 5.5 Mathematics Computation 250 7.4 24.8 8.0 25.4 6.3 Mathematics Applications 18.4 8.0 14.2 , 8.0 16.9 7.8 Spelling 35.2 13.6 35.5 14.2 34.7 12.5 Language 37.4 13.8 37.,9 15.0 3,7.0 11.5 Grade 4-Primary III Level Vocabulary .23.6 7.2 2,3:4 5.9 24.0 Reading Comprehension .42.3 11..5 ',42.2 11.3; 42.3 11,31 'ord Study Skills 29.8 10.0 .30.8 9.2, 8.7 10.8 .1thematics Concepts kinematics Computation 15.5 5.3 15.0 , 20,4 6.1 15.6 4.4 _5.1 16.0 6.2 21.4 7.0 ..chematics Applications 13.8 5.8 13.8 5.5 13.8 6:0, ,t.11ing 30.9 .10.2 30.4 9.2 31.5 11.2 inguage 28.9 8.8 28.1 8.1 29,8 9.4 -2 3- U.S.7V.I. System STT/STJ Test Mean Stand. Dev. , Mean Stand. Dev. STX Mean Stand. Dev. Grade 2-Primary i Level Vocabulary 21.7 5.1 - 22.7 4.8 'N, 20.1 5.1 Reading (Part A) 34.1. 13.8 36.3 15.2 30.5 10.3 Reading (Part B) 29.5 A8.9 30.7 8.4 ' 27.6 95 Wqrd Study Skills 47.5 9.4 48.7 8.7 45.6 . 10.1 Mathematics Concepts 19,0 4.4 19.7 4.3 18.7 5.6 Mathematici-Computation. 21.-5 5.0 22.0 4.6 20.7 5.4 histening Comprehension 16.8 4.3 17.9 4.0 15.2 4.4 , 40, 9 J fr . -24- , Teachers who administered the tests in their class- rooms were asked to review the test publi.sher'..$ obj,ectis and to determine the degree of match beeween these objectives and'the basic skills they .expected theix students to have obtained. Using these techniques, the researchers 1.4ere satisfied . that the test did, indeed, test a sample of objectives.that was consistent with-t.he objectives used in eeaching in the public schools of the Virgin Islands of the United States. The estimates of reliability of the test scores are pre- sented in Table 5. The kR-2O reliability estimate2 for each test is reported along with' the.(KR-20 estimate'for the mainland standardization samples as presented in the Technical Data Report. The issue of interpreting these reliability estimates is a complex one and,will be dealt with in more detail at the conclusion of this rei)ortc The'author felt the need to have at least a tentative criterion for making decisions regarding the acCept- ability of the reliability vtimates obtained from the V.I. sample of examinees. IThe Stanford AChievement Test is considered rxx =.[n/(n-1)] [0.2x-EIN/(o2x)] where rxx= the reliability estimate 11.:- the number of scores 2 - 0,x the variance of the dis'tribution of scores p= the proportion of examinees marking the ,correct answer on a particular Item q= 1_p 99 -(From Stanfoid Achieve- ment Test: Technical Data Report, p. 35) Ift to have more than acceptable reliability whet administered to the population Of examinees upon which it was standardized (i.e.: continental U.S. students). Among the indications of this are numerous.reviews of the test in the literature (Kasdon, 1974; Lehmann, 1975; Chase, 1978;, Ebel, 1978; Thorndike, 1978), and the fact that it is widely uged in the schools. However, the literature is replete with studies which indicate that standardized tests of academic achievement tend to produce le.s reliable scores when administered to students from low socioeconomic status homes and to those who are culturally different from the mafority of those on whom the test was normed (see reviews and discussions in Anastasi, 1958; Tyler, 1956; and Deuesch, 1960). Therefore, if the reliability estimates obtaited from a sample of U.S. Virgin Islpds students who took the Stanford Achievement Test are at least equal to the reliability estimates-obtained from the standardization samples, it is reasonable to conc1u4 that the test scores are reliable indicators of acadeMic achievement for these students. For each reliability estimate obtained from the V.I. sample, a reliability difference was found by subtracting the standard- ization groups' reliability estimate from the local groups' reliability estimates. Thd datribution of these differences 'is shown by the histogram in1Figure 1. The.median reliability difference was -.038 with a range from -.20 to +.05-with the distribution skewed'to the left (i.e. negatively) quite markedly. In addition in at attempt to observe these reliability differences from another perspective, for each pair of relia- bility estimates (the standardization group estimate and the Table 5 Stanford Achievemeht Test Raw Score Reliability Estimates STAND. USV1 GROUPS SYSTEM TEST ,KR-20 KR-20 ST THOMAS/ ST JOHN KR-20 ST CROIX KR-20 Grade 12-TASK II Level .91* .90* .84* .95 .91 ..90 Reading .94 .93 'Mathematics .94 .91* English .94 .87* Grade 10-TASK I Level Reading , .95 .93 .94 '.94 Mathematics ' .94 .98 .99 .89* English .95 .92*. .90* .95 - Grade 8-Advanced Level Vocabulary .89 .8k* .78* .87 Reading Conprehension .94 .95 :97 .93 Mathematics Concepts, .86 .74* .81* 75* Mathematics CoMputation .89 , .85* .83* .87 'Mathematics'Application .91 .83* .83* .82* Spelling ,94 .9,3 .92* .93 language .94 .88* .84* .91* Grade 6-Intermediate II Level .86* .80* ' ... Vocabulary .90 .8).Y Reading Comprehension .94 .93 .94 .91 Word Study Skills .95 .93* .94 .92* Mathematics Concepts .85 ,.79* .78* .77* Mathematics CompUtation .90 4.85* .88 .78* Mathematics Application .92 .89 $pelling .94 .94 .95 ".93 Language .94 .92* .93 .87' Grade 4-Primary III Level, .75* .88 Vocabulary .88 .83* Reading .96 .91* :91* .92* Word Study Skills .94 .90* .88* .92 Mathematics Concepts .86 77 .66* .84 Mathematics, Computation .87 .83* 75* .87 Mathematics Application .92 .86* .84* .81* Spellin .93 .93 .91* .94 Language .92 .86* .84* .38* -27- Table 5 (cont.) TEST STAND. USVI ST THOMAS/ GROUPS SYSTEM ST JOHN ST CROIX KR-.20 KR-20 KR-20 KR-20 Grade 2-Primary I Level Vocabulary .86 .72* .71* .71* fReading Part A .94 .98 .99 .94 -Reading Part B .95 .92* .91 .92* Word Study Skills .93 .92 .91 .93 Mathematics Concepts .81 .71* .69* 74* Mathematics Computation .87 .81 .78* .83* Listening Comprehension .77 :74 .70* .72' *Significantly lower than the standardization groups KR-20 at p=.05 o -28- V.I. sample estima,te), the hypothesis that the differences in A reliability obtained were less than zero was tested. Reliability estimates" were tran*sformed using Z-transformations to normalb.ze the skewness of the distribution of the correlation measures and the hypotl'esis tested with t-tests3. 'On-e tailed signifi- cance tests were used. As indicated by Table 5, 64 of 198 com- parisons show lower reliability'in 'the V.1.-sample (i.e. differ-. 7m-ices less than ',:ero). A note of caution is in order in interpreting the results of 'hese t(Sts of,5,ignificant differences. As prviously poihte'd out, cluster sampling was used in obtaining the sample of Vir,in Island studentt to be-tested rather than Simple random rsampling.. The result of this is that the actual standard error of the sample may very well be larger than the one used in cal- culating the t statisticAestimatecebyi [1/(N-3).01). The result of this would be that the values of C obtained were larger than they should have-been and that someof the differences from zero that were noted in Table 5 to be significant at the p=.05- level may actually not have been. -To put it in technical terms, the'probability of Type I error is probably.greater than .05 in each of these hypothesis tests. This is a definite weakness in any conclusions we might draw from these tests. However, from a practical point of view, given the decisions 'to be made, 3 Z = loge (1.-I-rxyl (1-r ) xy T = Z E(Z) V 1/(N-3) 33 (Hayes, 1973,'pp. 662-667) OOP' Figure 1 - Frequency Distribution of Differences Between the Standardization Group.Reliability Estimates and the V.I. Sample Reliability Estimates (Ar) 14 F. 12 e- q 10 e n :8 6 4 2 -4 -.20 -.18 -.16 -.14 -.12 -.10 -.08 '.06 -.04 -.02 0 .02 .04 adr -31 Type I error is the error oi preTerence.. That is, the cpnse- quences of mistakenly assuming that scores are less reliabli: for V.1. would be that we would either look more closely at these tests from which the scores cdme or discard Ehe results,of the testing as being unreliable for V.t; students. In this case .p what i lost is mach time and, possibly ziome money. On the other hand, the consequene of Type IT errors (mistakenly assuming that;local scores are at least as reliable as the 4 standardization group's" scores). would be,to go,ahead and use the unreliable seores,to make Aecisionsp.about levels of V.I. students and,possibly, to make decisions regard- ingiastructionalstrategies that will be used in the schools. In essence, then, what results is a rather liberal test of ihe hypotheses and, givenThe nature of the decisions to be made, this May not be totally undesirable. However, it must be kept in mind when interpreting çihese results that the actual level of Type I error iS not known and that it is probablY,higher than .05. In any.event, we can'use Table 5 to flag tests where reliab-ilities may be less than acceptable. It was noted that, in the majority of cases, the variances of Ole raw scores obtained by the V.I. sample were considerably fower than those reported for the standardization groups. This homogeneity is a phenomendh commonly found when teiting.samples drawn from' populations composed largely of person.from low socibeconomic status homes. "The reliability of any test is partially dependent oh the sample of individuals tested to.. obtain the Coefficient. In general, the more heterogeneous -32- the sample with respect to whatever the test is measurinL the higher the reliability coefficient will be" (Technical Data Report), p. 35). The standardization groups' reliability co- efficients can be adjusted for homogeneity using the variances obtained from the loc.41 sample making reliability comparisons more meaningful.4 Using the adjusted reliability estimates for the standard- , ization groups, the differences between this group's and the V.I. sample's reliabilitylestimates,was calculated employing She same procedure used with the unadjusted estimates. The distribution of these differences is shown in the histogram in Figure 2. The median reliabilPty difference using the adjusted estimates was -.002 with a range from -.06 to .02. As with the unadjusCed scores, the' distLbution is negatively skewed, but. not as markedly as with the unadjusted reliability estimate differences. When the standardization group's reliability estimateare adjusted for homogeneity, the diTYerences between the reliability estimates of the two samples be&ope fewer and smaller. Table 6 presents the adjusted estimates of reliability for the standardization groups' and the results of tests of the hypotheses that the differences between the standardization . groups' relfability estimates and the estimates of relthbility 4 Using the formula pxy = (l_c52 x */62 x ) (l_p x *y*) . 4' . where pxy and(' 2x are, in this case, the reliability coefficient and variance of the standardization groups and px*y* and 02x* are the same statistics for the V.1. sample. (Hayes, 1973) Figure 2 Frequency Distribution of Differences Between tip Standardization Group Adjusted,ReliabOity Estimates and the V.:I. Sample Reliability EstimaCes (Ar) 50, F 45 40 c 35, 30 )5 20 .% 10 5 t r o Table 6 Adjusted:StanfOrd Achievement Test Raw Score,Reliability Estimates TEST ." USVI SYSTEM ST THOMAS/ST YOHN ST. CROIX Adj. Stand. Groups KR-20. Local Sample KR-20 'Adj. Stand. Groups KR-20 Local Sample KR-20 Adj. Stand. Groups KR-20 'Local Sample KR-20 . , 4 , Reading Mathematics English Reading Mathematics . English ,--' .93 .88 .91 93 .97 .93 Grade 12 TASK.II Level .91 .84 .90 .94 .99 .90 .94 .91 .91 .94 .90 .95 .95 .90 .91 .94 .89 . .95 . .93) .87 .91 Grade .91 .85 .91 10 TASK I_Level .98 .92 .97 -.92 Table 6 (cont.) TEST USVI SYSTEM ST THOMAS/ST JOHN ST. CROIX Adj. Suand. Local7d1 Adj. Stand. Local Adj. Stand. Local Groups KR-20 Sample'"1, KR-20 Groups KR-20* Sample KR-20 Groups KR-20 Sample KR-20 Grade 8 Advanced Level Vo,cabulary .32. .81 .79 .78 .87 .87 1 Reading Comprsension .94' .95 .95 .97 .92 .93 as_hmatics Concepts .79 ;79 .80 .81 .75 .75 . . MIt;1..nlatics Computation .85 . ..85' .83 .83 .87 .87 .> nntht:matics Application .83 .83 - .83 .83 .83 .82 , ) Speiling .93, .93 :92 A .93 '.93 . Language .90 .88* .86 .84 .92. .91 Grade'6 - Intermediate II Level VoL:ab_ulary .36 .31) *.87 '.86 .81 *.80 . Reciding ComPrehension .92 .93 '.,,93 ' .94 .91 .91 , WorK Study , _Skills .93 .93 .94 .94 .92 .92 , \ Mathematics Concepts .79, .79 .78 , .78 .77 .77 MathematicsComputation .85 .85 .87 .. .88 .79 .78 n Mathematics Application .90 .89 .90 .89 .819 Spelling .95 .95 .93 .93 Language 4 i .9? .92 1 :92 .93 .88 k Table 6 (cont.) TEST ym SYSTEM ST THOMAS/ST JOHN ST.. CROIX Adj. Stand. Local Adj. Stand. Local Groups -Sample ' ,Groups Sample KR-20 KR-20 KR-20 KR-20 Adj. Stand. Groups KR-20 Local Sample KR-20 Grade 4 Primary III , Vocaoulaty- .84 .83 .76 .75 .88 ,.Reading Comprehension .93 91* .92 _ .91 .93 .92 Word Study Skills .91 .90 .89 .88 .92, .92 iMatheMatics Concepts .78 .77 .68 .66 .84 ,84 Mathematics Computation' :83 :83 .76 .75 .87 .87 Mathematics Appliation .87 .86 ' ".35 :84 .88 .87 Spelling .93 .93 .91 :91 .94 .94 Language .86 .86 .84 . 84- .88 .88 Grade 2 -'Primary I Level Vocabulary .76 .72 .72 .71 .76 .72 Reading -,Part A .97 .98., .97 .99 .94 .94 :- Redding - Part B .93 .92 .92 .91 , .94 . .92 Work Study Skills .91 .92 .89 .91 , ' .92. .93 Mathematics ConCepts .72 .71 .70 .69 .74 .74 Mathematics Computation .80 .81 .76 .78 ,82 .83 Listening Computation .78 .74 .73 .70 .78 .72 f4 *Significantly lOtqer than the standw-dization groups' KR-20 at p=.05 4 ..-- 1 t. f-37 for che scores obtained by thy V.I. .iariply are less than ,,:ero. Again, the caution mentioned in discussing the hypothesis tests using the unadjusted reliability estimates holds true. The actual level! of Type I error involved in these tests is not really known and is most probably higher than .05: Even under s condition, however, only 2 out of 108 comparisons showed differences significantly less than zero; Since the p=.05 level was formally used, these two differences could be expected as a result of Type 1 error (i.e. aS' a result of.chance). In fact, slightly more than 5 differences significantly less than zero .4ould have been expected on a chance basis. The standard error of measurement5 for the raw scores of the 'V.I. sample are shown in Table 7. ". . .when the reliability of a test is interpreted in terms of the standard error of' measurement, the problem of the influence of heterogeneity [or homogeneity) is taken into account, since the formula for the standard error of measurement includes the standard devia- tion of the sco'res" .(Technical,Data Report', p. 35). The standard error or measuxement can be thought of as the stan- .. dard deviation of the difgerences between the scores obtained on the Lest anci the true scores (the scores the examinees would have received if the`test were perfectly reliable). As such, it can be used to determine an interval within which we can be' confident tnat the true score falls. For instance, we can be confident tLat the'true score\would be within one standard 5 SE = 1-rxx (Gronlund, 1976) chore SE,is the .standard error of measure- ment S is the 'standard deviation of the scores and r\K is the reliability coefficient. Table 7 . Stanford Achievement Test Standard Error of Measurement_Estimates TEST STAND. GROUPS S.E.M. USVI SYSTEM S.E.M. ST THOMAS/ ST JOHN ST CROIX S.E,M. Grade 12 - TASK II Level Reading 2.60 3.65* 3.69** 3.40* Mathematics 2.80 3.01 2.98 2.97 .E4ag1ish 3.30- 3.36 3.45 3.44 Gradg 10 - TASK I Level Reading 2.50 3.70*. 3.48* 3.50* Mathematics 2.60 2.07 1.69 English 3.10 3.38* 3.41* 3.70* , Grade 8 - Advanced Level Vocabulary 3.10 3.15 3.10 3.05 'Reading Comprehension 3.60 3.45 3.02 3.46 MatheMatics Concepts . 3,,,(f0 2:65 2.61 2.67 Mathematics Computation 2.90 2.95 2'.92 2:94 A Mathematics Application 2.60 2,72 2.72 2.76 Spelling 3.30 3.27 3.33 3.40 Language , 3.90 4.20* 4.23 4.07 Table 7 (conl.) f STAND. USVI CROUPS SIISTEM TESF S.E.M. ST THOMAS/ ST JOHN -S.E.M. ST CROIX S.E.M. Grade 6 Intermediate II Let.el Vocabulary 3.0 3,.02 3.05- 3.05 Re,-idin,-, Com;)r,, it hs i on 3.50 3.29-- 3.11 3.46 Word Study Skills 2.90 2.93 2.82 2.91 * Marhematics Cricepts 2.60 2:.60' 2.59 2.64 MathematiCs.Computation 2.90 2.88 2.-78 2.94, Mathematics Applicatio.1 2.60 2.63 2.64 2.60 . Spellint-; 3.30 3.33 3.18 3.30 :Anguage 4.0 1.91 3.98 4.14 I .. t;rade 4 Primary III Level Vocabulary 2.30 2.96 2.95 2.91 Reading Comp'r,' nsion 3.20 3.46* 3.38 3.35 Word Study Skills 3.00 3.16 3.17 3.06 Mathcmaelcs Concepts 2.40 2.,55 2.55 - 2.47 Mathematics Computation 2.50 2.52 2.55 2.47, Mathe.natics Application 2.0 2.16* - 2.21* 2.18 Spelling 2.70 2.69 , . \ 2.76 2.74 Language 3.10 3.28* 3.26 3.25 Table 7 (cont.) TESTf STAND. USVI ST THOMAS/ GROUPS SYSTEM ST JOHN ST CROIX S.E.M. S.E.M. S.E.M. S.E.M. Grade 2 Primary I Level Vocabulary 2.50 2.68 2.57 2.72 Reading - Part A 2.50 1.95 1.52 2.51 Reading - Part B 2.40. 2.53 2.52 2.68 Word Study Skills 2.80 2.65 2.62 2.68 - Mathematics Concepts 2.30 2.38 2.38 2.37 Mathematics Computation 2.20 2.17 2.15 2.21 Listening Comprehension 2.0' 2.22* 2.17 2.34* *Significantly higher than the Standardization Groups at the 1)=.05 level. -41- error of the score the student actually received (the observed score) around 58Z of the time. The true score would be within two standari errors of the observed score approximately 96% of the time. Naturally, the lower the staodard error of measure- ment, the more reliable the scores.' x2 (chi-squared) tests6 were used to test the hypotheses that the standard errors oT measurement eor the'tesc scores in the Virgin Islands sample were 'greater than those for the standardization sample. Sixteen of the 108 tests show signifi- cantly higher staadard errors for the V.I. sample'at-the p=.05 level of significance. Nine of them occur in the 14.gh school tests in reading and English areas. These tests need.to be looked at closely. Among the remaining seven there seem to be no patterns. It .should be ncited, however, that in two of these cases, Mathematics Applications in grade 4 and Listening Com- prehension in grade 2, the differences are in one district and in the total system scores. Since the total system scores are obyiously affected by the individual district scores, it is possible that the large total system standard errors maybe a result of the lower reliability obtained from the district Scores. 6 6df = S2/°2 where df is the number of degrees of freedom, 52 is che square o the V.I. sample standard errorof .measuremOnt, 02 is the square of the standardization groups standard error of measurem nt. (Darlington, 1975) 49 WI% Summary and Conclusions The scores obtained from the testing of a representative ;sample of U.S. Virgin rslands students using the 1973 edition of the Stanford Achievement Test appear to be both content Valid and reliable. This is significant in that this test, and all standardized tests of academic achievement published in the United States, have been designed without including studies of noncontinental U.S. public school curriculum in the test plan- :, ning process or using noncontinental U.S. students in its standardization studies. 'It is clear that the test objectives, as steed by the pub- , lisher,.are a good match for those used in U.S. Virgin Islands public schools. -In addition, the reliability estimates cif the scores obtained from Virgin Islands stlidents are, in most cases,- not significantly different from those 6btained using the conti- nental U.S. standardization samples. At this point it ma)t, be' useful to examine the distinction between differenceshat are "statistically significant" and those that are "educationally significant." The statement that two values are "statistically significant" implies that we are confident that the difference between the two values is not zero. This'is no guarantee that the differences are'not-trivial. -For instance, we may weigh . two packages on the same, veryaccurate, scale and find that one weighs 25 kilOgrams while the other weighs 25.5 kilograms. If we were trying to decide which of these packages to assign each of two people to carry based on their relative strengths, we could probably conclude that'either person could carry either.package. The difference oT one half of a kilogram was real (i.e. nonzero), but it was so small that it was trivial. Likewise, differences in reliability estimates noted in this study may be statistically significant, but so small as to allOw us tb conclude that the test scores were reliable. enough , for u -o.use to make educational decisions (i.e. the differences . e not educationally. significant). With the exception ofthe 4 grade 12 and grade 10 Reading test scores and the grade 10' English test :Icores from the St. Croix district, the differences observed in standard error of measurement estimates seem to be so small as not to be educationally significant. Putting ,a'siOe the quesgion of the comparability of the obtained reliability eStimates between the standardization samples and the U.S. Virgin Islands sample, the question of whether or not the scores obtained from the U.S.V.I. sample are reliable enough for vus to use ithem to make educational decisions needs to be addressed. "The degree of reliabiligy we demand in our educational measures depends largely on the nature of die decision to be made", (Gronlund, 1976, p.124). Standardized test results are used by schooliersonnelas one source 'of information for 'making iiistruc- tional and curricular'decisions. Other sources of information such as teacher made clasSroom tests and observational techniques -are combined with the results of standardized tests before final educational decisions are made in schools. Finally, this partict-, ular study was designed to point out strengths and weaknesses in basic skills areas in U.S. Virgin Islands public schools. Those -414- 1, persons entrusted with the responsibility for making curricular .., and instructional decisions in the Department of Education will, use,this and other information before making changes in what goes on in schools. Further, decisions made will always be open to .confirma-tion and change. Cronbach (1970) points out that the reversability of decisions made o'n the basis of test data is an important factor to Like into consideration in making judgements concerning desired levels of reliability. The reli- ability estimates obtained from the_U.S.V.1. sample which seem to cluster from the middle .80's to the middle .90's are more than adequate to allOw the confident use of the obtained scores. , , 1 , r t 4 . ' 73 0 / t - -45- References Anastasi,,A. DifferenCial Psychology (3rd ed.). New York: Macmillan, 1958. Asher, W.J. Educational Research and Evaluation Methods: Boston: Little, Brown & Co., 1976. Chase, C.f. Rtview of the Test .of Academic Skills. In 0.1:7. Burns (Ed.), -The Eighth Mental Measurements Yearbook (vol. 1). Highland Park, N.J.: Gryphon Press, 778. Cronbach, L.J. Essentials of Psychological Testing (3rd ed1). New York: Harper and Row, 1970. Cronb'ach, L. J . & P.E. Construct validity in psychological test's. Pycholooical Bulletin, ,1955, 52, 281-302. Darlington, R.B. Radical and Squares. Ithaca, N.Y.: Logan Hill Press, 1975. ; Deutsch, M. :iiaority group and class status Vs related to social and personality factors in schoUstic achievement (Monograph No. 2). , Ithaca, N.Y.: Tile Society for Applied Anthropology, 1960. . Ebel, R.L. Must all tests be valid? American Psychologist, 1961, 16, 640-643. Ebel, R.L. Review of the 1973 edition of the Stanford Achievement Ten:. In O.K. Buros (Ed.), The Eighth NIental Measurements Yea'rbook (vol.1). Highland Park, N.J.: Gryphon Press, 1978. Ebel, R.L. Essentials of Educational ,Measurements (3rd ed.). - Englewood Cliffs, N.J.: Prentice-Hall, 1979. French, J.1L & Michael, W.B. (Cochairmen) Standards for Educational and Psychological Tests and Manuals. Washington, D.C.: American Psychological Association, 1966. Gronlund, N.E. Measurement and Evaluation in Teaching (3rd ed.). New York:- MacMillan, 1976. Hayes, W.L. Statistics for the Social Sciences (2nd ed.). New York: HoTt, Rinehart, and Winston, 1973. Kasdon, L.11. The Stanforl Achievement Test (Review of the.1973- edit:ion of the Stanford Achievement Test). Reading 'Teacher, 1974, 27, 743+. -46- Lehmann, I.J. The Stanford Achievement Test Serips 1973 (Review of the 1973 edition of the Stanford Achievement Test). Journal cif Educational Measurement, 1975, 12, 297-306. Passow, A.H. Review of the 1973 edition of the Stanford Achieve- , ment Test. In O.K. Buros (Esi.), The Eighth Mental'Measure- ments Yearbook (vol. 1). Highland Park, N.J.: Gryphon Press, 1978. Stanford Achievement Test. Technical Data Repor.t. New York:v Harcourt Brace Jovanovich, 1975. Thorndike, R.L. Review of Ole Tes,t bf Academic Skills. In O.K. Buros (Ed.), The Eighefi Mental Measurements Yearbook (vol. 1). Highland Park, N.J. : Gryphon Press, 1978. Tyler, L. The Psychology of Individual Differences (2nd ed.). New York: Appleton-Century-Crofts, 1956. Warwick, D.P. & Lininger, C.A. The Sample Survey: Theory and -Practice. New York: McGraw-Hill, 1975.' 4, %,