USVI Public Records

Document text

DOCUMENT RESUME ED 267 120 TM 860 212 AUTHOR Bliss, Leonard B. TITLE Item Analysis and Report of Student Skills of Elementary School StLJents. Virgin Islands of the United States Public School Basic Skills Achievement Survey. Technical Report #3. PUB DATE Jan 8! NOTE 117p , For related documents, see ED 227 150 and ED 243 964. Some pages may be marginally legible. PUB TYPE Reports - Research/Technical (143) EDRS PRICE MF01/PC05 Plus Postage. DESCRIPTORS *Academic Achievement; *Achievement Tests; *Basic Skills; Elementary Education; Evaluation Methods; Grade 2; Grade 4; Grade 6; *Item Analysis; Language Skills; Mathematics Skills; Reading Skills; Standardized Tests IDENTIFIERS *Stanford Achievement Tests; *Test of Academic Skills ABSTRAC. A sample of slightly over lc,i5C students from even numbered grades in public schools in the U.S. Virgin Islands were given the 1973 edition of the Stanford Achievement Test (in grades 2, 4, 6, and 8) and the Test of Academic Skills (in grades 10 and 12) in an attempt to assess student academic achievement in the basic skill areas of mathematics, reading, and English Language. Technical Report 1 detailed the procedures used in test selection, sampling, and tent administration. Technical Report 2 reported the results of the analysis of scores for secondary students (grades 8, 10, and 12) including an item analysis and summary skills on all objectives. This report describes Phase III of the data analysis which involves a detailed item analysis of each item on each test given to the sample of students in grades 2, 4, and 6 as well as a summary of student skills based on their achievement along objectives nrovided by the test publisher and keyed to individual test items. Measures of item difficulty and item discrimination were calculated for the entire territorial school system and for the individual school districts. (Author/PN) *********************************************************************** * Reproductions supplied by EDRS are the best that can be made * * from the original document. * *************************************************************t********* O CV 1-1 t=t I-1J . v-.140,40144, vx, Virgin Islands of the United States Public School Basic Skills Achievement Survey Technical Report #3: Item Analysis and Report of Student Skills of Elementary School Students "PERMISSION TO REPRODUCE THIS MATERIAL HAS BEEN GRANTED BY TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) Leonard B. Bliss, Ph.D Caribbean Research Institute College of the Virgin Islands January 1984 2 . . NATIONAL INSTITUTE OF EDUCATION EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) -"/"This document has been reproduced as received from the person or organization originating it I 1 Minor changes have been made to improve reproduction quality Points of view or opinions stated in this docu ment do not necessarily represent official NIE position or policy BEST COPY AVAILABLE Abstract Introduction Contents Background Item analysis Difficulty indices Discrimination indices Summary of student skills Availability of data 2nd Grade (Primary Level I) V Page ii 1 3 4 4 10 12 13 -Vocabulary 15 Reading (Part A) 18 Reading (Part B) 20 lord Study Skills 23 lathematics Concepts 29 Iftithenatics Computation 33 11itening Comprehension 37 . 4w-1i-tad. (Primary Level III) 41 Nocabulary ;;Reading Comprehension Study Skills thematics Concepts thematics Computation 8 page -de (Intermediate Level II) 42 44 47 52 56 61 63 70 Lary 71 Comprehension 75 Study Skills 80 thenatics Concept$-. 84 tbtsaatics Computation 88 Mathematics Application 92 Aliening 96 language 99 scussion 108 arences 113 BEST COPY AVAILABLE 3 Abstract A sample of slightly over 1500 students from even numbered grades in public s- 3ols of the U.S. Virgin Islands were given the 1973 edition of the Stanford Achievement Test (in grades 2, 4, 6, and 8) and the Test L.. Academic Skills (grades 10 an 12) in an attempt to assess student academic achievement in the basic skill areas of mathematics, reading, -and English Language. This report describes Phase III of the data analysis which involved a detailed item analysis of each item on each test given to the sample of students in grades 2, 4, and 6 as well as a summary of student skills based on their achievement along objectives provided by the ",test publisher and keyed to individual test items. Measures 'item difficulty and item discrimination were calculated *46: the entire territorial school system and for the individual school districts. 4 BEST COPY AVAILABLE 1 At the request of the U. S. Virgin Islands Department of Education and the Board of Trustees of the College of the Virgin Islands, the Caribbean Research Institute embarked on a study of basic skills achievement in U. S. Virgin Islands public schools. It soon became clear that any strategy designed to improve basic skills achievement needed to start off with a fairly detailed description of current achievement levels of students in territorial public schools. It was found that this was not available. The task force set up to d,3sign the study decided that the most efficient way to obtain information on levels of basic skills achievement was to administer a standardized -achievement test to a representative sample of students and to analyze the results of this test. Technical Report #1 (Bliss, 1982a) describes, in detail, the process used to Choose an appropriate test. Finally, the Stanford Achieve- ment Test (1973 version) was chosen as the instrument of -choice. Briefly, the reasons for choosing this achievement test battery were that 1) it covered the grades K-12, 2) it seemed, on initial observation, to be a good match with subject content taught in the schools, 3) it was technically sound given the population on which it was standardized a seemingly representative sample of mainl-rd U. S. students), 7111- it would report out criterion referenced results. 5 BEST COPY AVAILABLE 2 Due to various organizational and fiscal contraints, only students in even numbered grades were tested. This seemed acceptable since many of the objectives tested by the Stanford Achievement Test carry across adjacent levels of the test and there was no reason to suspect that the patterns of academic achievement of students in odd numbered grades were different from those in even numbered grades. These constraints are detailed in Technical Report #1. Sampling of students to be tested was done using a cluster sampling technique with classes as the cluster unit. The details of the sampling procedure can be found in Technical Report #1 along with a discussion of the effects of using this sampling technique on the information obtained. Table 1 presents a breakdown of the sample as it finally emerged. Table 1 U. S. Virgin Islands Sample Sizes Test Level Total System St. Thomas St. John District St. Croix District Task II 129 74 55 Task I 254 J,si 87 Advanced 345 173 172 Intermediate II 227 146 81 Primary III 346 186 160 Primary I 234 143 91 Total 1535 889 646 6 BEST COPY AVAILABLE 3 Testing was done at the grade level recommended by the test publisher during the week of October 21, 1980 in the St. Thomas/St. John district and during the week of December 1, 1980 in the St. Croix district. Testing materials and completed answer sheets were collected, answer sheets checked to determine compliance with narking instructions, and answer sheets sent off to be machine recorded. Background Technical Report #3 This is the second of four reports that will deal with the results of the basic skills assessment described above. -40-7 Technical Report #1 detailed the procedures used in test selection, sampling, and test administration. More impor- , tautly, it established empirically, the content validity and the reliability of the Stanford Achievement test when it was administered to a sample of U. S. Virgin Islands public school students. This is particularly important since there masts no standardized test of academic achievement which includes USVI students in its standardization group. Technical Report #2 (Bliss, 1982b) reported the results of the analysis of scares for secondary students (grade 8, ancl2 ) including an item analysis and summary skills 1st-Objectives. This report examines the scores of elementary school is (grades 2, 4, and 6) and presents: an item analysis of each item on each test of the battery which includes indices of item difficulty and discrimination and BEST COPY AVAILABLE 4 2) a summary of student skills based on their scores on the items keyed to specific objectives. Item Analysis Difficulty indices The difficulty index of an item is found by taking the number of examinees scoring correctly on an item and divid- ing it by the total number cf students taking the test. In short, it is the proportion of examinees who scored correctly on an item and has a range from 0 (no students scoring correctly) to 1 (all studetns scoring correctly). Because of the fact it is a proportion, it is often designated in the literature as "p" (e.g. p = .75). It may be worthwhile to point out that the term "difficulty index" may be somewhat of a misnomer since items with high difficulty indices are actually less difficult than items wita low. difficiilty `indices. Nevertheless, the term and its definition have become standard in the area of psychometrics throughout the United States. Difficulty indices for each item on each test were reported out by the test scoring service. In addition, 'difficulty indices for examinees in the standardization A.M the same grade as local examinees at approximately 0 same time of year are reported. The test scoring used a Chi-squared test for proportions of each lty index to test the hypotheses that the proportions ocal students scoring correctly on individual items in 8 BEST COPY AVAILABLE 5 greater or less than the proportions of examinees in the standardization group scoring correctly at the .05 level of significance. Significant differences in either direction were reported out. Discrimination Indices The item discrimination index indicated the degree to which responses on one item are related to responses on other items on the test. The statistic indicates whether a person who does well on the test as a whole (that is, a -,person who is presumably high on the trait being measured) is more likely to get the particular item correct than a person who does poorly on the test as a whole. In other words, the item discrimination index indicates whether an 'item discriminates between those who do well and those who do poorly on the test as a whole. Taking the item diffi- culty and the item discrimination index into consideration, the developers of tests desire to construct tests which -!:discriminates well among examinees with varying levels of a alt. The item discrimination index is calculated by the 102u1a U L d N the number of examinees who have total test scores in the upper range of total test scores and who also have the item correct. 9 :JEST COPY AVAILABLE k-ssi4i4 5 L = the number of examinees who have total test scores in the lower range of total test scores and who also have the item correct. N = the number of examinees in the upper or low range of the test scores. By definition, d is the difference between the proportion of high scoring examinees who got the item correct and the proportion of low scoring examinees who got the item cGrrect. Upper and lower ranges generally are defined as the upper and lower 10% to 33% of the sample, with examinees ordered On the basis of their total test score. When total scores are normally distributed using the upper and lower 27% pro- , r duces the best estimate of d (Kelly, 1939). If the distri- bution of total test scores is flatter than the normal curve, the optimum percentage is larger and approaches 33% (Cureton, 1957). However, Allen and Yen (1979) found that, for most $.;413piications, any percentage between 25 and 33 will yield similar estimates of d. In this study, 27% was used as the upper and lower percentage because examination of selected aistributiors of actual test scores revealed nearly normal tributions. The theoretical range of d is between -1 and +1. How- ever, maximum discrimination is likely to occur when p = .50. en p B .50 the variance in item scores, which is p (1 p), is maximized. As an item becomes more difficult, it is less eiy that any student will score correctly on it. As it ecomes less difficult it is more likely that any student BEST COPY AVAILABLE 7 will get it correct. This could lead to the suggestion that all items should have p = .50, but the usefulness of this suggestion is mitigated by intercor-relation among items. In an extreme case, if the items on a test all intercorre- lated perfectly and had difficulties of .50, half the examinees would receive a total test grade of zero and the other half would have perfect test scores. Hence, there would be no fine discrimination between examinees' levels of achievement or whatever trait was being measured. In general, test designers tend to try to choose items with a range of difficulties that average around .50. Items of particularly low difficulty are often included in tests Insually along the earlier:itemS) for motivational-reasons. Item discrimination indices were calculated in this study to provide indications that items may be flawed when used with USVI students. Such flaws are ambiguity, the presence of clues, the presence of more than one correct answer, and other technical defects, If none was found p,n examination of the item, and it was determined that item did, indeed, appear to measure the objective it d to, the item was included in the overall iii of results. Any item that discriminates positively can mmke a contribution to the measurement of pupil achieve- , dlow indices of discrimination are frequently lobtained for reasons other than item defects. --*-tiiiaiidized achievement tests are designed to measure everal different types of learning outcomes (e.g. knowledge, 11 BEST COPY AVAILABLE '14 8 understanding, application, etc.). Where this is the case, test items that represent an area receiving "elatively little emphasis will tend to have poor discriminating power. For example, if a test ha_, forty items measuring knowledge of specific facts and ten items measuring understanding, the latter items can be expected to have low discrimination indices. This is because the items measuring understanding have less representation in the total test score and there is typically a low correlation between measures of knowledge and measures of understanding. Low discrimination indices here merely indicate that these items are measuring some- thing different from what the major part of the test is measuring. Removing such items from the test would make It a more homogeneous measure of knowledge outcomes, but it would also damage the content validity of the test because . it would no longer measure objectives in the understanding area. Since achievement test batteries need to measure a wide variety of objectives in a reasonably short period of time, they tend to be fairly heterogeneous in nature and moderately low discrimination indices tend to be the rule ,rather than the exception. To summarize, a low discrimination index alerts test Users to the possible presence of defects in test items but 7'-" does not cause them to discard these items if they appear to be functioning as they should. A well constructed achieve- Oft,test will of necessity, contain items with low discri- 1 iflating power and to discard them would -esult in a test 12 BEST COPY AVAILABLE 9 which is less, rather than more, valid. Due to these consi- derations, in this study itt.ns were examined if they had discriminatien indices lower than .20. This is a rather conservative criterion since items that discriminate as low as this may prov;_de useful information, but given the un- known test taking characteristics of USV1 students, it was decided to be particularly cautious in the item analysis. 13 BEST COPY AVAILABLE :, 10 Summary of Student Skills The items of the tests of the various levels of the Stanford Achievement Test battery were keyed to behaviorally stated instructional objectives. These objectives are grouped into a series of item groups. Tables are available which present objectives within item groups and the difficulty and discrimination indices for the item or items which evaluate tiNose objectives. These may be obtained from the _Caribbean Research Institute. In addition, the difficulty index for each item for the examinees in the standardization group is available. This standardization group consists of examinees who were in the same grade at approximately the ame time during the school year as the U. S. Virgin Islands simple. The national p values are used not as a means to compare '.-U. S. Virgin Islands students with mainland U. S. examies. storical and cull.ural differences between these two groups examinees makes this comparison an inappropriate one. lAcsophical cunsic ons aside, lowever, such comparisons are of little use to the people who make curricular decisions scho4Is. What these people need to know are the partiou- Aevels of skills of students as measured against well efined objectives, not how well their students achieved -skil3s as compared to other students. 'Nevertheless, since the skills and knowledges taught in is are seldom taught once, but are dealt with at a umber of grade levels where they are rtAnforced and broadened, 14 dbl. COPY AVAILABLE 11 the level of achievement on specific objectives should be expected to change from grade to grade for a particular student or group of students. The publishers of the Stanford Achievement Test take this factor into considera- tion by testing particular objectives across a number of levels (grades) of the test. In this study the national p values are used to indicate the relative level of difficulty of an item by which the performance of the local sample may be judged. For instance, it would be foolish to be dissat- isfied if 20% of the USVI students have indicated that they have reached an objective when only 18% of comparable students in the standization group had reached that same objective. What is more likely the case is that this is a -''clifficult and complex objective that had just been taught :7tecently and would be retaught and enlarged upon at a later Therefore, the following criteria were used in summariz- N,I, - "student skills. Skills are described as "adequate" if `Troportions of local examinees scoring correctly on .10iiii'iiasitring those skills are not significantly higher or $ Ak the proportions of the standardization group pwrectly or, if significantly higher or lower, the ,,,ions correct are within 10% of the standardization On correct-as reported by the test scoring lls are described as "strong" if the proportions e' 'scoring correctly on items measuring those *ftificantly higher and more than 10% greater dEST COPY AVAILABLE 12 than the sLandardization group proportion correct as reported by the test sccr3ng service. Skills are described as "weak" if the proportion of vocal examinees scoring correctly on items measuring those are significantly lower and mere than 10% less than the standardization group proportion correct. The 10% standard was set in the realization that some differences, whjle statistically significant may be educationally trivial and it was noted that most differences indicated as significant exceeded 10%. Finally, in cases where the scores of examinees from both the St. Thomas/St. John and the St. Croix districts were the same based on the criteria stated above summaries were based.on the entire USVI sample. Where differences were noted, the skills of examinees within each district were summarized separately. . AAvailability of Data Tables listing each objective on every test keyed to -.test items with national difficulty indices, difficulty jnaices for the entire USVI sample and the individual dis- Arict subsamples as well as item discrimination indices for eriA.Ole sample and each subsample are available and may be obtained from the Caribbean Research Institute, College of the Virgin Islands. 16 6LST COPY AVAILABLE 13 Grade 2 - Primary I Level The Primary I Level of the Stanford Achievement Test consists of sev.n tests with the number of items listed below: 1. Vocabulary 37 items 2. Reading Comprehension (Part A) 45 items 3. Reading Comprehension (Part B) 42 items 4. Word Study Skills 60 items S. Mathematics Concepts 32 items 6. Mathematics Computation 32 items 7. Listening Comprehension - 26 items The u:eans, standard deviations, and reliability estimates these tests, determined previously (see Bliss, 1982a), ro presented below: Mean Standard KR-20 Standard "Test . Raw Score Deviation Reliability Error bOlary Ciinpre- Oart A) 21.7 34.1 B) 29.5 ills 47.5 Con- tics Com- Compre- 1- 19.0 21.5 16.8 (r 1 Total USVI Sample 5.1 .72 2.6 13.8 .98 1.95 8.9 .92 2.53 9.4 .92 2.65 4.4 .71 2.38 5.0 .81 2.71 4.3 .74 2.22 BEST COPY AVAILABLE 'k Vocabulary Reading Compre- hension (Part A) Reading Compre- hension (Part F,) Word Study Skills Mathematics Con- cepts Mathematics Compre- hension Listening Compre- hension Vocabulary 171""" -Reading Compre- hension (Part Al ; 4 eading Compre- , .".,:heinsion (Part B) ;Mord Study Skills , Mathematics Com- prehension Mathematics Com- . utation on g Compre- 14 St. Thomas/St. John Sample 21.7 4.8 .71 2.57 36,3 15.2 .99 1.52 30.7 8.4 .91 2.52 48.7 8.7 .91 2.62 19.7 4.3 .69 2.38 22.0 4.6 .78 2.15 17.9 4.0 .70 2.17 St. Croix Sample 20.1 5.1 .71 2.72 30.5 10.3 .94 2.51. 27.6 9.5 -.92 2.68 45.6 10.1 .93 2.68 18.7 5.6 .75 2.37 20.7 5.4 .83 2.21 15.2 4.4 .72 2.34 Following are the item analyses and summaries of student evement on the objectives evaluated by each of the tests. lb BEST COPY AVAILABLE. 15 Vocabulary Item Analysis All items showed acceptable discrimination indices with the fol:owing exceptions: Item #1 was particularly easy for students in-the USVI sample (p = .94). This accounts for the low level of discri- mination by this item and is not significantly different from the standardization group value of p = .91. Item #2 was particularly easy for examinees in the St. Thomas/ St. John group in all classes tested (p = .99). This would account for the low level of discrimination level observed (d = .05). Item #3 and #4 were very easy for the examinees in the St. Thomas/St. John sample (p = .93 and .90, respectively. This accountVfortthe-low. levels-of the. observed,difficulty indices. Item #8 was a moderately difficult item for USVI examinees (p = ,66). The teacher asked, "If a place is not ." The alternatives were, 1) accidental, 2) secure, and 3) dangerous. Many students who scored in upper 27% of the examinees on this test chose "acciden- while the incorrect choices were more evenly spread --between the two distractors for those in the lower 27%. ere is some factor operating that draws higher achieving 4tudents to Choice #1, but an inspection of the item, itself to,reveal this factor. -4v 19 BEST COPY AVAILABLE 16 Item #9 was an easy one for St. Thomas/St. John school district examinees (p = .87). This accounts for the low level of discrimination that was observed (d - .13). Ittm #20 discriminated negatively for St. Croix district examinees (d = -.12). This moderately difficult item (p = .31) had the teacher ask "A small man is a ." The alternatives were 1) boy, 2) girl, and 3) woman. Most exam- inees chose "woman" as the correct choice and, in the case of St. Croix students this was true of slightly more of the upper 27% of students than of the lower 27%. There may be a cultural influence in the way language is used in the Virgin Islands which accounts for this. . , Item #32 did not discriminate between higher and lower achieving students in th St. Croix school district (d = 0). This was a difficult item for these examinees (p = .23) and this very likely accounts for the lack of discrimination observed. Item #27 showed low discrimination for St. Croix s:hool district examinees (d = .16). This was a moderately diffi- cult item (p = .47) and it appeared more difficult for these students than for the St. Thomas/St. John district examinees (p = .54). On the surface, the problem seems to be that a lower proportion of the upper 27% of examinees in St. Croix t.:,an in the St. Thomas/St. John district scored correctly on this item (25% vs 82%) and a higher proportion of the lower 27% of the St. Croix district examinees than of 20 BEST COPY AVAILABLE Av. 17 the St. Thomas/St. John examinees scored correctly on it (20% vs 48%). USVI examinees in both districts did as well on this item as the standardization group (p = .49) and there appeared to be no consistent pattern of incorrect reponses. The reason for this low level of discrimination is not evident from the infcrmation available. The same phenomenon is evident on item #28. Item #33 discrimin; ed poorly for St. Croix district examinees (d = -.04). he same phenomenon observed in Items 27 and 28 was observed. Summary of Student Skills USVI examinees from both school districts showed weak skills in standard English vocabulary which was made of words generally encountered in reading oral language and listening. The average difficulty indices for the St. Thomas/ St. John district, the St. Croix district, and the standardi- zation group of students were .57, .51 and .68, respectively with St. Thomas/St. John district students scoring indices significantly below those of the .tandardization group on seven of the 13 items and St. Croix students scoring lower n 11 of the 13 items. St. Thomas/St. John school district examinees scored adequately compared to the standardization group on recogni- 444ftel.'7 i on of words from nonfiction and reference sources (mean 4-et": .69 vs .72) while St. Croix district examinees showed weak vocabulary skills in this area (mean p = .57). 21 BEST COPY AVAILABLE 18 St. Thcmas/S,.. John school district examinees scored adequately on recognition of vocabulary found in the areas of mathematics ..-Ld science (mean p = ,64 vs .69 for the _standz.rdization group) while St. Croix district examinees showed weak vocabulary skills in this area (mean p = .57). USVI examinees ,,.nth school districts demonstrated adequate skills on recognition of vocabulary found in the area of the social sciences (mean = .60 for the St. Thomas/ St. John district, mean p = .56 for the St. Croix district examinees, and p = .61 for the standardization group). Reading Comprehension (Part I) Item Analysis All items on this test discriminated satisfactorily of St. Croix school district examinees. The following items ,,,4hatted poor discrimination for St. Thomas/St. John district t. tudents: Items 11 and #2 were very easy for these examinees p .99 and .93, respectively). Since most examinees :scored correctly on these items, it was almost as likely for student in the lower 27% of the group to score correctly is .it was for a student in the upper 27% of the group. This -t-mould account for the low discrimination indices (d = .05 and .18, respectively) observed for these items. Items #4 and #7 were very easy items for the St. Thomas/ John school district sample of studel.ts (p = .95 and respectively) and this would accou.4_ for the low 22 BEST COPY AVAILABLE 19 levels of discrimination that was observed on these items (c = .15 and d = .13). Items #10 and #13 were very easy for this group of examinees (p- = .98 and p = .95, respectively) accounting for the low discrimination indices which were observed d = .08 and d = .13). Items #16 and #19 were also very easy for this group of second graders from the St. Thomas/St. John school district (p = .95 for both items), explaining the low levels of observed discrimination (d = .15 for both items). Summary of Student Skills USVI examinees from both the St. Thomas/St. John and the St. Croix school districts displayed adequate skills on the test which evaluated their ability to match words which appear in Teading materials for primary grades and in general listening vocabulary with illustrations. The mean difficulty indices for the St. Thomas/St. John school district examinees, the St. Croix district examinees, and the standardization group were .78, .68, and .72, respectively. St. Thomas/ St. John examinees scored as well or better than the stan- dardization group examinees on all items. St. Croix examinees scored significantly lower on 15 of the 45 items on this test than did the standardization group, and did not score significantly higher on any of them. 2:3 BEST COPY AVAILABLE 20 Reading Comprehension (Part II) Item Analysis All items on this test discriminated satisfactorily except the following: Item #1 was very easy for the St. Croix school district sample of examinees (p = .93). This would account for the low level of discrimination (d = .16) which was observed for this question. Item #2 and #3 were very easy for the examinees in the St. Thomas/St. John school district (p = .99 and .95, respectively). This would account for the low levels of discrimination observed for these items. Item #16 rias a moderately easy (p = .70) item that did not discriminate satisfactorily among St. Croix examinees (d... .12). The item presented a picture of a little girl lifting a doll out of a gift box. The item read, "Dad gave Silly a surprise on her birthday. It was in a The alternatives were 1) box, 2) car, 3) toy. Ninety percent of the upper 27% of the St. Thomas/St. John examinees scored correctly on this item compared to only 76% of the :upper 27S of examinees in the St. Croix sample. Most of the upper group students in St. Croix sample chose alternative #2 giving the impression that they may have been reading beyond what was actually presented since on St. Croix, a ' gift would almost certainly be brought home by automobile. Even though the directions explicitly reminded examinees to look at the pictures in answering the questions, high scor- ing examinees found the distractor "car" attractive. 24 BEST COPY AVAILABLE 20 Reading Comprehension (Part II) Item Analysis All items on this test discriminated satisfactorily except the following: Item #1 was very easy for the St. Croix school district sample of examinees (p = .93). This would account for the low level of discrimination (d = .16) which was observed for this question. Item #2 and #3 were very eas:' for the examinees in the St. Thomas/St. John school district (p = .99 and .95, respectively). This would account for the low levels of discrimination observed for these items. Item #16 was a moderately easy (p = .70) item that did not discriminate satisfactorily among St. Croix examinees (d= .12). The item presented a picture of a little girl lifting a doll out of a gift box. The item read, "Dad gave PSally a surprise on her birthday. It was in a ;The alternatives were 1) box, 2) car, 3) toy. Ninety 4 ercent of the upper 27% of the St. Thomas/S.c.. John examinees cored correctly on this item compared to only 76% of the per 27% of examinees in the St. Croix sample. Most of the upper group student_ in St. Croix sample chose alternative #2 giving the impression that they may nave been reading eYond what was actually presented since on St. Croix, a ift would almost certainly be brought home by automobile. Even though the directions explicitly reminded examinees to kat the pictures in answering the questions, high scor- aminees found the distractor "car" attractive. BEST COPY AVAILABLE 25 22 Item #37 was a moderately easy (p = .67) item that discriminated poorly (d= .12) among St. Croix school district examinees. The item presented a picture of a little boy and a woman, both in bathing attire sitting on the ground with water behind them. The boy is building a sand castle. The item reads, "then he went swimming in the ocean while his mother sat in the ." The alternatives were 1) shed, 2) sun, 3) summer. All of the upper 27% of the St. Thomas/ St. John examinees scored correctly on this item while of 76% of the St. Croix examinees scored correctly. On the other hand, only 46% of the lower 27% of St. Thomas/St. John examinees vs 64% of the lower 27% of the St. Croix examinees Scored this item correctly. There was no pattern of distrac- tors chosen by students scoring incorrectly in either the high or the luw group and the overall discrimination indices were virtually identical (p = .66 on St. Thomas/St. John and p = .67 on St. Croix). For some reason, this item correlated poorly with other items on the test for St. Croix examinees, but the reason for this is not apparent from the information that is available. Summary of Student Skills The problem encountered at higher grade levels where examinees did not seem to attempt items coming up later in the reading comprehension test (Bliss,1982t) did not arise °' t this glade level. There were 42 items on this test 'compared to as many as 71 on the reading comprehension st-on -Intermediate Level II (grade 6). In addition, 26 BEST COPY AVAILABLE 23 none of these iems were of the form where examinees were required to read a passage of text before answering questions. This further supports the speculation that either USVI exami- nees tire quickly when a-'ed to perform on items where they must read longer passages, or that they are more deliberate readers who take a longer period of time to read these .passages. On this lcve), only 7% of examinees omitted the last item on the test. USVI examinees from both school districts showed adequate achievement in determining meaning that is explicitly stated " .2-.4-1 AL sentence or short paragraphs. St. Thomas/St. John and St. Croix iistrict examinees answered these With a mean proportion correct of .74 and .65, 'respectively, compared to a mean p = .69 for the stapdardiza- 3-, on group . local examinees showed adequate achievement in determin- i4, .e k I g that is stated implicity in a sentence or short .St. Thomas/St. John district and St. Croix aminees answered items testing this skill with a on correct of .72 and .65, respectively, com- ean p = .71 for the standardization sample. Skills lit Anal sis titems on the word stuuy skills test discrimina rily for students in the USVI sample. In each ra an item showed poor discrimination it appeared to .*ID a high difficulty index (i.e. examinees found the 27 BEST COPY AVAILABLE 24 item very easy to answer correctly). These items and the discrimination indices (d) and difficulty indices (p) are listed below. For examinees in both the St. Thomas/St. John and St.' Croix districts (d and p are for the combined res- ponses from both districts): Item# d 1 .06 .90 2 .13 .97 3 .16 .95 5 . 0 7 .(j.9 26 .11 .97 examinees in the St. Thomas/St. Croix district, only: Iteml d 4 .13 .97 8 .08 .89 10 .03 .97 11 .18 .94 13 .18 .92 20 .08 .197 24 .08 .97 25 .18 .94 33 .08 .97 38 .15 .94 53 .05 .97 28 BEST COPY AVAILABLE 25 Summary of Student Skills Consonant sounds USVI examinees from both the St. Thomas/St. Min and the St. Croix school districts showed adequate skills in all areas of discriminating among graphemes for consonant sounds that were measured by this test. These sounds were: a) the initial consonant sound /k/ b) the initial consonant sound /d/ c) the initial consonant sound /p/ d) the initial consonant sound /r/ e) the initial consonant sound /j/ . f) the final consonant sound /t/ g) the final consonant sound /1/ h) the final consonant sound /d/ i) the final consonant sound /k/ Consonant clusters St. Croix school district showed weak skill in their ability to discriminate among graphemes for the constant cluster of'an initial consonant plus r (gr). Thomas /St. John district examinees showed adequate gielilevemnnt in this skill. USVI examinees from both school districts demonstrated adequate achievement on all other items on this test which measured their abilities to discriminate among graphemes for consonant clusters. These clusters included consonant _clusters of: a) an initial consonant plusr (tr) b) an initial s plus a consonant (sp) 29 BEST COPY AVAILABLE 26 c) an initial s plus a consonant (sk) d) an initial consonant plus 1 (cl) e) a final ,onsonant plus t (nt) f) a final n plus a consonant (ns) g) a final n plus a consonant (ns) h) a final consonant plus d (nd) i) a final consonant plus k (lk) Consonant degraphs USVI examinees from both the St. Thomas/St. John and the St. Croix school districts showed adequate skills in discriminating between consonant degraphs on all _items of this test which measured this skill. These ,consonant degraphs included: a),../sh/ in the initial position b) /ch/ in the initial position c) /th/ in the initial position d) /hw/ in the initial position Ly r e) /ng/ in the final position Vowel sounds USVI examinees from both school districts demonstrated exceptionally strong skill in discriminating among graphemes for the short vowel sound /e/ spellea e. St. Thomas/St. John school district examinees showed strong skills in discriminating among graphemes for the following vowel sounds: --- a) the long vowel sound /o/ spelled o-e b: the long vowel sound /yoo/ spelled u-e c) the long vowel sound /e/ spelled by the diagraph ee 30 BEST COPY AVAILABLE 27 St. Croix school district examinees displayed adequate achieve- ment in these skills. St. Croix examinees displayed weak skills in the ability to discriminate among graphemes for the long vowel sound /e/ spelled ea. St. Thomas/St. John district students-showed adequate achievement in this skill. USVI examinees from both school districts showed ade- quate ability to discriminate among graphemes for the follow- ing vowel sounds. The: a) short vowel sound /a/ spelled a b) short vowel sound /i/ spelled i v.) Short vowel sound /o/ spelled o icr; :_short vowel sound /u/ spelled u - e) ,;,long vowel sound /a/ spelled a-e 'long vowel sound /i/ spelled i-e long vowel sound /a/ spelled by the diagraph ai - long vowel sound /a/ spelled by the diagraph ay long vowel sound /oo/ spelled by the diagraph oo long vowel sound /o/ spelled by the diagraph oa ong vowel sound /o/ spelled by the diagraph ow us r combinations - USVI examinees from both riots showed weak skill in the ability to discimi- - . -1graphemes for the 'vowel plus r combination fur/ 41 1011- 4,a,thomas/St. John school district examinees showed ty in discriminating among graphemes for the vowel plus r combinations: 31 BEST COPY AVAILABLE'" 28 a) /ur/spelled er b) /or/ spelled or St. Croix district examinees showed adequate achievement in these two skills. Examinees in both school districts displayed adequate skills in discririnating among graphemes for the following vowel plus r combinations: a) /er/ spelled er b) /ar/ spelled ar c) /ur/ spelled it d) Air/ spelled ur Silent consonant letters - USVI examinees from both the St. Thomas/St. John and the St. Croix school districts dis- played adequate skills in discriminating among graphemes for the: a) hn spelling of /n/ b) wr spelling of In c) ght spelling of /t/ St. Croix school district examinees showed weak skills in discriminating among graphemes for the : a) lk spelling of /k/ b) mb spelling of /m/ St. Thomas/St. John examinees showed adequate achievement in these skills. Diphthongs St. Thomas/St. John school district examinees showed strong skills in their ability to discrimi- nate among graphemes for the diphthong /ou/ spelled ow. 32 BEST COPY AVAILABLE 29 St. Croix district examinees showed adequate skills in this area. USVI examinees from both school districts displayed adequate achievement in discriminating among graphemes for the following diphthongs: a) /ou/ spelled ou b) /oi/ spelled of c) /oi/ spelled oy Mathematics Concepts Item Analysis Item 11 and 82 were very easy for USVI examinees from --both school districts (p = .97 for the two items) and this would account for the low discrimination indices which were observed (d a .05 and .06 respectively). Item 13 discriminated poorly for USVI examinees from -4.both school districts (d = .02). This was a moderately easy item (p = .83) which required examinees to count the number of dots in a set of dots arranged 4,4 ,4nd choose the number of dots in the set from the alternatives 1) 15, 2) 16, 3) 17, 4) 18. It appears that examinees in the er 27% of student scores on this test are just as likely 67; ; to-score correctly on the item as are those in the lower 27% -theHproportion correct for the whole sample. Examinees .81 and .80, respectively) and this is almost the same 33 BEST COPY AVAILABLE 30 scoring incorrectly seemed to choose alternative #2 (16) most often indicating that the problem they are having is not due to their ability to recognize the numeral 17, but may be due to the counting method or pattern they are using. This latter skill may not correlate highly with other skills measured by the test. Item #2 did not discriminate well for examinees in either the St. Thoams/St. John or St. Croix school districts (com- . bined d = .09). This was a very easy item (combined p = .95) and this would account for the low discrimination index which Mp ,was obtained. Item #14 did not discriminate satisfactorily for :aainees in ic t'ie St. Thome. 'St. John school district (d = .08). was _a difficult item for these examinees (p = . 29) and this _should account for the low level of discrimination that was observed. Item #19 did not discriminate well among St. Croix school district examinees (d = .12). In this moderately cult (p = .38) item, examinees were asked to observe Circles arranged in a random manner and to choose the eh-half the circles are colored in. There appeared ttern of incorrect responses. As in Item #3, :were required to come up with a strategy for count- rcles and it may be that this skill is not highly with the others on the test. itei#22 discriminated poorly for St. Croix district zees (d "1 .08). This was an item of average difficulty 34 BEST COPY AVAILA131.1 31 (p = .59) which showed the numeral 431 and required examinees to identify the digit in the 100's place from among the alternatives, 1) 1, 2) 3, 3) 4, 4) 0. St. Croix examinees did well on this item compared to the standardization group (1) = .56). No pattern of incorrect responses could-be noted. Item #23 did not discriminate well for the smaple of USVI examinees (d = .06). This was due to the fact that it was a very easy item for these students (p = .98). Item #25 was a very difficult item (p = .25) for USVI examinees and this accounts for the low level of the observed discrimination index (d = .17). Summary of Student Skills Number, notation, and operations. St. Thomas/St. John school district examinees showed particular skill in their ability to identify the sign associated with an operation. St. Croix district examinees showed adequate achievement in this skill. St. Thomas/St. John examinees also showed strong skill in indicating the position of a block of.dots relative to other blocks. St. Croix district examinees showed weak achievement in this skill. St. Thomas/St. John school district examinees showed weak skills in the following: a) indentifying a set having a given number of objects. b) selecting a set whose number lies within a given interval. 35 BEST COPY AVAILABLE 32 St. Croix district examinees displayed adequate achievement in these skills. Examinees from both school districts showed weak skills in the following: a) identifying a group in which half of the abidct are marked b) indicating the additive identity in set theory USVI examinees from both the St. Thomas/St. John and the St. Croix school districts showed adequate skills on the following: a) choosing the numeral representing a set of objects b) choosing the number name representing a set of objects c) choosing the missing numeral in a series d) choosing a numeral representing the greatest relative value e) choosing a set matching a given set in number f) matching the number of a set with its representation as the sum of its subsets g) identifies the digit having a specified place value h) chooses the set which is equivalent to a given set i) identifies a numeral represented as tens and ones Geometry, measurement, and logical thinking. USVI examinees from both the St. Thomas/St. John and the St. Croix school districts demonstrated adequate skills in: 36 BEST COPY AVAILABLE V*. 33 a) identifying the object furthest from a specified point b) distinguishing between simple geometric figures c) determining the field having the greatest area d) identifying the figure having a given geometric relationship e) solving a problem in elementary logic involving transitivity f) determining which object has the greatest area unused g) identifying a c'-,ck showing a specified time h) ordering containers by volume They showed weak skills in the following: a) indicating the 3-dimensional block figure having the greatest number of blocks b) identifying the longest of four paths c) solving problems in elementary logic involving spatial relationships d) finding the total value of 3 coins e) identifying appropriate units of measure f) identifying a group of objects according to two properties Mathematics Computation Item Analysis Item #1 and #2 discriminated poorly for examinees in the St. Thomas/St. John school district (d = .08 and .18, respectively). These were easy items for these students 3 7 BEST COPY AVAILABLE 34 (p = .90 for both) and this would account for these low discrimination indices. Item #S discriminated poorly among St. Croix district examinees (d = .00). This was a very easy item for these students (p = .92), accounting for the low observed-discri- mination. Item #8 discriminated poorly for the whole USVI sample (d = .02). This was a moderately difficult item (p = .40) where examinees were asked to determine the number of pieces of candy in five boxes each containing two boxes of candy. The alternatives were 1) S, 2) 2, 3) 3, 4) 10. Examinees in the lower 27% group were almost just as likely to score correctly on this item as those in the upper 27%. As a whole, there was no significant difference between the pro- portion scoring correctly the local sample and the proportion scoring correctly in the standardization group. No patterns of incorrect responses was evident. Item #13 discriminated poorly for the whole USVI sample (d = .11). The fact that this was a very difficult item for these examinees (p = .28) would account for this. Item #14 showed poor discrimination for St. Croix district examinees (d = .04). This was due to the fact that these students found the item extremely difficult (p = .11). Item #10 was a very difficult one for St. Thomas/ St. John examinees (p = .18) and because of this it discri- minated poorly among these examinees (d = .09). 38 BEST COPY AVAILABLE 35 Item #16 and #20 were very easy items for St. Thomas/ St. John examinees (p = .96 and .94) and this accounts for the low levels of discrimination which weic observed (d = .08 and .18). Item #15 discriminated poorly among USVI examinees from both districts (d = .09). This was most likely due to the fact that this was an extremely easy item for this sample of students (p = .95). Items #20, #22 and #31 were each very easy for St. Thomas/St. John school district examinees (p = .94, .92, and .92, respectively), resulting in unacceptably low discri- mination indices (d = .18 and .15). Summary of Student Skills Addition and subtraction facts. St. Thomas/St. John school district examinees demonstrated particularly strong skills in subtracting number facts horizontally. St. Croix district examinees demonstrated adequate achievement in this skill. Examinees from both school districts showed adequate skills in adding two number facts horizontally and in adding a column of three 1-digit numbers. Mathematical sentences. St. Thomas/St. John examinees showed strong skill in finding the subtrahend in a subtrac- tion problem which is presented horizontally. St. Croix school district examinees demonstrated adequate achievement in this skill. 39 BEST COPY AVAILABLE 36 USVI examinees from both districts showed adequate skills in: a) finding the audend in an addition problem presented horizontally b) finding the minuend in a subtraction problem pre- sented horizOntally Verbal problems. Examinees from both the St. Thomas/ St. John and the St. Croix school districts showed adequate skills in their abilities to solve verbal problems by: a) adding the cost of two small purchases b) adding to find a total c) subtracting to find how many are left d) adding to find how many there are all together e) adding the cost of three small purchases They showed weak skills in solving verbal problems by: a) subtracting to find how many more are needed b) multiplying (or adding) to find a total when an equal number is given in each group c) subtracting to make a comparison d) using an elementary rate formula St. Thomas/St. John school district examinees showed weak skills in solving verbal problems by: a) finding the minuend in a subtraction problem b) using simple division St. Croix school district examinees demonstrated adequate achievement in these skills. 40 BEST COPY AVAILABLE 37 Listening Comprehension Item Analysis Item 01 was a very easy one for St. Thomas/St. John school district examinees (p - .95) and this should account for the low level of discrimination which was observed (d = .08). Item #2 discrimiaated poorly for St. Thomas/ -.. John school district examinees (d = .15). In th-_s mof, -ately easy (p = .82) item students had been read a passage by the teacher which related an outing by three children in which they parti- cipat'd in a number of activities. One of these was picnic- ing in the shade of a tree. They were then presented with pictures of a pond, a tree, and a rock and asked what the chldren in the story had their picnic by. It was noted that the proportion of examinees scoring correct in the upper 27% of the group was the same as that of the Atire sample of St. Thomas/St. Croix examinees (.82). It appears that students in the upper group who scored incorrectly very often chose the pond as the correct answer. The item seemed clear and concise and the passage was also quite clear and simple, yet higher achievement examinees seemed to be attrac- ted to the pond distractor. The reason -fir this is not apparent from the avaiidble information. Item #12 did not seem to discriminate at all for St. Croix school district examinees (d = 0). The probabi- lity of an examinee in the upper 2.g of the group scoring correctly on this item (.40) was the same as that c_ a 41 BEST COPY AVAILABLE. 38 student in the lower 27% and slightly lower than the propor- tion correct for the entire St. Croix sample (p =.42). Most examinees scoring incorrectly in this item chose the picture of a boy over the correct reponse of the picture of a girl although the concept in the read passage is clearly a feminine one with feminine adjectives and pronouns used, thoughout. Item #7 discriminated poorly for St. Thomas/St. John school district examinees (d = .08). This was an easy item for these students (p = .85) and this would account for the low level of discrimination that was observed. Items #12 and #14 did not discriminate well for St. Thomas/St. John school district examinees. Both items referred to the same passage of material and had the same choices. These were pictures of 1) an artists' paint brush, 2) a tray, and 3) a pair of scissors Approximately the same proportion of examinees scored correctly in the upper and low 27% of students in both items, although, the latter item seemed somewhat easier for these examinees (p = .41 for item #13 and p = .49 for item #14) There appeared to )e no visible pattern of incorrect responses on either item. Roth items require examinees to determine conceptual mean ing from the passage and relate this to knowledge they already possess and it maybe that these examinees do not possess the requisite knowledge due to geographical or cultural reasons. 42 BEST COPY AVAILABLE 39 Summary of Student Skills Explicit detail In this set of items examinees were required to identify specific details which were present-d in nearly the same language as in the passage which was read to them or recognize that a specific detail was not mentioned; identify a sequence of events; or follow specific directions. USVI examinees from both districts showed weak skills in understanding explicit detail. The mean propor- tion answering correctly in this group of items was .74 for the St. Thomas/St. John sample and .69 for the St. Croix saAple, compared to .82 for the standardization group. St. Thomas/St. John examinees scored significantly lower on POI of the items in this grouping while St. Croix district .examinees scored significantly lower than the standardiza- tion group on all items. Implicit meaning This set of items required examinees to identify what was heard when it was Lot presented expli- citly, thus requiring some degree of interpretation. USVI examinees from both the St. Thomas/St. John and the St. Croix school districts demonstrated weak skills in understanding implicit meaning. The mean proportions of examinees cor- rectly answering these item was .53 for tne St. Thomas/ St. John sample and .r,4 for the St. Croix sample, compared with .62 in the standardization group. Conceptual meaniag and classification In this group of items, examinees were required to identify and classify 43 BEST COPY AVAILABLE 40 concepts by relating what was heard to known concepts, or identify emotional attitudes or character traits or discern the meaning of words from content; or employ a conception of language to identify the doer or receiver of an action; or use concepts of comparative terms. St. Thomas/St. _John examinees display adequate skills in this area (mean p = .66 compared to mean p = .68 in the standardization group) while St. Croix examinees demonstrated weak skills (mean p = .52) scoring significantly below :he standardization sample in 80% of the items in this grouping. Inference and logical analysis In this set items, examinees were required to make inference and draw conclu- sions from what was heard; cr: to analyze problem situations and passages and make logical deducations. St. Thomas/ St. John examinees showed adequate achievement in these skills (mean p = .68 compared with a mean p = .72 for the standardization ?roup), scoring significantly below the standardization group on only 18% of the items. St. Croix school district examinees showed weak skills on this group- ing of items (mean p = ..8) and scored below the standard- ization group on about 92% of the items in this item grouping. 4 4 BEST COPY AVAILABLE 41 Grade 4 - Primary III Level The Primary III Level of the Stanford Achievement Test consists of eight tests with the number of items listed below: 1. Vocabulary - 45 items 2 Reading Comprehension 70 items 3. Word Study Skills 55 items 4. Mathematics Concepts 32 items S. Mathematics Computation 36 items 6. Mathematics Appications 28 items 7. Spelling - 47 items 8. Language 55 items The descriptive statistics for these test scores, determined previously (see Bliss, !982a), are presented below. Test Mean Standard Raw Score Deviation KR-20 ., tandard Reliability Error Total USVI Sample Vocabulary 23.6 7.2 .83 2.96 Reading Comprehension 42.3 11.5 .91 3.46 Word Study Skills 29.8 10.1 .90 3.16 Mathematics Concepts 15.5 5.3 .77 2.55 Mathematics Computa- tion 20.4 6.1 .83 2.52 Mathematics Applica- tions 13.8 5.S .86 2.16 Spelling 30.9 10.2 .93 2.69 Language 28.9 8.8 .86 3.28 45 BEST COPY AVAILABLE 42 St. Thomas /St. John Sample Vocabulary 23.4 5.9 .75 2.95 Reading Comprehension 42.2 11.3 .91 3.38 Word Study Skills 30.8 9.2 .88 3.17 Mathematics Concepts 15.0 4.4 .66 2.55 Mathematics Computa- tions 19.6 5.1 .76 2.55 Mathematics Applica- tions 13.8 5.5 .84 2.21 Spelling 30.4 9.2 .91 2.76 Language 28.1 8.1 .84 3.26 St. Croix Sample Vocabulary 2.40 8.4 .88 2.91 Reading Comprehension 42.3 11.8 .92 3.35 Word S Skills 28.7 10.8 .92 3.06 Mathematics Concepts 16.0 6.2 .84 2.47 Mathematics Computa- tions 21.4 7.0 .87 2.47 Mathematics Applica- tions 13.8 6.0 .87 2.18 Spelling 31.5 11.2 .94 -2.74 Language 29.8 9.4 .88 3.25 Following are the item analyses and summaries of student achievement on the objectives evaluated by each of the tests: Vocabulary Item Analysis All items showed discriminated satisfactorily except the following: 4 6 BEST COPY AVAILABLE 43 Item #1 was a very easy item for St. Thomas/St. John school district students (p = .91) and this accounts for the low level of discrimination which was observed. Item #21 This item asked examinees to choose the term which finished the statement, "If an object is shut up on all sides it is ." The alternatives provided were 1) released, 2) empty, 3) detached, 4) enclosed. Alternative #3 was scored as correct. Most examinees scoring incorrect- ly on this moderately difficult item (p = .62 chose "detached" as their response. Students in the upper 27% were almost as likely to chose this response as were those in the lower 27% of total scores on the vocabulary test. There may be a fine distinction in the mind of fourth graders between something being "enclosed" and being "detached" from the outside world and many of the examinees may not have been able to make this distinction. This explanation is strengthened by the fact that almost all of the upper 27% of students scor- ing incorrectly on this item chose alternative #3 ("detached") . Item 133 was a very difficult item for St. Thomas/ St. John school district examinees (p = .14) and this would account for the low level of discrimination which was ob- served. Item #45 was a very difficult item for USVI students (p = .23) and this would account frr the low levels of discrimination which were observed for examinees in both school districts. 4 7 BEST COPY AVAILABLE 1 I 1 44 Summary of Student Skills The sample of USVI students tended to be generally weak in the area of standard English vocabulary recognition. These weakness was seen across all areas of the best which included vocabulary found in a) general reading and literature b) nonfiction and reference wards c) mathematics and science d) social science On 31 of the 45 items on the test (69%) USVI examinees had a significantly lower percentage of examinees scoring cor- rectly than the standardization group with only one item where they scored correctly a significantly greater propor- tion Of the time than the standardization group. Aside from the issue of statistical significance, the proportions correct were very low for USVI samples on most items on the test when compared with those of the standardization group. Reading Comprehension Item Analysis The phe-,-- -.Jr' of large numbers of students having trouble in completing the reading comprehension tests in secondary oracles (Bliss, 1982b) and in the sixth grade sample was not as obvious in the fourth grade sample. In fact the proportioL of omits for item #70 (the last item in the test) was only 23% for the entire USVI sample compared 48 BEST COPY AVAILABLE 45 to 71% on the last item (#71) of the test given to the s:xth grade sample. In addition, the percentage of omits for fourth graders did rot appear to rise dramatically until item #53 where it was only 8%. Previously, the rise in omits was attributed to some phenomenon attached to the way USVI students respond to items which require them to read passages of text and respond to questions about them. This idea is further by observing the patterns of omits for fourth graders. Unlike the reading comprehension tests of more advanced levels which are composed entirely of reading passages fol- lowed by questions about the passages, Primary Level II con- sists of stories where questions are placed after almost every sentence. For instance: Mary ran across the street holding a bottle of pop. It slipped from her hand and 1 1) bent 2) ran 3) broke 4) cried A car drove over it and cut its 2 1) window 2) roof 3) door 4) tire This pattern continues through item #35 after which the familiar type of passage and questions form begins. The firft two passages are short (6 and 8 lines, respectively) and this is followed by a longer passage (21 lines). The number of omits begins to increase after this longer pas- sage and its questions (item #53). 49 BEST COPY AVAILABLE 46 All items showed satisfactory discrimination indices except the following: Item #1 and #6 were very easy items for IJSVT students (P = .95 for both items) and this would account for the low levels of Discrimination observed. Item #8 was a very easy item for St. Croix district examinees (p = .91) which would account for the low discri- mination index seen on this item. Item #9 was a very easy item for Se. Croix students (p = .93) and this accounts for the low discrimination levels observed. Item #35, #59, and #70 were very difficult items for USVI students (p = .14, .11, and .17, respectively) accounting for the low levels of discrimination observed for these items. Summary of Student Skills The mean difficulty index for items testing global or general meaning of iaterial read was .67 for the USVI sample compared to .73 for the standardization group. The local sample of examinees scored significantly lower than the standardization group on only two of the six items measuring this skill. USVI students appear to have adequate skills in determining global or general meaning from readings. The St. Thomas/St. John and the St. Croix sample of students showed mean difficulty indices of .50 and .53, respectively, compared to p = .59 for the standardization 50 BEST COPY AVAILABLE 47 group on items testing the ability to identify explicit meaning from readings. Local examinees appear to have adequate skills in this area. In the area of determining implicit meaning from readings USVI examinees showed a mean difficulty index of .72 in both districts compared to a mean p = .74 for the standardization group. It appears that the local sample of students dis- plays adequate skills in this area of reading comprehension. In determining meaning by content, St. Thomas/St. John school district examinees had a mean difficulty index of .67 and St. Croix district examinees scored a mean difficulty index of .66 compared to a mean of .73 for the standardiza- tion group. USVI students seem to perform adequately in this area. In the area of determining inferential meaning from reading both the St. Thomas/St. John and the St. Croix sample of students had mean difficulty indices of .48 compared to p = .60 for the standardization group. On 10 of the 13 items in this grouping, local examinees scored significantly lower than the standardization group. It appears the USVI students are weak in their ability to determine inferential meaning from readings. Word Study Skills Item Analysis All items showed satisfactory discrimination indices except for the following: 51 BEST COPY AVAILABLE 48 Item #10 shows d = .00 for both local samples. In this item the examinees are supposed to recognize the vowel sound /0/ in "awkward" when the word is pronounced by the teacher and choose from among the four words, 1) after, 2) always, 3) other, 4) again written in their text books to find a word that "has the same beginning sound." Local students almost consistently chose alternatives 3 and 4 ("other" and "again") over the correct choice "always." It is possible that -,:re are regional differences in pronounciation at work here. The word "always" is pronounced with a primary /0/ sound in the northeastern U.S., but is sounded like /ah/ in other parts of the U.S. and among U.S. Virgin Islands speakers of English. The pronounciation "ahl' ways", being the common pronounciation, it was very likely the one used by students who chose other alternatives. Since there is very likely no correlation between where the persons who students imitated while learning to speak (most likely their parents) and their word study skills, in general, the lack of discriminating power of this item is no mystery. Further, the value of this item in measuring the skill it purports to measure is dubious for the local population of Vat,' students. Item #27 was a very easy item for USVI examinees (p = .94) and this accounts for the low level of discrimina- ,- .,,.,, tion observed. 52 JEST COPY AVAILABLE 49 Summary of Student Skills Consonant sounds These items examine the ability of students to recognize sounds of consonant letter within words. USVI examinees showed adequate skills in discr'imir tang among graphemes for: a) the initial consonant sound /b/ b) the final consonant sound /n/ c) the final consonant sound /f/ They showed weak skills in discriminating among graphemes for: a) the initial consonant sou,d /s/ b) the initial consonant sound /f/ c) the initial consonant sound /j/ d) the final consonant sound /k/ e) the initial consonant sound /y/ Consonant digraphs and consonant clusters USVI examinees showed weak skills in discriminating among graphemes for the consonant digraph /th/ in the initial position. St. Croix school district examinees weak ability in discriminating between graphemes for: a) the cluster of initial s + a consonant (sp) b) the digraph /ch/ in the initial position c) the digraph /sh/ in the final position d) the digraph /sh/ in the initial position St. Thomas/St. John school district examinees displayed adequate skills in these areas. 53 BEST COPY AVAILABLE SO USVI examinees from both school districts showed adequate skills in discriminating between graphemes for: a) the cluster of an initial consonant + r (gr) b) the cluster of final sk c) the cluster of a final consonant + d (rd)- d) the digraph /ng/ in the final position Short vowel sounds USVI examinees showed weak skills in discriminating among graphemes for: a) the short vowel sound /i/ b) the short vowel sound /o/ St. Croix school district examinees showed weak skills in discriminating among graphemes for: a) the short vowel sound /u/ b) the short vowel sound /0o/ St. Thomas/St. John examinees displayed adequate skills in this area. USVI examinees from both school districts displayed adequate skills in discriminating among graphemes for; a) the short vowel sound /e/ b) the short vowel sound /a/ Long vowel sounds St. Croix school district examinees showed weak skills in discriminating among graphemes for: a) the long vowel sound /e/ b) the long vooel sound /a/ St. Thomas/St. John examinees showed adequate skills in these areas, 54 BEST COPY AVAILABLE 51 USVI examinees from botu cistricts displayed adequate skills in discriminating among graphemes for: a) the long vowel sound /1/ b) the long vowel sound /oo/ c) the long vowel sound /0/ d) the long vowel sound /yoo/ Variant vowel sounds USVI examinees from both school districts showed weak skills in discriminating among graphe- mes for: a) the diphthong /o / b) the schwa sound c) a vowel sound before r These examinees showed adequate skills in discriminating among graphemes for: a) the vowel sound /o/ b) the diphtong /oi/ Affixes - USVI examinees from both school districts show(A strong ability to discriminate among graphemes for the suffix "-tion." St. Croix examinees showed strong skill 4..ri discriminating among graphemes for the prefix "-en." St. Thomas/St. John district students showed adequate achievement in this skill. Examinees from bath school districts showed adequate skills in discriminating among graphemes for: a) the prefix "-re -," b) the suffix "-ance." 55 BEST COPY AVAILABLE 52 Mathematics Concepts Item Analysis All items seemed to discriminate satisfactorily except item #22 for the St. Croix school district sample of examinees. This was a very difficult item --- these students (p = .16) and there was no pattern of incorrect responses that could be noted. Summary of Student Skills Number This item grouping refers to concepts which deal with the relationships between different types of numbers (e.g. primes, odds vs evens, fractions vs wholes). USVI examinees from both school districts showed weak skills in theil ability to choose a number belonging in a given number series. St. Thomas/St. John district examinees showed weak skills in the following areas: a) representing a whole number in terms of a specified fraction b) determining the point at which a given number is located on the number line c) adding in a finite system of clock module St. Croix district examinees demonstrated adequate skill in these areas. USV1 students from both school districts displayed adequate skills in the following areas: 56 BEST COPY AVAILABLE 53 a) identifying a number at a particular point on a number line b) identify a sec consisting only of even numbers c) indicating the position of a number relative to three given numbers Notation refers to the technique by which numbers are expressed in written form. Some forms include expanded nota- tion, differing base values, decimals, and fractions. USVI examinees from both school districts showed weak skills in: a) identifying the remainder of a subtraction problem when the minuend is represented by a place-value device b) choosing the expanded notation representing an addition example St. Thomas/St. John school district examinees displayed weak ability in determining the difference between two num- bers represented on a plaL_ value device. St. Croix district examinees showed adequate achievement in this area. St. Croix school district examinees showed weak ability in the following skills: a) translating the word form of a given number into Arabic numerals b) identifying the expanded notation representing a a given numeral St. Thomas/St.John examinees displayed adequate skills in these areas. 57 BEST COPY AVAILABLE 54 USVI examinees from both school districts displayed adequate ability in the following skills: a) selecting the numeral in which the digit in the thousands place has the greatest value b) trans: ting a given Arabic numeral into word form c) recognizing the form of an expression not repre- senting a given number d) choosing the Roman numeral equivalent to a given Arabic numeral e) marking the hundreds place digit in a given numeral Operations - refers to the manipulation of numbers based on the use of a set of previously defined functions (e.g. addition, subtraction, multiplication, and division). St. Croix district examillees showed strong skills in: a) identifying the sign associated with an operation b) choosing a number sentence describing a given operation They displayed weak skills in: a' identifying zero as the additive identity b) indicating that the inverse of a given number sentence is true if the sentence is true St. Thomas/St. John school district examinees displayed adequate skills in all these areas. St. Thomas/St. John examinees showed weak skill in identifying the missing number in a number sentence 58 BEST COPY AVAILABLE t. 55 illustrating the commutative property of addition. St. Croix school district examinees performed adquately on this skill. USVI student from both school districts displayed adequate skills in: a) identifying the missing terms in a numbef sentence illustrating the associative property of addition b) identifying the number of sets of a number which are equal to a set of another number c) interpreting a number of sentence in terms of a second sentence whose operation is the inverse of the first Geometry and measurement refers to the concepts which involve shapes, measurement, and logic. USVI examinees in both school districts showed weak skill in,marking,a:row where measures are arranged in order from small to large. St. Thomas/St. John school district examinees displayed weak skill in estimating the length of a line segment while St. Croix examinees showed adequate achievement in this skill. Examinees from both school districts showed adequate skills in: a) identifying the area of the interior region of a figure b) indicating size relationships between two similar geometric figures c) indicating a point within the interior of two over- lapping geometric figures. 59 BEST COPY AVAILABLE 56 Mathematics Ccr -utation Item Analysis Item #2 was very easy (p = .89) for St. Thomas/St. John district examinees and this should account for the low level of discrimination observed (d = .06). Item #25 was a very easy item (p = ,92) for USVI students from both districts, thus accounting for the observed low discrimination index (d = .10). Item #34 was a moderately difficult item for St. Thomas/ St. John district examinees (p = .21). It required students t. divide 129 by 3 with the problem presented in standard form. No pattern of incorrect responses could be found and the item seemed clear and unambiguous. The low discrimi- nation index (d =.12) could be accounted for by the diffi- culty of the item or the fact that, being the only item on the test requiring long division, the intercorrelation between it and other items was particularly low for this sample of students. Summary of Student Skills Knowledge of primary facts USVI examinees from both school districts displaye-1 adequate skills in all areas of knowledge of primary facts testes' by this examination. These skills were: a)/ b ) adding primary facts to obtain sums of twelve or less adding primary facts to obtain sums greater than twelve 6o BEST COPY AVAILABLE 57 c) subtracting primary facts with a minuend less than twenty d) multiplying primary facts with a product of 25 or less e) multiplying primary facts wIth a product greater than 25 f) dividing primary facts with a dividend of 25 or less g) dividing primary facts with a dividend greater than 25 Addition and substraction algorithms This set of skills involves the use of a series of prelearned steps for finding sums and remainders. St. Croix school district showed particularly strong skills in: a) subtracting 2-digit subtrahends from 3-digit minuends, renaming tens b) subtracting 3-digit subrahends from 3-digit minuends, renaming hundreds St. Thomas/St. John district examinees showed adequate skills in these areas. USV1 examinees from both school districts showed ade- quate skills in: a) adding a 3-digit numeral and a 2-digit numeral without renaming b) adding two 3-digit numerals, renaming ones c) subtracting a 2-digit subtrahend from a 3-digit minuend without renaming 61 BEST COPY AVAILABLE 58 d) adding two 3-digit addends, renaming ones and tens and obtaining a 4-digit sum. Multiplication and division algorithms This set of skilis involves the use of a series of prelearned steps for finding products and quotients. St. Croix school district examinees displayed particu- larly strong skills in multiplying 2-digit by a one-digit multiplier with and without renaming ones. St. Thomas/ St. John district showed adequate ability in these skills. Examinees from both school districts showed adequate skills in: a) finding the devisor in the horizontal algorthm b) dividing a 3-digit numeral by a 1-digit divisor without renaming, and obtaining a 2-digit quotient. Mathematics Applications Item Analysis All items showed acceptable discrimination indices except the following: Item #1 was a very easy item for USVI examinees in both school districts (p = .90) and this would appear to account for the poor discrimination which was observed. Item #15 was a very difficult item for USVI examinees ( = .11). It required Students to find the number of yards in six feet. The question seemeu clear and unambiguous and there was no discernable pattern of incorrect responses. 62 BEST COPY AVAILABLE 59 Summary of Student Skills Solution of a one-step problem These items required students to find the solutions to word problems that could be solved using a single mathematical operation. USVI examinees from both school districts showed weak ability in solving word problems requiring subtracting to compare two quantities. St. Croix district examinees showed weakness in their ability to solve word problems requiring subtraction to find a remainder while St. Thomas/St. John examinees showed adequate achievement in this skill. USVI examinees from both school districts performed adequately in solving word problems which required: a) subtracting to find "how many are left" b) adding to find a total Analysis and development of a solution design - These items required students to carry out the following series of steps: 1) determine the operations needed to solve a word problem, 2) order the operations, and 3) carry out the operations in the appropriate manner in order to arrive at a correct solution. St. Croix school district examinees displayed adequate skills in solving word problems by: a) using knowledge of 1/2 to determine specific rate determining what additional information is needed to solve a given problem 63 BEST COPY AVAILABLE 60 They showed weak skills in solving word problems by: a) adding and subtracting in a two-step sequence b) determining specific rate c) multiplying and subtracting in a two-step sequence d) multiplying to determine the total number-in a group e) subtracting and adding in a two-step sequence f) separating relevant and irrelevant data St. Thomas/St. John examinees showed weak abilities in all of the problem solving situations listed above. Measurement and graphs examinees from both the St. Thomas/St. John and the St. Croix school districts displayed adequate skills in solving word problems which required them to: a) read a tally sheet to quantify a category b) read a tally sheet to determine the category of greatest number c) read a tally sheet to find a total d) find the total cost of three items in a price display They showed weak skills in solving word problems which required them to: a) convert standard units of linear measure b) convert standard units of linear measure and:-ubtract c) compare two items in a price display d) add and subtract in a two-step sequence using a price display BEST COPY AVAILABLE 61 Spelling Item Analysis All items on the spelling test functioned appropriately for examinees in both school districts. Summary of Student Skills Phonics St. Thomas/St. John school district examinees showed strong skills in identifying misspelled words where /ch/ is spelled cs instead of ct. St. Croix district examinees displayed adequate skill on this objective. USVI examinees from both districts showed adequate levels of achievement on all other phonics spelling skills tested. These constituted the ability to identify misspelled words where: a) the final e of a vowel-consonant-e pattern is omitted b) /u/ is spelled o instead of u c) the vowel from a vowel digraph before r is ommited d) /e/ is spelled e instead of ea e) a consonant letter is unnecessarily doubled f) /k/ is spelled ck instead of k g) /ch/ is spelled sh h) /k/ is spelled le- instead of ck i) one letter of a doubled consonant is omitted j) /f/ is spelled v instead of f k) initial wh is spelled w 1) /s/ is spelled se instead of ce 65 BEST COPY AVAILABLE 62 m) /o/ is spelled ow instead of 0 n) /el/ is spelled el instead of le o) vowel-consonant-e pattern of-ite is substituted for-eight p) /e/ is spelled ee instead of ea q) /o/ is spelled oa instead of o r) schwa syllable is omitted s) final silent e is omitted t) /el/ is spelled le instead of el u) final e is added to a word ending in a short vowel plus consonant v) loo/ is spelled on instead of oo w) final /e/ is spelled y instead of ey x) /e/ is spelled a instead of e Word building USVI examinees in both the St. Thomas/ St. John and the St. Croix school districts displayed ade- quate skills in all word building spelling skills. These constituted the ability to identify misspelled words in which: a) /t/ is spelled -t instead of -ed in forming past tense b) the final e of a vowel-consonant -e patterns is omitted in forming a compound c) the final e is omitted when adding -ly d) the final consonant letter of a cluster is doubled when adding -ed 66 BEST COPY AVAILABLE 63 e) the final e of a vowel-consonant -e pattern is retained when adding -ly f) a compound is formed without changing all to al g) the final consonant letter is doubled when adding -er h) the final consonant letter is doubled when adding -ing i) the final consonant letter is doubled when adding -ed j) the final e is retained when addi-g.-ing k) the suffix -16 is misspelled 1) the plural is formed without changing the final y to i m) -s instead of -es is used to formed the plural of a word ending in sh n) the suffix -ful is misspelled o) the final -es instead of -s is used to spell the tense p) -es instead of -s is used to form the plural Language Item Analysis All items on the .anguage test showed acceptable discri- mination indices except the follwoing: Item #9 was a very easy item for St. Thomas/St. John examinees (p = .91) and this would account for the low level of discrimination observed (d = .14). 67 BEST COPY AVAILABLE 64 Item #13 was an easy item fo- USVI examinees in both school districts (p = .82). The item required students to choose the correct form of a formal name from among the alernatives, 1) Mrs. finn, 2) Mrs. Finn, 3) mrs. finn, and 4) Mrs Finn. The vast majority of examinees who scored incorrectly chose alternative #4, obviously missing the fact that the period ii the honorific was missing. This was true among the upper and lower 270 of the examinees and it is likely that the abi.k.ity to detect t'lis difference does not correlate highly with the othe skills tested on the eAamina- tion. The fact that other items testing use of capitaliza- tion convention, but do not use an abbreviated honorific show good discrimination supports the idea that this item was not acutally discriminating between students who did and did not have the concept of capitalization in proper names as purported by the test publishers. Item #16 was a difficult item (p = .25) for USVI students. It required them to finish the sentence, I asked, "Why can't we have , with the alternatives being, 1) school?", 3) school?, and 4) school." Most examinees chose alternatives #3 and #4 and this was equally true in the upper and lower 27% groups. The item seems clear and unambiguous. Most examinees were aware that the sentence was either a question cr a quotation, but few realized that it was both. Item #33 was a difficult item for USVI students (p = .22). It actually discriminated negatively for St. Croix school district students (d = -37). Examinees when asked Lo choose sS BEST COPY AVAILABLE 65 an appropriate completion tc, the sentence, "I was glat that Mrs. Finn wrong." The alternatives were, 1) couldn't find nothing, 2) didn't find nothing, 3) couldn't find anything, and 4) couldn't never find anything. Analyis of item responses showed that students in the upper 27% of total language scores tended to choose alternative #1. These use of the double negative may be a dialectical artifact since the phenomenon did not show itself on any of the other items which tested the ability avoid double negatives and examines scored well on these other items (p . .73 and p = .58). Summary of Student Skills Capitalization conventions - These are skills which involve examinees' ability to recognize appropriate rses of capitalization of words. St. Croix school district examinees showed particularly strong skill in tneir ability to identify the correct capi- talizatian convention for holidays. St. Thomas/St. John examinees displayed adequate skill ir, this area. USVI examinees from both school districts showed weak skill in identifying the capitali.ation convention for common nouns. St. Thomas/St. John school district examinees showed weak skills it identifying the correct capitalization con- ventions for: a) cities and states b) the closing of a letter 6i BEST COPY AVAILABLE 6f St. Croix district students displayed adequate skills in this area. USVI examinees from both school districts showed adequate skills in identifying the correct capitalization conventions for a) street addresses b) the saltation of a letter c) a proper name d) the beginning of a sentence e) the days of the week f) titles g) the name of a school Punctuation conventions These skills involve the ability to recognize the appropriate use of punctuation in a given sentence. St. Thomas/St. John school district examinees showed weak skills in identifying the correct punctuation convention of using: a) a comma between city and state b) a comma after the closing of a letter St. Croix district examinees displayed adequate skills in this area. USVI examinees in both school districts showed adequate skills in identifying correct punc*,Iation conventions in using a) no r-nctuLtion after the street address in the heading of a letter 70 BEST COPY AVAILABLE 67 b) a comma between the day of the month and year c) a comma after the salutation in a friendly letter d) a period to end a declarative sentence e) a question mark to end an interrogative sent nce f) a period after an abbreviation or title g) a question mark to end an interrogative sentence enclosed in quotation marks h) commas to separate items in a series i) a comma after a name used in direct address j) an apostrophe in a contraction k) a comma after an introductory "yes" Usage conventions These refer to skills involving the ah'lity to choose appropriate forms of words depending on the content of the sentence in which they are used. USVI examinees from both the St. Thomas/St. John and the St. Croix districts showea adequate skills in the following usage situations: a) indefinite article b) where there is a pronoun form in compound subject and identifying the word order for a proper noun and a pronoun c) using the past tense of the word "to know" d) using a negative form to avoid a double negative e) using the past tense of the verb "to bring" f) using the past tense of the verb "to tell" They showed weak skills in the following usage areas: a) using the present tense of the verb "to hurt" 71 BEST' cou AVAII AM r 68 b) using the past-perfect tense of the verb "to come" c) using the past tense of the verb "to teach" d) using the past tense of the verb "to give" e) using the past tense of the verb "to run" f) using a pronoun form in a compound subject and a form of the verb "to be" g) using the present tense of the verb "to ask" h) using a negarive form of the verb "to be" i) using the present perfect tense of the verb "to grow" j) using the present tense of the 7erb "to become" Language sensitivity : word sensitivity These skills relate to the ability of examinees to perceive appropriate usage of words in a particular content. USVI examinees from both school districts displayed adequate skills in their ability to complete a sentence using" a) gerunds b) a form of the word "to float" d) a verb e) a pronoun to agree with an antecedent They showed weak skills in their abilities to complete a sentence using: a) an antecedent for a given pronoun b) a verb form St. Thomas/St. John school district examinees showed weak abilities in completing sentences using: 72 BEST COPY AVAILABLE 69 a) a conjunction b) an adverb c) a verb and verb form St. Croix district examinees showed adequate skills in these areas. Language sensitivity : sentence sensitit t) St. Thomas/ St. John examinees displayed weak skills in choosing phrases which complete sentence fragments and recognizing complete sentences. St. Croix district examinees showed adequate achievement in both of the skills. 73 BEST COPY AVAILABLE 0 Grade 6 Intermediate II Level The Intermediate II Level of the Stanford Achievement Test consists of eight tests with the number of items listed below: 1. Vocabulary 45 items 2. Reading Comprehension 71 items 3. Word Study Skills 50 items 4. Mathematics Concepts 35 items 5. Mathematics Computation 45 items 6. Mat::ematics Applications 40 items 7. Spelling 60 items 8. Language 80 items The means, standard deviations, and reliability esti- mates for these tests, determined previously (see Bliss, 1982a), are presented below. Test Mean Standard Raw Score Deviation KR-20 Standard ReliAbjelity Error Total USVI Sample Vocabulary 21.6 7.8 .85 3.05 Reading Comprehension 32.5 12.4 .93 3.29 Word Study Skills 28.6 11.1 .93 2.93 Mathematics Concepts 18.2 5.7 .79 2.(J' Mathematics Computa- tion 25.0 7.4 .85 2.88 Mathematics Applica- tions 18.4 8.i .89 2.65 Spelling 35.2 13.6 .94 3.33 Language 37.4 13.8 .92 3.91 74 BEST COPY AVAILABLE 71 St. Thomas/St. John Sample Vocabulary 22.6 8.1 .86 3.05 Reading Comprehension 31.8 12.7 .94 3.11 Word Study Skill's 29.4 11.5 .94 2.82 Mathematics Concepts 19.3 5.5 .78- 2.59 Mathematics Computa- tion 24.8 8.0 .88 2.78 Mathematics Applica- tions 19.2 8.0 .89 2.64 Spelling 35.5 14.2 .95 3.18 Language 37.9 15.0 .93 3.98 St. Croix Sample Vocabulary 19.7 6.8 .80 3.05 Reading Comprehension 33.5 11.5 .91 3.46 Word Study Skills 27.1 10...3 .92 2.93 Mathematics Concepts 16.4 5.5 .77 2.64 Mathematics Computa- tion 25.4 6.3 .78 2.94 Mathematics Applica- tions 16.9 7.8 .89 2.60 Spelling 34.7 12.5 93 3.30 Language 37.0 11.5 .87 4.14 Following are the item analyses and summaries of student achievement on the objectives evaluated by each of the tests: Vocabury. Item Analysis All items showed acceptable discrimination indices with the exception of the following: 75 BEST COPY AVAILABLE 72 Item #37 This was a particularly difficult item for students in the USVI sample (p = .19) and this accounts for the low level of dicrimination observed on it. The item required students to recognize that a ?bolt? is a unit used to measure cloth. A large number of those answering incor- rectly chose the alternative "tile." Item #40 This was also a difficult item for the local sample of students (p = .16). It required students to indi- cate that a person who has little respect for another looks at that person with "scorn." A large number of examinees responding ii.correctly chose "a scowl," and there is some question of the item being ambiguous since a case could be made for this answer. Item #41 - This item was also extremely difficult for the USVI examinees (p = .11). It asked students to indi.cate that the perfect person for a job was "ideal." There seemed to be no recognizable patterns of incorrect responses. Item #43 - This discriminated satisfactorily among St. Thomas/St. John district examinees, but poorly among examinees in the St. Croix district. The latter group found the item more difficult than the former (p = .21 is 0 = .31) A,, and the same number of examinees 'n the upper and lower scor- ing groups obtained correct scores in the St. Croix sample. The item required examinees to know that hot metal is shaped on "an anvil." Incorrect responses were primarily "a chisel" a lathe." The item appears to be functioning properly with the low discrimination index being the result of the small number of students who knew what an anvil was. 76 BEST COPY AVAILABLE 73 Item #44 This proved to be a difficuli question for the local sample of examinees (p = .7). The item asked examinees to identify "endure" as meaning the undergoing of an experience for a long period of time. A large proportion of those answering incorrectly chose "undertake" as the cor- rect answer. This was probably the Jesuit of the use of the recognition of the prefix "under." In short, the distractor seemed to work well. Item #45 This was an extremely difficult item for USVI students (p = .08). It required students to know that people seek a shallow portion of a river in order to "ford" it. Students overwhelmingly chose "swim" and "drift" and ignored the distractor "shoot." It seems clear that Virgin Islands students, inexperienced with rivers and streams, were not aware of this term, but could associate "swim" are "drift" with water in the form that they have experienced it (i.e. ocean). The item seems to be operating as it should. Item #47 This was a difficult item for the USVI sample of examinees (p = .18). It required examinees to indicate that the term "processing" referred to an ongoing situation. There appeared to be no pattern to the incorrect responses. Item #50 This item required examinees to indicate that "secede" is a term used to indicate the leaving of a state from the union of states. It was very unfamiliar to the USVI sample of students (p = .19) and did not seem to discriminate at all among examinees (d = .07). There seemed to be no pattern of incorrect responses. It could be that 77 BEST COPY AVAILABLE 74 this term (most commonly encountered) in studies of the U.S. Civil War was familiar to examinees since this period of history is not emphasized in USVI elementary schools owing to the fact that the Danish West Indies were not affected by the war. Summary of Student Ski_ls In general, USVI students tended to be weak in the area of standard English vocabulary recognition. This weakness was especially pronounced in the areas of words encountered in the areas of social science and arts and crafts and less so in those of math and science. In addition weaknesses were apparent in the areas of vocabulary found in general litera- ture and in references. Of the fifty items on the test, a significantly smaller proportion of USVI students were able to respond correctly when compared to the standardization group. Beyong the question of statistical significance, the proportions of USVI students who successfully answered many items was substantially lower than the standardization group. The differenLez, in iteem mean difficulty values are a clear indication of this problem. Most items seem to function well with poor discrimination values of some being the result of extremely low difficulty indices. 78 BEST COPY AVAILME 75 Reading Comprehension Item Analysis Technical Report #2 indicated that students in grades 8, 10, and 12 had difficulty completing the Reading Comprehension test. This same phenomenon was observed for grade 6 examinees. able I presents the number of examinees who omitted each item on the test. It should be remembered that since number right scoring precedures were used, an omitted item is counted as an incorrect response and that this process tends to de- flate the difficulty indices and individual student scores. Because of this situation items which less than 2/3 of the examinees were able to attempt were omitted from the analysis. Technical Report #2 (Bliss, 1982b, pp. 96-97) attempts to speculate as the causes of this phenomenon. All items appeared to have satisfactory discrimination indices with the exception of item #9 which appeared to be very easy (p = .95) for the St. Croix group of examinees. St. Thomas/St. John also found ksy (p = .93 an d = .21). The low levels of discrimination was due to the simplicity of the item (which required examinees to identify meaning inferred in a pass:4e). Summary of Student Skills Restricting the analyis to items 1 to 47, the following conclusions can be drawn. The mean difficulty index for items testing global or general meaning of a portion of reading material was p = .53 79 BEST COPY AVAILABLE 76 Proportions of the Sixth Grade Sample Omitting Reading Comprehension Test Items (Percents) ITEM # USVI STT/STJ STX 1 0 o 0 2 1 1 0 3 1 1 c 4 0 o o 5 1 1 1 6 0 o 0 7 0 o 0 8 0 o 0 9 0 o 0 10 0 o o 11 0 o 0 12 0 0 0 13 0 0 0 14 0 0 0 15 0 0 0 16 0 0 0 17 2 3 0 18 0 1 0 19 1 1 0 20 1 2 0 21 1 2 0 22 1 1 0 23 4 5 1 SO BEST COPY AVAILAKE 77 Table 2 continued 24 2 3 0 25 3 4 0 26 3 4 1 27 3 3 3 28 4 7 0 29 4 6 30 4 6 0 31 5 5 0 32 8 12 1 33 8 12 1 34 9 13 1 35 11 14 4 36 11 15 4 37 15 21 4 38 17 22 9 39 18 24 6 40 22 28 11 41 21 29 8 42 26 34 11 43 23 33 13 4i 26 34 11 as 30 4C 13 46 28 37 13 G7 32 43 14 48 34 45 14 49 36 15 19 81 BEST COPY AVAILABLE . . 78 Table 2 continued 50 40 50 23 51 NI 43 54 24 52 44 54 24 53 45 55 26 54 47 57 28 55 53 65 31 56 53 66 31 57 56 68 33 58 55 69 33 59 57 70 34 60 59 72 35 61 61 77 34 62 62 76 38 63 64 77 39 64 64 7, 40 65 65 79 40 66 66 81 39 67 70 86 41 68 70 85 44 69 71 86 43 70 '7 2 87 14 71 71 87 43 8 L BEST COPY AVAILABLE 79 for each of the samples in the two school district as well as for the whole USVI sample The standardization group mean was p = .56. USVI students sampled seem to have adequate skills in tne ability to determine global meaning from written material. The mean difficulty index for items testing the ability of students to determine meaning ev'1icity written in textual material was .66 for the total USVI qmplc, .67 for the St. Thomas /c'_ John district sample and .61 for the St. Croix district sample For the continental U.S. standardization grout, the meal difficulty index was .68. USVI students appear to exhibit adequate skills in their ability to deter- mine explicitly stated meaning from text. The mean difficulty index for items testing the ability of students to determine meanings that are implicit written presentations was .65 for the entire USVI sample and for the samples from each of the individual districts. For the standardization group the mean was .65. USVI students appear to have adequate skills in the area of deteyminii; Larlicit meaning. The mean difficulty index for items testing students' abilities to determine meanings inferred in reading selections was .49 fir the entice USVI sample RI for the samples from each of the individual districtc,. For the standardization group the mean was .58. USVI students appear to Ix: somewhat weak in the area of determining inferential meaning. 83 BEST COPY AVAILABV. 80 Word Study Skills Item Analysis All items appeared to have satisfactory discrimination indite- and appear to be functioning satisfactorily. Summary of Student Skills Consonant sounds- Students in both the St. Thomas/ St. John and the St. Croix school district had difficulty recognizing that the grapheme dg as in "edge" was sounded /j/. St.Croix students seemed weak in recognizing the following consonant sound graphemes. a) g pronounced /g/ as in "guide: b) _ pronounced /k/ as in "scout" c) b 1 ounouced /b/ as in "ribbon" d) ch prounounced /ch/ as in "cheese" e) ough pronounced /f/ as in "enough" f) sh pronounced /sh/ as in "shout" St. Croix district students appear to be very weak in the area of discrimination of consonant sounds with a mean difficulty index of .35 compared with .51 for the students in the St. Thomas/St. Johm district and .50 for the standard- ization group. Short and long vowel sounds- USVI sixth glade students, in general. -ppear to have difficulty recognizing the grapnemes for the ciilowing regular vowel sounds: a) short vowel sound /e/ as in 'feather" b) short vowel sound /0/ as in "boner" 1 1 81 BEST COPY AVAILABLE 1 81 c) short vowel sound /u/ as in "must" d) short vowel sound /i/ as in "trip" St. Croix district students also have difficulty recognizing the graphemes for the following vowel sounds: a) long vowel sound /e/ as in "tea" and "only" b) long vowel sound /o/ as in "home" It appears that St. Thomas/St. John students only have difficulty identifying aphemes for the short vowel sounds whi?e being fairly proficient in recognizing those long vowel sounds. St. Croix district students seem to have difficulty with both long and short vowel sounds. Variant vowel sounds- these refer to vowel sounds that cannot be classified as "long" or "short" and include such things as diphthongs and the "schwa" sound. USVI students appear to have difficulty in recognizing graphemes f3r all of the variant vowel sounds tested. These were: a) the dipthong /ol/ b) the schwa sound c) the vowel sound /o/ as in "order" and "jaw" d) the dipht7!ong /ou/ e) the vowel sound /u/ as in "fur" 71s° particular difficulty the local students appear to have with chese variant vowel sounds is also demonstrated mean difficulty indices for these items which was .54 for the USVI examinees and .65 for the standardization example. 85 BEST COPY t ;ARABLE 82 Of particular note is the pattern of incorrect responses by examinees on items which required them to recognize graphemes for vowel sounds. More often than not, examinees scoring incorrectly on items chose distractors having the same letters in approximately the sane positions the criterion word. For example, when asked to identify the word having the same sound as the "dge" in the word "edge" from among the choices 1) league 2) jelly 3) goods 4)grades, most examinees scoring incorrectly choose 1) rather than 2). Of course, the item, and i Ins like it, may be functioning well and these may simply be cases of good distractors (after all, the items did discriminate satis- factorily). The directions are verclear and concise. However, the sample item used to illustrate the process requires students to choose the word that has the same sound as the "i" in did from among 1)bud 2)hid 3)by 4)bat, with the correct answer being 2). This example is one where the correct answer does have the same letter in the same position as the criterion and this may have been confusing to some sixth graders. These items need to be examined more closely to determine what is operating, here. The rest of the items on the Word Study Skills test required students to identity extra syllables in words presented under various language conventions. For example, 1)re 2)en 3)mem 4)ber. In this case, the answer is 2). The syllable "en" is not required to produce the word "remember." Affixes In these items the examinee is required to identify the extra syllable when part of a word stem and its affix are 86 BEST COPY AVAILABLE 83 are separated. For example. 1)el 2)er 3)trav 4)tai. The correct answer is 4) since the word is "traveler". USVI students showed strong skills in this alca with a mean prcability of answering correctly of .60 in the St. Thomas/St. John school district and .61 in the St. Croix district. The mean difficulty index for the standardization group was .54. Syllables (VC-CV)- In these items students were required to identify the extra syllable when part of a word was divided according to the Vowel/Consonant-Consonant/Vowel rule. For example: 1)ence 2)dif 3)fer 4)i. Here, the extra syllable is "i! (choice#4). USVI students showed strong abilities in this skill with mean difficulty indices over these items of .68 in the St. Thomas/St. John district and .74 in the St. Croix district. The mean difficulty index for the staldavlization sample was .59. Syllables- (VC-V or V-CV)- In these items students were required to identify the extra syllable -,.hen nart of a worel was divided according to the Vowel/Consonant-Vowel or Vowel- Consonant/Vowel rule. For example: 1)er 2)1y 3)gov 4)nor. Her,2, the extra syllable is "1)" (choice #2) , USVI examinees showed strong abilities in this skill with mean difficulty indices of .63 in the St. Thomas/St. John school district and ..65 in the St. Croix district. The mean index of difficulty for the standardization sample was .53. Accent- in these items examinees were required to identify the extra syllable when part of a word was divided according to 8 7 BEST COPY AVAILABLE 84 accent rules. USVI students showed strong abilities in this skill with mean difficulty indices of .60 in the St. Thomas/ St. John district and .62 in the St. Croix district. The mean difficulty index for the standardization was .54. Math Concepts Item Analysis All items appeared to discriminate satisfactorily except the following: Item #4 showed lowed discrimination for students in the St. Thomas/St. John district. This was a moderately difficult (p=.71) item for these students which asked them to cnoosa the number of the empty set. The distribution of re-Lpolises showed that the probability of a student in the upper 27% of total scores was virtually the same as the probability of as that of a student in the lower 27% scoring corre,:tly (.77 vs.64). There appeared to be no recogrizable pattern of incorrect scores. Item #6 showed poor discrimination in the St. Croix school district. This moderately difficult item (p=.51) required examinees to convert numbers from ,xpandedto place value notation. In this case the probability of an examinee in the lower group obtaining the correct answer was apprrximately the same as one in the upper group (.6i vs.55). Here there was a pattern in the set of incorrect responses. Examinees who answered the item incorrect?- failed t) use zero as a place holder in the hundreds place when the expanded number v.as (nx1000) + (k x 10) + (j x 1). They either .:hose a three place number in the form 86 BEST COPY AVAILABLE 85 "n k j" or placed the zero in the wrong position. It may be that the skill tested here is not intercorrelated with the other math concepts being tested to a high degree and this could account for the low level of discrimination observed. Item #23 showed poor discrimination for examinees_ in the St. Thomas/St. John district. This was a very difficult item for these examinees (p=.22) and this could account for the poor discrimination. The pattern of responses which were incorrect casts some light on what was happening here. The item consisted of a 3 x 7 rectangular grid:_ with a shaded parallelogram inside the grid . The question called for the student to choose the number of "square Ltits" which were shaded and the distractors chosen most often were those that described the number of grid squares either totally shaded or the sum of those partially or totally shaded. Students either ignored the word "unit" in the question or were un- aware of the meaning of the term "square unit." Summary of Student Skills Number- This item grouping refers to concepts which deal with the relationships between different types of numbers (e.g. primes, odds vs evens, fractions vs wholes). USVI students showed particular strength in being able to determine the fractional part of an area which is shaded. They were particularly weak however in the ability to express this shaded portion as a percent of the total area. In addition, tht local sample of students showed weak skills in the ability to subtract in a finite system using a clock 'nodule. BEST COPY AVARABLF 86 St. Croix school district examines were weak in their ability to select a number line graph of a defined set of numbers. St. Thomas/St. ..John uistr_ct examinees appeared to have rdequate skills in this area. In addition, examinees in the USVI sample aisplayed adequate skills in the following concepts: a) choosing the number belonging in a given number series. b) selecting a set consisting only of odd numbers c) selecting the number line graph of a defined set of numbers. d) identifying prime numbers. e) determining length on a number line. Notation- refers to the technique by which numbers are expressed in written form. Some forms include expanded notation, differing base values, decimals, and fractions. USVI students show weak skills in converting from exponential to standard notation. St. Thomas/St. John school district students show particularly strong abilities in trahslating numbers in word form into Arabic numbers and in rounding off whole numbers. St. Croix district examinees display great weakness in the former of these two skills. St. Croix district students also show weakness in their ability to convert from expanded to numeral form. St. Thomas/ St. John district students display adequate ability in this skill. Notation skills in which all USVI students app?ar to perform adequately are: a) identification of place value in a numeral BEST COPY AVAILABLE A.* 87 b) identification of digits in a period of a numeral c) identification of a number that is 100 greater than another number d) estimations of products to the nearest hundreth Operations- refers to the manipulation of numbers based on the use of a set of pn:viously defined functicAs (d.g. addition, subtraction, multiplication). The St. Croix sample examinees showed weakness in their ability to identify division as the inverse of multiplication while examinees in the St.Thomas/ St. John district showed adequate ability in this area. The entire USVI sample of examinees were very weak in their ability to choose an example illustrating that division is distributed over addition. The local sample displayed adequate ability in all other operations skills sampled by the test. Geometry & Measurement- refers to the concepts which involve shapes, measurement, and logic. Students in the St. Thomas/ St. John district showed strong ability in the skill of estimating the distance between two points using metric units. St. Croix district students performed adequately in this skill. The USVI sample, in general, showed weaknesses in the ability to determine the measurement of a shaded region and to identify two disjoint sets. St. Croix school district students displayed weaknesses on the following concepts: a) Identification of the missing number in and ordered pair b) choosing a figure having a right angle c) solving a logical problem using Venn diagrams. 91 BEST COPY AVAILABLE 88 St. Thomas/St. John district students displayed adequate achievement in these skills. The entire USVI sample showed adequate achievement in the following areas: a) identification of the number of the empty set b) distinguishing quadrilaterals from other geometric shapes- c) solving elementary logic problems d) .sing similar triangles e) identifying line segments f) recognizing description of line segments Mathematics Computations Item Analysis Item #16- This item showed poor discrimination with the St. Croix sample (d=.05). While students in both districts found the item to be approximately equal in difficulty (p=.32 in the St. Thomas/St. John district vs p-.34 in the St. Croix district), it was found that examinees in the upper 27 1 of the St. Croix district sample were less likely to score correctly on it than those in the St. Thomas/St. John district (41% vs 46%). In fact the upper and lower portions of the St. Croix district were almost equally divergent in their probabilities of s-xoring correctly on the item (41% vs 36%). There seems to be the same discrimination between the two upper groups as there is between the upper and lower St. Croix district groups. The item requires examinees to examine the sum of two fractions in the 1/N + 1/N+1 and to determine whether 92 BEST COPY AVAILABLE 89 this sum is greater than, equal to, or less than one. The majority of the students scoring incorrectly chose "greater than" as the correct answer in both districts. The item seems clear and unambiguous, and it is very likely that this poor discrimination is simply a chance event. Item #17- this was a very difficult item for the local sample of stuL.eIts (p=.17).- It required examinees to compare two fractions in the form n /rn and n/m-1 and to determine whether the first was less than, equal to, or greater than the second. Most students who scored incorrectly chose "less thannas the correct response. Summary of Student Skills Knowledge of primary facts and solution of simple, mathematical sentences. Students from both school districts displayed adequate skills on items testing the following skills: a) deriving two addition facts b) deriving two subtraction facts c) deriving two multiplication facts d) multiplying within parentheses, then adding e) relating two products with a common factor and an unequal second factor f) deriving division facts with and without remainders g) multiplying a three digit factor with a medial zero by a one digit factor ST. Thomas/St. John examinees showed weak performance on the following skills while St. Croix examinees displayed adequate achievement on them: 93 BEST COPY AVAILABLE 00 a) comparing two fractional representations of a whole number. b) comparing the value of a decimal fraction with a common :raction. St. Crcix district examinees showed weak performance on the following skills while St. Thomas/St. John examinees displayed adequate achievement on them: a) recognizing a numeral distributed into two factors b) simplifying an expression within parenthese before performing other operations. Examinees in both districts showed weak achievement in the following sicills: a) adding two unit fractions with different denominators b) comparing values of ':wo common fractions with equal numerators. c) naming improper fractiol.- as "mixed numbers. Addition and subtraction algorithms-this set of skills involves the use of a series of prelearned steps for finding sums and remainders. Examinees from both districts showed at leaEt adequate skills on all objectives tested in this area. Strong skills were demonstrated on the following objectives: a) subtracting a three digit subtrahend from a 4 digit minuend renamin7 tens and hundreds with a disappearing left digit. b) adding a broken column of four addends with renaming. Examinees from both districts displayed adequate skills in the following areas: a) finding the value of a sentence placeholder by subtraction b) finding the minuend in a horizontal algorithm when the subtrahend and the -remainder are known 94 BEST COPY AVAILABLE 91 cj subtracting a four digit subtrahend from a five digit minuend, with zero in the minuend and renaming in all positions Multiplication and division algorithms-this set of skills involves the use of a series of prelearned steps for finding products and quotients. Examinees in both districts displayed at least adequate skills on all the objectives tested in this area. They showed particularly strong skills in the following areas: a) multiplying a two digit numeral by a one digit multiplier, renaming ones b) dividing a three digit numeral by a one digit divisor, renaming tens c) multiplying a three digit numeral by a two digit multiplier with zero in the multipliciand, renaming ones and tens St. Thomas/St. John school district examinees showed particularly strong skills on the following objectives, while St. Croix district examinees showed adequate skills on them: a) multiplying-two three digit numerals, with zero in the ones position of the multiplier and renaming b) multiplying a three digit numeral by a two digit multiplier with renaming St. Croix district examinees showed particularly strong skills in the area of dividing a two digit numeral by a two digit divisor while St. Thomas/St. john district examinees showed adequate skill on this objective. All USVI examinees showed adequate skills in the following areas: a) estimating the value of a quotient when the dividend and divisor are known 95 BEST COPY AVAILABLE 92 b) dividing a four digit numeral by a one digit divisor obtaining a quotient with zero in the tens place c) dividing a two digit numeral by a one digit divisor obtaining a quotient with a remainder Common fractions-St. Croix district examinees showed particularly strong ability in finding the least common denominator for three fractions with unlike denominators. St. Thomas/St.John district students snowed adequate skill in this area. USVI examinees showed adequate skill in adding two fractions 1_ having common denominators and a sum less then one. They showed weak skill in finding a unit fraction of a multiple of a denominator. Mathematics Applications Item Analysis All items showed acceptable discrimination indices except the following. Item #26- This item sets up a one to many relationship while supplying inadequate data. Examinees are required to recogniv: the piece of data which would be needed in order to solve the problem. The item was only moderately difficult for the local sample of students (p=.43). It is likely that the low level of discrimination seen here is a rt,,ult of the item requiring a different type of skill than the others tested on the examination. All other items required students to solve word problems or Lnterpret grilphs. In short, the intercorrelation between this and other items is probably low for this sample of students. One explanation for this is that the skill is not specifically taught as part of the USVI math curriculum, but 96 BEST COPY AVAILABLE 93 may be learned, independently, by many students. Item #36- This item discriminated sati5factorily among St. Thomas/St. John district examinees, but unsatisfactorily among St. Croix school district examinees. This was a very difficult question (p=.11) for all St. Croix examinees and this would account for the low level of discrimination. Item #38- This was a very difficult item for the local sample of examinees (p=.16) and this would account for the low level of discrimination observed. Item #40-This was also a very difficult item for the local sample of examinees (p=.11) and this fact most likely accounts for the low level of discrimination which was found. Summary of Student Skills Selection of appropriate operation-These items tested the ability of examinees to choose the mathematical operation required to solve a word problem. USVI examinees showed adequate ability in eposing the operation for finding the total number of objects expressed in different measures of quantity. They showed weak skills in their ability to choose the appropriate operation for finding: a) a numl.er of equal units by dividing b) "how many" of a unit number by dividing St. Croix district students displayed weak skills in their abilities to determine the appropriate operation for finding. a) specific rate by division b) finding the remainder by subtraction c) the rate per unit of time by multiplication. 9 7 BEST COPY AVAILABLE 94 St. Thomas/St. John examinees showed adequate achievement in these skills. Analysis and development of solution designs- These items required students to carry out the following series of steps: 1) determine the operations needed to solve a word problem, 2) order the operations, and 3) carry out the operations in the appropriate manner in order to arrive at a correct solution. Examinees in both school districts showed adequate ability on the following objectives: a) adding and subtracting in a two step sequence b) translating verbal problems into mathematical sentences c) determining correct change by adding and subtracting St. Thomas/St. John school district examinees showed weak skills in the ability to determine what additional information is needed to solve a word problem. St. Croix district examinees showed adequate skills in this area. St. Croix district examinees showed weak skills in the following area while St. Thomas/St. John examinees displayed adeql Ite skills: a) rounding three prices to estimate total price b) determining and analyzing specific rate Examinees in both districts showed weak skills on the following objectives: a) multiplying and adding in a two step sequence b) rounding price of item and estimating cost for a number of items 98 BEST COPY AVAILABLE 95 c) multiplying and subtracting in a two step sequence Rate and Scale Problems- These items involve solving word problems dealing with the compulation of rate of change and the use of map scales. Examinees in both sci districts showed adequate skills in determining what percent of a figure is shaded. St. Croix school district examinees displayed weak skills in determining a specific rate based on an average while St. Thomas/St. John examinees showed adequate achievement in this skill. Examinees in both school districts showed weak achievement on the following objectives: a) determining specific rate b) finding a unit fraction of a number c) finding 50% of a given amount of money d) finding a total when the part of its fractional representation of the whole is known e) interpreting a map scale to find relative distan ,:s f) expressing part of a total as a fraction Measurement- Students in both school districts showed weak achievement on all of the objectives tested in this area. They were: a) adding times expressed in hours and minutes b) converting standard units of linear measure c) finding the area of a region d) converting standard units of liquid measure e) subtracting two measures and converting into another unit of measure 99 BEST COPY AVAILABLE 96 Graph Reading and Interpretation Examinees in both school districts displayed adequate achievement in using bar graphs to make comparisons. St. Croix school district examinees showed weak achieve- ment in the following areas: a) finding the difference between two dates in a table b) using a bar graph to determine specific information St. Thomas/St. Jchn school strict examinees showed +- adequate achievement in the above two areas. Examinees in both districts showed weak _aility in using bar graphs to combine frequencies. Spelling Item Analysis All items on the spelling test functioned appropriately for examinees in both school districts. Summary of Student Skills Homophones- These are woras which sound the same, but have different meanings and are spelled differently. On these items students had to determine the appropriate form of the homophone from content and identify its correct spelling. Examinees in both school districts showed adequate skills in distinguishing between the following homophones: a) modal would instead of noun wood b) noun mind instead of verb mined c) verb heard instead of noun herd d) noun tide instead of verb tied e) adjective plain instead of noun plane No BEST COPY AVAILABLE 97 St. Croix district examinees showed weak abilities to distinguish between the following homophones: a) verb seems instead of noun seams b) noun tier instead of verb tear St. Thoras/St. Joh', district examinees exhibited adequate skills on these objectives. Students in both districts were weak in distinguishing the verb sealing from the noun ceiling. Phonics- These objectives dealt with spelling skills that were a function of the examinees' ability to use phonetic rules. Examinees from both school districts showed particularly strong skill in identifying misspelled words where /e/ is spelled e instead of ea. St. Thomas/St. John district examinees showed particularly high ability in identifying misspelled words where /en/ is spelled en instead of eon. St. Croix examinees displayed average ability on this skill. St. Croix district examinees showed particularly high . ability in identifying misspelled words where a) a syllable is omitted b) /en/ is spelled en instead of on c) one letter of a double consonant is omitted d) /el/ is spelled al instead of le St. Thomas/St. John students showed adequate abilities in these skills. Examinees in both districts showed adequate ability in 101 BEST COPY AVAILABLE 98 identifying misspelled words where a) the schwa syllable is omitted b) /i/ is spelled i instead of ui following /g/ c) the schwa sound is omitted d) the vowel digraph r is misspelled e) /k/ is spelled ck instead of c f) ie is reversed g) /e/ is spelled a instead of i h) a "silent" letter is omitted i) /s/ is spelled s instead of c j) /s/ is spelled se instead of ce k) /s/ is spelled c instead of s 1) /ks/ is spelled xs instead of x m) p is omitted in ( m (p) xY cluster n) consonant letter is double,, unnecessarily St. Thomas/St. John examinees showed weak ability to identify misspelled words when a final /n/ is spelled in instead of ine. St. Croix examinees showed adequate ability in this skill. Word Building- St. Thomas/St. John examinees demonstrated particularly strong ability in identifying misspelled words in which a) the final consonant letter is doubled when adding-ing b) the morpherem /en/ is spelled in instead o: en c) -s instead of -es is added to the plural form. St. Croix school district examinees displayed adequate performance levels on these skills. 102 BEST COPY AVAILABLE 99 St. Croix district examinees demonstrated particularly strong skills in identifying misspelled words in which a) final e is omitted when adding -s to form the plural b) the final 1 is omitted when adding the suifi-f. -ly c) the plural is formed without changing f to v d) -ed is added without doubling the final consonant letter St. Thomas/St. John examinees showed adequate performance levels on these skills. Examinees in both districts showed adequate skills in identifying misspelled words:. in which a) y is changed to i when adding -ed b) a consonant letter is deleted from a prefix assimilation c) final e is retained when adding -ing d) a suffix is added without changing ye to i e) morpheme /er/ is spelled er instead of or f) compound is formed without changing all to al g) -ing is added without doubling the final consonant letter h) suffixes -ery is confused with -ary i) suffix is added without doubling final consonant j) plural is formed without changing y to i k) -ed is added without doubling final consonant letter 1) the suffix -ours is spelled -us Language Item Analysis All items on the Language test descriminated satisfactorily except the following. Item #2 was a moderately easy item for 103 BEST COPY AVAILABLE 100 St. Croix students (p=.79) which required students to recog- nize that a comma needed to be placed between the day of a month and the year. Examinees in the lower 27% of the distribution were almost as likely as those in the upper 27% to score correctly cn this item (.68 vs. 77). There appeared to be no pattern of responses and it is likely that this skill simply does not correlate highly with those measured on other parts of the test. Items #3 & 4 were particularly difficult for St. Croix examinees (p=.14 and .19, respectively) and this could account for the low levels of discrimination observed. These items dealt with the use of commas to set off parenthetical expressions. Item #6 was a particularly difficult one for St. Croix students 9p=.30). Only three more students in the upper group scored correctly on this item than in the lower group. It dealt with the placement of a question mark at the end of a quotation. Item #10 was a difficult item for USVI students (p=.34). Students in the upper 27% of the distribution of scores were only slightly more likely to score correctly than those in the lower 27% (p=.39 vs .34). This item dealt with the formation of past participles from infinitive verb forms. Item #11 was a moderately easy item (p=.78) for St. Thomas/St. John examinees. Examinecs in the lower group were not much less likely to score incorrectly on this item 104 BEST COPY AVAILABLE 101 than those in the upper group.. It involved the choosing of the correct present perfect tense of the verb "to choose." The distractor "have chose" was chosen almost always by examinees who scored incorrectly. The item seems to be operating properly, but it does not correlate highly with most other items on the test. Item #21 was a very difficult item for St.Croix school district examinees (p=,19). This accounted for the low discrimination index observed for this item which test students' knowledge of the use of quotation marks to set off direct quotes. Item #22 was a moderately difficult item for St. Croix students (p=.43), but examinees in the upper and lower 27% of the distribution were equally likely (p=.50) to score correctly on this item which tested to see if students knew how to deal with indirect quotations. Item #33 showed poor discrimination for St. Croix district students (d=.14) The item response pattern suggests a reason for this. The item purports to measure whether students can identify the appropriate use of the present perfect tense of the verb "to become." The item reads: "With power in most U.S. towns, the electric clock became very common." The choices presented were: 1) became, 2) has becomed, 3) has became, and 4) has become with alternative #4 scored as the correct choice. A large number of the upper 27% of examinees who scored incorrectly chose alternative #1. Alternative #1 produces a gramatically correct sentence, but choice #4 is 105 BEST COPY AVAILABLE 102 said to be correct since preceding items deal with the history of electricity. Most interesting is the fact that item #33 appears at the head of a left hand page in the test booklet and students must remember the flow of questions or look back to the previous page. Nowhere in the directions are examinees told about relationships between items. In summary, this item contains two correct answers and this could account for the poor discrimination which was observed. Item #40 was a very difficult (p=.23) item for St. Croix district examinees. This would account for the low level of discrimination that was observed. Item #42 was a very difficult item for USVI students (p=.14). The item examined students' ability to choose an appropriate verb to complete the sentence, " The loudspeaker his voice into the crowd." The choices were 1)sputtered, 2) splashed, 3) stressed, 4) boomed with alternative #4 scored correctly. The majority of examinees chose alternative #3. This accounts for the low level of discrimination observed. There may be something in the general language use of USVI students that could explain these responses. Summary of Student Skills Capitalization conventions-These are skills which involve examinees' ability to recognize appropriate uses of capitali- zation of words. St. Croix district students showed weak skills in recognizing 106 BEST COPY AVAILABLE 103 the appropriate use of capitalization at the beginning of a direct quotation. St. Thomas/St. John district examinees displayed adequate skills in these areas. USVI examinees showed adequate skills in the recognition of capitalization conventions in the following situations: a) at the beginning of a principal clause following an introductory clause b) with a common noun c) at the beginning of an indirect quotation d) with the name of a building Punctuation conventions- These skills involve the ability to recognize the appropriate use of punctuation in a given situation. St. Croix school district examinees displayed weak ability to recognize a siutation where quotation marks and a comma are used to set off a direct quotation from the rest of a sentence. St. Thomas/St. John examinees showed adequate performance in this skill. USVI students displayed adequate skills in recognizing the appropriate use of punctuation where: a) a comma is used after an introductory adverb clause b) a comma is used betwen a day and a month c) commas are used to set off parenthetical expressions d) a question mall( is used at the end of an interrogative sentence and it is enclosed in quotation marks e) no punctuation marks are used to set off indirect quotations f) commas are used to separate ideas within series 107 BEST COPY AVAILABLE 104 Usage conventions- This refers to skills involving the ability to choose appropriate forms of words depending on the content of tie sentence in which they are used. USVI students showed strong ability to choose an appropriate set of words consisting of a demokstrative pronoun and an auxiliary verb from content. St. Thomas/St. John district students displayed strong ability in selecting the correct form of the present perfect tense of the verb "to choose" from content. St. Croix district students showed adequate skills in this area. St. Croix district students showed strong skills in the following usage situations: a) selecting the past participle of the verb "to burn" and the past perfect progressive form of the verb "to speak" from content b) selecting the tense of the verb "to invent" and avoiding a double negative in content. c) selecting the past participle of the verb "to bring" from content. St. Thomas/St. John district students displayed adequate skills in these areas. U.S. Virgin Islands students displayed low ability in the following areas of usage: a) selecting an appropriate comparative adjective from content. b) selecting the past tense of the verbs "to draw" and "to drive" from content. c) selecting an appropriate adverb from content. St. Thomas/St.John school district examinees displayed BEST COPY AVAILABLE 108 ., 105 poor skills in being able to select the present perfect tense of the verb "to bcceme" from content. St. Croix district examinees showed adequLe ability in this skill. St. Croix school district examinees showed poor skills in the following usuage areas: a) selecting the past participles of the verbs "to Know" and "to come' from content. b) selecting the past tense of the verbs "to know" and "to begin" from content. St. Thomas/St. John school district examinees scored adequately in these areas. USVI students all performed adequately in the following usage areas: a) selecting the present perfect form of the verb "to take" from content. b) selecting the negative forms of a sentence avoiding double negatives c) selectini; the past participles of the verbs "to breek" and "to eat" from content d) selecting the past participle of the verb "to begin" from content e) selecting the past perfect tense of tne verb "to drink" in content f) selecting the past participles of the verbs "to freeze" and "to go" in content g) selecting the appropriate reflexive pronoun in content h) selecting a verb to agree with a subject in number and to avoid a double negative i) selecting a pronoun form in a compound subject and identifying the proper word order for a proper name and pronoun 1@9 BEST COPY AVAILABLE 106 j) selecting the present perfect tense of the verb "to write" from content k) selecting the past participle of the verb "to take" and the past tense of the verb "to choose" in content 1) selecting the passive voice form of the word "to drive" from content m) selecting the appropriate verb to agree with a subject in number and the appropriate comparative adjective to modify a plural noun Language sensitivity: Word sensitivity- These skills relate to the ability of examinees to perceive appropraite usage of words in a particular context. USVI examinees showed adequate skills in the ability to complete a sentence by selecting an appropriate clause. They showed weak skills in the ability to complete a sentence by selecting an appropriate independent clause. St. Croix district examinees showed strong skills in the ability to complete a sentence by selecting an appropriate verb phrase. St. Thomas/St. John examinees showed weak skills in this area. St. Thomas /St. John _ i district examinees showed weak abilitie'- to complete sentences by selecting an appropriate: a) preposition b) posessive pronoun to begin with an antecedent c) verb d) predicate adjective Language sensitivity: Sentences- USVI examines demonstrated adequate skills in the following areas: 110 BEST COPY AVAILABLE 107 a) choosing the phrase which completes a sentence fragment b) recognizing a complete sentence c) recognizing that a group of words forms a single complete sentence d) recognizing that a group of words forms two or more sentences e) recognizing that a group of words do not form a complete sentence Dictionary skills- These consist of the abilities needed to locate words in a dictionary ani to use the dictionary entry to find needed information. USVI examinees displayed adequate ability in using sample dictionary entrees to: a) determine the appropriate meaning of a word in content b) identify the part of speech of a word in content c) determine syllabication d) determine the pronounciation of a word and choose a word that rhymes with it e) indicate the appropriate content for a particular meaning of a word f) identify guide words using alphabetical order USVI examinees showed weak skill in their ability to use a sample dictionary entry to determine the pronounciation of a word. 111 BEST COPY AVAILABLE 108 Discussion A detailed sumrdary of overall strengths and weaknesses will be presented in a final report on conclusions and recommendations. Howeler, at this time a number of points can be made concerning test performance and academic achi3ve- ment of USVI examinees across grades. The problem of examinees having insufficient time to complete the reading test is still evident in grade six examinees. The fact that it ceases to exist in grade four and two sheds some light on the nature of the causes of this phenomenon. In the grade four test (Primary Level III) only the second half of the test involves items where students were required to read lenghthy passages and answer questions based on them, while the second grade test (Primary Level I) has no such items. this seems to reinforce the idea that local examinees may either require more time to complete these items because they are slower readers or must refer back to the passages more often to answer questions, or that they tire easily given these type of questions and do not finish the test. Unlike the situation with secondary students it was noticed that when the analysis was restricted to items which two-thirds or more of the examinees, USVI students performed weil on higher level reading objectives such as globa2 and implicit meaning, although they still were weak in de- termining inferential meaning. BEST COPY AVAILABLE 112 . 109 Secondary examinees showed difficulty in all reading areas besides explicit meaning. As was the ease with secondary examinees, examinees in grades four and six show weak skills in the area of standard English vocabulary. In grade two, however, St. Thomas/St.John district examinees showed adequate skills in recognizing vocabulary that would be found in subject matter material in school, while St. Croix examinees were weak in these vocabulary areas. Examinees from both districts showed weakness in general standard English vocabulary. The fact that at least part of the local sample if students showed adequate vocabulary achieve- ment in subject matter areas may be the function of the emphasis places on these areas in schooling. It suggests a need to explicitly teach all areas of vocabulary. Grade twn examinees seem to show adequate skill in the use of phonetic skills for word building. This skill seems to deteriorate in grades four and six with grade four students having difficulty discriminations among certain vowel sounds and grade six examinees showing difficulty discriminating among consonant sounds, as well. This brings.to question the effectiveness of the instructional reading program in the intermediate grades and the opportunity examinees have to practice and have these phonics skills reinforced. In mathematics, the pattern seen with secondary students (see Poliss, 1982b) is evident with elementary students, as well. Students seem to have adequate computational skills 113 BEST COPY AVAILABLE 110 and seem to have good grasp of basic number facts and algorithms. They have moderate skills in dealing with one step word problems, but are very weak 'ealing with operations and word problems which require more than one step in order to arrive at solutions. USVI examinees also seem to have weak skills in seeing relation- ships between numbers and geometric shapes and -.his phenomenon is obvious across all grades. Local examinees display excellent spelling skills and this '- phenomenon is also seen across all grades. The pattern that was observed by Poliss (1982b, pp97-98) where secondary students seemed to achieve higher on objectives in the lower three levels of Bloom's Taxonomy of educational objectives (Bloom,1956) was also observed across elementary students, as well. Jensen (1968) suggested a paradigm which seems to fit this observed pattern when he suggested his level I and level II abilities idea. Level I abilities would be those which require rote memorization and simple manipulative skills (the first three levels of the taxonomy) while level II skills require higher level reasoning (the higher three levels of the taxonomy). Jensen postulates that differences in the ease with which persons learn level I abilities are smaller between socioeconomic-groups'thandifferences,in the_ease,with which persons learn level II abilities with lower socioeco-_ class persons having the greater difficulty in learning these skills. USVI examinees appear to have achieved at least adequately on objectives which would be classified in the lower three cate- gories of the taxonomy (i.e. knowledge, comprehension, and application). 114 BEST COPY AVAILABLE 111 They achieved less well on objectives that would 1-e classified in the upper three categories (analysis, synthesis, and evaluation). Local examinees seem to achieve well on objectives which can be achieved by memori,ation (knowledge objectives) or the application of a general procedure, rule, or algorithm (comprehension and application levels). They scored extremely well in spelling and single step multiplication computations. They scored lower on items requiring them to organize data, choose information relevent to solving problems, and to choose a series of rules and/or algorithms to use in solving problems (analysis, synthesis, and evaluation). Note the weaker achievement in mathematics applications problems, multistep computations, and the use of dictionary entries, for example-This phenomenon is apparent as early as grade two and becomes more obvious as the grade level of examinees increases. Another area that may be fruitful to examine is the area of instruction. While the Stanford Achievement Test seems content valid based on an aAalysis of written curricula, texts, and what teachers and administrators say they are doing in classrooms, the poorer showing of examinees on higaer level objectives may be a function of the emphasis placed on these objectives during instruction. If these higher level objectives are not taught in the classroom or are taught in such a way as to send a message to students that they are not very im- portant, low achievement on these objectives is a plausible outcome. For instance, if classroom tests only evaluate low level objectives, students quickly learn that these are the 115 BEST COPY AVAILABLE PI 112 really "important" things for them to learn. Likewise, if the amount of time spent dealing with these objectives in the class- room test only evaluate low level objectives, students quickly learn that thes' are the really "important" things for them to learn. Likewise, if the amount of time spent dealing with these objectives in the classroom is disproportionate to that spent on higher level objectives, a similar message goes out to examinees. It may be worth knowing precisely what is going on in classrooms during instruction. Finally, as with the secondary stud,mts, it was observed that elementary students omitted itmes that they had an opportunity to attempt to respond to. Although the directions did not tell students this, the raw score of an examinee was the number of . items which he or she answered correctly. There was no correction for guessing (i.e. no points were deducted for incorrect responses). Under these " number right" conditions, examinees should have guesse' at all items they did not know the answer to in order to maximize their scores. The fact that there were omits on many items indicates that many examinees did not use this strategy. This may have been a function of inadequate test directions (i.e. the directions did not indicate the method of scoring being used. but this is a somewhat dubious assumption since most tests these examinees take are "number right" tests. Rather, the problem is most likely one of poor test taking skills and these skills can and should be taught in the classroom. 116 BEST COPY AVAILABLE 113 References Allen, M.J. 5 Yen, W.M. Introduction to measurement theory, Monterey, CA: Brooks/Cole Publishing Co., 1979. Bliss, L.B. Virgin Islands of the United States public school basic skills achievement survey, Technical Report #1: Validation of the use of the Stanford Achievement Test with U.S.V.I. students. St. Thomas, USVI: Caribbean Research Institute, College of the Virgin Islands, 1982(a) Bliss, L.B. Virgin Islands of he United States public school basic skills survey, Technical Report #2: Item analysis and report of student skills of secondary school students. St. Thomas, -Ina.: Caribbean Research Institute, College of the Virgin Islands, 1982 (b) Bloom, B. (Ed.) Taxonom of educational ob'ectives, Handbook I: Cognitive domain. New Yor : Longmans, Green F Co., 1956. Cureton, E.E. The upper and lower twenty-seven per cent rule. Psychometrika, 1957, 22, 293-296. Jensen, A.k. Patterns of mental ability and socioeconomic status. Proceedin s of the National Academ of Sciences, 1968, 60, 1331 -1337. Kelly, T.L. The selection of upper and lower groups for the validation of test items. Journal .-)f. Educational Psychology, 1939, 30, 17-24. 117 40, dES1 COPY AVAILABLE