USVI Public Records

Document text

DOCUMENT RESUME ED 267 120 T™ 860 212 AUTHOR Bliss, Leonard B. TITLE Item Analysis and Report of Student Skills of Elementary School Students. Virgin Islands of the United States Public School Basic Skills Achievement Survey. Technical Report #3. PUB DATE Jan 84 NOTE 117p.; For related documents, see ED 227 150 and ED 243 964. Some pages may be marginally legible. PUB TYPE Reports - Research/Technical (143) EDRS PRICE MFO1/PCO5 Plus Postage. DESCRIPTORS *Academic Achievement; *Achievement Tests; *Basic Skills; Elementary Education; Evaluation Methods; Grade 2; Grade 4; Grade 6; *Item Analysis; Language Skills; Mathematics Skills; Reading Skills; Standardized Tests IDENTIFIERS *Stanford Achievement Tests; *Test of Academic Skills ABSTRACT A sample of slightly over 1500 students from even numbered grades in public schools in the U.S. Virgin Islands were given the 1973 edition of the Stanford Achievement Test (in grades 2, 4, 6, and 8) and the Test of Academic Skills (in grades 10 and 12) in an attempt to assess student academic achievement in the basic skill areas of mathematics, reading, and English Language. Technical Report 1 detailed the procedures used in test selection, sampling, and test administration. Technical Report 2 reported the results of the analysis of scores for secondary students (grades 8, 10, and 12) including an item analysis and summary skills on all objectives. This report describes Phase III of the data analysis which involves a detailed item analysis of each item on each test given to the sample of students in grades 2, 4, and 6 as well as a summary of student skills based on their achievement along objectives provided by the test publisher and keyed to individual test items. Measures of item difficulty and item discrimination were calculated for the entire territorial school system and for the individual school districts. (Author/PN) KEEETETKEEKEEETEKEKETKTEKAEKKKCKKKKKAKAKKEKEEKEKEEREEERKEEEKKKEKEEKEKEKEKEKEEKREREKEEEEE Reproductions supplied by EDRS are the best that can be made e * from the original document. as TEKEETEEKTERTEEEEEETKEETKEKEKAARAAAKKEARAAREKARERRRKEKKERKEKEKEREEKEEEREREEREREEE ED267120 Leonard B. Blis Caribbean Research College of the Virgin NATIONAL INSTITUTE OF EDUCATION ED Ai RES RCE NeORVA ATIONAL E . v9 January 1984 Abstract Introduction Background Item analysis Difficulty indices Discrimination indices Summary of student ski Availability of data 2nd Grade (Primary Level Vocabulary. Reading (Part A) Reading (Part B) Word Study Skills Mathematics Concepts Mathematics Computation Listening Comprehension 4th Grade (Primary Level I11) —_ Vocabulary .. Reading Comprehension hens Word Study Skills. ee Rene SS Mathematics Concepts SpoSiathenatics si eieeanlabinan eFeesuaes : wee ; NDNaAwWWN > > & ts & th Wk OV 1) ~J Lan) ~) —" ding Comprehension *~ Word Study Skills * Mathematics ‘Concepts Mathematics Computation Mathematics ida ication Spelling .. Language aononTw~]™ moro Ul = Discussion erences . BEST COPY AVAILABLE A sample numbered grades were given the 197 (in grades 2, 4, and 8) and th (grades 10 an 12) in an attempt achievement in the basic skil and English Language. This report describe the data analysis which involved a detailed item analysi each item on each test given to the sample of grades 2, 4, and 6 as well a based on their achievement alo test publisher and keyed to individual 1 items. Measures of item difficulty and item discriminati se calculated for the entire territorial individual school districts. St Gate “ee At the reques of Education and the Board 0Oi the Virgin Islands, the C: on a study of basi Islands public schools. It soon becam h any ements wel Strategy designed to improve basi ki chievement needed to start off with a fairly detailed description of current achievement levels students in territorial public schools. It was found that this was n available. iesign the study decided that the most efficient way btain information on levels of basic skills achievement was to administer a standardized achievement test to a representative sample of students and to analyze the results his test. Technical Report #1 (Bliss, 1982a) describes, in detail, the process used to choose an appropriate test. Finally, the Stanford Achieve- ment Test (1973 version) was 1osen as the instrument of choice. Briefly, the reasons yr choosing this achievement test battery were that 1) it cov d the grades K-12, 2) it seemed, on initial observation, t e a good match with subject content taught in the sc! ) it was technically sound given the population on which it was standardized (a seemingly representative sample of mainland U., S. students), >and 8) it would report out criterion referenced results. BEST COPY AVAILABLE : ~ n+ ra ~ S 4 é Y 2ints ~tadi —VUillliagidile Due to various organi only students in even numbered ades were tested. This seemed acceptable since many h bjective Stanford Achievement Test the test and there was nc of academic achievement were different from constraints are detailed in Sampling of students t cluster sampling technique wi The details of the sampling Technical Report #1 along wi of using this sampling technique on Table 1 presents a breakdown emerged. Table U. S. Virgin Island Thomas I . John Test Level Sy istrict <5 | Sere Task II ror pe scores ~a pe SS ~ es F Task I Advanced Intermediate I] Primary III Primary Total - “aC OMMe 5 a hy +> Testing was done recommended by the test publisher during week of Octcber 21, 1980 in the St. Thomas/St. John district and during the week of December 1, 1980 in the St. Croix district. Testing materials and A completed answer sheets were collected, answer sheets checked to determine compliance with marking instructions, and answer sheets sent off to be machine recorded. Technical Report #3 Background | This is the second of four reports that will deal with the results of the basic skills assessment described above. Technical Report #1 detailed the procedures used in test selection, sampling, and test administration. More impor- tantly, it established empirically, the content validity and the reliability of the Stanford Achievement test when it was ee ee ee eee) administered to a sample of U. S. Virgin Islands public school students. This is particularly important since there exists no standardized test of academic achievement which includes USVI students in its standardization group. Technical Report + (Bliss, 1982b) reported the results en ee eee ee ee of the analysis of scores for secondary students (grade 8, 10, and 12 ) including an item analysis and summary skills This report examines the scores of elementary school ‘Students (grades 2, 4, and 6) and presents: 1) an item analysis of each item on each test of the battery which includes indices of item difficulty and discrimination and BEST COPY AVAILABLE a summary of skills based on their scores on the items specific objective Item Analysis Difficulty indices The difficulty index of an item is found by taking the number of examinees correctly on an item and divid- ing it by the total I f students taking the test. In short, it is the proportion of examinees who scored correctly on an item and has a range from 0 (no students scoring correctly) to 1 (all studetns scoring correctly). Because the fact it is a proportion, it is often designated in the literature as "p" (e.g. p = .75). It may be worthwhile to point out that the term "difficulty index" may be somewhat a misnomer since items with high difficulty indices are actually less difficult than items witn low difficulty indices. Nevertheless, the term and its definition have become standard in the area of psychometrics throughout the United States. Difficulty indices for each item on each test were reported out by the test scoring service. In addition, YY difficulty indices for examinees in the standardization S ; >, group in the same grade as local examinees at approximately ; the same time of year are reported. The test scoring BEST COPY AVAILABLE greater or less than the proport ° y | £ Standardization group scoring C e .05 level of Significance. Significant differences 1n either direction were reported out. Discrimination Indices The item discrimination index indicated the degree to which responses on one item are related to responses on other items on the test. The statistic indicates whether a person who does well on the test as a whole {that is, a person who is presumably high on the trait being measured) is more likely to get the particular item correct than a person who does poorly on the test as a whole. In other words, the item discrimination index indicates whether an item discriminates between those who do well and those who do poorly on the test as a whole. Taking the item diffi- culty and the item discrimination index into consideration, the developers of tests desire to construct tests which , discriminates well among examinees with varying levels of a trait. The item discrimination index is calculated by the the number of examinees who have total test scores in the upper range of total test scores and who @also have the item correct. 3EST COPY AVAILABLE the number of examinees who have total test in the lower range of total test scores and also have the item correct. the number of examinees in the upper or low of the test s By definition, d is tl Tr between the proportion of high scoring examinees who got the item correct and the proportion of low scoring examinees who got the item correct. Upper and lower ranges generally are defined as the upper and lower 10% to 33% of the sample, with examinees ordered on the basis of their total test score. When total scores are normally distributed using the upper and lower 27% pro- duces the best estimate of d (Kelly, 1939). If the distri- bution of total test scores is flatter than the normal curve, the optimum percentage is larger and approaches 33% (Cureton, 1957). However, Allen and Yen (1979) found that, for most applications, any percentage between 25 and 33 will yield Similar estimates of d. In this study, 27% was used as the upper and lower percentage because examination of selected distributions of actual test scores revealed nearly normal distributions. The theoretical range of d is between -1 and #1. ever, maximum discrimination is likely to occur when p s © . . hen p= .50 the variance in item scores > which is p (1 - p), 2S Maximized. As an item becomes more difficult, it is less Be tikely that any student will score correctly on it. As it ~becomes less difficult it is more likely that any student BEST COPY AVAILABLE oe Ferenm ame + oe treme se + “+e +a er eew + will get it correct. This could lead to the suggestion that all items should have p = .50, bi the usefulness of this Suggestion is mitigated by intercorrelation among items. In an extreme case, if the items on t all intercorre- lated perfectly and had diffic €xaminees would receive a tota and the other half would have perfect test scores. Hen there would be no fine discrimination between examinees' levels of achievement or whatever trait was being measured. In general, test designers tend to try to choose item range of difficulties that average around .50. particularly low difficulty are often included in tests (usually among the earlieritems) for motivational reasons. — Gee are aw - Item discrimination indices were calculated in this Study to provide indications that items may be flawed when used with USVI students. Such flaws are ambiguity, the presence of clues, the presence of more than one correct answer, and other technical defects. If none was found ‘upon examination of the item, and it was determined that ‘the item did, indeed, appear to measure the objective it was intended to, the item was included in the overall “analysis of results. <Any item that discriminates positively can make a contribution to the measurement of pupil achieve- peo SR ment~and low indices of discrimination are frequently _ obtained for reasons other than item defects. Standardized achievement tests are designed to measure BEST COPY AVAILABLE Where hj ic +} es S ft ee HNnere tmas 385 tre CcCasc¢c, understanding, application, ©€ test items that represent an area receiving relatively little emphasis will tend to have poor discriminating power. For + + example, if a test ! y it specific facts and 1 items measuring latter items can be (pe i to have low discrimination indices. This is becaus he ns measuring understanding have less representation in the total test score and there is typically a low correlation between measures of knowled and measures of understanding. Low discrimination indices here merely indicate that these items are measuring some- thing different from what the major part of the test is measuring. Removing such items from the test would make it a more homogeneous measure of knowledge outcomes, but it would also damage the content yalidity of the test because it would no longer measure objectives in the understanding area. Since achievement test batteries need to measure a wide variety of objectives in a reasonably short period of time, they tend to be fairly heterogeneous in nature and moderately low discrimination indices tend to be the rule rather than the exception. To summarize, a low discrimination index alerts test users to the possible presence of defects in test items but does not cause them to discard these items if they appear to be functioning as they should. A well constructed achieve- “ment test will of necessity, contain items with low discri- pee eang power and to discard them would result in a test BEST COPY AV AILABLE 12 which is less, rather than more, valid. Due to these consi- derations, in this study items were examined if they had discrimination indices lower than .20. This is a rather conservative criterion since items that discriminate as low as this may provide useful information, but given the un- known test taking characteristics of USVI students, it was decided to be particularly cautious in the item analysis. Summary of Student Skills The items of the tests of the various levels of the Stanford Achievement Test battery were keyed to behaviorally stated instructional objectives. These objectives are grouped into a series of item groups. Tables are available which present objectives within item groups and the difficulty and discrimination indices for the item or items which evaluate those objectives. These may be obtained from the Caribbean Research Institute. In addition, the difficulty index for each item for the examinees in the standardization group is available. This standardization group consists of examinees who were in the same grade at approximately the F “same time during the school year as the U. S. Virgin Islands Sample. The national p values are used not as a means to compare U. S. Virgin Islands students with mainland U. S. examinees. Historical and cultural differences between these two groups ~~ : - ° e ° - “a of examinees makes this comparison an inappropriate one. = *"* Philosophical considerations aside, however, such comparisons are of little use to the people who make curricular decisions in schools. What these people need to know are the particu- ages “Yar ‘levels of skills of students as measured against well aye mt s# - defined objectives, not how well their students achieved 14 est COPY AVAILABLE 4 a3 expected to change from grade to grade for a particular Student or group of students. The publishers of the Stanford Achievement Test take this factor into considera- tion by testing particular objectives across a number of levels (grades) of the test. In this study the national p values are used to indicate the relative level of difficulty of an item by which the performance of the local sample may be judged. For instance, it would be foolish to be dissat- isfied if 20% of the USVI students have indicated that they have reached an objective when only 18% of comparable Students in the standization group had reached that same objective. What is more likely the case is that this is a difficult and complex objective that had just been taught ‘recently and would be retaught and enlarged upon at a later Therefore, the following criteria were used in summariz- sane student skills. Skills are described as "adequate" if es “the proportions of local examinees scoring correctly on i itens measuring those skills are not Significantly higher or Lower, than the proportions of the standardization group tg gfesring correctty or, if significantly higher or lower, the Biezinees scoring correctly on items measuring those p*lsnificantly higher and more than 10% greater ee BEST COPY AVAILABLE than the standardization group proportion correct as reported by the test scoring service. Skills are described as "weak" if the proportion of local examinees scoring correctly on items measuring those skills are significantly lower and more than 10% less than the standardization group proportion correct. The 10% standard was set in the realization that some differences, while statistically significant may be educationally trivial and it was noted that most differences indicated as significant exceeded 10%. Finally, in cases where the scores of examinees from both the St. Thomas/St. John and the St. Croix districts were the same based on the criteria stated above summaries were based on the entire USVI sample. Where differences were noted, the skills of examinees within each district were summarized separately. Availability of Data Tables listing each objective on every test keyed to test items with national difficulty indices, difficulty indices for the entire USVI sample and the individual dis- =;,.,trict subsamples as well as item discrimination indices for eet, the whole sample and each subsample are available and may be obtained from the Caribbean Research Institute, College of the Virgin Islands. 3EST COPY AVAILABLE 13 Grade 2 - Primary I Level The Primary I Level of the Stanford Achievement Test consists of seven tests with the number of items listed below: 1. Vocabulary - 37 items 2. Reading Comprehension (Part A, 45 items 3. Reading Comprehension (Part 42 items 4. Word Study Skills items 5S. Mathematics Concepts items 6. Mathematics Computation items 7. Listening Comprehension items The means, standard deviations, and reliability estimates ee for these tests, determined previously (see Bliss, 1982a), tot Sage Ae ee: are presented below: Mean Standard KR- 20 Standard Raw Score Deviation Reliability Error Total USVI Sample re 72 Reading Compre - , ension (Part A) 13. . 98 mY ~92 rematics ‘Con- re Se. , ee : Met Mathematics Com- orf Ree aod ee cae § kh: t = £ wd Tie . A 2 at z tion - ° e 81 ‘ us a ening Compre- sion 3EST COPY AVAILABLE St. Thomas/St. John Sample Vocabulary ; .8 71 Reading Compre- re oe hensiogn (Part A) , , . 99 Reading Compre- hension (Part B) ; 91 Word Study Skills 48.7 ay 91 Mathematics Con- cepts ° od .69 Mathematics Compre- hension Listening Compre- hension Croix Sample Vocabulary 20. ; a Reading Compre- _» hension (Part A) ; . .94 | = rie. | Reading Compre- hension (Part B) ; : .92 .68 Word Study Skills 93 .68 Mathematics Com- prehension , : 75 a F ._ Mathematics Com- _,,.putation 20. 83 21 Daf SAAN onde rae aed ae ; PY 34 Following are the item analyses and summaries of student ... achievement on the objectives evaluated by each of the tests. BEST COPY AV AILABLE Vocabulary Item Analysis All items showed acceptable discrimination indices with the following exceptions: Item #1 was particularly easy for students in the USVI Sample (p , . This accounts for the low level of discri- Mination by this item and is not significantly different from the standardization group yalue of p = .91. Item #2 was particularly easy for examinees in the St. Thomas/ St. John group in all classes tested (p = .99). This would account for the low level of discrimination level observed (d * .05). Item #3 and #4 were very easy for the examinees in the St. Thomas/St. John sample (p = .93 and .90, respectively. This accounts for: the low Jevels-of the observed. difficulty indices. Item #8 was a moderately difficult item for USVI examinees (p = .66). The teacher asked, "If a place is not safe, it is -"' The alternatives were, 1) accidental, 2) secure, and 3) dangerous. Many students who scored in the ‘upper 27% of the examinees on this test chose "“acciden- eis - tal” while the incorrect choices were more evenly spread between the two distractors for those in the lower 27%. “There is some factor operating that draws higher achieving : Students to Choice #1, but an inspection of the item, itself BEST COPY AVAILABLE ee all he nil. lle lI te ll a All Nt a nt — 16 Item #9 was an easy one for St. Thomas/St. John school district examinees (p = -87). This accounts for the low level of discrimination that was observed (d = .15). Item #20 discriminated negatively for St. Croix district examinees (d = -.12). This moderately difficult item (p = .31) had the teacher ask "A small man is a .” The alternatives were 1) boy, 2) girl, and 3) woman. Most exam- inees chose "woman" as the correct choice and, in the case of St. Croix students this was true of slightly more of the upper 27% of students than of the lower 27%. There may be a cultural influence in the way language is used in the Virgin Islands which accounts for this. Item #32 did not discriminate between higher and lower achieving students in th St. Croix school district (d = 0). This was a difficult item for these examinees (p = .23) and this very likely accounts for the lack of discrimination observed. Item #27 showed low discrimination for St. Croix school district examinees (d = .16). This was a moderately diffi- cult item (p = .47) and it appeared more difficult for these students than for the St. Thomas/St. John district “examinees (p = .54). On the surface, the problem seems to be that a lower proportion of the upper 27% of examinees in es “St. Croix than in the St. Thomas/St. John district scored correctly on this item (25% vs 82%) and a higher proportion of the lower 27% of the St. Croix district examinees than of BEST COPY AVAILABLE the St. Thomas/St. John examinees scored correctly on it (203 vs 48%). USVI examinees in both districts did as well on this item as the standardization group (p = .49) and there appeared to be no consistent pattern of incorrect reponses. The reason for this low level of discrimination is not evident from the information available. The same phenomenon is evident on item f28. Item #33 discriminated poorly for St. Croix district examinees (d = -.04). The same phenomenon observed in Items 27 and 28 was observed. Summary of Student Skills USVI examinees from both school districts showed weak skills in standard English vocabulary which was made of words generally encountered in reading oral language and listening. The average difficulty indices for the St. Thomas/ St. John district, the St. Croix district, and the standardi- zation group of students were .57, .51 and .68, respectively with St. Thomas/St. John district students scoring indices Significantly below those of the standardization group on seven of the 13 items and St. Croix students scoring lower on 11 of the 13 items. St. Thomas/St. John school district examinees scored _ adequately compared to the standardization group on recogni- tion of words from nonfiction and reference sources (mean ae” Pp .69 vs .72) while St. Croix district examinees showed @ > weak vocabulary skills in this area (mean p = .57). BEST COPY AVAILABLE 18 St. Thomas/St. John school district examinees scored adequately on recognition of vocabulary found in the areas of mathematics and science (mean p = .64 vs .69 for the standardization group) while St. Croix district examinees showed weak vocabulary skills in this area (mean p © .57). USVI examinees in both school districts demonstrated adequate skills on recognition of vocabulary found in the area of the social sciences (mean = .60 for the St. Thomas/ St. John district, mean p = .56 for the St. Croix district examinees, and p = .61 for the standardization group). Reading Comprehension (Part 1!) Item Analysis All items on this test discriminated satisfactorily “for St. Croix school district examinees. The following items Fee 2, Showed poor discrimination for St. Thomas/St. John district ii, “Students: Items #1 and #2 were very easy for these examinees (p = .99 and .93, respectively). Since most examinees scored correctly on these items, it was almost as likely for ya Student in the lower 27% of the group to score correctly “as it was for a student in the upper 278 of the group. This » would account for the low discrimination indices (d = .05 and gee -18, respectively) observed for these items. Items #4 and #7 were very easy items for the St. Thomas/ St. John school district sample of students (p = .95 and eee 98, respectively) and this would account for the low BEST COPY AVAILABLE 19 levels of discrimination that was observed on these items (d = .15 and d = .13). Items #10 and #13 were very easy for this group of examinees (p = .98 and p= .95, respectively) accounting for the low discrimination indices which were observed ( d = .08 and d = .13). Items #16 and #19 were also very easy for this group of second graders from the St. Thomas/St. John school district (p = .95 for both items), explaining the low levels of observed discrimination (d = .15 for both items). Summary of Student Skills USVI examinees from both the St. Thomas/St. John and the St. Croix school districts displayed adequate skills on the test which evaluated their ability to match words which appear in reading materials for primary grades and in general ' listening vocabulary with illustrations. The mean difficulty indices for the St. Thomas/St. John school district examinees, the St. Croix district examinees, and the standardization group were .78, .68, and .72, respectively. St. Thomas/ St. John examinees scored as well or better than the stan- dardization group examinees on all items. St. Croix examinees scored significantly lower on 15 of the 45 items on this test than did the standardization group, and did not score significantly higher on any of them. re BEST COPY AVAILABLE = 20 Reading Comprehension (Part I1) Item Analysis All items on this test discriminated satisfactorily except the following: Item #1 was very easy for the St. Croix school district sample of examinees (p = .93). This would account for the low level of discrimination (d = .16) which was observed for this question. Item #2 and #3 were very easy for the examinees in the St. Thomas/St. John school district (p = .99 and .95, respectively). This would account for the low levels of discrimination observed for these items. Item #16 was a moderately easy (p = .70) item that did not discriminate satisfactorily among St. Croix examinees (d= .12). The item presented a picture of a little girl lifting a doll out of a gift box. The item read, "Dad gave Sally a surprise on her birthday. It was in a _ The alternatives were 1) box, 2) car, 3) toy. Ninety percent of the upper 27% of the St. Thomas/St. John examinees scored correctly on this item compared to only 76% of the upper 27% of examinees in the St. Croix sample. Most of the upper group students in St. Croix sample chose alternative #2 giving the impression that they may have been reading “beyond what was actually presented since on St. Croix, a gift would almost certainly be brought home by automobile. Even though the directions explicitly reminded examinees to look at the pictures in answering the questions, high scor- ing exami , P ies 8 aminees found the distractor "car" attractive. 24 BEST COPY AVAILABLE 20 Reading Comprehension (Part I1) Item Analysis All items on this test discriminated satisfactorily except the following: Item #1 was very easy for the St. Croix school district sample of examinees (p = .93). This would account for the low level of discrimination (d = .16) which was observed for this question. Item #2 and #3 were very easy for the examinees in the St. Thomas/St. John school district (p = .99 and .95, respectively). This would account for the low levels of discrimination observed for these items. Item #16 was a moderately easy (p = .70) item that did not discriminate satisfactorily among St. Croix examinees .12). The item presented a picture of a little girl lifting a doll out of a gift box. The item read, "Dad gave Sally a surprise on her birthday. It was in a ‘ ... The alternatives were 1) box, 2) car, 3) toy. Ninety percent of the upper 27% of the St. Thomas/St. John examinees scored correctly on this item compared to only 76% of the SY upper 27% of examinees in the St. Croix sample. Most of the Ay upper group students in St. Croix sample chose alternative #2 giving the impression that they may have been reading beyond what was actually presented since on St. Croix, a _gift would almost certainly be brought home by automobile. Se 7d we Byen though the directions explicitly reminded examinees to New gy sook at the pictures in answering the questions, high scor- examinees found the distractor "car" attractive. 29 BEST COPY AVAILABLE 22 Item #37 was a moderately easy (p = .67) item that discriminated poorly (d= .12) among St. Croix school district examinees. The item presented a picture of a little boy and a woman, both in bathing attire sitting on the ground with water behind them. The boy is building a sand castle. The item reads, "then he went swimming in the ocean while his mother sat in the ___."" The alternatives were 1) shed, 2) sun, 3) summer. All of the upper 27% of the St. Thomas/ St. John examinees scored correctly on this item while of 76% of the St. Croix examinees scored correctly. On the other hand, only 46% of the lower 27% of St. Thomas/St. John examinees vs 64% of the lower 27% of the St. Croix examinees scored this item correctly. There was no pattern of distrac- tors chosen by students scoring incorrectly in either the high or the low group and the overall discrimination indices were virtually identical (p = .66 on St. Thomas/St. John and p = .67 on St. Croix). For some reason, this item correlated poorly with other items on the test for:St. Croix examinees, but the reason for this is not apparent from the information that is available. Summary of Student Skills The problem encountered at higher grade levels where examinees did not seem to attempt items coming up later in the reading comprehension test (Bliss,1982b) did not arise at this grade level. There were 42 items on this test compared to as many as 71 on the reading comprehension 4 es S34 AES sy as fo test on Intermediate Level I] (grade 6). In addition, 26 BEST COPY AVAILABLE 23 none of these items were of the form where examinees were required to read a passage of text before answering questions. This further supports the speculation that either USVI exami- hees tire quickly when asked to perform on items where they must read longer passages, OF that they are more deliberate readers who take a longer period of time to read these passages. On this level, only 7% of examinees omitted the last item on the test. USVI examinees from both school districts showed adequate achievement in determining meaning that is explicitly stated in a sentence or short paragraphs. St. Thomas/St. John pestis and St. Croix district examinees answered these eo 2 “Btems with a mean proportion correct of .74 and .65, respectively, compared to a mean p = .69 for the standardiza- ~ tion group. Local examinees showed adequate achievement in determin- St. Thomas/St. John district and St. Croix examinees answered items testing this skill with a Ftion correct of .72 and .65, respectively, com- -71 for the standardization sample. -Item Analysis — Most items on the word study skil lls test + g fattsfactority for students in the USVI sample. In each Nease where an item showed poor discrimination it appeared to Be due = a high difficulty index (i.e. examinees found the ve 27 BEST COPY AVAILABLE 24 item very easy to answer correctly). These items and the discrimination indices (d) and difficulty indices (p) are listed below. For examinees in both the St. Thomas/St. John and St. Croix districts (d and p are for the combined res- ponses from both districts): Item# For examinees in the St. Thomas/S x district, only: Itemt 4 8 10 11 13 20 25 Summary of Student Skills Consonant sounds - USVI examinees from both the St. Thomas/St. John and the St. Croix school districts showed adequate skills in all areasof discriminating among graphemes for consonant sounds that were measured by this test. These sounds were: a) the initial consonant sound /k/ b) the initial consonant sound /d/ c) the initial consonant sound /p/ d) the initial consonant sound /r/ e) the initial consonant sound /j/ f) the final consonant sound /t/ g) the final consonant sound /1/ h) the final consonant sound /d/ i) the final consonant sound /k/ Consonant clusters - St. Croix school district showed weak skill in their ability to discriminate among graphemes for the constant cluster of’ an initial consonant plus r (gr). St. Thomas/St. John district examinees showed adequate “achievement in this skill. USVI examinees from both school districts demonstrated adequate achievement on all other items on this test which measured their abilities to discriminate among graphemes for consonant clusters. These clusters included consonant clusters of: a) am initial consonant plusr (tr) b) an initial s plus a consonant (sp) - ae BEST COPY AVAILABLE =< ‘ ‘<- a= a SAS. os" 5 a 2st => Lage: i> 5 Saeed E. Si 5 sys 17 c) an initial s plus a consonant (sk) d) an initial consonant plus 1 (cl) e) a final consonant plus t (nt) f) a final n plus a consonant (ns) g) a final n plus a consonant (ns) h) a final consonant plus d (nd) i) a final consonant plus k (1k) Consonant degraphs - USVI examinees from both the St. Thomas/St. John and the St. Croix school districts showed adequate skills in discriminating between consonant degraphs on all . items of this test which measured this skill. These consonant degraphs included: a)_/sh/ in the initial position /ch/ in the initial position /th/ in the initial position /hw/ i initial position /ng/ i final position Vowel sounds - USVI examinees from both school districts demonstrated exceptionally strong skill in discriminating among graphemes for the short vowel sound /e/ spelled e. St. Thomas/St. John school district examinees showed strong skills in discriminating among graphemes for the following vowel sounds: a) the long vowel sound /o/ spelled o-e b) the long vowel sound /yoo/ spelled u-e c¢) the long vowel sound /e/ spelled by the diagraph ee BEST COPY AVAILABLE 27 St. Croix school district examinees displayed adequate achieve- ment in these skills. St. Croix examinees displayed weak skills in the ability to discriminate among graphemes for the long vowel sound /e/ spelled ea. St. Thomas/St. John district students showed adequate achievement in this skill. USVI examinees from both school districts showed ade- quate ability to discriminate among graphemes for the follow- ing vowel sounds. The: a) short vowel sound /a/ spelled DO on (BORE short vowel sound /i/ spelled short vowel sound /o/ spelled short vowel sound /u/ spelled ae) 40 at th hah _ long vowel sound /a/ spelled long vowel sound /i/ spelled 7 te ll. lll, ily long vowel sound /a/ long long ‘long vowel vowel vowel sound sound sound /a/ /oo/ /o/ spelled spelled spelled spelled diagraph diagraph diagraph diagraph long vowel sound /o/ spelled diagraph ow Sl-plus r combinations - USVI examinees from both istricts Showed weak skill in the ability to discimi- Oe a kel nl ats htt nttn mong graphemes for the vowel plus r combination /ur/ = : iminating among graphemes for the jollowing vowel plus r combinations: BEST COPY AVAILABLE a) /ur/spelled er b) /or/ spelled or St. Croix district examinees showed adequate achievement in these two skills. Examinees in both school districts displayed adequate skills in discriminating among graphemes for the following vowel plus r combinations a) /er/ spelled er b) /ar/ spelled ar c) /ur/ spelled ir d) /ur/ spelled ur Silent consonant letters -~ USVI examinees from both the St. Thomas/St. John and the St. Croix school districts dis- played adequate skills in discriminating among graphemes for the: a) hn spelling of /n/ b) wr spelling of /r/ c) ght spelling of /t/ St. Croix school district examinees showed weak skills in discriminating among graphemes for the a) 1k spelling of /k/ | b) mb spelling of /m/ St. Thomas/St. John examinees showed adequate achievement in these skills. Diphthongs - St. Thomas/St. John school district examinees showed strong skills in their ability to discrimi- nate among graphemes for the diphthong /ou/ spelled ow. Raped tee tes ‘ & ; 51S 3s ee. > - ~ fe i “" SWE sat 30 BEST COPY AVAILABLE = 29 St. Croix district examinees showed adequate skills in this area. USVI examinees from both school districts displayed adequate achievement in discriminating among graphemes for the following diphthongs: a) /ou/ spelled ou b) /oi/ spelled oi c) /oi/ spelled oy Mathematics Concepts Item Analysis Item #1 and #2 were very easy for USVI examinees from _. both school districts (p = .97 for the two items) and this would account for the low discrimination indices which were observed (d = .05 and .06 respectively). Item #3 discriminated poorly for USVI examinees from ‘both school districts (d = .02). This was a moderately easy item (p = .83) which required examinees to count the number “of dots in a set of dots arranged a ae ‘ a soe : -and choose the number of dots in the set from the alternatives _ 24) 15, 2) 16, 3) 17, 4) 18. It appears that examinees in the ypupper 27% of student scores on this test are just as likely _ to score correctly on the item as are those in the lower 27% » respectively) and this is almost the same —as-the proportion correct for the whole sample. Examinees ms-—~F BEST COPY AVAILABLE 30 scoring incorrectly seemed to choose alternative #2 (16) most often indicating that the problem they are haying is not due to their ability to recognize the numeral 17, but may be due to the counting method or pattern they are using. This latter skill may not correlate highly with other skills measured by the test. Item #2 did not discriminate well for examinees in either the St. Thoams/St. John or St. Croix school districts (conm- bined d = .09). This was a very easy item (combined p = .95) and this would account for the low discrimination index which was obtained. ; Item #14 did not discriminate satisfactorily for | sexaminees in the St. Thomas/St. John school district (d = .08). “This was a difficult item for these examinees (p = . 29) and yo. this should account for the low level of discrimination that was observed. a ee Item #19 did not discriminate well among St. Croix “school district examinees (d are * 2 In this moderately “@igficult (p = .38) item, examinees were asked to observe Boos circles arranged in a random manner and to choose the i-which half the circles are colored in. There appeared tad pattern of incorrect responses. As in Item #3, ie. were required to come up with a strategy for count- penese €ircles and it may be that this skill is not related highly with the others on the test. “Item #22 discriminated poorly for St. Croix district -08). This was an item of average difficulty BEST COPY AVAILABLE © 31 (p = .59) which showed the numeral 431 and required examinees to identify the digit in the 100's place from among the alternatives, 1) 1, 2) 3) 4, 4) 0. St. Croix examinees did well on this item compared to the standardization group No pattern of incorrect responses could’ be noted. Item #23 did not discriminate well for the smaple of USVI examinees (d = .06). This was due to the fact that it was a very easy item for these students (p -98). Item #25 was a very difficult item (p .25) for USVI examinees and this accounts for the low level of the observed discrimination index (d = .17). Summary of Student Skills Number, notation, and operations. St. Thomas/St. John school district examinees showed particular skill in their ability to identify the sign associated with an operation. St. Croix district examinees showed adequate achievement in this skill. St. Thomas/St. John examinees also showed strong skill in indicating the position of a block of dots relative to other blocks. St. Croix district examinees showed weak achievement in this skill. St. Thomas/St. John school district examinees showed weak skills in the following: a) indentifying a set having a given number of objects. b) selecting a set whose number lies within a given interval. eA ot ee ee thats oe 32 St. Croix district examinees displayed adequate achievement in these skills. Examinees from both school districts showed weak skills in the following: a) identifying a group in which half of the object are marked b) indicating the additive identity in set theory USV] examinees ‘from both the St. Thomas/St. John and the St. Croix school districts showed adequate skills on the following: a) choosing the numeral representing a set of objects b) choosing the number name representing a set of objects c) choosing the missing numeral in a series choosing a numeral representing the greatest relative value choosing a set matching a given set in number matching the number of a set with its representation as the sum of its subsets identifies the digit having a specified place value h) chooses the set which is equivalent to a given set i) identifies a numeral represented as tens and ones Geometry, measurement, and logical thinking. USVI examinees from both the St. Thomas/St. John and the St. Croix school districts demonstrated adequate skills in: BEST COPY AVAILABLE 33 identifying the object furthest from a specified point distinguishing between simple geometric figures determining the field having the greatest area identifying the figure having a given geometric relationship solving a problem in elementary logic involving transitivity determining which object has the greatest area unused g) identifying a clock showing a specified time h) ordering containers by volume showed weak skills in the following: a) indicating the 3-dimensional block figure having the greatest number of blocks b) identifying the longest of four paths c) solving problems in elementary logic involving Spatial relationships finding the total value of 3 coins identifying appropriate units of measure identifying a group of Objects sccording to two properties Mathematics Computation Item Analysis Item #1 and #2 discriminated poorly for examinees in the St. Thomas/St. John school district (d = .08 and .18, respectively). These were easy items for these students BEST COPY AVAILABLE ils deel an el tlt MM a nella 34 (p = .90 for both) and this would account for these low discrimination indices. Item #5 discriminated poorly among St. Croix district examinees (d = .00). This was a very easy item for these Students (p = .92), accounting for the low obseryed- discri- mination. Item #8 discriminated poorly for the whole USVI sample (d = .02). This was a moderately difficult item (p = .40) where examinees were asked to determine the number of pieces of candy in five boxes each containing two boxes of candy. The alternatives were 1) 5, 2, 3) 3, 4) 10. Examinees in the lower 27% group were almost just as likely to score correctly on this item as those in the upper 27%. As a whole, there was no significant difference between the pro- portion scoring correctly the local sample and the proportion scoring correctly in the standardization group. No patterns of incorrect responses was evident. Item #13 discriminated poorly for the whole USVI sample (d = .11). The fact that this was a very difficult item for these examinees (p = .28) would account for this. Item #14 showed poor discrimination for St. Croix district examinees (d = .04). This was due to the fact that these students found the item extremely difficult (p = .11). Item #10 was a very difficult one for St. Thomas/ St. John examinees (p = .18) and because of this it discri- minated poorly among these examinees (d = .09). BEST COPY AVAILABLE 35 Item #16 and #20 were very easy items for St. Thomas/ St. John examinees (p = .96 and .94) and this accounts for the low levels of discrimination which were observed (d = .08 and .18). Item #15 discriminated poorly among USVI examinees from both districts (d = .09). This was most likely due to the fact that this was an extremely easy item for this sample of students (p = .95). Items #20, #22 and #31 were each very easy St. Thomas/St. John school district examinees (p and .92, respectively), resulting in unacceptably low discri- mination indices (d = .18 and .15). Summary of Student Skills Addition and subtraction facts. St. Thomas/St. John school district examinees demonstrated particularly strong skills in subtracting number facts horizontally. St. Croix district examinees demonstrated adequate achievement in this skill. Examinees from both school districts showed adequate skills in adding two number facts horizontally and in adding a column of three 1l-digit numbers. Mathematical sentences. St. Thomas/St. John examinees showed strong skill in finding the subtrahend in a subtrac- tion problem which is presented horizontally. St. Croix school district examinees demonstrated adequate achievement in this skill. BEST COPY AVAILABLE USVI examinees from both districts showed adequate skills finding » add d i! an addition problem presented horizontally finding the minuend in a subtraction problem pre- sented horizontally Verbal problems. Examinees from both the St. Thomas/ John and the St. Croix school districts showed adequate their abilities to solve verbal problems by: adding the cost of two small purchases adding to find a total Subtracting to find how many are left adding to find how many there are all together adding the cost of three smali purchases showed weak skills in solving verbal problems by: a) subtracting to find how many more are needed b) multiplying (or adding) to find a total when an equal number is given in each group ¢) subtracting to make a comparison d) using an elementary rate formula St. Thomas/St. John school district examinees showed Skills in solving verbal problems by: a) finding the minuend in a subtraction problem Simple division St. Croix school district examinees demonstrated adequate achievement in these skills. BEST COPY AVAILABLE Listening Comprehension Item Analysis Item #1 was a very eas school district examinees for the low level of discrimi! (d = .08). Item #2 discriminated poorly for St. Thomas/St. John school district examinees (d i} e In this moderately e (p = .82) item students had been read a passage by the teacher which related an outing by three children in which they parti- Cipated in a number of activities. Ome of these was picnic- ing in the shade of a tree. They were then presented with pictures of a pond, a tree, and a rock and asked what the children in the story had their picnic by. It was noted that the proportion of examinees scoring correct in the upper 27% of the group was the same as that of the entire sample of St. Thomas/St. Croix examinees (.82). It appears that Students in the upper group wh« -ored incorrec often chose the pond a he correct answer. The item seemed clear and concise and passage wa: S\ lite clear and Simple, yet higher achievement examinees seeme: > attrac- ted to the pond distra > reason apparent from the avail: in mation. Item #12 did not seem discriminate Croix school dist ity of an examinee j; 1e t 27% of the group scoring correctly on this item the same as that of a BEST COPY AVAILABLE 38 Student in the lower 27% and slightly lower than the propor- for the entire St. Croix sample (p =.42). Most examinees scoring incorrectly in this item chose the picture of a boy over the correct reponse of the picture of a girl although the concept in the read passage is clearly a feminine one with feminine adjectives and pronouns used, thoughout. T > discriminated poorly for t. Thomas/St. John istrict examinees (d ~08). This was an easy item for these students (p = .85) and this would account for the low level of discrimination that was observed. Items #12 and #14 did not discriminate well for St. Thomas/St. John school district examinees. Both items referred to the same passage of material and had the same choices. These were pictures of 1) an artists' paint brush, 2) a tray, and 3) a pair of scissors. Approximately the same proportion of examinees scored correctly in the upper and low 27% of students in both items, although, the latter item seemed somewhat easier for these examinees (p = .41 for item #13 and p = .49 for item #14). There appeared to be no visible pattern of incorrect responses on either item. Both items require examinees to determine conceptual mean- ing from the passage and relate this to knowledge they already possess andit maybe that these examinees do not possess the requisite knowledge due to geographical or cultural reasons. BEST COPY AVAILABLE 39 Explicit detail - In this set of items examinees were required to identify specific details which were presented in nearly the same language as in the passage which was read to them or recognize that a specific detail was not mentioned; identify a sequence of events; or follow specific directions. USVI examinees from both districts showed weak Skills in understanding explicit detail. The mean propor- tion answering correctiy in this group of items was .74 for the St. Thomas/St. John sample and .69 for the St. Croix sample, compared to .82 for the standardization group. St. Thomas/St. John examinees scored significantly lower on 80% of the items in this grouping while St. Croix district examinees scored significantly lower than the standardiza- tion group on all items. Implicit meaning - This set of items required examinees to identify what was heard when it was not presented expli- citly, thus requiring some degree of interpretation. USVI examinees from both the St. Thomas/St. John and the St. Croix school districts demonstrated weak skills in understanding implicit meaning. The mean proportions of examinees cor- rectly answering these item was .53 for the St. Thomas/ St. John sample and .54 for the St. Croix sample, compared with .62 in the standardization group. Conceptual meaning and classification - In this group of items, examinees were required to identify and classify 43 BEST COPY AVAILABLE 40 concepts by relating what was heard to known concepts, or identify emotional attitudes or character traits or discern the meaning of words from content; or employ a conception of language to identify the doer or receiver of an action; or use concepts of comparative terms. St. Thomas/St. John examinees display adequate skills in this area (mean p = .66 compared to mean p = .68 in the standardization group) while St. Croix examinees demonstrated weak skills (mean p = .52) scoring significantly below the standardization sample in 80% of the items in this grouping. Inference and logical analysis - In this set items, examinees were required to make inference and draw conclu- sions from what was heard; or to analyze problem situations and passages and make logical deducations. St. Thomas/ St. John examinees showed adequate achievement in these skills (mean p = .68 compared with a mean p = .72 for the standardization group), scoring significantly below the Standardization group on only 18% of the items. St. Croix school district examinees showed weak skills on this group- ing of items (mean p = .58) and scored below the standard- ization group on about 92% of the items in this item grouping. BEST COPY AVAILABLE 41 Grade 4 - Primary III Level The Primary III Level of the Stanford Achievement Test consists of eight tests with the number of items listed below: Vocabulary - 45 items Reading Comprehension - 70 items Word Study Skills items Mathematics Concepts items Mathematics Computation items Mathematics Appications items 7. Spelling items 8. Language " items The descriptive statistics for these test scores, determined previously (see Bliss, 1982a), are presented below. Mean Standard KR-20 utandard Raw Score Deviation Reliability Error Total USVI Sample Vocabulary , 2 83 .96 Reading Comprehension , / 91 .46 Word Study Skills : ee : .16 Mathematics Concepts : a Be oo Mathematics Computa- —_— 52 Mathematics Applica- tions ; 16 Spelling : 69 Language 28 BEST COPY AVAILABLE Thomas/St. John Sample Vocabulary . . eh Reading Comprehension » . 91 Word Study Skills . : . 88 Mathematics Concepts , . .66 © Mathematics Computa- ; tions 9. ‘ sto Mathematics Applica- tions ‘ ‘ 84 Spelling , . 9] Language . Be 84 Croix Sample Vocabulary , P . 88 owe Reading Comprehension ‘ ; 96 + Word Study Skills , ‘ ~92 .06 Mathematics Concepts : , . ~47 Mathematics Computa- tions 21.¢ i 87 .47 Mathematics Applica- tions 3. ; -18 Spelling ; . 2.74 Language ‘ a ; ta Following are the item analyses and summaries of student achievement on the objectives evaluated by each of the tests: Vocabulary Item Analysis All items showed discriminated Satisfactorily except the following: 46 BEST COPY AVAILABLE 43 Item #1 was a very easy item for St. Thomas/St. John school district students (p = .91) and this accounts for the low level of discrimination which was observed. Item #21 - This item asked examinees to choose the term which finished the statement, "If an object is shut up on all Sides it is .'"' The alternatives provided were 1) released, 2) empty, 3) detached, 4) enclosed. Alternative #3 was scored as correct. Most examinees scoring incorrect- ly on this moderately difficult item (p = .62 chose "detached" as their response. Students in the upper 27% were almost as likely to chose this response as were those in the lower 27% of total scores on the yocabulary test. There may be a fine distinction in the mind of fourth graders between something being "enclosed" and being "detached" from the outside world and many of the examinees may not have been able to make this distinction. This explanation is strengthened by the fact that almost all of the upper 27% of students scor- ing incorrectly on this item chose alternative #3 ("detached"). Item #33 was a very difficult item for St. Thomas/ St. John school district examinees (p = .14) and this would account for the low level of discrimination which was ob- served. Item #45 was a very difficult item for USVI students (p = -23) and this would account for the low levels of discrimination which were observed for examinees in both school districts. 47 BEST COPY AVAILABLE 44 Summary of Student Skills The sample of USVI students tended to be generally weak in the area of standard English vocabulary recognition. These weakness was seen across all areas of the best which included vocabulary found in a) general reading and literature b) nonfiction and reference words c) mathematics and science d) social science On 31 of the 45 items on the test (69%) USVI examinees had a significantly lower percentage of examinees scoring cor- rectly than the standardization group with only one item where they scored correctly a significantly greater propor- tion Of the time than the standardization group. Aside from the issue of statistical significance, the proportions correct were very low for USVI samples on most items on the test when compared with those of the standardization group. Reading Comprehension Item Analysis The phenomenon of large numbers of students having trouble in completing the reading comprehension tests in secondary grades (Bliss, 1982b) and in the sixth grade sample was not as obyious in the fourth grade sample. In fact the proportion of omits for item #70 (the last item in the test) was only 23% for the entire USVI sample compared 48 BEST COPY AVAILABLE 45 to 71% on the last item (#71) of the test given to the sixth grade sample. In addition, the percentage of omits for fourth graders did not appear to rise dramatically until item #53 where it was only 8%. Previously, the rise in Omits was attributed to some phenomenon attached to the way USVI students respond to items which require them to read passages of text and respond to questions about them. This idea is further by observing the patterns of omits for fourth graders. at he lil as itl tw lle ll Re Unlike the reading comprehension tests of more advanced levels which are composed entirely of reading passages fol- lowed by questions about the passages, Primary Level II con- Sists of stories where questions are placed after almost Se a lial A wld sade every sentence. For instance: Mary ran across the street holding a bottle of pop. It slipped from her hand and i 4 E 4 { : broke cried A car drove over it and cut its 2 1) window door = 2) roof ; tire This pattern continues through item #35 after which the familiar type of passage and questions form begins. The first two passages are short (6 and 8 lines, respectively) and this is followed by a longer passage (z number of omits begins to increase after this longer pas- sage and its questions (item #S3). BEST COPY AVAILABLE 46 All items showed satisfactory discrimination indices except the following: Item #1 and #6 were very easy items for USVI students (p = .95 for both items) and this would account for the low levels of discrimination observed. Item #8 was a very easy item for St. Croix district examinees (p = .91) which would account for the low discri mination index seen on this item. Item #9 was a very easy item for St. Croix students (p = .93) and this accounts for the low discriminatiolr levels observed. Item #35, #59, and #70 were very difficult items for USVI students (p = -11, and .17, respectively) accounting for the low levels of discrimination observed for these items. Summary of Student Skills The mean difficulty index for items testing global or general meaning of material read was .67 for the USVI Sample compared to .73 for the standardization group. The local sample of examinees scored significantly lower than the standardization group on only two of the six items measuring this skill. USVI students appear to have adequate skills in determining global or general meaning from readings. Tne St. Thomas/St. John and the St. Croix sample of students showed mean difficulty indices of .50 and .53, respectively, compared to ov = .59 for the standardization BEST COPY AVAILABLE 47 group on items testing the ability to identify explicit meaning from readings. adequate skills in this area. In the area of determining USVI examinees showed a districts compared to a mean p group. It plays adequate skills In determining meaning by content, St. mean difficulty index of Local examinees appear to have implicit meaning from readings -72 in both .74 for the standardization appears that the local sample of students dis- in this area of reading comprehension. Thomas/St. John school district examinees had a mean difficulty index of -67 and St. index of .66 compared to a mean tion group. USVI students seem this area. In the area of determining reading both the St. Thomas/St. of students had mean difficulty p = .60 for the standardization local in this grouping, than the standardization group. Croix district examinees scored a mean difficulty of .73 for the standardiza- to perform adequately in inferential meaning from John and the St. Croix sample indices of .48 compared to group. On 10 of the 13 items examinees scored significantly lower It appears tne USVI students are weak in their ability to determine inferential meaning from readings. Word Study Skills Item éAnalysis All items showed Satisfactory discrimination indices except for the following: BEST COPY AVAILABLE 48 Item #10 shows d = .00 for both local samples. In this item the examinees are supposed to recognize the vowel sound /o/ in “awkward" when the word is pronounced by the teacher and choose from among the four words, 1) after, 2) always, 3) other, 4) again written in their text books to find a word that "has the same beginning sound."" Local students almost consistently chose alternatives 3 and 4 ("other" and "again") over the correct choice "always." It is possible that there are regional differences in pronounciation at work here. The word "always" is pronounced witn a primary /o/ sound in the northeastern U.S., but is sounded like /ah/ in other parts of the U.S. and among U.S. Virgin Islands speakers of English. The pronounciation "ahl' ways", being the common pronounciation, it was very likely the one used by students who chose other alternatives. Since there is very likely no correlation between where the persons who students immitated while learning to speak (most likely their parents) and their word study skills, in general, the lack of discriminating power of this item is no mystery. Further, the value of this item in measuring the skill it purports to measure is dubious for the local population of students. Item #27 was a very easy item for USVI examinees (p = .94) and this accounts for the low leyel of discrimina- tion observed. 4ST COPY AVAILABLE 49 Summary of Student Skills These items examine the ability of letter within USVI examinees showed adequate among graphemes for: a ) the initial consonant sound b) the final consonant sound c) the final consonant sound They showed weak skills in discriminating among graphemes for: a ) initial consonant sound b) initial consonant sound c) initial consonant sound d) the final consonant sound e) the initial consonant sound Consonant digraphs and consonant clusters USVI examinees showed weak skills in discriminating among graphemes for the consonant digraph /th/ in the initial position. St. Croix school district examinees weak ability in discriminating between graphemes for: a) Cluster of initial s + a consonant (sp) b) he digraph /ch/ in the initial position c) ne digraph /sh/ in the final position Gc) digraph /sh/ in the initial position St. Thomas/St. John school district examinees displayed adequate skills in these areas. BEST COPY AVAILABLE 50 from both school districts showed adequate ;criminating between graphemes for: cluster of an initial consonant cluster of final sk cluster of a final consonant + digraph /ng/ in the final position vowel sounds - USVI examint¢es showed weak inating among graphemes for: the short vowel sound /i/ the short vowel sound /o/ St. Croix school district examinees showed weak in discriminating among graphemes for: a ) the short yowel sound /u/ b) the short vowel sound /oo/ St. Thomas/St. John examinees displayed adequate skills in area. ISVI examinees from both school districts displayed adequate skills in discriminating among graphemes the short vowel sound b) the short vowel sound /a/ Long vowel sounds - St. Croix school district examinees showed weak skills in discriminating among graphemes for: a) the long vowel sound /e/ b) the long vowel sound /a/ St. Thomas/St. John examinees showed adequate skills in these areas, BEST COPY AVAILABLE $1 USVI examinees from both districts displayed adequate skills in discriminating among graphemes for: the long vowel sound /i/ the long vowel sound /90/ the long vowel sound /o/ d) the long vowel sound /yoo/ Variant vowel sounds - USVI examinees from both school districts showed weak skills in discriminating among graphe- mes for: a) the diphthong /o / b) the schwa sound c) a vowel sound before r These examinees showed adequate skills in discriminating among graphemes for: a) the vowel sound /o/ b) the diphtong /oi/ Affixes - USVI examinees from both school districts showed strong ability to discriminate among graphemes for the ' suffix "-tion."' St. Croix examinees showed strong skill in discriminating among graphemes for the prefix "-en." St. Thomas/St. John district students showed adequate achievement in this skill. Examinees from both school districts showed adequate Skills in discriminating among graphemes for: a) the prefix “pre-," b) the suffix "ance." BEST COPY AVAILABLE Mathematics Concepts Item Analysis All items seemed to discriminate satisfactorily except item #22 for the St. Croix school district sample of examinees. This was a very difficult item for these students (p = .16) and there was no pattern of incorrect responses that could be noted. Summmary of Student Skills Number - This item grouping refers to concepts which deal with the relationships between different types of numbers (e.g. primes, odds vs evens, fractions vs wholes). USVI examinees from both school districts showed weak skills in their ability to choose a number belonging in a given number series. St. Thomas/St. John district examinees showed weak skills in the following areas: a) representing a whole number in terms of a specified fraction determining the point at which a given number is located on the number line c) adding in a finite system of clock module St. Croix district examinees demonstrated adequate skill in these areas. USVI students from both school districts displayed adequate skills in the following areas: BEST COPY AVAILABLE 53 identifying a number at a particular point on a number line identify a set consisting only of even numbers indicating the position of a number relative to three given numbers Notation - refers to the technique by which numbers are expressed in written form. Some forms include expanded nota- tion, differing base values, decimals, and fractions. USVI examinees from both school districts showed weak skills in: a) identifying the remainder of a subtraction problem when the minuend is represented by a place-value device b) choosing the expanded notation representing an addition example St. Thomas/St. John school district examinees displayed weak ability in determining the difference between two nun- bers represented on a place value device. St. Croix district examinees showed adequate achievement in this area. St. Croix school district examinees showed weak ability in the following skills: a) translating the word form of a given number into Arabic numerals b) identifying the expanded notation representing a a given numeral St. Thomas/St.John examinees displayed adequate skills in these areas. BEST COPY AVAILABLE 54 USVI examinees from both school districts displayed adequate ability in the following skills: a) selecting the numeral in which the digit in the thousands place has the greatest value b) translating a given Arabic numeral into word form c) recognizing the form of an expression not repre- senting a given number choosing the Roman numeral equivalent to a given Arabic numeral e) marking the hundreds place digit in a given numeral Operations - refers to the manipulation of numbers based on the use of a set of previously defined functions (e.g. addition, subtraction, multiplication, and division). St. Croix district examinees showed strong skills in: a) identifying the sign associated with an operation b) choosing a number sentence describing a given operation displayed weak skills in: a) identifying zero as the additive identity b) indicating that the inverse of a given number sentence is true if the sentence is true St. Thomas/St. John school district examinees displayed adequate skills in all these areas. St. Thomas/St. John examinees showed weak skill in identifying the missing number in a number’_ sentence o8 BEST COPY AVAILABLE 3S illustrating the commutative property of addition. St. Croix school district examinees performed adquately on this skill. USVI student from both school districts displayed adequate skills in: a) identifying the missing terms in a number sentence illustrating the associative property of addition identifying the number of sets of a number which are equal to a set of another number interpreting a number of sentence in terms of a second sentence whose operation is the inverse of the first Geometry and measurement - refers to the concepts which involve shapes, measurement, and logic. USVI examinees in both school districts showed weak skill in: marking. a: row where measures are arranged in order from small to large. St. Thomas/St. John school district examinees displayed weak skill in estimating the length of a line segment while St. Croix examinees showed adequate achievement in this skill. Examinees from both school districts showed adequate skills in: a) identifying the area of the interior region of a figure indicating size relationships between two similar geometric figures indicating a point within the interior of two over- lapping geometric figures. og BEST COPY AVAILABLE ‘wer eee Mathematics Computation Item Analysis Item #2 was very easy (p = .89) for St. Thomas/St. John district examinees and this should account for the low level of discrimination observed (d = .06). Item #25 was a very easy item (p = ,92) for USVI students from both districts, thus accounting for the observed low discrimination index (d = .10). Item #34 was a moderately difficult item for St. Thomas/ St. John district examinees (p = .21). It required students to divide 129 by 3 with the problem presented in standard form. No pattern of incorrect responses could be found and the item seemed clear and unambiguous. The low discrimi- nation index (d =.12) could be accounted for by the diffi- culty of the item or the fact that, being the only item on the test requiring long division, the intercorrelation between it and other items was particularly low for this sample of students. Summary of Student Skills Knowledge of primary facts - USVI examinees from both school districts displayed adequate skills in all areas of knowledge of primary facts tested by this examination. These skills were: a) adding primary facts to obtain sums of twelve or less b) adding primary facts to obtain sums greater than twelve BEST COPY AVAILABLE SAV ae PR a Abe cee WORE RTs bs 57 subtracting primary facts with a minuend less than twenty multiplying primary facts with a product of 25 or less multiplying primary facts with a product greater than 25 f) dividing primary facts with a dividend or less g) dividing primary facts with a dividend greater than 25 Addition and substraction algorithms - This set of skills involves the use of a series of prelearned steps for finding sums and remainders. St. Croix school district showed particularly strong skills in: a) subtracting 2-digit subtrahends from 3-digit minuends, renaming tens b) subtracting 3-digit subrahends from 3-digit minuends, renaming hundreds St. Thomas/St. John district examinees showed adequate skills in these areas. USVI examinees from both school districts showed ade- quate skills in: a) adding a 3-digit numeral and a 2-digit numeral without renaming b) adding two 3-digit numerals, renaming ones Subtracting a 2-digit subtrahend from a 3-digit minuend without renaming 6j BEST COPY AVAILABLE 58 d) adding two 3-digit addends, renaming ones and tens and obtaining a 4-digit sum. Multiplication and division algorithms - This set of skills involves the use of a series of prelearned steps for finding products and quotients. St. Croix school district examinees displayed particu- larly strong skills in multiplying 2-digit by a one-digit multiplier with and without renaming ones. St. Thomas/ St. John district showed adequate ability in these skills. Examinees from both school districts showed adequate Skills in: a) finding the devisor in the horizontal algorthm b) dividing a 3-digit numeral by a 1-digit divisor without renaming, and obtaining a 2-digit quotient. Mathematics Applications Item Analysis All items showed acceptable discrimination indices except the following: Item #1 was a very easy item for USVI examinees in both school districts (p = .90) and this would appear to account for the poor discrimination which was observed. Item #15 was a very difficult item for USVI examinees (p = .11). It required students to find the number of yards in six feet. The question seemed clear and unambiguous and there was no discernable pattern of incorrect responses. BEST COPY AVAILABLE 59 Summary of Student Skills Solution of a one-step problem - These items required students to find the solutions to word problems that could be solved using a single mathematical operation. USVI examinees from both school districts showed weak ability in solving word problems requiring subtracting to compare two quantities. St. Croix district examinees showed weakness in their ability to solve word problems requiring subtraction to find a remainder while St. Thomas/St. John examinees showed adequate achievement in this skill. USVI examinees from both school districts performed adequately in solving word problems which required: a) Subtracting to find "how many are left" b) adding to find a total Analysis and development of a solution design - These items required students to carry out the following series of steps: 1) determine the operations needed to solve a word problem, 2) order the operations, and 3) carry out the operations in the appropriate manner in order to arrive at a correct solution. St. Croix school district examinees displayed adequate skills in solving word problems by: a) using knowledge of 1/2 to determine specific rate b) determining what additional information is needed to solve a given problem BEST COPY AVAILABLE 60 They showed weak skills in solving word problems by: a) adding and subtracting in a two-step sequence b) determining specific rate Cc) multiplying and subtracting in a two-step sequence multiplying to determine the total number in a group e) subtracting and adding in a two-step sequence f) separating relevant and irrelevant data St. Thomas/St. John examinees showed weak abilities in all of the problem solving situations listed above. Measurement and graphs - examinees from both the St. Thomas/St. John and the St. Croix school districts displayed adequate skills in solving word problems which required them to: a) read a tally sheet to quantify a category b) read a tally sheet to determine the category of greatest number c) read a tally sheet to find a total d) find the total cost of three item: a price display They showed weak skills in solving word problems which required them to: a) convert standard units of linear measure b) convert standard units of linear measure and subtract ¢) compare two items in a price display d) add and subtract in a two-step sequence using a price display BEST COPY AVAILABLE Spelling Item Analysis All items on the spelling test functioned appropriately for examinees in both school districts. Summary of Student Skills Phonics - St. Thomas/St. John school district examinees showed strong skills in identifying misspelled words where /ch/ is spelled cs instead of ct. St. Croix district examinees displayed adequate skill on this objective. USVI examinees from both districts showed adequate levels of achievement on all other phonics spelling skills tested. These constituted the ability to identify misspelled words where: a) the final e of a vowel-consonant-e pattern is omitted b) /u/ is spelled o instead of u 6} the vowel from a vowel digraph before r is ommited d) /e/ is spelled e instead of ea e) a consonant letter is unnecessarily doubled f) /k/ is spelled ck instead of k /ch/ is spelled sh /k/ is spelled k instead of ck one letter of a doubled consonant is omitted /f£/ is spelled v instead of f initial wh is spelled w /s/ is spelled se instead of ce 65 BEST COPY AVAILABLE /o/ is spelled ow instead of o /el/ is spelled el instead of le vowel-consonant-e pattern of-ite is substituted for-eight /e/ is spelled ee instead /o/ is spelled oa instead schwa syllable is omitted final silent e is omitted /el/ is spelled le instead of el final e is added to a word ending in a short vowel plus consonant loo/ is spelled on instead of oo final /e/ is spelled y instead of ey /e/ is spelled a instead of e Word building - USVI examinees in both the St. Thomas/ St. John and the St. Croix school districts displayed ade- quate skills in all word building spelling skills. These constituted the ability to identify misspelled words in which: /t/ is spelled -t instead of -ed in forming past tense the final e of a vowel-consonant -e patterns is omitted in forming a compound the final e is omitted when adding -ly the final consonant letter of a cluster is doubled when adding -ed BEST COPY AVAILABLE 63 the final e of a yowel-consonant retained when adding -ly a compound is formed without changing the final consonant letter is doubled -eT the final consonant letter is doubled when -ing the final consonant letter is doubled adding -ed the final e is retained when adding -ing the suffix -16 is misspelled the plural is formed without changing the final y to i -S instead of -es is used to formed the plural of a word ending in sh the suffix -ful is misspelled the final -es instead of -s is usé tense -€s instead of -s is used to form the plural Language Item Analysis Allitems on the language test showed acceptable discri mination indices except the follwoing: Item #9 was a very easy item for St. Thomas/St. John examinees (p = .91) and this would account for the low level of discrimination observed BEST COPY AVAILABLE 64 Item #13 was an easy item for USVI examinees in both school districts (p = .82). The item required students to choose the correct form of a formal name from among the :lernatives, 1) Mrs. finn, 2) Mrs. Finn, 5) mrs. finn, and 4) Mrs Finn. The vast majority of examinees who scored incorrectly chose alternative #4, obviously missing the fact that the period in the honorific was missing. This was true among the upper and lower 27% of the examinees and it is likely that the ability to detect this difference does not correlate highly with the othe skills tested on the examina- tion. The fact that other items testing use of capitaliza- tion convention, but do not use an abbreviated honorific show good discrimination supports the idea that this item was not acutally discriminating between students who did and did not have the concept of capitalization in proper names as purported by the test publishers. Item #16 was a difficult item (p = .25) for USVI students. It required them to finish the sentence, I asked, "Why can't we have , With the alternatives being, 1) school?", 3) school?, and 4) school." Most examinees chose alternatives #3 and #4 and this was equally true in the upper and lower 27% groups. The item seems clear and unambiguous. Most examinees were aware that the sentence was either a question or a quotation, but few realized that it was both. Item #33 was a difficult item for USVI students (p It actually discriminated negatively for St. Croix school district students (d = -37). Examinees where asked to choose 68 BEST COPY AVAILABLE 65 an appropriate completion to the sentence, "I was glat that Mrs. Finn wrong." The alternatiyes were, 1) couldn't find nothing, 2) didn't find nothing, 3) couldn't find anything, and 4) couldn't never find anything. Analyis of item responses showed that students in the upper 27% of total language scores tended to choose alternative #1. These use of the double negative may be a dialectical artifact Since the phenomenon did not show itself on any of the other items which tested the ability avoid double negatives and examinees scored well on these other items (p = .73 and > @ 38). Summary of Student Skills Capitalization conventions - These are skills which involve examinees' abiPity to recognize appropriate uses of Capitalization of words. St. Croix school district examinees showed particularly Strong skill in their ability to identify the correct capi- talization convention for holidays. St. Thomas/St. John examinees displayed adequate skill in this area. USVI examinees from both school districts showed weak Skill in identifying the capitalization convention for common nouns. St. Thomas/St. John school district examinees showed weak skills in identifying the correct Capitalization con- ventions for: a ) Cities and states b) the closing of a letter 69 BEST COPY AVAILABLE 66 St. Croix district students displayed adequate skills in this area, USVI examinees from both school districts showed adequate skills in identifying the correct capitalization conventions for a) street addresses b) the salutation of a letter Cc) a proper name d) the beginning of a sentence e) the days of the week f) titles g) the name of a school Punctuation conventions - These skills involve the ability to recognize the appropriate use of punctuation in a given sentence. St. Thomas/St. John school district examinees showed weak skills in identifying the correct punctuation convention of using: a) a comma between city and state b) a comma after the closing of a letter St. Croix district examinees displayed adequate skills in this area. USVI examinees in both school districts showed adequate skills in identifying correct punctuation conventions in using a) no punctuation after the street address in the heading of a letter BEST COPY AVAILABLE 67 comma between the day of the month and year comma after the salutation in a friendly letter period to end a declarative sentence question mark to end an interrogative sentence a period after an abbreviation of title a question mark to end an interrogative sentence enclosed in quotation marks commas to separate items in a series a comma after a name used in direct address an apostrophe in a contraction k) a comma after an introductory "yes" Usage conventions - These refer to skills involving the ability to choose appropriate forms of words depending on the content of the sentence in which they are used. USVI examinees from both the St. Thomas/St. John and the St. Croix districts showed adequate skills in the following usage Situations: a) indefinite article b) where there is a pronoun form in compound subject and identifying the word order for a proper noun and a pronoun using using using using the past tense of the word a negative form to avoid a the past tense of the verb the past tense of the verb "to know" double negative "to bring" “to tell” Showed weak skills in the following usage areas: using the present tense of the verb "'to hurt" 71 68 using the past-perfect tense of the verb ''to come" using the past tense of the verb "to teach'"' using the past tense of the verb "to give" using the past tense of the verb "to run" using a pronoun form in a compound subject and a form of the verb ''to be" using the present tense of the verb "to ask" uSing a negative form of the verb "to be" using the present perfect tense of the verb "to grow" j) using the present tense of the verb "to become" Language sensitivity : word sensitivity - These skills relate to the ability of examinees to perceive appropriate usage of words in a particular content. USVI examinees from both school districts displayed adequate skills in their ability to complete a sentence using" a) gerunds b) a form of the word "to float" d) a verb e) a pronoun to agree with an antecedent They showed weak skills in their abilities to complete a sentence using: a) an antecedent for a given pronoun b) a verb form St. Thomas/St. John school district examinees showed weak abilities in completing sentences using: BEST COPY AVAILABLE a) a conjunction b) an adverb c) a verb and verb form St. Croix district examinees showed adequate skills in these areas. Language sensitivity : sentence sensitivity - St. Thomas/ St. John examinees displayed weak skills in choosing phrases which complete sentence fragments and recognizing complete sentences. St. Croix district examinees showed adequate achievement in both of the skills. BEST COPY AVAILABLE 70 Grade 6 - Intermediate II Level The Intermediate II Level of the Stanford Achievement Test consists of eight tests with the number of items listed below: 1. Vocabulary 45 items Reading Comprehension 71 items Word Study Skills 50 items Mathematics Concept: 35 items Mathematics Computation 45 items Mathematics Applications 40 items Spelling 60 items 8. Language - 80 items The means, standard deviations, and reliability esti- mates for these tests, determined previously (see Bliss, 1982a), are presented below. Mean Standard KR- 20 tandard Raw Score Deviation Reliability Error Total USVI Sample Vocabulary ; 7.8 .85 Reading Comprehension , 12.4 Word Study Skills Mathematics Concepts Mathematics Computa- tion Mathematics Applica- tions Spelling Language BEST COPY AVAILABLE Thomas/St. John Sample Vocabulary ° 8.1 . 86 Reading Comprehension , Leet Word Study Skilis Mathematics Concepts Mathematics Computa- tion Mathematics Applica- tions Spelling Language Croix Sample Vocabulary ‘ ‘ .80 Reading Comprehension ‘ ‘ -91 Word Study Skills , ‘ 92 Mathematics Concepts ‘ ‘ mre Mathematics Computa- tion : , .78 - 94 Mathematics Applica- tions bs % J .89 .60 Spelling P : .93 .30 Language ‘ Let .87 -14 Following are the item analyses and summaries of student achievement on the objectives evaluated by each of the tests: Vocabulary Item Analysis All items showed acceptable discrimination indices with the exception of the following: 25 BEST COPY AVAILABLE 72 Item #37 - This was a particularly difficult item for students in the USVI sample (p = .19) and this accounts for the low level of dicrimination observed on it. The item required students to recognize that a ?bolt? is a unit used to measure cloth. A large number of those answering incor- rectly chose the alternative "tile." Item #40 - This was also a difficult item for the local sample of students (p = .16). It required students to indi- cate that a person who has little respect for another looks at that person with "scorn." A large number of examinees responding incorrectly chose "a scowl,'' and there is some question of the item being ambiguous since a case could be made for this answer. Item #41 - This item was also extremely difficult for the USVI examinees (p = .11). It asked students to indicate that the perfect person for a job was “ideal."" There seemed to be no recognizable patterns of incorrect responses. Item #43 - This discriminated satisfactorily among St. Thomas/St. John district examinees, but poorly among examinees in the St. Croix district. The latter group found the item more difficult than the former (p = .21 vs 0 = .31) and the same number of examinees in the upper and lower scor- ing groups obtained correct scores in the St. Croix sample. The item required examinees to know that hot metal is shaped on “an anvil." Incorrect responses were primarily "a chisel" a lathe."' The item appears to be functioning properly with the low discrimination index being the result of the small number of students who knew what an anvil was. 76 BEST COPY AVAILABLE 73 Item #44 - This proved to be a difficult question for the local sample of examinees (p = .7). The item asked examinees to identify "endure" as meaning the undergoing of an experience for a long period of time. A large proportion of those answering incorrectly chose "undertake" as the cor- rect answer. This was probably the result of the use of the recognition of the prefix "under." In short, the distractor seemed to work well. Item #45 - This was an extremely difficult item for USVI students (p = -08). It required students to know that people seek a shallow portion of a river in order to "ford" it. Students overwhelmingly chose "swim" and "drift" and ignored the distractor "shoot." It seems clear that Virgin Islands students, inexperienced with rivers and streams, were not aware of this term, but could associate "swim" and "drift" with water in the form that they have experienced it (i.e. ocean). The item seems to be operating as it should. Item #47 - This was a difficult item for the USVI sample of examinees (p = .18). It required examinees to indicate that the term "processing" referred to an ongoing situation. There appeared to be no pattern to the incorrect responses. Item #50 - This item required examinees to indicate that "secede" is a term used to indicate the leaving of a State from the union of states. It was very unfamiliar to the USVI sample of students (p = .19) and did not seem to discriminate at all among examinees (d = .07). There seemed to be no pattern of incorrect responses. It could be that BEST COPY AVAILABLE 77 74 this term (most commonly encountered) in Studies of the U.S. Civil War was familiar to examinees since this period of history is not emphasized in USVI elementary schools owing to the fact that the Danish West Indies were not affected by the War. Summary of Student Skills In general, USVI students tended to be weak in the area of standard English vocabulary recognition. This weakness was especially pronounced in the areas of words encountered in the areas of social science and arts and crafts and less so in those of math and science. In addition weaknesses were apparent in the areas of vocabulary found in general litera ture and in references. Of the fifty items on the test, a significantly smaller proportion of USVI students were able to respond correctly when compared to the standardization group. Beyong the question of statistical significance, the proportions of USVI students who successfully answered many items was substantially lower than the standardization group. The differences in item mean difficulty values are a clear indication of this problem. Most items seem to function well with poor discrimination values of some being the result of extremely low difficulty indices. BEST COPY AVAILABLE Reading Comprehension Item Analysis Technical Report #2 indicated that students in grades 8, 10, and 12 had difficulty completing the Reading Comprehension test. This same phenomenon was observed for grade 6 examinees. Table I presents the number of examinees who omitted each item on the test. It should be remembered that since number right scoring precedures were used, an omitted item is counted as an incorrect response and that this process tends to de- flate the difficulty indices and individual student scores. Because of this situation items which less than 2/3 of the examinees were able to attempt were omitted from the analysis. Technical Report #2 (Bliss, 1982b, pp. 96-97) attempts to speculate as the causes of this phenomenon. All items appeared to have satisfactory discrimination indices with the exception of item #9 which appeared to be very easy (p = .95) for the St. Croix group of examinees. St. Thomas/St. John also found it easy (p = .93 an d : . The low levels of discrimination was due to the simplicity of the item (which required examinees to identify meaning inferred in a passage). Summary of Student Skills Restricting the analyis to items 1] to 47, the following conclusions can be drawn. The mean difficulty index for items testing global or eneral meaning of a portion of reading material was p = “yg BEST COPY AVAILABLE 76 Proportions of the Six } port xth Grade Sample Omitting Reading Comprehension Test Items (Percents) ITEM # USVI STT/STJ 0 l l 0) BEST COPY AVAILABLE continued BEST COPY AVAILABLE continued BEST COPY AVAILABLE 79 for each of the samples in the two school district as well as for the whole USVI sample. The standardization group mean was p = .56. USVI students sampled seem to have adequate skills in the ability to determine global meaning from written material. The mean difficulty index for items testing the ability of students to determine meaning explicity written in textual material was .66 for the total USVI sample, .67 for the St. Thomas/St. John district sample and .61 for the St. Croix district sample. For the continental U.S. standardization group, the mean difficulty index was .68. USVI students appear to exhibit adequate skills in their ability to deter- mine explicitly stated meaning from text. The mean difficulty index for items testing the ability of students to determine meanings that are implicit written presentations was .65 for the entire USVI sample and for the samples from each of the individual districts. For the standardization group the mean was .65. USVI students appear to have adequate skills in the area of. determining implicit meaning. The mean difficulty index for items testing students’ abilities to determine meanings inferred in reading selections was .49 for the entire USVI sample and for the samples from each of the individual districts. For the standardization group the mean was .58. USVI students appear to be somewhat weak in the area of determining inferential meaning. BEST COPY AVAILABLE Word Study Skills Item Analysis All items appeared to have satisfactory discrimination indices and appear to be functioning satisfactorily. Summary of Student Skills Consonant sounds- Students in both the St. Thomas/ St. John and the St. Croix school district had difficulty recognizing that the grapheme dg as in “edge"' was sounded /j/. St.Croix students seemed weak in recognizing the following consonant sound graphemes. a) g pronounced /g/ as in "guide: b) c pronounced /k/ as in "scout" c) b prounouced /b/ as in "ribbon" ch prounounced /ch/ as in "cheese" ough pronounced /f/ as in “enough" sh pronounced /sh/ as in "shout" Croix district students appear to be very weak in the area of discrimination of consonant sounds with a mean difficulty index of .35 compared with .5t for the students in the St. Thomas/St. Johm district and .50 for the standard- ization group. Short and long vowel sounds- USVI sixth grade students, in general, appear to have difficulty recognizing the graphemes for the following regular vowel sounds: a) short vowel sound /e/ as in ‘'feather' b) short vowel sound /o/ as in "boner" S 4 BEST COPY AVAILABLE short vowel sound /u/ as in "must" short vowel sound /i/ as in "trip" Croix district students also have difficulty recognizing the graphemes for the following vowel sounds: a) long vowel sound /e/ as in "tea" and "only" b) long vowel sound /o/ as in "home" It appears that St. Thomas/St. John students only have difficulty identifying graphemes for the short vowel sounds while being fairly proficient in recognizing those long vowel sounds. St. Croix district students seem to have difficulty with both long and short vowel sounds. Variant vowel sounds- these refer to vowel sounds that cannot be classified as “long" or "short" and include such things as diphthongs and the "schwa" sound. USVI students appear to have difficulty in recognizing graphemes for all of the variant vowel sounds tested. These were: a) the dipthong /oi/ b) the schwa sound c) the vowel sound /o/ as in "order" and "jaw" d) the diphthong /ou/ e) the vowel sound /u/ as in "fur" ~ The particular difficulty the local students appear to have with these variant vowel sounds is also demonstrated mean difficulty indices for these items which was .54 for the USVI examinees and .65 for the standardization example. BEST COPY AVAILABLE 82 Of particular note is the pattern of incorrect responses by examinees on items which required them to recognize graphemes for vowel sounds. More often than not, examinees scoring incorrectly on items chose distractors having the same letters in approximately the same positions as the criterion word. For example, when asked to identify the word having the same sound as the "dge" in the word "edge" from among the choices 1) league 2) jelly 3) goods 4)grades, most examinees scoring incorrectly choose 1) rather than 2). Of course, the item, and items 11ike it, may be functioning well and these may simply be cases of good distractors (after all, the items did discriminate satis- factorily). The directions are veryclear and concise. However, the sample item used to illustrate the process requires students to choose the word that has the same sound as the "i" in did from among 1)bud 2)hid 3)by 4)bat, with the correct answer being 2). This example is one where the correct answer does have the same letter in the same position as the criterion and this may have been confusing to some sixth graders. These items need to be examined more closely to determine what is operating, here. The rest of the items on the Word Study Skills test required students to identify extra syllables in words presented under various language conventions. For example, 1)re 2)en 3)mem 4)ber. In this case, the answer is 2). The syllable "en" is not required to produce the word "remember." Affixes - In these items the examinee 1s required to identify the extra syllable when part of a word stem and its affix are 56 BEST COPY AVAILABLE are separated. For example: ljel 2)er 3)trav 4)tar. The correct answer is 4) since the word is “traveler”. USVI students showed strong skills in this area with a mean probability of answering correctly of .60 in the st. Thomas/St. John school district and .61 in the St. Croix district. The mean difficulty index for the standardization group was .54. Syllables (VC-CV)- In these items students were required to identify the extra syllable when part of a word was divided according to the Vowel/Consonant-Consonant/Vowel rule. For example: l)ence 2)dif 3)fer 4)i. Here, the extra syllable is "i" (choice#4). USVI students showed strong abilities in this skill with mean difficulty indices over these items of _68 in the St. Thomas/St. John district and .74 in the St. Croix district. The mean difficulty index for the standardization sample was .59. Syllables- (VC-V or V-CV)- In these items students were required to identify the extra syllable when part of a word was divided according to the Vowel/Consonant-Vowel or Vowel- Consonant/Vowel rule. For example: l)er 2)ly 3)gov 4)nor. Here, the extra syllable is "ly" (choice #2) , USVI examinees showed strong abilities in this skill with mean difficulty indices of .63 in the St. Thomas/St. John school district and .65 in the St. Croix district. The mean index of difficulty for the standardization sample was .53. ? - . ° ; x ‘ a Accent- In these items examinees were required to identify the extra syllable when part of a word was divided according to BEST COPY AVAILABLE 84 accent rules. USVI students showed strong abilities in this Skill with mean difficulty indices of .60 in the St. Thomas/ St. John district in the St. Croix district. The mean difficulty index for the standardization was .54. Math Concepts item Analysis All items appeared to discriminate satisfactorily except the following: #4 showed lowed discrimination for students in the John district. This was a moderately difficult item for these students which asked them to choose the The distribution of responses showed of a student in the upper 27% of total €s was virtually the same as the probability of as that student in the lower 27% scoring correctly (.77 vs.64). appeared ) be no recognizable pattern of incorrect scores. >m #6 showed poor discrimination in the St. Croix school This moderately difficult item (p=.51) required vert numbers from x] lto pli value notation. obability of an examinee in the lower group g the correct answer.was approximately the same as one there was a pattern in xaminees who answered the incorrectly iled to us >ro as a place holder in the hundreds place when the expanded number w: (nx1000) + (k x 10 - i = ae , ; : (] x 1). They either chose a three place number in the form 88 BEST COPY AVAILABLE seh k + fs that the Skill tested here is not Other math concepts being could account for the low Item #23 showed poor St. Thomas/St. 1tem the poor discrimination. incorrect casts some light on what item consisted of a a = .7 parallelogram inside the grid . Or placed the zero in the wrong position. John district. for these examinees (p=.22) 85 It may be intercorrelated with the tested to a high degree and this level of discrimination observed. discrimination for €xaminees in This was a very difficult and this could account for The pattern of responses which were was happening here. The rectangular grid” with a shaded The question called for the Student to choose the number of "square units" which were Shaded and described the number of grid or €ither ignored the word aware the sum of those Partially "unit" the distractors Chosen most often were those that Squares either totally shaded Or totally shaded. Students in the question or were un- of the meaning of the term "square unit." Summary of Student Skills Ent OKIlisS Number- This With (@.@. Students the fractional part particularly portion Sample of students in a item grouping refers the relationships between different types primes, odds ys evens, Showed particular strength in being able of an area which is weak however in the ability #5 a percent of the total area. Showed weak Skills in the to concepts which of numbers fractions vs wholes). USV] to determine Shaded. They were to express this shaded In addition, the local ability to subtract finite system uSing a clock module. Sy BEST COPY AVAILABLE 86 Croix school district examinees were weak in t} ability to select a number line graph of a defined set of lo numbers. St. Thomas/St. John district examinees appeared to have adequate skills in this area. In addition, examinees in the USVI sample displayed adequate ‘cills in the following concepts: choosing the number belonging in a given numbe! selecting a set consisting only of odd numbers selecting the number line graph of a defined set of numbers. identifying prime numbers. determining length on a number line. Notation- refers to the technique by which numbers are expressed in written form. Some forms include expanded notation, differing base values, decimals, and fractions. USVI students show weak skills in converting from exponential to standard notation. St. Thomas/St. John school district students show particularly strong abilities in translating numbers in word form into Arabic numbers and in rounding off whole numb¢ . St. Croix district examinees display great weakness in e former of these two St. Croix distric s also show weakness in their ability to convert from expanded to numeral form. St. Thomas/ St. John district s jents display adequ: ility in this skill. Notation s to perform adequately are: a) identification of place value in a numeral aa BEST COPY AVAILABLE 87 identification of digits in a period of a numeral identification of a number that is 100 greater than another number d) estimations of products to the nearest hundreth Operations- refers to the manipulation of numbers based on the use of a set of previously defined functions (d.g. addition, subtraction, multiplication). The St. Croix sample examinees showed weakness in their ability to identify division as the inverse of multiplication while examinees in the St.Thomas/ St. John district showed adequate ability in this area. The entire USVI sample of examinees were very weak in their ability to choose an example illustrating that division is distributed over addition. The local sample displayed adequate ability in all other operations skills sampled by the test. Geometry §& Measurement- refers to the concepts which involve shapes, measurement, and logic. Students in the St. Thomas/ St. John district showed strong ability in the skill of estimating the distance between two points using metric units. St. Croix district students performed adequately in this skill. The USVI sample, in general, showed weaknesses 1n the ability to determine the measurement of a shaded region and to identify two disjoint sets. St. Croix school district students displayed weaknesses on the following concepts: a) Identification of the missing number in and ordered pair. b) choosing a figure having a right angle c) solving a logical problem using Venn diagrams. hy BEST COPY AVAILABLE St. Thomas/St. John district students displayed adequate achievement in these skills. The entire USVI sample showed adequate achievement in the following areas: a) identification of the number of the empty sét distinguishing quadrilaterals from other geometric shapes solving elementary logic problems choosing similar triangles identifying line segments recognizing description of line segments Mathematics Computations Item Analysis Item #16- This item showed poor discrimination with the St. Croix sample (d=.05). While students in both districts found the item to be approximately equal in difficulty (p=.32 in the St. Thomas/St. John district vs p=.34 in the St. Croix district), it was found that examinees in the upper 27 $% of Croix district sample were less likely to score correctly than those in the St. Thomas/St. John district (41% vs In fact the upper and lower portions of the St. Croix district were almost equally divergent in their probabilities of scoring correctly on the item (41% vs 36%). There seems to be the same discrimination between the two upper groups ls between the upper and lower St. Croix district groups. The item requires examinees to examine the sum of two fractions in the 1/N + 1/N+1 and to determine whether IZ BEST COPY AVAILABLE this sum is greater than, equal to, or less than one. The majority of the students scoring incorrectly chose "greater than" as the correct answer in both districts. The item seems Clear and unambiguous, and it is very likely that this poor discrimination is simply a chance event. Item #17- this was a very difficult item for the local Sample of students (p=.17).- It required examinees to compare two fractions in the form n/m and n/m-1 and to determine whether the first was less than, equal to, or greater than the second. Most students who scored incorrectly chose "less than"as the correct response. Summary of Student Skills Knowledge of primary facts and solution of Simple, mathematical sentences. Students from both school districts displayed adequate skills on items testing the following skills: deriving two addition facts deriving two subtraction facts deriving two multiplication facts multiplying within parentheses, then adding relating two products with a common factor and an unequal second factor deriving division facts with and without remainders multiplying a three di git factor with a medial by a one digit factor ST. Thomas/St. John examinees Showed weak performance on the following skills while St. Croix examinees displayed adequate achievement on them: BEST COPY AVAILABLE 90 comparing two fractional representations of a whole number. b) comparing the value of a decimal fraction with a common fraction. St. Croix district examinecs showed weak performance on the following skills while St. Thomas/St. John examinees displayed adequate achievement on them: a) recognizing a numeral distributed into two factors b) simplifying an expression within parenthese before performing other operations. Examinees in both districts showed weak achievement in the following skills: a) adding two unit fractions with different denominators b) comparing values of two common fractions with equal numerators. c) naming improper fractions as "mixed numbers. Addition and subtraction algorithms-this set of skilis involves the use of a series of prelearned steps for finding sums and remainders. Examinees from both districts showed at least adequate skills on all objectives tested in this area. Strong skil were demonstrated on the following objectives: a) subtracting a three digit subtrahend from a 4 digit minuend renaming tens and hundreds with a disappearing left digit. b) adding a broken column of four addends with renaming. Examinees from both stricts displayed adequate skills in the following areas: a) c finding the value of a sentence placeholder by subtraction b) Saga the minuend in a horizontal algorithm when the subtrahend and the remainder are known 94 BEST COPY AVAILABLE 9] c) subtracting a four digit subtrahend from a fj minuend, with zero in the minuend and renaming in all positions Multiplication and division algorithms-this set of skill: involves the use of a series of prelearned steps for finding products and quotients. Examinees in both districts displayed at least adequate on all the objectives tested in this area. They showed ularly strong skills in the following areas: multiplying a two digit numeral by a one digit nultiplier, renaming dividing a three digit numeral by a one digit divisor, renaming tens multiplying a three digit numeral by a two digit multiplier with zero in the multipliciand, renaming ones and tens Thomas/St. John school district examinee: strong skills on the following objecti\ rict examinees showed adequat iplying’two three digit numerals, with > nes position of the multiplier and renami lying a three digit numeral] ler with renaming examinees showed pa dividing a Thoma thi: reas estimating the value and divisor are known dividing a four it ral by a one digit divisor! obtaining a quot t ‘ro in the tens place dividing a two digit numeral by a one digit divisor! obtaining a quotient with ; remainder ommon fractions Cri lis ict examinees showed trong al the least common denominator inators. St. Thomas/St.John in this area. in adding two frac then one. They raction of a multiple BEST COPY AVAILABLE was a of examinee the low lev Summary of Student Skill the ability of examine required to solve a USVI examinees showed adequate abi operation for finding the total number of objec’ in different measures of quantity. They showed weak in their ability to choose the appropriate operation for finding: a) a number of equal units by dividing b) "how many"' of a unit number by dividing St. Croix distri Students displayed weak skills 3 their abilities to determine the appropriate operation for finding. a) specific rate by division b) finding the remainder by subtraction c) the rate per unit of time by multiplication. BEST COPY AVAILABLE a ems into mathematical sentences determining correct change by adding and subtracting John school di showed weak information cist ct examinees examine Th YOmac iNnomas, following objective , +4 mul L rounding humber of items BEST COPY AVAILABLE 95 5 lying and subtracting in cale Problems- These items figure xaminees displayed weak skills ic rate based on an average while examinees showed adequate achievement in in both school districts showed weak achievement following objectives: determining specific rate inding a unit fraction of a number finding ® Of a given amount of money finding a total when the part of its fractional representation of the whole is known interpreting a map scale to find relative distances f a total as a fraction Vv ieasurement- in both school districts showed achievement on all the objectives tested in this They were: a) adding times expressed in hours and minutes converting standard units measure finding the area of converting standard unit: f liquid m re Subtracting two measures and converting into another unit of measure BEST COPY AVAILABLE 96 Graph Reading and Interpretation Examinees in both school districts displayed adequate achievement in using bar graphs to make comparisons. Croix school district examinees showed weak achieve- ment in the following areas: finding the difference between two dates in a table uSing a bar graph to determine specific information Thomas/St. John school district examinees showed adequate achievement in the above two areas. Examinees in both districts showed weak ability in using bar graphs to combine frequencies. Spelling Item Analysis All items on the spelling test functioned appropriately for examinees in both school districts. Summary of Student Skills Homophones- These are words which sound the same, but have different meanings and are spelled differently. On these items students had to determine the appropriate form of the homophone from content and identify its correct spelling. Examinees in both school districts showed adequate skills in distinguishing between the following homophones: a) modal would instead of noun wood noun mind instead of verb mined verb heard instead of noun herd noun tide instead of verb tied adjective plain instead of noun plane Lu BEST COPY AVAILABLE Croix district examinees showed weak abilities to guish between the following homophones: verb seems instead of noun seams noun tier instead of verb tear Thomas/St. John district examinees exhibited adequate yn these objectives. idents in both districts were weak in distinguishing the verb sealing from the noun ceiling. Phonics- These objectives dealt with spelling skills function of the examinees' ability to use phonetic from both school districts showed particularly Strong sk in identifying misspelled words where /e/ is spelled e instead of ea. st. Thomas/St. John district examinees showed particularly high ability in identifying misspelled words where /en/ is spelled en instead of eon. St. Croix examinees displayed average ability on this skill. St. Croix district examinees showed particularly high ability in identifying misspelled words where a) a syllable is omitted /en/ is spelled en instead of on one letter of a double consonant is omitted /el/ is spelled al instead of le St. Thomas/St. John students showed adequate abilities in these skills. Examinees in both districts showed adequate ability in oh BEST COPY AVAILABLE words where “Ollowing + instead of c instead of ce / is spelled xs instead of x is omitted in ( m (p) xY cluster consonant letter is doubled unnecessarily showed weak ability to Thomas/St. John examinees words when a final /n/ is spelled in Yaar a & a misspe Croix examinees showed adequate abi John examinees demonst1 ability in identifying misspelled words ing particularly) is doubled when adding- in which final conscnant morpherem /en/ BEST COPY AVAILABLE Languas [ten rict examinees demonstrated particularly identifying misspelled words in which omitted when adding -s to form the plural the final 1 is omitted when adding the formed without changing f ee added without doubling the final consonant John examinees showed adequate performance districts showed adequate skills ing misspelled words in which is changed to i when adding -ed a consonant letter is deleted from a prefix assimilation is retained when adding -ing 1x is added without Changing ye to i morpheme /er/ is spelled er instead of or is formed without changing all to al added without doubling the final consonant letter confused with -ary uffix is added without doubling final consonant plural is formed without changing y to i -€d is added without doubling final consonant letter + hy the suffix -ours is spelled -us items on the Language test descriminated Satisfactorily tTOllowing. Item #2 was a moderately easy item for 103 BEST COPY AVAILABLE 100 St. Croix students (p=.79) which ré nize that a comma needed to be placed between the day of a month and the year. Examinees in the lower 27% of the distribution were almost as likely as those in the upper 27% to score correctly on this item (.68 vs. 77). There be no pattern of responses and it is likely correlate highly with those measured on other parts of the tes #3 §& 4 were particularly difficult for . Croix (p=.14 and .19, respectively) and thi account for the low levels of discrimination observed. These items dealt with the use of commas to set off parenthetical a particularly difficult one for St. Croix Students 9p=.30). Only three more students in the upper group scored correctly on this item than in the lower group. It dealt with the placement of a question mark at the end of a quotation. Item #10 was a difficult item for USVI students (p=.34). Students in the upper 27% of the distribution of scores were only slightly more likely to score correctly than those in the lower 27% (p=.39 vs .34). This item dealt with the formation of past participles from infinitive verb forms. Item #11 was a moderately easy item (p=.78) for St. Thomas/St. examinees. Examinees in the lower group were not much less likely to score incorrectly on this item BEST COPY AVAILABLE 101 : ; ; ; ; > : thocea ; thea wnhner ye AIT Tt +MArTD red > > | LIIUSC in Lil€ upperY TTOUT e .s invoivyed the Cnoosin rt i the correct present perfect tense of the verb "to choose. e . ~ * th « - > ec ott . -} . no » | , rc } The distractor "have chose" was chosen almost always by - : i. 7 cr } ; “Arrea lv The me > } €xaminees who scored incorrectly. The item seems to be Operating properly, but it does not < rrelate highly with mOSt other items on the test uo a + ~ ~ = ina an a. Item #21 was a very diffic t em tor St.Croix school ' Gistrict examinees (p=.19). This accounted for the low eae pena a eee i ies oye ee ee ee Seren a =e a ae ai a ee - ? discrimination index observed for this item which test students knowledge of the use of quotation marks to set off direct quotes. — cr 14) = t r t = eb) fi a moderately difficult item for St. Croix Students (p=.43), but examinees in the upper and lower 27% of the distribution were equally likely (p=.50) to score correctly on this item which tested to see if students knew how to deal with indirect quotations. a ul tf Item #: owed poor discrimination for St. Croix district Students (d=.14) The item response pattern suggests a reason for this. The item purports to measure whether student ¢ v4) ie) 8) 4 =) Pant +r tha me ih he ae -43 9 +c : . — - 4 ~ , Cc £ identify the appropriate use of the present perfect tense of the verb "to become."' The item reads: "With power in most U.S. towns, the electric Clock became very common." The choices Presented were: 1) became, 2) has becomed, 3) has became, and 4) has become with alternative #4 scored as the correct cnoice. A large number of the upper 27% of examinees who scored incorrectly Chose alternative #1. Alternative produces a gramatically correct sentence 105 BEST COPY AVAILABLE " 1 ¢pn ha . - c+ c nce nrec Iino ++eme jeal With the 5 tory Saic LC D Correct inmee. t SCeCEGIALSC Ltems ae<¢ A1T tne Nistor of electricity. Most interesting is the fact that item #33 appears at the head of a left hand page in the test booklet ; . } } - os - sc + SC a ] h- -} nd students must remember the tlow of questions or look back the previous page. Nowhere in the directions are examinees told about relationships between items. In summary, this he poor discrimination which was observed. item #40 was a very difficult (p=.23) item for St. Croix ict examinees. This would account for the low level of Gast discrimination that was observed. Item #42 was a very difficult item for USVI students (p=.14). The item examined students' ability to choose an r P appropriate verb to complete the sentence, " The loudspeaker his voice into the crowd."' The choices were 1)sputtered, Fs) 12) = 4 ~ 4) pail a 49) C . tl — 7) tressed, 4) boomed with alternative #4 scored correctly. The majority of examinees chose alternative #3. =) ro iNilS accounts for the low level of discrimination observed. snere may be something in the general language use of USVI th a : ce 3 } 2 mee . . COuld explain these responses. c a4 99) summary of Student Skills Capitalization conventions-These are skills which involve examinees' ability to recogni ze appropriate uses of capitali- zation of words. cr ™ . ae i an ae ; = » 2 ° : ot. Croix district students showed weak skills in recognizing : or 3eST COPY AVAILABLE 10Nn conventi appropri “mance a ques senten ho punctua Quotations commas geqQ uc poor skills in being able to select the present perfect tense of the verb "'to become" from content. St. Croix district Cxaminees showed adequate ability in this skill. St. Croix school district examinees showed poor skills the following usuage areas: a) selecting the past participles of the verbs "to Know" and "'to come" from content. b) selecting the past tense of the verbs "to know" and "to begin" from content. Thomas/St. John school district examinees scored adequately these areas. USVI students all performed adequately in the following uSage areas; a) selecting the present perfect form of the verb "to take" from content. b) selecting the negative forms of a sentence avoiding double negatives selecting the past participles of the verbs "to break" and "to eat" from content selecting the past participle of the verb "to begin" from content selecting the past perfect tense of the verb "to drink" in content selecting the past participles of the verbs "to freeze" and "to go" in content selecting the appropriate reflexive pronoun in content selecting a verb to agree with a subject in number and to avoid a double negative Selecting a pronoun form in a compound subject and identifying the proper word order for a proper name and pronoun BEST COPY AVAILABLE selecting the present perfect tense of the yerb "to write" from content selecting the past participle of the verb "to take" and the past tense of the verb "to choose" in content selecting the passive voice form of the word "to drive'' from content selecting the appropriate verb to agree with a Subject in number and the appropriate comparative adjective to modify a plural noun Language sensitivity: Word sensitivity- These skills relate to the ability of examinees to perceive appropraite usage of words in a particular context. USVI examinees showed adequate skills in the ability to complete a sentence by selecting an appropriate clause. They Showed weak skills in the ability to complete a sentence by selecting an appropriate independent clause. St. Croix district examinees showed strong skills in the ability to complete a sentence by selecting an appropriate verb phrase. St. Thomas/St. John examinees showed weak skills in this area. St. Thomas/St. John school district examinees showed abilities to complete sentences by selecting an appropriate: a) preposition b) posessive pronoun to begin with an antecedent Cc) verb d) predicate adjective Language sensitivity: Sentences- USVI examinees demonstrated adequate skills in the following areas: 110 BEST COPY AVAILABLE 107 a) choosing the phrase which completes a sentence fragment b) recognizing a complete sentence c) recognizing that a group of words forms a single . “- complete sentence d) recognizing that a group of words forms two or more sentences €) recognizing that a group of words do not form a complete sentence Dictionary skills- These consist of the abilities needed ’ to locate words in a dictionary and to use the dictionary entry to find needed information. USVI examinees displayed adequate ability in using sample dictionary entrees to: a) determine the appropriate meaning of a word in content b) identify the part of speech of a word in content c) determine syllabication d) determine the pronounciation of a word and choose a word that rhymes with it e) indicate the appropriate content for a particular meaning of a word f) identify guide words using alphabetical order USV] examinees showed weak skill in their ability to use a sample dictionary entry to determine the pronounciation of a word. 11] BEST COPY AVAILABLE Discussion A detailed summary of overall strengths and weaknesses will be presented in a final report on conclusions and recommendations. However, at this time a number of points Can be made concerning test performance and academic achieve- ment of USVI examinees across grades. The problem of examinees having insufficient time to Complete the reading test is still evident in grade six examinees. The fact that it ceases to exist in grade four and two sheds some light on the nature of the causes of this phenomenon. In the grade four test (Primary Level III) only the second half of the test involves items where students were required to read lenghthy passages and answer questions based on them, while the second grade test (Primary Level I) has no such items. this seems to reinforce the idea that local examinees may either require more time to complete these items because they are slower readers Or must refer back to the passages more often to answer questions, or that they tire €asily given these type of questions and do not finish the test. Unlike the situation with secondary students it was noticed that when the analysis was restricted to items which two-thirds or more of the examinees, USVI students performed well on higher leve}] reading objectives such as globai and implicit meaning, although they still were weak in de- termining inferential meaning. BEST COPY AVAILABLE Secondary examinees showed difficulty in all reading areas besides explicit meaning. As was the ease with secondary examinees, examinees in grades four and six show weak skills in the area of Standard English vocabulary. In grade two, however, St. Thomas/St.John district examinees showed adequate skills in recognizing vocabulary that would be found in subject matter material in school, while St. Croix examinees were weak in these vocabulary areas. Examinees from both districts showed weakness in general Standard English vocabulary. The fact that at least part of the local sample of students showed adequate vocabulary achieve- ment in subject matter areas may be the function of the emphasis placed on these areas in schooling. It Suggests a need to explicitly teach all areas of vocabulary. Grade two examinees seem to show adequate skill in the use of phonetic skills for word building. This skill seems to deteriorate in grades four and six with grade four students having difficulty discriminations among certain vowel sounds and grade six examinees Showing difficulty discriminating among consonant sounds, as well. This brings.to question the effectiveness of the instructional reading program in the intermediate grades and the Opportunity examinees have to practice and have these phonics skills reinforced. In mathematics, the pattern seen with secondary students (see Poliss, 1982b) is evident with elementary students, as well. Students seem to have adequate computational skills 113 BEST COPY AVAILABLE 110 and seem to have good grasp of basic number facts and algorithms. They have moderate skills in dealing with one Step word problems, but are very weak in dealing with operations and word problems which require more than one Step in order to arrive at solutions. USVI examinees also seem to have weak Skills in seeing relation- Ships between numbers and geometric shapes and this phenomenon 1s obvious across all grades, Local examinees display excellent spelling skills and this phenomenon is also seen across al] grades. The pattern that was observed by Poliss (1982b, pp97-98) where secondary students seemed to achieve higher on objectives in the lower three levels of Bloom's Taxonomy of educational objectives (Bloom,1956) was also observed across elementary Students, as well. Jensen (1968) suggested a paradigm which Seems to fit this observed pattern when he Suggested his level I and level II abilities idea. Level I abilities would be those which require rote memorization and Simple manipulative Skills (the first three levels of the taxonomy) while level I] Skills require higher level reasoning (the higher three levels of the taxonomy). Jensen postulates that differences in the €ase with which Persons learn level I abilities are smaller between socioeconomic groups than-differences in the ease.-with which persons learn level II abilities with lower socioeco Class persons having the greater difficulty in learning these skills. USVI examinees appear to have achieved at least adequately on objectives which would be classified in the lower three cate- ; ‘ ication). gories of the taxonomy (i.e. knowledge, comprehension, and applicat ) ‘14 BEST COPY AVAILABLE 11] They achieved less well on objectives that would be classified in the upper three categories (analysis, synthesis, and evaluation). Local examinees seem to achieve well on objectives which can be achieved by memorization \knowledge objectives) or the application of a general procedure, rule, or algorithm (comprehension and application levels). They scored extremely well in spelling and single step multiplication computations. They scored lower on items requiring them to organize data, choose information relevent to solving problems, and to choose a series of rules and/or algorithms to use in solving problems (analysis, synthesis, and evaluation). Note the weaker achievement in mathematics applications problems, multistep computations, and the use of dictionary entries, for example-This phenomenon is apparent as early as grade two and becomes more obvious as the grade level of examinees increases. Another area that may be fruitful to examine is the area of instruction. While the Stanford Achievement Test seems content valid based on an analysis of written curricula, texts, and what teachers and administrators say they are doing in classrooms, the poorer showing of examinees on higher level objectives may be a function of the emphasis placed on these objectives during instruction. If these higher level objectives are not taught in the classroom or are taught in such a way as to send a message to students that they are not very im- portant, low achievement on these objectives is a plausible outcome. For instance, if classroom tests only evaluate low level objectives, students quickly learn that these are the 115 BEST COPY AVAILABLE 3iz really "important" things for them to learn. Likewise, if the amount of time spent dealing with these objectives in the class room test only evaluate low leyel objectives, students quickly learn that these are the really "important" things for them to learn. Likewise, if the amount of time spent dealing with these objectives in the classroom is disproportionate to that spent on higher level objectives, a similar message goes out to examinees. It may be worth knowing precisely what is going on in classrooms during instruction. Finally, as with the secondary students, it was observed that elementary students omitted itmes that they had an opportunity to attempt to respond to. Although the directions did not tell Students this, the raw score of an examinee was the number of items which he or she answered correctly. There was no correction for guessing (1.e. no points were deducted for incorrect responses). Under these "number right" conditions, examinees should have guessed at all items they did not know the answer to in order to maximize their scores. The fact that there were omits on many items indicates that many examinees did not use this strategy. This may have been a function of inadequate test directions (i.e. the directions did not indicate the method of scoring being used, but this is a somewhat dubious assumption since most tests these examinees take are "number right" tests. Rather, the problem is most likely one of poor test taking skills and these skills can and should be taught in the classroom. BEST COPY AVAILABLE References Allen, M.J. §& Yen, W.M. Introduction to measurement theory. Monterey, CA: Brooks/Cole Publishing Co., 1979. Bliss, L.B. Virgin Islands of the United States public school basic skills achievement Survey, Technical Report #1: Validation of the use of the Stanford Achievement Test with U.S.V.I. students. St. Thomas, USVI: Caribbean Research Institute, College of the Virgin Islands, 1982(a) Bliss, L.B. Virgin Islands of the United States public school basic skills survey, Technical Report #2: Item analysis and report of student skills of secondary school students. St. Thomas, USVI: Caribbean Research Institute, College of the Virgin Islands, 1982 (b) Bloom, B. (Ed.) Taxonomy of educational objectives, Handbook I: Cognitive domain. New York: Longmans, Green § Co., 1956. Cureton, E.E. The upper and lower twenty-seven per cent rule. Psychometrika, 1957, 22, 293-296. Jensen, A.R. Patterns of mental ability and socioeconomic status. Proceedings of the National Academy of Sciences, 1968, 60, 1330-1337. » T.L. The selection of upper and lower groups for the validation of test items. Journal of Educational Psychology, + 17 JA ous bt 2a. . > BES] COPY AVAILABLE