USVI Public Records

Document text

ED 243 964 AUTHOR TITLE INSTITUTION PUB DATE NOTE AVAILABLE FROM PUB TYPE EDRS PRICE DESCRIPTORS IDENTIFIERS ABSTRACT DOCUMENT RESUME TM 840 287 Bliss, Leonard B. Item Analysis and Report of Student Skills of Secondary School Students. Technical Report #2, Public School Basic Skills Achievement Survey. - College of the Virgin Islands, St. Thomas. Caribbean Research Intt. Jun 82 103p.; For Technical Report No. 11 see ED 227 150. Caribbean Recaz.,.h Institute, College of the Virgin Islands, St. Thomas, Virgin Islandt 00801 ($4.00). Reports Research/Technical (143) MF01/PC05 Plus Postage. *Academic Achievement; *Achievement Tests; *Basic Skills; Difficulty Level; Elementary Secondary Education; *Item Analysis; Language Skills; Mathematics Skills; Reading Skins; *School Districts; Test Items Stanford Achievement Tests; Stanford Test of Academic Skills; *Virgin Islands A sample of slightly over 1,500 students from even-numbered grades in public schools of the U.S. Virgin Islands were given the 1973 edition of the Stanford Achievement Test (in grades 2, 4, 6, and 8) and the Test of Academic Skills (grades 10 and 12) in an attempt to assess student academic achievement in the basic skill areas of mathematics, reading, and English Language. This report describes the data analysis which involved a detailed item analysis of each item on each test given to the sample of students in grades 8, 10, and 12 as well as a summary of student skills based on their achievement along objectives provided by the test publisher and keyed to individual test items. Measures of item difficulty and item discrimination were calculated for the entire territorial school system and for the individual school districts. (Author) ***********************************************************************. * Reproductions supplied by EDRS are the best that can be made * * from the original document. * ********************************************************),************* Virgin Islands of the United State's Public School Basic Skills Achievement Survey Technical Report # 2 Item Analysis and Report of Student Skills of Secondary School Students Caribbean Research Ingtitute; College of the Virgin Islands Leonard B. Bliss, Ph.D. - Principal Investigator June 1982 Contents 1 Abstract 2 Introduction 3 Background 5 Item analysis 6 Difficulty indices 6 Discrithination indices 7 Summary of student skills 11 Availability of data 13 8th Gi.'ado (Advanced Level) 14 Vocabulary 16 Reading Comprehension 18 Mathematics Concepts 29 Mathematics Computations 32 mathortics Applications 36 Spelling 40 Language 43 10th Grade (TASK Level I) 51 Reading 53 English 63 Mathematics 67 12th Grade (TASK Level II) 73 Reading 75 English 84 Mathematics 88 Discussion 96 References 100 4 Abstract A sample of slightly over 1500 students from even numbered grades in public schools of the U.S. Virgin Islands were given the 1973 editiOn. of the Stanford Achievement Test (in grades 2, 4, 6, and 8) and the TeSt of Academic Skills (grades IQ and 12) in an attempt to assess student academic achievement in the.basic skill areas of mathematics, reading; and English Language. This report deSdribeS Phase II of the data analysis which involved a detailed item analysis of each item each test giv6n to the sample of students in grades ;--_ 8, 10; and 12 as well as a summary of student skills based on their achievement along objectives provided by the test publisher and keyed to individual test items; Measures. of item difficulty and item discrimination were calculated for the entire territorial Sdhbbl system and for the individual school districts At the request of the US. Virgin Islands Department of 8-du-cation and the iioard. of Trustees Of the College of the Virgin Islands, the Caribbean Research Institute embarked of basic skills achieVeMent i It soon became clear that virgin islands any Strategy designed to skills achievement needed to Start Off with on a study public schools, improve baSid _ \ a fairly detailCd description of current achieVeMent levels of student:s in territorial public schools It Was founa that this was not available. The task force set up to design the study decided that the Most efficient way to obtain information on levels of basic skills achievement was to administer a Standardized achievement test to a representative sample of students and to analyze the 1982) results of this test. Technical_ Repo_r_t_l (Bliss, describes, in detail, the process used to choose an appropriate test; Finally, the Stanford Achievement Test (1973 version) was chosen as the instrument of choice. Briefly, the reasons for choosino this adhieVetent test battery were that 1) it covered the grades K-12, 2) it seemed, on initial oCServation, to be a good match with subject content taught in the schools, 3) it was technically sound given the population on which it was standardiZed (a seemingly representative sample of mainland U.S. students); and 4) it would report out criterion referenced results; Due to various organizational and fiscal contraints only students in even numbered grades were tested. This seemed acceptable since many of the objectives tested by the Stanford Test carry across adjacent levels of the test an-, no reason to suspect that the patLerns of academic of students in odd numbered gra s were difforen in even numbered grades. These constraints are in Technical Report #1. of students to be tested was done using a technique with classes as the cluster unit. Ti-ii Ciet; 17=iplino procedure can be found in Technical lepOi-t discussion of the effects of using this sami, On the information obtained. Table I presents a awn of the sample as it finally emerged; Table 1 U.S. Virgin Islands Sample Sizes Test Level Total St.Thomas System St. John District St. C. 1)i st. r TASK II 129 74 TASK I 254 167 / Advanced 345 173 1 7 Intermediate II 227 146 Primary TIT 346 186 Primary I 234 143 ji Total 1535 889 (.16 3 'sting was done at the grade level recommended by the `c.. c. i-,UbliSher during the Week of October ni 1980 in the r:MiS/St. John district and dUting the week of December 1, in thO St. Croix district. Testing materials and completed -et.-0 collected; answer Sheets checked to deterMine with marking instructions; and answer sheets sent to ')o machine scored. BackgrbUnd - Technical Report 42 is the socond of four reports that Will deal with thc :_:osults or the basic skills assessment deSeribed above. Tc!chnil Report U. detailed the procedures used in test selection SaMpling; and test administration; More importantly; it estblished empitically; the content validity and the rilLibility of the Stanford AchieVeMent Test when-it was a,Thihistered to a Sample of b;S. Virgih Islands bdbli6 school This iS particularly important since there exists i:JcLr(lizod test Of academic achievement which includes t(idonts in its Standardization group. report examines the scores of secondary school students 0; and 12) and presents : I) al: item analysis of each item on each test of the battery which includes indices of item difficulty and discrimination and 2) a summary of student skills based on their scores on the itOM8 keyed to specific objectives. 6 ii-em _Analysis Dfficulty indite8 The difficulty index of an item is found by taking the number of examinees scoring correctly on an item and diViding it by the total number of students taking the test. In shOrt, it is the proportion of examinees who scored correctly on an item and has a range from 0 (no students SCoting correctly) to 1 (all students scoring correctly). BecauSe of the fact it is a proportion, it is often designated in the literature as "p" (e.g. p=.75). It may be worthwhile to point out that the term "difficulty indo" may be somewhat of a misnomer since items with high difficulty indices are actually leS8 difficult than items with low diffitulty indices-. NevertheleSS, the term and its definition have become standard in the area Of psycho- metrics throughout the United States. Difficulty indices for each item on each test were reported out by the test scoring service. In addition; difficulty indiC88 for examinees in the standardization group in the same grade as local examinees at approximately the same time of year are reported. The test scoring service used a Chi-squared test for proportions Oh each difficulty index to test the hypotheses that the proportions of local students scoring correctly on individual items in greater or less than the proportions of examinees in the standarditatiOn group scoring correctly at the .05 level of significande. Significant differences in either direction were reported out. 9 7 DiscriMinatien incicos The itern Oicrr1natLon :ndex indicates the degree to Which resp6nse:, Ch ltaM related to responSeS On Other items on he teeL. ThO itic indicates whether a person who does well on thc? tt Whole (that is a person who iS presumably high On the trait betricj measured) is more likely to -9iet the particular iern correct than a person who 6OeS poorly on the test as a whole. In other words; the item diSCriMinatic:i whethet ail item discriminates between thdSe who lo an those who do poorly on the test as a whele. Taking the item difficulty and the item discrimina- tion indek ibtb COnSideratioru the developerS of tests desire to construct tests WhiCh discriminate well among examinees with varying' levels of a trait, The item disc:rtMination index is calculated by the formula , L Where = thL:newiher Ohb have total test scores tOlal test scores and who also have the it.pr; cnrrct. L = the numb'el: of examineos Who have total test scores in the lbwy range of total test 'scores and who also have the item coi-ect N - the in the upper or low range C) 1. the test score,. 8 By definition; d is the difference between the proportion of high scoring examinees who got the item correct and the proportion Of loW scoring examinees who got the item correct; Upper and lower ranges generally are defined as the upper and lower 10% to 33% of the sample, With-examinees ordered on the basis of their total test score. When total scores are normally distributed, using the upper and lower 27% produces the best estimate of d (Kelly, 1939). If the distribution of total test scores is flatter than the normal curve, the optimum percentage is larger and approaches 33% (Cureton, 1957). However; Allen and Yen (1979) fOund that; for most applications, any percentage between 25 and 33 will yield similar estimates of d. In this study; 27% was used as the upper and lower percentage because examination of selected distributions Of actual test scores revealed nearly normal distributions. The theoretical range of d is between -1 and ±1. However; maximum discrimination is likely to occur when p=.50: When p=.50 the Varian-C-6 in item scores, which is p(1-p), is maximized. As an item becomes more difficult, it is less likely that any student will score correctly on it. As it becomeS less difficult it is more likely that any student will get it correct. This could lead to the suggestion that all items should have p=.50, but the useful- ness of this suggestion is mitigated by interrcorrelations among items. In an extreme case; if the items on a test all interrcorre- latod Perfectly and had difficulties of .50i half the examinees would receive a total test grade of zero and the other halt would have perfect test scores. Hence, there would be no fine discrimination between examinees' levels of achievement or whatever trait Was being measured; _., general, test designers tend to try to choose items with a range of diffitUltieS that average around .50. IteMs of particularly low diffietlty are often included in tests (usually among the earlier items) for motivational reasons. Itefii discrimination indices were calculated in this study to provide indications that items may be flawed when used with U VI students: Such flaws are ambiguity; the presence of clues; the presence of more than one correct answer, and other technical defects;- If none was found upon examination of the iteM, and it was determined that the item did; indeed; appear to measure the objective it was intended to; the i-6-2:M was indlUded in the overall analysis of results. Any item that discriminate positively can make a contribution to the measure- ment of pupil achievement and low indices Of discrimination are frequently Obtained for reasons other than iteM defects; Standardized achievement tests are designed to measure several different types of learning outcomes (e.g. knowledge; understanding; application, etc.). Where this is the case, test items that represent an area receiving relatively little emphasis will tend to have pbbr discriminating POw6r. For example, if a test has forty item measuring kh6isi16d4e of specific facts and ten items measuring understanding; the latter items can be expected to have low disatimination indices. 12 This is because the items measuring understandihg have less representation in the total test score and there is typically .a low correlation between measures of knowledge and measures of unde-standihg. LOW discrimination indices here merely irldicate that these items are measuring something different_ from what the major part of the test is measuring; Removing such items from the test would make it a more homogenous measure of knowledge outcomes; but it would also damage the content validity of the test because it would no longer measure objectives in the understandihg area. Since achievement test batteries need to measure a wide variety of objectives in a reasonably short period of time; they tend to be fairly heterogeneous in nature and moderately low discrimination indices tend to be the rule rather than the exception. To summarize, a low discrimination index alerts test users to the possible presence of defects in test items but doeS not cause them to discard these items if they appear to be funetiOh- ing as they should. A well constructed achievement test will - of necessity; contain items with low discriminating power and to discrard them would result in a test which is less; rather than more; valid; Dthe to these considerations, in this study items were examined if they had diSeriminz:tion indices lower thah .20. This is a rather conservative criterion since items that diSCtiminate as low as this may provide useful information; but given the unknown test taking charactcristics of USVI students; it was decided to be particularly cautious in the item analysis. 13 11 Summary of Student Skills The items on the tests of the various levels of the Stanford AdhieVetent Test battery were keyeii3 tti behaviorally stated instructional Objectives. These objectives are grouped into a series Of item groups.. Tables are aVailable which present objectives within item groups and the difficulty and discrimination ildiceS for the item or items which evaluate those objectives; TheSe may be obtained from the Caribbean Research Institute; In addition, the difficulty index for each item for the examinees in the Standardization group is available. This standardization group consists of examinees who were in the same grade at approximately the same time during the school year as the U.S. Virgin Island8 sample; The national p values are used not as a means to compare U-S. Vitain Islands students with mainland US examinees. Historital and cultural differences ti-otweon these two groups of examinees makes this comparison an inappropriate one. Philosophital tonsiderations aside, hoticievet, such comparisons are of little use to the people who make curricular decisions in schools. What these people need to knOW are the particular levels of skills of students as measured aqaitist well defined objectives, not hOW well their students achieved thse skills as compared to other students. Nevertheless, since the skills and knowledges taught in schools are seldoM taught once, but are dealt with at a number of grade levels where they are reinforced and broadened, the level of achievement= on specific obje tives should_be expected 12 to change from grade to grade for a particular student or group of students. The publishers of the Stanford Achievement Test take this factor into consideration by testing particular objectives across a number of levels (grades) of the test; this study the national p valuea are used to indicate the relative level of difficulty of an item by which the performance of the local sample nay be judged For instance, it would be foolish to be dissatistified if 20% of the USVI students have indicated that they have reached an objective when only 18% of comparable students in the standardization group had reached that same objective. What is more likely the case is that this is a difficult and complex objective .t.hat had just been taught recently and would be retaught and elarged upon at a later time. Thetefbte; the following criteria were used in summarizing student skills. Skills are described as "adequate" if the propor- .tions of local examinees scoring correctly on items measuring those skills are not significantly higher or lower than the proportions of the standardization group scoring correctly or, if significantly higher or lower; the proportions correct are within 10% of the standardization group proportion correct as reported by the test scoring service; Skills are described as "strong" if the proportions of local examinees scoring correctly on items measuring those skills are significantly higher and more than 10% greater than the standardization group proportion correct as reported by the test scoring are described, a8 "weak", if the proportion of 15-dal examinees scoring cOrrectly'on items measuring those skills are 13 significantly 16Wet and more than 10% less than the standiza- tion group proportion correct. The 10% standard was set*in the realization that some differences, while statistically significant may be educationally trivial and it was noted that most differences indicated as significant exceeded 10 %. Findlly, in cases where the scores of examinees from both the St. Thomas/St; John and the St. Cr6iX districts were the same based OA the criteria stated above summaries were based on the entire USVI sample; Where differences were noted, the skills of e:-<atinee4 within each district were summarized separately Avallabil_l_ity of Data Tables listing each objective on every test keyed to test items with national diffidUlty indices, difficulty indices for the entire USVI sample and the individual district subsamples as well as item discrithinatitin indices for the whole sample and each subsample are available and may be obtained frbt the Caribbean Research Institute,College of the Virgin IslandS. Grade 8 -_Advanced Level The Advanced Level of the Stanford Achievement Test Basic Battery consists of seven tests with the number of items listed below: 1. Vocabulary 50 items 2; Reading Comprehension - 74 items 3; Mathematics concepts 35 items 4. Mathematics Computation - 45 items 5. Mathematics Applications - 40 items 6. Spelling == 60 items 7. Language = 70 items The means; standard deviations; and reliability estimates for theSe tests determined previously (see BlisS; 1982) are presented below: 14 'Test Mean Raw Score Standard DeviatiOn KR720_ Reliability Standard Error Total USVI Sample_ Vocabulary 21.0 7;2 .81 3.15 Reading Compre- hension 31.5 15.4 .95 3.45 Math Concepts 15.3 5;8 .74 ' 2.65 Math Computation 23.0 7.6 .85 2.95 Math Application 16.9 6.6 .83 2.72 Spelling 31;8 12.3 .93 3.27 Language 35;6 12.1 .88 4;20 17 15 St. Thomas/St. John Sample Vocabulary 20.7 .78 3.10 Reading Compre- hension 32.6 17.4 .97 3.02 Math Concepts 16.4 6;0 .81 2.61 Math Computation 23;3 1.1 .83 2;92 Math Applications 17.7 6.6 ;83 2;72 Spelling 32.8 11.8 ;92 3;33 Language 35.8 10.6 ;84 4.23 St. Croix Sample Vocabulary 21.2 8.5 .87 3.05 Reading Compre- hdrIsion 30.5 13.1 .93 3.46 Math Concept 14.2 5.3 .75 2.67 Math Computation 22.8 8.2 .87 2.94 Math Applications 16.1 6;5 .82 2.76 Spelling 30;8 12.9 ;93 3.40 Language 34;6 13.6 .91 4;07 Following are the item analyses and summa -ies of student aChievement on the objectives evaluated by each of the tests; 16 Vocabulary I_tet_ALnadvsis All items showed acceptable discrimination indices with the following exceptions: Item #16 - This item was an easy. one (p---.72) for USVI examinees regardless of overt.11 achievement; Item #32 Thib item was particularly difficult for USVI students (p=.16). It Called for them to select a synonymn for "poneer;" Many atudehtt Chbad "beat" over the correct responoe "think." Item #39 This item was particularly difficult (1)=.17); but there seemed *-o be no particular pattern of incorrect responses; Item #40 This item simply did not discriminate well between high and low achievers; although difficulty was moderate (p=.37). It required students to select a synonym for "contemplative" and many examinees chose the distractor "cautious" over the correct response "thoughtful." Item #43 - This item diSdriMinated very poorly between high and low achievers; High adhievers often choose "dejected" as a synonym for "exhilarated" rather than the correct response "stimuIated;" Item #47 - This item discriminated very poorly in spite of a moderate difficulty index (p -.29) and there appeared to be no pattern of incorrect responses. 17 Item # 29 - ThiS item shOWed very little discriminatibil With high achievers equally likely to answer the item correctly as were low achievers with a moderate difficulty indeX (p=.52). IteM # 34 = This item showed low diSdkimination between high and LOW achievers. The difficulty index (p=.31) was not partidUlarly loW nor was there a noticeable pattern of incorrect responseS0 Item # 1 = Thi8 item was very easy for USVI students (p=.96). Item # 30 - This item tended to be difficult for USVI students (p=.21) with no patterns observed in incorrect responses. Item # 31 - This item was a difficult one for USVI students (p=.21). Itet # 44 - This item discritinated satisfactorily for St. Crbix School district student80 but poorly for St. Thomas/St.' John diStrict examinees. These latter examinees in the high adhieving group chose "dangerbUt" very often as a synonym for "anomolbus" over the correct response. This pattern was not evident among St. Croix school diStridt students. Item # 41 - ThiS item discriminated satisfactorily for St. Croix district students, but did not discriminate at all among high and low achievers in the St. Thomas/St. John district; These examinees tended to choose-"active" as a synonym for "audacious" over the correct retpbnse; "bold." No such pattern was noticeable among St. CrOix district students. In general; this ;tended to be a difficUlt item for students in both St. Thomas/ St. John and St. Croix districts (p=;19 and .21,respectively). IteM # 19 - This moderately diffidtilt (p=.40) item tended to diSdriMinate poorly among high and low achievers. There appeared to be no ambiguity in the item and no observable pattern of incorrect response. 1C Summary of- Student S 1is In general, USVI students tested seem to be weak in tht area f standard EngliSh vocabulary recognition. Most items seem to function well with most poorly discriminating items being the result of either extremely high or low levels of difficulty; Since few of e incorrect response patterns (i.e. examinees choosing a particular incorrect item with a high frequency) which typically mark ambiguous items were presenti it is unlikely tAat these scores were the result of technical difficulties in the items themselves. Of the fifty items on the test; a significantly smaller proportion of USVI students were able to respond correctly when compared to the standarditatiOn group on 39 of these items. Beyond the question of statistical Signigicance, the proportions of USVI students who successfully answered many items was substantially lower than the the standardilation sample. The differences in item mean difficulty values are a clear indication of this problem; Of particular note is the single item on which the USVI sample outperformed tne standardization sample (but only in the St. Thomas/St. John district) This item asked examinees to chocLe the synonym for "capsized" by stating, "A capsized boat has been -.==" - With the correct response choice being "overturned." Since such an item may allude directly to personal experiences of USVI students; thiS is a phenomenon that bears examining in looking at other items. Reading Comprehension Item Analysis 19 All items showed satisfactory discrimination indices except the following: Item #12 - High and Iow achieving students seemed to be equally likely to correctly answer this moderately difficult (p=.30) question in the St. Croix district. In the St. Thomas district; low achievers appeared more likely to be able to correctly answer the item than were high adhieverS. High achieving students seemed to choose distractor #4 very often. The item requires examinees to choose the most appropriate title for a.passage they have read; Distractor # 4 refers to a particular section of the passage and uses names and ideas in that section. However; it does not refledt the overall idea of the passsage. Students choosing this distractor are not aware of this overall (or "global") ideaf. IteM # 54 = This item seemed to discriminate Satisfactorily for examinees in the St; Croix distridt; bUt less well for those in the St. Thomas/St; John district. This was 'a fairly diffictilt question (p=.18) which required examinees to choose the main idta in a short poem by Walt Whit:Man. A large number of StUdentS from both districts seem to have omitted this item which appeared toward the end of a 74 item examination: ThisjwiI1 be 'commented on in the summary. Item #54; 60; and 74 display the same phenomena as item #54, showing many omits on what well to be difficult items; Itet #3 = This appeared to be an easy item for most students (p=.90) IteM #57 = ThiS item appeared to be a difficult one for students in 22 20 both districts (p=.18). However, it should be noted that 48% of students tested omitted this item and this `could account for the low difficulty and discrimination indeces. Item # 73 This item seemed to exhibit the same phenomenon as item # 57 In this case 59% of students tested omitted this item; Item # 41 - In this fairly difficult item (p=.28) high achieving students were distracted by choice # 4; In this question students read a paragraph about home mortgages and were asked to choose a statement indicating the value of a mortgage to a lender although this advantage was not explicitly stated. Choice # 4 indicated that the value was that the lender would "eventually own the property," which the paragraph indicated was the consequence to default on the part of the borrower. The choice which was considered correct was his money earnsinteresti" and the fact that the borrower pays interest was also explicitly stated in the passag. Since either of these situations could be considered as "value" by a thoughtful examinee, it may be that the question was defective in that it had two answers that could be considered correct by students who lacked experience with mortgage. Item # 72 - This item appeared to be very difficult for USVI examinees. However, 59% of examinees omitted the item and thiS could account for the low difficulty and discrimination irdices which were observed in the St. Thomas/St. John examinees responses. Item # 65, 67, 69 = TheSe three iters showed satisfactory discrimina- tion in the St. Ckoix diSttitt, 'Alt poor discrimination in the 23 2 St. Thomas/St. John district. One phenomenon that was noted was that while the numbers of high achievers scoring correctly on these items was fairly constant between districts the number of lOW achievers scoring correctly WAS greater in St; 'homas /Stt JOhh than in St. Croix; AIS6 the proportion of omits was higher in the St; Thomas/St. John district (63,64, and 66%; re;pectively vs 52;51; and 52% on St. Croix) and this could ad-count for the differences item discrimination power that was observed. Item # 23 This moderately difficult item 4.34) discriminates little; if at all; between high and lOW achieving .tideEts; In general; LOW achieving students seem just as likely to obtain correct scores on this item as high achieving students ; There does; hOWeVer, seem to be differenceS in the patterns of incorrect responses between these two groups of examinees; The item asked Students who had read a passage which dealt with animals supported by the British governMent which included cat who was the "OLtidial Rat Catcher" in the storeage cellars of London's government offices; The passage indicated that the cats food cost the government the equivalent Of per week while neglecting to give any information on the cost of upkeep for any of the other animals in the passage; Lhe question; "the writer probably mentions exactly what it casts to feed Peta [the cat) because he thinkS it is ---;" should elicit "amusing" as the correct response. Most examinees failed td See this as being amusingi with the low achievers tending to select "too and the higher achievers tendifi4 22 to select "unfair" as their answers. The latter seems to be more of a moral judgement and it may be significant that this was the choice of the higher achieving students. It is also interesting to note that the Technical _Data Re ort published by the testmaker shdWs this item to discriminate less well than most other items (although much better than on the USVI sample of examinees) when the test was administered to the standardization groups. Item # 29 This Moderately difficult item (p=.36) discriminated poorly between high and low achievers. It required students to infer something about the climate of Tropical Africa from reading a passage concerning rivers in the region and their great potential for the production of hydroelectrioal power. Most students concluded that in this region there was "sun- shine most of the time" rather than the conclusion drawn from the passage that there was "quite a bit of rainfall" What may be happening is that examinees are draWing from their own previously learned concept: about the nature of the African climate and/or their personal experiences concerning the climate in a tropical zone instead of using the informa- tion provided in the written passage they were to read. Items # 58, 61; 63; 66 - These items appeared to be very difficUlt for USVI examinees (p=.15i .141 .17i \and .12i respectively). Howeveri as was the case"in other items- owa d the end of the test; the number of students omitting these items was very high (52 56 59 and 63%i respectively in the St. Thomas/ St: John district and 49; 52 54! and 56% in the St. Croix 23 district). These high proportions of omitted responses could account for the low diffieulty and discrimination indices which were obtained; Summary of Student Skills Table 2 shOWS the proportions (in percents) of omitted responses on each item in the Reading Comprehensidn test. From this table it is apparent that students in this sample did not have the opportunity to attempt all of the items on the test in the time allowed to them. This is an important observation. If students could not attempt items, they could not possibly score correctly on them and this would tend to deflate;the difficulty indices of items toward the end of the test; making scores of USVI students Seeth lower than they should actually be. In addition, since lesS than half the students were able to. attempt items 60 through 74, we would expect lower discrithihation indices than would be obtained had all students been able to complete the test. ThiS phenomenon seems to be unique to the Reading Cdthprehension test. On all other tests in the Advanced Level battery examinees appear to have ample time to finish within the time provided by the test publishers..Any attempt to explain why the USVI group required more time than the standardization groups 14-bad only be speculation; but further research should be done to investigate the posSibilities that such factors as a more deliberate reading/ I style (resulting in longer periods of time being required for students to read each passage), fatigue, a and differences in attention span might be contributing to this phenomenon. Given theSe observad.Oni it might be well to be 6icii-oti(JE 24 TabIe 2 Proportions of the Eighth Grade Sample Omitting Reading Comprehension Test Items (Percents) ITEM# USVI STTIST3 STX 0 3 4 1 1 C 5 1 1 6 0 0 0 1 8 9 0 o 10 1 11 1 2 12 13 0 1 2 14 1 1 15 1 16 1 17 0 18 1 1 0 19 1 2 "--, '20 21 1 27 Table 2 Continued 22 1 23 6 1 24 1 2 U 25 5 3 26 5 2 27 4 28 4 5 29- -7- 4 5 30 5 5 5 31 6 5 6 32 8 8 8 33 8 8 9 34 7 6 8 35 8 8 8 36 9 7 11 37 11 9 13 38 13 11 14 39 14 . 13 15 40 14 12 15 41 15 14 17 42 18 17 19 43 19 19 19 44 20 20 20 45 21 20 21 46 '23 23 23 47 29. 31 27 48 30 31 28 25 Table 2 Continued 34 38 38 30 30 11 49 50 51 32 34 35 52 39 44 34 53 41 45 37 54 42 45 40 55 44 48 40 56 47 49 49 57 48 52 44 58 49 52 46 59 50 - 53 48 60 51 55 48 61 52 56 47 62 53 58 48 63 54 59 56 64 55 60 51 65 57 63 52 66 56 63 50 67 58 64 51 68 57 64 51 69 59 66 52 70 59 6.7 .51 71 58 65 52 72 59 63 54 73 59 64 59 74 58 63 53 26 27 .drawing any conclusions from the results of items 50-74 where more than a third of the examinees omitted the items in question. Re8tridting the observations to items 1 thrOugh 49, however, the fbllowing generalizations can be made; USVI students tested seem to be somewhat weak in the area of determining the general idea of the passages they read, although this must be a somewhat tenuous judgement based upon the small number of items (five) from whidh thi8 judgement could be made because of the large number of examinees omitting later items of the test. However, in three of these fiVe items, local students had a significantly smaller proportion of correct answers than did the students in the standardization groups. In the area of determining meanings explicitly stated in a reading passage, USVI students appeared to have achieved well. In most of the items which most students seem to have had the opportunity to answer, the proportidn of local students answering correctly was as high asor only slightly lower than the standardization groups proportions correct. Implicit meaning refers to ideas in a piece of reading which are not specifically in the reading, but must be deduced from facts USVI examinees ear to have difficulty in determining explict meanings in pa sages they have read. In ten out of thirteen items dealing with is skill that most students were able to attempt to answer in the time allowed, the students in the local sample scored correctly a significantly lower proportion, f the time than standardization sample, Further, the mea proportion of students or other information presented in the test. The 28 scoring correctly on these itemE was particularly low (59% vs:70% for the standardization group); Often a reader must obtain information from a text by obtain- ing clues from what is explicitly stated within the text. The USVI examinees had great difficulty in determining meaning from written passages on the basis of contextual clues. In seven out the nine items that most examinees were able to attempt within the time allowed, the proportion of local students answering items correctly was significantly lower than the proportion of examinees in the standardization sample answering the -same items correctly. Again, these proportions were particularly lbw with a mean of 38% for tt local examinees versus a mean of.52 for the standardization group. The students in the USVI sample had great difficulty in tasks which required them to make inferences or draw conclusions from passages they had read. The proportion of USVI examinees answering items correctly was significantly lower than the proportion of ,examinees in th'e standardization sample who answered the same items correctly for all of the eleven item; which most examinees in the USVI sample were able to attempt'in the alloted time; Determining inferential meaning appeared tc, be the reading com-- prehension skill which gave USVI students the most difficulty with a mean correct proportion on the above mentioned eleven items of 28% versus 46% for the standardization group. An observation that may be significant. is that USVI students .showed strength in the area of determing explicit meaning from material which they read while showing weaknesses in aohivement 31 29 in other areas of reading-comprehension. The taxonomy of educaticnal objectives developed by Bloom (1956) may be a good theoretical framework from which to view this phenomenon. Being able to derive meanings that are explicit in readings inVOlVeS objectives at the comprehension leVel of the taxonomy while the other objectives types (deriving global meening; implicit meaning, inferential meaning, and meaning frbt context) involve higher level (and, therefore, more diffidUlt) objectives in the area of analysis and synthesis. It appears that the sample of USVI students may have, as a group, achieved reading comprehen- sion objectives at the comprehension level, but have not, as yet, fully mastered many of the higher level objectives in thi8 skill area. Mathamatics_Concepts Item_Analysis All items discriminated s4tisfadtbrily except the Itet 6 - This was-a)iparticularly diffidUlt item for USVI students (P=-15)- One distractor operated tj cause many students to choose it over the correct response. Specifically, the item asked students to complete the aritAmetid sentence: 6+12=. Most students, regardless of general leVel of athieve- ment, chose "2" as the correct answer rather tha 11". This indicates that the item was functioning properly, but that examinees had not mastered the objective being tested. 32 30 Item # 4 - This proved to be a relatively easy item for examinees (p=.75); but it did not discriminate satisfactorily in the St. Thomas/St. John district. This was primarily because low achieving-stude s were almost as likely to answer correctly as were high achiev_ students. Item # 16 - On this moderately difficult item (p=.36) high achieving students were not much more likely to answer correctly than were low achieving students; There appeared to be no pattern observable in the incorrect responses; Item # 29 - In this moderately difficult item (p=.35) high achieving students were not much more likely to answer correctly tha.. low achieving students. The item asked students to determine the effect on a factor if the quotient remained constant and the other factor decreased. Many students from both achieve- ment level groups chose the response that indicated that the factor decreased rather than the correct response indica- ting that it would increase. The item appears to be operating properlyi howeveri since thij; error is a common one and indicates a lack of understanding of the concept. Item # 33 - On this difficult item (p=.27) most students in both achievement level groups scored incoriectiy. The lack of discrimination was very likely due to the difficulty of the item rather than to any technical fault in its writing.. No response patterns were observed in the incorrect answers. Item # 35 - This was an extremely difficult item for all students (p=.22) resulting in a low level of discrimination.. There 33 31 appeared to be no pattern of incorrect responses. Summary of Student- Skills Number The itet grouping "Number" refers to the concepts which deal with the relationships betWeen different types of numbers (e.g. primes; oddS vs evens; fractions vs whole). USVI students showed no particular strengths in this area Concept areas that appeared to be particularly weak were: A) the concept of fraction and its relationship to other types of numbers. b) the' concept Of odd and even numbers c) the concept of Signed numbers: d) the concept of "modulo" Notation "Notation" refers to the technique by which numbers are expressed in written form; SUCh forms include expanded notation, differing base Values, decimals; and fractions. The students in the USVI.sampIe seemed to have a strong grasp of the.concopt of exponential notation. They seemed to be weakest in the following areas: A) .use of expanded notation b) decimal notation c) translating arable numbers into word f)rm Operations The term "operations" refers to the manipulation of numbers based on the use of a set of previously defined fanctions (e.g. addition; subtraction; multiplication). The students in the local 32 sample Showed strenghts in the area of multiplication of decimal fractiont, but seemed to be weak in the following areas: a) solving inequalities b) identifying multiplicative identity c) identifying the uses of the associative property d) the use of sighed numbers in multiplication and division e) identifying formulas fbr qualities of geometric functions .f) estimation in multiplication and division Geometry and measurement This appeared to be the atea,bf math concepts in which the USVI sample exhibited strongest levels of achievement; Examinees performed well in the following areas: a) metric conversion b) the use of Venn diagrams c) graphing Areas in which weaknesses were observed were: a) set theory b) estimation of angle sizes c) properties of geometric figures d) probability Mathematics Computation Item Analysis All items discriminated satisfadtbrily with the exception of the following. J i5 33 Item # 36 This item appeared to be vary difficult for examinees in the USVI sample (p=.10). The question required students to find the average of 12 and 6. In the low achieving group there appeared to be no pattern of incorrect responses, but high achievers were noted to choose the response "18" a large proportion of the time. This response would occur if examinees summed to two numbers; but failed to follow through on the second part of the averaging operation and did not diVide. Item # 45 - This Appeared to be a difficult item for the examinees in the local sample (p=.24). It should be noted that the item was also difficult and discriminated at low levels (d=;29) for the standardization sample. An examination of the item; however; indicates that it is quite clearly worded and the task seems to be simply presented. The lack of diScritiliriatinq power is most likely due to the general difficulty of the item. Summary of Student SkIlla Knowledge of Primary Facts and Solution of Simple Mathematical Sentences On these skills there appears to be some substantial differehdeS between the achievement leVelS of students in the St; Thomas/St.Johh and the St; Croix district. Because of this it would be useful to summarize achievement charAdteriStieS inch district; individually. In the St. Thomas/St.John district students showed strong skills in the following areas.: a) use of the associative property of multiplication 36 34 b) multiplication and diViSibn with integers; fractions; and mixed numbers c) addition; subtraction; and renaming of fractions d) use of the distributive property Weaknesses were observed in these areas: A) multiplication with carry facts b) operations with negative terms c) renaming exponential numerals In the St. Crbix district examinees exhibited strong achieve- ment in these areas: a) multiplication and diVisibti of integers; fractions and mixed numberS b) use of the distributive property d) appropriate use of the distributiVe property Weaknesses in the following areas were obsetVed: a) use of the associative property in multiplication b) operations involving negative terms multiplidatic)n with carry facts Addition and Subtraction_A1orithms Objectives in thit item grouping deal with the mechanics of addition and substraction. Students in ti.e USVI sample showed good achievement in this area on all objectives= Multipl_icatiah and Division Algorithms This item grouping tested objectives dealing with the mechanics of multiplication and diviSiOn. Students showed good achieVement levels on all objectives in thiS area with the exception i of the ability to find the quotient in a division problem with a 35 two digit diVisor and a zero in the ones column of the quotient. majority of students missing the item which tested thiS objective indicated "NH" (not tiOre) as their choice leading the researchers to suspect that they came up with an answer through faulty use of the algorithm; -rnmiTnn FractiOti St. Thomas/St. John district studentt showed particular strengthS in finding common denominators and in working problems where common denominators must be found. They achieved adequately on the objective of finding the common fraction of a whole number; However, they appeared weak in the area of fraction multiplication. St; Croix dittriCt students appear to haVe achieved adequately in the area of fraction multiplication, but were weak an_objectives which required them to find and work with common denominators; Students in both schbol districts appeared to have particular difficulty in the area of division by iiaC16ns; Other nparatiOnal Models All USVI students achieved strongly in the objectives involving exponential notation and prime factbrization. Their achievement seemed adequate on objectives dealing with decimal notation and number lines. Weak areas were On objectives having to do with finding averages and working with percentages. St. Thomas/St. John district studentS exhibited good achievement levels in 'all other areas. St. Croix students showed good achievement levels in other aleas with the exception of objectives dealing with common and simpae algebraic manipulation. Mathematics applications Item_Analvs1s All items discriminated adequately except the following. Item # 11 - This was a particularly difficult item for USVI students (p=.20). High achieving students were only slightly more likely to score correctly on this item than were low achieving students. There appeared to be no pattern in the distribution of incorrect answers, except that choice number 4 (which indicated that a shaded area of a square took up the entire square when it obviously did not) was chosen very seldomly. Item # 35 - This appeared to be a difficult item for the local sample students (p=.29); It was the only problem dealing with metric units on the test. Howeveri the metric equiva- * lents were given in the stem of the item; so a student, knowing nothing of the metric system could have determined the correct answer. A large number of students missing this item either left in blank or indicated the choice "NH" (not here); it is possible thz.t these students were intimidated by the metric units and did not even attempt the item and that this phenomenon carried through both high and low 39 37 achieving students Item # 37 = This item discriminated poorly. It tended to be a difficult item (p=;21) and asked students to determine the sum of six angles marked off in triangle with a perpehdicUlar drawn from one Of the angles: to the opposite Side. The student seledtitig the proper answer needed ,to put together the fadt8 that -the sum of the angles Of a triangle is 1800 and that a perpendicular drawn to a side produces two angles of 90° . The incorrect choice chosen most frequently was "180°" which indicated that examinees may have been aware of this first fact, but were unaware of the second or were unable to use the two facts together; Item # 40 - This was a very difficult item for USVI students (p=;14). Thi8 less than chance ptcportion of students scoring correctly on this item and the large proportion of omits (20%) evokes the possibility that; although it is partof the OUrriculumi the objective was never really taught in the scho61s, and/or that being the last item on the test; a substential number of Students never had the chance to attempt to answer its Summary- of Skills Anal sis and Dr.velomeht of a Solttion Design, Selection-of a Solution SentPnce. andAdequacT of Data St. Thomag/St. John district examinees exhibited adequate skills in solving problems which required them to: 40 a) divide to determine shares b) add and divide in multistep sequences F) translate verbal problems into mathematical sentences d) determine correct change in a two step sequence They displayed weaknesses on items requiring them to: a) determine the adequacy of given data for solving a problem b) express verbally the process for solving a problem c) estimate the ratio of two'numbers greater than 1,000 St; Croix examinees showed adequate achievement on none of the objectives in thit item grouping. They showed particular weaknesses on items requiring them to: a) divide to deterMine shares b) translate verbal problems into mathematicai sentences . c) determine the adequacy of given data for solving a problem d) determine correct change in a two step sequence- e) express verbally the process for solving a problem f) estimate the ratio of two numbers greater than 1,000 Date, Scale; and Percent The USVI sample showed adequate achievement.only on 41 39 the objective of deterMining a Specific rate when one rate is expresses as an average; Particularly marked weaknesses were fOthd on items which required the examinee to: a) express part of a total as a fradtion b) determine specific rate when no average rate.is given c) interpret map scales d) find a total When a part and its fractional representation are known e) determine percents Measurement Examinees in the USVt sampledetOnstrated adequate adhieVetent in solving problems whiCh required them to: a) find the volume of a rectangular solid b) find the area of a rectangle c) find the sum of six angles They showed weaktie88e8- it solving problems Which required them to: a) calCUlate the area of a shaded region b) convert Standard masures of weight in order to divide , c) add negative numbers _ d) match EngliSh unit with metric approximations e) find and compare area of squares'and parallelograms f) identify finishing time when given starting and elapsed times 40 Students in the USVI sample demonstrated adequate achieve ment in identifying a function in a mathematical sentence With two variables and choosing the solution set for the sentence. They showed Weaknettet in items dealing with the following objectives: a) choosing the graph of a solution set b) using line graphs Statistics; Avera and Probabilit St. Thomas/St; John district students demonstrated ado irate. achievement in the ability to compute an average. They showed great weakness in their ability 4o determine the probability of the occurance of a given event; _ St. CrOix examinees exhibited weaknesses across both of these objectives. Spelling temAnalysis All items discriminated at a satisfactory level except the following; Item # 10 - This was a very difficult item. for USVI students (p=;18); Both high and low scoring studentt appeared to find it equally difficult; Summary of Student Skills Homophones HbitibphOhet are words which sound alike but have different meanings and are spelled differently; On these items, 43 41 student8 were presented with four homophones used in context and were requited to cnoose the phrase in which the homophone was used inappropriately., StudehtS in, the S Thomas/St; John distriet showed particularly strong ability to recognize the distinctibn between the'homophones "choral" and "coral" when used in context. They showed particular weakneases in distinguishing missuse of the following homophones in context: a) cereal/serial b) straight/strait c) miner/minor S d) arc/ark Oh other homograph recognition pairs these students performed at an adequate level; St. CroiX district students showed particular weaknesses in distinguishing missuse of the following homophones in context: a) cereal/serial b) bored/board el wrap/rap d) straight/strait e) miner/minor They showed adequate achievement in dealing with all other homophoneS in context. Phonics The students in the USVI sample showed a particularly strong grasp Of the use of phonics in spelling. Specific objectives where local students achieved very highly were where 42 misspelled wordS had situations where a) /1e/was spelled e-e instead of ee b) final /e/ was spelled ey instead of y c) schwa sound was omitted Objectives on which local students showed weaker achievement were tested by items where the misspelled words had situations where a) /k/ was spelled c instead of ch b) /kt/ was spelled ck instead of ct c) /e/ was spelled i instead of d) /e/ was spelled e instead of ea e) /ch/ was spelled d instead of t f) /e/ was spelled e instead of a It might be useful to note here that difficulties appeared to be of two' types. FirSt, in the use Of vowels, particularly of those having the sound /e/. This sound is represented in a great number of ways in the English language, ver. y ofte quite arbitrarily; As a result, it would be difficult to apply phonetic rules to its use; Second, in words that are generally mispronounced by the population at large. We tend to say "strickIy" rather than "strictly" and "congradulatione instead of "congratulations." Again; applying phonetic ruleS to such words could cause difficulties. Word Building Word building skills refer to those which are involved in adding suffixes and prefixes to root words. USVI students showed very'satisfactory achievement in this area .. The few weak 43 areas if achievement are those in.WhIch students were required to identify misspelled words where A) /ens/ is spelled ense instead of ence b) a final 1 is omitted when adding ly Laaa.6.8a-s Item_ Analysis All it discriminated at a satisfactory level except the following. Item # 16 - This item was an extremely difficult item for USVI students (p=.08). Low achieving students were just as likely to score correctly on it as high achieving,studetns. Incorrect answers were spread in no particular pattern except that the incorrect answer was chosen lest often than could be expected by chance (there were four choices). Only 2% of students tested omitted the item; However, it was noted that high achieving students responded in a particular way. The item required students to fill in the blanks in the sentence, "Beddbet goon saw that Humphrey was as good as in fact, Humphrey was the --- of the two." The choices were 1) he-best, 2) him-better; 3) he-better; and 4) him-best; with choice # 3 litted as the correct answer. High achieving students tended to choose response 1 and 4 over the other two responses, indicating that they believed the Superlative form of "good" was the corrcet response. cooper (personal communication) points 4s' Item # out that persons who are considered to be exemplars of the use of formal standard Englith have adopted thit usage. He suggests that it will beCOMO the fottally accepted usage within ten years. Given this situation, the item may have been ambiguous since it could be thought of as having two correct answers. - This item proved to be very difficult for students in the VSVI sample (p=.16). It tested students' knowledge of both the capitalization conventions for school subjects and the use of commas to offset words in a series; Most studentb Chete choices # 2 (Latin, Greek, Math and Science) and 4 3 (latin, greek, math and science) with the farmer indicating they were not aware of the capitalization convention and the latter that they were unaware of both the capitalization convention and the comma usage. Choice #2 was, by far; the most popular. High achieving student§ who missed the item chose 4 2 in almost every instance. Item # 61 - This was a difficult item for examinees in the local sample (p=.23). There appeared to be no pattern of incorrect responses either for the high achieving or low achieving group. The item gave a dictionary entry for the word "versatile" including a pronounciation guide. Examinees had to be able to determine form the pronounciation guide that the last syllable of the word contained no vowel sound. In writing, and often in common pronounciation the word 116.8 an "i" in the 45 final syllablet but the letter is not sounded; What may have contributed to the difficulty here that students attempted to answer the item without using the pronounciatiOn guide as thdy were instructed to do and thust tended to rely on their own pronountiation of the word be-Cat:Se they could or would not make use of the pronounciation guide. Summary of Student Skil s Capitalization Conventions- in the St.Thomas/St; John school district students seemed to have a good grasp of capitalization conventions tested except those used when naming school subjects. In this latter case they tended to capitalize the names of 11 school subjettS. Students in the St; Croix distritt had difficulty in dealing with the following capitalization conventions: a) "Sir" as an honorific (e;q. Sit Walter Raleigh) b) the name of a play c) school subjects d) names of religions e) names of institutions and cities Oh other capitaIizatitin conventions tested, these students appear to have achieved Adequately; Punctuation Conventions Students in the St; thomaS/St. Croix school diStrict seem to have difficulty in the use of commas in certain Situations; They seem to be adequately familiar with comma'usage under the following circumstances: 48 4 a) in dates b) after an introductory participial phrase c) in setting off appositive phrases d) in Setting off direct quotations e) after introductory adverb clauses f) before "and" when it joins two .independent tlaUSOS. g) between adjectives modifying the sate noun h) after introductory prepositional phrases These students do seem to have difficulty in the use of commas under these circumstances. a) setting off a parenthetical expression b) when they are not used to se off indirect quotations c) Setting off direct quotationfi d) setting off nonessential cli,uses In the St. Croix Sdhool district examinees appeared to be adequately famil_ '7 with the use of commas in these situations: a) in dates b) in setting off appositive phrases c) after introductory adverb,cliuses d) when they are not used to se off indirect quotations e) before "and" when it joins tlo independent.clauses These students seemed to show weaknesses it the use of commas under the following circumstances: a) after an introductory participial phrase b) Setting off a parenthetical Expression ) separating items in a series; 19 di separating adjectives modifying the same noun e) setting off direct quotations f) setting off nonessential clauses g) after introductory prepositional phrases 47 Usage Conventions Usage conventions refers to the choice of particular forms of a word which are appropriate in a given context. Eamples of thiS indltde the formation of past and present participles and the choide of appropriate verb tense. StUdentS in the USVI sample seemed to have a good grasp on the fOrtation and appropriate use of the past participle of irregular verbS, but showed some weakneSSet in determining the past participles of the verbs, "to write," "to begin," and "to wear." The latter verb only appeared to be troublesome to St. Thomas/St. John district students; , Examinees in both dittridtS showed weaknesses in achievement on the following objectives: a) choosing a verb form to avoid double negatives b) choosing reflexive pronouns c) choosing the appropriate verb when singular subjects are joined by "nor." The three item groupings noted above (i.e; capitalization; punctuation, and usage conventions) form Patt:A of the language test and constitute the-first 31 items of the test. This section is timed separately and it was noticed that after item #25, over 15% of the examinees tended to omit items. Twenty one percent Of the examinees omitted item # 31.The items in thiS section 50 48 of the test require examinees to carefully read one or two sentence passage and choose 'the pprOptiate words to fill one or two blanks in the sentence or sentences. It was noted in the Reading Comprehension test that local studentS appear to take more time to. read material presented to them with the result that they have difficUlty getting to the later items on tests requiring much reading. It is possible that the same phenomenon is occurring on this section of the Language test, alti;ough on a smaller scale; with the result that a substantial number of examinees are not able to attempt the last few items on the test. This would have the effect of deflating difficulty indices and Making the achievement of studentS in the local sample appear to be lower on the objectives tested by the-Se items, and on the test as Whble. It is suggested that the resUltb obtained on these later-items be viewed with caution when interpretations are made. Distinguishing_Fra_gments Sentences, and Run,0nSentences Students in the St, Thomas/St; John school district appear to have little difficulty recognizing complete sentences and sentence fragments. On the Other hand; they do seem to be weak in recognizing a run -on sentence when it is PieSeiited to them. Students in the St. CrOiX school district appear to be able to recognize sentence fragments and run -on sentences, but Seem to have difficulty recognizing complete sentences; English Skills This section of the test examir.es student achievement 51 49 in areas such as the use of dictionaries and other reference works, and in basic literary usage. In the area of dictionary skills, students in the USVI sample shOWed adequate skills in using dictionary entries to determine the appropriate meanings of words used'in context. They seemed to be weak in other areas of dittionary use such as use of the pronoundiatiOn guides and use of guide words; Students in bOth districts had diffibUlty identifYing the Reader's _Cuitle to Periodical L terature as a source for finding the titles of magazine articles when the author was known and St. Croix district students exhibited difficulty in identifying an atlas as the reference one would use to find maps. thd St. Thomas/St. John district students achieved adequately on the following objectives: a) recognizing a word formed by a prefix or suffix b) identifying meanings of morphemes c) selecting the modal in the present tense d) choosing sentences which have identical meanings These students showed lOW achievement on these objectives: a) identifying literary concepts b) recognizing determiners that are used incorrectly C) indicating reasons why given sentences are ungrammatical d) identifying irrelevant words in sentences. In'the St. Croix district students achieved adequately on the following objectives: a) identifying meanings of morphemes 50 These students showed weaknesses in achievement on the other objectives listed above; 51 Grade 10 - TASK I Level The Stanford Achievement TeSt - Test of Academic Skillsi 1,6V-el I consists of three tests with the number of items listed beloW: 1. Reading 78 items 2. EngliSh 69 items 3. Mathematics 48 items The means; standard deviatiOns and the reliability estimates obtained for raw scores on these tests; as previously reported (see Bliss; 1982), are presented below Test Mean Raw Score Standard KR-20 Deviatibn Reliability Standard Error Total USVI Sample Reading 45.6 14.0 .93 3.70 s'_ ,.) English 32.0 14.6 .98 2.07 Mathematics 48.0 12.0 .92 3.38 St. Thomas St. John Sample Reading 43.6 14.2 ..54 3.48 English 32.i .6.9 .99 1.69 Mathematics' 47.6 10.8 .90 3.41 St. Croix Sample Reading 48.5:: 14.3 .94 3.50 English -31.5 8.6 .89 2.85 MathematiCS' 49.0 14.0 .95 3.70 54 Following are the item analyses and summaries of student achievement on the objectives evaluated by each of the tests. 1 53 R'gAdia Item Analysis Most items seem to discriminate at a satisfactory level. The exception to this statement will be examined shortly. Before doin4 this; however; it will be useful-to note that; as in ( the Readinq\Comprehension test on the Advanced Level given to eighth grade Stwients;.large:numbers of students seem to have been unable to attempt all the items on the test in the time allowed. Table 3 indicates the proportion of omits on each item in the Reading test. One difference from the eighth grade sample which is readily apparent is that the proportion of students omitting items in the St. Thomas/St. John district i8 considerably higher than that for the Students in the St; Croix district. Also, the difficulty indidet on items where a, great number of omits are present seem to be higher for St. Croix studentt (i.e. more St. Croix students scored correctly on these items); whereas the proportions of students scoring correctly on items where there few omits do not differ greatly between districts; From thiS it is clear that students in the St. Croix district are more likely to be able to attempt to answer later items on the Reading test than are students in the St. ThOMaS/St. John di-EiiCt and that thiS phenomenn results in generally higher scores on this test fbt St. Croix students than for St. Thomas/ St. John students (mean raw scores are 48.5 and 43.6, respectively). 54 Reading Item Analysis- Table 10 presents the difficulty and disetiMination indices for items on the Reading test. Most items seem to discriminate at a satisfactory level. The exceptions to this statement will be examined shortly. Before doing this; however; it will be useful to note that; as in the Readitig Comprehension test on the Advanced Level given to eighth grade students; large numbers of students seem to have been unable to attempt all the items on the test in the time allowed. Table 11 indicates the proportion of omits on each item in the Reading test. One difference from the eighth grade sample which is readily tppatorit is that the proportion of students omitting items in the St. Thomas/St. John distriet it considerably higher than that for the Students in the St. CrbiX ditttiet., Also; the difficulty indidet on items where a great number of omits are present seem to be higher for St. Croix students (i.e. more St; Croix students scored correctly on these items); whereas the proportions of students scoring correctly on items where there we--e few omits do not differ greatly between districts. From this it is clear that studehtS the St. Croix district are more likely to be able to attempt to Anter later items on the Reading test than are students in the St. Thomas /St. John diStriet and that this phenomenon results in generally higher scores on this test for St. Croix students than fbt St. Thomas/ St; Jc- students (mean raw scores are 48.5 and 43.6, retpectively). 57 Table 3 Proportions of the Tenth Grade Sample Omitting Reading Test Items (PercentS) 55 ITEM # US VI STT/STJ STX 1 0 4 1 1 1 5 0 6 4 3 2 1 10 11 12 13 4 14 1 0 15 1 1 2 16 1 0 17 2 2 18 2 19 1 20 Table 3 Continued 21 2 2 0 22 0 23 24 2 25 6 3 26 5 7 1 27 6 8 2 28 8 II 29 14 18 30 14 18 6 31 15 21 32 17 23 5 33 20 27 8 34 21 29 8 37 31 41 14 38 30 41 10 39 32 42 14 40 38 47 20 41 37 48 16 42 39 50 18 43 27 35 11 44 28 35 13 45 30 38 14 46 35 43 20 47 35 44 21 48 37 46 21 49 37 46 21 56 57 Table -5 Continued 50; 39 48 23 51 41 49 26 52 5 7 1 53 9 11 6 54 8 10 6 55 8 10 5 56 9 10 9 57 14 14 ' 15 58 9 9 8 59 14 14 14 60 14 14 14 61 18 19 16 62 13 14 13 63 18 19 16 64 20 21 17 65 17 19 14 66 20 21 15 68 19 19 18 69 21 22 20 70 20 22 16 71 23 25 20 72 24 26. 21 73 25 28 20 74 -24 27 20 75 28 31 58 22 76 30 34 22 77 30 34 22 78 33 38 24 Any attempt to explain these differences would be speculative at this time; but there are some pOSSible explanations whidh come to mind. The most obvious is that St.' Croix district students who took thi8 test are more generally academically competent than their St. Thomas/St. John counterparts. This seem Unlii:ely since other tests in the TASK battery do not exhibit this difference; Proportions of omits and raw score means are very similar (see Blibp; 1982); Another possibility is that St, Croix counselors gave students time to finith the test and this violated the standard diteCtions of the test. There is some evidence that this may have been the case; The examinees who were supervised by one counselbt on St. Croix consistently left out fewer items toward the end Of the examination than did the studentS supervised by the other counselor. In fact; the proportion; of students omitting iterit in the latter class was approximately the same ( or. slightly higher) than the proportion Omitted by the entire St. Thomas/ St. John group of examinees. WhateVet the explanation; dedision makers Would do well to view the results of the Reading test as somewhat tentative; particularly on items where more than 1/3 of the students omitted the item, The following discussiOn will limit itself to those items which were answered by 2/3 or more of the examinees. All items showed satisfactory diterimination indices except the following: 60 Items # 1, 3, 4, 22 - These proved to be very easy ite:is for ) USVI students (p..931 .95i .93, .94, reSpedtively), and low achieving students were very likely to score correctly on these items; Item # 64 This item seemed to discriminate satisfactorily among St. Croix district examinees but poorly among St. Thomas/ St. John examinee-S. The item required students to match the word "aspire" to the one Word among five that was most closely related to it. This was a moderately difficult item for St. Thomas/St. John students (p==.26) and for some reason low achievers were just at likely to get the item correct in this district as were high achievers. No patterns of incorrect answers could be observed. 1111 PO- t Skills Glabal_Meaming Students in the USVI sample showed weaknesses in their _ability to determine the main idea of a reading passage. In the St. ThomaS/St. John district the proportion of students _ correctly responding to the fOtt items with omit proportions under 33% was significantly and considerably lower than the proportions of the standardizatidn group scoring correctly on all items (mean p for the St. Thomas/St. John group was .41 versus .55 for the standardization 4roup). In the St; Croix district the proportion of students correctly responding to the six items -leasuring global meaning was significantly and considerably lower than the standardization 63 61 group in 4 of the items; The mean p for the St. Croix group was .44 versus .56 for the standardiZation group. Explicit Detail The USVI sample of examinees appearg 'to gave achieved well in the .area of detertining explicit details from reading pas- sages. Ox a third of the items (six out of eighteen) the proportion of local examinees scoring correctly was significantly higher than that of the StandardizatiOn group. On only four of the eighteen items did local. stUdent8 score significantly below the standardization group. Implicit Detail IMplidit meaning refers to'inforMation which is present in a body of text but is not explicitly Stated.. Students in the St. Thomas/St. John diStridt seem to do well on items requiring them to determine implicit details from a reading passage. Exceptions to this rule seem to be possibly related to the nature Of the reading passage. For instance they tended to do poorly (p=.36 and .45 vs 60 and .59 for the Standardization group) on items from a very long passage on sail boat safety. It is possible that reading passage length may be a faCtOr here since other paSSages were considerably shorter. St. CrOix district examinees tended to score similarly to St. ThomaS/St. John examinees with the exception cf the fact that they seemed to have less trouble with one of the two items listed above where 52% of the sample scored correctly compared to 59% of the standardization group and only 45% of the St; Thomas/ 64 62 St. John sample. Inference and Logical Analysis This section of the test evaluated students' abilities to draw logical conolusiOnt ftbit materials they have read and to make inferences concerning SituatiOn8 beyond the content of the rradings; Students in the USVI sample appear to be adequately proficient in these skills with a Mean group p value of ;58 versus ;55 for the standarizatiOn group. Meanlng_from_Context These items appeared at the end of Part A Of the test. BedauSe Of this, the proportions of omitted responses were extremely high indicating that many students did not have sufficient time to attempt to respond to these items (the mean proportion of omits for the entire sample on these items was over 34%). Thit phenomenon makes any attempt to interpret tnis data extremely suspect and, -consequently, no attempt will bE made here; Word Meaning This portion of the test evaluated standard English vocabulary. In genera , the USVI sample of examinees showed weak achievement in this area. St; Thomas/St. John district students' proportions correct were significantly lower than the correspond- ing proportions on the same items in all bUt One of the 27 items on this subtest. In many cases the differences were quite substantial with a mean p value of .51 for the St. Thbilie.s/ 65 63 St. John students compared to .69 for the standardization group. In the St. Croix district examinees did somewhat better; }Jilt Still showed weaknesses in vocabulary achievement. On 17 out Of 27 items on this subtext the proportion of St. Croix diStritt StUCent correctly responding was significantly below that of the StAdardization group; Again; differences wer quite substahtial in many cases; The mean p value for the St. Croix district studentS on these items was .58 compared to a value of .69 for the standardiZation group. English Item Anajas 111 items showed satisfadtbry discrimination except the following: IteM # 4 This item discriminated satisfactorily between high and low achievers in the St. Croix diStrict, but not in the St. Thomas/St. John district. ThiS was a very easy iteM for studentS in both districts (p =.94 in bOth) with the result that most studenti correctly answered it. This accounts for the low diStrimination:values obtained. Item 4 12 - This it zti. did net discriminate well on St. Croix. fact; more students in the low achieving group scored correctly on it than stUdehtt in the high achieving group. It is a moderately easy item (0....8o on St; Croix); However; ah examination of the incorrect response pattern failed 66 E4 to show any reason for this phenomenon. The item asked the student to choose the situation when the term "love ya" would not be appropriately used; It provided the following choices: 1) in conversation with a friend 2) in conversation with an older person 3) in a letter to a friend 4) in a letter of application for a job Choice # 4 was considered the correct answer. It Was noted that high achieving students who responded incorrectly were no more likely to choose one distractor over another. Item # 30 This item discriminated poorly for examinees in the St. Croix district. It was a very easy item for these students (p =.94) and this should account for the low discrimination index obtained; item # 48 - This relatively easy item (p.,,;87) did not thscriminate satisfactorily between high and low :Achieving students in the St. Croix oistrict; From the three word; "askt"; "trouple"; "srtion ; and 'exspect" examinees were asked to choose the number of words spelled correctly. Low achieving students who scored incorrectly on the item showed no incorrect response pattern, but high achieving students who missed this item all chose "3" as the number of correctly spelled words. It seems that there is one of these two misspelled words that was not obviouSly misspelled even to students who spell well. 67 65 Itemt #53 and 54 were very easy for St. Croix studentt (1)=.97 and .90; respectively) and this should account for the LOW discrimination levelt observed for these items; Summary of Student_Skills Learning Skins_ St. Thomas/St. John schddl district examinees showed weakhettet in the following areas of dictionary usage skills: a) identifying a word that contains a given vowel sound b) the thahqo in principal stress in two forms of a word They appeared to have achieve- adequately on the other dictionary skills which were tested. St. CroiX examinees achieved adequately on all dittionaty use skills tested except the ability to identify the change in the principal stress in two forms of a word. While USVI students soon to be abIe to identify a reference to aid vocabulary development, they seem to have great difficulty in identifying a reference source for articles in periodical:S. The Students in the usvi sample showed. satisfactory achieVeMent in objectives dealing With appropriate English usage; They were particularly strong in objectives having to do with identifying the appropriate usage of slang exprestions and with identifying the action wordt in a sentence. Usage Conventions Examinees in the USVI sample thOWed weakness in their knowledge of the correct capitalization conventions for seasons 66 and common nouns. St; Croix school district students exhibited adequate achievement in the choosing of appropriate Conuna usagei but St. Thomas/St. John examinees showed weakneSSPS in the proper use of commas between items in a series and Of the use of commas after introductory words. St. ThOMat/St. John examinees showed some weakneSSe8 on items which measured the appropriate use of verb tenses; but showed particular strength in dealing with objectives concerning -; proper use of pronouns and adjectives. St. Croix school district students showed adequate achieve- r. Ment in all the areas listed above. Spelling USVI students showed particular strength in spelling; On all but two of the 15 spelling itemsi the proportion of USVI studentS scoring correctly was not signifiCantly different or was significantly higher than that of the standardization group The mean p value for spelling items of .82 compares favorably with the value of .81 obtained by the Standardization grour Sentence Sensitivity item. c-:aIuating'sentence sensitivity 2.erate by presenting t students with four similar sentences which eiffe: pri.mrilyin terms of punctuation and syntax; The student it .77.e.od th choose the sentence which 'expresses the idea LeSt." USVI examinees showed strong achievement along this 62 objective with a mean p value of ;82 on these items compared. .81 obtained by the standardization group. Paragraph Arrangement This section of the Ehi4ligh test evaluates the examinee's ability to organize a logical paragraph by ordering four sentences which are presented into their proper sequence. USVI students showed adequate ability to accomplish this task with a mean p value of .73 compared to a mean p value of .77 on these items when attempted by the standardization group; ,- Matheirtics All items appear to discriminate adequatel with the follow- ing exceptions. Items 1, 2, 5" and 30 == These were all very (=,t;ty items for the students in the USVI sample (p=;92 .94, .92, and .90, respectively). As a result of this; all studentt regardless of general mathematiCs achievement level, were able to score correctly on these items. IteM . 11 This,was a moderately difficult item (p=.63) which asked students to choose the name which "best desoribet a lbSed figure with four congruent sides and four con= gruent angles." The choices were, it.ectangIe"i "rhombus", . . _ "quadrilateral";-and "square." It was notec that examine-et, particularly the high achieving group, 'who missed this item tended tb choose "quadrilateral" rather than "square" as the correct answer. The described figure is, indeed, 70 68 a quadrilateral. It is also a rectangle and a rhoMbus. It is obviously the intent of the author of the item that examinees realize.that the four sides are equal (congruent), the angles are equal and must each measure 90 degrees,. and therefore,the opposite sides must be parallel. The tequitement that the student choose the "best" descriPtibh may be ambiguous. ft is possible that some students may not know A-711at "best" is in thit case. The item author obviUtly means, "most pretAse", but "best" Co!..ild also mean ''most generalizable ", which would make "quadrilateral" the correct answer. It Is not certain whether a student choosing "quadrilateral" did so because heir she was unable to go through the thought process described above or because he or shE defined "best" in a different manner than the author. The item may be ambiguous be-catise of this and it is uncertain what information concerning student achievement it provides; Sumr:iaty of Student SkiiIs Numbers, Symbols, andSets Students in the St. Tnomas/St: John school diStrict Showed adequate achievement levels in the following objectives: a) use of expanded notation b) converting standard numerals to word form t) determining place value d) defining "prime number" 69 They showed weaknesses in a) determining relationships between two sets b) determining the value of a linear expression with two unknowns' when given the values of the unknowns t) interpreting Venn diagrams. St CroiX school district examinees demonstrated adequate achievement in all the objectives listed above with the exception ,L identifying the relay Jnip between two sets. Number Properties and Operations Whole Numbers St Thomas/St; John schbbl district students showed adequate achievement levelS oh the following objectives: a) multiplication of common fractions b) reducing fractions tc their lowest terms C) locating decimal points in products d) subtracting decimal fr7actibhS These examinees showed weaknesses in achievement on the Following objectives: a) finding decimal equivalents ofsfraCticn b) addiI common fractions with reducing c) subtracting common fractions d) determining relatiVe values of fracticns e) adding mixed nUMbert St. Croix school distriet students demonstrated adequate achievement on the following objectives: a) multipl]cation of common fractions 70 b) reducing fractions to lowest terms c) locating decimal points in products d) subtracting decimal fractions They showed weaknesses in achievement in the following areas: J.) finding decimal equivalents of fractions b) adding and subtracting fractions c) determining relative values of fractions d) adding mixed numbers Nmbex_Properties -an-d- Oper=ations- Integers and Exponents St. Thomas/St. John students exhibited adequate achievement on all objectives in this section of the test except determining the distance between two points on a number line. St. Croix school district examinees showed adequate achievement on all objectives on this section of the test except the ability to add negative integers. Mathematical Sentences USVI students showed adequate achievement on all objectives evaluated in this section of the test; eagnremPnt St; Thomas/St;-John school district examinees showed weaknesses in determining the meaning of metric prefixes while Sty Croix district examinees achieved adequately on both objectives in this section of the test Ratio and Percent USVI students tested achieved adequately on all objectives dealing with ratio and percent except on the objectiVe of finding a total when a part and its percent of the total is given; f,raphsi probability; and Statistics Students in both USVI school diSttiCtS showed adequate achievement on the following objectives: a) identifying percents represented on a circe graph b) finding averages of groups of nu dbegs. These students showed weaknesses on the following objectives: a) computing percents represented on circle graphS b) computing the amounts represented by percents on circle graphs. Mathematical Reasoning St. Thomas/St; John district examinees showed adequate achievement levels on the following Objectives: a) solving three step problems involving money b) determing missing data needed to solve a problem. They showed weaker achievement on theSe objectives: a) stating the solution of a word problem as a mathematical expre:;sion b) SOlving word problems by deduction St. CroiX ;JiStriCt students achieved at adequate levels on all of'the objectives listed above with the exception of the ability to state solutions to word problems as mathematical expressions; 75 72 Grade 12 - AASK II Levol 73 The Stanford AchieVetent Test Test of Academic Skills; Level II consists of three tests With the number of items listed below: Reading - 78 items 2. English - 69 items 3. Mathematics - 48 items The means, standard deviations; and reliability estimates for the raw scores On these tests, as previously reported (see Bliss; 1982) are pre_ tinted below. Test Mean Standard KR-20 Standard Raw Score Deviation Reliability Error Total USVI Sample Reading 43.9 13.8 .93 3.65 English 25.3 8.3 .91 3.01 Mathematics 46.8 11.2 .87 3.36 St.__Thomas/St-_.__John Sample Reading 40.6 12.3 ;91 3.69 English 24.6 7.4 ;90 2.98 Mathematics 45.6 10.9. ;84 3.45 St. Croix Sample Reading 48.2 15.2 .95 3.40 English 26.3 9.4 .91 2.97 Mathematics 48.4 11.5 .90 3.44 aV Following are the item analyses and summaries of student ze"Eievnent on the objectives evaluated by each Of the tests. 77 74 75 Reading Item Analysis As'on the tenth and eighth grade levels; it was noted that a high proportidh of studentsomitted later items on the reading test; Table 4 presents the proportion of examinees omitting each item; It should be noted that items 52 through 78 make up Part B of the test and that these items are timed separately; This accounts for the drop in omits leginning with item 52; Again; this phenomenon very likely indicates that large numbers of U.S. virgin Islands students did not\have time to attempt later items on the test and the Vafldity Of\the resUlts on these Iatei items is definitely qUeStionablABecause Of this; \ _ only items which less than 33% of the examinees\dmitted will be considered in the item analysis and summary of student skillS. All items whiCh 67% or more-of the examinees did not omit discriminated at a satisfactory level except the following items. Item # 1; 3; 9, 11, and 15 - These were particularly easy items for the USVI sample (p=.95; ;84, .94, .83; and ;97, respectively) and low achieving studentS were just as likely to score correctly on these as were high achievers. ThiS accounts for the loW discrimination indices. 76 Table _4_ Ptopottions_of the Twelfth Grade Sample Omitting Reading Test Items (Percents) ITEM # USVI STT/STJ S TX 9 5 6 7 10 11 12 13 14 15 16 17 18 19 20 21 22 1 1 2 2 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 2 77 Table 4 Continued 5 7 23 24 8 4 25 7 5 9 26 6 5 27 9 9 28 10 9 11 29 14 15 13 30 14 15 13 31 14 15 13 32 16 16 16 33 16 15 16 34 18 16 20 35 18 16 20 36 23 23 24 37 29 27 33 38 29 24 35 39 30 26 36 40 35 34 36 41 33 28 38 42 34 30 40 28 30 25 44 31 31 31 45 32 31 33 45 32 31 33 46 34 34 35 47 37 35 40 80 Table Continued 37 38 38 39 35 34 34 35 78 40 44 44 44 48 49 50 51 52 5 53 10 14 5 54 6 9 2 55 9 11 5 56 8 11 4 57 7 9 58 6 9 2 59 7 II 60 9 12 61 11 16 62 10 16 63 12 19 64 10 15 4 65 13 19 66 17 24 67 15 23 4 68 16 24 5 69 16 24 70 16 24 7 71 19 24 13 72 15 23 4 73 16 24 4 81 TablO 4 Continued 79 74 16 24 4 75 19 26 11 76 17 26 5 77 19 26 11 78 18 28 4 80 1:e:7 18 This moderately difficult item (p=.63) showed elrticui,;Iiy IOW discrimination in the St; Thomas/St. ,L.)hn saran: (d=.05 With p=.57) . The item involved "subject" as it was used in reading Passage about music; the response 'object" was chbeh by most of the high achieving students who missed thiS item rather than "theme;" 'which was the correct response. Low achievers who scored incorrectly in the St. Thomas/St. Jbhn sample answered more or less randomly; High achieVing students who were not Certain Of the correct answer may have been distracted by the altorr,itive "object" since it is used often in lanQuage in connection with the term "subject." Low achieving students were likely act to hae perceived this reIationShib and to have answered more randomly with some students chobSing the correct response hy chance; This could account fbt the low discrimination observed in this item. This was a very diffiCult it 'm for St: Thomas /St. Jah:i students (p=;07); but was soM4hat easier for St Croix school districtstudents (0=.22). The poorer discrimination in the St. thOnia. JOhn district resulted from the fact that fewer high achieiing students in that district = cored correctly on this item. The proportions of loW aChieVing students scoring correctly was approxi- mately the same in both distracts; The item asked examinees to ChObSe ar eamp,e of a "geographic building 8J m-Fiterial"; defined in the reading as a material which "uses local resources . . . and suits the local climate." Choices were: a) stone for a pyramid in Egypt b) snow for an igloo in Northern Alaska c) adobe for a museum in New York City d) a steel girder for an office building in Pittsburgh The response considered correct was "b." Mbtt St, Thomas/ St. Johh dittriCt students chose "c" as the correct response while thb-se St. Croix examinees in the high achieving group who s)red incorrectly chose more or less evenly between "a" and "O." Adobe is the only material specifically identified F'7 a geographic .buiIding material in the passage (although only ir cf once to to a community in New Mexico and the "hot; dry scut. ,:ern part of the United States." Students may have seen adobe in the passagc acrd chosen response "C" on the basis of this; disregarding the location of the museum. On the other hand; they may not have been aware of New Y:.rk City geography and climate;' Neither explanation takes into account the differences observed in the behavior, of the St; Croix sample. In 9enetal May be a poor item for a number of reasons. requireS examinees to haiie a strong knowledge of .lecigralbhic facts n--)t given in the readings. This may make it a good geography test item but it is uncertain whet:ler a student a:swering incorrectly did so due to lack pf geographical %nowledge or lack of reading skilIG. 84 82 Tn addition; the item may_ really haVe tWo correct answers. one for a pyramid might well be considered a geographical building material since it is natural to Egypt and suits the local climate there. He: e, the results on this item should be examined with caution. Sumary of Student Skills lobal. Meaning Students in the USVI sample showed some weaknesses in the area of meaning determination. However, they seemed to_score well on earlier; easier items. Examinees Seem to score better, in absolute terms and in relation to the standard- iation groan, on earlier items dealing with global meaning. On the fii;ur teems with OMit rates of less 'than 33 %, the roan p value wczs .56 compared to .65 for th standardization group. Meaning of Explicit USVI students appear to have achieved quite weid on objectives dealing with determining explicit meaning from readings. The mean p an value for the It.z,ms measuring these Skills gas .74 compared to .75 for tl, standardization group. On three items the proportion of USVI students scoring correctly was significantly greater than the proportion Of the staiiJard- ization group. meaning of Implicit Detail The studentS In the USVI sample seen to have achieved adequately in objectives dealing with tne determinatioh Of Tracts which are iMplitit in readings; On items which an 85 83 appropriate number of :ittia-ents were able to attempt to respond to tne mean p values , .67 in the St. ThOmas St. John district and .74 in tFie St. Ci-oix district comared to .74 in the grcup; Again. it was observed that local students tend to do less well on items ocurring later in the test. Inference and Logical Analysts_ USVI students ShOWe-1/4.1 weaknesses in the area of inference and analySiS, Cf the items were appropriate t o eyamine; the rrle, p va3.ues were .511f.).7- the St; Thomas/ St. John sc1lool and for C.:oix school district compared to .64 for tne Ftandaiaizaton group. While the St. Croix grols auhie ement level could be Said to be adeclU there shculd be some concern ,Ibdut the level of achluvent,:Tit7 n the i Thomas/St. John distr_let. Wurd Mean is Wdrd mean involves vocabulary skins. Pere, market, iifferences were noted between examinees In the two school stricts. St CrOiX district students showed c:onsiderably higher achrevement "mean p value E. .68) than Lite St. ThOM6S/ St. John district examinees (p,i..52). It was noted that the Thorna/gi--; John examinees rig tcd items considerahIy more often than St; Croix examinees, For example, 11 of thi. St. Croix examinees -.)mitted the last item #78) on the test compared t 28% in C:ie St. Thomas/St. John district. This difference seems to be due to one St. Thomas/St. John class 8f; of the examins omitted the item compared to l4 it the othet. Sinte scores on the other parts of the battery are not greatly different between these two clasa,=, it is possib-1_e that he was a lem in test adminis in the former claSs; Such thin s inad-quate instruct!on on this part of test or error in timing could be the cause of this pheholion At any rate: the St. Thomas/St; John scores valiai are oper to question. The St. Croix examinees show good achievement on the items with their Mean p value (.68) comparing favorably with the standardization mean p value of .70. English Iter Analysis All ite:As discriminated adeqUate17? eci:tit the following: 3; 3i; 37; 42; 46; and 52 These wGre particularly it,is for the USVI sanple of examl:.ees (p-=, .96. .93, .96, .96; 6; and .92, respecitive: aG.i:unt for the io aisciimination This ,17.-ould observed on thG:. _;like low achieing students wer_ almost as ltkely to score correctly on these items as were high achieving :Ltd n c L .)eau:,e they were Frasy. tem 18 Thi' iLet Auat tor the St. thai6 st. J(i.nn fchool district &.kanin-ci: , bu poorly. for '0:16 t.c:-oix examinees. This was due to the fact that St; Cro;' examinees found ikie item much easier (p=55) than the St. Thcmas/St. John examine-6S (p=.86) and this produced the ldi.4 level of discriminatien typically found on items with high difficulty indie6S. It 54 - This was a moderately easy item for USVI examinees (p=;83): It was difficult to see why this item disoriminare(i as poorly as i i did. There appeared be no pattern of incorrect responses and no extraneous clueS in the item itscl. The item appeared to be clear and worded Simply. Summary Of Student Skills Fearning Skills On Objectives concerned with dictionary usage HSVI studes '6.-sted appeared to have difficulty using the rn-mutiti-ciAtL n key to intify words containing vowel sounds; 8E: CrbiX. examinees Fhowed some weakneSS their ability to locate entry words in a dictionary. On all other. dictionary skills tOrtt-ed; tIte USVI sample Showed adequate achicvemen Students in the St. Crei:;: sampL showed exceptionally s rbhg achievement o- objectives dealing with the use of reference sources; In tact students in this simple had proportions answering correctly significantly higher than the standardiatiOn group on all tii6e object St. '1'h-Otiia8/ct. to -r.hieved aduqw:teiy C ai_ objecLives de7'ilind with the use of referene, sources except the ability to identify the use of a hhesaUtus. 86 On bjectives dealing with knowledge of the use and structure of English USVI examinees showed strong achievement on the following: a) idenLifi-ati^n of a prepositional phiage b) identification of an adverb forming morpheme They deminstrated weakne'sSeS OP the following objectives: a) identifying an appropriate place for slang expressions b) identifying an appropriate Closing for a formal letter _ C) identif:2,ng a standard EncT1111 sentence d) ideotifving a sentence containing a prepositional phrase. usage convritions CapitLlization Examlneet in both districts shoWed weaknesses in their knowledge of ca:pitalization conventions for common nouns; In qenerali they tended to capitalize commor r uos that should have been written &,:ing leer case Initt,1 letters. St. Thomas /St. John district stuu.1 is showed weakness;_ _ in the knowledge that natiOnaiities shou7d be capitalized While St. Croix district students appeand to have achieved aleqoat,. y along tj..s oojective. EXaM_Te(:..; in both distiiCtS aLpeartd to have little diffiedltV recooni7ing that dayS Ot the t--k should hr - Puncr,.s'Lion .XEth -he ...YT -mple 5h6wnj aden,late achievement 87 on all items testing the appropriate use of punctuation. F--,:age Con ent ,s Parts o -,ech St. Thomas/St; John schobl dis-Aitt examinees demonstrate adequate achievement on all objecti. deling with usage parts of speech tested with the exce.pti,, of: a) verb redundancy b) the use of a participle St; Crci- school district students 3hOtvd adequate of achiment on all these object -ives except the use of a participle. Spelling USVI examinee F; Showed adequate achieVeMAitj_n spelling With p value of .70 compared to .72 for.the Standard- izat On t-he lc itrst8 testing spelling sxills ; -Agnificane.y nig,;er proportions scored correctly than the Standardization g :oup on three ite and :;ignificantly lower prop-dr: correcly on only three it(uris; Sonta_ce Studeti the sample achieVed well on items designed to teat the ob3ectives. Their prOjortions scored correctly were as high or higher than the Staidardization groul Gn all itoros Wi,:h a mean p of ;77 co:r. a '-cd to .76 for the st;.indardization group. Paragraph Arrangemel::- USVI examineeL: achiel;ed adaquatr'ly on c'hie-;;;tives designed k to test their abillties to arrange paragra hs with a mean p value of .61 compared to .67 for the stan4 ar-ILzatiOn group. CCU Mathemat Item Analysis All items discriMinated Leguately with the following exceptions: Item # 1 and 2 = ThS6 were particuIarl,J sy items for the USVI sample and .88 respectively). As a result of this phenomenon it was apparent tha IOW achieving students had almost the same probability of scoring correctly on these items as did high adhiVing students This would account for the low discriMination power which was obserVed. Item 4 44 =- This was a particularly difficult item for student8 in the USVI sample (p=;12); ThiS itt. is notable becauSe Of the moderate negative rimination in the St. Croix group However, it was nc tat 45-6 of the 2t; Thomas/ St. John examinees and 27% or the St. Croix examinees emit.Led this itetti (it was near the end of the 48 item test) 1.-t is believed that any t.:) interpret these z °esults, ma, be reading more validity into the item than is warranted. on the reading test; there may be a timing difficulty involved with this test; although it appears less apparent witn this teSt. For instance; thti. rise in the proportion of omits L; net nearl as smooth as the rise on the reading test (See Table 5). It seems as if studeLits ilave had a chance to attcTupt most items; bu+- left those they had diffic iinSWei[ng blank, This Table 5 Proportions of the Twelfth Grade Sample Omitting Mathematics Test Items (Percents) 89 ITEM # USYI STT/STJ S TX 1 0 0 0 2 0 0 0 0 0 0 4 7 5 9 0 0 6 0 0 7 4 9 8 0 0 0 9 5 5 4 IC 3 7 11 0 0 0 12 0 13 0 14 15 2 4 0 1 17 2 4 18 5 19 1 0 '7 21. 22 2 90 Table_5Colitinuez1 23 0 24 1 2 25 1 0 2 26 5 27 28 29 7 30 4 5 31 15 32 33 5 5 5 34 4 5 2 35 5 2 36 5 2 37 8 5 38 11 12 9 39 15 40 23 28 16 41 19 24 42 13 27 5 43 20 31 5 44 37 45 27 45 32 41 20 46 29 43 11 47 '6 39 7 48 33 46 15 91 seems apparent, fOr instance; in the St. CroiX sample Where 27% of 4:L=:aminees omitted item number 44, but only 7% c!r:7;_ :Htem # 47. Again; a larger number of omits was Observed for the St. Thomas/St; John sample than fer the St. Croix sample. Since the mean number correct fOr these two samples differed by only 1.7 points (See Bliss, 1982) it is unlikely that the differen,fes in Omits were due to differences in math ability between groups. Instead, as in the reading test, it is more likely the differences observed were due tb administration probleMS in the St.Thomas/St. John sample. SummarT__of_Student Skills Nur-lb_S and Sets Examinees in the St; ThomaS/St. John school distrt. snowed adeqthte achievement on the following objectives: a) identification of place ValUe of a digit in a decimal b) identifidation of the reIationShp between two Set, They showed weakneSS5?s on tie fallowing Objective ) rounding decimals- intarpreting Venn diagram., c) it tetp.L.E:ting abol.ttO values of e number. St. Croix district examinees showed precisely he opposite strengths and weaknesses. They appeared to show adequate achieveMent on the three objectVeS on which the St. Thmas/ St; 3.61in examinees ere weak and to show weak achievement on 92 the two objectives on which the former group Df examinees displayed adequate achievemnt. NUMber Properties and Operations_- Whole NUmbers St. Thomas/St. John district examinees displayed adequate achievement on ;fill objectives in this grouping; St. Croix district students showed adequate achievement the following objectives: a) adding a broken column of three numbers b) subtracting with renaming C) multiplying involving zeroes d) multiplication of prime numbers e) application Of the Multiplicative property over addition. They showed weakness in the area of identifying of prime numbers. mPmber Properties and Operations Common Inc; Decimal Fractions St. Thomas/St; John district examinees showed adequate achievement on all objectives this area except: a) adding fractions with reduction estimating the quotient of decimal diVisien c) identifying the least common denominator Of liter,t1 fractions St. Croik district examinees showed adequate achievement on all 5bjectives in this area except: a) adding fractions with reduction 1-0 estimating the quotient in decimal division 95 93 c) subtracting common fractions with unlike denominators after renaming. Number Properties and Geratioi - Integers and Exponents USVI examinees showed adequate achievement on all objectives tested in this area; Mathematical_Sentemoes_ St. ThoMas/St. John school district examinees showed adequate achievement on all objectives tented in this area except the ability to translate verbal prot:lems into mathematicaI equaions. Croix district students showed adequate achievement on all objectivc3 tested in this item grouping Geometry and Mr,asurement There wore subr':,,etiaI differences in achievement this area between exam...nees in the two school diSttiCtS. St. Thomas/St. John. district students di:;played adequate achievement on the ability to approximate the inglish equivalent of a lihbar tOttl-c unit. They showed weak evement on the fcIl- Hectl.ves: itifyng characteristics of commoh polygons ,rm:',ning the relatiV areas of potygohs given their dimensions c) naming the ang.:.c that result from e. setting a right angle . d) solving one step word problems that .nvolve convering: between English linear units. 94 St; Croix district examinees showed particularly strong achievement on the following objectives: a) identifying characteristics of common polygons b) naming the angles that result from bisecting a right angle They showed adequate achievement on these objectives: determining the relative areas of polygons; given their dimensions b) approximating the English equivalent of a linear metric unit These students showed weakness in solving one step woid problems that involve converting between engiisn linear units. Ratio and PPr_rent St. Thomas/St. John district examinees displayed adequate achievement on 411 Objectives tested in this area. St. Croix diStriet Students showed adequate achievement on the objectives tested in this item grouping except the ability' to solve rate problems. Graphs ;Probability; and Statistics St. Thomas/St; John district examinees demonstrated adequate achievement in the ability to use tables to make comparisons. They showed weaknesses in the following areas: a) solving word problems by finding the average two numbers b) finding the average of positive and negative numbers solving word problems by interpreting a table to find percent success 9C5 d) identifyinr the prObability of a particular eVt-4,I't occurring; TWO Other items had omits aboVe 40% and no attenpt will be made to derive information from theM. St. Croix diStriet examinees showed adequate achieveme:n't on the following objectives: -a) finds the average of a positive and negative numbcr b) using tables to make comparisons C) choosing a set of ordered pairs that would be on the same line as two given points; They showed weaknesses On the follbing objectives: a) solving word probleMs by finding the average of two nuMbers b) solVing word- problems by interpreting a table to find percent of success c) identifying the probability of a particular event occurring d) identifying coordinates on a graph off a linear equation e) choosing a set of ordered pairs that would be on the same line as two given Mathematical Reasoning USVI examinees displayed adequate achievement on all objectives tested in this item group. In the St. Themes/ 8t. John sample, one item had 46% omitted reSb6nSeS (item # 48) and was het US-0d in this analysis. 96 Discussion A detailed summary tudorit basic skills across grades and recommendations based on the results of the achievement testing will_ of necesSitY, have to be put off Until the results of z,1:7.3ntary school testing are analyzed and reported. Nevertheless; there are some facts brought out by thf..analytiS of the secondary school- examinee scores which are worth, at leaSt tentatively; touching upon. Most striking is the time situation on the reading comprehension tests. On all three levels tested it was obvious that many studentS did not have sufficient time to attempt all the items on these tests. -Teachers and counselors were instri :tea to Strictly adhere to the time limits SP-edified by the test publishers. These time had bean empirically determined to be sufficient for all studentS to complete the test during the standardization stuies described in Technical Report # I. It appears that those time limits may no'. be appropriate for USVI students who may 1.),: slower; more deiberate readers. This characteriStid of local students is not necessarily undesiruble provided students end up_understandina what theY read. Nevertheless, it tray' account for low rea ling scores since most standardized tests count'omits as incorrect answers. FrOm a technical point of view, the result of this phenomenon may be that students arc taking a timed test rather than a power test and this brings up numerous problem; in score interpretation. 97 It is notable that time seems to be sufficient on most other _ 7_ests. No_attempt is made here to describe the causes of this phenomenon; Many alternative hypothesis present themselves; HoweVer, it is important that research be conducted to establish the nature and causes of this difficulty. Second; there appears to be a pattern that can; at least tentatively. be observed in student achievement. USing the taxonomy of edUbatiOnal objectives in the cognitive deMain (Bloom; 1956) as a theoretical framework; some sense can be made out of this pattern. -U.S. Virgin Islands students generally appear to achieve at least adequately on- objectives that would be classified in the lower three categories of the taxonomy (i.e.-knOWledge;_comprehension; and appliCatioa). They achieve leS-fJ well on objectives that would be elaSSifi2d in the upper three categories (analysis synthesisi and eVa uation). Again the reading comprehension tests exhibit this phenomenon quite clearly. USVI examinees achieve adequately on items which test their abilities to understand facts that tte explicitly stated in the text (comprehension level object veS); bUt are weak iA determining implicit, global; and cont!xtual Meanings (analysis and synthesis objectives). Those stAdents seem to achieve well on objectives which can be achiev_:d by memorization (knowledge objectives) or the application of a general procedure; rule or algorithm (comprehension and application level); They scored extremely wall in spelling and Singl:.) step mathematical computations. They scored lower on itemS reqiiring 98 them to organize data; choose information relevant to solving problems, and to cheese a series of rules and/or algorithms to use in 'solving problems (analysis; synthesis; and evaluation objectives) . Note the weaker achievement in mathematics 4 applications problems; multistep mathematical computations) and the use of dictionary entries; for example; Again, there is no attempt here to determine the causes of this pnenowenon; but if analy8is of the elementary achievement tests substantiates this tentative observation, further research into this phenomenon is important. Third, U.S. Virgin Islands students in the grade levels tested displayed weakness in their knowledge of standard English vocabulary. Finally i there is the issue of test taking Skill8. one particular phenomenon was very apparent. Many examinees omitted items which they had a chance to attempt to answer. Although t was not explicitly stated in the directions; th_ score students received was the number of items they answered correctly. There was no correction for guessing and an omit was counted the Same as a wrong answer. This should have been explained in the directions and the lack of such an explanation shows _a oeficit in the test publishers° thorouginess in test con- struction. Nevertheless; most tests students take use this "number right" scoring procedure (exceptions are the Preliminary Scholastic Aptitude Test and the Scholastic Aptitude Test which are not given earlier than grade eleven) and the best strategy 99 for examinees wishing to maximize their scores when confronted with an item they did not know the answer to would have been to try to discard alternatives which they know were wrong and to guess randomly from the remaining alternatives or; if they could identify no incorrect answers, to simply take a random guess. Many examinees seemed to be unaware of this strategy or unwilling or unable to use it. 100 References Allen, M.J. & Yeti, W.M.Introduction_toneasurement theory. Monterey, CA: BrOOkS/Cole Publishing Co., 1979. Bliss, L.B. Virgin Islands of the United States public school ha'fc skills achievement survey; Technical_Report #1: Validation of the use of the Stanford Achievement Test with_ U-S.V-T. students._ St. Thomas, USVI: Caribbean - Research Institute, College of the Virgin Islands; 1982. Bloom; B. (Ed.) Taxonomy of educational objectiv_Randbook I: Cognitive_damain_._ New York: Longmans, Green & Co., 1956. Cooper, V. Personal communication, June I, 1982. Cureton, E.E. The upper and lower twenty -seven per cent rule; Psychometrika, 1957,72_, 293-296. Kelly, T.L. The selection of upper and lower groups for the validation -or test items. Journal of EduCational_Psychology, 1939, 30; 17- -24.