Being Smart(er) about the SMARTER Balanced Assessment System

By Nicholas F. Dus­sault, WASCD Board of Direc­tors, ASCD Leg­isla­tive Pol­icy Committee

By nDussaultow every­one should know that the cur­rent Wis­con­sin Knowl­edge and Con­cept Exam­i­na­tion (WKCE), pur­chased from CTB McGraw-​Hill, will be replaced by an assess­ment cur­rently being devel­oped by the SMARTER Bal­anced Assess­ment Con­sor­tium (SBAC). The assess­ments will stand in for the No Child Left Behind (NCLB) fed­eral account­abil­ity sys­tem, test­ing in Math­e­mat­ics and Eng­lish Lan­guage Arts (ELA), begin­ning in 201415. Stu­dents in grades 38 and grade 11 will be tested.

SBAC is a con­sor­tium of 28 states awarded a fed­eral grant to develop the next gen­er­a­tion of NCLB tests. Wis­con­sin elected to join this one of two win­ning con­sor­tia and is a gov­ern­ing state in SBAC. You may have heard of the other con­sor­tium, The Part­ner­ship for Assess­ment of Readi­ness for Col­lege and Careers (PARCC) which includes our neigh­bor­ing state, Illinois.

The SBAC assess­ment sys­tem will be pri­mar­ily com­puter adap­tive to start and will move toward almost totally com­put­er­ized as tech­nol­ogy allows. They will be extended response per­for­mance assess­ments com­pleted on paper that in its early years that will require some teacher participation.

Read­ing the SBAC pro­posal reveals that the assess­ment sys­tem has three com­po­nents: First, an end of the year sum­ma­tive assess­ment that is com­puter adap­tive; sec­ond, com­puter adap­tive interim/​benchmark assess­ments; and third, sup­port for a classroom-​based for­ma­tive assess­ment. Only the sum­ma­tive assess­ments will count for account­abil­ity pur­poses. The interim/​bench­mark assess­ments are finer-​grained ver­sions of the sum­ma­tive assess­ment with sim­i­lar items and cut­off scores that a school may (or may not) take at their dis­cre­tion, as many times as it wishes. The pro­posal promises on-​line sup­port for classroom-​based for­ma­tive assess­ment con­sist­ing of guides, train­ing and rubrics. The three parts are “inte­grated” but not nec­es­sar­ily “balanced.”

The state account­abil­ity test­ing sys­tem is not going away but is being replaced by a new sys­tem, admin­is­tered in a dif­fer­ent way, to mea­sure new stan­dards. If any­thing can be pre­dicted it will be a greater chal­lenge to Wis­con­sin schools than the sys­tem it replaces. These changes need to be heeded, stud­ied, accom­mo­dated, and pre­pared for if Wisconsin’s schools wish to remain as suc­cess­ful on NCLB assess­ments as they have been in the past. Although the test is not yet avail­able, one can make some pre­dic­tions about what the test will look like by care­fully read­ing the SBAC pro­posal and review­ing the steady stream of doc­u­ments being pub­lished by SBAC.

Challenges facing Wisconsin’s schools

First, the SBAC assess­ment will explic­itly mea­sure the new Com­mon Core State Stan­dards (CCSS) which are more rig­or­ous, demand­ing, explicit, and focused than the exist­ing Wis­con­sin Model Aca­d­e­mic Stan­dards. The assess­ment is being designed to mea­sure the extent that stu­dents are “col­lege or career ready,” or on track to do so, instead of being based on some vague notion of “pro­fi­ciency.” The bot­tom line is that the tests will be based on fewer, sig­nif­i­cantly more demand­ing stan­dards, and have a more spe­cific focus.

Sec­ond, there will be a dif­fer­ent mech­a­nism for set­ting cut­off or pro­fi­ciency scores – which are the scores that stu­dents, and groups of stu­dents, must attain to be suc­cess­ful on the test. Under the cur­rent WKCE, the Wis­con­sin DPI, with the assis­tance of Wis­con­sin edu­ca­tors and tech­ni­cal experts, set the score lev­els that rep­re­sent basic, pro­fi­cient, and advanced on the cur­rent assess­ment. They are, by all accounts, one of the eas­i­est set of cut­off scores in the nation. As a result, Wis­con­sin has one of the low­est rates in the nation of iden­ti­fy­ing schools and dis­tricts as not mak­ing ade­quate yearly progress.

This will change with the new SBAC assess­ment sys­tem. Wis­con­sin will have to agree with, or fol­low the con­sen­sus of, the 27 other states in the con­sor­tium when set­tling cut­off scores, and the process will be sub­ject to greater scrutiny than the cur­rent sys­tem was. It is most likely that the SBAC consortium-​wide cut­off scores will be sig­nif­i­cantly more demand­ing than Wisconsin’s cur­rent cut­off scores. In all like­li­hood de facto national cut­off scores will emerge as it is most likely that the cut­off scores will be com­pa­ra­ble to the cut­offs scores of the other major con­sor­tium, PARCC. The net result will be that Wisconsin’s his­tor­i­cally low expec­ta­tions will be sig­nif­i­cantly raised, and will allow for direct com­par­isons with other states which cur­rently have much higher expectations.

Third, the con­di­tions of the test­ing will change. The cur­rent WKCE con­sists of fall test­ing in a rel­a­tively nar­row time win­dow. The new SBAC sys­tem will have a much larger win­dow and give dis­tricts more options of when to test. The new sys­tem will col­lect data imme­di­ately with­out scan sheet entry and thus be able to reduce the return of results from months to weeks. The han­dling of hun­dreds of boxes of test mate­ri­als will dis­ap­pear. Test secu­rity will change and prob­a­bly become less oner­ous. Test accom­mo­da­tions will need to change for ELL and SWD students.

Since no two stu­dents will have the same test under com­puter adap­tive test­ing, item analy­sis will become a thing of the past. Other strate­gies will have to be devised in those dis­tricts that now use item analy­sis as a way to find cur­ric­u­lar or instruc­tional weak­nesses. For­tu­nately, SBAC pro­vides dif­fer­ent mech­a­nisms to pro­vide feed­back and learn from our stu­dents’ per­for­mances and improve our ped­a­gogy. It promises interim/​benchmark assess­ment and help in devel­op­ing for­ma­tive assess­ments for use in the class­room as an alter­na­tive feed­back process. How­ever, it is not clear that these processes will be in place for a suf­fi­cient period of time dur­ing the first few years when the test will count as our account­abil­ity mea­sure for edu­ca­tors to use them to improve prac­tice and to develop knowl­edge­able feedback.

The com­bi­na­tion of more demand­ing and rig­or­ous stan­dards, higher cut­off scores for pro­fi­ciency, and increased scrutiny of the test results, given national com­par­isons, will lead to more dif­fi­cul­ties for Wis­con­sin schools and dis­tricts to demon­strate the suc­cess required by NCLB. Even with the new mod­i­fi­ca­tions (they are not really waivers) to the NCLB expec­ta­tions, it is clear that Wis­con­sin edu­ca­tors will have many sig­nif­i­cant challenges

Some suggestions on the work that must be done

There is much to do if schools and dis­tricts in Wis­con­sin are to main­tain the suc­cesses of the past. Edu­ca­tors must first under­stand the instruc­tional and cur­ric­u­lar demands of the Com­mon Core State Stan­dards (CCSS). It is wholly insuf­fi­cient to “align” exist­ing cur­ric­ula with the CCSS as the new stan­dards are so sub­stan­tially dif­fer­ent from cur­rent stan­dards. Whole­sale upgrades must be made in cur­ricu­lum and instruc­tion. This includes increas­ing the rigor of learn­ing and class­room assess­ment, increas­ing the use of rea­son­ing and argu­ment, model build­ing in math­e­mat­ics, and a host of other strate­gies not explic­itly men­tioned in cur­rent standards.

Over­all, there is an expec­ta­tion of deep cog­ni­tive under­stand­ing on the part of stu­dents (and teach­ers) over the cur­rent empha­sis on fac­tual learn­ing. The SBAC pro­posal calls for deep dis­ci­pli­nary under­stand­ing, prob­lem solv­ing as the basis for learn­ing, close analy­sis of text, research skills in the early grades, the abil­ity to inte­grate knowl­edge across mul­ti­ple and diverse texts and con­texts, rea­son­ing from evi­dence, and other “higher order” think­ing skills only ref­er­enced in pass­ing in many cur­rent practices.

2014-15 Target

The SBAC assess­ment is sched­uled to replace the cur­rent WKCE assess­ment in the 201415 school year. Despite the large amount of money com­mit­ted to SBAC and the staffs of the 28 states con­tribut­ing to its com­ple­tion, the two con­sor­tia work­ing on the new state test­ing process still have mas­sive tasks. It is dif­fi­cult to get 28 states to agree on much very quickly about the sev­eral com­pet­ing test­ing the­o­ries in the field. Accord­ing to the plan, SBAC has less than two and one-​half years before imple­men­ta­tion is to begin and already its pub­lished sched­ules have begun to slip.

Most rea­son­ably, the first pri­or­ity for SBAC is to cre­ate the sum­ma­tive test item bank and the com­put­er­ized plat­form to admin­is­ter it. No edu­ca­tional project of this scope and com­plex­ity has ever been attempted, par­tic­u­larly with the need for high degrees of valid­ity and reli­a­bil­ity that high stakes tests demand. The new CCSS pro­vide chal­lenges of their own. Higher order think­ing skills, depth of under­stand­ing, and qual­ity of argu­ment are much more dif­fi­cult con­cepts to mea­sure than fac­tual recall.

This pri­or­i­ti­za­tion of the sum­ma­tive item bank first is appro­pri­ate as the other two com­po­nents of the assess­ment sys­tem are premised on the need to pre­pare stu­dents for the sum­ma­tive test. Thus, the other com­po­nents need to wait. Unfor­tu­nately, this places states, dis­tricts, and schools behind the prover­bial eight-​ball because the other two com­po­nents are crit­i­cal if dis­tricts and schools are to pre­pare. If the interim/​benchmark assess­ments are not in place then schools can­not use them to deter­mine how close their stu­dents are to being suc­cess­ful. Thus, the com­po­nents of the new assess­ment sys­tem that are the most use­ful and effec­tive for schools and dis­tricts will be among the last com­po­nents to be devel­oped. It is dif­fi­cult to pre­pare for some­thing that doesn’t yet exist and whose struc­ture and com­po­si­tion is unknown. And, the train­ing to sup­port those prac­tices is sched­uled to be deliv­ered late in the SBAC imple­men­ta­tion process, which will not give dis­tricts and schools much time to learn and imple­ment new practices.

The only ele­ment of the new assess­ment sys­tem that dis­tricts, schools, and teach­ers have the most con­trol over is the for­ma­tive assess­ment. If teach­ers can­not pro­vide feed­back and cor­rec­tions in the class­room toward the expec­ta­tions of the assess­ment, edu­ca­tors are left with the choice of “shoot­ing in the dark” or not shoot­ing at all. Both of these sit­u­a­tions are not con­ducive to improv­ing teach­ing and learning.

Given the nec­es­sary order of devel­op­ment, the SBAC assess­ment sys­tem is mov­ing down the tracks with­out ade­quate infor­ma­tion and resources for dis­tricts to pre­pare for the com­ing chal­lenges. Dis­tricts in Wis­con­sin and else­where are in a dif­fi­cult spot. Over­com­ing greatly lim­ited resources, they must quickly make sig­nif­i­cant changes in their prac­tices and poli­cies. They must under­stand the CCSS and the impli­ca­tions for instruc­tion and dra­mat­i­cally revise their cur­ric­ula to imple­ment those new standards.

You can’t do it alone

The CCSS can­not be imple­mented in “whole cloth.” For exam­ple, a sixth grade teacher can­not imple­ment the math or ELA cur­ricu­lum by him– or her­self, even with their grade level col­leagues col­lab­o­rat­ing. This is because the incom­ing stu­dents will not be ready for the sixth grade stan­dards because they haven’t met the pre­req­ui­sites taught in fifth grade because the fifth grade teach­ers are just begin­ning to imple­ment the stan­dards. The fifth grade teach­ers will need to rely upon the skills taught by the fourth grade teach­ers and so on back. The new stan­dards are such a sig­nif­i­cant leap in per­for­mance that a sin­gle teacher or a sin­gle grade level can­not do it alone because the stu­dents will need to be solidly pre­pared with the pre­req­ui­site skills taught in ear­lier grades.

Start­ing in kinder­garten and adding the stan­dards to one grade at a time will not work because the test­ing starts in two and one-​half years and this approach would take almost a decade to imple­ment. The most rea­son­able strat­egy is to learn about the learn­ing pro­gres­sions built into the stan­dards and pub­lished in sev­eral places and to strive to “close the gap” between cur­rent expec­ta­tions and the expec­ta­tions demanded by the stan­dards each year; simul­ta­ne­ously in each and every grade. This requires an unprece­dented high per­cent­age of stu­dents mak­ing sig­nif­i­cant progress every year for five to seven years. It also requires teach­ers to have a clear under­stand­ing of the learn­ing pro­gres­sions across the grade level stan­dards and the path to where the stu­dents are now and where they need to be to be suc­cess­ful on the SBAC assessment.

Addi­tion­ally, the research on well-​constructed for­ma­tive assess­ment shows that it is clearly among the most pow­er­ful strate­gies avail­able in the class­room for improv­ing stu­dent per­for­mance, even on the demand­ing con­tent of the CCSS. Imple­ment­ing for­ma­tive assess­ment in the class­room takes con­sid­er­able plan­ning and prac­tice to imple­ment well. It requires teach­ers to change many ele­ments of their prac­tice to focus on pro­vid­ing con­stant feed­back to stu­dents and increas­ing the student’s respon­si­bil­ity for their own learn­ing and grad­ual release of teacher respon­si­bil­ity through the meth­ods by which the teacher struc­tures the cul­ture of the class­room and the struc­ture of for­ma­tive assess­ment. Although it is men­tioned promi­nently in the SBAC pro­posal, the Consortium’s pub­lished imple­men­ta­tion plans have been silent on its implementation.

No time to waste (or wait)

Dis­tricts and schools that wish to be suc­cess­ful on the new assess­ment sys­tem must start now going “full bore” on their prepa­ra­tion for the SBAC assess­ment. Time is short and there is much crit­i­cal work to be done. Although every­thing is not now known about the SBAC test­ing, enough is known to start the work. There are sev­eral crit­i­cal tasks that can be done now, which are under the con­trol of edu­ca­tors, and can be done with min­i­mal risks of being off tar­get. All of these rec­om­men­da­tions are known or know­able in the lit­er­a­ture on school improvement.

  1. Study the steady stream of doc­u­men­ta­tion being pub­lished by SBAC and the experts work­ing with the Con­sor­tium. This will give you an idea of the phi­los­o­phy of SBAC and the under­ly­ing research that is guid­ing the devel­op­ment of the assessment.
  2. Dis­sem­i­nate what you learn widely to class­room prac­tice, and ensure that teach­ers are well pre­pared to adopt the prac­tices that are con­tained in the SBAC pro­posal and sup­port­ing documents.
  3. Begin study­ing and imple­ment­ing the CCSS in every class­room after an analy­sis of what the new stan­dards mean in terms of expec­ta­tions for stu­dent per­for­mance. Assume that your new cur­ricu­lum will be sub­stan­tially dif­fer­ent from what you have done in the past. Given the rapidly ris­ing expec­ta­tions, it will be bet­ter to over pre­pare than to take the stan­dards too lightly.
  4. Under­stand how learn­ing pro­gres­sions work, how they are woven into the new stan­dards, and how they can help orga­nize your early work and even­tu­ally inte­grate teach­ing, learn­ing, assess­ing, and inter­ven­tions. Although not a lot is known about this approach, it holds vast poten­tial for orga­niz­ing the for­mi­da­ble tasks that lie ahead.
  5. Imple­ment instruc­tion in the class­room based on the CCSS as soon and as rapidly as pos­si­ble; know­ing that it will be rough and uneven for a few years.
  6. Develop your own com­mon assess­ments based on the types of sam­ple ques­tions being released by SBAC. Set very high stan­dards and do not be deterred if few stu­dents meet them early on.
  7. Invest heav­ily in for­ma­tive assess­ment and learn how to do it deeply and well. It is not a quiz or an end of chap­ter test. It is the nexus between instruc­tion and assess­ment — nei­ther fish nor fowl, but per­haps the most effec­tive set of strate­gies in our tool boxes. Build the high stan­dards into the for­ma­tive assess­ment and pro­vide a steady stream of non-​evaluative feed­back to students.

Start now! Time is short and there is much to do. By the time things are clearly defined, it will be too late to pre­pare for the test­ing and a longer time until you will be able to expe­ri­ence success.

Nick Dus­salt is an assess­ment spe­cial­ist, cur­rently work­ing for CESA#7. He can be reached at This email address is being pro­tected from spam­bots. You need JavaScript enabled to view it.

Leave your comments

0
terms and condition.

ascd logo

dpi logo

Registered User Login