2009 NAEP High School Transcript Study

Welcome to the 2009 National Assessment of Educational Progress High School Transcript Study.
We will now start the presentation.
Thank you, Jerry.
My name is Cornelia Orr, and I also welcome you today to this briefing on the release of these results.
Today I will be taking David Gordon's place as your moderator.
We will miss David's being here and wish him well.
The Governing Board is an independent bipartisan board that sets policy for the National Assessment of Educational
Progress also called NAEP.
The assessment results are reported to the country as the nation's report card.
The Governing Board is pleased to host today's event.
As most of you know,
NAEP is the only ongoing nationally representative assessment of student performance in the United States
and the only nationally representative measure of 12th grade achievement.
Today's study is both of import and interest.
Before we begin the data presentation, Jerry, our webinar producer, will address logistics
and mechanics for using the WebEx.
But first I'd like to run through our agenda.
After Jerry makes sure we are WebEx savvy, Dr. Jack Buckley,
commissioner of the National Center for Educational Statistics,
will present the 2009 NAEP High School Transcript Study results.
Then Henry Kranendonk will offer his perspective of the study.
Henry is a member of the Governing Board and currently serves as a mathematics consultant
and specialist for a variety of entities, including the Milwaukee public schools.
Finally, we are pleased to welcome Kati Haycock, president of the Education Trust,
to share her response on the results of the 2009 High School Transcript Study.
We will conclude with a time for questions during the webinar and off air at its completion.
But first, Jerry, please share with us what the attendees need to know.
Thank you, Cornelia.
If you'd like to ask a question at any time today through our Q&A feature,
you will find it in the lower right-hand corner of our screen.
Simply look for the Q&A tab, type in your question, and make sure you submit the questions to all panelists.
When you do submit your question, please put in your name and your organization.
Now back to you, Cornelia.
Thanks, Jerry.
Let's begin.
It's my pleasure to introduce Dr. Jack Buckley.
Dr. Buckley is the commissioner of the National Center for Education Statistics.
He's on leave from his position as professor of Applied Statistics at NYU.
He is known for his research on school choice, particularly charter schools
and on statistical methods for public policy.
Dr. Buckley, thank you for being here.
We're look forward to hearing from you.
Thanks Cornelia, and good morning.
I'm very pleased to be here today to give you the results from the 2009 High School transcript study, or HSTS,
conducted periodically as part of the national Assessment of Educational Progress or NAEP.
The NAEP High School Transcript Study Report provides detailed information on the academic experiences
and performance of America's high school graduates.
First a few notes.
All High School Transcript Study results, whether percentages, averages, scale scores,
or achievement-level percentages, are based on samples.
This means that there is a margin of error associated with every score and percentage.
In the figures that I'll show you today, we used asterisk to indicate the statistically significant differences.
The results for private school students are included in the overall results presented today;
however not enough private schools participated to allow us to release reliable results for them.
The 2009 NAEP High School Transcript Study is the sixth study in the series that began in 1990.
The 2009 HSTS is a nationally-representative sample of students who graduated from high school in 2009.
Results are based on sample of 37,700 graduates and their transcripts.
These transcripts were collected from 740 public and private schools across the country.
Transcripts collection took place from June 2009 through January 2010.
The study examines student course-taking patterns and grades, showing trends in course taking over time.
Most of the time, I'll be comparing 2009 with the earliest assessment from 1990,
and the most recent prior assessment in 2005.
But in some cases, we'll be using a shorter timeline, only going back to 1998 or 2000.
In addition to overall results, we'll also be looking at gender and grade ethnic differences.
First we'll look at information on credits earned based on student transcripts.
HSTS identifies three types of courses.
Core academic credits are the main classes high school students take, such as English, Mathematic, Science,
and Social Studies.
Other academic courses include Fine Arts, Foreign Languages, and computer-related study.
Other credits include courses in Vocational Education, Personal Health, and Physical Education.
To standardize our reports of course taking, we used the Carnegie definition of a credit,
120 hours of classroom instruction.
HSTS reports on the average course credits earned, as well as grade point average.
In both cases we have overall results and separate results for each course type.
This slide shows the changes in the average course credits earned by high school graduates since 1990.
The 2009 graduates earned more credits in core academic courses than in any past study year,
increasing from an average of 13.7 credits in 1990, as shown by the bar at the top, to 16 credits in 2009,
shown by the bar at the bottom.
Graduates also earn more credits overall in other academic courses, increasing from 3.6 credits in 1990
to 5. 3 credits in 2009, as shown by the blue bars.
And when we bring in the other credits, shown by the orange bars, and add them up,
we see that the total average credits increased from 23.6 in 1990 to 27.2 in 2009,
more than any previous graduation class for which we have conducted a study.
Factors, then, may be related to increasing credits include online courses.
With a rise of these online courses, students may have more opportunities outside of the traditional classrooms.
Information from the High School Transcript Study indicates that about 5% of graduates took online courses for
credits, and also credit for classes taken in middle school;
students who are taking what were once considered high school courses now before entering high school; for example,
Algebra I, Biology, or Spanish I.
A substantial number of these students are earning high school credits for these courses.
Among the 2009 graduates, for example, 26% took Algebra I before entering high school.
This next graph shows the changes over time in the average number of credits earned by graduates in each of the four
major racial ethnic groups.
Comparing 1990 to 2009, all four racial ethnic groups earned a higher number of credits.
Comparing 2009 to 2005, only white and black students had increases in the number of credits earned.
The average number of credits for black students, shown by the light blue line, increased from 23.5 in 1990,
to 26. 9 in 2005, and to 27.4 in 2009.
This slide shows changes over time for all graduate's grade point average.
From 1990 to 2009, overall GPA increased from 2.68 to 3.0 points.
Over the same time period, GPA increased for each of the three course types other courses,
as shown by the orange line at the top, other academic courses, by shown by the black line, and core academics,
the gray line at the bottom.
From 2005 to 2009, however, GPAs did not change measurably for core courses or other academic courses.
HSTS examines course-taking patterns in two ways: by looking at the level of the curriculum a student takes,
whether a standard, which is the least demanding, mid level, or rigorous, and by looking at course taking in Science,
Technology, Engineering, and Mathematics, what we call STEM. In particular,
we'll be addressing STEM advanced Mathematics, STEM advanced Science and Engineering,
and STEM-related technical courses.
The first perspective on course taking is curriculum level,
how many academic credits a graduate takes during high school.
Curriculum levels are a measure of high school graduates' overall academic achievement.
They can help measure how well graduates are prepared for postsecondary education based on the number
and type of courses taken.
We define three curriculum levels for this study.
The standard level includes four credits in English and three each in the three remaining core subjects.
The mid-level includes all of the standard credits plus more challenging requirements for Mathematics and Science,
along with a Foreign Language requirement,
and the rigorous level includes all of the mid-level requirements plus additional requirements for Mathematic,
Science, and Foreign Language.
Any curriculum that does not meet the requirements for the standard level is considered below standard.
Since 1990, graduates have been completing more challenging curriculum.
As these black bars show, from 1990 to 2009,
the percentage of graduates completing a below-standard curriculum decreased by 35 points, from 60% to 25%.
The percentage completing a standard curriculum rose six points.
There was an increase of 20 points for graduates completing a mid-level curriculum, from 26% in 1990 to 46% in 2009,
and there was an 8-point increase for graduates completing a rigorous curriculum during the same time period.
When we compare the changes over time
and the percentages of graduates completing this various curriculum levels by race ethnicity,
we see a variety of patterns.
The percentages of black
and white graduates completing a mid-level curriculum since 1998 have increased for both groups.
The blue line here shows increases for white graduates, while the orange line shows increases for black graduates.
In 1998, 35% of white graduates and 34% of black graduates had completed a mid-level curriculum.
The difference of two percentages was not statistically significant.
In 2009, however, a greater percentage of black than white graduates completed this mid-level curriculum,
51% compared to 45.
This 6-point gap was statistically significant, as was the 6-point gap in 2005.
Looking at the percentages completing a rigorous curriculum since 1998, we see increases for white graduates only.
14% of white students completed this highest level of curriculum, compared to 6% of black students.
This 8-point gap is larger than the previous gap of 5 points in 2005, and the two previous assessments.
This next set of graphs compare patterns of curriculum completion for white and Hispanic students.
Again, the blue line shows the percentages for white graduates over time,
while the orange line show results for Hispanic students.
The percentages of both white
and Hispanic graduates completing a mid-level curriculum were higher in 2009 than in 1998.
The increase for Hispanic graduates was larger than the increase for white graduates,
eliminating the 10-point gap that existed in 1998.
The 2-point difference in 2009, favoring Hispanic graduates, was, however, not significantly significant.
For the rigorous level however,
only white graduates showed a statistically significant increase in their percentage of completion from 1998 to 2009,
resulting in an increase in the gap from 5 to 6 points.
This next slide compares completion percentages for white and Asian Pacific Islander students.
The percentage of both white and Asian Pacific Islander graduates
completing a mid-level curriculum was 35% in 1998.
In 2009, the percentage for white graduates was 45%,
7 points higher than the percentage for Asian Pacific Islander graduates.
A greater percentage of Asian Pacific Islander graduates than white graduates, however,
completed a rigorous curriculum.
This was true in all four comparison years.
In 2009, 29% of Asian Pacific Islander graduates completed a rigorous curriculum, compared to 14% of white graduates.
This 15-point gap in 2009 was larger than the 7-point gap in 1998.
For the 2009 HSTS Report, we examined the requirements that students lacked to reach the next higher curriculum level.
So among graduates who attained a standard curriculum but did not attain a mid-level curriculum,
we found that 35% lacked only the science requirement needed to achieve a mid-level curriculum.
This percentage was higher than the percentage who lacked only, say, the Mathematic or Foreign Language course
and higher than the percentage who lacked more than one required course, and of those,
29% of graduates who did lack more than one required course,
a majority of those were lacking a science course as one of their courses.
For the 2009 High School Transcript Study,
we also looked at factors that might be associated with achieving higher curriculum levels,
and Algebra I course taking before high school is one of those factors.
This slide provides 2005 and 2009 data for all graduates,
as well as individual results for the four major racial ethnic groups.
The blue bars indicate the percentages of graduates who took Algebra before high school in 2005,
while the orange bars give the percentages who did so in 2009.
For all graduates, the percentage increased from 20 to 26%,
and all four racial ethnic groups showed an increase as well.
In 2009, nearly half of the Asian Pacific Islander graduates took Algebra I before high school.
This next slide shows the average NAEP 12th grade Mathematic scores of graduates
according to the level of curriculum completed.
Graduates completing a below-standard curriculum had an average score of 142,
placing them at the cut point for students at the NAEP basic achievement level.
Graduates completing the standard and mid-level curricula also fell in the basic range.
Graduates completing a rigorous curriculum scored, on average, in the proficient range.
On this slide we look at the same information for the four major racial ethnic groups.
Within each group,
graduates completing a rigorous curriculum earned higher NAEP scores on average
than graduates completing lower curriculum;
however the completion of a rigorous curriculum does not indicate an elimination of racial ethnic gaps in the
mathematic performance, as you can see.
The average scores for black
and Hispanic graduates completing a rigorous curriculum were lower than the average score for white graduates, which,
in turn, were lower than the average score for Asian Pacific Islander graduates,
and this pattern holds true for each of the remaining curriculum levels,
as you can see when we bring in the bars for those levels, black for below standard, red for standard,
and gray for mid-level.
Now while the differences shown here aren't statistically significant,
this does not mean that race ethnicity is the determining factor in creating these differences.
Differences in student performance are influenced by many factors and not simply the ones shown here.
We also have results by gender.
Male graduates completing a rigorous curriculum, on an average,
had higher 12th grade NAEP Mathematic scores than female graduates.
In 2009, these male graduates had an average 12th grade NAEP Mathematics score of 192,
compared to 185 for female graduates.
This was also true for male and female graduates taking mid-level and below standard curriculum.
To compete globally and keep up with expanding scientific and technical expertise, educators
and policymakers have called for increasing emphasis on Science, Technology, Engineering, and Mathematics, or STEM,
course taking in your schools.
As part of the 2009 High School Transcript Study, we took a closer look at STEM course taking.
Here we provide the definitions we used for STEM courses today.
STEM-advanced Mathematics courses include Algebra II, other Advanced Mathematics, such as Trigonometry, Statistics
and Probability, Pre-Calculus, and Calculus.
STEM advanced Science and Engineering courses include Advanced Biology, Chemistry, Advanced Environmental
or Earth Science, Physics, and Engineering, and STEM-related technical courses include, Engineering, Science,
Technology, Health Science Technology, and Computer Science, and related courses.
We'll begin by looking at STEM Advanced Mathematics course taking, looking at differences by gender in particular.
In 2009, the percentage of female graduates taking STEM Advanced Mathematics was higher than that of males,
85% as compared to 82.
Higher percentages of female graduates also took Algebra II and Pre-Calculus, or Analysis.
And for other Advanced Mathematics and Calculus, there was no statistically significant difference.
Overall, a higher percentage of female graduates also took STEM-advanced Science and Engineering courses;
however this is not always the case when we look at individual courses.
Higher percentages of female than male graduates took Advanced Biology and Chemistry courses,
but higher percentages of male graduates took Physics and Engineering.
For Advanced Environmental or Earth Science there was no apparent difference.
And for the STEM-related technical courses, male graduates had higher personages overall, for Engineering
and Science Technologies and Computer Science courses, while for Health Science and Technology courses,
the percentage of female graduates was higher.
Now I'd like to look at the percentages of graduates who took, in particular, advanced placement
or International Baccalaureate courses in 2009 by race ethnicity.
The brown bars show the percentages taking AP or IB Mathematics courses, while the green bars are for AP, IB, Science.
For both types of courses, the percentages for Asian Pacific Islander graduates, 42% for Mathematics,
and 38% for Science, were higher than for any other group.
The percentages for white graduates were higher than the percentages for either black or Hispanic graduates.
And when we look at AP, IB, Mathematics, and Science course taking according to gender,
we see no significant difference for Mathematics, as shown, again, here by the brown bars.
For science, however, the percentage of female graduates is higher, 15%, as compared to 13.
That completes my overview of the results from the 2009 NAEP High School Transcript Study.
There is much more information in the full report, and in addition,
the NAEP website will give you access to more information on the study and earlier studies,
as well as access to the NAEP High School Transcript Study Data Explorer,
which allows you to perform your own analyses of the topics we have discussed here, as well a many others.
In closing, I would like to thank the schools and students who participated in these assessments.
Thank you very much.
Thank you, Dr. Buckley.
Next we turn to Henry Kranendonk, a Governing Board member.
Henry is a Mathematics curriculum consultant for the Milwaukee Public Schools
and the Mathematics specialist for Marquette's University Educational Opportunities Program.
Among other duties, he is also a lecturer and consultant for the University of Wisconsin Milwaukee
and is involved in curriculum writing projects for various organizations.
Henry will discuss his perspective of the results with a lend from his vast experience of Mathematics teaching in his
home city of Milwaukee.
Thank you for being with us today, Henry.
Thank you, Cornelia.
I am pleased to be part of this panel to discuss the National Assessment of Educational Progress 2009 High School
Transcript Study.
I served as the Mathematic curriculum specialist for seven years
and taught math at the high school level for 32 years in the Milwaukee public schools.
As a result, I have an obvious interest in seeing trends on some of the courses our nation's graduates have taken
and the grades and credits they have earned.
The report didn't necessarily surprise me.
As someone who has taught advanced math classes, I know that a higher-level curriculum makes the difference,
not just in students' NAEP scores but also in the ability to handle the challenging course work
and to be better prepared for life after high school.
This report made me think about the larger implications and the challenges of providing this kind of curriculum.
First, I'm a firm believer that science
and math are critical subjects in which students often need extra support throughout their high school experience.
Background data on this report that can be found online bears this out.
In 2009,
students who took their last Science course in tenth grade scored 11 points lower on NAEP than peers who took their
last Science course at tenth grade and scored 11 -- and 27 points lower than peers who took science on 12th grade.
Likewise,
students who took their last mathematics course in tenth grade scored eight points lower than peers who took their
last math course at 11th grade, and 24 points lower than peers who took math in 12th grade.
It is clear that students benefit from a more challenging curriculum.
Although more advanced course taking is certainly a tangible goal in this endeavor to improve student performance,
there are other important factors to be considered.
In my area of Milwaukee there are schools in the suburbs that consistently offer higher level math and science courses.
Good news.
But two miles down the road, in urban Milwaukee, there are schools with remarkably fewer course offering.
Providing the opportunity for a course like Calculus with an enrollment
anywhere from 8 to 12 students is definitely a challenge.
Schools find it hard to justify offering a class for so few students, especially when funding cuts have to be made.
Other options for meeting the needs of students are available and have been attempted,
including busing students from neighboring schools to consolidated classes and distance learning
or online learning opportunities; however these options also can be costly; for example, transportation,
and may not be suitable for all students.
My hope is that we can continue to explore ways for equitable access to these important courses for students.
Access to challenging math and science courses is not enough.
It's only part of the solution.
The content in these courses must be delivered in an accurate and rigorous manner by trained and certified teachers.
This is a great challenge for our schools, not just those in our urban areas.
A real concern of mine is how we will incorporate Algebra into the eighth grade curriculum so those students can get a
leg up on their math courses in high school.
Quite a few states, like California, are pushing for this more vigorously.
The common core state standards, which is coming up, certainly calls for more Algebra topics at the middle grade level.
For these initiatives to be successful and equitable for all students, the mathematics background of eighth grade
and seventh grade teachers much improve to meet this challenge.
This will require an investment in teacher training.
If algebra is taught nationally at the eighth grade level, middle grade mathematics teachers training
and certification will need to include more algebra and not just general middle school mathematics.
The University of Wisconsin Milwaukee, a campus where I teach,
offers a minor for middle grade certification in mathematics with genuine topic-specific courses.
We're finding more and more teacher-training candidates are taking advantage of that opportunity.
A similar offering is also designed in science and STEM.
The bottom line is we need to increase the quality and rigor of teaching.
For the class on 2009,
there was a 36-point gap in NAEP mathematic scores between student who is took Algebra I in high school
and those students who took Algebra I before high school, a gap similar to what you saw in 2005.
Early exposure makes a difference, but we must have targeted training to bring students to the highest level possible.
Even with universal eighth-grade Algebra,
there will be students whose developmental needs will require additional algebra as they enter high school.
We still have to provide multiple pathways for kids.
I am concerned about creating an environment that may be perceived as
a blocking students from preparing for postsecondary education
or employment simply because they did not successfully complete a math curriculum
that includes algebra in the middle grades.
During the four years of high school, there is exciting potential
and opportunity for our youths to achieve of in this country.
There will be a need for high schools to help prepare kids for various paths, whether it's a four-year university,
a technical college, military service, or employment, with appropriate rigor and substance.
The landscape in our schools is more complicated today.
As I look back at my first years of teaching nearly 40 years ago, I sometimes reflect on how much simpler it was.
There were only four or five math courses, Algebra, Geometry, Advanced Math, and Pre-Calculus.
Now I'm working at Marquette University with students who are behind
and who are taking classes like Statistics or the Mathematics of Finance,
and these students would have benefited from courses in these areas during their high school experience.
This NAEP study makes me wonder if we're catching up with that landscape.
Are we doing all we can to provide the best possible courses and instruction in learning?
We certainly need to.
Our students and our future depend on this.
I will now turn this back to Cornelia.
Thank you, Henry.
Now we turn to Kati Haycock.
We are very fortunate to have her, a leading advocate in the field of education, who is with us today.
Kati currently serves as president of the Education Trust, an organization that works for access
and high academic achievement of all students and at all levels, pre-K through college.
Welcome, Kati.
Thanks Cornelia.
For years national leaders have been sounding an alarm about the need to ensure that students graduated from high
school, all students graduating from high school, leave actually equipped with the knowledge
and skills that they need to success in college and the workplace.
It appears that our students are getting that message,
but the results from this transcript study are kind of a sobering reminder of how much work remains to be done.
Now, importantly, data from the study shows some real progress, especially for students of color, whom,
for many years, have been shunted into lower-level courses.
All groups of graduates are now earning more credits toward at least a standard curriculum,
and about half of African American students and about half of Latino graduates now complete a mid-level curriculum.
These trends matter because higher level course work is, indeed, associated, as Jack mentioned,
with better outcomes for students.
African American students, for example, who took a rigorous curriculum,
scored 45-points high on the NAEP mathematics exam that those who took a standard curriculum, a difference,
by the way, that is far greater than the 29-point achievement gap between all African American 12th graders
and their white counterparts.
And when it comes to postsecondary success,
eight of every ten students who is taking the most intense curriculums in high schools eventually complete that
bachelors degree that they need for lifetime earning success,
whereas as the success rates drop to just one in ten among students who took the lowest level course work.
Now despite the good news -- and it's good -- at the lower end of the curriculum spectrum,
it's very clear that we aren't making nearly enough progress getting students into the most rigorous curriculum that
many colleges are looking for.
Far too many students, especially those who are African American
or Latino -- excuse me -- still do not have the kind of high school experience that they need.
African American and Latino graduates, for example,
are about one-half as likely as white graduates to complete that most rigorous curriculum,
and there's been no improvement in that number since 2005.
In addition, when you look at grades you have to worry.
The grades earned by African American and Latino graduates are stagnating.
On an average, African American and Latino graduates have grade point averages somewhere between kind of B
and a B minus; whereas white and Asian graduates have GPAs closer to a B plus -- between a B and a B plus.
GPAs among Latino graduates haven't improved since 1998, and American African American students,
there have been no improvements since 2005.
Because, as Jack mentioned earlier, the GPA's of white students are increasing,
what that means is that the gaps between groups are actually widening.
For Latinos and whites, for example, the gap that separates them has doubled since 1990.
Despite the general link between taking a more rigorous curriculum and higher test performance,
improvements in the performance among students of color have actually lagged behind improvements
in high school course taking.
Among students who complete the same level of curriculum, African-American and Latino graduates' knowledge
and skills lag actually quite far behind those of their white and Asian counterparts.
Latinos, for example, who complete a mid-level curriculum are performing about as well on the NAEP math
and science assignments as white students who complete a below-standard curriculum.
And African Americans students who complete the most rigorous curriculum are
actually performing at about the same level as
white graduates completing a mid-level curriculum.
Now when you have a picture like that that shows higher level course taking, with little
or no progress in achievement,
you got to raise serious questions about the level of course rigor in schools serving many students of color.
Now I know that there are people who argue that there is kind of an unavoidable tradeoff here between greater
and more equitable access to courses like Algebra II or Chemistry and Physics, and the actual rigor of those courses.
People often suggest that if we let more students into these courses we'll inevitably lower their quality.
Certainly, we do see that lower pattern of course quality in some schools,
but we also know it doesn't have to be this way; that this idea that we have to choose somehow between access
and excellence is dead wrong.
My colleagues and I at The Ed Trust have studied high schools across the country that are working for all students,
and we know from them that increased course rigor and higher achievement can and should go hand in hand.
At schools like Elmont Memorial Junior/Senior High School in New York, for example,
more students are taking the most rigorous courses in another high school,
and achievement for all groups of students is far higher than at other high schools in New York State.
But, again, at far too many other schools, schools without a focus on strong instruction,
a rigorous curriculum is often rigorous in name only.
So what can we do to change this?
States are increasingly focused, as Henry mentioned, on ensuring their students are college and career ready,
and they're beginning to put in place higher standards
and more rigorous assessments that actually align with this new goal.
But in order for this sort of critical
and potentially game-changing work to actually live up to the promise of college and career readiness for all,
higher standards and assessments have to be coupled with access for students to rigorous coursework,
with strong well-supported teachers.
Unfortunately, we know that this not what happens now.
African-American and Latino students, in particular,
are still less likely to attend high schools that offer high-level math courses like Trig and Calculus,
which severely limits their ability to take the courses they'll need to be successful in STEM fields.
And even in schools where high-level courses are available, students of color too often can't take them,
unfortunately because of decisions made earlier in their academic career.
Consider, for example,
almost two-thirds of the graduates who complete a rigorous curriculum actually took algebra before high school.
But even when you look at the highest achieving African America fifth graders,
what you learn is that only about 35% of them are enrolled in eighth grade Algebra,
compared to nearly two-thirds of high-achieving white fifth graders.
Of course, enrolling in a course with a certain name is certainly not enough.
Even when two courses have the same name, or even use the same textbook,
there's no guarantee that they will be equally rigorous.
In our experience, students of color often receive less rigorous assignments than their peers in schools
or districts with lower concentrations of minority students.
A transition to more rigorous standards will certainly help this process,
but implementing those standards consistently across districts and classrooms is critical.
To support high achievement for all students, teachers will need access to curricular support materials
and professional development opportunities that provide examples of
and guidance on developing high-quality lessons and assignments.
We also need to assure that students have access to teachers who are qualified
to teach the subjects that they're assigned to teach.
Students certainly have higher math achievement when their teacher actually has a background in that subject,
especially at the high school level.
However,
nearly one in three math classes in high-minority high schools are taught by teachers who did not major in nor are
certified to teach mathematics.
Today's Transcript Study results represent, in other words,
not just failure on the part of kids but systemic failure on the part of our public schools.
Students, in fact, are doing what's asked of them, but they aren't being taught any more than the predecessors were.
While we've made progress in some areas,
far more work remains to be done to ensure that all students are equipped with the knowledge
and skills that they need to be successful in college and in life.
Thank you, Kati.
All of our presenters today have certainly given us plenty to think about.
Now we would like to address the questions that you have about our presentation today.
We will have Amy Buckley, who will facilitate the question-and-answer session, so I'll turn it over to her from now.
Thank you so much, Cornelia, and thank you to all of our panelists.
For those of you wishing to ask questions, please do so now.
As Jerry mentioned at the beginning,
we ask that you direct them to all panelists so that they don't slip through the cracks.
Also, we have had a great response thus far and have multiple questions that have already been submitted.
If we are not able to get to your question, please know that we will respond to you,
and we thank you so much for your interest.
Our first question comes from Jamal Abdul-Alim from Diverse Issues in Higher Education.
Jamal Asks,
What are the implications of this study for college administrators
who are charged with providing postsecondary education?
Should they plan on devoting more resource towards remediation, or should they expect fewer students to enroll,
or perhaps another scenario.
Cornelia, can you begin to address that.
Well I wish there were clear answers from the data that we have seen today.
I think that the board's research effort on preparedness is trying to provide some answers about how really prepared
students are based on what the NAEP source tells them.
But this research is in progress, and we do not have answers on all of these issues but should by this year.
I think, for now, colleges know the students they are receiving, they're working with their local high schools
and their local school districts, and they should continue with the plans that they have in place.
Thank you so much, Cornelia.
Our next question comes from Joy Dingle with Achieving Equality.
Joy asks, What are practical steps we can take to increase the quality of the courses our students take and complete?
Henry and Kati, you both addressed in your comments.
Henry, perhaps you could start.
Thank you. I would address this question by targeting three areas first.
Number one, I think schools should look at the quality and background of the teachers involved in these courses.
Certainly, teachers teaching more advanced level mathematics in the grade levels need that background.
Two, the rigor of the course designed by the depth and the topic, not just the amount of topic,
should be well organized around established standards.
Certainly we are very hopeful in this country that the topics --
or that the initiative around the common core state standards may bring state to the standards.
But there are many standards out there that can provide that organization.
And three, I think that teaching force should also be in partnership with universities
and colleges in their own content training and ongoing professional development in these courses.
Let me just add one quick note to that.
You know, the last time states rolled out their academic standards, they essentially left it to teachers,
to local districts, to local schools,
and often to individual teaches to figure out how to teach kids to those standards.
The net result of that was that we have very uneven interpretations of what kind of lessons, units, assignments;
teachers use very uneven interpretations of what student work is good enough to meet standards.
I think there is increasing consensus that we can't leave it to teachers this time.
They don't want to do the work, they're too busy doing other things,
and we've got to provide much more guidance this time around of banks, of lessons, units,
assignments that are really high quality that teachers can draw on
and adapt to their own students to make sure we get more consistency this time around.
Thank you so much, Kati.
Our next question comes from Maisie McAdoo from United Federation of Teachers.
Maisie asks, Can the study tell us whether online learning, either for AP coursework or for credit recovery,
is successful in readying students for college?
Jack or Peggy, can you start?
Well sure. That's a great question,
and I should first provide a general caveat that I always find myself finding on these, these days.
You know, a study like NAEP is great at letting us describe the state,
the condition of education in the United States,
not usually the best source for something causal like trying to determine whether or not a particular curriculum
or course of trajectory is actually going to improve or benefit students of a particular type.
That said, we do -- within the High School Transcript Study, we do have some data on online learning.
In the actual report, on Page 48 of the main report, for example, we desegregate
or show average scores on NAEP mathematics and science for graduates by whether
or not they took an online mathematics or science course. So, for example, in online Mathematics,
there was statistically significant difference, where students who did not take the course, on average, scored 156,
while students who did take at least one online mathematics course scored lower -- significantly lower at 141.
Now, of course, what I'm trying to say is that taking that online mathematics course causes those students to,
score lower on NAEP.
As you know from your question, that there is sort of a heterogeneous population that take online courses,
some because there are not advanced courses offered in their institutions,
others who are taking these courses for credit recovery,
and it's possible that their scores could have been lower still had they not taken the online course.
I would encourage people who are interested at looking at this in greater depth to use our online tool,
the NAEP data explorer, where you can fit, a somewhat more complicated model and try
and control for other factors while you're looking at that question.
Thank you very much.
Our next question is from Lynn Schaefer Willner with George Washington University.
She asks,
How do you address concerns about the validity of ELL scores if ELL accommodation rates are so variable across states?
Peggy.
Again, another good question.
So first I should point out that we don't have state-level data here for this study,
but it still is a good question because it's something that we are continuing to address.
Since the late 1990, in fact, the program has addressed two major issues associated with special needs students,
students with disabilities and English language learners.
The first, inclusion,
we have focused most predominantly on the NAEP -- I should say quite a bit of progress over this time that,
currently about -- we have made progress about two-thirds more of the students included in the samples in was the case
back in the �90s.
Nevertheless, the variation across states still remains.
There is more progress to be made, but we're gaining ground with inclusion this year.
The second area, as noted by your question, would be an accommodation.
Validity issues regarding accommodations are more difficult to address,
primarily because students often take more than one accommodation,
so having one accommodation is a rare occurrence actually.
And states vary quite a bit in whether their students are accommodated.
Some states, as much as 80% of their special needs students are accommodated, others hardly any.
We've done some experimental studies (INAUDIBLE) experimental studies,
focusing on specific types of accommodations
and gotten a lot more comfortable with the validity interpretation of those data.
But the program of research must continue.
Thank you so much, Peggy.
Our next question is from John Seffel with Colonel University and the New York State Center of Rural schools.
He asks, With the NAEP sample, are we able to reliably estimate courses taken
and performance in such courses for students attending rural school districts; specifically,
are there difference in course taking
and course performance in STEM courses in rural versus suburban versus urban schools?
Staff.
Another great question.
Fortunately,
we are able to desegregate results for various of the measures we have today by what we call low cal, where we
generally divide the world into cities, suburbs, towns, and rural areas, and other subdivisions within each of those.
And we do find some very interesting important differences.
I would say perhaps the most important one is, although course taking is important,
there's simply access to the courses, and Kati alluded to this in a couple areas.
If the courses aren't offered and there's not a good online infrastructure that can offer a comparable course,
students simply can't take them.
And we find that there is a very large difference within the rural areas in who has access to
or where courses are offered, particularly in Advanced Science.
So, for example, in some of the rural areas,
depending on how remote they are from the nearest urban population center,
we could find as few as 50% of students will have access to Advanced Science course taking.
The numbers tend to be higher for mathematics, and we suspect, although I can't show from the NAEP data,
we suspect that it's at the end of the day to hire a mathematics teacher
and offer a Calculus course than it is to actually have a Science Lab
or some of the facilities that you need to offer Advanced Science.
And I would also encourage you, again,
to look at the online data tools because you can use this low cal breakdown to look at a lot of interesting issues
in these rural education with these data.
Thank you very much.
Our next question is from Elaine Weiss.
She asks, It is clear that students across the board are making gains in course credits, rigor of courses,
and NAEP stores, yet we continue to hear that minority students, especially, are flat or losing ground.
How do we correct this?
Cornelia, do you want to start and perhaps Peggy.
Well, certainly, we want to thank you for the question
and say we certainly don't have all of the answers to getting all of the problems solved.
You've heard suggestions from Henry
and Kati today about ensuring that the course work that students have access to -- first of all,
have access to course work, and secondly, have access to rigorous instruction and comprehensive learning by students.
So I think those things, as well as what Henry mentioned,
about having stronger teachers who have the background needed to teach the curriculums in the classroom.
So those are some suggestions that I have.
Peggy.
Yes, this is Peggy Carr, associate commissioner for assessment.
I don't have any of the answers either, Cornelia.
But I think it would be helpful to sort of help everyone understand how to think about these data.
So there are a lot of confounds here.
The students of various racial ethnic backgrounds differ disproportionately, for example,
in their socioeconomic status.
Blacks and Hispanics are predominantly in these data and other NAEP data that we explore,
about 80% are eligible for free and reduced-price lunch, as compared to about 40% for their white counterparts.
So that's an important confound that I think that we need to keep in mind.
There are other factors that might be correlated with these scores that we see; the qualifications of their teachers,
whether they have degrees, major or minor, in the areas that they are given to teach, the experience they have,
in making -- teaching these courses, and just so many other factors.
I think Kati so eloquently described issues associated with the rigor of courses.
In fact, we have a study coming out later this year that will explore rigor of the courses by looking at textbooks,
the chapters covered in those textbooks, the questions asked of the students regarding those chapters,
and the outlines teachers use for those particular courses,
and these courses have the same title; Algebra I and Geometry.
So we will be able to give you a look into these classes that have been supported here.
So there are no silver bullet.
A lot of confounds to be considered, but many questions to be answers.
Yeah.
I think Peggy would agree though, that while there is a fair amount that explain this,
the real question in most people's minds is, okay, what do we do; right?
What do we do to change these patterns?
And when you look at the schools that are exceptions to these, the patterns we have been talking about nationally,
the answers are really pretty clear.
Number one, it's about making sure you get kids of color and poor kids into the tougher courses; number two,
it's about making sure their teachers know what they're doing, right, and are well supported; number three,
it's about assuring some consistency and rigor by providing those teachers with lessons, units, assignments,
examples of student work, and so on that they need to get consistent rigor; and, number four,
it's about providing kids the kind of on-time support that they need
when they're struggling to master the materials.
And schools and districts that are working on those four things are getting much better results.
We just need to do that at scale.
Thank you, Kati.
Our next question is from Marie O'Hara.
She asks, Were you able to look at the difference in achievement levels for states with higher
or more rigorous course requirements for graduation than those with lower requirements?
Jack.
Sure.
Unfortunately, no.
Although our sample size is large, it's not large enough to be state representative, and it turns out,
doing an in-depth transcript analysis at that scale,
right now it's too big a project that we haven't been able to take on,
so we're not able to actually give you state representative comparisons like that, although it's a great question.
Thank you.
Our next question is from Michael Straight.
He would like to know if there is any more information you can provide on the criteria for curriculum classification.
Was it just the course titles?
In California, for example, many CTE courses have been approved for a G credit.
Would this course count towards rigorous or mid-level, or would they be in the other category by virtue of title?
You know, A to G is the name for the Cal State
and University of California approved academic course requirements for colleges.
Thank you for that clarification.
No problem.
I'd be happy to answer that.
So the short answer is, no,
and the longer answer is something called a Common Course Coding System called the Classification of Secondary
School Courses or CSSC, which is itself a modification, something you may be familiar with.
It's a little more common, called the CIP, or Classification of Instructional Programs.
So the courses that appear on a student transcript get a six-digit code, which is based on the course content
and the level, not the title alone.
We have about 2,200 codes, and we use course catalogs
and other materials from the schools to figure out the content and the level of the courses,
so not simply taking the title from the transcript.
Great, thank you.
The next question is from Gayle Howell.
Gayle Asks, Why is Native American data not in this study?
This is Peggy Carr, the associate commissioner.
The sample size for Native Americans simply just was not large enough to report out.
We have reporting rules, limitations, for reporting samples, and they just did not have enough.
Now this is a huge sample,
but we would have to do some special targeting sample to include Native Americans for that study.
Great.
Thank you, Peggy.
Our next question is from Susan Rariff.
Susan would like to know, What about the Arts?
Recognizing that access to art can help at-risk or low-performing students, did this NAEP study access the arts
and the relationship of Arts course taking to overall high school performance and college readiness?
Again, this is Peggy Carr.
We do include Foreign Languages, as the commissioner indicated earlier, in mid-level
and the rigorous curriculum level that's (INAUDIBLE).
Fine Arts is also part of the inclusion in this particular level of Fine Arts -- yeah, mid-level.
Mid-level includes Fine Arts.
So we do have inclusion of some of these courses, but the more finer gradations, we would have to dig deeper.
Great.
Thank you, Peggy.
Our next question is from Holly Elliott with Shelton State Community College.
She would like to know if we can provide more details about the demographics of the 740 public
and private schools that were included in this study.
Jack or Peggy?
Sure.
So I mentioned at the very beginning of the presentation that we're dealing with a sample here,
and what I didn't say anything about was sort of how was that sample drawn.
And so if we took just sort of a simple random sample of 740 schools and then students within schools,
we would be very unlikely, in fact, to match the demographics of the country, which is why, instead,
we take a very complex sample, where we're, in some cases,
over-sampling places where we're likely to get certain types of students that we want to be able to report on.
In other places, we're ensuring that we draw schools from a particular district that's necessary,
otherwise we would not be representative of the United States, places like New York City, for example.
Once we have those data; however, because we know a lot about how we designed the sample,
we know everything about how we designed the sample,
we actually are able to construct weights that make the data representative of the demographics of the country.
And so I guess the short answer is, if you just had access to the raw data,
they would not match the national demographics, but once we use the correct weights and analysis, they do.
Thank you, so much.
Our next question is from Brad Sindell.
Brad states that the CCSF includes Algebra I for grade eight for all students,
complicating course taking patterns across middle and high schools
and perhaps invalidating continued longitudinal trend data.
Is the Governing Board and NCES anticipating this?
We have been following the work of the common core state standards
and the common core assessment consortium very carefully, and, actually, in his remarks,
Governing Board member Henry Kranendonk mentioned how this is going to be changing the landscape at middle schools.
So we are aware of it.
We do not know how it will affect the Transcript Study, because, as Dr. Buckley mentioned a minute ago,
we examine what's actually taught in the course work, and so we are hopeful that the longitudinal trends can continue.
I would like -- Henry, go ahead.
This is Henry Kranendonk.
I would like to add to that that I think many of us feel the many facets of our achievement look through the NAEP will
be enhanced by this new effort.
By that, I mean I am particularly, as a former math teacher and math curriculum specialist,
looking to see how NAEP achievement scores will change in the future as a result of the common core state standards.
My hope, of course, is that the standards actually are providing that definition
and articulation of the rigor that we addressed in our comments that is very inconsistent,
both in course descriptions and across the country, and by bringing that consistency, I'm hopeful.
I'm actually optimistic that the achievement of our kids will improve, and that improvement, I hope,
will definitely be shown in improved achievement scores in our NAEP assessments.
So I think it remains as a way to monitor and look at how successful this initiative will be.
And I was just going to add -- I mean I think you're exactly right that the common core State standards have the
potential to complicate course-taking patterns.
But I don't look at that as a threat to the longitudinal validity to the study.
I look at that as an opportunity for our study to, next time around, to tell you will whether
or not things have actually changed.
Thank you very much.
Our next question is from Annette Cook, also with Shelton State Community College.
She comments that she notes there were no students in the advanced category of results from the curriculum level.
Why is that?
Who falls into the advanced category, and what defines advanced?
Cornelia, can you start us off.
I will start us off. Because advanced representatives a very advanced course work and attainment on the NAEP scale.
What's recorded in the High School Transcript Study are the averages of groups.
I'll let Peggy and Jack speak to that a little more.
So what we have in the report -- this is Peggy Carr again -- on Page 21 you will see a mapping of the average scores;
that's sort of a profile of average scores for the various achievement levels.
But they're just that, they're averages, and so you have variation around those means.
So there are students that will be in the advanced level,
individual students that will be in the advanced level that are not shown here.
But the bulk of the students, the average designations that we have here,
show that they are not falling in the advanced level.
Thank you very much.
Our next question is from Christopher Baker.
Christopher comments that there seems to be a growing trend in high schools to eliminate waiting for AP
and Honors courses towards GPA calculations, and also the elimination of class rank on transcripts, this,
in spite of long-term knowledge that these are good indicators of postsecondary success.
Does your study show any correlation or draw any conclusions about GPA and/or class rank and postsecondary success.
Jack.
Well first let me just say in terms of our own GPA calculation, we use a 4-point scale
and we do not weight any courses any differently than any others.
That said, a couple folks have brought this up already,
this is a cross-sectional study looking at the transcript of graduates, and this particular study,
we are not following these folks longitudinally postsecondary.
And so from this data alone, we're not able to tell you something about their success.
Cornelia mentioned that the National Assessment Governing Board has an ongoing research project trying to identify,
can we use what we know about NAEP and map that back to postsecondary success,
and we hope to make some progress on that front.
Just one clarification, Jack.
At the present time,
the studies that have been commissioned are looking at preparedness for entry-level course work
or entry-level job training, not success.
And we hope to eventually be able to look at that over time, but right now that won't be in the 2011 report.
This is Peggy Carr Again.
Well what we do know, not attributing success to the outcomes, is that students who take AP courses,
IB courses do score higher on NAEP, both in math and science,
and these students are also in higher curriculum-level designation, and they perform higher on NAEP,
and in the higher achievement levels.
So we know that.
I think the issue is, how are those scores, those designations, correlated with success.
That's the unknown factor.
Great, thank you so much.
Our next question is from Patricia Porter.
She would like to know what recommendations you have for educator preparation programs for mathematics teachers to
address the findings of this study.
Henry, would you like to comment on that?
I alluded in my remarks to some exciting things that I think are happening at the institution that I am a part-time
instructor at; that's University Of Wisconsin Milwaukee.
We do offer a Minor in Mathematics or a Minor in Science for our teachers.
This is for middle grade level teachers, even our elementary teachers.
These are core-specific preparations in math and science.
I am currently attending the National Council of Teacher of Mathematics conference,
hearing many exciting -- really exciting efforts around STEM centers that are being developed in various areas of the
country, and these particular centers are working at improving teacher understanding
and preparation in the STEM courses.
Kati mentioned something that I'm finding extremely important myself in working with teachers,
and that is to identify, locate which sources of problems and student work
and what we can share with teachers so that their understanding of the courses that we're talking about here
and what's goes in them and how student work looks as a result of taking them consistently is made available.
There's many efforts looking for standing videotaped classes
and teachers in those classes that can be of help across the country.
So there is a part of me that's very excited with this effort.
And the bottom line is the preparation of teachers, and not only teachers coming into the field.
In fact, I think in any respects, that's very exciting and there is movement there,
but teachers also who are in the field and are missing out -- have missed out in some of that training,
also need to have professional development to please make sure that they're ready for this.
So it's extensive and it does bring some concerns to my mind,
but I think there are efforts going on that we can be excited about.
Thank you, Henry, and thank you to everyone, for your wonderful questions.
We have gone passed our hour, and we're mindful of everyone's time.
So we're going to close the Q&A portion at this time.
But if we were not able to address your question, please know that we will respond to you directly.
Thanks, Amy.
I also want to thank our panelists today, Jack Buckley, Henry Kranendonk, and Kati Haycock for their participation,
and also Dr. Peggy Carr, who is the associate commissioner of NCES,
for her participation in the question-and-answer session.
Please allow me just a few closing comments.
First, I encourage you to visit our NAEP site at WWW.NATIONSREPORTCARD.GOV
that will give you this study but also various tools,
including the Data Explorer you heard talked about today
that you can use to mind questions -- answers to you own questions.
Also, if you go to the NAGB.ORG site, you will see a release page with a press release
and the panelists' statements that you've heard today.
If the media outlets who are joining us today are interested in additional Q&A,
there is a conference call that's taking place noon today, Eastern Time.
Please contact Jessica Hoy at (202) 540-8823 or JHOY@REINGOLD.COM if you are interested in accepting this information.
Third, I just want to thank all of our participants on the webinar today.
We appreciate you taking time to listen today,
and we hope you will also take a few minutes to complete a brief survey that will appear
in the window when you end your session.
Please stay tuned for an announcement on the date which we will be releasing the 2010 NAEP report card on Civics,
which will present data from Grades 4, 8, and 12.
Thank you very much.