Today’s report is in the nature of a caveat emptor warning or “let the reader of educational statistics beware.”

Yesterday the Kentucky Department of Education was set to receive an update on its AdvanceKentucky program to increase success on Advance Placement exams. The program received millions in Race to the Top funds and has been heralded by the U.S. Department of Education as a great success.

The DOE applauded AdvanceKentucky results, noting in particular that, from the program’s inception in 2008 until 2013, the total number of exams taken is up 95 percent and the number of scores of 3 or better is up 100 percent. A score of 3-5 (scale: 1-5) is considered a “successful score” and can lead to advance placement or college credit.

But here’s the first caveat emptor:

Kentucky’s number of successful AP exams did rise 100%. In fact, it’s up 150% in five years. But numbers in many locations are similar: In the same five years, Alabama’s successful AP exams are up 210% and Arizona’s 240%. Here in DC, the increase ws 180%. Did those areas spend millions to achieve the increase? Were great lessons learned in these areas? The US DOE is mum on this.

In the DOE brag report, Kentucky Education Commissioner Terry Holliday is quoted as saying:

“With our Race to the Top funding, we’ve been able expand the AdvanceKentucky initiative so that more students, especially those who are traditionally underserved and underrepresented in Advanced Placement, are exposed to rigorous, college-level courses…With the proper supports provided, they learn they can be successful and more kids graduate ready for college, with college credit and an option for continuing their education they might otherwise have never considered.”

So perhaps the real success of AdvanceKentucky is in increasing the success rate of minority students?

We come to the second caveat emptor:

The College Board’s report, “Equity Gaps in Participation and Success Among Traditionally Underserved Students” (download report by clicking on 4th report, bottom left), offers information beyond one state’s results…which is all the US DOE seems to be sharing:

In Kentucky, the rate of successful AP scores among African American students rose from 2.4% in 2008 to 3.4% in 2013, an increase of 40%. But Alabama’s successful AP scores among the same subgroup rose from 4.5% to 11%, an increase of 144%, while the rate in Arizona rose from 1% to 3%, an increase of 200%.

Similarly, Kentucky’s increase in successful scores among Hispanic students went up from 1.3% to 3.5%. But Alabama’s went up from 2.1% to 3.2% and Arizona’s from 15.1% to 26.1%.

Again, just a few more numbers raise questions as to the extent U.S. tax money and an $18 million annual budget for AdvanceKentucky are solely — or even largely, or a little bit — responsible for increases in Kentucky AP scores. Perhaps the progrm is exceptional, but it is important to look carefully at the full picture before trumpeting apparent results.