Wednesday, June 19, 2013

DI Study: The Breakdown

If you follow the Index regularly, we recently posted a ten year study, with the draftability index score of every Division I college prospect selected in the NBA Draft from 2000-2009. What follows are the results of that study and some comments on the application and methodology.

                           GREEN LEVEL (1.00 and above - almost universally recommended)

Players selected in Round One scoring                            Players selected in Round Two scoring Green
Green that have compiled a .100 WP48                                 that have compiled a .100 WP48 
                     or higher                                                                                or higher
                (23/38) -  60.5%                                                                     (5/9) - 55.5 %

                                                         Overall (28/47) - 59.6%

As I was working out the kinks with the formula, the numbers I was getting were between 60-70%, so I'm pretty satisfied that this is roughly where the total number is going to sit. 

Before I go further, I just wanted to address the choice to use WP48 as the measuring stick here. WP48 works well here because it is an average, and therefore isn't heavily influenced by how many minutes the player gets or what team he's on.

If you don't find WP48 or similar methodology compelling, that would be another argument altogether, and more to the point, you would likely find this entire study invalid, as it is built upon the supposition that win-based metrics are useful representational models and describe outcomes accurately. I'd be interested to know how the same player selections perform using other metrics and methods of analysis and am pretty confident DI would still generate good prospect rankings in most cases. 

One last note specific to green, It's been mentioned to me that many of the recommended choices are lottery picks, and perhaps that negates some of the power of hitting on roughly 60% of them. In answer, I would cite a couple points we all to be true: that 'lottery pick' does not, by any stretch of the imagination equal 'productive' and as any draft observer can plainly see- there are very few productive players to be had outside of the top 15 picks. If our method were recommending large numbers of players outside of the top 15 picks, it would be a condemnation of the approach, not a cause for endorsement. 

         GOLD LEVEL (0.50 - 0.99 -  preferable to red and gray - not categorically recommended)

Players selected in Round One scoring                            Players selected in Round Two scoring Gold
Gold that have compiled a .100 WP48                                 that have compiled a .100 WP48 
                     or higher                                                                                or higher
                (25/72) - 34.7%                                                                    (13/53) - 24.5%

                                                         Overall (38/125) - 30.4%

In the development process I was getting roughly 1/3 of gold grades hitting on productive players and again we see the 10-year study yields results just slightly below those figures. 

To clarify, gold prospects are not recommended, I am not a mathematician per se but even I realize that when you can throw a dart and get a good player 26.4 percent of the time (the total number of players .100+ from 2000-2009) that by bringing it up to 30% or even 35% it has not been demonstrated that you've necessarily done anything at all. So why even make the categorization? I did consider lumping the gold and red categories together at one point, however, I've run quite a few different sets of numbers and the gold players are consistently outperforming the reds 8-16%. (This time it's 8.) It's a pretty hard and fast line and I don't find the distinction insignificant. 

One note on the discrepancy between the numbers by round: my theory is that enough guys scoring gold and being taken in the second round are either cut outright or shuffled around without being played or developed, so as to artificially create a bit of a chasm. Were they all given the rope a first rounder is, I think you'd see that number come back up somewhat.

The big takeaways from DI are grab the greens and avoid the grays- and the gold and red grades are basically descriptions of middling prospects' proximity to one or the other pole. There are almost certainly other ways of honing and refining the way we talk about these players and it will be an ongoing challenge to discover just what those are. 

                           RED LEVEL (0.25 to 0,50  - very inconsistently productive)

Players selected in Round One scoring                            Players selected in Round Two scoring Red
Red that have compiled a .100 WP48                                 that have compiled a .100 WP48 
                     or higher                                                                                or higher
                 (15/68) 22.1%                                                                     (16/70) - 22.9%
                                                          
                                                        Overall (31/138) - 22.5%

In my previous work, I've seen this reds coming in anywhere from 16-24% so certainly this is right at the upper crest of that range. While I am curious that there is no 'second round depression' as we saw in the gold category, I am encouraged by the consistently between the first and second rounds. Continuity suggests we may be tapping into the 'real' potential of the players and not just arbitrarily assigning values. 
Although this study suggests the gap between red and gold may be less than initially suspected, as long as they come in on either end of the line of random chance (26.4) I'll continue to use them as described. 

                                 GRAY LEVEL (0.25 and below - seldom productive)

Players selected in Round One scoring                            Players selected in Round Two scoring Green
Gray that have compiled a .100 WP48                                 that have compiled a .100 WP48 
                     or higher                                                                                or higher
                (6/36) -  16.7%                                                                      (5/63) - 7.9%

                                                         Overall (11/99) - 11.1%

My first calculations of the gray level yielded about 10% of players hitting and so these findings are very much in line. In all cases the numbers are not so large as to read everything in the percentages yielded, and I do encourage anyone reading this to go through the study, see which players fall into what categories, and  decide whether you find the predictions DI makes compelling. 

We'll be back in short order with some more study analysis and some final thoughts on next week's draft- thanks for reading and enjoy game seven, kids.


No comments:

Post a Comment