How to measure a QBs performance?

Hello, my name is CritMark and I am a stataholic. The first step in dealing with an addiction is to admit you have one. My spreadsheet just on QB stats for 2013 is 15 tabs and at least a hundred thousand cells with data. They are filled with Pivot Tables and Vlookups. Don’t worry if you don’t know what those are, it just means you have a better social life than I do. If you do know what those are remember, seek immediate medical attention if you have a Nested If Statement that lasts longer than 64 functions! (Did anybody get that?)

In the world of number crunchers there are the pros and the amateurs. The best pros are actuaries and arguably the best amateurs are sports fans. There is a very big difference between the two. Actuaries mostly work under two guiding principles, the law of large numbers and they don’t care why. Basically the law of large numbers says given a large enough sample the numbers will accurately reflect the truth of whatever it is they are testing. They don’t care why because it is not their job to care, just find data that can help to evaluate risk. An elderly person with a pet will statistically be in better health and live longer than one without a pet. You can theorize why but from an actuaries point of view they just know it’s true. I am not saying knowing why isn’t of value just from a risk mitigation standpoint the accuracy of the results is more important than the why.

Sports fans are a completely different kind of number cruncher. They seldom use a large enough sample of data to come to a statistically valid conclusion. More importantly, they demand to know why so they ascribe attributes to their analysis that have little or no basis in fact and ignore obvious attributes because they don’t fit the conclusion they predetermined before they began the analysis. Then they ascribe some conclusion which the data does not show and may have little to no real relevance. It’s called passion and it’s not a bad thing.

Finally, as Ramfan 1313 likes to say "Correlation vs. causation...the distinction is important..." He is correct but in those simple words is a great deal of complexity. In thinking along those same terms I would use slightly different words; "What’s the value of what you just proved?" Is it simply a post event statistical fact that shows has just happened or can it be used to guide future actions to influence future outcomes?

In the piece entitled Stunning Stat Reveals Sam Bradford's Achillies Heel! Enabling Ewe (nothing personal but it illustrates a point) makes the argument that yards/attempt is a valid measure of a QBs likelihood of having a winning record. The stat quoted is 100% accurate: "Only 1 of the top 15 quarterbacks in YDS/A had a losing season in 2013." Bradford was not among those 15 and as a result Ewe came to this conclusion: "Sam doesn't throw the ball down field often enough, because he doesn't go through his progressions, instead Sam locks onto the safe receiver for a short completions."

My issue with this conclusion is it assumes one singular reason for a QB having a low yards/attempt number, failure to go through progressions. I believe that is a flawed conclusion. In the lengthy exchange that ensued from my article I think someone owes Sam Bradford an apology! this stat came up and there seemed to be agreement that there were a lot of factors that could impact that specific stat like:

*Is a low yds/att a function of poor QB play or good defensive play vs the QB (see the Super Bowl)?

*Does a good pass rush change the offense’s play calling to more short passes?

*Does a large lead allow a D to ignore the run and therefore rush harder and play better pass D?

*Does a large deficit force the O into forcing longer passes thereby reducing the completion % which in turn can reduce the yds/att?

*Does poor O Line play force more check downs?

*Does a good ground game or lack thereof change the play calling on both sides of the ball?

*How do you factor in O & D schemes, player personnel, quality of receivers and progressions?

*How about YAC? Tavon’s 81 yard TD catch vs the Colts was a 5 yard catch and 76 yard run yet in the stats it’s 1 reception for 81 yards. It helped swell Clemons yds/att to 15.4 yards for the game.

Please understand I am not criticizing the stat or the post, just pointing out the potential flaw in the conclusion. I greatly appreciate Ewe’s passion and welcome her input. With this and input from others, I am going to dig into that stat and see if I can make any headway in associating it with other stats to find some deeper meaning.

Because of the complexity inherent in valuing some stats my analysis tends to focus more on statistical probability and a QBs ability to beat the odds. In my previous post I mentioned that when a team is held to under 50 yards rushing they lose 80% of the time. That was based on a sample of only 40 games in 2013. Now I am sure that if I went back 10 or 20 years that number would change some. It might improve to 70% or fall to 85% but I am confident in saying it would never be less than 50%. I was also comfortable using the stat because the 12 playoff teams were a combined 1-6 when held under 50 yards rushing.

So, in the mean time I am working on a new measure that determines what the statistical likelihood of a QB winning each game based on things outside their control vs actual outcome to create a +/- score to that probability. In simple terms, how well does a QB overcome the odds or succumb to them?

So with that very long route back to my question, what stats do you like to use to evaluate a QB? Let me know which and why you think they are valuable in your comments and I will see about incorporating them into my new QB score. Then just think, we will have another talking point to argue about when calmly debating the value of our QB over a brisk cup of tea!

Log In Sign Up

Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Turf Show Times

You must be a member of Turf Show Times to participate.

We have our own Community Guidelines at Turf Show Times. You should read them.

Join Turf Show Times

You must be a member of Turf Show Times to participate.

We have our own Community Guidelines at Turf Show Times. You should read them.




Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.