The research covers mostly NFL...but some college towards the bottom.
Football Outsiders Basics (a.k.a. "Pregame Show")
by Aaron Schatz
While reading Football Outsiders or the Football Outsiders Almanac, new readers will often come across an offhand comment about, for example, the idea that fumble recovery is not a skill, and wonder what in the heck we are talking about. In each edition of Football Outsiders Almanac, we now include an essay called "Pregame Show" which gives a basic look at some of the most important precepts that have emerged during nine years of Football Outsiders research. That essay is republished here, along with links to the original research online when possible, or mentions of where that research appeared in print.
Please note that some of our basic research findings, such as the split between offense, defense, and special teams, were never addressed in one specific article, but instead developed over time. Therefore, there is no specific article we can point out.
For now, this article only includes precepts from our NFL research, although we plan to include precepts from college research in the near future.
You run when you win, not win when you run.
The first article ever written for Football Outsiders was devoted to debunking the myth of "establishing the run." There is no correlation whatsoever between giving your running backs a lot of carries early in the game and winning the game. Just running the ball is not going to help a team score; it has to run successfully.
There are two reasons why nearly every beat writer and television analyst still repeats the tired old-school mantra that "establishing the run" is the secret to winning football games. The first problem is confusing cause and effect. There are exceptions, usually when the opponent is strong in every area except run defense, like last year's Green Bay Packers. However, in general, winning teams have a lot of carries because their running backs are running out the clock at the end of wins, not because they are running wild early in games.
The second problem is history. Most of the current crop of NFL analysts came of age or actually played the game during the 1970s. They believe that the run-heavy game of that decade is how football is meant to be, and today's pass-first game is an aberration. As we addressed in an essay in Pro Football Prospectus 2007 about the history of NFL stats, it was actually the game of the 1970s that was the aberration. The seventies were far more slanted towards the run than any era since the arrival of Paul Brown, Otto Graham, and the Cleveland Browns in 1946. Optimal strategies from 1974 are not optimal strategies for today's game.
A sister statement to "you have to establish the run" is "team X is 5-1 when running back John Doe runs for at least 100 yards." Unless John Doe is ripping off six-yard gains Jamaal Charles-style, the team isn't winning because of his 100-yard games. He's putting up 100-yard games because his team is winning.
This is a corollary to the absurdity of "establish the run." If you don't believe us, meet our good friends the 2006-2007 Minnesota Vikings (or, for a more recent example, the 2014 St. Louis Rams). With rare exceptions, teams win or lose with the passing game more than the running game -- and by stopping the passing game more than the running game. The reason why teams need a strong run defense in the playoffs is not to shut the run down early, it's to keep the other team from icing the clock if they get a lead. You can't mount a comeback if you can't stop the run.
Note that "good pass defense" may mean "good pass rush" rather than "good defensive backs."
Running on third-and-short is more likely to convert than passing on third-and-short.
On average, passing will always gain more yardage than running, with one very important exception: when a team is just one or two yards away from a new set of downs or the goal line. On third-and-1, a run will convert for a new set of downs 36 percent more often than a pass. Expand that to all third or fourth downs with 1-2 yards to go, and the run is successful 40 percent more often. The gap has narrowed in recent years, but the numbers still favor running. Since 2010, teams have converted 68 percent of runs on third-and-1 (including fourth-down plays), compared to 55 percent of runs in the same situations. Expand that to all third or fourth downs with 1 or 2 yards to go, and running still has the edge, 66 percent to 52 percent. With these percentages, the possibility of a long gain with a pass is not worth the tradeoff of an incomplete that kills a drive.
This is one reason why teams have to be able to both run and pass. The offense also has to keep some semblance of balance so they can use their play-action fakes, and so the defense doesn't just run their nickel and dime packages all game. Balance also means that teams do need to pass occasionally in short-yardage situations; they just need to do it less than they do now. Teams pass 67 percent of the time on third-and-2 even though 60 percent of runs in that situation convert, while passes convert 51 percent of the time. They pass 77 percent of the time on fourth-and-2 even though 69 percent of runs in that situation convert, while passes on fourth-and-2 convert only 48 percent of the time.
When you open your newspaper on Sunday morning, you'll see that the little agate-type previews of each game list team rankings by total yardage. That is still how the NFL "officially" ranks teams, but these rankings rarely match up with common sense. That is because total team yardage may be the most context-dependent number in football.
It starts with the basic concept that rate stats are generally more valuable than cumulative stats. Yards per carry says more about a running back's quality than total yardage; completion percentage says more than just a quarterback's total number of completions. The same thing is true for teams; in fact, it is even more important because of the way football strategy influences the number of runs and passes in the game plan. Poor teams will give up fewer passing yards and more rushing yards because opponents with a late game lead stop passing in order to run out the clock. For winning teams, the opposite is true.
In 2014, who had a better pass defense: San Diego or Arizona? According the official NFL rankings, San Diego (3,249 yards allowed on 550 passing plays) was the fourth-best pass defense in football, while Arizona (3,908 yards allowed on 614 passing plays) ranked 29th. The Arizona pass defense, though, ranked 14th in DVOA (3.2%), while San Diego was 25th (16.4%). That's a big reason the Cardinals made the playoffs and the Chargers just missed.
Total yardage rankings are also skewed because some teams play at a faster pace than other teams. In 2014, Philadelphia's offense ranked fifth with 6,348 total yards, while Green Bay was sixth with 6,178 yards. Philadelphia, though, led the league with 201 offensive drives, while Green Bay was last with 163 drives. On a per-drive basis, Green Bay's offense gained substantially more yards than that of the Eagles.
This sounds absurdly basic, but when people consider team and player stats without looking at strength of schedule, they are ignoring this. In 2007, Tony Romo and Derek Anderson had similar numbers, but Romo faced a much tougher schedule than Anderson did. Romo was better that year, and better in the long run.
In 2014, Le'Veon Bell played 12 games that finished in the bottom half of the league in run defense DVOA. He averaged 96.8 yards per game and 4.9 yards per carry in those contests. In his four outings against upper-half run defenses, those averages fell to 49.8 yards per game and 3.8 yards per carry.
Because players and teams don't give the exact same performance every week, this is more of a general law, and it doesn't necessarily apply in the short term.
Our brethren at Baseball Prospectus believe that the most precious commodity in baseball is outs. Teams only get 27 of them per game, and you can't afford to give one up for very little return. So imagine if there was a new rule in baseball that gave a team a way to earn another three outs in the middle of the inning. That would be pretty useful, right?
That's the way football works. You may start a drive 80 yards away from scoring, but as long as you can earn 10 yards in four chances, you get another four chances. Long gains have plenty of value, but if those long gains are mixed with a lot of short gains, you are going to put the quarterback in a lot of difficult third-and-long situations. That means more punts and more giving the ball back to the other team rather than moving the chains and giving the offense four more plays to work with.
The running back who gains consistent yardage is also going to do a lot more for you late in the game, when the goal of running the ball is not just to gain yardage but to eat clock time. If you are a Baltimore Ravens fan watching your team with a late lead, you don't want to see three straight Justin Forsett stuffs at the line followed by a punt. You want to see a game-icing first down.
A common historical misconception is that our preference for consistent running backs means that "Football Outsiders believes that Barry Sanders was overrated." Sanders wasn't just any boom-and-bust running back, though; he was the greatest boom-and-bust runner of all time, with bigger booms and fewer busts. Our play-by-play database goes back to 1989, the first year of Sanders' career. Sanders finished in the top five of rushing DYAR in 1989, 1990, 1991, 1994, 1996, and 1997.
Rushing is more dependent on the offensive line than people realize, but pass protection is more dependent on the quarterback himself than people realize.
Some readers complain that this idea contradicts the previous one. Aren't those consistent running backs just the product of good offensive lines? The truth is somewhere in between. There are certainly good running backs who suffer because their offensive lines cannot create consistent holes (Matt Forte is a recent example). Most boom-and-bust running backs, however, contribute to their own problems by hesitating behind the line whenever the hole is unclear, looking for the home run instead of charging forward for the four-yard gain that keeps the offense moving.
As for pass protection, some quarterbacks have better instincts for the rush than others, and are thus better at getting out of trouble by moving around in the pocket or throwing the ball away. Others will hesitate, hold onto the ball too long, and lose yardage over and over.
Note that "moving around in the pocket" does not necessarily mean "scrambling." In fact, a scrambling quarterback will often take more sacks than a pocket quarterback, because while he's running around trying to make something happen, a defensive lineman will catch up with him.
Over the past five seasons, offenses have averaged 5.8 yards per play from Shotgun (not counting the Wildcat or other college-style option plays), but just 5.0 yards per play with the quarterback under center. This wide split exists even if you analyze the data to try to weed out biases like teams using Shotgun more often on third-and-long, or against prevent defenses in the fourth quarter. Shotgun offense is more efficient if you only look at the first half, on every down, and even if you only look at running back carries rather than passes and scrambles.
Clearly, NFL teams have figured the importance of the Shotgun out for themselves. In 2001, NFL teams only used Shotgun on 14 percent of plays. Five years later, in 2006, that had increased slightly, to 20 percent of plays. Shotgun usage has risen steadily since then, shattering the halfway point in 2013 (when NFL teams used Shotgun 59 percent of the time) and then creeping over the 60 percent threshold in 2014. Before 2007, no team had ever used Shotgun on more than half its offensive plays. In 2014, 26 different teams used Shotgun over 50 percent of the time, led by Philadelphia which used Shotgun on an NFL-record 86 percent of plays. (That was actually a very slight decline for Philadelphia, from 86.0 percent in 2013 to 85.6 percent last year.) It is likely that if teams continue to increase their usage of the Shotgun, defenses will adapt and the benefit of the formation will become less pronounced. But it certainly isn't happening yet; the difference between success on Shotgun and non-Shotgun plays in 2001 was bigger than in 2008, 2009, or 2010.
Terrell Davis, Jamal Anderson, and Edgerrin James all blew out their knees. Larry Johnson broke his foot. Earl Campbell and Eddie George went from legendary powerhouses to plodding, replacement-level players. Shaun Alexander broke his footand became a plodding, replacement-level player. This is what happens when a running back is overworked to the point of having at least 370 carries during the regular season.
The "Curse of 370" was expanded in Pro Football Prospectus 2006 to include seasons with 390 or more carries in the regular season and postseason combined. Research also shows that receptions don't cause a problem, only workload on the ground.
Plenty of running backs get injured without hitting 370 carries in a season, but there is a clear difference. On average, running backs with 300 to 369 carries and no postseason appearance will see their total rushing yardage decline by 15 percent the following year and their yards per carry decline by two percent. The average running back with 370 or more regular-season carries, or 390 including the postseason, will see their rushing yardage decline by 35 percent, and their yards per carry decline by eight percent. However, the Curse of 370 is not a hard and fast line where running backs suddenly become injury risks. It is more of a concept where 370 carries is roughly the point at which additional carries start to become more and more of a problem.
Research in Pro Football Prospectus 2008 suggests that overuse in college does not create a problem for top prospects, but also shows that players chosen after the first round rarely have a successful NFL career after a college season with over 330 carries.
It’s worth noting that the return to the committee backfields that dominated the '60s and '70s nearly meant an end to the Curse of 370. No running back had gone over 370 carries since Michael Turner in 2008 until DeMarco Murray ran the ball 392 times last year. The Cowboys, perhaps fearing for Murray's health, allowed him to enter unrestricted free agency.
In 2014, for example, Jeremy Maclin had 1,318 receiving yards, while Odell Beckham, Jr., was close behind with 1,305 yards. Both players ran routes of roughly the same average length. But there was a huge difference between them: Maclin caught 59 percent of passes, while Beckham had a catch rate of 70 percent.
We don't yet know enough to precisely parse the blame for incomplete passes, but we know that wide receiver catch rates are as consistent from year to year as quarterback completion percentages. However, it is also important to look at catch rate in the context of the types of routes each receiver runs. We recently expanded on this idea with a new plus/minus metric.
There are three units on a football team, but they are not of equal importance. Our DVOA ratings provide good evidence for this. For a long time, the saying from Football Outsiders was that the total quality of an NFL team is three parts offense, three parts defense, and one part special teams. Further recent research suggests that offense is even more important than we originally believed. Recent work by Chase Stuart, Neil Paine, and Brian Burke suggests a split between offense and defense of roughly 58-42, without considering special teams. Our research suggests that special teams contributes about 13 percent to total performance; if you measure the remaining 87 percent with a 58-42 ratio, you get roughly 4:3:1. When we compare the range of offense, defense, and special teams DVOA ratings, we get the same results, with the best and worst offenses roughly 130 percent stronger than the best and worst defenses, and roughly four times stronger than the best and worst special teams.
Offense is more consistent from year to year than defense, and offensive performance is easier to project than defensive performance. Special teams is less consistent than either.
Nobody in the NFL understands this concept better than Indianapolis Colts general manager Bill Polian. Both the Super Bowl champion Colts and the four-time AFC champion Buffalo Bills of the early 1990s were built around the idea that if you put together an offense that can dominate the league year after year, eventually you will luck into a year where good health and a few smart decisions will give you a defense good enough to win a championship. (As the Colts learned in January 2007, you don't even need a year, just four weeks.) Even the New England Patriots, who are led by a defense-first head coach in Bill Belichick, have been more consistent on offense than on defense since they began their run of success in 2001.
This theory, which originally appeared in the New York Times in October 2006, is one of our most controversial, but it is hard to argue against the evidence. Measuring every kicker from 2000 to 2014 who had at least ten field goal attempts in each of two consecutive years, the year-to-year correlation coefficient for field-goal percentage was just 0.135. The patron saint of this theory might be Mike Vanderjagt, whose field-goal percentage bounced around from 82 percent in 2001 to 74 percent in 2002, then a perfect 100 percent in 2003, followed by 80 percent in 2004, 92 percent in 2005, and 72 percent in 2006. Adam Vinatieri has long been considered the best kicker in the game. But even he had never enjoyed two straight seasons with accuracy better than the NFL average of 85 percent until 2011, when he followed up his 26-for-28 2010 campaign by going 23-for-27 (85.2 percent).
On the other hand, the year-to-year correlation coefficient for kickoff distance, over the same period as our measurement of field-goal percentage and with the same minimum of ten kicks per year, is 0.474, and the correlation coefficient for touchback rate is 0.814. The same players consistently lead the league in kickoff distance, particularly Graham Gano, Billy Cundiff, Olindo Mare, and Stephen Gostkowski.
Specific defensive penalties of course lose games; we’ve all sworn at the television when the cornerback on our favorite team gets flagged for a 50-yard pass interference penalty. Yet overall, there is no correlation between losses and the total of defensive penalties or even the total yardage on defensive penalties. One reason is that defensive penalties often representgood play, not bad. Cornerbacks who play tight coverage may be just on the edge of a penalty on most plays, only occasionally earning a flag. Defensive ends who get a good jump on rushing the passer will gladly trade an encroachment penalty or two for ten snaps where they get off the blocks a split-second before the linemen trying to block them.
In addition, offensive penalties have a higher correlation from year to year than defensive penalties. The penalty that correlates highest with losses is the false start, and the penalty that teams will have called most consistently from year to year is also the false start.
Stripping the ball is a skill. Holding onto the ball is a skill. Pouncing on the ball as it is bouncing all over the place is not a skill. There is no correlation whatsoever between the percentage of fumbles recovered by a team in one year and the percentage they recover in the next year. The odds of recovery are based solely on the type of play involved, not the teams or any of their players.
Fans like to insist that specific coaches can teach their teams to recover more fumbles by swarming to the ball. Chicago's Lovie Smith, in particular, is supposed to have this ability. However, in Smith’s first three seasons as head coach of the Bears, their rate of fumble recovery on defense went from a league-best 76 percent in 2004 to a league-worst 33 percent in 2005, then back to 67 percent in 2006.
Fumble recovery is equally erratic on offense. In 2008, the Bears fumbled 12 times on offense and recovered only three of them. In 2009, the Bears fumbled 18 times on offense, but recovered 13 of them.
Fumble recovery is a major reason why the general public overestimates or underestimates certain teams. Fumbles are huge, turning-point plays that dramatically impact wins and losses in the past, while fumble recovery percentage says absolutely nothing about a team's chances of winning games in the future. With this in mind, Football Outsiders stats treat all fumbles as equal, penalizing them based on the likelihood of each type of fumble (run, pass, sack, etc.) being recovered by the defense.
Other plays that qualify as "non-predictive events" include blocked kicks and touchdowns during turnover returns. These plays are not "lucky," per se, but they have no value whatsoever for predicting future performance.
Every yard line on the field has a value based on how likely a team is to score from that location on the field as opposed to from a yard further back. The change in value from one yard to the next is the same whether the team has the ball or not. The goal of a defense is not just to prevent scoring, but to hold the opposition so that the offense can get the ball back in the best possible field position. A bad offense will score as many points as a good offense if it starts each drive five yards closer to the goal line.
A corollary to this precept: The most underrated aspect of an NFL team's performance is the field position gained or lost on kickoffs and punts. This is part of why players like Devin Hester and Josh Cribbs can have such an impact on the game, even when they aren't taking a kickoff or punt all the way back for a touchdown.
Although play in the red zone has a disproportionately high importance to the outcome of games relative to plays on the rest of the field, NFL teams do not exhibit a level of performance in the red zone that is consistently better or worse than their performance elsewhere, year after year. The simplest explanation why is a small(er) sample size and the inherent variance of football, with contributing factors like injuries and changes in personnel.
We discovered this when creating our first team projection system in 2004. It said that the lowly San Diego Chargers would have one of the best offenses in the league, which seemed a little ridiculous. But looking closer, our projection system treated the previous year's performance on different downs as different variables, and the 2003 Chargers were actually good on first and second down, but terrible on third.
Teams get fewer opportunities on third down, so third-down performance is more volatile -- but it's also is a bigger part of a team's overall performance than first or second down, because the result is usually either very good (four more downs) or very bad (losing the ball to the other team with a punt). Over time, a team will play as well in those situations as it does in other situations, which will bring the overall offense or defense in line with the offense and defense on first and second down.
This trend is even stronger between seasons. Struggles on third down are a pretty obvious problem, and teams will generally target their off-season moves at improving their third-down performance ... which often leads to an improvement in third-down performance.
However, we have discovered something surprising over the past three years: The third-down rebound effect seems to have disappeared on offense, as we explain in the Philadelphia chapter of Football Outsiders Almanac 2010. We don't know yet if this change is temporary or permanent, and there is no such change on defense.
There are no doubt teams with streaks of good or bad health over multiple years. However, teams who were especially healthy or especially unhealthy, as measured by our Adjusted Games Lost (AGL) metric, almost always head towards league average in the subsequent season. Furthermore, injury -- or the absence thereof -- has a huge correlation with wins, and a significant impact on a team's success. There's no doubt that a few high-profile teams have resisted this trend in recent years. The Patriots seem to overcome injuries every year, and the last three Super Bowl champions were among the most-injured teams in the league in each of those seasons. Nonetheless, the overall rule still applies. The Patriots and Giants made it to Super Bowl XLVI despite being among the most injured teams in the league, but they also were the only two teams in the top 15 of our AGL metric that made the playoffs at all. In 2014, five of the seven least-injured teams in the league made the playoffs, and only one playoff team (Indianapolis) finished outside the top 20.
Teams with a high number of injuries are a good bet to improve the following season. However, while injury totals tend to regress towards the mean, there's also no doubt that certain teams have a record of staying healthier than others. Indianapolis finished 30th in 2012, 2013, and 2014, and has ranked 24th or worse for nine straight years.
Connected to the previous statement, because teams need to go into the season expecting that they will suffer an average number of injuries no matter how healthy they were the previous year. The Redskins went into 2006 with a Super Bowl-quality starting lineup, and finished 5-11 because they had no depth. You cannot concentrate your salaries on a handful of star players because there is no such thing as avoiding injuries in the NFL. Every team will suffer injuries; the only question is how many. The game is too fast and the players too strong to build a team based around the idea that "if we can avoid all injuries this year, we'll win."
Running backs usually decline after age 28, tight ends after age 29, wide receivers after age 30, and quarterbacks after age 32.
This research was originally done by Doug Drinen (editor of Pro-Football-Reference.com). In recent years, a few players have had huge seasons above these general age limits (most notably Tiki Barber, Tony Gonzalez, and Terrell Owens), but the peak ages Drinen found a few years ago still apply to the majority of players.
During the summer of 2007, ESPN The Magazine asked us to research when players decline at "non-skill" positions. This research was not as rigorous as our usual work, and needs a little more attention before we're ready to stand by it. For the curious, however, the preliminary results said that defensive ends and defensive backs generally begin to decline after age 29, linebackers and offensive linemen after age 30, and defensive tackles after age 31.
QBASE, our latest attempt to project the NFL careers of draft-eligible quarterbacks, was developed by Andrew Healy and unveiled in April of 2015. The math behind QBASE is complex, but the idea is simple: quarterbacks who played well in college over a large sample size and also impressed scouts with their intangible qualities are more likely to succeed in the pro ranks than their peers. The trick is identifying prospects who meet all three criteria.
From 1997 to 2010, the top QBASE scores for quarterbacks taken in the first 100 picks of the draft belonged to Philip Rivers, Carson Palmer, Donovan McNabb, Peyton Manning, Byron Leftwich, Aaron Rodgers, Ben Roethlisberger, John Beck, Matthew Stafford, Chad Pennington, and Daunte Culpepper. With the obvious exceptions of Leftwich and Beck (hey, nobody's perfect), each of those quarterbacks started for at least five years in the NFL and led his team(s) to multiple playoff berths.
At the other extreme, the worst QBASE scores among highly drafted quarterbacks in that timeframe belonged to Josh McCown, Trent Edwards, Charlie Batch, Kevin O'Connell, Michael Vick, Charlie Whitehurst, Marques Tuiasosopo, Chris Simms, Patrick Ramsey, and Brodie Croyle. None of those players ever developed into a consistently efficient passer.
Prior to 2015, we evaluated collegiate quarterback prospects using the Lewin Career Forecast, named after its creator David Lewin and later re-named the Long-term Career Forecast. It was a simpler system that used completion percentage and games started in college to project each prospect's career in the NFL. We've left the links here if you'd like to explore how our methodology has evolved over time.
Football Outsiders Almanac 2009 introduced a new metric called Playmaker Score, which measures rookie wide receivers by simply multiplying average yards per reception and total career touchdowns in college. Players who score high in this metric do not necessarily become stars in the NFL, but no first- or second-round pick with a score below 8.0 has yet to live up to his draft position. Like the Lewin Career Forecast, Playmaker Score is far more accurate with receivers chosen in the first two rounds, and it doesn't seem to work for hybrid slot receiver/running backs such as Percy Harvin and Dexter McCluster.
It may seem strange because graduation enforces constant player turnover, but college football teams are actually much more consistent from year to year than NFL teams. Thanks in large part to consistency in recruiting, teams can be expected to play within a reasonable range of their baseline program expectations each season. Our Program F/+ ratings, which represent a rolling five-year period of play-by-play and drive efficiency data, have an extremely strong (.76) correlation with the next year’s F/+ rating.
Championship teams are generally defined by their ability to dominate inferior opponents, not their ability to win close games.
Football games are often decided by just one or two plays -- a missed field goal, a bouncing fumble, the subjective spot of an official on fourth-and-1. One missed assignment by a cornerback, or one slightly askew pass that bounces off a receiver's hands and into those of a defensive back five yards away and the game could be over. In a blowout, however, one lucky bounce isn't going to change things.
Championship teams beat their good opponents convincingly and destroy the cupcakes on the schedule. Certainly there are exceptions to this rule, including last year's Super Bowl champion. However, in the DVOA era (1989-2014), 23 of 26 Super Bowl champions have had more blowouts against sub-.500 teams than close wins against above-.500 teams.
Football Outsiders Basics (a.k.a. "Pregame Show")
by Aaron Schatz
While reading Football Outsiders or the Football Outsiders Almanac, new readers will often come across an offhand comment about, for example, the idea that fumble recovery is not a skill, and wonder what in the heck we are talking about. In each edition of Football Outsiders Almanac, we now include an essay called "Pregame Show" which gives a basic look at some of the most important precepts that have emerged during nine years of Football Outsiders research. That essay is republished here, along with links to the original research online when possible, or mentions of where that research appeared in print.
Please note that some of our basic research findings, such as the split between offense, defense, and special teams, were never addressed in one specific article, but instead developed over time. Therefore, there is no specific article we can point out.
For now, this article only includes precepts from our NFL research, although we plan to include precepts from college research in the near future.
You run when you win, not win when you run.
The first article ever written for Football Outsiders was devoted to debunking the myth of "establishing the run." There is no correlation whatsoever between giving your running backs a lot of carries early in the game and winning the game. Just running the ball is not going to help a team score; it has to run successfully.
There are two reasons why nearly every beat writer and television analyst still repeats the tired old-school mantra that "establishing the run" is the secret to winning football games. The first problem is confusing cause and effect. There are exceptions, usually when the opponent is strong in every area except run defense, like last year's Green Bay Packers. However, in general, winning teams have a lot of carries because their running backs are running out the clock at the end of wins, not because they are running wild early in games.
The second problem is history. Most of the current crop of NFL analysts came of age or actually played the game during the 1970s. They believe that the run-heavy game of that decade is how football is meant to be, and today's pass-first game is an aberration. As we addressed in an essay in Pro Football Prospectus 2007 about the history of NFL stats, it was actually the game of the 1970s that was the aberration. The seventies were far more slanted towards the run than any era since the arrival of Paul Brown, Otto Graham, and the Cleveland Browns in 1946. Optimal strategies from 1974 are not optimal strategies for today's game.
A sister statement to "you have to establish the run" is "team X is 5-1 when running back John Doe runs for at least 100 yards." Unless John Doe is ripping off six-yard gains Jamaal Charles-style, the team isn't winning because of his 100-yard games. He's putting up 100-yard games because his team is winning.
- The Establishment Clause, July 2003
This is a corollary to the absurdity of "establish the run." If you don't believe us, meet our good friends the 2006-2007 Minnesota Vikings (or, for a more recent example, the 2014 St. Louis Rams). With rare exceptions, teams win or lose with the passing game more than the running game -- and by stopping the passing game more than the running game. The reason why teams need a strong run defense in the playoffs is not to shut the run down early, it's to keep the other team from icing the clock if they get a lead. You can't mount a comeback if you can't stop the run.
Note that "good pass defense" may mean "good pass rush" rather than "good defensive backs."
Running on third-and-short is more likely to convert than passing on third-and-short.
On average, passing will always gain more yardage than running, with one very important exception: when a team is just one or two yards away from a new set of downs or the goal line. On third-and-1, a run will convert for a new set of downs 36 percent more often than a pass. Expand that to all third or fourth downs with 1-2 yards to go, and the run is successful 40 percent more often. The gap has narrowed in recent years, but the numbers still favor running. Since 2010, teams have converted 68 percent of runs on third-and-1 (including fourth-down plays), compared to 55 percent of runs in the same situations. Expand that to all third or fourth downs with 1 or 2 yards to go, and running still has the edge, 66 percent to 52 percent. With these percentages, the possibility of a long gain with a pass is not worth the tradeoff of an incomplete that kills a drive.
This is one reason why teams have to be able to both run and pass. The offense also has to keep some semblance of balance so they can use their play-action fakes, and so the defense doesn't just run their nickel and dime packages all game. Balance also means that teams do need to pass occasionally in short-yardage situations; they just need to do it less than they do now. Teams pass 67 percent of the time on third-and-2 even though 60 percent of runs in that situation convert, while passes convert 51 percent of the time. They pass 77 percent of the time on fourth-and-2 even though 69 percent of runs in that situation convert, while passes on fourth-and-2 convert only 48 percent of the time.
- 'Tis Better to Have Rushed and Lost Than Never to Have Rushed at All, January 2004
- Pro Football Forecast 2004, Buffalo chapter
- Pro Football Prospectus 2005, Detroit chapter
When you open your newspaper on Sunday morning, you'll see that the little agate-type previews of each game list team rankings by total yardage. That is still how the NFL "officially" ranks teams, but these rankings rarely match up with common sense. That is because total team yardage may be the most context-dependent number in football.
It starts with the basic concept that rate stats are generally more valuable than cumulative stats. Yards per carry says more about a running back's quality than total yardage; completion percentage says more than just a quarterback's total number of completions. The same thing is true for teams; in fact, it is even more important because of the way football strategy influences the number of runs and passes in the game plan. Poor teams will give up fewer passing yards and more rushing yards because opponents with a late game lead stop passing in order to run out the clock. For winning teams, the opposite is true.
In 2014, who had a better pass defense: San Diego or Arizona? According the official NFL rankings, San Diego (3,249 yards allowed on 550 passing plays) was the fourth-best pass defense in football, while Arizona (3,908 yards allowed on 614 passing plays) ranked 29th. The Arizona pass defense, though, ranked 14th in DVOA (3.2%), while San Diego was 25th (16.4%). That's a big reason the Cardinals made the playoffs and the Chargers just missed.
Total yardage rankings are also skewed because some teams play at a faster pace than other teams. In 2014, Philadelphia's offense ranked fifth with 6,348 total yards, while Green Bay was sixth with 6,178 yards. Philadelphia, though, led the league with 201 offensive drives, while Green Bay was last with 163 drives. On a per-drive basis, Green Bay's offense gained substantially more yards than that of the Eagles.
- Pro Football Prospectus 2005, Cleveland chapter
- Pro Football Prospectus 2005, New York Jets chapter
This sounds absurdly basic, but when people consider team and player stats without looking at strength of schedule, they are ignoring this. In 2007, Tony Romo and Derek Anderson had similar numbers, but Romo faced a much tougher schedule than Anderson did. Romo was better that year, and better in the long run.
In 2014, Le'Veon Bell played 12 games that finished in the bottom half of the league in run defense DVOA. He averaged 96.8 yards per game and 4.9 yards per carry in those contests. In his four outings against upper-half run defenses, those averages fell to 49.8 yards per game and 3.8 yards per carry.
Because players and teams don't give the exact same performance every week, this is more of a general law, and it doesn't necessarily apply in the short term.
- Pro Football Prospectus 2005, Cincinnati chapter
Our brethren at Baseball Prospectus believe that the most precious commodity in baseball is outs. Teams only get 27 of them per game, and you can't afford to give one up for very little return. So imagine if there was a new rule in baseball that gave a team a way to earn another three outs in the middle of the inning. That would be pretty useful, right?
That's the way football works. You may start a drive 80 yards away from scoring, but as long as you can earn 10 yards in four chances, you get another four chances. Long gains have plenty of value, but if those long gains are mixed with a lot of short gains, you are going to put the quarterback in a lot of difficult third-and-long situations. That means more punts and more giving the ball back to the other team rather than moving the chains and giving the offense four more plays to work with.
The running back who gains consistent yardage is also going to do a lot more for you late in the game, when the goal of running the ball is not just to gain yardage but to eat clock time. If you are a Baltimore Ravens fan watching your team with a late lead, you don't want to see three straight Justin Forsett stuffs at the line followed by a punt. You want to see a game-icing first down.
A common historical misconception is that our preference for consistent running backs means that "Football Outsiders believes that Barry Sanders was overrated." Sanders wasn't just any boom-and-bust running back, though; he was the greatest boom-and-bust runner of all time, with bigger booms and fewer busts. Our play-by-play database goes back to 1989, the first year of Sanders' career. Sanders finished in the top five of rushing DYAR in 1989, 1990, 1991, 1994, 1996, and 1997.
Rushing is more dependent on the offensive line than people realize, but pass protection is more dependent on the quarterback himself than people realize.
Some readers complain that this idea contradicts the previous one. Aren't those consistent running backs just the product of good offensive lines? The truth is somewhere in between. There are certainly good running backs who suffer because their offensive lines cannot create consistent holes (Matt Forte is a recent example). Most boom-and-bust running backs, however, contribute to their own problems by hesitating behind the line whenever the hole is unclear, looking for the home run instead of charging forward for the four-yard gain that keeps the offense moving.
As for pass protection, some quarterbacks have better instincts for the rush than others, and are thus better at getting out of trouble by moving around in the pocket or throwing the ball away. Others will hesitate, hold onto the ball too long, and lose yardage over and over.
Note that "moving around in the pocket" does not necessarily mean "scrambling." In fact, a scrambling quarterback will often take more sacks than a pocket quarterback, because while he's running around trying to make something happen, a defensive lineman will catch up with him.
- Passing: Fun With Sacks, Part II, December 2003
- Rushing: Anything written about the Arizona Cardinals during 2006
Over the past five seasons, offenses have averaged 5.8 yards per play from Shotgun (not counting the Wildcat or other college-style option plays), but just 5.0 yards per play with the quarterback under center. This wide split exists even if you analyze the data to try to weed out biases like teams using Shotgun more often on third-and-long, or against prevent defenses in the fourth quarter. Shotgun offense is more efficient if you only look at the first half, on every down, and even if you only look at running back carries rather than passes and scrambles.
Clearly, NFL teams have figured the importance of the Shotgun out for themselves. In 2001, NFL teams only used Shotgun on 14 percent of plays. Five years later, in 2006, that had increased slightly, to 20 percent of plays. Shotgun usage has risen steadily since then, shattering the halfway point in 2013 (when NFL teams used Shotgun 59 percent of the time) and then creeping over the 60 percent threshold in 2014. Before 2007, no team had ever used Shotgun on more than half its offensive plays. In 2014, 26 different teams used Shotgun over 50 percent of the time, led by Philadelphia which used Shotgun on an NFL-record 86 percent of plays. (That was actually a very slight decline for Philadelphia, from 86.0 percent in 2013 to 85.6 percent last year.) It is likely that if teams continue to increase their usage of the Shotgun, defenses will adapt and the benefit of the formation will become less pronounced. But it certainly isn't happening yet; the difference between success on Shotgun and non-Shotgun plays in 2001 was bigger than in 2008, 2009, or 2010.
- Pro Football Prospectus 2007, Tampa Bay chapter
Terrell Davis, Jamal Anderson, and Edgerrin James all blew out their knees. Larry Johnson broke his foot. Earl Campbell and Eddie George went from legendary powerhouses to plodding, replacement-level players. Shaun Alexander broke his footand became a plodding, replacement-level player. This is what happens when a running back is overworked to the point of having at least 370 carries during the regular season.
The "Curse of 370" was expanded in Pro Football Prospectus 2006 to include seasons with 390 or more carries in the regular season and postseason combined. Research also shows that receptions don't cause a problem, only workload on the ground.
Plenty of running backs get injured without hitting 370 carries in a season, but there is a clear difference. On average, running backs with 300 to 369 carries and no postseason appearance will see their total rushing yardage decline by 15 percent the following year and their yards per carry decline by two percent. The average running back with 370 or more regular-season carries, or 390 including the postseason, will see their rushing yardage decline by 35 percent, and their yards per carry decline by eight percent. However, the Curse of 370 is not a hard and fast line where running backs suddenly become injury risks. It is more of a concept where 370 carries is roughly the point at which additional carries start to become more and more of a problem.
Research in Pro Football Prospectus 2008 suggests that overuse in college does not create a problem for top prospects, but also shows that players chosen after the first round rarely have a successful NFL career after a college season with over 330 carries.
It’s worth noting that the return to the committee backfields that dominated the '60s and '70s nearly meant an end to the Curse of 370. No running back had gone over 370 carries since Michael Turner in 2008 until DeMarco Murray ran the ball 392 times last year. The Cowboys, perhaps fearing for Murray's health, allowed him to enter unrestricted free agency.
- Ricky Williams is Pretty Much Screwed, July 2004
- 370 Carries Revisited, Seattle chapter of Pro Football Prospectus 2006, republished online January 2007
- Pro Football Prospectus 2008, Detroit chapter
In 2014, for example, Jeremy Maclin had 1,318 receiving yards, while Odell Beckham, Jr., was close behind with 1,305 yards. Both players ran routes of roughly the same average length. But there was a huge difference between them: Maclin caught 59 percent of passes, while Beckham had a catch rate of 70 percent.
We don't yet know enough to precisely parse the blame for incomplete passes, but we know that wide receiver catch rates are as consistent from year to year as quarterback completion percentages. However, it is also important to look at catch rate in the context of the types of routes each receiver runs. We recently expanded on this idea with a new plus/minus metric.
- Ch-Ch-Chambers, December 2006, Boston Sports Media Watch
- Receiving Plus/Minus, August 2009
- Scaled Plus/Minus, July 2010
There are three units on a football team, but they are not of equal importance. Our DVOA ratings provide good evidence for this. For a long time, the saying from Football Outsiders was that the total quality of an NFL team is three parts offense, three parts defense, and one part special teams. Further recent research suggests that offense is even more important than we originally believed. Recent work by Chase Stuart, Neil Paine, and Brian Burke suggests a split between offense and defense of roughly 58-42, without considering special teams. Our research suggests that special teams contributes about 13 percent to total performance; if you measure the remaining 87 percent with a 58-42 ratio, you get roughly 4:3:1. When we compare the range of offense, defense, and special teams DVOA ratings, we get the same results, with the best and worst offenses roughly 130 percent stronger than the best and worst defenses, and roughly four times stronger than the best and worst special teams.
Offense is more consistent from year to year than defense, and offensive performance is easier to project than defensive performance. Special teams is less consistent than either.
Nobody in the NFL understands this concept better than Indianapolis Colts general manager Bill Polian. Both the Super Bowl champion Colts and the four-time AFC champion Buffalo Bills of the early 1990s were built around the idea that if you put together an offense that can dominate the league year after year, eventually you will luck into a year where good health and a few smart decisions will give you a defense good enough to win a championship. (As the Colts learned in January 2007, you don't even need a year, just four weeks.) Even the New England Patriots, who are led by a defense-first head coach in Bill Belichick, have been more consistent on offense than on defense since they began their run of success in 2001.
- Turnovers and the Unpredictability of Defense, April 2004
- Mentioned in 2005 DVOA Projections, September 2005
This theory, which originally appeared in the New York Times in October 2006, is one of our most controversial, but it is hard to argue against the evidence. Measuring every kicker from 2000 to 2014 who had at least ten field goal attempts in each of two consecutive years, the year-to-year correlation coefficient for field-goal percentage was just 0.135. The patron saint of this theory might be Mike Vanderjagt, whose field-goal percentage bounced around from 82 percent in 2001 to 74 percent in 2002, then a perfect 100 percent in 2003, followed by 80 percent in 2004, 92 percent in 2005, and 72 percent in 2006. Adam Vinatieri has long been considered the best kicker in the game. But even he had never enjoyed two straight seasons with accuracy better than the NFL average of 85 percent until 2011, when he followed up his 26-for-28 2010 campaign by going 23-for-27 (85.2 percent).
On the other hand, the year-to-year correlation coefficient for kickoff distance, over the same period as our measurement of field-goal percentage and with the same minimum of ten kicks per year, is 0.474, and the correlation coefficient for touchback rate is 0.814. The same players consistently lead the league in kickoff distance, particularly Graham Gano, Billy Cundiff, Olindo Mare, and Stephen Gostkowski.
- "NFL Kickers Are Judged on the Wrong Criteria," New York Times, November 12, 2006
- Pro Football Prospectus 2007, Arizona chapter
Specific defensive penalties of course lose games; we’ve all sworn at the television when the cornerback on our favorite team gets flagged for a 50-yard pass interference penalty. Yet overall, there is no correlation between losses and the total of defensive penalties or even the total yardage on defensive penalties. One reason is that defensive penalties often representgood play, not bad. Cornerbacks who play tight coverage may be just on the edge of a penalty on most plays, only occasionally earning a flag. Defensive ends who get a good jump on rushing the passer will gladly trade an encroachment penalty or two for ten snaps where they get off the blocks a split-second before the linemen trying to block them.
In addition, offensive penalties have a higher correlation from year to year than defensive penalties. The penalty that correlates highest with losses is the false start, and the penalty that teams will have called most consistently from year to year is also the false start.
- Pro Football Prospectus 2007, St. Louis chapter
Stripping the ball is a skill. Holding onto the ball is a skill. Pouncing on the ball as it is bouncing all over the place is not a skill. There is no correlation whatsoever between the percentage of fumbles recovered by a team in one year and the percentage they recover in the next year. The odds of recovery are based solely on the type of play involved, not the teams or any of their players.
Fans like to insist that specific coaches can teach their teams to recover more fumbles by swarming to the ball. Chicago's Lovie Smith, in particular, is supposed to have this ability. However, in Smith’s first three seasons as head coach of the Bears, their rate of fumble recovery on defense went from a league-best 76 percent in 2004 to a league-worst 33 percent in 2005, then back to 67 percent in 2006.
Fumble recovery is equally erratic on offense. In 2008, the Bears fumbled 12 times on offense and recovered only three of them. In 2009, the Bears fumbled 18 times on offense, but recovered 13 of them.
Fumble recovery is a major reason why the general public overestimates or underestimates certain teams. Fumbles are huge, turning-point plays that dramatically impact wins and losses in the past, while fumble recovery percentage says absolutely nothing about a team's chances of winning games in the future. With this in mind, Football Outsiders stats treat all fumbles as equal, penalizing them based on the likelihood of each type of fumble (run, pass, sack, etc.) being recovered by the defense.
Other plays that qualify as "non-predictive events" include blocked kicks and touchdowns during turnover returns. These plays are not "lucky," per se, but they have no value whatsoever for predicting future performance.
- Pro Football Prospectus 2005, New Orleans chapter
Every yard line on the field has a value based on how likely a team is to score from that location on the field as opposed to from a yard further back. The change in value from one yard to the next is the same whether the team has the ball or not. The goal of a defense is not just to prevent scoring, but to hold the opposition so that the offense can get the ball back in the best possible field position. A bad offense will score as many points as a good offense if it starts each drive five yards closer to the goal line.
A corollary to this precept: The most underrated aspect of an NFL team's performance is the field position gained or lost on kickoffs and punts. This is part of why players like Devin Hester and Josh Cribbs can have such an impact on the game, even when they aren't taking a kickoff or punt all the way back for a touchdown.
- How Many Points is a Turnover Worth?, September 2003
- Also, see the book The Hidden Game of Football by Pete Palmer, Bob Carroll, and John Thorn
Although play in the red zone has a disproportionately high importance to the outcome of games relative to plays on the rest of the field, NFL teams do not exhibit a level of performance in the red zone that is consistently better or worse than their performance elsewhere, year after year. The simplest explanation why is a small(er) sample size and the inherent variance of football, with contributing factors like injuries and changes in personnel.
- Football Outsiders Almanac 2009, "The Red Menace"
We discovered this when creating our first team projection system in 2004. It said that the lowly San Diego Chargers would have one of the best offenses in the league, which seemed a little ridiculous. But looking closer, our projection system treated the previous year's performance on different downs as different variables, and the 2003 Chargers were actually good on first and second down, but terrible on third.
Teams get fewer opportunities on third down, so third-down performance is more volatile -- but it's also is a bigger part of a team's overall performance than first or second down, because the result is usually either very good (four more downs) or very bad (losing the ball to the other team with a punt). Over time, a team will play as well in those situations as it does in other situations, which will bring the overall offense or defense in line with the offense and defense on first and second down.
This trend is even stronger between seasons. Struggles on third down are a pretty obvious problem, and teams will generally target their off-season moves at improving their third-down performance ... which often leads to an improvement in third-down performance.
However, we have discovered something surprising over the past three years: The third-down rebound effect seems to have disappeared on offense, as we explain in the Philadelphia chapter of Football Outsiders Almanac 2010. We don't know yet if this change is temporary or permanent, and there is no such change on defense.
- Pro Football Prospectus 2005, San Diego chapter
- "Third Down is the Charm for NFL Turnarounds", New York Times, August 2005
- Football Outsiders Almanac 2010, Philadelphia chapter
There are no doubt teams with streaks of good or bad health over multiple years. However, teams who were especially healthy or especially unhealthy, as measured by our Adjusted Games Lost (AGL) metric, almost always head towards league average in the subsequent season. Furthermore, injury -- or the absence thereof -- has a huge correlation with wins, and a significant impact on a team's success. There's no doubt that a few high-profile teams have resisted this trend in recent years. The Patriots seem to overcome injuries every year, and the last three Super Bowl champions were among the most-injured teams in the league in each of those seasons. Nonetheless, the overall rule still applies. The Patriots and Giants made it to Super Bowl XLVI despite being among the most injured teams in the league, but they also were the only two teams in the top 15 of our AGL metric that made the playoffs at all. In 2014, five of the seven least-injured teams in the league made the playoffs, and only one playoff team (Indianapolis) finished outside the top 20.
Teams with a high number of injuries are a good bet to improve the following season. However, while injury totals tend to regress towards the mean, there's also no doubt that certain teams have a record of staying healthier than others. Indianapolis finished 30th in 2012, 2013, and 2014, and has ranked 24th or worse for nine straight years.
- Pro Football Prospectus 2008, "The Injury Effect"
Connected to the previous statement, because teams need to go into the season expecting that they will suffer an average number of injuries no matter how healthy they were the previous year. The Redskins went into 2006 with a Super Bowl-quality starting lineup, and finished 5-11 because they had no depth. You cannot concentrate your salaries on a handful of star players because there is no such thing as avoiding injuries in the NFL. Every team will suffer injuries; the only question is how many. The game is too fast and the players too strong to build a team based around the idea that "if we can avoid all injuries this year, we'll win."
Running backs usually decline after age 28, tight ends after age 29, wide receivers after age 30, and quarterbacks after age 32.
This research was originally done by Doug Drinen (editor of Pro-Football-Reference.com). In recent years, a few players have had huge seasons above these general age limits (most notably Tiki Barber, Tony Gonzalez, and Terrell Owens), but the peak ages Drinen found a few years ago still apply to the majority of players.
During the summer of 2007, ESPN The Magazine asked us to research when players decline at "non-skill" positions. This research was not as rigorous as our usual work, and needs a little more attention before we're ready to stand by it. For the curious, however, the preliminary results said that defensive ends and defensive backs generally begin to decline after age 29, linebackers and offensive linemen after age 30, and defensive tackles after age 31.
- Offensive age estimates based on How Important is Age? by Doug Drinen, written in 2000.
QBASE, our latest attempt to project the NFL careers of draft-eligible quarterbacks, was developed by Andrew Healy and unveiled in April of 2015. The math behind QBASE is complex, but the idea is simple: quarterbacks who played well in college over a large sample size and also impressed scouts with their intangible qualities are more likely to succeed in the pro ranks than their peers. The trick is identifying prospects who meet all three criteria.
From 1997 to 2010, the top QBASE scores for quarterbacks taken in the first 100 picks of the draft belonged to Philip Rivers, Carson Palmer, Donovan McNabb, Peyton Manning, Byron Leftwich, Aaron Rodgers, Ben Roethlisberger, John Beck, Matthew Stafford, Chad Pennington, and Daunte Culpepper. With the obvious exceptions of Leftwich and Beck (hey, nobody's perfect), each of those quarterbacks started for at least five years in the NFL and led his team(s) to multiple playoff berths.
At the other extreme, the worst QBASE scores among highly drafted quarterbacks in that timeframe belonged to Josh McCown, Trent Edwards, Charlie Batch, Kevin O'Connell, Michael Vick, Charlie Whitehurst, Marques Tuiasosopo, Chris Simms, Patrick Ramsey, and Brodie Croyle. None of those players ever developed into a consistently efficient passer.
Prior to 2015, we evaluated collegiate quarterback prospects using the Lewin Career Forecast, named after its creator David Lewin and later re-named the Long-term Career Forecast. It was a simpler system that used completion percentage and games started in college to project each prospect's career in the NFL. We've left the links here if you'd like to explore how our methodology has evolved over time.
- College Quarterbacks Through the Prism of Statistics, March 2006
- Pro Football Prospectus 2006, "Projecting College Quarterbacks"
- Pro Football Prospectus 2007, "Projecting College Quarterbacks Revisited"
- Introducing QBASE, April 2015
Football Outsiders Almanac 2009 introduced a new metric called Playmaker Score, which measures rookie wide receivers by simply multiplying average yards per reception and total career touchdowns in college. Players who score high in this metric do not necessarily become stars in the NFL, but no first- or second-round pick with a score below 8.0 has yet to live up to his draft position. Like the Lewin Career Forecast, Playmaker Score is far more accurate with receivers chosen in the first two rounds, and it doesn't seem to work for hybrid slot receiver/running backs such as Percy Harvin and Dexter McCluster.
- Football Outsiders Almanac 2009, "Introducing Playmaker Score"
It may seem strange because graduation enforces constant player turnover, but college football teams are actually much more consistent from year to year than NFL teams. Thanks in large part to consistency in recruiting, teams can be expected to play within a reasonable range of their baseline program expectations each season. Our Program F/+ ratings, which represent a rolling five-year period of play-by-play and drive efficiency data, have an extremely strong (.76) correlation with the next year’s F/+ rating.
Championship teams are generally defined by their ability to dominate inferior opponents, not their ability to win close games.
Football games are often decided by just one or two plays -- a missed field goal, a bouncing fumble, the subjective spot of an official on fourth-and-1. One missed assignment by a cornerback, or one slightly askew pass that bounces off a receiver's hands and into those of a defensive back five yards away and the game could be over. In a blowout, however, one lucky bounce isn't going to change things.
Championship teams beat their good opponents convincingly and destroy the cupcakes on the schedule. Certainly there are exceptions to this rule, including last year's Super Bowl champion. However, in the DVOA era (1989-2014), 23 of 26 Super Bowl champions have had more blowouts against sub-.500 teams than close wins against above-.500 teams.
- Guts and Stomps, December 2005
