NFL Team Rating#

NFL_Team_Rating.ipynb Open In Colab Kaggle Gradient Open In SageMaker Studio Lab Hits

Description: NFL Team Rating problem from the Analytical Decision Modeling course at the Arizona State University.

Tags: educational, quadratic, amplpy, gurobi

Notebook author: Yimin Wang <yimin_wang@asu.edu>, Marcos Dominguez Velad <marcos@ampl.com>

Model author: Yimin Wang <yimin_wang@asu.edu>

References:

  1. Analytical Decision Modeling course at W. P. Carey School of Business. Syllabus: https://catalog.apps.asu.edu/catalog/classes/classlist?keywords=86683&searchType=all&term=2237&collapse=Y

Objective and Prerequisites#

This NFL team rating problem shows you how to determine the optimal rating of sporting teams based on their past performances. We use the root mean squared error (RMSE) as a criteria to determine what set of ratings is most accurate. The objectives of the sports rating problem are:

  • Find the best possible set of ratings for all teams,

  • Encorporate possible home team advantages,

  • Minimize the errors when predicting outcomes of future matches, and

  • Ensure that the ratings are consistent and identifiable.

Problem Description#

picture

We consider the NFL games from 32 teams. The teams are indexed 1 to 32, for example, team 1 is Arizon, team 2 is Atlanta, and so on.

Team Number

Team name

Team Number

Team name

1

Arizona Cardinals

17

Miami Dolphins

2

Atlanta Falcons

18

Minnesota Vikings

3

Baltimore Ravens

19

New England Patriots

4

Buffalo Bills

20

New Orleans Saints

5

Carolina Panthers

21

New York Giants

6

Chicago Bears

22

New York Jets

7

Cincinnati Bengals

23

Oakland Raiders

8

Cleveland Browns

24

Philadelphia Eagles

9

Dallas Cowboys

25

Pittsburgh Steelers

10

Denver Broncos

26

St. Louis Rams

11

Detroit Lions

27

San Diego Chargers

12

Green Bay Packers

28

San Francisco 49ers

13

Houston Texans

29

Seattle Seahawks

14

Indianapolis Colts

30

Tampa Bay Buccaneers

15

Jacksonville Jaguars

31

Tennessee Titans

16

Kansas City Chiefs

32

Washington Redskins

Table below illustrates some of the results of the 256 regular season NFL games from the 2015 season. The first game is team 10 Denver versus team 3 Baltimore, played at Denver. Denver won the game by a score of 49 to 27, and the point spread (home team score minus vistor team score) is calculated as the difference between the home team score and the visiting team score. In the above example, the point spread is 49-27=22.

Week

Match

Home team index

Visiting team index

Home team score

Visiting team score

Point spread

1

1

10

3

49

27

22

1

2

5

29

7

12

-5

1

3

15

16

2

28

-26

1

4

22

30

18

17

1

1

5

26

1

27

24

3

1

6

9

21

36

31

5

1

7

28

12

34

28

6

1

8

20

2

23

17

6

1

9

14

23

21

17

4

1

10

6

7

24

21

3

1

11

11

18

34

24

10

1

12

4

19

21

23

-2

1

13

25

31

9

16

-7

…

…

…

…

…

…

…

…

…

…

…

…

…

…

…

…

…

…

…

…

…

17

256

17

22

7

20

-13

We would like to find a best rating system that most accurately predicts the matches.

Model Formulation#

Indices#

\(i,j \in \{1..32\}\): Index to represent teams

\(t \in \{1..256\}\): Index to represent different games

Parameters#

\(p_{ijt}\): point spread between team \(i\) and team \(j\) in game \(t\)

Decision Variables#

\(x_{i}\): Ratings of team \(i\), \(i \in \{1..32\}\)

\(y\): Home team advantage

Objective Function#

  • Prediction Accuracy. We want to minimize the prediction error.

\[ \text{Min}_{x_{i},y} \quad \frac{\sqrt{\sum_{t \in \{1..256\}} (x_i + y - x_j - p_{ijt})^2}}{256} \]

Constraints#

\[ \sum_i x_i = 32*85 \ \ \ \text{(fix average rating to be 85)} \]

Python Implementation#

We now import the AMPL Python Module and other Python libraries.

# Install dependencies
%pip install -q amplpy

# Google Colab & Kaggle integration
from amplpy import AMPL, ampl_notebook

ampl = ampl_notebook(
    modules=["gurobi"],  # modules to install
    license_uuid="default",  # license to use
)  # instantiate AMPL object and register magics

Set up the inputs#

#####################################################
#                    Model Formulation
#####################################################

# from 1 to 32
team = [*range(1, 33)]
# from 0 to 255
game = [*range(1, 257)]

team_label = [
    "Arizona Cardinals",
    "Atlanta Falcons",
    "Baltimore Ravens",
    "Buffalo Bills",
    "Carolina Panthers",
    "Chicago Bears",
    "Cincinnati Bengals",
    "Cleveland Browns",
    "Dallas Cowboys",
    "Denver Broncos",
    "Detroit Lions",
    "Green Bay Packers",
    "Houston Texans",
    "Indianapolis Colts",
    "Jacksonville Jaguars",
    "Kansas City Chiefs",
    "Miami Dolphins",
    "Minnesota Vikings",
    "New England Patriots",
    "New Orleans Saints",
    "New York Giants",
    "New York Jets",
    "Oakland Raiders",
    "Philadelphia Eagles",
    "Pittsburgh Steelers",
    "St. Louis Rams",
    "San Diego Chargers",
    "San Francisco 49ers",
    "Seattle Seahawks",
    "Tampa Bay Buccaneers",
    "Tennessee Titans",
    "Washington Redskins",
]

# past game performance
p = [
    [10, 3, 22],
    [5, 29, -5],
    [15, 16, -26],
    [22, 30, 1],
    [26, 1, 3],
    [9, 21, 5],
    [28, 12, 6],
    [20, 2, 6],
    [14, 23, 4],
    [6, 7, 3],
    [11, 18, 10],
    [4, 19, -2],
    [25, 31, -7],
    [8, 17, -13],
    [32, 24, -6],
    [27, 13, -3],
    [19, 22, 3],
    [1, 11, 4],
    [29, 28, 26],
    [30, 20, -2],
    [13, 31, 6],
    [21, 10, -18],
    [2, 26, 7],
    [6, 18, 1],
    [16, 9, 1],
    [14, 17, -4],
    [3, 8, 8],
    [23, 15, 10],
    [24, 27, -3],
    [4, 5, 1],
    [12, 32, 18],
    [7, 25, 10],
    [24, 16, -10],
    [5, 21, 38],
    [22, 4, 7],
    [25, 6, -17],
    [29, 15, 28],
    [18, 8, -4],
    [17, 2, 4],
    [7, 12, 4],
    [32, 11, -7],
    [28, 14, -20],
    [3, 13, 21],
    [19, 30, 20],
    [20, 1, 24],
    [31, 27, 3],
    [9, 26, 24],
    [10, 23, 16],
    [26, 28, -24],
    [2, 19, -7],
    [16, 21, 24],
    [4, 3, 3],
    [13, 29, -3],
    [18, 25, 7],
    [23, 32, -10],
    [30, 1, -3],
    [8, 7, 11],
    [27, 9, 9],
    [10, 24, 32],
    [11, 6, 8],
    [15, 14, -34],
    [31, 22, 25],
    [20, 17, 21],
    [8, 4, 13],
    [1, 5, 16],
    [12, 11, 13],
    [14, 29, 6],
    [23, 27, 10],
    [9, 10, -3],
    [28, 13, 31],
    [6, 20, -8],
    [21, 24, -15],
    [17, 3, -3],
    [31, 16, -9],
    [26, 15, 14],
    [7, 19, 7],
    [2, 22, -2],
    [6, 21, 6],
    [3, 12, -2],
    [18, 5, -25],
    [28, 1, 12],
    [19, 20, 3],
    [22, 25, -13],
    [8, 11, -14],
    [16, 23, 17],
    [13, 26, -25],
    [29, 31, 7],
    [10, 15, 16],
    [4, 7, -3],
    [30, 24, -11],
    [9, 32, 15],
    [27, 14, 10],
    [1, 29, -12],
    [5, 26, 15],
    [2, 30, 8],
    [25, 3, 3],
    [22, 19, 3],
    [11, 7, -3],
    [12, 8, 18],
    [14, 10, 6],
    [32, 6, 4],
    [15, 27, -18],
    [17, 4, -2],
    [31, 28, -14],
    [16, 13, 1],
    [24, 9, -14],
    [21, 18, 16],
    [30, 5, -18],
    [18, 12, -13],
    [23, 25, 3],
    [1, 2, 14],
    [19, 17, 10],
    [11, 9, 1],
    [20, 4, 18],
    [15, 28, -32],
    [10, 32, 24],
    [7, 22, 40],
    [24, 21, -8],
    [16, 8, 6],
    [26, 29, -5],
    [17, 7, 2],
    [26, 31, -7],
    [5, 2, 24],
    [13, 14, -3],
    [32, 27, 6],
    [22, 20, 6],
    [8, 3, 6],
    [19, 25, 24],
    [9, 18, 4],
    [4, 16, -10],
    [23, 24, -29],
    [29, 30, 3],
    [12, 6, -7],
    [18, 32, 7],
    [14, 26, -30],
    [21, 23, 4],
    [1, 13, 3],
    [28, 5, -1],
    [27, 10, -8],
    [6, 11, -2],
    [31, 15, -2],
    [2, 29, -23],
    [12, 24, -14],
    [25, 4, 13],
    [3, 7, 3],
    [20, 9, 32],
    [30, 17, 3],
    [31, 14, -3],
    [29, 18, 21],
    [6, 3, 3],
    [21, 12, 14],
    [13, 23, -5],
    [30, 2, 13],
    [15, 1, -13],
    [25, 11, 10],
    [20, 28, 3],
    [17, 27, 4],
    [10, 16, 10],
    [24, 32, 8],
    [4, 22, 23],
    [7, 8, 21],
    [5, 19, 4],
    [2, 20, -4],
    [13, 15, -7],
    [11, 30, -3],
    [16, 27, -3],
    [12, 18, 0],
    [21, 9, -3],
    [8, 25, -16],
    [26, 6, 21],
    [19, 10, 3],
    [1, 14, 29],
    [23, 31, -4],
    [17, 5, -4],
    [3, 22, 16],
    [32, 28, -21],
    [3, 25, 2],
    [11, 12, 30],
    [9, 23, 7],
    [4, 2, -3],
    [27, 7, -7],
    [24, 1, 3],
    [18, 6, 3],
    [32, 21, -7],
    [14, 31, 8],
    [22, 17, -20],
    [28, 26, 10],
    [8, 15, -4],
    [5, 30, 21],
    [16, 10, -7],
    [13, 19, -3],
    [29, 20, 27],
    [15, 13, 7],
    [25, 17, -6],
    [7, 14, 14],
    [1, 26, 20],
    [27, 21, 23],
    [28, 29, 2],
    [10, 31, 23],
    [3, 18, 3],
    [30, 4, 21],
    [20, 5, 18],
    [32, 16, -35],
    [12, 2, 1],
    [19, 8, 1],
    [22, 23, 10],
    [24, 11, 14],
    [6, 9, 17],
    [10, 27, -7],
    [8, 6, -7],
    [31, 1, -3],
    [17, 19, 4],
    [18, 24, 18],
    [5, 22, 10],
    [15, 4, -7],
    [30, 28, -19],
    [14, 13, 22],
    [25, 7, 10],
    [21, 29, -23],
    [23, 16, -25],
    [26, 20, 11],
    [9, 12, -1],
    [2, 32, 1],
    [11, 3, -2],
    [16, 14, -16],
    [11, 21, -3],
    [13, 10, -24],
    [26, 30, 10],
    [7, 18, 28],
    [29, 1, -7],
    [22, 8, 11],
    [4, 17, 19],
    [15, 31, -4],
    [32, 9, -1],
    [5, 20, 4],
    [27, 23, 13],
    [3, 19, -34],
    [12, 25, -7],
    [24, 6, 43],
    [28, 2, 10],
    [29, 26, 18],
    [1, 28, -3],
    [9, 24, -2],
    [6, 12, -5],
    [31, 13, 6],
    [27, 16, 3],
    [20, 30, 25],
    [21, 32, 14],
    [14, 15, 20],
    [7, 3, 17],
    [18, 11, 1],
    [19, 4, 14],
    [2, 5, -1],
    [25, 8, 13],
    [23, 10, -20],
    [17, 22, -13],
]

Compute the index set to facilitate setting up the model#

# Computing the index set (ijt)

# Valid set of tuples
A = []
for t in game:
    # record team pairs and match sequence
    i = p[t - 1][0]
    j = p[t - 1][1]
    tp = i, j, t
    A.append(tp)

# print(np.matrix(A))

Setup decisions, objective, and constraints#

%%ampl_eval

# Parameters
param num_teams;
param num_games;

# Sets
set teams := 1..num_teams;
set games := 1..num_games;
# data for point spread is a subset of teams x teams x games
set games_data within {teams, teams, games};

# Point spread between team i and team j in game t
param p{games_data};

# Build decision variables: team ratings and home team advantage
var x{teams} >= 0;
var y >= 0;

%%ampl_eval
# Objective function: Minimize SSE
minimize sse:
    sum{(i,j,t) in games_data} (x[i] + y -x[j] - p[i,j,t])^2;

%%ampl_eval
#Constraints
# Fix average rating to be at 85
s.t. fix_average_rating:
    sum{i in teams} x[i] = 85*num_teams;

# Load data into ampl
ampl.param["num_teams"] = 32
ampl.param["num_games"] = 256
ampl.set["games_data"] = A
ampl.param["p"] = {(i, j, t): p[t - 1][2] for (i, j, t) in A}

Solve the model#

# Run optimization engine
ampl.option["solver"] = "gurobi"
ampl.solve()

Gurobi 10.0.1: Gurobi 10.0.1: optimal solution; objective 30972.98811
0 simplex iterations
6 barrier iterations

Examine outputs - The minimum SSE#

# check the SSE
print(
    "The minimum sum of squared errors are ",
    round(ampl.get_objective("sse").value(), 2),
)

The minimum sum of squared errors are  30972.99

Check the optimal team ratings#

# print optimal ratings by team

print(
    "\033[1m Home team advantage is \033[0m (", round(ampl.var["y"].value(), 2), ") \n"
)

print("\033[1m Optimal team ratings")
print("------------------------------------------\n")
# loop through all destinations

x = ampl.var["x"]

average_rating = 0
team_count = 0
for i in team:
    print(
        "\033[1m",
        i,
        " ",
        team_label[i - 1],
        "\033[0m:",
        round(x[i].value(), 2),
        "\n",
        end="",
    )
    average_rating += x[i].value()
    team_count += 1

print("------------------------------------------")
print(
    "\033[1m Average team ratings: \033[0m", round(average_rating / team_count, 2), ""
)

 Home team advantage is  ( 3.11 ) 

 Optimal team ratings
------------------------------------------

 1   Arizona Cardinals : 91.45 
 2   Atlanta Falcons : 82.24 
 3   Baltimore Ravens : 81.47 
 4   Buffalo Bills : 81.79 
 5   Carolina Panthers : 94.2 
 6   Chicago Bears : 80.87 
 7   Cincinnati Bengals : 90.35 
 8   Cleveland Browns : 77.31 
 9   Dallas Cowboys : 84.31 
 10   Denver Broncos : 96.37 
 11   Detroit Lions : 83.36 
 12   Green Bay Packers : 81.89 
 13   Houston Texans : 77.42 
 14   Indianapolis Colts : 89.04 
 15   Jacksonville Jaguars : 73.9 
 16   Kansas City Chiefs : 91.08 
 17   Miami Dolphins : 84.16 
 18   Minnesota Vikings : 78.38 
 19   New England Patriots : 90.89 
 20   New Orleans Saints : 93.77 
 21   New York Giants : 79.6 
 22   New York Jets : 78.91 
 23   Oakland Raiders : 77.01 
 24   Philadelphia Eagles : 86.85 
 25   Pittsburgh Steelers : 83.05 
 26   St. Louis Rams : 87.22 
 27   San Diego Chargers : 87.66 
 28   San Francisco 49ers : 95.13 
 29   Seattle Seahawks : 98.04 
 30   Tampa Bay Buccaneers : 82.33 
 31   Tennessee Titans : 84.23 
 32   Washington Redskins : 75.72 
------------------------------------------
 Average team ratings:  85.0

Conclusion#

The NFL team rating problem shows that an nonlinear optimization model can be used to predict game match performances. The above example illustrates that teams can be rated in such a way to minimize the prediction errors for game outcomes.

A key take away of the above example is that the team ratings should be controlled at a standard level (85 in this case). The reason is that without such a standard level, there exists an infinite set of optimal solutions that achieve the same RMSE. The standard level, however, can be set at any number.

One potential issue with the above team rating optimization approach is that the ratings can over fit historical performance. An exponential weighted performance approach can partially overcome this issue but putting more weight on more recent matches and less weight on distant matchings. However, given that the regular season is not very long, such an issue is unlikely to be a significant concern.

References#

[1] AMPL python reference. https://amplpy.readthedocs.io/en/latest/reference.html

[2] This notebook is developed by Yimin Wang (yimin_wang@asu.edu). Marcos Dominguez (marcos@ampl.com) also contributed to this notebook content.