Can we predict cricket match score in Excel? Using machine learning, ensemble modeling, multiple regression and Excel formulas we can. This tutorial explains how.
Cricket world cup is on. Both my homes (India & New Zealand) have done well so far in the tournament and if things go OK in the last couple of matches, they should qualify for semi-finals. The games are happening in UK, which is 12 hours behind New Zealand. You know that means?
Yes, lots of sleepless nights in Wellington.
As I watch these games, I notice that every once in a while they show a “score predictor“. It will tell you what the final score could be based on the proceedings of the game so far.
So I thought, hmm, May be I should make one of those in Excel?
That is what I did. I created a machine learning model in Excel to predict cricket score. Sounds interesting? Read on.
Cricket?!? What now?
If you are thinking “the only cricket that keeps me up all night is the damned chirping one in my basement”, then don’t worry. You need very little knowledge of cricket to understand the techniques. Once you know the ideas you can apply them to many other real life problems like predicting sales next year or student absences next term or electricity usage in the new plant.
Defining the problem – Cricket Score Prediction
Just a quick note if you are not familiar with cricket. In a typical one-day match, two sides compete. The game starts with one of teams batting first and scoring some runs in 50 overs. The next team then tries to beat that target set by first team in 50 overs.
Let’s start by defining the problem. We want to create a cricket score predictor that takes the inputs:
- Country playing – C
- Runs scored so far in the match – Rs
- Wickets remaining in the match – WR
- Overs remaining – OR
Our predictor should tell us what could be the final score at end of 50 overs.
For example, we could ask, “Australia scored 52 runs in 10 overs losing 1 wicket. What would be the final score?” and our predictor can provide a guess – say 305 runs.
Things we ignore:
The final score of a team in a cricket match depends on many things, including:
- The playing team and their batsmen
- The opposition team and their bowlers, fielders
- The ground and pitch they play on
- The weather and historical weather
- Crowd attending the game and how much they are cheering
- What the umpires had for lunch
- etc.
If we try to incorporate every little thing that matters, we will never be able to construct our prediction model. So let’s ignore everything except those 4 parameters (C, Rs, Wr, Or) listed above.
Our model as an equation
Let’s say Rp denotes our predicted Runs. We can define Rp as
Rp = p(C, Rs, Wr, Or)
where
- C = Country
- Rs = Runs already scored
- Wr = Wickets Remaining
- Or = Overs Remaining
- p is a prediction function that does some magic to calculate Rp
But we know that Rp = Rs + something
This is because total runs at the end of 50 overs will be something more than Runs Scored at the time of prediction.
If we can find something our problem is solved. But how?
Introducing Run Rate
Run rate is ratio between runs scored and overs completed. So if India scores 342 runs in 50 overs, their run rate is 6.84. As our prediction model is for 50 overs, if we know the Run Rate, we will know final score.
Let’s define few more variables.
- Os (Overs so far) = 50 – Or
- RRs (Run Rate so far) = Rs / (50-Or)
- RRr (Run Rate for remaining overs) – this should be predicted
Given these variables, we can rewrite Rp (Runs predicted) as
- Rp (Runs Predicted) = Rs + Or * RRr
So if we can build a model to predict RRr, we can calculate Predicted score.
We can argue that Run Rate in remaining overs will be a function of (country, run rate so far, overs remaining, wickets remaining)
RRr = f(C, RRs, Wr, Or)
Creating country specific prediction functions
We can further argue that each country has specific strengths and abilities when it comes to batting. So, if we define a set of functions, f1(), f2()…, fn() where fn is
RRr for country n = fn(RRs, Wr, Or)
We can then call the relevant function based on which country we are predicting the score for.
So what is this magical prediction function?
Regression of course. There are many sophisticated machine learning algorithms. But for something straight forward like Run Rate (remaining overs), we can create a simple multiple regression model.
Say RRr can be written as
RRr = m1*RRs + m2*Wr + m3*Or + const
Given a set of training data with RRs, Wr, Or and RRr, we can use LINEST() function in Excel to calculate {m1, m2, m3, const} that fits the sample data. Once we have the multipliers and constant value for each country, we can predict the score for any situation. Its that simple.
Why just one equation per country? Why not more?
As with everything else in life, cricket matches too have significant variability. For that reason, rather than one regression model per country, why not create 10 of them per country and the average the prediction?
As management consultants say,
“When you are not sure what to say, just run a survey and tell them what they said.”
If this sounds like a bunch of bs, don’t worry. This is an actual machine learning technique known as Ensemble Modeling.
Ensemble Modeling
The idea of ensemble modeling is simple. We build multiple models from the training data and then combine the results of all models when making predictions.
See this picture to understand how a typical Ensemble Model works.
This way, we can create more variation in input scenarios and create a robust model.
For the sake of simplicity, let’s say we want to build 10 regression models for each country.
How to aggregate the ensemble model results?
We will end up with 10 predicted Run Rates (Remaining overs). We can simply average these 10 to come up with final prediction. You can also assign weights to the models and do a weighted average. Let’s stick with simple average.
Let’s get building then.
Machine Learning 101
Learn first, Predict next.
That’s it. But if you want more text, here we go. Almost all machine learning models follow this pattern. They look at some data to construct the model. Once that model is ready, we then test it on a different data set to see how satisfactorily it performs. If the results are not up to scratch, we back to step one and fine tune the model.
The easiest of all these is a regression model. That is the same one we will be using too.
Where is the data?
There are many fine websites for finding current score or recent match results. But in order to train our model, we need a collection of historical match data by each over. This is notoriously hard to get. Thankfully, there is cricksheet. They have historical one day match data at ball by ball level for 1,400 + matches in CSV format. (here is the downloads page)
Note about data: I noticed that cricsheet doesn’t have all matches data. For example I could not find any 2018 games in the data set I downloaded few days ago. It doesn’t really matter as we are using a large sample of data spanning several years.
As you can see, this data is at a too detailed level than what we need.
So I used Power Query to combine 1,400 files, reshape the data to over by over scores and then calculate total score, overs and wickets at every 5 over interval until end of the game. I then took only the recent 300 games as very old performances have little impact on current scoring patterns.
Sorry for not explaining the Power Query or Excel formula steps. That would get too technical and this post will never end.
Once reshaped, my data looks like this:
We can then derive additional columns,
- Last over of the game: =MAXIFS([Over], [Source.Name],[@[Source.Name]], [Country], [@Country])
- RRs – Run Rate so far: =[@[Cum Runs]]/[@Over]
- Or – Overs Remaining: =[@[Max Over?]]-[@Over]
- Wr – Wickets Remaining: =10-[@[Cum Wickets]]
- Score at end: =SUMIFS formula
- RRr – Run Rate (remaining overs)
=([@[Score at end]]-[@[Cum Runs]]) / [@[Overs Remaining]]
Using 2,3,4 & 6 we can create our regression models.
But before we go there, let’s split the data in to training & test data sets. For this example, I choose the latest 50 games as test data set and everything older as training data. Instead of creating two separate tables, I just added a column at the end to look at [Match ID] to tell me whether something is test or training.
Also, we do not need to use last over data for training. At the end of game there is nothing left to predict, so there is no point of using last over data when training the model.
I have added a data point ID as column to this table so I can uniquely identify all data points when sampling training data. It is [Data point number]
Bagging & Bootstrapping
Don’t freak out. We are still on topic. Bagging is the technical term for the concept of randomly sampling data, building models and then aggregating (ie bagging) at the end.
Bootstrapping refers to random sampling of data.
Our bootstrap approach is rather simple and naive.
- For each country, we want 10 bags – 10 data sets
- For each data set, we want a random sample of 50 data points
- We then create a multiple regression model to fit
RRr = m1*RRs + m2*Wr + m3*Or + const - We average all 10 model results when predicting outcomes.
A note on const: When I was building my cricket score prediction models, I realized that setting const=0 gave me a better R2 (ie the model fits well with training data). So I set the 3rd parameter of LINEST() to FALSE (ie no need for const). You may want to keep it on for other types of models.
In my training data from cricsheet, we have 16 countries. That means we need 16*10*50 = 8,000 data points to construct the models.
Using a bunch of RANDBETWEEN, INDEX+MATCH and COUNTIFS, I was able to construct this grid.
Constructing Multiple Regression Models
Once data grid is ready, we can create a bunch of LINEST() formulas to tell us the multipliers (m1, m2, m3) for each model. This can be done 160 times (each of the 16 countries need 10 models). But I am very lazy. So I used INDEX() formula to fetch arrays of 50 cells so that LINEST results can be tabulated nicely. This is how our multiple regression model looks:
As you can see, our model has very high R2 values. This is promising.
Mind the F
While high R2 values are good, you should not trust the model blindly. You should also check if the relationship between output (Run Rate in remaining overs) and inputs (RRs, Or, Wr) is chance. This can be done by looking at F statistic and F distribution probability. I have not bothered with this step for all of the data, but I did check for few samples to see if the F probability is low (low means relationship is not random).
Learn more about F statistic and how to interpret the results.
Calculating final prediction
As you can see, each model predicts Run Rate (in remaining overs). But we need to predict the score. Given the inputs:
- C = Country
- Rs = Runs already scored
- Wr = Wickets lost so far
- Or = Overs Remaining
We can calculate predicted Runs (Rp) as
- RRr = average of all 10 country specific predictions (RRs, Wr, Or)
- Rp = Rs + RRr * Or
In simple words, our final prediction is Runs already scored + average of 10 predicted run rates times remaining overs.
Testing our ensemble model against some of the recent matches
Now that we have our shiny ensemble models, let’s go test them. I have extracted score data from last 50 games by innings. I then filtered away any games with less than 50 overs played (canceled due to rain, chased before the last ball etc.)
This is what we have.
For prediction, we also need to know what were the runs scored and how many wickets they had in hand at certain over. I started by creating a scrollbar to select the over (any multiple of 5 between 5 and 40). Then we fetch the relevant inputs from test data and run the model against them to calculate predicted score. I then compared this against actual score to see what kind of error and accuracy our model is getting.
This involved using some crazy, but fun MMULT and INDEX functions (ofcourse, TRANSPOSE too). It is 2:19 AM as I am typing this. That means, Unfortunately, it is too late in the game to explain the formula logic here, so I will leave it to your imagination.
Here is how our model compares with actual results at 15 overs.
And this is how it works after 25, 35 and 40 overs. As you can see, accuracy improves the later in game you ask for prediction.
Predicting India’s score against Bangladesh – 2nd July, 2019
Right now, as I am typing this, India is playing against Bangladesh. India have score 314 in 50 overs. I wanted to see how our model predicts the score at various points in game. As you can see, it gets a little optimistic (as India didn’t loose a wicket until 30th over) but the prediction gets closer since 35th over.
Download cricket score prediction model & play with it
If you want to examine the calculations, predict your own scores or just want to see how its all done, here is the file.
In the download:
- You will find two models, not one. This is because I built two regression models to see which will give better prediction. The one presented in this article gives better results.
- You will find a simple score predictor too. Enter inputs (Country, runs so far, wickets so far and overs) and it will tell you what the predicted score is.
- All calculations and data.
Feel free to mash up the data to create your own prediction tool.
How I built the score predictor – video
I made a short (oh well, 37 minutes long) video explaining the process, machine learning concepts and Excel implementation. Watch it below or see it on Chandoo.org youtube channel.
References – on Machine learning, Excel and statistics
This is an interesting topic and I am sure you want to know more. See below references to understand the concepts better.
- Ensemble modeling:
- Ensemble modeling & learning
- Ensemble modeling basics, terminology and sample Python code
- Understanding bootstarpping technique
- LINEST() & Multiple Regression:
- LINEST Function – Microsoft Help
- LINEST example and explanation from MIT
- LINEST and other forecasting functions in Excel
- F statistic:
- Understanding F statistic
- Excel features:
- Power Query for combining files
- INDEX function
- Also see:
- Time series forecasting in Excel
How do you like the score predictor?
I had so much fun creating this. I did have a few false starts and made models with wrong equations, but eventually came up with something that provides sensible prediction. I am happy with the way it turned out. Although I couldn’t explain every little thing about the model in this post, I hope you are able to fill those gaps in.
Do you like this prediction model in Excel? Are you surprised to see a complex machine learning algorithm implemented in good ol’ spreadsheet? Share your thoughts in the comments.
55 Responses to “Quickly Fill Blank Cells in a Table [Reader Tip]”
this can be done in 3 steps:
1. select the blank cells (as described above)
2. select the cell with the value you want to copy (CTRL-CLICK to add to the selection)
3. place cursor into formula bar and hit CTRL-ENTER
please ignore or delete my comment - it solves a different problem: copying a single value to all blank cells. apologies.
That is a great method and it saves me a lot of time! I first heard about it from Mr Excel in this video - http://www.youtube.com/watch?v=jHmh_viESuw. He has a neat way of doing the paste special values at the end of his video.
Hi!
I fill blank cells with an almost identical method; go to any the first blank cell in any column and place the equation and enter (=D2, for the same example above); then copy that cell, select the columns/range you want to fill (even if in different columns), Special, go to Blanks, Paste (default), copy all range and paste as values.
although the two methods are almost identical, what i use might be less hectic regarding entering formulas without clicking any of the cells (step 7)
ie:
1. fill an empty cell with using =D2(cell above)
2. copy D3 (the cell with the formula)
2. Select blank cells after selecting the range with empty cells (steps 1,2,3,4 and 5)
3. paste (normal)
4. copy then paste as values
BR
AQ
Great tip. I'll use it later today!
Martin -Thank You! This wonderful tip will save me a great deal of time each week.
Thanks Martin! Up to this point, I've always used a clumsy combination of filters and fill-down's. This is much cleaner.
Fantastic. Thanks for sharing.
None of these steps are necessary, Excel has this feature built into the ribbon.
Click on any row label in the table where there are blanks under it.
Click on the PivotTable Tools>Design tab on the ribbon
Click the Report Layout button in the Layout group at the far left
Select the option in the list
Done
To remove the duplication, use the feature right below that option.
There is a slightly simpler way and more flexible. Hihglight the required cells - which could be the column only in your table. Do the Ctrl-G, Alt-S, K, Enter (or Goto, Special, Blank Cells) so that they are highlighted and Type ={up arrow}, Ctl-Enter. This will make the cells equal the cell above - you do not have to enter any address at all. The technique can obviously be adapted to many situations. An example of the practical use for this is when you have saved an Inventory report from an accounting program that prints a heading (or something) on one line and prints details of that group (the heading) on subsequent lines (without the heading).
Hi Martin,
great trick! If only I had known it earlier, it would have saved me quite some time...
Not again, thanks!
I came across this in a class recently myself and posted a tutorial on my blog. The Special area of the Go To dialogue box is wicked. Some great options in there, hidden away waiting to be found.
Good work Martin.
Hi Martin,
Many thanks for sharing this powerful trick. Saves alot of time.
Gabriel
Please give credit where credit is due. Posted on June 30, 1998: http://www.mvps.org/dmcritchie/excel/fillempt.htm
Ahhh... Very neat trick. Thank you, Martin.
Ken, I tried to follow your post but could not get it to work. Could not find options
I have been using this trick for ages and would be lost without it.
Thank you very much!!! I had other tricks to deal with it, but this one is way faster and easier!!!
@BigG: Good resource there. Thanks for sharing the link with us. Please note that, this technique is not new. I am sure many Excel users would have discovered this already. We have not copied or inspired from David's article. It was just a happy coincidence.
@Ken: Your technique works only with Pivot Tables made in Excel 2010 or above.
Thanks Martin!! Nice post 🙂
@Chandoo: I also use the ASAP utilities add- in available in the link below:
http://www.asap-utilities.com/download-asap-utilities.php
This summarizes lot of hidden features in excel (like using Find function on entire workbook, password protecting all sheets at once, copying print setting of sheets etc.,) and is quite useful for beginners like me 😉
Thanks Martin and Ahmad Qadah. This is useful. I previously used to ask the senders to retrieve the data again so that I did not have the blanks.
Nice trick. I always use the specialcells method of the range object in code to access this powerful goto special dialog box in vba - a trick that Chandoo taught me in vba school - which is another reason you should join (a free bit of promotion for you Chandoo..!)
🙂
Yes I have seen this one before so credit may belong elsewhere. Never the less still especially useful where a legacy system report is sent to a text file which is subsequently re-imported to Excel but the original report is indented by groups. You can then recreate a complete data record for each report line
NB Different Ken to above
Thanks Martin - great post. I often work with data in this form and I usually fill in the blanks manually, by copying and dragging a cell value down - this way is much less prone to human error!
One challenge.. the last step where I change formula to constants. This replaces any formulas that I have as well. What If I want to change the formula to constants only where I replaced them with blank ?
Hi martin, thanks a million 🙂
Nicely explained Martin, thanks for sharing this tip. As Tanja says, this method is far less error-prone. When I first learned this method it saved me lots of time, so I decided to create a video on Youtube to share it with others. In my 3 minute video I compare side-by-side two methods of filling in blanks on 500 rows of data (1) using the fill handle, (2) using Go To > Special > Select Blanks
Just like in Mr Excel's video shared by Andrew in comment (3), I used the right mouse button to drag the selection border to do paste special values at the last step.
If you want to check out my video, visit this link: http://www.youtube.com/watch?v=9TDcVOKbm34&hd=1
I've came across this a month ago, and it really is a gem of a tip!
Thanks. Great tip and useful for a range of excel projects 🙂
Vishy,
When you Ctrl Enter the formula into all blank cells, Excel keeps the formerly blank cells highlighted, revealing the new values.
At this point you can choose to Copy and Paste Special them as constants. All other formulas remain untouched.
BigG,
I was not familiar with that link and I certainly didn't copy the article from it. As Chandoo commented this is not a new technique, and I am hardly the first to have written about it.
@Martin,
using office 2007; you can not copy multiple selection, what version are you using?
Thanks
Thanks, Really nice, really helpful.
wow, how cool is that! Thank you for this tipp!! GREAT!
I thought this was a great tip. I had never done such things with tables in Excel (having only converted to 2007 a couple of months ago, I soon discovered what a versatile tool they can be). So I decided to create my own copy and duplicate the process. Taking it a step further, I recorded the steps in VBA and used those as a guideline to create this simple macro which accomplishes the same function.
Caveat: this will only work when a cell in the table is selected and it will replace ALL formulas in the table with their values.
Sub FillTableBlanks()
' Macro created 20 October 2011 by Jason B White
'Declare Variable
Dim strTable As String
'Get Current Table Name
strTable = ActiveCell.ListObject.Name
'Select Current Table
Range(strTable).Select
'Fill Blank Cells With Formulas
Selection.SpecialCells(xlCellTypeBlanks).FormulaR1C1 = "=R[-1]C"
'Paste Values Of Formulas
Selection = Selection.Value
End Sub
I hope that submitting macros is sanctioned in this forum. My previous post was my first ever attempt at contributing to an Excel blog. And I'm unaware if there is a way to differentiate macro snippets by using tags as I've seen in other Excel VBA forums.
I just wanted to mention that I figured out a way to modify my macro so that it doesn't overwrite ALL formulas in the table, but only those which were filled in by the macro.
Modifying the fourth section (Fill Blank Cells With Formulas) as shown below accomplishes that:
'Fill Blank Cells With Formulas
Selection.SpecialCells(xlCellTypeBlanks).Select
Selection.FormulaR1C1 = "=R[-1]C"
Hi,
I face a similar situation in office and use the below macro after selecting the range of data across which I want to duplicate the data below.
Sub FillBlankCellsSelectionDown()
Dim rAcells As Range, rLoopCells As Range
Set rAcells = Selection
For Each rLoopCells In rAcells
If rLoopCells.Value = "" Then
rLoopCells.FillDown
End If
Next rLoopCells
End Sub
re: paste special -> values
Drag the Paste Values toolbutton on to the standard toolbar next to the Paste button and save a couple of clicks.
Hi everyone many thanks for sharing this solutions but do not work Excel 2003? right? Thanks
@Alejandra:
I know that the macro I created was in Excel 2007. I assume that it's probably specific to 2007 (or 2010), but can't be sure, as I no longer have access to a PC running Excel 2003.
I have to admit that I didn't even realize that tables existed when I was using 2003.
Filling blank cells (cleaning-up the pivot-table aftermath) is one of our "daily-ritual", to dealing with those, we've create a short-cut (one of the many) to very quickly fill-up those blanks.
Basically what we need to do is to select the whole area to be filled-up (with the value above), and click a button, VBA automatically deals with the rest.
We use VBA to handle this problem just as mentioned above by several other people, however, I think we'll also need to consider the extreme (well, actually not that extreme if you're dealing with lots of data on a day-to-day basis) case: that the "blank" cells are highly fragmented, e.g. the maximum "areas" that Excel 2003 can handle is around 6500 (sorry I couldn't find the exact spec).
Thus, in our function, there's another step to cut-off the number of cells going into the "specialcells" function, just to make sure that the function will run in every condition.
I just wanna give a solution to similar problem which i face regularly while copying the data from a pivot as it is. I apply the following solution which i think is the easiest one on earth. Select a cell F2 (considering that column E is the last column filled with data) and type the following formula =IF(ISBLANK(A2),F1,A2). Now just drag the formula equivalent to the length and breadth of the entire range of data which want to fill in this case drag it from F2:I21 , remember do not apply on the cost column.
Now just copy whole new range i.e: F2:I21 and paste special it over the former range A2:D21. That's it 🙂
If u find any problem related to this formula u r welcome to contact me.
thanks martin
This doesn't work in excel 2007. So request to Martin , if he can confirm which version he has used. Guess 2010.
@BK
my method (comment #4) which is almost the same as Martins works on excel 2007... i've been using it since 2007 came out actually.
Excelent trick, thanks Martin.
[...] Quickly Fill Blank Cells in a Table [...]
eXCEEELLTOOOOOOOOOOOOOOOO......!
Many thanks to Martin.
im getting an error no cells were found why is this
Very cool trick!
I'm facing a similar problem, but I'd like to use a formula to pick the first non-empty above the referenced cell, and keep the empty cells empty. Any solution?
Example case:
I've got 3 columns, 1) consecutive dates, 2) my current weight, 3) my BMI. The first data row would be like: A2) jan-1, B2) 70 (kg), C2) =70/1,75^2 (because my height, 175cm, is pretty constant)
Now of course I forget to write down my weight on jan-2nd, so the formula would return 0. If my weight is blank, I'd like to refer to the last 'non-blank' weight (up the list of course, so jan-1st).
The solution on this page would solve my problem partially, but every time I leave cells blank, I have to repeat these steps. A formula would prevent this, AND I can still see which days were actually not filled in.
[…] http://chandoo.org/wp/2011/10/17/fill-blank-cells-in-a-table/ […]
Thxs! Yes, "knew" you could do this with "one" col of data...never thought to try it with >>multiple<< cols...Cool!
Thanks a lot i was searching this thing for many days ,
Thanks a lot to martin
Thanks a lot to martin
Thanks a lot i was searching this thing for many days ,
Thanks a lot to martin
The north, on the contrary, is the land of mighty and sometimes creepy-looking pinetrees, often compared to monsters from fairy tales.
Pages 1 through 3 of the tentative budget are also printed in portrait format so
the writing on those pages is also sideways.
There are occasionally long discussions of the cost of nuclear relative to the cost of renewables in the technical literature.