Over on Twitter, I came across this beautiful chart, aptly titled – Joyplot. It is the kind of chart that makes you all curious and awed. So I did what any Excel nerd would do. Recreated it in Excel of course. This post takes you thru the process.
Peak time for sports and leisure #dataviz. About time for a joyplot; might do a write-up on them. #rstats code at https://t.co/Q2AgW068Wa pic.twitter.com/SVT6pkB2hB
— Henrik Lindberg (@hnrklndbrg) July 8, 2017
First let me share the final outcome.
Joyplot in Excel – Peak time of the day for sports and leisure
Here is the final overlapped area chart with a bit of formatting thrown in. It is a pretty close imitation of Henrik’s original chart. Click on it to enlarge.
Creating Joyplot in Excel – Tutorial
As you can guess, the chart is a just an overlapped area chart (ie each area sits behind another, unlike stacked area chart where they are umm, well, stacked!)
Let’s start with a look at data. Henrik’s original data has 10,656 rows, each row containing activity name, time and p value – how much survey respondents enjoyed [@activity] at that time.
Here is a snapshot of first few rows.
Scrubbing and re-arranging the data
As you can see, while this format is excellent for storing, it is very tedious if we want to make one chart with all series. So let’s scrub.
- We need to figure out if an activity should be included or not. I am using the same criteria as Henrik’s. Exclude activities with p value less than 0.003 or activity title “Playing sports n.e.c. *” (not elsewhere classified)
- To do this, we first pivot the data on activity and max(p). Then filter this pivot two ways – max(p) >=0.003 and label not equal Playing sports n.e.c. *
Tip: You may need to enable multiple filters per field in the field settings of row labels. - We will end up with 28 activities.
- Then add a helper column to original table that looks up the pivot and tells if an activity should be included or not
- To do this, we first pivot the data on activity and max(p). Then filter this pivot two ways – max(p) >=0.003 and label not equal Playing sports n.e.c. *
- Add two more columns to original table to tell peak time and modified time. This will help us in rearranging and sorting the data. Modified time just moves time by 3 hours (Henrik’s chart is plotted from 3AM to 3AM). At this stage our data looks like this:
- Now, pivot the data once again. This time,
- exclude activities by using report filter on include? column.
- Set up peak and activity in row labels area, modified time in column labels area and p in values area.
- Arrange the report in tabular format, turn off sub-totals.
- We get this:
- Calculate normalized values by dividing each p value with maximum p value for that activity. We can use another range of 28×288 cells to do this. We get this:
- The next 2 steps may seem confusing. It will become clear once you look at the charts.
- Define an offset value. Start with 0.5. You can change this later. In a separate 28×288 cell range, calculate gaps by multiplying offset with position of an activity. Something like this:
- Now, finally calculate activity + gap values by adding up respective cells in each of the 28×288 ranges. We get this:
At this stage, our data is a shape ready for visualizing.
Creating and formatting overlapped area chart
The chart creation process has 5 steps.
- Select the 28×288 range of cells created in step 7 and insert an overlapped area chart.
- Now, copy the gaps range (created in step 6 above) and paste them on to area chart as new series (just ctrl+c your data and select the chart, press ctrl+v)
- Adjust the order of series so that each activity is sandwiched by appropriately named gap series
- Tip: adjusting 56 series is painful with the chart select data > move series up/down buttons. Instead, just select the series, look at formula bar. The SERIES formula has last parameter as order. Change this number. It is easy to figure out the number once you try a few.
- Change all gap series fill color to white. This instantly creates the floating area chart effect.
- Change the colors of activity series. Apply white / off-white border to these series. Your joyplot is ready.
Quick overview of the chart creation process:
Let’s examine the result of each those 5 steps with a smaller dataset so you can see how everything fits together. Here is the data for this example:
- Create an overlapped area chart with activity+gaps data. We get this:
- Add gaps as new series to chart. You get this:
- Move the gap series so that they sandwich activity series. Use Chart Data > Move series up/down buttons or SERIES formula
- Apply white color fill formatting for gap series. This creates floating area chart effect as below:
- Finally, format the chart by apply some colors and border formatting etc.
So there you go. The final outcome does look joyful.
Alternatives to Joyplot
While joyplot is awesome, it is not easy to make. Fortunately, there are a few simpler alternatives that we can whip up in Excel as soon as you have either the pivot or normalized values. Below I have shown two such examples. Read about sparklines or conditional formatting heatmaps for more.
Joyplot alternative – using sparklines:
Tip: to get axis on your sparkline, just type the times separated by a single space. Then go to format cell (ctrl+1) and set horizontal alignment to distributed. Viola, Excel will fill the cell by adjusting spaces.
Joyplot alternative – Conditional Formatting Heatmap
Download Joyplot Workbook
Click here to download Joyplot Excel workbook. Examine the data scrubbing formulas, pivot and chart settings to learn how this is created.
If you are familiar with R, then go thru Henrik’s R code. It is much shorter than the Excel gymnastics we did with circular pivot table referencing. That said, some of the data re-arrangement could be done with same ease in Power Query too.
Your thoughts on Joyplot?
The only step we missed in Excel implementation is moving average smoothing of the area charts. It can be easily added as a step between 3 and 4 in data stage.
How do you like Joyplot? Would you create something like this for your business / personal data? Share your stories and thoughts in the comments section.
More joy for you…
If you love this, you are going to enjoy these charts too.
28 Responses to “FIFA Worldcup 2018 Excel Tracker – FREE Download”
Good work as always - I liked the way you did the "menu" on the left hand side (although the buttons aren't lined up between tabs if I'm being ultra picky)
Have you previously written about the method of extracting the Wikipedia page into Power Query? It's not something I recall seeing before.
ps other geeky observsations:
- the bracket columns are too narrow for the date & match number - and will need to be wider still when the team names get populated
- match 51 should be Moscow (Luzhniki) for consistency
- it's not possible to be 23 hours ahead of GMT - the International Dateline gets in the way! I think the maximum is 14. There are also a couple of countries who work to a quarter hour to make it really complicated!
- There's a typo in the how-to - "compated" instead of compared
Thanks for the lovely feedback. I have fixed almost all of them.
1) button alignment: this is tricky as row heights can change between sheets.
2) Column width is fixed now so bracket view looks better
3) Updated the stadium name
4) Did not bother with the 23 hours ahead thingie. This is more of a novelty feature 😛
5) Fixed the type
6) Fixed an issue with live score table. This should work as long as the points table is maintained in wikipedia page - https://en.wikipedia.org/wiki/2018_FIFA_World_Cup
7) I have not discussed the technique of reading all tables on webpage to one big table. Watch out for a blog post on this soon.
Button alignment is one for the ultra-OCD sufferers 😉 There are ways, but only for those with too much time on their hands.
Aah, Excel. The perfect tool for people like us. Everything (cells) is in same shape and size by default and aligned perfectly. 😀
Is there actually a way to copy row heights (in the same way you copy column widths?)
By the way Chandoo, great post. I'm forwarding to my department. I actually use another query from the same page to automatically fill in the team names for the knock-out stages (I made one for round of 16 which I then duplicated and edited for quarter-finals etc.) This is incredible, I was always wondering how to do these type of queries from the web, and now I know 🙂
Hello!
This is quite amazing and incredibly cool to use 🙂
Testing the constraints of this sheets a few errors popped I noticed:
- Vlookup Group E-H refers to column J instead of E (eg. Brazil gets the same points as Russia because the formula looks up Russia twice)
- Power query only has 29 lines, the overview of has 32 but the 3 countries from group A are lost as the overview is refreshed - causing N/A in the group stage colums
@Jake.. thank you. I am sorry for the errors. I could not test the live points table until the games began. I see my folly now. I have fixed both issues and uploaded a new file. As the points table relies on a wikipedia page, if someone decides to change the layout or rename a column it can seriously harm this template. I took some precautions in the Power Query layer to adjust column names dynamically etc, but it is not foolproof.
Try downloading the newer version and let me know if you see something funny.
No worries!
Was able to fix the vlookup myself but the power query had me bit stumped 🙂 And wanted to give you a heads-up to everyone can enjoy it!
Thanks for the awesome sheet!
Hi,
Thanks for sharing this world cup tracker. Certainly makes it more interesting when the data is current. As a newbie, it also helps to have a couple of mistakes to find whether unintentional or not.
Thanks again
Hi,
Your v-lookups in the "Group Stage" tab for groups E, F, G, and H (all the ones under column O) are pointing to the wrong country. They all point to column J, so whatever happens to the countries in column J will also be reflected for the countries in the groups in column O for that same row.
Just thought I'd call that out. Thanks for the great work on this!
@Christian... Thanks for trying this and letting me know about lookups. I have fixed the issue now. Please download latest version for that and few more fixes.
Refresh All did not work correctly. Team names vanished though points were updated.
@Sheeloo... Can you please try with latest version (download again using above links). I tested up to latest Iran's stunning win over Morocco and it works.
Dear Chandoo
Thanks a lot for this worksheet.
However, while refreshing the data, I am getting error message as "Initialisation of Data Source failed".
May I know what version of Excel you are using? Do you have internet connectivity? If you are familiar with Power Query, try tracing the steps in the query editor. And oh, first start with the latest version of file (link above).
@Etienne - yes. Copy row, paste formats will do it, although obviously that will bring the formats of every cell in the row as well as the height.
Latest version seems to be working well.
One request: the Groups & Points tables on the Group Stage sheet have the team names pre-entered. This means they don't get sorted according to the results.
On my copy, I've changed them to a lookup, so they appear in the same order as the points table. It would be good if you can do the same if/when you release a new update!
Here's what I did. It's not the most elegant, but it works, and I didn't have much time to spend on it!
Using helper values of 1,2,3,4 in columns I and N for each group, the formula for the first team name in group A (cell J4) is:
=INDEX(points[Team],MATCH(OFFSET(J4,-(I4),0),points[Group],0)+(I4-1))
This can be copied & pasted to the other team name cells.
Cheers!
Good suggestion. I have made changes to the points table to remove lookups and just show teams in the order they appear in the detailed table. This way, You will see top two teams on first two rows. We could highlight them as well (figured this would make it look like a bowl of M&Ms, so didn't bother) or highlight *YOUR* team.
I consider my Excel skills as above average but far from guru and I love how your little projects like this get me to look at data in a new way. I would like to expand on the data in the points table through the use of some calculations but I am a little challenged by the data coming across as text. The Pts column is easy to deal with, but I'm having problems with the GD. The negative goal differential looks like it may be noted with an en dash instead of a minus sign, but if I search for an en dash in the data Excel doesn't find any. I would like to include conversion to a minus sign in my little macro so I can get everything to numbers but so far I am not having any luck. Any thoughts? Thanks for your help.
Thanks for such kind words 🙂
I suggest adding an extra step in Power Query to convert points, GD & other columns to numbers. You can replace em dash in PQ. I did not do it as this will add another layer of dependency and should the wikipedia page change, one more reason for the query to fail.
As always, an awesome spreadsheet from Chandoo. I love the Power Query score update without macros. The country watch-out is a unique feature as well!
For those who like a predictor template with flag lookup and a ribbon UI, here is our spreadsheet:
https://www.spreadsheet1.com/fifa-world-cup-2018-russia-free-prediction-templates-for-excel.html
Here is our World Cup 2022 template with LAMBDA functions:
https://www.spreadsheet1.com/fifa-world-cup-2022-qatar-free-prediction-templates-for-excel.html
[…] Interesting World Cup Tracker here at chandoo.org : https://chandoo.org/wp/fifa-worldcup-2018-tracker/ […]
Great template!
I came across another one with image vlookups for country flags
https://eexcel.co.uk/downloads (World_Cup_2018_Sweepstake.xlsx)
This is a great Template.
I am running Excel 2010 with the PowerQuery add-in running.
The scores will not update, so I followed the error and the second operation (Fitlered rows) says that the table is empty.
After a few minutes on Wikipedia, I realise that my PowerQuery skills are not good enough to work out what the issue is.
Any suggestions?
I would like to fix it myself is possible.
Thanks,
Sean.
@Sean... Can you try the latest version mate? I think it should work.
Where can I see the results for a specific match?
Thanks!
@Juan... You can now. I have included a results tab that shows match scores. This too is a live table. Just refresh data to get new results. Please download latest version file from links above to use this feature.
PS: There is another version coming soon with all goals too. I just have to spend some more time polishing the Json to table Power Query thingie.