Recently Microsoft announced Python support for Excel. This is a BIG news for everyone using Excel to analyze data or find insights. In this article, let me give you a proper introduction to the Python in Excel feature and how to use it.
If you prefer video, check out my Excel for Python is here video.
What is Python for Excel feature?
You can now write Python code natively in Excel cells and return the output as either Python objects or Excel values. For example, you want to perform quick statistical analysis of your sales data in the range A1:D10. You can use the below Python code to do this now.
=XL(“A1:D10”, headers=True).describe()
How do I enable Python for Excel?
This “preview” feature is only available with Excel 365 beta users as of September 2023.
If you have Excel 365, you can go to File > Account to enable “insider” program. More details on eligibility and instructions are here – https://insider.microsoft365.com/en-us/join/windows
After you’ve joined the program, update your Office from File > Account page.
After the update, if you have Python for Excel, it will show up in the formula ribbon, as depicted below.
If you don’t have it yet, just wait a few weeks. It will show up eventually.
How to use Python in Excel:
A Quick Tutorial
Open Excel and load any of your data files. Alternatively, if you need sample data, copy paste the below table into Excel.
| Sales Person | Product | Country | Date | Sales | Boxes |
|---|---|---|---|---|---|
| Gigi Bohling | Manuka Honey Choco | India | 20-Jul-23 | 8162 | 742 |
| Barr Faughny | White Choc | Canada | 16-Aug-23 | 2485 | 355 |
| Marney O’Breen | Peanut Butter Cubes | India | 14-Jul-23 | 10255 | 733 |
| Wilone O’Kielt | Mint Chip Choco | India | 2-Jul-23 | 16800 | 800 |
| Gunar Cockshoot | Orange Choco | New Zealand | 2-Aug-23 | 2842 | 203 |
| Andria Kimpton | Baker’s Choco Chips | Canada | 18-Jul-23 | 9373 | 427 |
| Beverie Moffet | Fruit & Nut Bars | India | 14-Jul-23 | 6573 | 598 |
| Mallorie Waber | Baker’s Choco Chips | India | 24-Jul-23 | 3598 | 150 |
| Barr Faughny | Spicy Special Slims | Canada | 11-Jul-23 | 5138 | 571 |
| Dennison Crosswaite | White Choc | Canada | 22-Jul-23 | 1547 | 258 |
| Ches Bonnell | 99% Dark & Pure | New Zealand | 16-Aug-23 | 12901 | 993 |
| Andria Kimpton | Organic Choco Syrup | USA | 16-Jul-23 | 7161 | 651 |
| Gunar Cockshoot | Fruit & Nut Bars | New Zealand | 19-Jul-23 | 11935 | 1492 |
| Beverie Moffet | After Nines | India | 18-Aug-23 | 5089 | 268 |
| Gunar Cockshoot | Peanut Butter Cubes | USA | 11-Jul-23 | 9247 | 578 |
| Andria Kimpton | Peanut Butter Cubes | India | 22-Jul-23 | 10731 | 826 |
| Gigi Bohling | After Nines | Australia | 4-Jul-23 | 9730 | 609 |
| Gunar Cockshoot | Eclairs | USA | 1-Aug-23 | 3150 | 287 |
| Karlen McCaffrey | 99% Dark & Pure | USA | 6-Aug-23 | 2247 | 205 |
| Roddy Speechley | Peanut Butter Cubes | USA | 1-Jul-23 | 2765 | 213 |
| Brien Boise | Caramel Stuffed Bars | India | 3-Aug-23 | 7112 | 647 |
| Wilone O’Kielt | Organic Choco Syrup | UK | 27-Aug-23 | 3787 | 345 |
| Dennison Crosswaite | Peanut Butter Cubes | Canada | 29-Aug-23 | 2674 | 168 |
| Gigi Bohling | White Choc | India | 14-Aug-23 | 378 | 54 |
| Karlen McCaffrey | Raspberry Choco | Australia | 7-Jul-23 | 7217 | 401 |
| Marney O’Breen | Spicy Special Slims | New Zealand | 19-Aug-23 | 735 | 147 |
| Mallorie Waber | Organic Choco Syrup | UK | 3-Jul-23 | 4690 | 427 |
| Karlen McCaffrey | Manuka Honey Choco | India | 24-Jul-23 | 8008 | 572 |
| Wilone O’Kielt | Spicy Special Slims | Australia | 18-Jul-23 | 12586 | 2518 |
- Once you have some data in Excel, press CTRL ALT SHIFT P to enable Python mode. If you get a “welcome to Python screen” complete the tour and then press the shortcut again.
- Using your mouse or keyboard, select the data in your workbook. Excel should write the necessary XL() command to capture your data into Python as a dataframe.
- To see the dataframe you just built, press CTRL Enter. Excel will display a “Python Object” in the cell.
DATAFRAME: a dataframe is a python concept for storing data. They are like Excel tables. Each column of dataframe has one kind of data.
To see the output as values
instead of Python object
You can see the “actual” values of your Python code anytime. Just select the cell with Python output and either press CTRL+ALT+SHIFT+M or right click on the cell and choose “Python Output” > Excel values option.
10 Python Coding Examples
Use these code samples to play with Python in Excel. Before starting.
- Copy the above table of sample data and paste it in Excel (in range A1:F30). Alternatively, download this file with the data.
- To type the code, enter python mode (CTRL ALT SHIFT P) or use the formula =PY( in a cell.
Example 0
Construct dataframe 👩💻
df = xl("A1:F30", headers=True)
Explanation & Output 💻
This will just create a dataframe named df and return that to the cell. You can either leave it or see the underlying data (which will be same as A1:F30) by changing the output style.
Example 1
Description of the data 👩💻
df.describe()
Explanation & Output 💻
This will generate a dataframe with statistical descriptions for all your number columns. Example output is shown below.
Example 2
Description of the data, all columns 👩💻
df.describe(include="all")
Explanation & Output 💻
This will generate a dataframe with statistical descriptions for all your columns. Perfect for situations when you have some text, dates and numbers in your data. Sample output shown below:
Example 3
Unique Product Names 👩💻
df["Product"].unique()
Explanation & Output 💻
This will generate a python array (ndarray) that has all the product names with duplicate values removed.
Example 4
Add “Sales per Box” calculated column to the dataframe 👩💻
df["sales per box"]=df["Sales"]/df["Boxes"]
Explanation & Output 💻
This will add a new column [“sales per box”] to the dataframe with the calculation logic: sales divided by boxes. You can use the same approach to add many other columns
Example 5
Add “Sales as percentage” calculated column to the dataframe 👩💻
total_sales = sum(df.Sales)
df["Sales as a percentage"] = df["Sales"]/total_sales
df
Explanation & Output 💻
First, we calculate the “total_sales” and keep it in a variable. Then we use that variable to calculate the sales as a percentage.
💡 TIP: Do you notice the different ways in which you can refer dataframe columns? You can use dot notation (ex: df.Sales) or bracket notation (ex: df[“Sales”])
HOMEWORK PROBLEMS
Can you add below columns to the df dataframe?
- Sales value rounded to nearest thousand.
- Month number of the sales date
- Flag each record as “Canada” or “Non-Canada”
Example 6
Group Sales by Date and Show a Pivot 👩💻
df.groupby(by="Date").sum()
Explanation & Output 💻
This creates a default groupby (similar to pivot in Excel) of your data by showing totals by date. This will sum() all the number columns in your dataframe. See the below sample output.
Example 7
Group Sales by Date but only show Sales & Boxes columns 👩💻
df.groupby("Date")[["Sales", "Boxes"]].sum()
Explanation & Output 💻
This creates a customized groupby with Sales & Boxes columns totals by Date. Use this pattern when you don’t want to summarize certain things (like Sales per box).
Example 8
Create a bar graph with Daily Sales 👩💻
import matplotlib.pyplot as plt
plt.bar(df["Date"], df["Sales"])
Explanation & Output 💻
We import the plotting library matplotlib.pyplot and use that to generate a bar graph with default settings.
Sample output is shown below:
Example 9
Create a bar graph with Daily Sales – another method 👩💻
df_groups = df.groupby("Date")["Sales"].sum()
df_groups.plot(kind="bar")
Explanation & Output 💻
This code uses the built-in plotting function of the pandas library to generate the bar graph. Notice how this doesn’t show missing dates.
Sample output is shown below:
Example 10
Filter the dataframe to show all records where the product has the word “Choc” 👩💻
df[df["Product"].str.contains("Choc")]
Explanation & Output 💻
This code generates a new dataframe that contains all rows where the Product column has the word “Choc” in it.
MORE HOMEWORK PROBLEMS
- Can you filter all the records that have either “Choc” or “choc”?
- Create a bar graph of this data to show total sales by each product
How does Python in Excel work?
You need internet connection to run Python code in Excel. All the code you write is executed in Microsoft Cloud. This also means your data travels on the network to Microsoft Cloud and returns with the result.
What happens if your code has an error?
If there is an error in your Excel Python code, you will see a new error message #PYTHON! in Excel.
You will also see #BUSY! when Excel is running your Python code (in Microsoft Cloud).
In case of an error in your code, Excel automatically opens the Python Diagnostics tab and displays more information there.
Execution order of your code
The python code you write in Excel will run in row-major order. This means, the code runs row by row, left to right. See this illustration to understand the process.
Resources to Learn Python 🐍
Now that you are familiar with Python in Excel, you may want to learn more. May I suggest using the below approach.
- See if you can enable use Python in Excel to get a feel of the technology.
- Install a proper Python IDE like Anaconda, VS Code or something else to learn & practice Python properly.
- Understand the Python programming concepts like variables, conditions, list comprehension, dataframes and EDA. Here is a good article on the process.
- Apply these concepts on your own / business data to solidify your understanding.
- If you need practice datasets, try Kaggle.
📺 Python Videos
Python in Excel (video by Chandoo)
[NEW]
How to use Python as an Excel Person – FREE Masterclass + 3 Projects
[300k+ views, 1.5 hours long]
End to End data manipulation with Pandas – 10 Examples
[35k views, 18 mins]
📚 Python Books
- Python Crash Course 2nd Edition by Eric Matthes – https://amzn.to/3PBzYRK
This is the book we all (Jo, kids & I) read and really loved it. The explanations and examples are easy enough to get started. There is enough variety to please everyone.
- Automate boring stuff with Python – https://amzn.to/3Py5T5w
More practical if you want to get things done with Python. I read it a few times and really like the practicality of the book.
- Python Data Science Handbook – https://amzn.to/3MFKOUK
Python is particularly useful for doing data science & building machine learning models. This is an area of focus for me in the next months. I suggest getting the Python Data Science book once you have strong foundation in the language.
Note: I am using affiliate link for these books.
💻 Microsoft Resources
As part of the Python for Excel launch, Microsoft also added many resources and example pages to their website. Check out these pages.
35 Responses to “75 Excel Speeding up Tips Shared by YOU! [Speedy Spreadsheet Week]”
I see most are saying that array formulas are bad. But I thought that when you use array formulas it grabs all the data at once and performs the calculations in one fell swoop. At least that is how the UDFs that I created work. When I did the time test it was much faster that way. Maybe I'll go back and check to make sure my work is right, but that's what I did.
When I work with array formulas I get a full column of data then work on that column and return a full column of data all at once. Which has shown to be much faster than the alternative.
Anyone have special insights on this?
wow..! thats a pretty impressive list, some real gems in there. I read somewhere the other day that spreadsheet development should be 80% planning and 20% implementation - taking the time to think about layout, how you're going to calculate things and how to structure the data often results in a lot less headaches, and more time for deeper analysis...
Hi Chandoo...
Mixed feeling about this article, while I completely agree with you when you share everyone's points under their name giving credit to contributors, but at the same time I see alot of repeated tips and few those may not be applicable under all cases. (I may have sounded harsh). Some kind of sorting of tips were required than mere 3 categorization. (Tough ask, I know). Some really worthy and awesome tips get lost in occean.
I would have been happy to read only non repititive tips and more like standard chandoo articles.
Well, I have also got two tips (may get lost in the list above), which is not presented above:
1. When using too many pivots - Disable data drilling option. This reduces file size, cache memory and speeds up file.
2. If your macro has used too many files for gathering input, close the file and open. This is also release or kill unwanted space in memory and speeds up. Again this can be achieved by macro to close self file and reopen same file, using timer function.
Please do not count me negative.
Regards,
Prasad DN
Hey, one thing that really speeds up VBA processing of data is to extract ranges into arrays & then working on those arrays. Arrays are a lot faster than using cell offset or any other method for working with a database. After you are done you can also paste the value directly into the range with one single command. This will speed up the macros considerably.
Here is a msdn article with examples in it:
msdn.microsoft.com/en-us/library/aa139976(v=office.10).aspx
Hope this helps
oops, noticed a typo in the tip I posted:
Re directly assigning values in VBA rather than copying & pasting should have read:
Sheet2.Range( "B1:B200 ").Value= Sheet1.Range( "A1:A200 ").Value
not
Sheet2.Range( "B1:B200 ").Value= Sheet1.Range( "A1:A100 ").Value
Corrected with double quotes:
Sheet2.Range("B1:B200").Value= Sheet1.Range("A1:A200").Value
[...] posts on speeding up Excel worksheets, one of the posts focuses on formulas and another he let the general readers make their suggestions. I made the suggestion that people use array formulas. But most of the other suggestions said not [...]
OK, I tested it. My UDFs were faster as array functions (like a couple thousand times). But Excel's built in functions are faster when not doing arrays. Not sure this is the case for all situations.
I can't help but to speak up.
Deleting a PivotTable will not speed up your workbook. It will only reduce the file size. There is zero memory processing for a pivot table if it just sits there.
Instead of destroying the pivot table, why not remove the redundant raw data. That would equally reduce the file size of the workbook, while keeping the pivot table functionality.
http://datapigtechnologies.com/blog/index.php/cut-the-size-of-your-pivot-table-workbooks-in-half/
Thanks a lot guys for your valuable tips !!
It really worked for me
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
...
...
...
Application.ScreenUpdating = True
Application.Calculation = xlCalculationAutomatic
I would add:
Application.StatusBar = "Wait a second..."
............
Application.StatusBar = False
Optimizing Speed using Pivot Tables:
If you find that Pivot Tables are becoming sluggish becomes of the sheer volume of tables there is a way to increase performance. By default (xls 2010) pivot tables are designed to autofit the data within the columns as well as preserving formatting. I discovered that unchecking these two options alone will exponentially increase performance.
Here's how to do it:
Right click on your pivot table(s) --->choose 'Pivot Table Options' -->Layout & Format' Tab --->uncheck two boxes at the bottom. Done!
here is my list:
1. Avoid Variants when possible
2. Use long instead of integer
3. Use double instead of single
4. Use booleans as much as possible
5. pull data into arrays, manipulate, then dump back to workbook
6. use .value2 instead of .value if you are looking at strings or numeric values
7. set object variables
dim Wks as Excel.Worksheet
set wks = thisworkbook.Sheets("Sheet1")
8. use with statements...
with wks.cells(x,y)
.value2 = "abcd"
.interior.colorindex=4
with .font
.bold=true
.size =20
.underline=xlunderlinestylesingle
end with
9. use string version of functions (Left$() instead of Left())
10. Test for empty string variables with len() or lenb()...if lenb(String1)=0 then
11. Use the Mid$() function if it is possible instead of split()
12. use the join$() function instead of concatenating strings
13. AscW() to evaluate first characters
14. combine if statements and booleans together
boolean = (lenb(String1) = 0)
16. InStr(), InStrB(), InStrRev() are very fast, InStr() can be used to quickly return a substring occurrence of a string
17. DICTIONARIES!!!!!!!!!!!
18. FileSystemObject
19. My machine runs slightly faster when i fill in all the inputs of a function (instead of InStrB(String1, "abcd"), i use (InStrB(1,String1,"abcd",vbBinaryCompare)
20. Short Circuit If statements: If x = 2 then if y >3 then if z = 5 then b=true or
if x=2 then
if y >3 then
if z=5 then
b=true
c=true
end if
end if
end if
21. use ElseIf
22. my testing indicates ElseIf is slightly faster than a Case Select
23. set strings to empty by: = vbnullstring
24. with application
.screenupdating = false
.displayalerts = false
.enableevents = false
end with
25. UserForms can be very beneficial
26. User-Defined Types are a very neat way to encapsulate data
27. User-Defined Functions are handy, but can increase run-time if called thousands of times
28. if using ElseIfs, nested And Ifs or Select Case statements, put the argument that will occur most frequently at the beginning
29. Looping is not the worst thing...just got to figure out how to do it the most efficiently
30. Use dynamic arrays instead of static arrays
31. if you can figure out Win32 APIs, then they are usually much faster than VBA functions
I worked on a massive spreadsheet and it had become very slow over time as I developed it. I tried stripping down more and more formulas by replacing with pasted values, removed all conditional formatting etc. In the end what finally did the trick was when I removed the last single SUMPRODUCT fomula. It changed the updating time after one change from 7-8 seconds to instantaneously. The SUMPRODUCT I had used the full columns, and if I put it to only look at rows 1-500 it was fine.
[...] Speeding up Excel – 75 tips [Visitors: 36,157 ] Using Excel as your database [ 32,455 ] Comprehensive guide to VLOOKUP [ 23,745 ] 66 Dashboards visualizing Excel salary survey data [ 26,148 ] Interactive Sales chart in Excel [ 21,444 ] Compare 2 Excel sheets – howto? [ 21,820 ] Send mails using Excel VBA & Outlook [ 22,294 ] Customer Service Dashboard in Excel [ 18,136 ] Making your dashboards interactive [ 15,294 ] Extract numbers from text in Excel [ 18,490 ] [...]
TIPS FOR SPEEDING UP EXCEL
(1) Instead of writing a lot of formulas to organise data, you can VLOOKUP() the data in a Pivot table, thereby combining the advantages of Pivot table and VOOKUP().
(2) If you have a range named ‘TotalTaxForTheCurrentFinancialYear’, then it is not compulsory to use this name when making the worksheet. Naming the range as ‘Tax’ or simply ‘T’ will be sufficient. The formula =SUM(T) will be shorter and easier to use.
After completing typing all the formulas, simply edit the name of the range from ‘T’ to ‘TotalTaxForTheCurrentFinancialYear’, in the name box. The formula =SUM(T) will automatically change to =SUM(TotalTaxForTheCurrentFinancialYear).
Vijaykumar Shetye, India
VBA
I don't write many macros and like most of you when doing a recalculation it sometimes takes forever.
I have found that when I looked at my spreadsheet, I could determine which order of calcuations (by column) would produce the least number of iterations. So I wrote a macro to do my calculations on my terms. I picked the order of the columns I wanted to calcuate and it sped up my recalc 5-10X.
I did this so long ago, I believe I used "expression .Calculate".
TIPS TO SPEED UP EXCEL by Vijaykumar Shetye, India
You can view all the formulas in the entire worksheet quickly by pressing [ctrl] and [~] keys simultaneously.
To view results, press the key combination again.
i need help about excle lerning and reports making with dash bord i have no facility to join the on classes.
sajjad.hussain165@gmail.com
Is there any command to get time with seconds
Is there any way to create an excel file for specific time period, afterwards it will not open
[...] are looking for , but give it a try : Optimize Slow VBA Code. Speed Up Efficient VBA Code/Macros 75 Excel Speeding up Tips - How to speed-up & optimize slow Excel workbooks? | Chandoo.org - Lea... [...]
I've had to do a lot of mass calculations for reports etc. that involved repetitive identical, yet complex formulae, which took forever...
I hit on a great time-saver: sort the spreadsheet data; if, for example, the same result was due to a lookup of Hotel Name (A column), Date (B column), and Room Type (F column), the formula (Z column) would be: "=IF(A2&B2&F2=A1&B1&F1,Z1,VLOOKUP(A2&B2&F2,LookUps!A:G,7,FALSE))".
This meant that if the result is the same as the row above, just use the same answer, thus saving loads of time instead of VLOOKUPs. (This is a simplified example, the actual one had INDIRECT(ADDRESS...) in it, too!)
Check files for invalid range names, invalid links and names that aren't needed any longer.
Clearing out some 200 old references in a template made the file open go from 30 seconds to 2.
You can improve the speed by stopping calculation during changing cell value and after that you can enable it. please follow the link.
http://webtech-training.blogspot.in/2013/10/how-to-stop-heavy-formula-calculation.html
[…] 75 Excel Speeding Tips Its a long List, many repeated but worth a visit. […]
[…] 75 Speed-up tips by Chandoo (smartly done by crowd sourcing) […]
[…] Are you opening slow excel files? Use this reference to speed up your excel sheets […]
If you want to highlight the content or result within a cell with colour, use content colour not cell fill colour. This make a large data sheet fast as full colour takes up more resource.
There are already so many useful replies, so don't be mad at me if I repeat someone with the following hints.
While using pivot tables:
1. Link (raw) data from external files, rather then building pivots in the same workbook of the data. => Reduces file size.
2. don't flag "keep source data" in pivot settings. => reduces cache.
Downside is when you want to use slicers, you must allow refresh of the source data and thus people need access to that file.
One extra when using tables above ranges: replace the table header references by cell references in heavy duty formulas. I'm not sure but it seems to be faster and lighter (in #MBs).
hi
iam student and need xloptimizer( no demo) for solving the mathematical model
can you help me
thanks alot
@Reza
We cannot give away XLoptimizer
Why not post a question in the Chandoo.org Forums
http://forum.chandoo.org/
Dear Sir,
Thanks a lot for sharing tips & tricks of excel....
I read it , understand it and then use it in job and that has helped me a lot....
Thanks a lot...
Himanshu.
Mumbai, India.
In VBA, send out values to the worksheet all together as an array then excel will only re-calculate once rather than each time a cell that is output.
to the guy who said avoid looping in VBA - easier said than done, it's one of the most powerful uses for VBA out there. I'd therefore recomend the half way house and break the loop as soon as you've got what you want, don't let it run until the end. Use While etc.
What I would say on VBA in general is minimise sheet to code interations. Suck all your data into a VBA array THEN do the maths don't use cells themselves as stand alone visual variables.
And to the lady who said it's faster offline - that's because Microsoft are constantly contacting their own website be it security verification and/or update checks
Apply some logic to the order of criteria in sumifs / countifs formula
order the most exclusive criteria first. Once one criteria fails the others do not execute.
Bing AI, given the following query, confirms this: "excel countifs. if one criteria is likely to exclude most of the data range then should this criteria go first in the list to prevent frivilous executions".