Ah, you thought I will let you go to the next decade without a “year in review” post? No so fast my friend, not so fast.
[This post is a bit long by our standards, but full of gooey excel goodness. So get a cup of coffee or choco and get going]
January 2009
The year 2009 has been pivotal in PHD’s life. It all started with a friendly email from Microsoft on Jan 1st telling me that PHD become an MVP. I felt wonderful knowing that. Even though the award meant little in terms of benefits, it is a great tribute to our little community and the passion we share here.
Later in the first week we wrote a post on how to make combination charts in excel. The discussion on combo charts continued throughout the year, so much that when I posted a holiday greeting card at the end of the year, Santhosh, one of our regulars said that they card looked like a combo chart.
Most importantly we have crossed the milestone of 2000 RSS subscribers in the Jan 2009. To celebrate that I posted one hundred excel tips. That was fun (plus Jo hated me for sitting in front of computer that long).
February
February is fun. I started off the month with an excel twitter client. Which became a wild hit on internet (ok, not so wild, but few of the other blogs in excel community did mention it. Also, JP, the rockstar VBA blogger at codeforoutlookandexcel made an add-in out of it)
Later in the month I got too excited to discover that you can use excel data filters to make a dynamic chart. By far the cheapest and easiest way to make a dynamic chart. We continued the discussion on dynamic charts for the rest of the year and posted several ways to make them.
March
I celebrated the one year anniversary of “conditional formatting rockstar” post by writing 5 more posts on excel conditional formatting. The series started with conditional formatting basics and went on to talk about how you can solve 4 most common problems using excel CF.
We also started writing about excel array formulas and continued that discussion off and on. Array formulas area fun and easy to write (once you have the basics right).
April
This has been a dull month what with my transfer from India to Sweden and sudden lack of internet connectivity. Despite all that we wrapped our first visualization contest on budget vs actual charts and posted some really excellent charting alternatives to the familiar problem.
PHD is also featured on Lifehacker for the Excel Formulas Errors – How to handle them? post.
May
We have proposed “Tweetboards” as an alternative to traditional dashboards and generated good bit of discussion in May. Later several readers emailed me their tweetboard implementations. Slowly tweetboards are spreading in the wild 😉
We also rounded up all the Excel 2007 Productivity Tips.
June
I have stared the Project Management using Excel series in this month with Project Management Gantt Charts. The 6+1 posts soon became legendary and helped me launch the project management templates. In total these posts had more than 200 comments, 150k page views in a short time.
June also was the best month PHD’s history as the blog got featured again on Lifehacker and Delicious home pages for the Excel Mouse Tricks post. Later that month we have rounded up all the techniques you can use to convert excel files to pdfs.
July
We focused on charting more and had the 14 skills you must have for making better charts. I have also written about the all too familiar sumif with multiple criteria problem and some formula solutions for it.
Later that month my post on Using Excel Goal Seek and Finding how much you need for retirement got mentioned in Lifehacker and fetched me a ton of new visitors.
August
Thanks to Aaron, who guest posted about excel waterfall charts in August. In august, I have turned my attention towards the pivot tables and wrote Excel Pivot Tables Tutorial. I have been playing with pivots off and on for a while and this post was my first serious attempt to explore the features. Later I wrote more about them and I am planning to explore pivot tables further in 2010.
In august, we have also crossed the 5000 RSS subscriber mark and celebrated it with a huge contest. Later that month I have wrapped up all the contest entries in the Excel Formulas – 29 tips post.
September
I have started the month with a discussion on Pareto Charts and how to make them in excel. Later that month I wrote about Excel Data Tables features. Both of these posts attracted a lot of discussion and helped me learn valuable new tricks in excel.
Later that month, on September 24th, I became a dad. My life has been the most wonderful and beautiful ever since.
October
In October we wrapped up the project management series with a Project Status Dashboards using Excel. Later that month I have launched the project management templates for excel product. I met several new customers and started to believe that I can make a living out of this blog.
November
In November, we started our most ambitious visualization challenge ever with the Zoho Sales Data Visualization challenge. We now have more than 30 excellent entries and I am waiting for Jan4th when we announce the voting for winner.
Also I have posted about the sumproduct formula and reviewed excel 2010.
December
We started the month with a discussion on using drawing shapes along with charts to make better dashboards. Later in the month I have written about making a quick thermo-meter chart and posted alternatives to compare targets using charts.
Finally I have released the free 2010 calendar excel for you to download and print copies.
To wrap up,
I liked this year thoroughly. Personally it has been nothing short of an exciting ride. We became parents, Jo got promoted, we purchased small piece of land (where we are going to build our dream house) and things couldn’t be better.
Blogwise, the year is equally exciting. I am extremely thankful to all of you for being there for me and encouraging me to learn and share. I met several new people thru this medium and made new friends.
I hope the year had been a memorable experience for you as well.
I sincerely wish you a prosperous new year 2010. Thank you.
PS: Those of you who visited the site yesterday must have seen the nagging “Database error”. I am sorry, but there was a problem when I moved the blog to a different server and the DB went down for almost 24 hours. Now it is up and running smoothly. Let me know if you see something funny.
30 Responses to “Rescue oddly shaped data – Battle between Formulas, VBA and Power Query”
Nice use of Power Query! Power Query is simply awesome! But somehow a lot of people are punishing themselves by not using it (not learning it).
An imperfect 4th approach for consideration... no codes at all...
Select myrange.
Go to Special --> Blank
Delete Cell --> Shift cell left
90% done... now we just need to move the data of 2nd column to the bottom of 1st column
Of course... Power Query is the best.
Cheers,
There is another way but it involves multiple steps:
Copy the values in column E, move the cursor to F5, Paste Special with Skip Blanks, OK
Copy the values in column D, move the cursor to F8, Paste Special with Skip Blanks, OK
And so on.
This works perfectly, albeit a little clumsily apart from the values in B17 and C16, which can be moved with simple copy and paste
Power Query Forever! I do not know how I survived for so long without knowing and using this tool, I can not recommend it to my colleagues, but by the way they prefer to suffer to learn.
My congratulations here from Brazil.
I rolled my eyes when I saw that data
Using decimal places is a nice trick to order data, thanks for that
And tweaking the first formula a bit, you can use OFFSET instead of INDIRECT
=OFFSET($A$1, MIN(IF(myrange, ROW(myrange)), ROWS(A$1:A1))-1, RIGHT(TEXT(MIN(IF(myrange, ROW(myrange) + COLUMN(myrange)*0.00001), ROWS(A$1:A1)), ".00000"), 5)-1)
Tried the above formula with the downloaded oddly shaped data file and I could not get it to work. I get #value without ctrl+shift+enter, and #ref with ctrl+shift+enter.
Sorry, it was SMALL, not MIN.
Add with CTRL+SHIFT+ENTER.
Thank you for your formula. Like the indirect formula I tested this one in older versions of EXCEL and it worked without ALTERATION in EXCEL 95. Very impressive.
Too complicated
Use =Sum to summarize all the sells to the left and Bobs Your Uncle
@Bertie... I am afraid that won't work when you have more than one value in a row.
I tested this formula in versions of Excel all the way back to Excel 95
=IF(ISERROR(INDIRECT("R"&SUBSTITUTE(TEXT(SMALL(IF(MyRange"",ROW(MyRange)+COLUMN(MyRange)*0.00001),ROWS(A$1:A9)),"00000.00000"),".","C"),FALSE)),"",(INDIRECT("R"&SUBSTITUTE(TEXT(SMALL(IF(MyRange"",ROW(MyRange)+COLUMN(MyRange)*0.00001),ROWS(A$1:A9)),"00000.00000"),".","C"),FALSE)))
So there are multiple ways of cleaning up messy data by formulas.
Wow.. Excel 95. Who knew people still use that. But as you have shown, Excel has all these beautiful and powerful functions for 23 years. It has data sciency stuff before DS was even a thing.
I had a problem with pasting the formula in the original post.
Formula should be: =IF(ISERROR(INDIRECT("R"&SUBSTITUTE(TEXT(SMALL(IF(myrange"",ROW(myrange)+COLUMN(myrange)*0.00001),ROWS(A$1:A1)),"00000.00000"),".","C"),FALSE)),"",(INDIRECT("R"&SUBSTITUTE(TEXT(SMALL(IF(myrange"",ROW(myrange)+COLUMN(myrange)*0.00001),ROWS(A$1:A1)),"00000.00000"),".","C"),FALSE)))
EXCEL even in a 16 bit version, is a very robust and capable program.
I don't like the VBA code. If you have a blank row in MyRange, the last entry in the range is doubled up in the paste.here range.
Not really. The macro is writing one cell at a time from paste.here. You have to clean the range before, which I was too lazy to write. But a line like Range(range("paste.here"), range("paste.here").end(xldown)).clearcontents should do the trick.
Adding Range(range("paste.here"), range("paste.here").end(xldown)).clearcontents fixed the problem.
for step split column by delimiter i am not getting option of split into rows or columns. Can you help me in this
Thanks Chandoo for promoting Power Query.
To simplify further, you can "Unpivot Columns" instead of right click on the newly created column and split it by comma in to rows in step 3 of Power Query.
i used
=LOOKUP(10000,B5:F5)
and got the answers. I just plagiarized this formula somewhere and use it, maybe you can explain why it works.
Regards
@Johan... I am not sure if the formula works correctly. When I tested it with the sample data in this post, it showed #N/As in two cells. Essentially, it will only give first value in each row. So if a row has multiple values, then subsequent values are missed. LOOKUP() function goes thru a list and finds the first value that is less than or equal to the input - in this case 10000 in B5:F5.
I have the need to convert pdf's to excel on occasion and they often come out a mess like this. I have used:
Cell G2 =COUNT(myrange)
Cell G3 =IFERROR(IF(G2-1<1,"",G2-1),"") copied down to G100
Cell H2 =IFERROR(LARGE(myrange,G2),"") copied down to H100
Waouw...
=IFERROR(INDIRECT("R" & SUBSTITUTE(TEXT(SMALL(IF(myrange "", ROW(myrange) + COLUMN(myrange)*0.00001),
ROWS(A$1:A1)), "00000.00000"), ".", "C"), FALSE), "")
but CTRL Shift Enter with {} before and after 🙂 😀
Here's a way with pivot table
https://www.bookkempt.com/2018/02/aligning-non-contiguous-data.html
This is brilliant. Bookmarked 🙂
Another possibility.
This assumes that you have a row index 'k' to use in the SMALL function and a column index 'h' to identify the columns of 'myRange'.
If you define 'coord' to refer to
=k+h/10 [assuming h<10]
then it will be possible to recover values later based upon location within 'myRange'. The formula 'nb' that identifies non-blanks by coordinates is given by
= SMALL( IF(myRange"", coord), k )
Finally, to unpick the pieces
= INDEX( myRange, INT(nb), 10*MOD(nb, 1) )
Whilst I am here and making trouble the PQ solution is also a tad over-complicated. All that is needed is to unpivot the entire table and remove the Attribute column.
The advanced editor would show
let
Source = Excel.CurrentWorkbook(){[Name="myRange"]}[Content],
#"Unpivoted Columns" = Table.UnpivotOtherColumns(Source, {}, "Attribute", "Value"),
#"Removed Columns" = Table.RemoveColumns(#"Unpivoted Columns",{"Attribute"})
in
#"Removed Columns"
1.fill the blank cells with 0
2.the requested column value=sum of those mess number column
but this can be used in only one column has value
Chandoo
And if we use the formula SEARCH (100000000, B5: F5)
JC
Another approach with Power Query, it will still work if the number of columns changed:
let
Source = Excel.CurrentWorkbook(){[Name="myrange"]}[Content],
#"Added Custom" = Table.AddColumn(Source, "List", each Record.ToList(_)),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom",{"List"}),
#"Expanded LIst" = Table.ExpandListColumn(#"Removed Other Columns", "List"),
#"Filtered Rows" = Table.SelectRows(#"Expanded LIst", each ([List] null))
in
#"Filtered Rows"
Cool idea to use Record.ToList as added column. Thanks for sharing this.
Nowadays, you can just use TOCOL on Excel 2024, MS 365, and Web Excel. It has a parameter to ignore blanks/errors/both.