18 ways to turn analysis projects into a nightmare
Every week, we read news about failed analysis projects. If you listen carefully, you can hear the grunts, screams and curses of thousands of analysts all over the world about their analysis nightmares.
At Chandoo.org, we talk a lot about best practices for data analytics. So today, let’s peek in to the dark side and understand the mistakes that can turn your analysis project into a nightmare.
There are 3 parts in any analysis project
To understand these worst practices in analysis world, first let’s break analysis projects in to 3 parts.
- Requirements
- Data Structure
- Tools & Construction
Let’s deep dive in to each area of the analysis projects to see what can go wrong.
Analyzing 300,000 calls for help [case study]
Over the weekend, I got an email from Mr. E, one of my students. Mr. E works at a police department in California and as part of his work, he was looking at calls received by police. Whenever police get a call for help, multiple teams can respond to the call and go to the location. All of these dispatches are recorded. So a single call can have several such dispatches. And Mr. E wanted to findout which team responded the first. The problem?
Finding the first responded team is tricky.
Today let’s take up this problem as a case study and understand various methods to solve it. We are going to learn about writing better lookups, pivot tables, power pivot and optimization. Put on your helmets, cause this is going to be mind blowingly awesome.
Excel Links – Getting used to life in Windy Wellington Edition
So we moved to Wellington, New Zealand few weeks back (on 17th of July 2016, to be precise). After spending first 3 weeks in Jeff’s house and a hotel, we moved in to our rental home over the weekend (on 6th of August). Around the same time, the worst of Wellington winter waved welcome to us. We quickly learned how to stay warm indoors (layers, hot water bottles, rugs and more layers). Kids started going to school few days back and they are loving it. I bought a bike and managed to go out on few rides on the hilly roads of Wellington and found a strange for sale sign too.
Anyhow, Since we didn’t have internet connection until today, I thought I will start by sharing a few Excel links with you. Check them out to get your fix of spreadsheets.
Read on…
Analyzing half a million consumer complaints [Part 1 of 3]
How would you analyze data when you have lots of it? That is the inspiration for this series.
Let’s meet our data – Finance Industry Consumer Complaints
As part of open data initiatives, US government & Consumer Financial Protection Bureau maintain a list of all consumer complaints made against financial institutions (banks, credit unions etc.) You can download this data from the catalog page here. I have obtained the data on 1st of February, 2016. The download has 513,824 records. Each row contains one complaint.
In this and next two parts of the series, we are going to analyze these half a million complaints to find insights.
Reconcile debits & credits using Solver [Advanced Excel]
Here is a tricky problem often faced by accountants and finance professionals: Let’s say you have 5 customers. Each of them need to pay you some money. Instead of paying the total amount in one go, they paid you in 30 small transactions. The total amount of these transactions matches how much they need to pay you. But you don’t know which customer paid which amounts. How would you reconcile the books?
If you match the transactions manually, it can take an eternity – after all there are more than 931 zillion combinations (5^30).
This is where solver can be handy. Solver can find optimal solution for problems like this before you finish your first cup of coffee.
CP030: Detecting fraud in data using Excel – 5 techniques for you
Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | RSS
In the 30th session of Chandoo.org podcast, let’s learn how to uncover fraud in data.
What is in this session?
In the wake of hedge fund scams, accounting frauds and globalization, We, analysts are constantly second guessing every source of data. So how do you answer a simple question like, “am I being lied to?” while looking at a set of numbers your supplier has sent you.
That is our topic for this podcast session.
In this podcast, you will learn
- Quick announcements about 50 ways & 200k BRM
- Introduction to fraud detection
- 5 techniques for detecting fraud
- Benford’s law
- Auto correlation
- Discontinuity at zero
- Analysis of distribution
- Learning systems & decision trees
- Implementing these techniques in Excel
- A word of caution
Who is the most consistent seller? [BYOD]
Who is the most consistent of all?
Imagine you are a category manager at a large e-commerce company. Your site offers various products, but you don’t really make these products. You list products made by other vendors on your site. Every day, these vendors would send you invoices for the amount of product they have sold. Above is a snapshot of such invoices.
Looking at this list, you have a few questions.
- Who is the best seller?
- Who is the most active seller?
- Who is the most consistent seller?
- Which seller has fewest invoices?
Let’s go ahead and answer these using Excel. Shall we?