In a recent project, we were tasked with analyzing customer behavior from a dataset of 30 shop customers, specifically focusing on satisfaction and loyalty. The aim was to segment customers into groups to help improve business strategy.
The data set that we've got is already loaded in the variable data. A good preliminary step in most analyses is to visualize the data and examine it. One of the better tools to do that is a scatter plot. As we looked at the scatter plot, we wondered, "How many groups of points can you see here?" We immediately noticed two distinct groups standing out. In data science, we would normally call these groups clusters, and we could identify these two clusters instantly, with no machine learning whatsoever.
To dig deeper, we applied the Kmeans algorithm, a popular unsupervised machine learning technique, to identify four distinct customer clusters. We've applied an algorithm on our data to reach an insight. Naturally, we must analyze what we see. Cluster analysis indicates the problem. These clusters were:
- Alienated: Low satisfaction, low loyalty.
- Fans: High satisfaction, high loyalty.
- Supporters: Low satisfaction, high loyalty.
- Roamers: High satisfaction, low loyalty.
Normally, we would have to dig deeper to find the drivers of dissatisfaction for these customers. Maybe it is long queues or unfriendly staff, or perhaps high prices. Whatever the reason, we must take actionable steps to fix the issue and make our supporters happier. Simultaneously, we can do something else. We can turn the roamers into fans by increasing their brand loyalty. Loyalty cards, gifts, personalized discounts, vouchers, and raffles are different strategies used to make such clients loyal in the long run.
The insights we gained were clear: while fans are the ideal customer base, we needed strategies to convert supporters and roamers into fans. This project highlighted the importance of storytelling in data science—translating numbers into actionable insights that drive business decisions.
=======================================================================
importance of data and common terminology in data science:
Importance of Data:
- Foundation of Business: Data is essential for informed decision-making and strategy development.
- Role of Data Teams:
- Data Team: Analyzes data to solve business problems.
- Business Intelligence Team: Creates dashboards for insights.
- Data Science Team: Develops predictive models to forecast outcomes.
Complexity of Terminology:
- Evolving Buzzwords: Terms like data mining and predictive analytics reflect the industry's growth but have similar foundational goals.
- Job Title Confusion: Misleading titles can lead to misunderstandings about the roles needed (e.g., data analyst vs. data analytics specialist).
Clarifying Data Science Concepts:
- Understanding Key Terms: Familiarity with terms such as business analytics, data analytics, data science, business intelligence, and machine learning is crucial for effective communication in the field.
Key Differences Between Analysis and Analytics
Definition:
Analysis:
- Focuses on examining historical data.
- Involves breaking down a large dataset into manageable parts to understand past events (e.g., sales trends).
- Answers questions like how and why something happened.
Analytics:
- Primarily concerned with predicting future outcomes.
- Involves applying logical and computational reasoning to insights gained from analysis.
- Seeks patterns and actionable strategies for future decision-making.
Types of Analytics:
- Qualitative Analytics:
- Utilizes intuition and experience to inform decisions (e.g., choosing clothing styles based on trends).
- Quantitative Analytics:
- Applies mathematical formulas and algorithms to data for forecasting (e.g., predicting the best month to launch a new collection based on past sales).
- Qualitative Analytics:
Combining Analysis and Analytics:
- You can perform qualitative analysis to understand past outcomes and quantitative analysis to explain historical data (e.g., analyzing why sales decreased last summer).
Related Terms:
- Similar distinctions apply to terms like data analysis, data analytics, business analysis, and business analytics, each having specific meanings based on their focus on past or future data usage.
Techniques for Working with Traditional Data
Overview:
- The section covers techniques used in traditional data, big data, business intelligence (BI), traditional data science methods, and machine learning. The goal is to help viewers relate terms from the infographic to their corresponding techniques and applications.
Data Definition:
- Data: Refers to raw facts, processed data, or information.
- Raw Data: Untouched data collected from various sources that must be processed for analysis.
Data Collection:
- Methods include:
- Surveys: Gathering opinions on products using a rating scale (e.g., 1 to 10).
- Automated Collection: Using tools like cookies to track user activity on websites.
- Methods include:
Data Pre-processing:
- Purpose: Converts raw data into a usable format by addressing issues that arise during data collection.
- Common Issues:
- Invalid entries (e.g., unrealistic ages).
- Inconsistent data.
Techniques in Data Pre-processing:
- Class Labeling: Categorizing data points into types (e.g., numerical vs. categorical).
- Numerical Data: Quantifiable values (e.g., goods sold).
- Categorical Data: Qualitative information (e.g., profession).
- Data Cleansing (Cleaning/Scrubbing): Correcting inconsistencies in data.
- Handling Missing Values: Decisions on how to manage incomplete data entries (e.g., using average values).
- Class Labeling: Categorizing data points into types (e.g., numerical vs. categorical).
Specific Approaches for Traditional Data:
- Balancing: Ensuring equal representation in survey data (e.g., equal numbers of male and female respondents).
- Data Shuffling: Randomizing dataset observations to eliminate patterns and improve predictive performance.
Visualizations in Traditional Data:
- Entity Relationship Diagram (ER Diagram): Illustrates the architecture of a database and how tables relate to each other.
- Relational Schema: Represents data tables and their relationships through rectangles and lines.
Recap:
- Emphasis on the importance of data collection and pre-processing for quantitative analysis.
- Recognition that there are numerous techniques available for working with traditional data, focusing here on the most frequently used.
====================================================================
1. Customer IDs as Categorical Data
- Imagine you have a table of customer data, and each customer has a unique ID. For example:
- Customer ID 1
- Customer ID 2
- Customer ID 3
- And so on...
These numbers 1, 2, 3, etc. are just labels for different customers, and they don't have numerical meaning. You can't calculate the average customer ID, and it wouldn't give you any meaningful information. They are simply used to identify or categorize customers.
- Example:
Customer ID Name 1 Alice 2 Bob 3 Charlie
Here, the customer IDs 1, 2, 3 are categorical because they just represent different people, not something you can perform calculations on.
2. Dates as Categorical Data
Similarly, Dates like 24-10-2021 or 25-10-2021 are also considered categorical. Dates provide a reference point in time but don't represent numbers you can do arithmetic with in most cases (e.g., you can't find the "average" date in the same way you can with numbers).
Example:
Date Stock Price 24-10-2021 $150 25-10-2021 $155 26-10-2021 $152
Here, 24-10-2021, 25-10-2021, and 26-10-2021 are all categorical data because they just indicate when an event (like a stock price change) occurred.
Key Difference:
- Both Customer IDs and Dates are categorical because they don't involve any meaningful arithmetic operations. They are identifiers, not values you can calculate like you would with numerical data (e.g., number of complaints or stock prices).
These examples show that although Customer IDs and Dates may look like numbers or numerical values, they are treated as labels or categories in data analysis, making them categorical data.
=================================================================
1. Traditional Data vs. Big Data
Traditional Data:
- Think of customer information stored in tables, like a list of customers with their names, ages, and how many times they bought something.
- Example:
- Here, "Purchases" is numerical (you can add them), but "Name" is categorical (you can’t do math on names).
Big Data:
- This refers to huge and complex data that isn’t easily stored in a simple table. Think of all the text, images, videos, and audio files collected from social media, websites, or sensors.
- Example: Customer reviews from an online store, stored in a large database:
- You need special tools to process this kind of data since it’s not organized neatly in rows and columns like traditional data.
2. Dealing with Missing Data
Missing Data in Traditional Data:
- Sometimes, you might not have complete information for every customer. For example, maybe a customer didn't give their age.
- Example:
- One way to deal with this is to fill in the missing age with the average age of the other customers.
Missing Data in Big Data:
- In big data, there could be missing pieces in large datasets, like incomplete product reviews. You need more advanced methods to handle missing values. For example, you might use a machine learning model to predict the missing information based on other details.
3. Text Data Mining (Working with Text Data in Big Data)
- Example:
- Imagine you want to analyze customer reviews to understand common issues with your product. Instead of reading through thousands of reviews one by one, you use a process called text data mining to automatically find important words or phrases like "poor quality" or "fast delivery."
- Outcome: This helps you quickly find patterns in what customers are saying, without manually reading every review.
4. Data Masking (Protecting Sensitive Data)
- Example:
- Let’s say a company is analyzing customer data but wants to keep personal information private. To do this, they use data masking, which hides details like customer names or phone numbers.
- Before Masking:
- After Masking:
- This way, the company can still analyze purchase trends without exposing personal data.
5. Visualizing Big Data
- Example:
- Let’s say a retailer wants to see how customers interact with their website and social media. They gather data from different sources like website clicks, customer reviews, and social media comments.
- Visualization: A graph with many connected lines shows how the data from different sources is related. This helps the retailer see patterns in customer behavior.
Simple Recap:
- Traditional data is easy to organize in tables (like customer details).
- Big data is more complex and includes text, images, and videos from many sources.
- Text data mining helps find useful information in huge amounts of text.
- Data masking protects private information while still allowing data analysis.
- Big data visualizations help make sense of complex data connections.
These techniques help companies turn raw data (whether small or big) into useful insights!