A/B Testing for Marketing Campaign
This project aimed to determine the effectiveness of a marketing ad campaign in increasing conversion rates. Using a dataset from Kaggle, we conducted an A/B test to compare the conversion rates of users exposed to the ad (treatment group) with those who were not (control group). Through data wrangling, hypothesis testing, and statistical analysis (bootstrapping, binomial sampling, and two proportion z-test), we found that the ad campaign significantly improved conversions. Additionally, a logistic regression analysis confirmed the statistical significance and practical impact of the ad campaign.
Introduction
In this project, we aimed to determine whether an ad campaign led to an increase in conversions. We assessed whether this increase was both statistically and practically significant. The analysis was conducted using a dataset from Kaggle called "Marketing A/B Testing."
Objective
The primary goal was to evaluate the effectiveness of the ad campaign by comparing the conversion rates of a treatment group (exposed to ads) and a control group (not exposed to ads or shown a public service announcement). We employed A/B testing, regression analysis, and various statistical methods to draw conclusions.
Technologies Used
Python: For data manipulation, analysis, and visualization.
Libraries: pandas, numpy, matplotlib, statsmodels, scipy.
Statistical Methods: Bootstrapping, Binomial Sampling, Two Proportion z-test, Logistic Regression.
Data Wrangling
We started by reading and cleaning the dataset, removing unnecessary columns, and ensuring there were no duplicate entries. We also transformed categorical variables for easier analysis.
Exploratory Data Analysis
Initial analysis showed:
No missing values in the dataset.
Conversion rate was represented as a boolean value (True/False) which was conve
rted to binary (1/0) for analysis.
Treatment group (exposed to ads) and control group (not exposed) were identified.
Statistical Analysis
Bootstrapping Method:
Simulated samples to create a sampling distribution.
Observed that the conversion rate difference was significantly in favor of the treatment group.
Binomial Sampling Method:
Created samples using binomial distribution.
Similar results to bootstrapping, reinforcing the findings.
Two Proportion z-test:
Calculated z-score and p-value to confirm statistical significance.
Resulted in a very small p-value, indicating a significant difference in conversion rates.
Regression Analysis
Performed a logistic regression to further validate findings:
Created a dummy variable for the treatment group.
Found that the treatment group was 1.44 times more likely to convert compared to the control group.
The coefficient for the treatment group was statistically significant.
Conclusion
All tests indicated that the ad campaign significantly increased conversion rates. However, to assess the financial success of the campaign, additional data on costs and revenue would be required.
Key Takeaways
The ad campaign effectively increased conversions.
Statistical and practical significance were established through multiple methods.
Further financial analysis is needed for a comprehensive evaluation of the campaign's success.