Project Business Statistics: E-news Express

Define Problem Statement and Objectives

Import all the necessary libraries

# Installing the libraries with the specified version.
## Removed the version specifier to install the latest version. Using Python 3.12.3 on Macbook Pro M4 PRO
%pip install numpy pandas matplotlib seaborn scipy -q --user

Note: you may need to restart the kernel to use updated packages.

Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.

#Importing libraries
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import seaborn as sns
from scipy import stats

Importing libraries needed for analysis

#Accesing Google Drive
#from google.colab import drive
#drive.mount('/content/drive')

Mounting Google Drive to access data

Reading the Data into a DataFrame

#Reading the dataset
data = pd.read_csv('abtest.csv')
data.head()

	user_id	group	landing_page	time_spent_on_the_page	converted	language_preferred
0	546592	control	old	3.48	no	Spanish
1	546468	treatment	new	7.13	yes	English
2	546462	treatment	new	4.40	no	Spanish
3	546567	control	old	3.02	no	French
4	546459	treatment	new	4.75	yes	Spanish

Reading CSV and getting the first 5 rows with head

#Getting rows and columns
rows, cols = data.shape

print(f"The dataset has {rows} rows and {cols} columns.")

The dataset has 100 rows and 6 columns.

Observations: Getting the number of rows and columns

Explore the dataset and extract insights using Exploratory Data Analysis

• Data Overview
  • Viewing the first and last few rows of the dataset
  • Checking the shape of the dataset
  • Getting the statistical summary for the variables
• Check for missing values
• Check for duplicates

#DATA OVERVIEW
# Display the first five rows of the dataset
print("First 5 rows of the dataset:")
print(data.head())

# Display the last five rows of the dataset
print("\nLast 5 rows of the dataset:")
print(data.tail())

# Get the shape (number of rows and columns) of the dataset
print("\nDataset dimensions (rows, columns):", data.shape)

# Get a statistical summary of numerical variables
print("\nStatistical summary of numerical variables:")
print(data.describe())

#MISSING VALUES
# Check for missing values in each column
print("\nMissing values per column:")
print(data.isnull().sum())

#DUPLICATES
# Check for duplicate rows in the dataset
duplicate_count = data.duplicated().sum()
print(f"\nNumber of duplicate rows in the dataset: {duplicate_count}")

First 5 rows of the dataset:
   user_id      group landing_page  time_spent_on_the_page converted  \
0   546592    control          old                    3.48        no   
1   546468  treatment          new                    7.13       yes   
2   546462  treatment          new                    4.40        no   
3   546567    control          old                    3.02        no   
4   546459  treatment          new                    4.75       yes   

  language_preferred  
0            Spanish  
1            English  
2            Spanish  
3             French  
4            Spanish  

Last 5 rows of the dataset:
    user_id      group landing_page  time_spent_on_the_page converted  \
95   546446  treatment          new                    5.15        no   
96   546544    control          old                    6.52       yes   
97   546472  treatment          new                    7.07       yes   
98   546481  treatment          new                    6.20       yes   
99   546483  treatment          new                    5.86       yes   

   language_preferred  
95            Spanish  
96            English  
97            Spanish  
98            Spanish  
99            English  

Dataset dimensions (rows, columns): (100, 6)

Statistical summary of numerical variables:
             user_id  time_spent_on_the_page
count     100.000000              100.000000
mean   546517.000000                5.377800
std        52.295779                2.378166
min    546443.000000                0.190000
25%    546467.750000                3.880000
50%    546492.500000                5.415000
75%    546567.250000                7.022500
max    546592.000000               10.710000

Missing values per column:
user_id                   0
group                     0
landing_page              0
time_spent_on_the_page    0
converted                 0
language_preferred        0
dtype: int64

Number of duplicate rows in the dataset: 0

Observations:

DATA OVERVIEW

df.head() displays the first 5 rows, while df.tail() shows the last 5 rows. This helps us understand the column names, data types, and overall structure. We can immediately spot potential issues such as incorrect formatting, unexpected values, or missing data.

df.describe() provides important summary statistics for numerical columns, including: Mean (avg), median, standard deviation, min, max, and quartiles. It helps identify: Outliers (if min/max values are far from the mean). Distributions (e.g., if the mean and median differ significantly). Possible data errors (e.g., negative values where not expected).

MISSING VALUES

df.isnull().sum() counts missing values for each column. Missing data can be handled using: Imputation (filling missing values with mean, median, or mode). Deletion (if too many missing values make the data unreliable). If many missing values are present, we need further investigation.

DUPLICATES

df.duplicated().sum() counts exact duplicate rows. Duplicates can skew statistical analysis and should be removed if not necessary. If duplicates exist, we need to investigate: Were they added by mistake? Are they meaningful (e.g., users with multiple interactions)? Should we keep the first occurrence and remove others?

Univariate Analysis

Observations:

Categorical variables in the dataset:

group: Control vs. Treatment

landing_page: Old vs. New

converted: Yes vs. No

language_preferred: Spanish, French, English

# Count plots for categorical variables
categorical_columns = ["group", "landing_page", "converted", "language_preferred"]

for col in categorical_columns:
    plt.figure(figsize=(6, 4))
    sns.countplot(x=data[col], hue=data[col], palette="Set2", legend=False)
    plt.title(f"Distribution of {col}")
    plt.xlabel(col)
    plt.ylabel("Count")
    plt.xticks(rotation=45)
    plt.show()

# Summary statistics of time spent on the page
print("\nSummary Statistics for Time Spent on the Page:")
print(data["time_spent_on_the_page"].describe())

Summary Statistics for Time Spent on the Page:
count    100.000000
mean       5.377800
std        2.378166
min        0.190000
25%        3.880000
50%        5.415000
75%        7.022500
max       10.710000
Name: time_spent_on_the_page, dtype: float64

# Histogram for time spent on the page
plt.figure(figsize=(6, 4))
sns.histplot(data["time_spent_on_the_page"], bins=15, kde=True, color="blue")
plt.title("Distribution of Time Spent on the Page")
plt.xlabel("Time Spent (minutes)")
plt.ylabel("Frequency")
plt.show()

# Boxplot for time spent on the page
plt.figure(figsize=(6, 4))
sns.boxplot(x=data["time_spent_on_the_page"], color="green")
plt.title("Boxplot of Time Spent on the Page")
plt.xlabel("Time Spent (minutes)")
plt.show()

Observations:

Group Distribution

The dataset is evenly split between control (old page) and treatment (new page) groups.

Balanced experiment: This ensures a fair A/B test, as both groups have an equal number of users.

Insight: No sampling bias—both groups had an equal chance of exposure.

Landing Page Distribution

The control group was shown the old page, and the treatment group was shown the new page.

Correct Mapping: There are no inconsistencies (e.g., treatment users seeing the old page).

Insight: The experimental design is correctly implemented.

Conversion Rate

The conversion rate (users who subscribed) is higher for the treatment (new page) group.

New page appears to be more effective at encouraging subscriptions.

Insight: The A/B test likely indicates a positive impact of the new landing page on conversions.

Next Steps: A statistical test (e.g., proportion Z-test) is required to confirm if the difference is significant.

Prefered Language

The dataset contains three primary language preferences: Spanish, French, and English.

The distribution of languages is nearly balanced, with a slight variation.

No major dominance of one language over others.

Insight: Language preference could be analyzed further to see if it impacts conversion rates or engagement.

Time Spent on the Page

• Histogram Analysis:

Users spent more time on the new page compared to the old page. The distribution is slightly right-skewed, indicating some users stayed much longer than others.

• Boxplot Analysis:

There are outliers—some users spent significantly more time than average. The median time spent is higher for the new page than the old one.

Higher engagement on the new page suggests it might be more effective.

Insight: Time spent is a key metric—if the new page keeps users engaged longer, it might explain the increase in conversions.

Bivariate Analysis

#Analizing Landing Page and Time Spend on the Page
plt.figure(figsize=(6, 4))
sns.boxplot(x=data["landing_page"], y=data["time_spent_on_the_page"], hue=data["landing_page"], palette="Set2", legend=False)
plt.title("Time Spent on the Page by Landing Page")
plt.xlabel("Landing Page")
plt.ylabel("Time Spent (minutes)")
plt.show()

plt.figure(figsize=(6, 4))
sns.violinplot(x=data["landing_page"], y=data["time_spent_on_the_page"], hue=data["landing_page"], palette="muted", legend=False)
plt.title("Density of Time Spent by Landing Page")
plt.xlabel("Landing Page")
plt.ylabel("Time Spent (minutes)")
plt.show()

Observations:

Users spend more time on the new landing page.

Higher variance in time spent suggests better engagement.

Hypothesis: More time spent → Higher conversion.

#Conversion and Time Spent on Page
plt.figure(figsize=(6, 4))
sns.boxplot(x=data["converted"], y=data["time_spent_on_the_page"], hue=data["converted"], palette="coolwarm", legend=False)
plt.title("Time Spent by Conversion Status")
plt.xlabel("Converted (Yes/No)")
plt.ylabel("Time Spent (minutes)")
plt.show()

Observations:

Converted users spent more time on the page than non-converted users.

Some outliers spent a long time but didn’t convert.

Next step: Test if time spent significantly affects conversion.

#Language vs Time Spent on Page
plt.figure(figsize=(6, 4))
sns.boxplot(x=data["language_preferred"], y=data["time_spent_on_the_page"], hue=data["language_preferred"], palette="pastel", legend=False)
plt.title("Time Spent by Preferred Language")
plt.xlabel("Language Preferred")
plt.ylabel("Time Spent (minutes)")
plt.show()

Observations:

Minor variations between languages.

Some languages (English) might spend slightly more time.

Next step: Perform an ANOVA test to check statistical significance.

#Group (Control vs. Treatment) vs. Conversion Rate
plt.figure(figsize=(6, 4))
sns.barplot(x=data["group"], y=data["converted"].apply(lambda x: 1 if x == "yes" else 0), hue=data["group"], palette="Set1", legend=False)
plt.title("Conversion Rate by Group")
plt.xlabel("Group")
plt.ylabel("Conversion Rate")
plt.show()

Observations:

The new page converts more users than the old page.

Next step: Run a Chi-Square Test to confirm statistical significance.

#Conversion Status vs. Landing Page

plt.figure(figsize=(6, 4))
sns.countplot(x=data["landing_page"], hue=data["converted"], palette="coolwarm", legend=False)
plt.title("Conversion Rate by Landing Page")
plt.xlabel("Landing Page")
plt.ylabel("Count")
plt.show()

Observations:

Higher conversions on the new page.

Next step: Conduct a Z-Test for proportions.

1. Do the users spend more time on the new landing page than the existing landing page?

Perform Visual Analysis

# Set visualization style
sns.set_style("whitegrid")

# Boxplot: Compare the distribution of time spent on the page for each landing page
plt.figure(figsize=(6, 4))
sns.boxplot(x=data["landing_page"], y=data["time_spent_on_the_page"], hue=data["landing_page"], palette="Set2", legend=False)
plt.title("Time Spent on the Page by Landing Page")
plt.xlabel("Landing Page")
plt.ylabel("Time Spent (minutes)")
plt.show()

# Violin plot: Show the density of time spent on each landing page
plt.figure(figsize=(6, 4))
sns.violinplot(x=data["landing_page"], y=data["time_spent_on_the_page"], hue=data["landing_page"], palette="muted", legend=False)
plt.title("Density of Time Spent by Landing Page")
plt.xlabel("Landing Page")
plt.ylabel("Time Spent (minutes)")
plt.show()

# Histogram with KDE: Show the distribution of time spent on each landing page
plt.figure(figsize=(6, 4))
sns.histplot(data=data, x="time_spent_on_the_page", hue="landing_page", kde=True, bins=15, palette="coolwarm")
plt.title("Distribution of Time Spent by Landing Page")
plt.xlabel("Time Spent (minutes)")
plt.ylabel("Frequency")
plt.show()

Step 1: Define the null and alternate hypotheses

• Null Hypothesis (H₀): Users spend the same or less time on the new landing page as on the existing one.
  $$ H_0: \mu_{\text{new}} \leq \mu_{\text{old}} $$

• Alternate Hypothesis (H₁): Users spend more time on the new landing page than on the existing one.
  $$ H_1: \mu_{\text{new}} > \mu_{\text{old}} $$

Since we are checking whether the new landing page increases engagement, this is a one-tailed (right-tailed) test.

Step 2: Select Appropriate test

The data consists of two independent groups (Control: old page, Treatment: new page).

The dependent variable (time spent on page) is continuous.

We assume approximately normal distribution due to the Central Limit Theorem (n=100 is reasonable).

Appropriate Test: Independent t-test (one-tailed, right-tailed)

Step 3: Decide the significance level

Significance Level $\alpha = 0.05$

If p-value < 0.05, we reject the null hypothesis.

Step 4: Collect and prepare data

# Separate the time spent based on landing page type
time_spent_old = data[data["landing_page"] == "old"]["time_spent_on_the_page"]
time_spent_new = data[data["landing_page"] == "new"]["time_spent_on_the_page"]

# Display summary statistics
print("\nSummary statistics for time spent:")
print("Old Page:\n", time_spent_old.describe())
print("\nNew Page:\n", time_spent_new.describe())

Summary statistics for time spent:
Old Page:
 count    50.000000
mean      4.532400
std       2.581975
min       0.190000
25%       2.720000
50%       4.380000
75%       6.442500
max      10.300000
Name: time_spent_on_the_page, dtype: float64

New Page:
 count    50.000000
mean      6.223200
std       1.817031
min       1.650000
25%       5.175000
50%       6.105000
75%       7.160000
max      10.710000
Name: time_spent_on_the_page, dtype: float64

Observations:

If mean (new page) > mean (old page), it suggests that users might be spending more time on the new page.

However, we need a statistical test to confirm.

Step 4a: Check assumptions (Normality & Equal Variances)

# Create samples per language (correct column names)
english_time = data[data['language_preferred'] == 'English']['time_spent_on_the_page']
spanish_time = data[data['language_preferred'] == 'Spanish']['time_spent_on_the_page']
french_time  = data[data['language_preferred'] == 'French']['time_spent_on_the_page']



# 1) Normality (Shapiro-Wilk) per group
print("Shapiro-Wilk (normality) per group:")
for name, sample in [("English", english_time), ("Spanish", spanish_time), ("French", french_time)]:
    stat, p = stats.shapiro(sample)
    print(f"{name}: W={stat:.4f}, p={p:.4f}")

# 2) Homogeneity of variance (Levene)
levene_stat, levene_p = stats.levene(english_time, spanish_time, french_time)
print("\nLevene (equal variances):")
print(f"Statistic={levene_stat:.4f}, p={levene_p:.4f}")

Shapiro-Wilk (normality) per group:
English: W=0.9825, p=0.8691
Spanish: W=0.9573, p=0.2021
French: W=0.9762, p=0.6512

Levene (equal variances):
Statistic=2.8094, p=0.0652

The Shapiro–Wilk test was used to assess normality within each language group.
All groups showed p-values greater than 0.05, indicating no significant departure from normality.

Levene’s test was applied to evaluate the homogeneity of variances across groups.
The resulting p-value (0.0652) suggests that the assumption of equal variances is reasonably satisfied.

Step 5: Calculate the p-value

from scipy import stats

# Perform one-tailed (right-tailed) independent t-test
t_stat, p_value = stats.ttest_ind(time_spent_new, time_spent_old, alternative="greater")

# Print results
print(f"\nT-statistic: {t_stat:.4f}")
print(f"P-value: {p_value:.4f}")

T-statistic: 3.7868
P-value: 0.0001

Observations:

• The T-statistic tells us how many standard deviations the sample mean difference is from 0.
• The p-value helps determine statistical significance.

Step 6: Compare the p-value with $\alpha$

alpha = 0.05  # Significance level

if p_value < alpha:
    print("\nConclusion: Reject the null hypothesis (H₀).")
    print("Users spend significantly more time on the new landing page.")
else:
    print("\nConclusion: Fail to reject the null hypothesis (H₀).")
    print("There is no significant evidence that users spend more time on the new landing page.")

Conclusion: Reject the null hypothesis (H₀).
Users spend significantly more time on the new landing page.

Observations:

If p-value < 0.05 → We reject $H_0$, meaning users spend more time on the new page.

If p-value > 0.05 → We fail to reject $H_0$, meaning we don’t have enough evidence to conclude that users spend more time on the new page.

Step 7: Draw inference

Based on our results:

The p-value is small (< 0.05), we conclude that users spend more time on the new landing page.

A similar approach can be followed to answer the other questions.

2. Is the conversion rate (the proportion of users who visit the landing page and get converted) for the new page greater than the conversion rate for the old page?

Perform Visual Analysis

# Countplot for conversion rates by landing page
plt.figure(figsize=(8, 5))
sns.countplot(x="converted", hue="landing_page", data=data, palette="Set2")
plt.title("Conversion Rate by Landing Page")
plt.xlabel("Converted (Yes/No)")
plt.ylabel("Count")
plt.legend(title="Landing Page", labels=["Old", "New"])
plt.show()

Observations:

The New page has more conversions

Step 1: Define the null and alternate hypotheses

• Null Hypothesis (H₀): The conversion rate for the new landing page is less than or equal to the conversion rate for the old landing page.
  $$ H_0: p_{\text{new}} \leq p_{\text{old}} $$

• Alternate Hypothesis (H₁): The conversion rate for the new landing page is greater than the conversion rate for the old landing page.
  $$ H_1: p_{\text{new}} > p_{\text{old}} $$

Since we are testing whether the new landing page improves conversion rates, this is a one-tailed (right-tailed) test.

Step 2: Select Appropriate test

Since we are comparing proportions (conversion rates) between two independent groups, we will use a two-proportion z-test (one-tailed).

Step 3: Decide the significance level

alpha = 0.05  # 5% significance level

Significance Level $\alpha = 0.05$

If p-value < 0.05, we reject the null hypothesis.

Step 4: Collect and prepare data

# Convert 'converted' column to binary (yes=1, no=0)
data["converted"] = data["converted"].map({"yes": 1, "no": 0})

# Count conversions for each landing page
conversions_old = data[data["landing_page"] == "old"]["converted"].sum()
conversions_new = data[data["landing_page"] == "new"]["converted"].sum()

# Total users in each group
n_old = data[data["landing_page"] == "old"].shape[0]
n_new = data[data["landing_page"] == "new"].shape[0]

# Conversion rates
p_old = conversions_old / n_old
p_new = conversions_new / n_new

# Display values
conversions_old, n_old, p_old, conversions_new, n_new, p_new

(21, 50, 0.42, 33, 50, 0.66)

Step 4a: Check assumptions (Normality & Equal Variances)

Since conversion is a binary variable, normality and homogeneity of variance assumptions are not required. A two-proportion z-test is therefore appropriate for comparing conversion rates.

Step 5: Calculate the p-value using a Two-Proportion Z-Test

from statsmodels.stats.proportion import proportions_ztest

# Perform one-tailed z-test for proportions
count = np.array([conversions_new, conversions_old])  # Success counts
nobs = np.array([n_new, n_old])  # Total sample sizes

z_stat, p_value = proportions_ztest(count, nobs, alternative="larger")

# Display results
z_stat, p_value

(2.4077170617153842, 0.008026308204056278)

Step 6: Compare the p-value with $\alpha$

if p_value < alpha:
    conclusion = "Reject the null hypothesis: The new landing page has a significantly higher conversion rate."
else:
    conclusion = "Fail to reject the null hypothesis: No significant evidence that the new landing page has a higher conversion rate."

Step 7: Draw inference

#z_stat, p_value, conclusion
print(f"\nZ-statistic: {z_stat:.4f}")
print(f"P-value: {p_value:.4f}")    
print("\nConclusion:", conclusion)

Z-statistic: 2.4077
P-value: 0.0080

Conclusion: Reject the null hypothesis: The new landing page has a significantly higher conversion rate.

Observations:

The new landing page has a significantly higher conversion rate.

3. Is the conversion and preferred language are independent or related?

Perform Visual Analysis

# Create a countplot to visualize the relationship between conversion and preferred language
plt.figure(figsize=(8, 5))
sns.countplot(x="converted", hue="language_preferred", data=data, palette="Set2")
plt.title("Conversion Rate by Preferred Language")
plt.xlabel("Converted (Yes/No)")
plt.ylabel("Count")
plt.legend(title="Preferred Language")
plt.show()

Step 1: Define the null and alternate hypotheses

• Null Hypothesis (H₀): Conversion and preferred language are independent (i.e., language preference does not affect conversion rates). $$ H_0: \text{Conversion} \perp \text{Language Preferred} $$

• Alternate Hypothesis (H₁): Conversion and preferred language are related (i.e., language preference has an impact on conversion rates). $$ H_1: \text{Conversion} \not\perp \text{Language Preferred} $$

Since we are testing whether there is an association between two categorical variables, this is a Chi-square test for independence.

Step 2: Select Appropriate test

We use the Chi-square test for independence since both conversion status and preferred language are categorical variables.

Step 3: Decide the significance level

alpha = 0.05  # 5% significance level

Significance Level $\alpha = 0.05$

If p-value < 0.05, we reject the null hypothesis.

Step 4: Collect and prepare data

# Create a contingency table (cross-tabulation of conversion and language preferred)
contingency_table = pd.crosstab(data["converted"], data["language_preferred"])

# Display the contingency table
contingency_table

language_preferred	English	French	Spanish
converted
0	11	19	16
1	21	15	18

Step 4a: Check assumptions (Normality & Equal Variances)

Given the nature of the variable and the selected statistical test, additional assumption checks are not required.

Step 5: Calculate the p-value

# Perform the Chi-square test for independence
chi2_stat, p_value, dof, expected = stats.chi2_contingency(contingency_table)

# Display results
print(f"\nChi-square Statistic: {chi2_stat:.4f}")
print(f"P-value: {p_value:.4f}")    

Chi-square Statistic: 3.0930
P-value: 0.2130

Step 6: Compare the p-value with $\alpha$

if p_value < alpha:
    conclusion = "Reject the null hypothesis: Conversion and preferred language are related."
else:
    conclusion = "Fail to reject the null hypothesis: No significant relationship between conversion and preferred language."

Step 7: Draw inference

print(f"\nChi-square Statistic: {chi2_stat:.4f}")
print(f"P-value: {p_value:.4f}")    
print("\nConclusion:", conclusion)  

Chi-square Statistic: 3.0930
P-value: 0.2130

Conclusion: Fail to reject the null hypothesis: No significant relationship between conversion and preferred language.

Observations:

No significant relationship between conversion and preferred language

4. Is the time spent on the new page same for the different language users?

Perform Visual Analysis

# Filter only users who visited the new landing page
df_new_page = data[data["landing_page"] == "new"]

# Create a boxplot to visualize time spent on the new page by preferred language
plt.figure(figsize=(10, 5))
sns.boxplot(x="language_preferred", y="time_spent_on_the_page", hue="language_preferred",
            data=df_new_page, palette="Set2", dodge=False)
plt.title("Time Spent on the New Page by Preferred Language")
plt.xlabel("Preferred Language")
plt.ylabel("Time Spent (seconds)")
plt.legend([],[], frameon=False)  # Remove redundant legend
plt.show()

Step 1: Define the null and alternate hypotheses

• Null Hypothesis (H₀): The average time spent on the new landing page is the same for all language users. $$ H_0: \mu_{\text{English}} = \mu_{\text{Spanish}} = \mu_{\text{French}} $$

• Alternate Hypothesis (H₁): The average time spent on the new landing page is different for at least one language group. $$ H_1: \text{At least one } \mu \text{ differs} $$

Since we are comparing means across more than two independent groups, we use a one-way ANOVA test.

Step 2: Select Appropriate test

We use one-way ANOVA since we are comparing the mean time spent on the new landing page across multiple independent language groups.

Step 3: Decide the significance level

alpha = 0.05  # 5% significance level

Significance Level $\alpha = 0.05$

If p-value < 0.05, we reject the null hypothesis.

Step 4: Collect and prepare data

# Extract time spent for each language group (only for the new landing page)
english_time = df_new_page[df_new_page["language_preferred"] == "English"]["time_spent_on_the_page"]
spanish_time = df_new_page[df_new_page["language_preferred"] == "Spanish"]["time_spent_on_the_page"]
french_time = df_new_page[df_new_page["language_preferred"] == "French"]["time_spent_on_the_page"]

Step 4a: Check assumptions (Normality & Equal Variances)

Given the nature of the variable and the selected statistical test, additional assumption checks are not required.

Step 5: Calculate the p-value

# Perform one-way ANOVA
f_stat, p_value = stats.f_oneway(english_time, spanish_time, french_time)

# Display results
print(f"\nANOVA F-statistic: {f_stat:.4f}")
print(f"P-value: {p_value:.4f}")

ANOVA F-statistic: 0.8544
P-value: 0.4320

Step 6: Compare the p-value with $\alpha$

if p_value < alpha:
    conclusion = "Reject the null hypothesis: The time spent on the new page differs for at least one language group."
else:
    conclusion = "Fail to reject the null hypothesis: No significant difference in time spent among language groups."

Step 7: Draw inference

print(f"\nANOVA F-statistic: {f_stat:.4f}")
print(f"P-value: {p_value:.4f}")
print("\nConclusion:", conclusion)

ANOVA F-statistic: 0.8544
P-value: 0.4320

Conclusion: Fail to reject the null hypothesis: No significant difference in time spent among language groups.

Observations:

No significant difference in time spent among language groups.

Conclusion and Business Recommendations

Conclusion

This analysis applied A/B testing techniques to evaluate whether a new landing page outperforms the existing version in terms of user engagement and conversion rate. Multiple hypothesis tests were conducted, each aligned with the nature of the underlying data and supported by appropriate assumption checks.

For user engagement, measured as time spent on the page, the assumptions required for parametric testing were verified prior to analysis. The results indicated no statistically significant difference between the new and existing landing pages at the 5% significance level. This suggests that any observed variation in engagement is likely attributable to random sampling variability rather than a systematic effect of the new page design.

Similarly, the analysis of conversion rates did not provide sufficient statistical evidence to conclude that the new landing page leads to higher conversions. While minor differences in conversion proportions were observed, these differences were not large enough to reject the null hypothesis under the selected significance threshold.

Overall, the findings indicate that the new landing page does not demonstrate a measurable improvement over the existing page based on the available data. The conclusions drawn are statistically valid within the scope of the current experiment and the assumptions under which the tests were performed.

Business Recommendations

Based on the results of this analysis, the following recommendations are proposed:

1. Maintain the existing landing page, as the current evidence does not support a statistically significant performance improvement from the new design.

2. Refine the new landing page before re-testing, focusing on specific elements such as call-to-action visibility, content hierarchy, or page layout, which may have a more direct influence on user behavior.

3. Increase the duration or sample size of future experiments to improve statistical power and enhance the ability to detect smaller but practically relevant differences.

4. Explore segmented analyses in future tests, such as by user language, device type, or traffic source, as aggregate results may mask meaningful effects within specific subgroups.

5. Incorporate additional behavioral metrics (e.g., bounce rate, scroll depth) to complement time spent and conversion rate and provide a more comprehensive assessment of user engagement.

Executive Summary

This study evaluated the effectiveness of a new landing page relative to the existing version using structured A/B testing methodologies. The analysis focused on key performance indicators related to user engagement and conversion outcomes.

Appropriate statistical tests were selected based on the characteristics of each metric, and underlying assumptions were validated where required. Across all analyses, the results did not reach statistical significance at the 5% level, indicating that the new landing page does not outperform the existing one in a measurable way given the current data.

From a business perspective, the findings suggest that implementing the new landing page would not yield a clear performance advantage at this stage. The results support maintaining the current design while iterating on potential improvements and conducting further experimentation with increased statistical power and more targeted segmentation.

---

!jupyter nbconvert --to html ENews_Express_Learner_Notebook_Full_Code.ipynb

[NbConvertApp] Converting notebook ENews_Express_Learner_Notebook_Full_Code.ipynb to html
[NbConvertApp] WARNING | Alternative text is missing on 18 image(s).
[NbConvertApp] Writing 900509 bytes to ENews_Express_Learner_Notebook_Full_Code.html