├── leab
├── __init__.py
├── before
│ ├── __init__.py
│ ├── leTemplate.html
│ └── leSample.py
├── leDataset
│ ├── data
│ │ ├── evan_miller_ttest_default_1.csv
│ │ ├── evan_miller_ttest_default_2.csv
│ │ ├── evan_miller_chi2_default_1.csv
│ │ ├── evan_miller_chi2_default_2.csv
│ │ ├── evan_miller_chi2_default_3.csv
│ │ ├── evan_miller_chi2_default_4.csv
│ │ ├── evan_miller_chi2_default_5.csv
│ │ └── data_chi2.csv
│ ├── __init__.py
│ ├── sample_le_average.py
│ └── sample_le_success.py
└── after
│ ├── __init__.py
│ ├── leSuccess.py
│ └── leAverage.py
├── docs
├── reference
│ ├── leAverage.md
│ ├── leSample.md
│ └── leSuccess.md
├── _static
│ ├── logo.py
│ └── logo.svg
├── index.md
└── notebooks
│ ├── sample_size.ipynb
│ ├── success_comparaison.ipynb
│ └── average_comparaison.ipynb
├── .github
├── PULL_REQUEST_TEMPLATE.md
├── workflows
│ ├── deploy-docs.yml
│ ├── python-tests.yml
│ └── release.yml
└── ISSUE_TEMPLATE
│ ├── feature_request.md
│ └── bug_report.md
├── mkdocs.yml
├── tests
├── test_leAverage.py
├── test_leSuccess.py
└── test_leSample.py
├── pyproject.toml
├── LICENSE
├── SECURITY.md
├── .gitignore
├── CODE_OF_CONDUCT.md
├── README.md
└── CONTRIBUTING.md
/leab/__init__.py:
--------------------------------------------------------------------------------
1 | __all__ = ["leDataset", "before", "after"]
2 |
--------------------------------------------------------------------------------
/docs/reference/leAverage.md:
--------------------------------------------------------------------------------
1 | ## leAverage.py
2 |
3 | ::: leab.after.leAverage
--------------------------------------------------------------------------------
/docs/reference/leSample.md:
--------------------------------------------------------------------------------
1 | ## leSample.py
2 |
3 | ::: leab.before.leSample
--------------------------------------------------------------------------------
/docs/reference/leSuccess.md:
--------------------------------------------------------------------------------
1 | ## leSuccess.py
2 |
3 | ::: leab.after.leSuccess
--------------------------------------------------------------------------------
/leab/before/__init__.py:
--------------------------------------------------------------------------------
1 | from .leSample import leSample
2 |
3 | __all__ = [
4 | 'leSample'
5 | ]
6 |
7 |
--------------------------------------------------------------------------------
/leab/leDataset/data/evan_miller_ttest_default_1.csv:
--------------------------------------------------------------------------------
1 | 64.2
2 | 28.4
3 | 85.3
4 | 83.1
5 | 13.4
6 | 56.8
7 | 44.2
8 | 90.0
--------------------------------------------------------------------------------
/leab/leDataset/data/evan_miller_ttest_default_2.csv:
--------------------------------------------------------------------------------
1 | 45.0
2 | 29.5
3 | 32.3
4 | 49.3
5 | 18.3
6 | 34.2
7 | 43.9
8 | 13.8
9 | 27.4
10 | 43.4
--------------------------------------------------------------------------------
/leab/after/__init__.py:
--------------------------------------------------------------------------------
1 | from .leSuccess import leSuccess
2 | from .leAverage import leAverage
3 |
4 | __all__ = [
5 | 'leSuccess',
6 | 'leAverage'
7 | ]
8 |
9 |
--------------------------------------------------------------------------------
/leab/leDataset/__init__.py:
--------------------------------------------------------------------------------
1 | from .sample_le_success import SampleLeSuccess
2 | from .sample_le_average import SampleLeAverage
3 |
4 | __all__ = [
5 | 'SampleLeSuccess',
6 | 'SampleLeAverage'
7 | ]
8 |
9 |
--------------------------------------------------------------------------------
/.github/PULL_REQUEST_TEMPLATE.md:
--------------------------------------------------------------------------------
1 | ## Describe your changes
2 |
3 | ## Issue ticket number and link
4 |
5 | ## Checklist before requesting a review
6 |
7 | - [ ] I have performed a self-review of my code
8 | - [ ] If it is a core feature, I have added thorough tests.
9 | - [ ] Do we need to implement analytics?
10 | - [ ] Will this be part of a product update? If yes, please write one phrase about this update.
--------------------------------------------------------------------------------
/docs/_static/logo.py:
--------------------------------------------------------------------------------
1 | from PIL import Image, ImageDraw, ImageFont
2 |
3 | img = Image.new('RGBA',
4 | (840, 450),
5 | color = (255, 255, 255,0))
6 |
7 | fnt = ImageFont.truetype('Le Havre Light.ttf',
8 | 500)
9 | d = ImageDraw.Draw(img)
10 | d.text((0,0),
11 | "le AB",
12 | font=fnt,
13 | fill=(0, 0, 0))
14 |
15 | img.save('logo_le_ab.png')
--------------------------------------------------------------------------------
/mkdocs.yml:
--------------------------------------------------------------------------------
1 | site_name: LeAB Documentation
2 | use_directory_urls: false
3 | site_url: https://tlentali.github.io/leab/
4 | theme:
5 | name: material
6 |
7 | plugins:
8 | - search
9 | - mkdocstrings:
10 | default_handler: python
11 |
12 | nav:
13 | - Home: index.md
14 | - Reference:
15 | - leAverage: reference/leAverage.md
16 | - leSuccess: reference/leSuccess.md
17 | - leSample: reference/leSample.md
--------------------------------------------------------------------------------
/tests/test_leAverage.py:
--------------------------------------------------------------------------------
1 | from leab import after
2 | from leab import leDataset
3 |
4 |
5 | def test_get_size_absolute():
6 | data = leDataset.SampleLeAverage()
7 | ab_test = after.leAverage(data.A, data.B)
8 | assert ab_test.sample_A.confidence_interval == [
9 | 34.75214007684581,
10 | 81.59785992315418,
11 | ]
12 |
13 |
14 | def test_get_size_absolute():
15 | data = leDataset.SampleLeAverage()
16 | ab_test = after.leAverage(data.A, data.B)
17 | assert ab_test.get_verdict() == "Sample A mean is greater"
18 |
--------------------------------------------------------------------------------
/leab/leDataset/sample_le_average.py:
--------------------------------------------------------------------------------
1 | import pandas as pd
2 | from pathlib import Path
3 |
4 |
5 | class SampleLeAverage:
6 | """
7 | Sample data to run with leAverage, from E. Miller default example in app.
8 | """
9 | def __init__(self):
10 |
11 | root = Path(__file__).parent
12 | file_1 = root / 'data' / 'evan_miller_ttest_default_1.csv'
13 | file_2 = root / 'data' / 'evan_miller_ttest_default_2.csv'
14 |
15 | self.A = pd.read_csv(
16 | file_1, names=["values"]
17 | )
18 | self.B = pd.read_csv(
19 | file_2, names=["values"]
20 | )
21 |
--------------------------------------------------------------------------------
/.github/workflows/deploy-docs.yml:
--------------------------------------------------------------------------------
1 | name: Deploy MkDocs to GitHub Pages
2 |
3 | on:
4 | push:
5 | branches:
6 | - master
7 |
8 | permissions:
9 | contents: write
10 |
11 | jobs:
12 | deploy:
13 | runs-on: ubuntu-latest
14 |
15 | steps:
16 | - name: Checkout code
17 | uses: actions/checkout@v4
18 |
19 | - name: Set up Python
20 | uses: actions/setup-python@v5
21 | with:
22 | python-version: '3.11'
23 |
24 | - name: Install dependencies
25 | run: |
26 | pip install mkdocs mkdocs-material mkdocstrings[python]
27 |
28 | - name: Deploy to GitHub Pages
29 | run: |
30 | mkdocs gh-deploy --force
31 |
--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/feature_request.md:
--------------------------------------------------------------------------------
1 | ---
2 | name: Feature request
3 | about: Suggest an idea for this project
4 | title: ''
5 | labels: ''
6 | assignees: ''
7 |
8 | ---
9 |
10 | **Is your feature request related to a problem? Please describe.**
11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
12 |
13 | **Describe the solution you'd like**
14 | A clear and concise description of what you want to happen.
15 |
16 | **Describe alternatives you've considered**
17 | A clear and concise description of any alternative solutions or features you've considered.
18 |
19 | **Additional context**
20 | Add any other context or screenshots about the feature request here.
21 |
--------------------------------------------------------------------------------
/.github/workflows/python-tests.yml:
--------------------------------------------------------------------------------
1 | name: Python Tests
2 |
3 | on:
4 | push:
5 | branches: [ master ]
6 | pull_request:
7 | branches: [ master ]
8 |
9 | jobs:
10 | test:
11 | runs-on: ubuntu-latest
12 |
13 | steps:
14 | - name: Checkout code
15 | uses: actions/checkout@v3
16 |
17 | - name: Install uv
18 | run: |
19 | curl -Ls https://astral.sh/uv/install.sh | bash
20 | echo "$HOME/.cargo/bin" >> $GITHUB_PATH
21 |
22 | - name: Create the virtual environnement using uv
23 | run: uv venv
24 |
25 | - name: Activate the venv and install dépendancies
26 | run: |
27 | source .venv/bin/activate
28 | uv sync
29 |
30 | - name: Launch tests
31 | run: |
32 | source .venv/bin/activate
33 | pytest
--------------------------------------------------------------------------------
/tests/test_leSuccess.py:
--------------------------------------------------------------------------------
1 | from leab import after
2 | from leab import leDataset
3 |
4 |
5 | def test_get_confidence_interval():
6 | data = leDataset.SampleLeSuccess()
7 | ab_test = after.leSuccess(data.A, data.B, confidence_level=0.95)
8 | assert ab_test.sample_A.confidence_interval == [
9 | 8.526343659939133,
10 | 22.13718821096384,
11 | ]
12 |
13 |
14 | def test_get_p_value():
15 | data = leDataset.SampleLeSuccess()
16 | ab_test = after.leSuccess(data.A, data.B, confidence_level=0.95)
17 | assert ab_test.p_value == 0.25870176105718934
18 |
19 |
20 | def test_get_verdict():
21 | data = leDataset.SampleLeSuccess()
22 | ab_test = after.leSuccess(data.A, data.B, confidence_level=0.95)
23 | assert ab_test.get_verdict() == "No significant difference"
24 |
--------------------------------------------------------------------------------
/leab/leDataset/data/evan_miller_chi2_default_1.csv:
--------------------------------------------------------------------------------
1 | 1
2 | 0
3 | 1
4 | 1
5 | 0
6 | 0
7 | 1
8 | 1
9 | 0
10 | 0
11 | 0
12 | 1
13 | 1
14 | 1
15 | 0
16 | 0
17 | 1
18 | 0
19 | 0
20 | 0
21 | 0
22 | 0
23 | 0
24 | 1
25 | 1
26 | 1
27 | 1
28 | 0
29 | 0
30 | 0
31 | 0
32 | 0
33 | 0
34 | 0
35 | 0
36 | 0
37 | 0
38 | 0
39 | 0
40 | 0
41 | 0
42 | 0
43 | 0
44 | 0
45 | 0
46 | 0
47 | 0
48 | 0
49 | 0
50 | 0
51 | 0
52 | 0
53 | 0
54 | 0
55 | 0
56 | 0
57 | 0
58 | 0
59 | 0
60 | 0
61 | 0
62 | 0
63 | 0
64 | 0
65 | 0
66 | 0
67 | 0
68 | 0
69 | 0
70 | 1
71 | 0
72 | 0
73 | 0
74 | 0
75 | 0
76 | 0
77 | 0
78 | 0
79 | 0
80 | 0
81 | 0
82 | 0
83 | 0
84 | 0
85 | 0
86 | 0
87 | 0
88 | 0
89 | 0
90 | 0
91 | 0
92 | 0
93 | 0
94 | 0
95 | 0
96 | 0
97 | 0
98 | 0
99 | 0
100 | 0
--------------------------------------------------------------------------------
/leab/leDataset/data/evan_miller_chi2_default_2.csv:
--------------------------------------------------------------------------------
1 | 1
2 | 0
3 | 0
4 | 1
5 | 0
6 | 0
7 | 0
8 | 1
9 | 0
10 | 0
11 | 0
12 | 0
13 | 1
14 | 0
15 | 0
16 | 0
17 | 0
18 | 0
19 | 0
20 | 1
21 | 0
22 | 0
23 | 0
24 | 0
25 | 0
26 | 0
27 | 0
28 | 0
29 | 0
30 | 0
31 | 1
32 | 0
33 | 0
34 | 1
35 | 0
36 | 1
37 | 0
38 | 0
39 | 0
40 | 1
41 | 0
42 | 0
43 | 1
44 | 1
45 | 0
46 | 1
47 | 0
48 | 0
49 | 1
50 | 1
51 | 1
52 | 0
53 | 0
54 | 1
55 | 0
56 | 0
57 | 0
58 | 0
59 | 1
60 | 0
61 | 0
62 | 1
63 | 0
64 | 0
65 | 0
66 | 1
67 | 1
68 | 0
69 | 0
70 | 0
71 | 0
72 | 0
73 | 0
74 | 0
75 | 0
76 | 0
77 | 0
78 | 0
79 | 0
80 | 0
81 | 0
82 | 0
83 | 0
84 | 0
85 | 0
86 | 0
87 | 0
88 | 0
89 | 0
90 | 0
91 | 0
92 | 0
93 | 0
94 | 0
95 | 0
96 | 0
97 | 0
98 | 0
99 | 0
100 | 0
--------------------------------------------------------------------------------
/leab/leDataset/data/evan_miller_chi2_default_3.csv:
--------------------------------------------------------------------------------
1 | 0
2 | 0
3 | 0
4 | 0
5 | 0
6 | 0
7 | 0
8 | 0
9 | 0
10 | 0
11 | 0
12 | 1
13 | 0
14 | 0
15 | 0
16 | 0
17 | 1
18 | 0
19 | 0
20 | 0
21 | 0
22 | 0
23 | 0
24 | 0
25 | 0
26 | 0
27 | 0
28 | 0
29 | 0
30 | 0
31 | 0
32 | 0
33 | 0
34 | 0
35 | 0
36 | 0
37 | 0
38 | 0
39 | 0
40 | 0
41 | 0
42 | 0
43 | 0
44 | 0
45 | 0
46 | 0
47 | 0
48 | 0
49 | 0
50 | 0
51 | 0
52 | 0
53 | 0
54 | 0
55 | 0
56 | 0
57 | 0
58 | 0
59 | 1
60 | 0
61 | 0
62 | 1
63 | 0
64 | 0
65 | 1
66 | 0
67 | 0
68 | 0
69 | 0
70 | 0
71 | 0
72 | 0
73 | 1
74 | 0
75 | 0
76 | 0
77 | 0
78 | 0
79 | 1
80 | 0
81 | 0
82 | 0
83 | 0
84 | 1
85 | 0
86 | 0
87 | 0
88 | 1
89 | 0
90 | 0
91 | 0
92 | 0
93 | 1
94 | 0
95 | 0
96 | 0
97 | 0
98 | 0
99 | 0
100 | 0
--------------------------------------------------------------------------------
/leab/leDataset/data/evan_miller_chi2_default_4.csv:
--------------------------------------------------------------------------------
1 | 0
2 | 0
3 | 0
4 | 0
5 | 0
6 | 0
7 | 0
8 | 0
9 | 0
10 | 0
11 | 0
12 | 0
13 | 0
14 | 0
15 | 0
16 | 0
17 | 0
18 | 0
19 | 0
20 | 0
21 | 0
22 | 0
23 | 0
24 | 0
25 | 0
26 | 0
27 | 0
28 | 0
29 | 0
30 | 0
31 | 0
32 | 0
33 | 0
34 | 0
35 | 0
36 | 0
37 | 0
38 | 0
39 | 0
40 | 0
41 | 0
42 | 0
43 | 0
44 | 0
45 | 0
46 | 0
47 | 0
48 | 0
49 | 0
50 | 0
51 | 0
52 | 0
53 | 0
54 | 0
55 | 0
56 | 0
57 | 0
58 | 0
59 | 0
60 | 0
61 | 0
62 | 0
63 | 0
64 | 0
65 | 0
66 | 0
67 | 0
68 | 0
69 | 0
70 | 0
71 | 0
72 | 0
73 | 0
74 | 0
75 | 0
76 | 0
77 | 0
78 | 0
79 | 0
80 | 0
81 | 0
82 | 0
83 | 1
84 | 0
85 | 0
86 | 0
87 | 1
88 | 0
89 | 0
90 | 0
91 | 0
92 | 0
93 | 0
94 | 0
95 | 0
96 | 0
97 | 0
98 | 0
99 | 0
100 | 0
--------------------------------------------------------------------------------
/leab/leDataset/data/evan_miller_chi2_default_5.csv:
--------------------------------------------------------------------------------
1 | 1
2 | 1
3 | 1
4 | 1
5 | 1
6 | 1
7 | 1
8 | 1
9 | 1
10 | 1
11 | 1
12 | 1
13 | 1
14 | 1
15 | 1
16 | 1
17 | 1
18 | 1
19 | 1
20 | 1
21 | 1
22 | 1
23 | 1
24 | 1
25 | 1
26 | 1
27 | 1
28 | 1
29 | 1
30 | 1
31 | 1
32 | 1
33 | 1
34 | 1
35 | 1
36 | 1
37 | 1
38 | 1
39 | 1
40 | 1
41 | 1
42 | 1
43 | 1
44 | 1
45 | 1
46 | 1
47 | 1
48 | 1
49 | 1
50 | 1
51 | 1
52 | 1
53 | 1
54 | 1
55 | 1
56 | 1
57 | 1
58 | 1
59 | 1
60 | 1
61 | 1
62 | 1
63 | 1
64 | 1
65 | 1
66 | 1
67 | 1
68 | 1
69 | 1
70 | 1
71 | 1
72 | 1
73 | 1
74 | 1
75 | 1
76 | 1
77 | 1
78 | 1
79 | 1
80 | 1
81 | 1
82 | 1
83 | 1
84 | 1
85 | 1
86 | 1
87 | 1
88 | 1
89 | 1
90 | 1
91 | 0
92 | 0
93 | 0
94 | 0
95 | 0
96 | 0
97 | 0
98 | 0
99 | 0
100 | 0
--------------------------------------------------------------------------------
/.github/ISSUE_TEMPLATE/bug_report.md:
--------------------------------------------------------------------------------
1 | ---
2 | name: Bug report
3 | about: Create a report to help us improve
4 | title: ''
5 | labels: ''
6 | assignees: ''
7 |
8 | ---
9 |
10 | **Describe the bug**
11 | A clear and concise description of what the bug is.
12 |
13 | **To Reproduce**
14 | Steps to reproduce the behavior:
15 | 1. Go to '...'
16 | 2. Click on '....'
17 | 3. Scroll down to '....'
18 | 4. See error
19 |
20 | **Expected behavior**
21 | A clear and concise description of what you expected to happen.
22 |
23 | **Screenshots**
24 | If applicable, add screenshots to help explain your problem.
25 |
26 | **Desktop (please complete the following information):**
27 | - OS: [e.g. iOS]
28 | - Browser [e.g. chrome, safari]
29 | - Version [e.g. 22]
30 |
31 | **Smartphone (please complete the following information):**
32 | - Device: [e.g. iPhone6]
33 | - OS: [e.g. iOS8.1]
34 | - Browser [e.g. stock browser, safari]
35 | - Version [e.g. 22]
36 |
37 | **Additional context**
38 | Add any other context about the problem here.
39 |
--------------------------------------------------------------------------------
/leab/leDataset/sample_le_success.py:
--------------------------------------------------------------------------------
1 | import pandas as pd
2 | from pathlib import Path
3 |
4 |
5 | class SampleLeSuccess:
6 | """
7 | Sample data to run with le Mean, from E. Miller default example in app.
8 | """
9 | def __init__(self):
10 |
11 | root = Path(__file__).parent
12 | file_A = root / 'data' / 'evan_miller_chi2_default_1.csv'
13 | file_B = root / 'data' / 'evan_miller_chi2_default_2.csv'
14 | file_C = root / 'data' / 'evan_miller_chi2_default_3.csv'
15 | file_D = root / 'data' / 'evan_miller_chi2_default_4.csv'
16 | file_E = root / 'data' / 'evan_miller_chi2_default_5.csv'
17 |
18 | self.A = pd.read_csv(
19 | file_A, names=["success"]
20 | )
21 | self.B = pd.read_csv(
22 | file_B, names=["success"]
23 | )
24 | self.C = pd.read_csv(
25 | file_C, names=["success"]
26 | )
27 | self.D = pd.read_csv(
28 | file_D, names=["success"]
29 | )
30 | self.E = pd.read_csv(
31 | file_E, names=["success"]
32 | )
--------------------------------------------------------------------------------
/tests/test_leSample.py:
--------------------------------------------------------------------------------
1 | from leab import before
2 |
3 |
4 | def test_get_size_per_variation_absolute():
5 | ab_test = before.leSample(conversion_rate=20, min_detectable_effect=2)
6 | assert ab_test.get_size_per_variation() == 6347
7 |
8 |
9 | def test_get_size_per_variation_relative():
10 | ab_test = before.leSample(
11 | conversion_rate=20, min_detectable_effect=2, absolute=False
12 | )
13 | assert ab_test.get_size_per_variation() == 157328
14 |
15 |
16 | def test_get_total_size_absolute():
17 | ab_test = before.leSample(conversion_rate=20, min_detectable_effect=2)
18 | assert ab_test.get_total_size() == 12694
19 |
20 |
21 | def test_get_total_size_relative():
22 | ab_test = before.leSample(
23 | conversion_rate=20, min_detectable_effect=2, absolute=False
24 | )
25 | assert ab_test.get_total_size() == 314656
26 |
27 |
28 | def test_get_duration_absolute():
29 | ab_test = before.leSample(conversion_rate=20, min_detectable_effect=2)
30 | assert ab_test.get_duration(avg_daily_total_visitor=1000) == 13
31 |
32 |
33 | def test_get_duration_relative():
34 | ab_test = before.leSample(
35 | conversion_rate=20, min_detectable_effect=2, absolute=False
36 | )
37 | assert ab_test.get_duration(avg_daily_total_visitor=1000) == 315
38 |
--------------------------------------------------------------------------------
/pyproject.toml:
--------------------------------------------------------------------------------
1 | [project]
2 | name = "leab"
3 | version = "0.1.11"
4 | description = "Add your description here"
5 | readme = "README.md"
6 | requires-python = ">=3.12"
7 | dependencies = [
8 | "numpy>=2.2.4",
9 | "pandas>=2.2.3",
10 | "scipy>=1.15.2",
11 | "statsmodels>=0.14.4",
12 | ]
13 |
14 | [project.urls]
15 | Repository = "https://github.com/tlentali/leab"
16 | Documentation = "https://tlentali.github.io/leab/"
17 |
18 | [dependency-groups]
19 | dev = [
20 | "black>=25.1.0",
21 | "jupyter>=1.1.1",
22 | "mkdocs>=1.6.1",
23 | "mkdocs-material>=9.6.9",
24 | "mkdocstrings[python]>=0.29.0",
25 | "pylint>=3.3.6",
26 | "pytest>=8.3.5",
27 | ]
28 |
29 | [tool.pytest.ini_options]
30 | pythonpath = [
31 | "."
32 | ]
33 |
34 | [build-system]
35 | requires = ["setuptools>=61.0"]
36 | build-backend = "setuptools.build_meta"
37 |
38 | [tool.setuptools.packages.find]
39 | where = ["."]
40 | include = ["leab"]
41 | exclude = ["tests*", "notebook*", "misc*"]
42 |
43 | [tool.bumpver]
44 | current_version = "0.1.11"
45 | version_pattern = "MAJOR.MINOR.PATCH"
46 | commit_message = "bump: {old_version} → {new_version}"
47 | tag_message = "v{new_version}"
48 |
49 | [tool.bumpver.file_patterns]
50 | "pyproject.toml" = [
51 | 'current_version = "{version}"',
52 | 'version = "{version}"'
53 | ]
54 |
55 | [tool.uv]
56 | required-version = ">=0.6.11,<0.7"
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | BSD 3-Clause License
2 |
3 | Copyright (c) 2025, tlentali
4 | All rights reserved.
5 |
6 | Redistribution and use in source and binary forms, with or without
7 | modification, are permitted provided that the following conditions are met:
8 |
9 | 1. Redistributions of source code must retain the above copyright notice, this
10 | list of conditions and the following disclaimer.
11 |
12 | 2. Redistributions in binary form must reproduce the above copyright notice,
13 | this list of conditions and the following disclaimer in the documentation
14 | and/or other materials provided with the distribution.
15 |
16 | 3. Neither the name of the copyright holder nor the names of its
17 | contributors may be used to endorse or promote products derived from
18 | this software without specific prior written permission.
19 |
20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
--------------------------------------------------------------------------------
/docs/index.md:
--------------------------------------------------------------------------------
1 | 
2 |
3 | ---
4 |
5 | `LeAB` is a Python library for AB testing analysis.
6 |
7 | Get ready to take a decision !
8 |
9 | `LeAB` is a Python package designed to streamline A/B test analysis, from raw data to insightful results. It provides a clean API to define experiments, compute metrics, and interpret statistical outcomes with confidence.
10 | AB testing has never been more popular, especially on Internet based companies.
11 | Even if each test is unique, some questions seem to be asked again and again :
12 |
13 | - when is my test going to be statistically significant ?
14 | - is A more successful than B ?
15 | - does A generate more than B ?
16 |
17 | Strong statistical knowledge are required to handle it from start to end correctly.
18 | To answer those questions in a simple and robust way, we built **le AB**.
19 | Lets Python do AB testing analysis !
20 |
21 | ## 🛠 Installation
22 |
23 | `LeAB` is intended to work with Python 3.12 or above.
24 |
25 | Installation can be done by using `pip` :
26 |
27 | ```bash
28 | pip install leab
29 | ```
30 |
31 | There are [wheels](https://pypi.org/project/leab/#files) available for Linux, MacOS, and Windows.
32 |
33 | ## ✨ Why LEAB?
34 |
35 | A/B testing is a powerful tool, but often requires stitching together multiple tools and manual steps. LEAB aims to:
36 |
37 | - 📊 Simplify and standardize A/B test workflows
38 | - 🔍 Ensure statistical rigor with built-in checks
39 | - 🧪 Support simulations and power analysis
40 |
41 | Let the data speak—with confidence.
42 |
43 | Made with 💙 from Bordeaux 🍷 and Montréal 🍁.
44 |
--------------------------------------------------------------------------------
/.github/workflows/release.yml:
--------------------------------------------------------------------------------
1 | name: Publish to PyPI
2 |
3 | on:
4 | push:
5 | branches:
6 | - master
7 |
8 | jobs:
9 | release:
10 | runs-on: ubuntu-latest
11 | steps:
12 | - name: Checkout code
13 | uses: actions/checkout@v4
14 |
15 | - name: Set up Python
16 | uses: actions/setup-python@v5
17 | with:
18 | python-version: "3.12"
19 |
20 | - name: Install tools
21 | run: |
22 | python -m pip install --upgrade pip
23 | pip install bumpver build twine
24 |
25 | - name: Determine version bump type
26 | id: version
27 | run: |
28 | git fetch --tags
29 | LAST_COMMIT_MESSAGE=$(git log -1 --pretty=%B)
30 | echo "Last commit: $LAST_COMMIT_MESSAGE"
31 |
32 | if [[ "$LAST_COMMIT_MESSAGE" == *#major* ]]; then
33 | echo "bump=major" >> $GITHUB_OUTPUT
34 | elif [[ "$LAST_COMMIT_MESSAGE" == *#minor* ]]; then
35 | echo "bump=minor" >> $GITHUB_OUTPUT
36 | else
37 | echo "bump=patch" >> $GITHUB_OUTPUT
38 | fi
39 |
40 | - name: Bump version
41 | run: |
42 | bumpver update --${{ steps.version.outputs.bump }}
43 |
44 | - name: Commit bumped version
45 | run: |
46 | git config user.name "GitHub Actions"
47 | git config user.email "actions@github.com"
48 | git add .
49 | git commit -m "chore: bump version [skip ci]"
50 | git push
51 |
52 | - name: Build and publish
53 | env:
54 | TWINE_USERNAME: __token__
55 | TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
56 | run: |
57 | python -m build
58 | twine upload dist/*
59 |
--------------------------------------------------------------------------------
/SECURITY.md:
--------------------------------------------------------------------------------
1 | # `leAB` Open Source Security Policies and Procedures
2 |
3 | This document outlines security procedures and general policies for the `leAB` Open Source projects as found on [https://github.com/tlentali/leab](https://github.com/tlentali/leab).
4 |
5 | * [Reporting a Vulnerability](#reporting-a-vulnerability)
6 | * [Disclosure Policy](#disclosure-policy)
7 |
8 | ## Reporting a Vulnerability
9 |
10 | The `leAB` team and community take all security vulnerabilities
11 | seriously. Thank you for improving the security of our open source
12 | software. We appreciate your efforts and responsible disclosure and will
13 | make every effort to acknowledge your contributions.
14 |
15 | Report security vulnerabilities by emailing the `leAB` security team at thomas.lentali@gmail.com
16 |
17 | The lead maintainer will acknowledge your email within 24 hours, and will send a more detailed response within 48 hours indicating the next steps in handling your report.
18 | After the initial reply to your report, the security team will endeavor to keep you informed of the progress towards a fix and full announcement, and may ask for additional information or guidance.
19 |
20 | Report security vulnerabilities in third-party modules to the person or team maintaining the module.
21 |
22 | ## Disclosure Policy
23 |
24 | When the security team receives a security bug report, they will assign it to a primary handler.
25 | This person will coordinate the fix and release process, involving the following steps:
26 |
27 | - Confirm the problem and determine the affected versions.
28 | - Audit code to find any potential similar problems.
29 | - Prepare fixes for all releases still under maintenance.
30 |
31 | These fixes will be released as fast as possible to `Pypi`.
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | # Byte-compiled / optimized / DLL files
2 | __pycache__/
3 | *.py[cod]
4 | *$py.class
5 |
6 | # C extensions
7 | *.so
8 |
9 | # Distribution / packaging
10 | .Python
11 | build/
12 | develop-eggs/
13 | dist/
14 | downloads/
15 | eggs/
16 | .eggs/
17 | lib/
18 | lib64/
19 | parts/
20 | sdist/
21 | var/
22 | wheels/
23 | pip-wheel-metadata/
24 | share/python-wheels/
25 | *.egg-info/
26 | .installed.cfg
27 | *.egg
28 | MANIFEST
29 |
30 | # PyInstaller
31 | # Usually these files are written by a python script from a template
32 | # before PyInstaller builds the exe, so as to inject date/other infos into it.
33 | *.manifest
34 | *.spec
35 |
36 | # Installer logs
37 | pip-log.txt
38 | pip-delete-this-directory.txt
39 |
40 | # Unit test / coverage reports
41 | htmlcov/
42 | .tox/
43 | .nox/
44 | .coverage
45 | .coverage.*
46 | .cache
47 | nosetests.xml
48 | coverage.xml
49 | *.cover
50 | *.py,cover
51 | .hypothesis/
52 | .pytest_cache/
53 | cover/
54 |
55 | # Translations
56 | *.mo
57 | *.pot
58 |
59 | # Django stuff:
60 | *.log
61 | local_settings.py
62 | db.sqlite3
63 | db.sqlite3-journal
64 |
65 | # Flask stuff:
66 | instance/
67 | .webassets-cache
68 |
69 | # Scrapy stuff:
70 | .scrapy
71 |
72 | # Sphinx documentation
73 | docs/_build/
74 |
75 | # PyBuilder
76 | .pybuilder/
77 | target/
78 |
79 | # Jupyter Notebook
80 | .ipynb_checkpoints
81 |
82 | # IPython
83 | profile_default/
84 | ipython_config.py
85 |
86 | # pyenv
87 | # For a library or package, you might want to ignore these files since the code is
88 | # intended to run in multiple environments; otherwise, check them in:
89 | # .python-version
90 |
91 | # pipenv
92 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
93 | # However, in case of collaboration, if having platform-specific dependencies or dependencies
94 | # having no cross-platform support, pipenv may install dependencies that don't work, or not
95 | # install all needed dependencies.
96 | #Pipfile.lock
97 |
98 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow
99 | __pypackages__/
100 |
101 | # Celery stuff
102 | celerybeat-schedule
103 | celerybeat.pid
104 |
105 | # SageMath parsed files
106 | *.sage.py
107 |
108 | # Environments
109 | .env
110 | .venv
111 | env/
112 | venv/
113 | ENV/
114 | env.bak/
115 | venv.bak/
116 |
117 | # Spyder project settings
118 | .spyderproject
119 | .spyproject
120 |
121 | # Rope project settings
122 | .ropeproject
123 |
124 | # mkdocs documentation
125 | /site
126 |
127 | # mypy
128 | .mypy_cache/
129 | .dmypy.json
130 | dmypy.json
131 |
132 | # Pyre type checker
133 | .pyre/
134 |
135 | # pytype static type analyzer
136 | .pytype/
137 |
138 | # Cython debug symbols
139 | cython_debug/
140 |
141 | # static files generated from Django application using `collectstatic`
142 | media
143 | static
144 |
145 | # docs
146 | =0.5.0
147 | =1.8.3
148 | make.bat
149 |
150 | # vscode
151 | .vscode
152 |
153 | # uv
154 | .python-version
155 | *.lock
156 |
--------------------------------------------------------------------------------
/leab/before/leTemplate.html:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | {{ title }}
6 |
7 |
85 |
86 |
87 |
88 |
89 |
90 |
{{ title }}
91 |
92 |
93 |
94 |
100 |
101 |
102 |
Baseline conversion rate and Minimum Detectable Effect :
103 |
104 | {{ html_plot_baseline_conversion_rate }}
105 |
106 |
107 |
108 |
109 |
112 |
113 |
114 |
115 |
116 |
--------------------------------------------------------------------------------
/docs/notebooks/sample_size.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# Le Sample\n",
8 | "\n",
9 | "This tuto shows you how to get the number of sample needed per variation to reach a statistical significant result. \n",
10 | "First, let's import the right tools :"
11 | ]
12 | },
13 | {
14 | "cell_type": "code",
15 | "execution_count": null,
16 | "metadata": {},
17 | "outputs": [],
18 | "source": [
19 | "from leab import before"
20 | ]
21 | },
22 | {
23 | "cell_type": "markdown",
24 | "metadata": {},
25 | "source": [
26 | "As we want to know the sample needed, we stand **before** launching the AB test. \n",
27 | "Then, from the `before` import, we create a `leSample` instance that we call `ab_test`. \n",
28 | "In this example we want to get a volume of sample per variation needed if we expect a minimum detectable effect of 2% on a baseline conversion rate of 20% :"
29 | ]
30 | },
31 | {
32 | "cell_type": "code",
33 | "execution_count": null,
34 | "metadata": {},
35 | "outputs": [
36 | {
37 | "data": {
38 | "text/plain": [
39 | "6347"
40 | ]
41 | },
42 | "execution_count": 2,
43 | "metadata": {},
44 | "output_type": "execute_result"
45 | }
46 | ],
47 | "source": [
48 | "ab_test = before.leSample(conversion_rate=20,\n",
49 | " min_detectable_effect=2)\n",
50 | "ab_test.get_size_per_variation()"
51 | ]
52 | },
53 | {
54 | "cell_type": "markdown",
55 | "metadata": {},
56 | "source": [
57 | "So the result is `6347` visits per variation (`6347` for the A variation and `6347` more for the B variation). \n",
58 | "If we know the trafic expected on this test, we can ends up with a approximate number of days needed to run the test :"
59 | ]
60 | },
61 | {
62 | "cell_type": "code",
63 | "execution_count": 3,
64 | "metadata": {},
65 | "outputs": [
66 | {
67 | "data": {
68 | "text/plain": [
69 | "13"
70 | ]
71 | },
72 | "execution_count": 3,
73 | "metadata": {},
74 | "output_type": "execute_result"
75 | }
76 | ],
77 | "source": [
78 | "ab_test.get_duration(avg_daily_total_visitor=1000)"
79 | ]
80 | },
81 | {
82 | "cell_type": "markdown",
83 | "metadata": {},
84 | "source": [
85 | "You will need `13` days to run the test. "
86 | ]
87 | }
88 | ],
89 | "metadata": {
90 | "file_extension": ".py",
91 | "kernelspec": {
92 | "display_name": "Python 3",
93 | "language": "python",
94 | "name": "python3"
95 | },
96 | "language_info": {
97 | "codemirror_mode": {
98 | "name": "ipython",
99 | "version": 3
100 | },
101 | "file_extension": ".py",
102 | "mimetype": "text/x-python",
103 | "name": "python",
104 | "nbconvert_exporter": "python",
105 | "pygments_lexer": "ipython3",
106 | "version": "3.6.9"
107 | },
108 | "mimetype": "text/x-python",
109 | "name": "python",
110 | "npconvert_exporter": "python",
111 | "pygments_lexer": "ipython3",
112 | "version": 3
113 | },
114 | "nbformat": 4,
115 | "nbformat_minor": 4
116 | }
117 |
--------------------------------------------------------------------------------
/docs/notebooks/success_comparaison.ipynb:
--------------------------------------------------------------------------------
1 | {"cells":[{"cell_type":"markdown","metadata":{},"source":["# Le Success\n","\n","Here we will compare success reached in A versus B. \n","First, lets import some data from `leDataset` :"]},{"cell_type":"code","execution_count":2,"metadata":{},"outputs":[],"source":"from leab import leDataset\n\ndata = leDataset.SampleLeSuccess()"},{"cell_type":"markdown","metadata":{},"source":["Then we get `data.A` and `data.B`, where **1** means *success* and **0** means *fail* :"]},{"cell_type":"code","execution_count":3,"metadata":{},"outputs":[{"data":{"text/html":["
\n","\n","
\n"," \n","
\n","
\n","
success
\n","
\n"," \n"," \n","
\n","
0
\n","
1
\n","
\n","
\n","
1
\n","
0
\n","
\n","
\n","
2
\n","
1
\n","
\n","
\n","
3
\n","
1
\n","
\n","
\n","
4
\n","
0
\n","
\n"," \n","
\n","
"],"text/plain":[" success\n","0 1\n","1 0\n","2 1\n","3 1\n","4 0"]},"execution_count":3,"metadata":{},"output_type":"execute_result"}],"source":"data.A.head()"},{"cell_type":"markdown","metadata":{},"source":["Now, we call the `after` tool from `leab`, because, as we are comparing results, we assume that it appends after launching the AB test."]},{"cell_type":"code","execution_count":4,"metadata":{},"outputs":[],"source":"from leab import after"},{"cell_type":"markdown","metadata":{},"source":["We build an instance `ab_test` including our data for a confidence level fixed at 95% :"]},{"cell_type":"code","execution_count":5,"metadata":{},"outputs":[],"source":"ab_test = after.leSuccess(data.A, data.B, confidence_level=0.95)"},{"cell_type":"markdown","metadata":{},"source":["We have acces at some complementary infos like the *confidence interval* of a sample or the *p value* obtained :"]},{"cell_type":"code","execution_count":6,"metadata":{},"outputs":[{"data":{"text/plain":["[8.526343659939133, 22.13718821096384]"]},"execution_count":6,"metadata":{},"output_type":"execute_result"}],"source":"ab_test.sample_A.confidence_interval"},{"cell_type":"code","execution_count":7,"metadata":{},"outputs":[{"data":{"text/plain":["0.25870176105718934"]},"execution_count":7,"metadata":{},"output_type":"execute_result"}],"source":"ab_test.p_value"},{"cell_type":"markdown","metadata":{},"source":["To conclude on the test, we call the `get_verdict()` methode :"]},{"cell_type":"code","execution_count":8,"metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["No significant difference\n"]}],"source":"ab_test.get_verdict()"},{"cell_type":"markdown","metadata":{},"source":["So, at 95% of confidence level, A and B variations are quite as succeful."]}],"nbformat":4,"nbformat_minor":2,"metadata":{"language_info":{"name":"python","codemirror_mode":{"name":"ipython","version":3}},"orig_nbformat":2,"file_extension":".py","mimetype":"text/x-python","name":"python","npconvert_exporter":"python","pygments_lexer":"ipython3","version":3}}
--------------------------------------------------------------------------------
/CODE_OF_CONDUCT.md:
--------------------------------------------------------------------------------
1 | # Code of Conduct
2 |
3 | ## Our Pledge
4 |
5 | In the interest of fostering an open and welcoming environment, we as
6 | contributors and maintainers pledge to making participation in our project and
7 | our community a harassment-free experience for everyone, regardless of age, body
8 | size, disability, ethnicity, gender identity and expression, level of experience,
9 | nationality, personal appearance, race, religion, or sexual identity and
10 | orientation.
11 |
12 | ## Our Standards
13 |
14 | Examples of behavior that contributes to creating a positive environment
15 | include:
16 |
17 | * Using welcoming and inclusive language
18 | * Being respectful of differing viewpoints and experiences
19 | * Gracefully accepting constructive criticism
20 | * Focusing on what is best for the community
21 | * Showing empathy towards other community members
22 |
23 | Examples of unacceptable behavior by participants include:
24 |
25 | * The use of sexualized language or imagery and unwelcome sexual attention or
26 | advances
27 | * Trolling, insulting/derogatory comments, and personal or political attacks
28 | * Public or private harassment
29 | * Publishing others' private information, such as a physical or electronic
30 | address, without explicit permission
31 | * Other conduct which could reasonably be considered inappropriate in a
32 | professional setting
33 |
34 | ## Our Responsibilities
35 |
36 | Project maintainers are responsible for clarifying the standards of acceptable
37 | behavior and are expected to take appropriate and fair corrective action in
38 | response to any instances of unacceptable behavior.
39 |
40 | Project maintainers have the right and responsibility to remove, edit, or
41 | reject comments, commits, code, wiki edits, issues, and other contributions
42 | that are not aligned to this Code of Conduct, or to ban temporarily or
43 | permanently any contributor for other behaviors that they deem inappropriate,
44 | threatening, offensive, or harmful.
45 |
46 | ## Scope
47 |
48 | This Code of Conduct applies both within project spaces and in public spaces
49 | when an individual is representing the project or its community. Examples of
50 | representing a project or community include using an official project e-mail
51 | address, posting via an official social media account, or acting as an appointed
52 | representative at an online or offline event. Representation of a project may be
53 | further defined and clarified by project maintainers.
54 |
55 | ## Enforcement
56 |
57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be
58 | reported by contacting the project team at **thomas.lentali@gmail.com**. All
59 | complaints will be reviewed and investigated and will result in a response that
60 | is deemed necessary and appropriate to the circumstances. The project team is
61 | obligated to maintain confidentiality with regard to the reporter of an incident.
62 | Further details of specific enforcement policies may be posted separately.
63 |
64 | Project maintainers who do not follow or enforce the Code of Conduct in good
65 | faith may face temporary or permanent repercussions as determined by other
66 | members of the project's leadership.
67 |
68 | ## Attribution
69 |
70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
71 | available at [http://contributor-covenant.org/version/1/4][version]
72 |
73 | [homepage]: http://contributor-covenant.org
74 | [version]: http://contributor-covenant.org/version/1/4/
--------------------------------------------------------------------------------
/docs/_static/logo.svg:
--------------------------------------------------------------------------------
1 |
2 |
54 |
--------------------------------------------------------------------------------
/leab/after/leSuccess.py:
--------------------------------------------------------------------------------
1 | import pandas as pd
2 | import statsmodels.stats.proportion as smp
3 | from scipy.stats import chi2
4 | import scipy.stats
5 |
6 |
7 | class Chi2Sample:
8 | def __init__(self, sample: pd.DataFrame, confidence_level: float = 0.95):
9 | self.sample = sample
10 | self.confidence_level = confidence_level
11 | self.compute()
12 |
13 | def compute(self) -> None:
14 | self.get_success()
15 | self.get_trial()
16 | self.get_confidence_interval()
17 |
18 | def get_success(self) -> None:
19 | self.success = self.sample.sum().iloc[0]
20 |
21 | def get_trial(self) -> None:
22 | self.trial = len(self.sample)
23 |
24 | def get_confidence_interval(self) -> None:
25 | self.conf_int = smp.proportion_confint(
26 | self.success, self.trial, alpha=1 - self.confidence_level, method="wilson"
27 | )
28 | self.conf_int_inf = self.conf_int[0] * 100
29 | self.conf_int_sup = self.conf_int[1] * 100
30 | self.confidence_interval = [self.conf_int_inf, self.conf_int_sup]
31 |
32 |
33 | class leSuccess(Chi2Sample):
34 | """
35 | Build leSuccess object.
36 |
37 | Parameters:
38 |
39 | sample_A (pd.DataFrame): A sample data.
40 | sample_B (pd.DataFrame): B sample data.
41 | confidence_level (float): desired confidence level, default : 95%.
42 |
43 | Example:
44 |
45 | ::
46 |
47 | >>> from leab import leDataset
48 | >>> from leab import after
49 |
50 | >>> data = leDataset.SampleLeSuccess()
51 | >>> ab_test = after.leSuccess(data.A,
52 | ... data.B,
53 | ... confidence_level=0.95)
54 | >>> ab_test.sample_A.confidence_interval
55 |
56 | [8.526343659939133, 22.13718821096384]
57 |
58 | >>> ab_test.p_value
59 |
60 | 0.25870176105718934
61 |
62 | >>> ab_test.get_verdict()
63 |
64 | 'No significant difference'
65 | """
66 | def __init__(
67 | self,
68 | sample_A: pd.DataFrame,
69 | sample_B: pd.DataFrame,
70 | confidence_level: float = 0.95,
71 | ):
72 | self.confidence_level = confidence_level
73 | self.sample_A = Chi2Sample(sample_A, self.confidence_level)
74 | self.sample_B = Chi2Sample(sample_B, self.confidence_level)
75 | self.compute()
76 |
77 | def compute(self) -> None:
78 | self._get_contingency_table()
79 | self._get_observed_values()
80 | self._get_expected_values()
81 | self._get_chi_square_statistic()
82 | self._get_degree_of_freedom()
83 | self._get_p_value()
84 |
85 | def _get_contingency_table(self) -> None:
86 | sample_A_value_counts = self.sample_A.sample["success"].value_counts()
87 | sample_B_value_counts = self.sample_B.sample["success"].value_counts()
88 | self.contingency_table = pd.DataFrame(
89 | [sample_A_value_counts, sample_B_value_counts]
90 | )
91 |
92 | self.contingency_table.index = ["sample_A", "sample_B"]
93 | self.contingency_table.columns = ["fail", "success"]
94 |
95 | def _get_observed_values(self) -> None:
96 | self.observed_values = self.contingency_table.values
97 |
98 | def _get_expected_values(self) -> None:
99 | b = scipy.stats.chi2_contingency(self.contingency_table)
100 | self.expected_values = b[3]
101 |
102 | def _get_chi_square_statistic(self) -> None:
103 | chi_square = sum(
104 | [
105 | (o - e) ** 2.0 / e
106 | for o, e in zip(self.observed_values, self.expected_values)
107 | ]
108 | )
109 | self.chi_square_statistic = chi_square[0] + chi_square[1]
110 |
111 | def _get_degree_of_freedom(self) -> None:
112 | no_of_rows = len(self.contingency_table.iloc[0:2, 0])
113 | no_of_columns = len(self.contingency_table.iloc[0, 0:2])
114 | self.degree_of_freedom = (no_of_rows - 1) * (no_of_columns - 1)
115 |
116 | def _get_p_value(self) -> None:
117 | self.p_value = 1 - chi2.cdf(
118 | x=self.chi_square_statistic, df=self.degree_of_freedom
119 | )
120 |
121 | def get_verdict(self) -> None:
122 | if self.p_value < 1.0 - self.confidence_level:
123 | if (
124 | self.sample_A.success / self.sample_A.trial
125 | > self.sample_B.success / self.sample_B.trial
126 | ):
127 | return("Sample A is more successful")
128 | else:
129 | return("Sample B is more successful")
130 | else:
131 | return("No significant difference")
132 |
--------------------------------------------------------------------------------
/docs/notebooks/average_comparaison.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# Le Average\n",
8 | "\n",
9 | "Here we will compare means of value obtained from A versus B. \n",
10 | "First, lets import some data from `leDataset` :"
11 | ]
12 | },
13 | {
14 | "cell_type": "code",
15 | "execution_count": 2,
16 | "metadata": {},
17 | "outputs": [],
18 | "source": [
19 | "from leab import after\n",
20 | "from leab import leDataset\n",
21 | "\n",
22 | "data = leDataset.SampleLeAverage()"
23 | ]
24 | },
25 | {
26 | "cell_type": "markdown",
27 | "metadata": {},
28 | "source": [
29 | "`data` is composed by `data.A` and `data.B`"
30 | ]
31 | },
32 | {
33 | "cell_type": "code",
34 | "execution_count": 3,
35 | "metadata": {},
36 | "outputs": [
37 | {
38 | "data": {
39 | "text/html": [
40 | "
13 | le AB is a Python library for AB testing analysis.
14 |
15 |
16 | ## ⚡️ Quick start
17 |
18 | [](https://colab.research.google.com/drive/1R3z8uoCvpqhVfH0X6eU_-TaRRPbKUqbM?usp=sharing)
19 |
20 | Before launching your AB test, you can compute the needed **sample size** per variation :
21 |
22 | ```python
23 | >>> from leab import before
24 |
25 | >>> ab_test = before.leSample(conversion_rate=20,
26 | ... min_detectable_effect=2)
27 | >>> ab_test.get_size_per_variation()
28 |
29 | 6347
30 | ```
31 |
32 | After reaching the needed sample size, you can **compare means** obtained from A VS B :
33 |
34 | ```python
35 | >>> from leab import after
36 | >>> from leab import leDataset
37 |
38 | >>> data = leDataset.SampleLeAverage()
39 |
40 | >>> ab_test = after.leAverage(data.A, data.B)
41 | >>> teab_testst.get_verdict()
42 |
43 | 'Sample A mean is greater'
44 | ```
45 |
46 | ## 🛠 Installation
47 |
48 | 🐍 You need to install **Python 3.12** or above.
49 |
50 | Installation can be done by using `pip`.
51 | There are [wheels available](https://pypi.org/project/leab/#files) for **Linux**, **MacOS**, and **Windows**.
52 |
53 | ```bash
54 | pip install leab
55 | ```
56 |
57 | You can also install the latest development version as so:
58 |
59 | ```bash
60 | pip install git+https://github.com/tlentali/leab
61 |
62 | # Or, through SSH:
63 | pip install git+ssh://git@github.com/tlentali/leab.git
64 | ```
65 |
66 | ## 🥄 Philosophy
67 |
68 | > "*Life is a sum of all our choices.*"
69 | > **Albert Camus**
70 |
71 | Get ready to take a decision !
72 |
73 | AB testing has never been more popular, especially on Internet based companies.
74 | Even if each test is unique, some questions seem to be asked again and again :
75 |
76 | - when is my test going to be statistically significant ?
77 | - is A more successful than B ?
78 | - does A generate more than B ?
79 |
80 | Strong statistical knowledge are required to handle it from start to end correctly.
81 | To answer those questions in a simple and robust way, we built `le AB`.
82 | Lets Python do AB testing analysis !
83 |
84 | ## 🔥 Features
85 |
86 | Here are some benefits of using `Le AB` :
87 |
88 | - **Sample size** : How many subjects are needed for my AB test ?
89 | - **Test duration** : How many days are needed for my AB test ?
90 | - **Rate of success** : Does the rate of success differ across two groups ?
91 | - **Average value** : Does the average value differ across two groups ?
92 |
93 | ## 🔗 Useful links
94 |
95 | - [Documentation](https://tlentali.github.io/leab/)
96 | - [API reference](https://tlentali.github.io/leab/reference/leAverage/)
97 | - [Issue tracker](https://github.com/tlentali/leab/issues)
98 |
99 | ## 🙏 Thanks
100 |
101 | This project takes its inspiration from [Evan Miller](https://www.evanmiller.org/) great work, especially the following :
102 |
103 | - [Sample Size Calculator](https://www.evanmiller.org/ab-testing/sample-size.html)
104 | - [Chi-Squared Test](https://www.evanmiller.org/ab-testing/chi-squared.html)
105 | - [2 Sample T-Test](https://www.evanmiller.org/ab-testing/t-test.html)
106 |
107 | Thank you so much Evan M. for your work, it saved our lives so many times !
108 |
109 | A big thanks to [Max Halford](https://maxhalford.github.io/) too, who inspired us in the structure of this project, particularly for docs and tests.
110 | Have a look at [Creme-ml](https://github.com/creme-ml/creme), it's just amazingly done !
111 |
112 | To finish, thanks to all of you who use or are going to use this lib, hope it helps !
113 |
114 | ## 🖖 Contributing
115 |
116 | Feel free to contribute in any way you like, we're always open to new ideas and approaches. If you want to contribute to the code base please check out the [CONTRIBUTING.md](https://github.com/tlentali/leab/blob/master/CONTRIBUTING.md) file. Also take a look at the [issue tracker](https://github.com/tlentali/leab/issues) and see if anything takes your fancy.
117 |
118 | This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Again, contributions of any kind are welcome!
119 |
120 |
121 |
122 |
123 |
9 |
10 | 
