Skip to content

Commit

Permalink
Merge pull request #122 from e10v/dev
Browse files Browse the repository at this point in the history 
Round revenue to 2 decimals
  • Loading branch information
@e10v
e10v authored Jan 7, 2025
2 parents 35c1ff8 + 55791de commit 057ddbd
Show file tree
Hide file tree
Showing 10 changed files with 187 additions and 187 deletions.
4 changes: 2 additions & 2 deletions docs/custom-metrics.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ user: [[0,1,2,3,4,...,3995,3996,3997,3998,3999]]
variant: [[1,0,1,1,0,...,0,0,0,0,0]]
sessions: [[2,2,2,2,1,...,2,2,3,1,5]]
orders: [[1,1,1,1,1,...,0,0,0,0,2]]
revenue: [[9.166147128806545,6.4340787057460656,7.943873223822707,15.928674729738708,7.136917019113867,...,0,0,0,0,17.162458516177704]]
revenue: [[9.17,6.43,7.94,15.93,7.14,...,0,0,0,0,17.16]]
has_order: [[1,1,1,1,1,...,0,0,0,0,1]]

```
Expand Down Expand Up @@ -225,7 +225,7 @@ Now we can perform the Mann-Whitney U test:
>>> print(result.to_string(("metric", "pvalue", "statistic")))
metric pvalue statistic
mwu_orders 0.0263 2069092
mwu_revenue 0.0300 2068063
mwu_revenue 0.0300 2068060

```

Expand Down
28 changes: 14 additions & 14 deletions docs/data-backends.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -83,17 +83,17 @@ Ibis Table is a lazy object. It doesn't fetch the data when created. You can use
```pycon
>>> ibis.options.interactive = True
>>> print(data.head(5))
┏━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┓
┃ user ┃ variant ┃ sessions ┃ orders ┃ revenue ┃
┡━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━┩
│ int64 │ int64 │ int64 │ int64 │ float64 │
├───────┼─────────┼──────────┼────────┼───────────┤
│ 0 │ 1 │ 2 │ 1 │ 9.166147 │
│ 1 │ 0 │ 2 │ 1 │ 6.434079 │
│ 2 │ 1 │ 2 │ 1 │ 7.943873 │
│ 3 │ 1 │ 2 │ 1 │ 15.928675 │
│ 4 │ 0 │ 1 │ 1 │ 7.136917 │
└───────┴─────────┴──────────┴────────┴───────────┘
┏━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┓
┃ user ┃ variant ┃ sessions ┃ orders ┃ revenue ┃
┡━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━┩
│ int64 │ int64 │ int64 │ int64 │ float64 │
├───────┼─────────┼──────────┼────────┼─────────┤
│ 0 │ 1 │ 2 │ 1 │ 9.17 │
│ 1 │ 0 │ 2 │ 1 │ 6.43 │
│ 2 │ 1 │ 2 │ 1 │ 7.94 │
│ 3 │ 1 │ 2 │ 1 │ 15.93 │
│ 4 │ 0 │ 1 │ 1 │ 7.14 │
└───────┴─────────┴──────────┴────────┴─────────┘

>>> ibis.options.interactive = False

Expand Down Expand Up @@ -142,8 +142,8 @@ Aggregate[r0]
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
│ int64 │ float64 │ float64 │ float64 │ float64 │
├─────────┼───────────────────┼────────────────────┼─────────────────┼──────────────────┤
│ 0 │ 1.996045 │ 0.265726 │ 0.530400 │ 5.241079 │
│ 1 │ 1.982802 │ 0.289031 │ 0.573091 │ 5.730132 │
│ 0 │ 1.996045 │ 0.265726 │ 0.530400 │ 5.241028 │
│ 1 │ 1.982802 │ 0.289031 │ 0.573091 │ 5.730111 │
└─────────┴───────────────────┴────────────────────┴─────────────────┴──────────────────┘

>>> ibis.options.interactive = False
Expand Down Expand Up @@ -234,7 +234,7 @@ DatabaseTable: memory.main.users_data_with_cov
sessions_per_user 2.00 1.98 -0.68% [-3.2%, 1.9%] 0.603
orders_per_session 0.262 0.293 12% [4.2%, 21%] 0.00229
orders_per_user 0.523 0.581 11% [2.9%, 20%] 0.00733
revenue_per_user 5.12 5.85 14% [3.8%, 26%] 0.00675
revenue_per_user 5.12 5.85 14% [3.8%, 26%] 0.00674

```

Expand Down
58 changes: 29 additions & 29 deletions docs/multiple-testing.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,23 +38,23 @@ As an example, consider an experiment with three variants, a control and two tre
... ))
>>> print(data)
shape: (6_046, 5)
┌──────┬─────────┬──────────┬────────┬───────────┐
│ user ┆ variant ┆ sessions ┆ orders ┆ revenue │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 ┆ i64 ┆ f64 │
╞══════╪═════════╪══════════╪════════╪═══════════╡
│ 0 ┆ 1 ┆ 2 ┆ 1 ┆ 9.58279 │
│ 1 ┆ 0 ┆ 2 ┆ 1 ┆ 6.434079 │
│ 2 ┆ 1 ┆ 2 ┆ 1 ┆ 8.304958 │
│ 3 ┆ 1 ┆ 2 ┆ 1 ┆ 16.652705 │
│ 4 ┆ 0 ┆ 1 ┆ 1 ┆ 7.136917 │
│ … ┆ … ┆ … ┆ … ┆ … │
│ 3989 ┆ 2 ┆ 4 ┆ 4 ┆ 34.931448 │
│ 3991 ┆ 2 ┆ 1 ┆ 0 ┆ 0.0 │
│ 3992 ┆ 2 ┆ 3 ┆ 3 ┆ 27.964647 │
│ 3994 ┆ 2 ┆ 2 ┆ 1 ┆ 17.217892 │
│ 3998 ┆ 2 ┆ 3 ┆ 0 ┆ 0.0 │
└──────┴─────────┴──────────┴────────┴───────────┘
┌──────┬─────────┬──────────┬────────┬─────────┐
│ user ┆ variant ┆ sessions ┆ orders ┆ revenue │
│ --- ┆ --- ┆ --- ┆ --- ┆ --- │
│ i64 ┆ i64 ┆ i64 ┆ i64 ┆ f64 │
╞══════╪═════════╪══════════╪════════╪═════════╡
│ 0 ┆ 1 ┆ 2 ┆ 1 ┆ 9.58 │
│ 1 ┆ 0 ┆ 2 ┆ 1 ┆ 6.43 │
│ 2 ┆ 1 ┆ 2 ┆ 1 ┆ 8.3 │
│ 3 ┆ 1 ┆ 2 ┆ 1 ┆ 16.65 │
│ 4 ┆ 0 ┆ 1 ┆ 1 ┆ 7.14 │
│ … ┆ … ┆ … ┆ … ┆ … │
│ 3989 ┆ 2 ┆ 4 ┆ 4 ┆ 34.93 │
│ 3991 ┆ 2 ┆ 1 ┆ 0 ┆ 0.0 │
│ 3992 ┆ 2 ┆ 3 ┆ 3 ┆ 27.96 │
│ 3994 ┆ 2 ┆ 2 ┆ 1 ┆ 17.22 │
│ 3998 ┆ 2 ┆ 3 ┆ 0 ┆ 0.0 │
└──────┴─────────┴──────────┴────────┴─────────┘

```

Expand All @@ -74,7 +74,7 @@ variants metric control treatment rel_effect_size rel_effect_size_ci
(0, 1) sessions_per_user 2.00 1.98 -0.66% [-3.7%, 2.5%] 0.674
(0, 1) orders_per_session 0.266 0.289 8.8% [-0.89%, 19%] 0.0762
(0, 1) orders_per_user 0.530 0.573 8.0% [-2.0%, 19%] 0.118
(0, 1) revenue_per_user 5.24 5.99 14% [2.1%, 28%] 0.0212
(0, 1) revenue_per_user 5.24 5.99 14% [2.1%, 28%] 0.0211
(0, 2) sessions_per_user 2.00 2.02 0.98% [-2.1%, 4.1%] 0.532
(0, 2) orders_per_session 0.266 0.295 11% [1.2%, 22%] 0.0273
(0, 2) orders_per_user 0.530 0.594 12% [1.7%, 23%] 0.0213
Expand All @@ -100,7 +100,7 @@ False discovery rate (FDR) is the expected value of the proportion of false disc
>>> print(adjusted_results_fdr)
comparison metric control treatment rel_effect_size pvalue pvalue_adj
(0, 1) orders_per_user 0.530 0.573 8.0% 0.118 0.245
(0, 1) revenue_per_user 5.24 5.99 14% 0.0212 0.0592
(0, 1) revenue_per_user 5.24 5.99 14% 0.0211 0.0592
(0, 2) orders_per_user 0.530 0.594 12% 0.0213 0.0592
(0, 2) revenue_per_user 5.24 6.25 19% 0.00218 0.0182

Expand All @@ -122,7 +122,7 @@ The method also adjusts the significance level alpha and saves it as `alpha_adj`
... )))
comparison metric control treatment rel_effect_size pvalue alpha_adj
(0, 1) orders_per_user 0.530 0.573 8.0% 0.118 0.0240
(0, 1) revenue_per_user 5.24 5.99 14% 0.0212 0.0120
(0, 1) revenue_per_user 5.24 5.99 14% 0.0211 0.0120
(0, 2) orders_per_user 0.530 0.594 12% 0.0213 0.0180
(0, 2) revenue_per_user 5.24 6.25 19% 0.00218 0.00600

Expand All @@ -134,9 +134,9 @@ By default, **tea-tasting** assumes arbitrary dependence between hypotheses and
>>> print(tt.adjust_fdr(results, metrics, arbitrary_dependence=False))
comparison metric control treatment rel_effect_size pvalue pvalue_adj
(0, 1) orders_per_user 0.530 0.573 8.0% 0.118 0.118
(0, 1) revenue_per_user 5.24 5.99 14% 0.0212 0.0284
(0, 1) revenue_per_user 5.24 5.99 14% 0.0211 0.0284
(0, 2) orders_per_user 0.530 0.594 12% 0.0213 0.0284
(0, 2) revenue_per_user 5.24 6.25 19% 0.00218 0.00873
(0, 2) revenue_per_user 5.24 6.25 19% 0.00218 0.00872

```

Expand All @@ -148,9 +148,9 @@ Family-wise error rate (FWER) is the probability of making at least one type I e
>>> print(tt.adjust_fwer(results, metrics))
comparison metric control treatment rel_effect_size pvalue pvalue_adj
(0, 1) orders_per_user 0.530 0.573 8.0% 0.118 0.118
(0, 1) revenue_per_user 5.24 5.99 14% 0.0212 0.0635
(0, 2) orders_per_user 0.530 0.594 12% 0.0213 0.0635
(0, 2) revenue_per_user 5.24 6.25 19% 0.00218 0.00873
(0, 1) revenue_per_user 5.24 5.99 14% 0.0211 0.0634
(0, 2) orders_per_user 0.530 0.594 12% 0.0213 0.0634
(0, 2) revenue_per_user 5.24 6.25 19% 0.00218 0.00872

```

Expand All @@ -165,9 +165,9 @@ By default, **tea-tasting** assumes arbitrary dependence between hypotheses and
... ))
comparison metric control treatment rel_effect_size pvalue pvalue_adj
(0, 1) orders_per_user 0.530 0.573 8.0% 0.118 0.118
(0, 1) revenue_per_user 5.24 5.99 14% 0.0212 0.0422
(0, 1) revenue_per_user 5.24 5.99 14% 0.0211 0.0422
(0, 2) orders_per_user 0.530 0.594 12% 0.0213 0.0422
(0, 2) revenue_per_user 5.24 6.25 19% 0.00218 0.00870
(0, 2) revenue_per_user 5.24 6.25 19% 0.00218 0.00869

```

Expand All @@ -188,9 +188,9 @@ In the examples above, the methods `adjust_fdr` and `adjust_fwer` received resul
... ))
comparison metric control treatment rel_effect_size pvalue pvalue_adj
Experiment 1 orders_per_user 0.530 0.573 8.0% 0.118 0.245
Experiment 1 revenue_per_user 5.24 5.99 14% 0.0212 0.0588
Experiment 1 revenue_per_user 5.24 5.99 14% 0.0211 0.0587
Experiment 2 orders_per_user 0.514 0.594 16% 0.00427 0.0178
Experiment 2 revenue_per_user 5.10 6.25 22% 6.27e-04 0.00523
Experiment 2 revenue_per_user 5.10 6.25 22% 6.27e-04 0.00522

```

Expand Down
4 changes: 2 additions & 2 deletions docs/user-guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ user: [[0,1,2,3,4,...,3995,3996,3997,3998,3999]]
variant: [[1,0,1,1,0,...,0,0,0,0,0]]
sessions: [[2,2,2,2,1,...,2,2,3,1,5]]
orders: [[1,1,1,1,1,...,0,0,0,0,2]]
revenue: [[9.166147128806545,6.4340787057460656,7.943873223822707,15.928674729738708,7.136917019113867,...,0,0,0,0,17.162458516177704]]
revenue: [[9.17,6.43,7.94,15.93,7.14,...,0,0,0,0,17.16]]

```

Expand Down Expand Up @@ -251,7 +251,7 @@ Example usage:
sessions_per_user 2.00 1.98 -0.68% [-3.2%, 1.9%] 0.603
orders_per_session 0.262 0.293 12% [4.2%, 21%] 0.00229
orders_per_user 0.523 0.581 11% [2.9%, 20%] 0.00733
revenue_per_user 5.12 5.85 14% [3.8%, 26%] 0.00675
revenue_per_user 5.12 5.85 14% [3.8%, 26%] 0.00674

```

Expand Down
Loading

0 comments on commit 057ddbd

Please sign in to comment.