Total ROI is very high for a couple of channels. Whereas the optimized percentage allocation is still low for these channels

[rohan1997vk]

Contributing to FB NextGen MMM R script

I ran Robyn on my data. It has 19 channels. For a couple of those channels that ROI is way to high. While after budget allocation, the new spend share is still not high.

Issue

A summary of what the issue is about.

Steps to Reproduce

Please provide instructions that can be used to reproduce your issue. Usually this will include a test case that produces the wrong output.

Expected Behavior

What you expected to happen. For example the error message you expected to see.

Actual Results

What actually happened. For example, an error message you did not expect to see.

Environment

R version (R --version)

[Leonelsentana]

Hi @rohan1997vk hope you are well. Have you tried decreasing channel_constr_low (range is 0.01-1) for your biggest spend channels while increasing channel_constr_up for the smaller spend channels in your budget allocator (E.g. 2 means up to 2x channels average investment from past data) ? That should help you get a better balance on the allocation.
Let us know if that helps!

[rohan1997vk]

Channel1 - (0.65,1.5) ; Historical the total spend on this channel is 0.72% - ROI is ~ 48[generated by Robyn]
Channel2 - (0.65,1.5); Historical the total spend on this channel is 3.57% - ROI is ~ 8[generated by Robyn]
Channel3 - (0.7,1.2); Historically the total spend on this channel is 46.6% - ROI is ~0 .34[generated by Robyn]

P.S Channel 1 and 2 are search channels; whereas channel-3 is TV. My concern here is regarding the extremely high ROI for channels 1 and 2

[gufengzhou]

Is this the chosen model? Are all Pareto models showing the same/high ROI for these channels?

[rohan1997vk]

Yes, it is one of the models. However, for all the generated models by Robyn, the end results are comparable to each other. There is no major deviations

[gufengzhou]

You can try increasing the pareto_front argument in robyn_run() to get more pareto results. Regarding the high search channels ROI, if you have some numbers in mind that you're very certain, you can use those numbers to calibrate search. That'd be the best way to deal with it.

[gufengzhou]

To consolidate our recommendations for either too high ROI or 0 effect for certain channels here:

• Increase hyperparameter bounds for this channel to allow more room for both adstock & saturation transformation to obtain probably different fit/effect size/ROI.
• Increase trial numbers to collect more diversified results from different trials (random walks)
• Increase pareto_fronts in robyn_run() to output more results to choose from
• The ultimate way to get desired effect size/ROI for certain channel is through calibration

We're also considering adding an extra parameter in robyn_run() to control for stricter calibration constraints when necessary. However, please understand that Robyn or MMM in general is correlational by nature and all the options above won't guarantee an certain effect size.