🎉 Source Facebook Marketing: Added support for multiple Facebook accounts #8237

mlavoie-sm360 · 2021-11-24T16:03:12Z

What

*Closes #7740

How

Updated the spec to allow one of two strategies regarding Facebook accounts:
- All accounts linked to access token
- One or many accounts as specified

🚨 User Impact 🚨

Are there any breaking changes? If yes, please make sure to include it here and in any changelogs with the 🚨🚨 emoji
What is the end result perceived by the user?

🚨 The spec definition has been modified
The end user will gain support for syncing all Facebook accounts associated to token

Pre-merge Checklist

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here
After the new connector version is published, connector version bumped in the seed directory as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

This change is

Source Facebook Marketing: Support multiple Facebook accounts airbytehq#7740

Closes airbytehq#7740

CLAassistant · 2021-11-24T16:03:25Z

All committers have signed the CLA.

marcosmarxm · 2021-11-30T02:11:14Z

@mlavoie-sm360 please run ./gradlew format to format all files?

airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/api.py

keu · 2021-11-30T08:05:49Z

airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/api.py

        except FacebookRequestError as exc:
            raise FacebookAPIException(f"Error: {exc.api_error_code()}, {exc.api_error_message()}") from exc
-


what will happen if we can't find some accounts?

Not finding some accounts raised an Exception. Now that we handle one or many accounts, I'm not sure what the behaviour should be. If I provide 5 accounts and I don't have access to one of them, do I want to raise an exception and break the flow, log a warning one of the accounts is missing or simply let the list have 4 accounts? I'm not sure what is the cleanest behaviour.

@mlavoie-sm360
I think the check should succeed only if all specified accounts are accessible.

@keu, thanks for your input. Is it all the same to you if that check is done in source.py under def check_connection() rather than here? At the moment, check_connection fails if:

strategy is subset and at least one account is missing from the list

strategy is all and no accounts are available

strategy selected is unsupported

@mlavoie-sm360 I agree with the above if the check is done in the source.py, my only concern is that how the user will know that some accounts are not accessible anymore? How about we will have a warning for each missing account in _find_account method?

@keu, how about this?

airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/spec.json

airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/common.py

airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/streams.py

…ccount-support

…-multiple-account-support' into source-facebook-marketing-added-multiple-account-support # Conflicts: # airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/api.py # airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/streams.py

marcosmarxm · 2022-02-21T15:28:36Z

@keu now it's possible to merge this?

keu · 2022-02-21T18:04:55Z

@keu now it's possible to merge this?

@marcosmarxm this PR will need to merge with the latest master (will do it myself), currently working on #6225.

marcosmarxm · 2022-03-25T01:42:05Z

I'm closing this due inactivity.

misteryeo · 2022-03-30T23:19:41Z

Update: First off, thank you for your contribution @mlavoie-sm360, we really appreciate it! We're going to hold off reviewing this as it looks like this would require a non-trivial migration of configuration. This is something we're scheduled to implement this coming quarter which you can follow along here https://github.com/airbytehq/airbyte-internal-issues/issues/206. Once complete, we'll come back and make sure we can get this merged.

Thanks for your patience!

mlavoie-sm360 · 2022-03-31T06:17:29Z

Alright, thanks for the follow-up @misteryeo

martiningolingo · 2022-06-30T17:02:44Z

Hey, everyone! This would be a great improvement in the connection.
I'm wondering when this can be available in the official release.
Thank you for your work!

jagannathsrs · 2022-08-03T00:53:53Z

Is it non-trivial migration in the previous comment referring to existing pipelines? Will there be any issues if they create a new pipeline on this version?

mlavoie-sm360 · 2022-08-03T05:23:18Z

@jagannathsrs, if you are creating a new pipeline on this version, there are no issues. The configuration is straight forward, the major difference being that you are not restricted to a single Facebook account.

…-multiple-account-support' into staging--source-facebook-marketing-added-multiple-account-support # Conflicts: # airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/api.py

t0hai

@mlavoie-sm360 could you fix the mentioned points? then your branch works for me in production and others might want to use your solution since FB just deprecated API v.12

t0hai · 2022-08-11T12:48:07Z

airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/api.py


 import pendulum
 from cached_property import cached_property
 from facebook_business import FacebookAdsApi
 from facebook_business.adobjects import user as fb_user
 from facebook_business.adobjects.adaccount import AdAccount
 from facebook_business.exceptions import FacebookRequestError
-from source_facebook_marketing.common import FacebookAPIException
+from source_facebook_marketing.common import FacebookAPIException, SourceFacebookMarketingConfig


must be ConnectorConfig instead of SourceFacebookMarketingConfig

t0hai · 2022-08-11T12:48:48Z

airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/api.py

-                    return account
+            accounts_found = list(fb_user.User(fbid="me").get_ad_accounts())
+            if self.__config.account_selection_strategy_is_subset:
+                account_ids = self.__config.account_ids


must be self.__config.accounts.ids

t0hai · 2022-08-11T12:49:29Z

airbyte-integrations/connectors/source-facebook-marketing/source_facebook_marketing/api.py

-        raise FacebookAPIException("Couldn't find account with id {}".format(account_id))
+    def _accounts_missing_from_config(self, accounts:List[Type[AdAccount]]) -> Set[str]:
+        """Returns a list of account ids missing from config accounts"""
+        config_account_ids = set(self.__config.account_ids)


must be self.__config.accounts.ids

marcosmarxm · 2022-11-22T20:22:15Z

Hello 👋, first thank you for this amazing contribution.

We really appreciate the effort you've made to improve the project.
We ask you patience for the code review. Last month our team was focused on Hacktoberfest event and that probably left some PR without the proper feedback. And this week, due to the Thanksgiving US Holiday, most our team is out of office with their families. Another important piece of information why code won't be merge this week is: as a safety measure the core team has decided to freeze merging code to main branch to keep the release stable. Next week we'll return to you with the proper code review and update the status of your contribution.

If you have any questions feel free to send me a message in Slack!
Thanks!

marcosmarxm · 2022-12-20T19:49:29Z

Hello 👋:skin-tone-2: and thank you for your contribution!

Airbyte has instituted a code freeze between 19 and 30 December, to make sure there are no disruptions during the holidays.
Because of this, reviewing and merging your contribution may take longer than usual.
We apologize for the delay, but we want everyone to have a quiet and happy holiday.

If you have any questions or need further clarification, please don't hesitate to ping via Slack.

sherifnada · 2023-03-07T20:14:26Z

Closing for inactivity -- will reopen or reimplement once the blocking config migration issue airbytehq/airbyte-internal-issues#206 is resolved

mlavoie-sm360 added 2 commits November 24, 2021 08:25

Added support for multiple Facebook account syncing

0344a65

Source Facebook Marketing: Support multiple Facebook accounts airbytehq#7740

Source Facebook Marketing: Added support for multiple Facebook accounts

21f050f

Closes airbytehq#7740

github-actions bot added the area/connectors Connector related issues label Nov 24, 2021

octavia-squidington-iii added the community label Nov 24, 2021

marcosmarxm requested review from yevhenii-ldv and keu November 30, 2021 02:07

marcosmarxm added the zzm label Nov 30, 2021