Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GRPC fix #2225

Merged
merged 4 commits into from
Sep 20, 2024
Merged

GRPC fix #2225

merged 4 commits into from
Sep 20, 2024

Conversation

fedml-dimitris
Copy link
Contributor

Fixing grpc compatibility with the fedml.ai platform and simplifying the user-end requirements to define the grpc server ip and port. The values defined in the grpc config file will override the default ip and port values.

fedml-dimitris added 3 commits September 4, 2024 00:20
…the user-end requirements to define the grpc server ip and port. The values defined in the grpc config file will override the default ip and port values.
Copy link

gitguardian bot commented Sep 14, 2024

⚠️ GitGuardian has uncovered 17 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request
GitGuardian id GitGuardian status Secret Commit Filename
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/master_job_runner.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/comm_utils/constants.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/device_client_runner.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/worker_protocol_manager.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/model_device_client.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/scheduler_core/compute_cache_manager.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/api/modules/device.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/worker_job_runner.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/master_job_runner.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/model_device_server.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/device_model_inference.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/job_runner_msg_sender.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/master_job_runner.py View secret
5451874 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/model_scheduler/master_protocol_manager.py View secret
5692101 Triggered Generic High Entropy Secret 87ae30a python/fedml/computing/scheduler/model_scheduler/device_model_deployment.py View secret
9453265 Triggered Generic High Entropy Secret 87ae30a python/fedml/api/api_test.py View secret
8762943 Triggered Generic Password 87ae30a python/fedml/computing/scheduler/scheduler_core/compute_cache_manager.py View secret
🛠 Guidelines to remediate hardcoded secrets
  1. Understand the implications of revoking this secret by investigating where it is used in your code.
  2. Replace and store your secrets safely. Learn here the best practices.
  3. Revoke and rotate these secrets.
  4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Copy link

@RahulVadisetty91 RahulVadisetty91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

w20

Update Backend Initializations: Ensure that all communication managers for different backends are correctly initialized and that any new backend types are handled.

Verify Exception Messages: Ensure exception messages are accurate and reflect the current backend configurations.

Ensure Consistency: Check if all configuration settings are applied consistently for new backends.

def _init_manager(self):
    # Check if the backend is correctly initialized for new types or existing backends.
    if self.backend == "MQTT_S3":
        from .communication.mqtt_s3.mqtt_s3_multi_clients_comm_manager import MqttS3MultiClientsCommManager
        mqtt_config, s3_config = self.get_training_mqtt_s3_config()
        self.com_manager = MqttS3MultiClientsCommManager(
            mqtt_config,
            s3_config,
            topic=str(self.args.run_id),
            client_rank=self.rank,
            client_num=self.size,
            args=self.args,
        )
    elif self.backend == "MQTT_WEB3":
        from .communication.mqtt_web3.mqtt_web3_comm_manager import MqttWeb3CommManager
        mqtt_config, web3_config = self.get_training_mqtt_web3_config()
        self.com_manager = MqttWeb3CommManager(
            mqtt_config,
            web3_config,
            topic=str(self.args.run_id),
            client_rank=self.rank,
            client_num=self.size,
            args=self.args,
        )
    elif self.backend == "GRPC":
        from .communication.grpc.grpc_comm_manager import GRPCCommManager

        HOST = "0.0.0.0"
        PORT = CommunicationConstants.GRPC_BASE_PORT + self.rank
        self.com_manager = GRPCCommManager(
            HOST, PORT, ip_config_path=self.args.grpc_ipconfig_path, client_id=self.rank, client_num=self.size,
            grpc_ipconfig_path=self.args.grpc_ipconfig_path,
            client_rank=self.rank,
            client_num=self.size,
            args=self.args,
        )
    elif self.backend == "TRPC":
        from .communication.trpc.trpc_comm_manager import TRPCCommManager
        self.com_manager = TRPCCommManager(
            self.args.trpc_master_config_path, process_id=self.rank, world_size=self.size + 1, args=self.args,
        )
    else:
        if self.com_manager is None:
            raise Exception("no such backend: {}. Please check the comm_backend spelling.".format(self.backend))
        else:
            logging.info("using self-defined communication backend")

@fedml-alex fedml-alex merged commit f6b8c44 into dev/v0.7.0 Sep 20, 2024
0 of 37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants