fix(ingest/gc): reduce logging, remove unnecessary sleeps

[anshbansal]

• Reducing logs was required otherwise logs were full of useless Deleted 0 ....
• Now the final report will have structured details on number of data jobs and data flows
• We don't want to wait in case there were 0 deletions. Otherwise it will take too long. This fixes that. In testing this changed things from ~1 hour to 15 minutes

• No need to query data flows in case the deletion of empty data flows is disabled

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

[skrydal]

Good catch

[skrydal]

It seems we are repeating code here, how about refactoring both ifs to

Suggested change

	if deleted_count_last_n > 0:
	logger.info(f"Deleted {deleted_count_last_n} DPIs from {job.urn}")
	if deleted_count_last_n > 0:
	logger.info(f"Deleted {deleted_count_last_n} DPIs from {job.urn}")
	if deleted_count_last_n % self.config.batch_size == 0 and self.config.delay:
	logger.info(f"Sleeping for {self.config.delay} seconds")
	time.sleep(self.config.delay)

[skrydal]

Great job, left some minor suggestions.

[skrydal]

You might consider using dictionary comprehension to reduce indentations:

          dataFlows = {flow.urn: flow for flow in self.get_data_flows()}