[Serve] Refactor replica record handling using enum

[andylizf]

This PR introduces the following improvements:

1. Replaces the unclear condition:

   if removal_reason is not None:

   with a more explicit enum-based approach.
2. Adds a ReplicaRecordAction enum to clearly define possible actions for replica records.
3. Refactors the logic for determining replica record actions into a separate function.
4. Simplifies the main code flow by using the new enum and function.

These changes make the replica management process more explicit and easier to extend in the future.

[cblmemo]

Thanks for the refactoring @andylizf ! It is awesome. Left some discussions ;)

[cblmemo]

Suggested change

	class ReplicaRecordAction(Enum):
	KEEP = auto()
	REMOVE_SCALE_DOWN = auto()
	REMOVE_PREEMPTION = auto()
	REMOVE_VERSION_MISMATCH = auto()
	class ReplicaRecordRemoveReason(Enum):
	SCALE_DOWN = 'scale_down'
	PREEMPTION = xxx
	VERSION_MISMATCH = xxx

nit: how about we only keep the remove reason and return a None in the get_replica_record_action function to indicate keep it? Please also rename this function accordingly.
also, use a str for its value for more information.

[andylizf]

I will do this after the #4032 merge, and see how properly handle the relationship between the new classes ReplicaRecordAction and ReplicaStatusProperty.

[github-actions]

This PR is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[cblmemo]

is this still planned?

[andylizf]

is this still planned?

Yes, sure. Let me reschedule it.

[andylizf]

We definitely need a clearer ReplicaStatus and ReplicaStatusProperty. And a class representing the removal in serve_state in the proper place.