diff --git a/dev/README_RELEASE_AIRFLOW.md b/dev/README_RELEASE_AIRFLOW.md
index eef015028235c..e206ef419bb03 100644
--- a/dev/README_RELEASE_AIRFLOW.md
+++ b/dev/README_RELEASE_AIRFLOW.md
@@ -129,9 +129,7 @@ branch you can run:
 ./dev/airflow-github compare 2.1.2 --unmerged
 ```
 
-Be careful and verify the hash commit specified. This is a 'best effort' to find it, and
-could be inaccurate if the PR was referenced in other commits after it was merged. You can start
-cherry picking from the bottom of the list. (older commits first)
+You can start cherry picking from the bottom of the list. (older commits first)
 
 When you cherry-pick, pick in chronological order onto the `vX-Y-test` release branch.
 You'll move them over to be on `vX-Y-stable` once the release is cut. Use the `-x` option
diff --git a/dev/airflow-github b/dev/airflow-github
index 8109521e4ab60..5a8a17cbf708d 100755
--- a/dev/airflow-github
+++ b/dev/airflow-github
@@ -266,7 +266,9 @@ def compare(target_version, github_token, previous_version=None, show_uncherrypi
             url="URL",
         )
     )
-
+    milestone_issues = sorted(
+        milestone_issues, key=lambda x: x.closed_at if x.closed_at else x.created_at, reverse=True
+    )
     for issue in milestone_issues:
         commit_in_main = get_commit_in_main_associated_with_pr(repo, issue)
         status = issue.state.capitalize()