Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement AI-powered Code Review System with Line-by-Line Comments #6

Merged
merged 2 commits into from
Nov 9, 2024

Conversation

bstanga
Copy link
Contributor

@bstanga bstanga commented Nov 9, 2024

Major enhancement introducing an AI-powered code review system that analyzes PRs and provides detailed feedback. The system evaluates code quality, security concerns, and generates specific line-by-line comments as GitHub review comments. The implementation includes a new review prompt system, comment submission logic, and improved diff handling for better code analysis.

Copy link

github-actions bot commented Nov 9, 2024

📖 Walkthrough

Major enhancement implementing an AI-powered code review system. The changes include a new review prompt system that analyzes PR content, generates detailed code comments, and submits them as GitHub review comments. The system evaluates code quality, security concerns, and provides actionable feedback with specific line-by-line comments.

Changes

File Summary
src/diff.ts Export File type and simplify generateFileCodeDiff function by removing the simplified parameter. The function now always uses full hunk formatting for consistent diff output.
src/messages.ts Rename buildOverviewMessage to buildLoadingMessage and standardize signature formatting. Add consistent PRESUBMIT_SIGNATURE to both overview and walkthrough messages.
src/octokit.ts Convert initOctokit from arrow function to regular function with explicit return type annotation for better type safety.
src/prompts.ts Implement comprehensive AI review system with new types AIComment and PullRequestReview. Add runReviewPrompt function that analyzes PR content and generates detailed code review feedback with specific comments.
src/pull_request.ts Add functionality to submit AI-generated reviews as GitHub comments. Implement robust error handling and fallback mechanisms for comment submission, supporting both single-line and multi-line review comments.

@github-actions github-actions bot changed the title Dev Implement AI Code Review System Nov 9, 2024
@bstanga
Copy link
Contributor Author

bstanga commented Nov 9, 2024

📖 Walkthrough

Major enhancement introducing an AI-powered code review system that analyzes PRs and provides detailed feedback. The system evaluates code quality, security concerns, and generates specific line-by-line comments as GitHub review comments. The implementation includes a new review prompt system, comment submission logic, and improved diff handling for better code analysis.

Changes

File Summary
src/diff.ts Simplified generateFileCodeDiff by removing the simplified parameter and standardizing diff generation. Exported File type for broader usage across the codebase.
src/messages.ts Renamed buildOverviewMessage to buildLoadingMessage and standardized message signatures. Added PRESUBMIT_SIGNATURE constant and improved formatting of generated messages with consistent signature placement.
src/octokit.ts Enhanced type safety by converting initOctokit from arrow function to regular function with explicit return type annotation for better TypeScript integration.
src/prompts.ts Implemented comprehensive AI review system with new types AIComment and PullRequestReview. Added runReviewPrompt function that generates detailed code reviews with comments, security analysis, and quality scoring.
src/pull_request.ts Added functionality to submit AI-generated reviews as GitHub comments. Implemented robust comment submission with fallback mechanisms for both single-line and multi-line comments. Handles review submission failures gracefully.

autogenerated by presubmit.ai

@bstanga bstanga changed the title Implement AI Code Review System Implement AI-powered Code Review System with Line-by-Line Comments Nov 9, 2024
Copy link
Contributor Author

@bstanga bstanga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review submitted

): Promise<PullRequestReview> {
let systemPrompt = `
<IMPORTANT INSTRUCTIONS>
You are an experienced senior software engineer tasked with reviewing a Git Pull Request (PR). Your goal is to provide comments to improve code quality, catcht typos, potential bugs or security issues, and provide meaningful code suggestions when applicable.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a typo in the word 'catcht' which should be 'catch'.

};

// Handle file comments
const fileComments = comments.filter((c) => !c.end_line);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filter condition !c.end_line for file comments is incorrect. A comment with end_line: 0 would be treated as a file comment. Consider using a more explicit condition like c.end_line === undefined || c.end_line === null.

}

// Handle line comments
const lineComments = comments.filter((c) => !!c.end_line);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The filter condition !!c.end_line is inconsistent with the previous filter. This could lead to comments being missed if end_line is 0. Consider using c.end_line !== undefined && c.end_line !== null.

Comment on lines +193 to +199
// If submitting all comments at once fails, try submitting them one by one
info("trying to submit comments one by one");
await Promise.allSettled(
lineComments.map((c) =>
submitInlineComment(c.file, c.end_line, c.content)
)
);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fallback error handling doesn't log individual comment submission failures. Consider adding error logging for each failed comment submission to help with debugging:

const results = await Promise.allSettled(
  lineComments.map((c) =>
    submitInlineComment(c.file, c.end_line, c.content)
  )
);
results.forEach((result, index) => {
  if (result.status === 'rejected') {
    warning(`Failed to submit comment for ${lineComments[index].file}: ${result.reason}`);
  }
});

Comment on lines +99 to +107
export type AIComment = {
file: string;
start_line: number;
end_line: number;
highlighted_code: string;
header: string;
content: string;
label: string;
};
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using zod schema for AIComment type definition instead of a plain type. This would allow runtime validation of the comment structure and provide consistent validation with the schema defined below:

export const AICommentSchema = z.object({
  file: z.string(),
  start_line: z.number(),
  end_line: z.number(),
  highlighted_code: z.string(),
  header: z.string(),
  content: z.string(),
  label: z.string()
});

export type AIComment = z.infer<typeof AICommentSchema>;

@bstanga bstanga merged commit f781abc into main Nov 9, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant