AI Python Code Tutor (Gemini)

This action analyzes test results and code to provide personalized feedback on student assignments, including identifying errors, suggesting improvements, and explaining concepts.

The AI tutor can identify logic errors, suggest more efficient algorithms, explain complex concepts in simpler terms, and even provide links to relevant documentation.

It would save instructors' time, provide more consistent feedback, and help students learn more effectively regardless of time & location.

Prerequisites

Before using this action, ensure you have the following:

• pytest-json-report plugin: This plugin is required to generate the JSON test reports that the action uses for analysis.
  • You can install it using pip:

    pip install pytest-json-report

  • To generate the JSON report, run the following command:

    python -m pytest --json-report --json-report-file=report.json tests/test_my_test_file.py

  • For more information on the plugin, see the pytest-json-report documentation.

Key Features

• Provide AI powered feedback on Python code assignments.
• Support multiple JSON files by pytest-json-report plugin.
• Support multiple student files.

Usage

• Please set GOOGLE_API_KEY in the repository's secrets.

• To use the action, create a workflow file (e.g., .github/workflows/classroom.yml) in your repository and add the following configuration:

on: [push]

jobs:
  grade:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install dependencies
        run: pip install pytest pytest-json-report
      - name: Run tests
        run: |
          python -m pytest --json-report --json-report-file=report.json tests/test_my_test_file.py
      - name: AI Python Tutor
        uses: kangwonlee/[email protected]
        if: always()
        with:
          report-files: report.json
          api-key: ${{ secrets.GOOGLE_API_KEY }}
          student-files: exercise.py
          readme-path: README.md
          explanation-in: English
        timeout-minutes: 5 

• This action analyzes the JSON output generated by the pytest-json-report plugin to understand the test results of your Python code and provide AI-powered feedback.
• You can exclude common content of README.md assignment instruction to save AI API data usage.
  • Please mark the common contents by the starting marker From here is common to all assignments. and the ending marker Until here is common to all assignments. in the README.md file, surrounded by double backtick (ascii 96) characters.

Inputs

• report-files: Comma-separated list of JSON report files generated by pytest-json-report. (Required)
• api-key: Your Google API key. (Required)
• student-files: Comma-separated list of Python files containing the student's code for review. (Required)
• readme-path: Path to the assignment instructions (README.md). (Optional)
• explanation-in: Language for explanations (e.g., English, Korean). (Optional, default: English)

Example with multiple JSON files and student files

with:
  report-files: 'report1.json, report2.json, reports/*.json'
  api-key: ${{ secrets.GOOGLE_API_KEY }}
  student-files: 'exercise1.py, exercise2.py' 
  readme-path: README.md
  explanation-in: English

Limitations

• The action currently supports only Python code assignments.
• The action requires the pytest-json-report plugin to generate JSON test reports.

Future Work

• Support more human languages.

  • Add language argument
  • Add language detection

• Add AI model choice argument to select different AI models.

  • Gemini advance
  • Gemini basic

• Support more programming languages.

  • Possibly in a different repository

• Add verbose argument to provide more detailed feedback.

  • More internal details

Troubleshooting

• If you encounter any issues, please check the action logs for more information.
  • To view the logs, go to the Actions tab of your repository, click on the workflow run, and then select the "AI Python Tutor" job. You'll find the logs for each step in the job.

Common Errors:

• API Key Errors:
  • "Invalid API key": Double-check your API key in the repository secrets and the workflow file.
  • "API key not found": Make sure you have set the GOOGLE_API_KEY in your repository secrets.
• Report File Errors:
  • "Report file not found": Ensure the JSON report is generated using pytest-json-report and placed in the correct location (e.g., report.json in the root of your repository).
  • "Invalid report format": Use the pytest-json-report plugin with the recommended options to generate a valid report.
• Student File Errors:
  • "Student file not found": Make sure the student's code file is in the correct location and has a .py extension.
• Timeout Errors:
  • "Action timed out": Increase the timeout-minutes value in your workflow file or optimize your code.

Debugging Suggestions:

• Check Action Logs: View the action logs in the GitHub Actions interface for detailed error messages.
• Test Locally: Use act to run the action locally and debug your workflow.

Contact

• If you have any questions or feedback, please contact the author : https://github.com/kangwonlee.

License

This project is licensed under the BSD 3-Clause License + Do Not Harm.

Copyright (c) 2024 Kangwon Lee

Acknowledgements

This project is based on the python-github-action-template by Vincent A. Cicirello, licensed under the MIT License.