Add course statistics #5440

zhuhanming · 2022-12-04T20:25:48Z

Majority of #4496 rebased onto phillmont/migrate-assessment-show.

This PR introduces:

Course Progression Statistics
Course Performance Statistics

There's some overlap between Course Performance Statistics and the existing Student Statistics. The changes made to Student Statistics are also not rebased yet. All that will be changed in the next PR.

Part of #619 and #1112 (after a few more minor changes).

TODO (which I'll likely be unable to work on)

Show in UI which submissions are late/overdue
Add groups filtering (and perhaps support group average vs class average comparisons)
Possible extension would be Student Watchlist #113, since now we can check specific rows in the table and perform operations. Instructors should be able to select these students and add them into the watchlist.
Write more tests, especially on the correctness of the statistics provided.

Course Performance Table

Screen.Recording.2022-12-05.at.9.27.30.PM.mov

zhuhanming · 2022-12-04T20:34:06Z

app/controllers/course/statistics/aggregate_controller.rb

+  def assessment_info_array
+    @assessment_info_array ||= Course::Assessment.published.with_default_reference_time.
+                               where.not(course_reference_times: { end_at: nil }).
+                               where(course_id: current_course.id).
+                               pluck(:id, :title, :start_at, :end_at)
+  end
+
+  def user_submission_array # rubocop:disable Metrics/AbcSize
+    submission_data_arr = Course::Assessment::Submission.joins(creator: :course_users).
+                          where(assessment_id: assessment_info_array.map { |i| i[0] },
+                                course_users: { course_id: current_course.id, role: :student }).
+                          group(:creator_id, 'course_users.name', 'course_users.phantom').
+                          pluck(:creator_id, 'course_users.name', 'course_users.phantom',
+                                'json_agg(assessment_id)', 'array_agg(submitted_at)')
+
+    submission_data_arr.map do |sub_data|
+      assessment_to_submitted_at = sub_data[3].zip(sub_data[4]).map do |assessment_id, submitted_at|
+        if submitted_at.nil?
+          nil
+        else
+          [assessment_id, submitted_at]
+        end
+      end.compact
+
+      [sub_data[0], sub_data[1], sub_data[2], assessment_to_submitted_at] # id, name, phantom, [ass_id, sub_at]
+    end
+  end
+
+  def correctness_hash
+    query = CourseUser.find_by_sql(<<-SQL.squish
+      SELECT
+        id,
+        AVG(correctness) AS correctness
+      FROM (
+        SELECT
+          cu.id AS id,
+          SUM(caa.grade) / SUM(caq.maximum_grade) AS correctness
+        FROM
+          course_assessment_categories cat
+          INNER JOIN course_assessment_tabs tab
+          ON tab.category_id = cat.id
+          INNER JOIN course_assessments ca
+          ON ca.tab_id = tab.id
+          INNER JOIN course_assessment_submissions cas
+          ON cas.assessment_id = ca.id
+          INNER JOIN course_assessment_answers caa
+          ON caa.submission_id = cas.id
+          INNER JOIN course_assessment_questions caq
+          ON caa.question_id = caq.id
+          INNER JOIN course_users cu
+          ON cu.user_id = cas.creator_id
+        WHERE
+          cat.course_id = #{current_course.id}
+          AND caa.current_answer IS true
+          AND cas.workflow_state IN ('graded', 'published')
+          AND cu.course_id = #{current_course.id}
+          AND cu.role = 0
+        GROUP BY
+          cu.id,
+          cas.assessment_id
+        HAVING
+          SUM(caq.maximum_grade) > 0
+      ) course_user_assessment_correctness
+      GROUP BY
+        id
+    SQL
+                                  )
+    query.map { |u| [u.id, u.correctness] }.to_h
+  end


Would appreciate any help with rewriting this code to use the ORM more.

Let's add this to TODO

zhuhanming · 2022-12-05T01:54:56Z

Changed the base to master as it contains fixes that allow the app to be run locally.

ekowidianto · 2022-12-05T09:50:51Z

Changed the base to master as it contains fixes that allow the app to be run locally.

I have already rebased the phillmont/migrate-assessment-showoff the latest master branch.

zhuhanming · 2022-12-08T04:20:38Z

Btw @ekowidianto I think this PR is more or less ready to review.

ekowidianto

Many thanks for the PR! @zhuhanming
See some comments below.

client/app/bundles/course/statistics/utils/parseCourseResponse.js

client/app/bundles/course/statistics/pages/StatisticsIndex/course/StudentProgressionChart.jsx

client/app/bundles/course/statistics/pages/StatisticsIndex/course/StudentPerformanceTable.jsx

app/views/course/statistics/aggregate/course_performance.json.jbuilder

client/app/bundles/course/statistics/pages/StatisticsIndex/course/StudentPerformanceTable.jsx

client/app/bundles/course/statistics/pages/StatisticsIndex/index.jsx

ekowidianto · 2022-12-09T04:43:16Z

client/app/bundles/course/statistics/pages/StatisticsIndex/index.jsx

@@ -70,11 +69,7 @@ const TabPanel = (props) => {
      role="tabpanel"
      {...other}
    >
-      {value === index && (
-        <Box sx={{ pt: 3, pb: 3 }}>
-          <Typography>{children}</Typography>


Actually is there any specific reason why Typography was used? Seems like the children is of JSX.Element type and if there’s div or other elements that cant be nested within p (default typography’s component), it will throw console error

ekowidianto · 2022-12-12T23:38:47Z

Btw @ekowidianto I think this PR is more or less ready to review.

@zhuhanming PR reviewed~ thanks

- fix proptypes shape error - change injectIntl to useIntl - remove obsolete underLinesOnHover props for <Link />

…level to student performance table

zhuhanming requested a review from ekowidianto December 4, 2022 20:25

zhuhanming changed the base branch from phillmont/migrate-assessment-show to master December 4, 2022 20:26

zhuhanming changed the base branch from master to phillmont/migrate-assessment-show December 4, 2022 20:27

zhuhanming force-pushed the hanming/add-course-statistics-2 branch from 52c919d to 1d63b11 Compare December 4, 2022 20:30

zhuhanming commented Dec 4, 2022

View reviewed changes

zhuhanming mentioned this pull request Dec 4, 2022

Add course statistics #4496

Closed

zhuhanming force-pushed the hanming/add-course-statistics-2 branch 2 times, most recently from cf9d23a to 338dad8 Compare December 5, 2022 01:53

zhuhanming changed the base branch from phillmont/migrate-assessment-show to master December 5, 2022 01:54

zhuhanming force-pushed the hanming/add-course-statistics-2 branch from 338dad8 to 4a5c8c1 Compare December 6, 2022 05:26

zhuhanming changed the base branch from master to phillmont/migrate-assessment-show December 6, 2022 05:26

ekowidianto requested changes Dec 9, 2022

View reviewed changes

ekowidianto reviewed Dec 9, 2022

View reviewed changes

purfectliterature force-pushed the phillmont/migrate-assessment-show branch 2 times, most recently from 570e6c8 to b1c6da4 Compare December 9, 2022 04:48

ekowidianto force-pushed the hanming/add-course-statistics-2 branch from 082bfb0 to 48d429d Compare December 9, 2022 06:43

purfectliterature force-pushed the phillmont/migrate-assessment-show branch 7 times, most recently from 5bfb659 to 13f225e Compare December 9, 2022 10:29

Base automatically changed from phillmont/migrate-assessment-show to master December 9, 2022 15:28

ekowidianto force-pushed the hanming/add-course-statistics-2 branch 3 times, most recently from 8004f49 to fd5e0e2 Compare December 10, 2022 15:18

zhuhanming and others added 17 commits December 22, 2022 13:21

chore: install chartjs deps

bea3cc9

feat: add chart-related components

5841f14

feat: add chart colors

6a39032

feat: add course stats fetching to statistics API

fccf1e8

feat: set up redux for course stats

2412c78

feat: add course stats UI

824f738

feat: update models to support course stats

30f9bd8

feat: add endpoints for course statistics

41727eb

tests: add basic tests for course stats

c7e4cd6

feat: shift some computations to jbuilder and remove repeated ops

fd5484a

feat: improve performance table to support tutor filtering

c9e801b

feat: sort managers by name

b48b3c4

fix(statistics): fix console error

8592007

- fix proptypes shape error - change injectIntl to useIntl - remove obsolete underLinesOnHover props for <Link />

refactor(course statistics): remove object destructuring

c094356

fix(statistics controller): fix duplicate title in joined table

89fb3ac

feat(statistics): add more information such as assessment count, max …

bb53f11

…level to student performance table

perf(student progression chart): add memo

4d69866

ekowidianto force-pushed the hanming/add-course-statistics-2 branch from fd5e0e2 to 4d69866 Compare December 22, 2022 09:11

feat(intl): regenerate translations

e5def8a

ekowidianto merged commit c3238f7 into master Dec 22, 2022

ekowidianto deleted the hanming/add-course-statistics-2 branch December 22, 2022 10:12

ekowidianto mentioned this pull request Dec 22, 2022

Course statistics TODO #5521

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add course statistics #5440

Add course statistics #5440

zhuhanming commented Dec 4, 2022 •

edited by ekowidianto

Loading

zhuhanming Dec 4, 2022

ekowidianto Dec 8, 2022

zhuhanming commented Dec 5, 2022

ekowidianto commented Dec 5, 2022

zhuhanming commented Dec 8, 2022

ekowidianto left a comment

ekowidianto Dec 9, 2022

ekowidianto commented Dec 12, 2022

Add course statistics #5440

Add course statistics #5440

Conversation

zhuhanming commented Dec 4, 2022 • edited by ekowidianto Loading

TODO (which I'll likely be unable to work on)

Course Performance Table

zhuhanming Dec 4, 2022

Choose a reason for hiding this comment

ekowidianto Dec 8, 2022

Choose a reason for hiding this comment

zhuhanming commented Dec 5, 2022

ekowidianto commented Dec 5, 2022

zhuhanming commented Dec 8, 2022

ekowidianto left a comment

Choose a reason for hiding this comment

ekowidianto Dec 9, 2022

Choose a reason for hiding this comment

ekowidianto commented Dec 12, 2022

zhuhanming commented Dec 4, 2022 •

edited by ekowidianto

Loading