{"id":773,"date":"2009-09-21T21:16:48","date_gmt":"2009-09-22T02:16:48","guid":{"rendered":"http:\/\/mikeconley.ca\/blog\/?p=773"},"modified":"2023-12-20T16:25:19","modified_gmt":"2023-12-20T21:25:19","slug":"integrating-pedagogical-code-reviews-into-a-cs-1-course-an-empirical-study","status":"publish","type":"post","link":"https:\/\/mikeconley.ca\/blog\/2009\/09\/21\/integrating-pedagogical-code-reviews-into-a-cs-1-course-an-empirical-study\/","title":{"rendered":"Integrating Pedagogical Code Reviews into a CS 1 Course: An Empirical Study"},"content":{"rendered":"<h3><a href=\"http:\/\/portal.acm.org\/ft_gateway.cfm?id=1508972&amp;type=pdf\">Integrating Pedagogical Code Reviews into a CS 1 Course:<br \/>\nAn Empirical Study<\/a><\/h3>\n<h4>by Christopher Hundhausen, Anukrati Agrawal, Dana Fairbrother, and Michael Trevisan<\/h4>\n<p><a href=\"http:\/\/portal.acm.org\/ft_gateway.cfm?id=1508972&amp;type=pdf\">Here&#8217;s another paper <\/a>talking about using code review in a first year programming course.\u00a0 Let&#8217;s give it a peek.<\/p>\n<p>The idea of this paper is to introduce a type of &#8220;studio-based&#8221; learning environment for programmers.\u00a0 The term originates from architecture, where architecture students bring various designs into class, and then the classroom as a unit critiques and evaluates the design.\u00a0 In the studio-based learning model, the instructor <em>guides<\/em> the critiques.\u00a0 This way, not only does the critiqued student get valuable feedback, but the critics also learn about <em>what to critique<\/em>.\u00a0 Remember when <a href=\"http:\/\/mikeconley.ca\/blog\/2009\/09\/13\/do-peer-code-reviews-seem-incompatible-with-the-traditional-classroom\/\">I talked about learning how to read code?<\/a> Studio-based learning seems to be one solution to that problem.<\/p>\n<p>Studio-based learning could also help foster community among the students.\u00a0 In a field where isolated cubicles are still quite normal (I&#8217;m sitting in one right now!), a sense of community could be a welcome relief.<\/p>\n<p>The paper also notes the following:<\/p>\n<blockquote><p>&#8230;the peer review process can (a) prepare students to deal with criticism [1], teach students to provide constructive criticism to others [1], provide students with experience with coming to a consensus opinion in cases where opinions differ [1], and build teamwork skills [8] . All of these are \u201csoft\u201d or \u201cpeople\u201d skills that are likely to be important in their future careers.<\/p><\/blockquote>\n<p>I whole-heartedly agree.<\/p>\n<p>The paper then goes on to give a quick survey of how peer review (not just for code, but peer review in general) is currently used in education:<\/p>\n<ul>\n<li>pair programming (which is what I experienced at UofT)<\/li>\n<li>&#8220;in-class conference&#8221; model<\/li>\n<li>peer feedback meetings (mostly for project documentation)<\/li>\n<li>web-based peer grading solutions (RRAS, and Peer Grader)<\/li>\n<li>peer code review<\/li>\n<\/ul>\n<p>Maybe I&#8217;m just getting tired of reading these papers, but I found this section particularly difficult to get through.\u00a0 I had to read it 3 or 4 times just to understand what was going on.<\/p>\n<p>Finally, they get on with it, and talk about their approach to peer code review &#8211; a student oriented version of a formal code review.\u00a0 Yikes &#8211; if by formal code review, they mean a <a href=\"http:\/\/en.wikipedia.org\/wiki\/Fagan_inspection\">Fagan Inspection<\/a>, then these poor students are in for some long meetings&#8230;<\/p>\n<p>Here is their 5 step outline of their approach:<\/p>\n<ol>\n<li>Plan the inspection of a specific piece of code.<\/li>\n<li>Hold a kick-off meeting with an inspection team to distribute the code to be inspected, train the team on the process, and set inspection goals.<\/li>\n<li>Have members of the inspection team inspect the code for defects on their own time.<\/li>\n<li>Hold an inspection meeting to log issues found by individual members&#8217; inspections, and to find additional issues.<\/li>\n<li>Edit the code to address the issues uncovered in the inspection, and verify that the issues have been resolved.<\/li>\n<\/ol>\n<p>Yep, that sounds a lot like a Fagan Inspection.\u00a0 As I&#8217;ve already found out, there are<a href=\"http:\/\/mikeconley.ca\/blog\/2009\/09\/14\/smart-bear-cisco-and-the-largest-study-on-code-review-ever\/\"> lighter, faster, and just-as-effective approaches<\/a> to peer code review&#8230;<\/p>\n<p>After using their approach, their study noticed:<\/p>\n<ol>\n<li>A positive trend in the quality of the students code, and a negative trend in the number of defects found per review<\/li>\n<li>Discussion transitioned from syntax and style to more high-level concepts, such as architecture and design of the system.\u00a0 Thus, meaningful discussion was generated using their technique.\u00a0 The study notes, however, that &#8220;&#8230;we<br \/>\ncannot provide any objective evidence that students were able to provide helpful critiques of each other\u2019s code within the code reviews themselves.&#8221;<\/li>\n<li>There is anecdotal evidence that suggests that this work helped to create more of a community feel among the students in the class<\/li>\n<\/ol>\n<p>Sounds like a lot of anecdotal evidence.<\/p>\n<p>I don&#8217;t know, reading the results of this paper, I was disappointed.\u00a0 The evidence\/data they collected feels light and insubstantial.\u00a0 I certainly support what the study was trying to do, and maybe I&#8217;ve read too many peer review papers, but I don&#8217;t find it surprising anymore to hear that peer code review improves students code.\u00a0 It&#8217;s good to have evidence for that, but I was hoping for something more.<\/p>\n<p>The paper closes with the writers almost agreeing with me:<\/p>\n<blockquote><p>There are several limitations to our results that suggest the need for more rigorous follow-up studies with both larger student samples and additional data collection methods.<\/p><\/blockquote>\n<p>I knew it wasn&#8217;t just me!<\/p>\n<p>Anyhow, the paper finishes with the writers outlining how they&#8217;d do future studies.<\/p>\n<p>And that&#8217;s that.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Integrating Pedagogical Code Reviews into a CS 1 Course: An Empirical Study by Christopher Hundhausen, Anukrati Agrawal, Dana Fairbrother, and Michael Trevisan Here&#8217;s another paper talking about using code review in a first year programming course.\u00a0 Let&#8217;s give it a peek. The idea of this paper is to introduce a type of &#8220;studio-based&#8221; learning environment [&hellip;]<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[454],"tags":[1218,489,490],"class_list":["post-773","post","type-post","status-publish","format-standard","hentry","category-code-reviews","tag-code-reviews","tag-fagan-inspection","tag-studio-based-learning"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/prmTy-ct","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/posts\/773","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/comments?post=773"}],"version-history":[{"count":14,"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/posts\/773\/revisions"}],"predecessor-version":[{"id":3222,"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/posts\/773\/revisions\/3222"}],"wp:attachment":[{"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/media?parent=773"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/categories?post=773"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mikeconley.ca\/blog\/wp-json\/wp\/v2\/tags?post=773"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}