The student-staff pairing for Law worked on assessment design for post-graduate law students. This example is from a Criminology module.

Please explore the links below to find specific information you are looking for on this page.

Learning Outcomes

The learning outcomes for this module included the following:

On successful completion of this module, students should be able to:

• Outline and trace changes in criminological theories over time
• Link these changes to shifts in societal concerns over criminality
• Evaluate the applicability of criminological theories to high profile Irish and international cases.

Current Assessment Design

Summative assessment, in the form of a 4000-word take-home essay, that accounts for 70% of the overall grade, and an oral exam, that accounts for 20%. Formative assessment, in the form of attendance and participation, that accounts for 10%.
For the purposes of this research, the student-staff pairing focused on the take-home essay.

Assignment Task

The essay question was as follows:

A number of the theories covered in this module have situated criminal behaviour in its social, cultural, political and/or economic context in order to understand it. Do you think that this wider context is useful?

Discuss with reference to at least one of the theories that you have encountered in this course.

Associated Risk of GenAI

The take-home essay is one of the more high-risk assessment styles. Students may attempt to use the GenAI to generate complete or partially plagiarised responses to the essay question.

Breakdown of process, completed by student participant

In the first instance, the student pasted the essay question directly into ChatGPT and was provided with a brief, two-paragraph answer.

The student clarified that the task was intended for an assignment within the context of law school in Ireland, expanding the response to three paragraphs accordingly.

Subsequently, the student emphasised that the assignment was intended student studying at a postgraduate level, requiring both depth and critical analysis. The response incorporated more critical terminology and phrases but retained a predominantly informative tone.

The student then requested the inclusion of legal sources in the answers. The sources provided were primarily secondary, consisting of journals and books. While some sources appeared genuine, others lacked verifiability through search results.

It was conveyed by the student that the answer must be at least 1200-words. However, the provided response fell short, only amounting to 500 words.

Following a critique of this initial response, the user reiterated the need for a 1200-word assignment. In response, a 600-word assignment was delivered.

Subsequently, the student instructed the AI to compose the first half of the assignment within 600 words. The AI initially produced 400-500 words for the first half and then indicated readiness to proceed with the second half, ultimately delivering 330 words, with a maximum word count limit of 750 for the entire assignment.

The student then copied the content into another ChatGPT window and sought the AI's opinion, providing context. ChatGPT assessed the response as a 1.1 standard answer, noting its nuanced approach.

Observations

The answer provided by ChatGPT initially appeared to be quite insightful and to be quite specific to Irish law, incorporating relevant authors.

However, on closer inspection, it was quite repetitive in its phraseology and showed no real depth in its critical analysis. Also, many of the authors referenced by ChatGPT appear to be fabrications.

The essay was well-structured, each aspect of the answer was clearly broken down in indefinable ways.

The language and style are formal in tone consistent with legal writing however, ChatGPT had significant issues creating an answer of a suitable length.

Despite being asked, it did not seem capable of being able to provide the critical element required to bring this answer up to the required standard.

In this essay, ChatGPT was poor at providing reputable references, and appears to have randomly generated a referencing style.

ChatGPT unquestionably possesses the skill to craft a strong law assignment. The way it constructs sentences and presents an argument is of a high standard. However, the issue lies with its depth of knowledge. The arguments, while well-articulated, lack any real substance.

Alternative Download