Evaluation & Rubrics

Rubrics contribute to greater transparency by clearly articulating the criteria and standards used to evaluate students’ achievement of learning outcomes.

Rubrics as a synthesis of criteria and standards 

Rubrics can be understood as bringing together criteria and standards within a single framework for making judgements about student work.

In this sense, rubrics connect the two: they specify what is being judged and what different levels of achievement in relation to those qualities look like. This is what enables a rubric to support criterion-referenced and standards-oriented assessment. Rather than simply assigning a score to student work, a rubric provides an explicit framework for interpreting evidence of learning against defined qualities and levels of achievement.

Identifying Criteria

The criteria are the assignment expectations: the qualities the final work should display. The performance level descriptions instantiate those expectations at different levels of competence (Brookhart, 2018)

Assessment criteria identify the qualities, dimensions or features of student work that are important when judging achievement.  Criteria therefore express what matters in the work, whether it is an artefact, performance, or process. For example, the quality of an argument, the accuracy of an analysis, the effectiveness of a performance or practical demonstration, the application of disciplinary methods, or the rigour of a research, design, or reflective process. Identifying appropriate assessment criteria is one of the most important decisions in rubric design. Criteria should represent meaningful aspects of the intended learning rather than superficial features of task completion that are easier to observe or quantify. In other words, criteria should direct attention to the knowledge, skills, understanding, or capabilities that students are expected to demonstrate, directing attention to the achievement of intended learning outcomes.

The issue with the 'superficial' features detailed in this table is not so much that the things being measured are irrelevant. In some contexts, quantitative measures are entirely appropriate, and assessments may legitimately specify minimum standards or threshold requirements. For example, an assessment may require a minimum level of attendance, a specified number of components to be completed, a particular level of accuracy, or compliance with professional, technical or safety requirements. The issue is rather whether such measures provide meaningful evidence of the learning or quality of performance that the assessment is intended to evaluate. Things like attendance, word count and grammar can all matter. The key question when selecting criteria is therefore not simply whether a feature can be measured, but whether it provides meaningful evidence of the learning or quality of performance the assessment is intended to evaluate.

Practical Tip:  Where a requirement is necessary for the assessment but does not provide meaningful evidence of the learning or quality of performance being evaluated, it may be better specified as an assessment condition or threshold requirement rather than as part of a rubric criterion. This distinguishes requirements for completing the assessment from the dimensions of quality that form the basis of the judgement of achievement.

Reflection 

Take one of your existing rubrics and consider whether each criterion represents a meaningful dimension of the learning or performance you intend to assess. For each criterion ask:

• Does the feature represent a meaningful quality of the student’s work or performance?
• Is this something that an assessor needs to evaluate rather than simply check?
• Does the criterion provide relevant evidence of the intended learning outcomes?

Are there any measures or criteria that are primarily requirements, conditions or proxies rather than meaningful dimensions of quality?

From Learning Outcomes to Assessment Criteria

Identifying rubric criteria generally involves working backwards from the module learning outcomes, but this is not a matter of simply reproducing the outcomes as criteria; the criteria must also be tailored to the specific knowledge, skills and performances elicited by the assessment task. Ideally, the assessment task will be selected on the basis of its capacity to generate valid evidence of the intended learning: that is, it will provide students with an appropriate opportunity to demonstrate the knowledge, skills and capabilities articulated in the learning outcomes. In practice, however, assessment design involves a range of considerations beyond validity alone. Particular assessment formats may be selected because they are manageable in terms of learner effort and assessment workload, while in other cases they may reflect established disciplinary, professional or programme-level conventions. The process of developing assessment criteria must therefore take account not only of what the learning outcomes require students to demonstrate, but also of what the chosen assessment task actually enables them to demonstrate.

If learning outcomes describe what students should know, understand, or be able to do, assessment criteria identify what aspects of student performance will be judged as evidence of that learning. Biggs and Tang (2007) describe assessment as involving three related steps: identifying the criteria, selecting evidence that will be judged against those criteria, and then making judgements about the extent to which the criteria have been met.

A good starting point, when identifying criteria, might be simply to ask ourselves:

• What would I need to see in a student's work to conclude that this learning outcome has been achieved?

     And more specifically: 

• What aspects of the student’s product, process or performance would provide convincing evidence that the learning outcome has been achieved?

Notice that the criteria do not simply repeat the learning outcome. Rather, they identify the key qualities in the work that provide evidence of achievement. As mentioned above, it is always useful to distinguish between features that represent learning and features that merely reflect compliance with task instructions (assessment requirements). 

As a rule of thumb, criteria should focus on:

• Knowledge students are expected to demonstrate.
• Skills they are expected to apply.
• Qualities of work that distinguish different standards of performance or quality
• Evidence that is genuinely relevant to the learning outcomes.

Reflection

Take one of your existing rubrics and consider the relationship (or 'alignment') between your learning outcomes, assessment task, and criteria. Consider:

• What matters? What qualities of the student’s work are genuinely important to the intended learning?
• What would provide evidence? What aspects of the student’s product, process or performance would provide convincing evidence of that learning?
• Is this criterion appropriate to the specific task? How does this particular assessment task make the learning visible?
• Is this meaningful evidence? Does the criterion assess something that demonstrates learning or quality, rather than simply something that can be easily counted or checked?

Performance Level Descriptors

In addition to specifying the essential features, qualities or dimensions of learning to be assessed (criteria) the other defining feature of a rubric is its performance-level descriptors.

True rubrics feature criteria appropriate to an assessment's purpose, and they describe these criteria across a continuum of performance levels (Brookhart, 2018)

These describe what achievement of each criterion looks like at different levels of quality, enabling evaluators to make judgements about student work against explicit standards. The descriptors therefore do more than indicate that one level is ‘better’ than another: they make explicit how and why the quality of performance differs across levels. The effectiveness of a rubric depends on the quality of its performance descriptors. Descriptors should clearly articulate the characteristics of performance at different levels of achievement, enabling assessors to make consistent judgements. 

This not-entirely scientific analytic rubric illustrates the difference between Criteria, Performance-level descriptors, and achievement levels. 

A rubric for making a 'good' cup of tea 

Here's an example of an analytic rubric for that most essential of Irish competencies: the ability to make a good cup of tea. It is an “analytic” rubric because it breaks performance down into multiple dimensions or criteria, such as “tea selection,” “steeping time,” and “presentation”, against which judgements can be made. A more complete rubric would probably also consider factors such as water temperature and, of course, the actual taste! For each criterion, a set of descriptors at different performance levels (exemplary, beginning, etc.) describes what performance looks like at each stage. These descriptors may capture actions taken, skills demonstrated, or qualities evident in the tea and the process of making it. Together, they make the competency more concrete and observable, helping everyone involved understand what mastery looks like, and what they might do to move towards it.

The table below demonstrates a process of constructive alignment in rubric development. It traces a clear pathway from the competency being assessed ('making a good cup of tea'), through the evidence needed to judge performance, to the assessment metric and minimum acceptable standard. The rubric therefore emerges as the final assessment tool rather than the starting point, ensuring that criteria and performance levels are directly derived from the learning outcome and the required standard of achievement.

Performance Levels

There isn’t one universally prescribed set of performance levels for rubrics. The most important thing is that they provide a structured scale that is clearly differentiated and consistently applied across criteria. Sometimes performance levels are aligned with percentage thresholds for grades, or a rubric will use a scoring system that can help with weighting criteria and converting results into traditional grades. The best terminology to use depends on the purpose of the assessment, the need to align your rubric with your institution’s assessment regulations, and the message you want to convey to learners.

In the example used above, we have opted for developmental labels (e.g. beginning, developing, proficient, exemplary), rather than more standards or attainment-based labels (e.g. below expectations, achieved, exceeds expectations, advanced). By developmental, we mean that the levels communicate a progression in capability, helping learners to see where they are now and what increasingly sophisticated performance might look like. This can avoid overly negative language and encourage a sense of progression and growth, although descriptive and developmental labels aren’t always practicable or appropriate. You can also assign numeric values to descriptive labels, making the language and performance levels commensurable with percentages and grading schemes.

Of course, what your descriptors say at each level is generally more important than the labels you use here, the important thing being that evaluators and learners can clearly understand the described differences in quality, capability, achievement, or progression. All of the examples above use four levels, which is a common approach, but there is no hard-and-fast rule about the number of levels you should use. Depending on your purpose and context, you might use three, five, or more (though it is best to keep things as simple as possible).

Nor do performance levels need to be aligned with grades, percentages, or numerical scores. In many cases, descriptive levels can be used simply to communicate different degrees of performance or development. However, aligning levels with scores, grade bands or percentage thresholds can become tremendously useful when the rubric is also being used to make judgements about the standard of students’ work in ways that need to map onto your university’s grading system. The key is to make that alignment deliberate and transparent, rather than allowing the scoring system to drive the design of the rubric.

You can find information about TU Dublin's Grading system and GPA Model on our Assessment Regulations page.

Scoring and Weighting

A rubric has three essential features: evaluative criteria, quality definitions, and a scoring strategy (Popham, W.J. 1997)

So far, we have mostly focused on rubrics as tools for making qualitative judgements about the quality of students’ work against clearly defined assessment criteria. This is an important part of what rubrics do: they help us articulate what we value, recognise different qualities of performance, and make our judgements more consistent and transparent. But rubrics also have to operate within the wider systems of higher education assessment. At some point, our professional and qualitative judgements about a student’s work need to be translated into standards-oriented outcomes, such as a percentage, grade, grade band, or classification, that can be used for purposes such as certification, progression and award. This inevitably involves some form of scoring, and often some decisions about the relative weighting of different criteria.

In essence, rubrics sit at the intersection of qualitative judgement and measurement: they help us interpret evidence and make judgements about the quality of a performance or artefact, while also providing a means of scoring, ranking and relating those judgements to defined standards. Reconciling these different purposes is not always straightforward and we may never achieve a perfect balance between them.

The challenge, then, is to make this translation in a way that supports rather than undermines the quality of our judgement. A rubric should not reduce assessment to a mathematical exercise: the numbers are a means of representing and communicating our judgements, rather than replacing them. 

Aligning performance levels with grades or percentages

As we saw above, it is relatively straightforward to align performance levels with percentage ranges or grade bands. This is perhaps the most basic way of translating the judgements captured in a rubric into results that can be reconciled with your university’s grading, classification or GPA system. For example, as in the table below, a rubric might define four levels of performance (e.g. beginning, developing, proficient, and exemplary )and associated each with a corresponding grade or percentage band. This allows the descriptive language of the rubric to sit alongside the more formal language of institutional grading, providing a bridge between qualitative judgements about performance and the grades that ultimately need to be recorded and reported.

Of course, as an analytic rubric, of the type describe earlier on this page, uses multiple criteria, each of which has its own performance levels, you will need a mechanism for bringing those separate judgements together into an overall result. For example, our tea-making rubric has three criteria:

If a student is Proficient in tea selection, Exemplary in steeping, and Developing in presentation, you now have three judgements to reconcile. One could of course just take midpoint of each band work out an average, but this raises a number of questions such as:

• Is it fair to use a midpoint when the percentage points are so wide?
• Does an average accurately represent the overall quality if, say, a learner was exemplary in one important area but weak in another?
• What happens at grade boundaries, should we just override the rubric?
• Are all of the criteria equally important? Do we need to weight them according to their relative importance and contribution to the learning outcomes?

Ultimately, we also need to step back and ask ourselves about the relationship between our percentage bands and performance levels. Once we start introducing numbers into our rubrics, we risk making the rubric seem much more objective and precise than the underlying, qualitative judgement may be. You may recognise this tension from your own assessment practice: the judgement we make can be nuanced, contextual and sometimes tacit, while the rubric we use to represent it can give the impression of mechanical precision and objectivity.

Think about your own experience of marking: how often does the final number feel rather more definite than the judgement that produced it?

The point here is that while aligning individual performance levels with grades is easy; deciding how those individual judgements should be aggregated into an overall grade is where the design of a rubric becomes more challenging. If you’re not a naturally quantitative person, this may be the point where your eyes start to glaze over. But it is worth resisting the temptation to get too hung up on numerical precision. More points, narrower percentage bands and increasingly elaborate calculations can create an illusion of objectivity without necessarily making our judgements any more accurate or defensible. Ultimately, a rubric is a tool for supporting informed qualitative judgement. The goal is not to make our judgements appear more precise than they really are, but to make them fair, rigorous, transparent and defensible, and to ensure that the inferences we make about students’ learning are valid. In other words, we should focus less on whether our rubric can distinguish between a 67 and a 68, and more on whether the assessment gives us good evidence of the knowledge, skills and capabilities we actually intend to assess, and whether we are using that evidence appropriately to make decisions about students’ achievement.

Some common approaches to translating our qualitative judgments into final grades

1. Avoid the false precision trap 

One of the key challenges in aligning our rubrics with an eleven-point grade scale is that, as you can probably imagine, trying to build all eleven distinctions directly into the rubric would quickly make it rather unwieldy for us, and potentially confusing for learners. So, we need to start by accepting that a rubric with three, four or five performance levels simply cannot, and doesn’t need to, distinguish reliably between every percentage point on a 0–100 scale. The aim is to capture meaningful differences in the quality of performance, rather than to manufacture a level of precision that the rubric cannot really support.

When designing rubrics, our focus should be on making robust judgements about quality, rather than attempting to justify highly precise marks. Inevitably, some level of professional judgement will be required to determine where a learner’s work ultimately falls within a given percentage or grade band.

And that’s OK! indeed, it is part of what we are there to do as educators. We bring disciplinary knowledge, experience and professional expertise to the process of interpreting evidence of learning and deciding what standard that evidence represents. A good rubric should therefore support and structure our professional judgement, rather than try to replace it with a calculation.

2. Consider grading by band rather than percentage 

The two types of rubrics most commonly found in higher education practice are broadly defined as 'analytic' rubrics and 'holistic' rubrics. Regardless of the type employed, rubrics will generally include the following essential components:

Analytic Rubrics

An analytic rubric is, in essence, a multidimensional rubric that breaks down performance or achievement into multiple criteria and provides a framework for making judgements about the quality of performance in relation to each criterion. These individual judgements are then brought together to inform an overall evaluation of the student’s work. It is termed “analytic” because the overall evaluation is broken down into different dimensions of performance, allowing each to be considered separately before the judgements are brought together to inform an overall evaluation, in much the same way that we might analyse something complex by breaking it into its component parts and considering how those parts work together.

Here is an example of a (rather complex) analytic rubric for an assessment with the following key learning outcome 'Select and combine herbs and spices effectively to create a balanced and flavourful home-cooked dish, and explain the reasons for your choice'.

So an analytic rubric is, in essence, distinguished by having multiple criteria (seen here in the left column) and associated descriptors for each criteria at each level of performance. They allow for more detailed feedback on different aspects of students’ work, can support more consistent scoring, help align assessment with multiple learning outcomes and skills, and give students a clearer understanding of how their work has been evaluated, including its strengths and areas for improvement.

Reflecting on rubric performance levels and descriptors

In many cases, the quality of a rubric depends less on the choice of criteria than on the clarity of the distinctions between performance levels. Developing these distinctions often requires iterative refinement, discussion with colleagues and testing against examples of actual student work to ensure that the standards are meaningful, defensible and usable in practice. When creating or reviewing an analytic rubric you might start by asking questions such as:

• Can I clearly explain what distinguishes each level of performance?
• Do the levels show meaningful differences in quality or achievement?
• Would different assessors interpret the differences in similar ways?
• Do the descriptors describe the work, rather than the student?
• Could a student use the descriptors to understand how to improve?

Holistic Rubrics

Another, less common type of rubric you may encounter in higher education assessment is the holistic rubric. It is described as 'holistic' because, in contrast to an analytic rubric, it does not break performance down into multiple separate criteria, each of which is considered independently. Instead, the assessor considers the student’s work as a whole and makes an overall judgement about its quality or level of achievement, using a single descriptor to describe what that level of performance looks like. The focus is therefore on the overall quality and coherence of the work, and on the extent to which it demonstrates the intended learning as a complete piece of work. In essence, rather than asking “How well has the student performed against each individual criterion?”, a holistic rubric asks “How well does this piece of work, taken as a whole, demonstrate the intended learning?”

Here is an example of our rubric for preparing a flavourful home-cooked dish, converted into a 'holistic' approach.

Holistic rubrics are generally more efficient to design and faster to apply during grading, making them well-suited to large classes, smaller formative tasks, or low-stakes assessments. They simplify the assessment process and are especially appropriate when the overall quality of work is the main concern, rather than specific elements. However, they tend to offer less detailed feedback for students and allow for greater subjectivity in evaluation. There is strong support for using holistic rubrics for creative tasks where it's hard to break performance down into rigid criteria and the overall impression or quality of the work may be more meaningful than measuring technical accuracy against strict criteria. They're also practical for live assessments, like oral presentations or performances, where educators need to make quick, consistent judgements, sometimes called 'rapid grading scenarios'.

Of course, these limitations should be understood in context. A rubric is primarily a tool to support professional judgement and evaluation, rather than a complete account of every aspect of a student’s performance. Where greater detail or diagnostic feedback is needed, written feedback can complement the rubric by drawing attention to particular strengths, areas for development, and next steps for learning.

Using the Brightspace Rubrics Tool

Whether or not you intend to assess directly within the VLE, Brightspace provides a capable rubric tool for building both analytic and holistic rubrics, enabling you to define criteria and performance levels, attach descriptors and feedback to levels of achievement, allocate weights or points where appropriate, and apply rubrics consistently when evaluating student work. Here are some resources to get you started:

• Create a Rubric using the Rubrics Tool (Webpage) from D2L, makers of Brightspace.
• Create a holistic rubric (YouTube video) from D2L
• Create an analytic rubric (YouTube video) from D2L
• Create a weighted rubric (YouTube video) from D2L
• Add a rubric to an existing activity  (YouTube video) from D2L
• TU Dublin guide 'Rubrics on Brightspace' (MS Sway Presentation, requires login)

Using the MS CoPilot Rubric Tool

If you are new to creating rubrics, you can use MS Copilot’s teaching tools to generate a draft rubric based on an existing assessment. This can be a useful way to explore different possibilities for rubric type, criteria, achievement scales, performance levels, scoring, accessibility and so on. However, it is important to treat AI-generated rubrics as starting points rather than finished products. Use the tool to generate ideas, test alternatives and act as a sounding board, but apply your own academic and disciplinary judgement to decide what should be assessed, what constitutes meaningful evidence of learning, and how different levels of achievement should be described. 

To create or edit assessment rubrics using Microsoft Copilot, follow these steps:

• Open Microsoft Copilot and sign in using your TU Dublin credentials.
• Once logged in, navigate to the 'Teaching Tools' area: In the top-left corner of the screen, locate the 'Teach' menu, which appears beneath the 'New Chat' button.
• Select 'Teach' to access the teaching-focused tools and features.
• From the available options, choose 'Homework and Assessments'.
• Within the Homework and Assessments section, select 'Rubric' You can then:
  • Create a new grading rubric for an assessment.
  • Generate rubric criteria and performance descriptors using AI assistance.
  • Edit and refine existing rubrics to align with module learning outcomes and assessment requirements.
  • Review the generated rubric before applying it to your assessment activity.

Tip: Before generating a rubric, have your assessment brief, learning outcomes, and grading criteria available. Providing this information to Copilot will help produce more relevant and accurate rubric recommendations. 

You can find a guide to the Copilot rubrics tool here. 

'Creating Effective Rubrics' with Linda Suskie (Webinar Recording)

• Sader, D.R (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119–144. http://www.jstor.org/stable/23369143
• Brookhart SM (2018) Appropriate Criteria: Key to Effective Rubrics. Front. Educ. 3:22. doi: 10.3389/feduc.2018.00022
• Popham, W. J. (1997). What's wrong--and what's right--with rubrics. Educational Leadership, 55(2), 72-75. https://tudublin.idm.oclc.org/login?url=https://www.proquest.com/trade-journals/whats-wrong-right-with-rubrics/docview/224857225/se-2?accountid=10594
• Biggs, J. B., Tang, C. S., & Society for Research into Higher Education. (2007). Teaching for quality learning at university (3rd ed.). Open University Press.
• University of Reading CQSD (n.d.) Developing and Using Assessment Criteria and Rubrics, Focus on: Assessment and Feedback. https://www.reading.ac.uk/cqsd/-/media/project/functions/cqsd/documents/ade/tandl-resources/prp-assessment-criteria-and-rubrics.pdf?la=en&hash=6ED29C8090668F9E63044BFBAE9CC2E3