Quality Performance in the Immigration Appeal Division 2024

On this page

• Overview
• Assessment methodology
• Limitations
• Results
• Summary of results
• Pre-proceeding readiness
• Fair and respectful proceedings
• Focused and robust proceedings
• Reasons state conclusions on all determinative issues
• Decisions provide findings and analysis necessary to justify conclusions
• Reasons are transparent and intelligible
• Recommendations
• Management response and action plan
• Annex A – IAD performance indicators

Overview

This report describes the results of the measurement of quality in decision-making in the Immigration Appeal Division (IAD). This report aims to provide a perspective to improve the Division's overall performance.^{Footnote 1}

To ensure neutrality, quality, and consistency in the assessment, two independent reviewers, Faisal Bhabha, and Anne Levesque performed the assessment. The reviewers were selected based on their in-depth knowledge of administrative law, professional ethics, legal pedagogy, and access to justice particularly for marginalised litigants.

Faisal Bhabha is an associate professor at Osgoode Hall Law School, York University. He also serves as the Faculty Director of the Professional LLM in Constitutional Law. He teaches and researches in the areas of constitutional law, employment law, legal professionalism, and access to justice. Previously, he sat as Vice-chair of the Human Rights Tribunal of Ontario (2008-2011).

Anne Levesque is an associate professor in the French Common Law Program at the Faculty of Law at the University of Ottawa. She has also served as Academic Director of Experiential Learning at the Faculty. Her research and publications focus on human rights, public interest litigation and access to justice. Professor Levesque is also the founding Director of the Legal Practice Program at the University of Ottawa.

The Audit and Evaluation Team provided the statistics found in the table accompanying each result section as well as the information in 1.0 “Context” and collaborated with the reviewers on 2.0 “Summary of results”. However, the findings in this report, including all strengths, areas for improvement and recommendations are solely those of the third-party reviewers.

Assessment methodology

The study reviewed 50 randomly selected appeals that were held before a single member and were decided on their merits (e.g. final decision of allowed or dismissed) between April 1, 2023, and February 29, 2024. The cases were balanced in proportion to region, language, and case type. Hearings between 45 minutes and seven hours were included. The average hearing length was three and a half hours.

The following charts illustrate the sampling design:

Regional office

Language of proceeding

Appeal type

The reviewers examined all evidentiary and administrative materials on file; listened to the complete audio recordings and reviewed the written decision; and assessed these against qualitative indicators in an assessment tool developed by the Strategic Planning, Accountability and Reporting (SPAR) Directorate and approved by the Deputy Chairperson of the IAD (see Appendix A). The assessment tool assesses 27 indicators across six performance categories. Fourteen of the indicators are mandatory for assessment, and thirteen are assessed only when applicable.

1. Pre-proceeding readiness
2. Fair and respectful proceedings
3. Focused and robust proceedings
4. Reasons state conclusions on all determinative issues
5. Decisions provide findings and analysis necessary to justify conclusions
6. Reasons are transparent and intelligible

Each indicator is assessed on a 1-to-3 rating scale. The 1-to-3 rating scale is as follows:

1=Does not meet expectations: The quality requirement was not met. The evidence showed one or more key instances where the proceeding or reasons would have markedly benefited had this requirement been met. There may have been an effort to apply the requirement, but the level of achievement fell short of expectations.

2=Meets expectations: This is a level of acceptable achievement. On balance, the member satisfied this quality requirement though there is margin for minor improvement.

3=Exceeds expectations: This is a level of consistent, above-average achievement. The evidence shows a grasp of the quality requirement and an understanding of its importance to a high-quality proceeding or decision, as the case may be.

Results are also expressed as a percentage of cases that meet expectations. A hearing is considered to meet the quality standard when 80% or more of the indicators for that hearing score a 2 or higher.

In support of Gender Based Analysis Plus (GBA+), the assessment tool includes items which directly address matters of accommodations, self-represented persons, intersectionality, and where applicable, other aspects of the Chairperson's Guidelines.

Methodology recommendation

The reviewers commend the IRB for including GBA Plus factors in the assessment tool as it is a testament of the high priority they are afforded in decision making. That said, the reviewers are of the view that intersectionality and trauma informed factors may be more suitably assessed in the performance category relating to how the hearing is conducted. To that end, in future quality performance assessments, the Audit and Evaluation Team may consider moving the indicator relating to intersectionality and trauma informed factors under the category of “Focused and robust proceedings”.

Limitations

The goal of the study was not to generate statistics but to identify areas of strength, areas for improvement, and patterns in decision-making quality. To mitigate the inherent limitations of qualitative research, detailed performance indicators and guidance were provided to the assessors to help focus the assessment. Moreover, the assessors were provided with orientation by the Division, and data was checked for quality by the Audit and Evaluation Team.

A small sample size limits the inferences that may be made about the broader caseload. Where sample sizes are too small for ‘if applicable' indicators, observations or recommendations may still have been provided but these are not based on representative findings. Observations do not lend themselves to firm conclusions on legal matters such as the correct application of the law, the weighing of the evidence, or the fairness of the proceedings from a natural justice perspective. Only a court reviewing the case can arrive at such conclusions.

Results

Summary of results

The primary performance target for this assessment balances realism with aspiration. It aims for 80% of cases within the sample to meet the quality assessment. A case meets the quality standard when 80% or more of the indicators for that case score a 2 or higher. This target was achieved with 94% (47 out of 50) of cases meeting or exceeding the expectation. Where an indicator had many cases that did not meet the target, it is addressed in the reviewer's observations following the table (“What we can improve”).

It is also noteworthy that a high number of cases in the sample earned a rating of “exceeds expectations” in one or more of the indicators (nearly 10%). This is indicative of a highly talented and experienced roster of members who excel in the broad range of skills relating to administrative decision making, including active listening; empathy and self-awareness; active adjudication, including issue framing; substantive impartiality, legal reasoning and point form writing.

What we did well

• Members were well-prepared for their hearings and demonstrated a good understanding and familiarity with the file.
  
• Members were strongly committed to resolve technical issues when they arose.
  
• Most members provided accessible explanations to witnesses on sworn testimony. When relevant, most members adapted hearings for self-represented appellants, to ensure that the latter understood the proceedings.
  
• Members generally began hearings by explaining the process and how the proceedings would unfold.
  
• Members used point first writing, and clear and concise headings to organise their reasons.
  
• In their reasons, members made clear connections between testimony and documentary evidence adduced during the hearing to the factual findings necessary to reach their conclusion.
  
• Reasons were written in an accessible language to the parties, including when self-represented, and to the general public. A strong uniformity in style and structure helped make the decisions easier to read, and easier to understand.
  

What we can improve

• Members do not always make use of available technical support when facing technical issues, solving them on their own. These issues could be resolved more quickly and efficiently using technical support.
  
• The reviewers observed uneven treatment of self-represented or under-represented appellants among the cases reviewed and identified conditions in some cases that increased obstacles for these appellants.
  
• Few members intervened during hearings and, before final representations members rarely clearly stated the issues to focus on.
  
• Better and, sometimes, more active adjudication would have improved some hearings.
  
• Awareness of intersectional and trauma informed factors in assessing testimony can be improved.
  

Pre-proceeding readiness

Why measure this

The groundwork for quality is set before the hearing when the member prepares for the proceeding and understands the facts and key issues of the case. Indicator 1 of this study assesses this principle.

Considerations

Indicator 1 applies to all cases.

Overall result

General observations

In all the matters reviewed, members consistently demonstrated thorough preparation for the hearings. They were well-versed in the issues at hand, fully aware of the decision on appeal, and familiar with the evidentiary record. Reviewers commend the members for their diligent efforts in preparing for these hearings, noting their strong understanding of the relevant jurisprudence and guidelines. Given this high level of preparation, members could further enhance the effectiveness of the hearings by clearly stating the legal issues to the parties at the outset, explaining the applicable legal tests, and specifying the type of evidence they wish to hear. They could also be sure to be familiar with and avail themselves of technical support in the event of a mishap. Both areas will be elaborated upon further in other sections of this report.

What we did well

Members were well-prepared for the hearings, demonstrated a solid understanding of the issues, and were familiar with both the decision and the evidentiary record. Reviewers praised their strong grasp of relevant jurisprudence and guidelines. This level of preparation likely represents countless hours of unseen work, and members should be commended for their dedication and diligence in ensuring they are fully equipped to handle the complexities of each case.

What we can improve

Members either met or exceeded exceptions relating to this indicator in all of the matters sampled. As such, no additional measures are required to improve upon timely and complete pre-hearing readiness.

Fair and respectful proceedings

Why measure this

Individuals appearing before the IRB expect that they will be treated with sensitivity and respect. Any shortcoming in this regard potentially undermines tribunal integrity and public confidence. Indicators 2 to 11 of this study apply these principles in the context of IRB decision-making.

Considerations

Indicators 2, 4, 5, and 11 are considered universal—they apply to all cases. Indicators 3, 6 to 10 are assessed as applicable.

Overall result

General observations

The reviewers observed several areas of both commendable performance and potential improvement in the hearing process. While members were proactive in resolving technical issues and demonstrated good familiarity with online platforms, there were recurring problems with maintaining communication during hearings, leading to lost time.

The high proportion of self-represented appellants, many of whom face intersecting barriers such as addiction, mental health issues, disability, and trauma, has highlighted a need for measures to be taken to ensure the fairness of the hearings for these individuals. These individuals often do not necessarily require a designated representative, but they do require assistance, as a matter of natural justice, to understand the legal tests, evidentiary requirements, and the mechanics of the overall hearing process. Current practices reveal uneven treatment of such individuals. Some members excelled at adapting their hearings to respond to the needs of unrepresented and under-represented parties, while others fall short.

What we did well

Members showed a keen willingness to address and resolve technical issues, displaying a strong familiarity with online platforms and problem-solving skills when difficulties arose.

Most members took care to explain in accessible language to witnesses what it means to give sworn testimony and how to testify through an interpreter.

Members who excel in adapting hearings for self-represented parties demonstrated a strong ability to address the unique needs of individuals facing significant barriers. For example, some members clearly explained to litigants the different parts of the hearing, the issues in dispute and the possible outcomes of the hearing. Members who exceeded expectations in relation to these indicators described in accessible language the burden of proof, the elements of the applicable legal test and the factors that would be relevant in the decision to be made. These explanations helped make the hearings more efficient and the testimony more focused.

What we can improve

When facing technical issues, members had different strategies to resolve them. Some of them used the technical support that is made available to them, while other resolved those issues on their own. Making use of available technical support may help members to resolve communication issues more efficiently and reduce lost time during hearings.

There is a need for a more consistent approach in dealing with marginalised appellants during hearings, particularly those who are self-represented and/or those who face intersecting barriers.

Recommendations

• The IAD should consult with members to understand the reasons behind the underutilization of available technical support during hearings and to find ways to increase its effectiveness.
  
• The IAD should collaborate with members to develop and implement comprehensive best practices and resources specifically designed for handling self-represented parties, especially for those facing intersecting barriers including addiction, mental health, disability, and trauma. For example, it could consider preparing a job aid or checklist for members for communicating information about the nature of the proceedings, the relevant legal and factual issues, the possible outcomes and the availability of appeal or judicial review routes.
  

Focused and robust proceedings

Why measure this

Proceedings that are efficient and well managed create favourable conditions for quality outcomes to emerge and support the IRB's efforts to make the most effective use of its resources. Indicators 12 to 17 of this study apply these principles in the context of IRB decision-making.

Considerations

Indicators 12 and 13 are considered universal—they apply to all cases. Indicators 14 to 17 are assessed as applicable.

Overall result

General observations

Members generally began hearings by explaining the process and how the proceedings would unfold. Some explained the legal test and the evidence required to make a decision. A small number of members intervened during the course of the hearing to seek to focus the evidence or narrow the issues. Though this is not an indicator that was formally assessed, it does constitute a best practice in active adjudication.

The reviewers identified conditions in certain hearings exacerbating obstacles for self-represented appellants. This underscores the need for members to make better use of active adjudication, to preserve fair and respectful hearings, and take action to promote access to justice.

What we did well

Most members in the cases sampled began hearings by going through the evidentiary record to ensure that all the parties had access to the full and complete record. Members generally described the process and their role to the parties. Some members clearly set out the issues to be determined.

Nearly all members explained to witnesses in clear and accessible language what it means to give sworn testimony and provided guidance on how to testify through an interpreter when that was the case.

Members generally concluded the hearing by explaining to the parties the next steps in the process and advising them of the expected timeline by which they would receive a decision.

What we can improve

The reviewers observed that few members intervened during the course of the hearing and members rarely clearly state before the final representation what issues they wanted the submissions to focus on.

Members can therefore improve how they actively focus the hearing:

• Some cases would have improved with better active adjudication from members, particularly during the course of the hearing and before the final representations. This could include clearly outlining the legal issues at the start of the hearing, explaining the applicable legal tests specifying the type of evidence needed and clearly telling the parties what issues they ought to focus on during their final representations.
  
• Members need to be proactive, intervening when necessary to maintain civility and fairness for appellants, particularly during cross examination. When the appellant is self-represented, these issues have the effect of exacerbating obstacles while not manifestly serving the interests of justice.
  

Adopting these practices would not only help ensure that all parties, especially those without representation, have a better understanding of the process and can participate more effectively, but it could also lead to more efficient hearings, as the evidence presented, and the closing submissions would be more focused.

Recommendations

• The IAD should work with members to develop best practices and training in active adjudication, specifically focusing on how to effectively open and direct hearings. These best practices should include clear explanations of the legal issues to be determined; the types of evidence that need to be adduced to address the legal issues; the applicable legal tests as determined by statute and binding precedent; and the different parts of a hearing. It is also a best practice to provide such explanations as needed throughout the hearing including before the final representations.
  
• The IAD should work with members to develop strategies and identify best practices in effectively managing interactions between parties to a hearing, aiming to promote civility and equitable treatment.
  

Reasons state conclusions on all determinative issues

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility and transparency of the decisions of an administrative tribunal. Through indicators 18 and 19, this study applies the Court's requirement in the context of IRB decision-making.

Considerations

Indicators 19 and 20 apply to all cases.

Overall result

General observations

The length of the reasons was generally appropriate considering the complexity of the matters and evidentiary record. Most decision makers were methodological in their analysis of factors relevant to their decisions particularly in the context of matters involving humanitarian and compassionate groups.

Substantively, reasons were justifiable, intelligible and transparent. While reasons did not always refer to binding precedent, this choice was appropriate since many litigants before the IAD were often self-represented. The legal test that was applied was always articulated in accessible language and the factual findings supporting the conclusions were clearly stated. Reasons were organised in a manner that made it easy to follow the reasoning and logic applied.

What we did well

In legal writing, form often impacts substance. In that regard, most members excelled at point first writing and used clear and concise headings to organize their reasons in a way that facilitated the understanding of the underlying reasoning and logic and made the factual findings clear.

Generally, decision makers struck the appropriate balance of addressing all relevant factors and keeping decisions concise to promote efficiency in the process.

What we can improve

While nearly all members excelled in point first writing, a very small proportion of reasons did not use the point first written style or accessible headings that make the underlying reasoning and logic easier to understand. The IAD may consider capitalizing on the talent and expertise of most members and showcasing the decisions that exemplify the best use of point first writing and effective headings during future training.

Recommendation

In light of the favorable results of the assessment relating to these indicators, no formal recommendation is required relating to the reasons.

Decisions provide findings and analysis necessary to justify conclusions

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility, and transparency in a decision of an administrative tribunal. Through indicators 20 to 24, this study applies the Court's requirement in the context of IRB decision-making.

Considerations

Indicators 20 and 21 apply to all cases. Indicators 22 to 24 are only assessed if applicable.

Overall result

General observations

The members all appear to abide by best practices in legal writing, employing language that is accessible to the broadest possible range of individuals in the general public. Connecting testimony and documentary evidence adduced at the hearing to the factual findings necessary to reach legal conclusions is one of the most important things a member does to ensure their decision will hold and not be vulnerable to misunderstanding by the parties and the public, let alone open to judicial review. The decisions in the sample, overall, drew out such connections in writing, making it clear what the basis was for the particular conclusion.

What we did well

The need to justify decisions with factual findings and legal analysis is well understood by members and is demonstrated in their written decisions. Although many of the cases reviewed could be plausibly decided either way (given the legal standard in most of the cases is whether the decision constitutes a valid exercise of discretionary power), members' diligence in appropriately explaining and justifying their decisions helps ensure that decisions are easy to understand.

What we can improve

Some members could benefit from an increased awareness of intersectional and trauma informed factors when assessing testimony. The difference between the members' experience and culture and those of the appellants could lead to false assumptions or stereotype-based reasoning that bias members' assessment of testimonies and their decision-making. The members' decision to prefer other evidence to witnesses' testimony, and their assessment of that testimony, should always be explained with sufficient details in the reasons.

Recommendation

Considering the positive results of the assessments relating to these indicators, no formal recommendation is required with regards to the findings and analysis justifying conclusions.

Reasons are transparent and intelligible

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility, and transparency in a decision of an administrative tribunal. Through indicators 25 to 27, this study applies the Court's requirement in the context of IRB decision-making.

Considerations

All indicators apply to all cases.

Overall result

General observations

The members performed extremely well by consistently using clear, transparent and intelligible writing in their decisions. Even where there were shortcomings in relation to accounting for intersectional or trauma factors, the reasoning was transparent, and the writing was intelligible.

What we did well

In all matters that were assessed, the reasons were drafted in language that is accessible to the parties, even when self-represented, and the general public. Written decisions of the IAD tend to follow a predictable structure in terms of the sections and the organization. This helps whether reading one decision or dozens of decisions. The reviewers also observed a strong uniformity in style and structure which helped make the decisions easier to read, easier to understand, and more likely to be cited positively.

What we can improve

Members either met or exceeded expectations relating to these indicators in all of the matters sampled. As such, no additional measures are required to improve upon timely and complete pre-hearing readiness.

Recommendation

Considering the positive results of the assessments relating to this indicator, no formal recommendation is required with regards to the findings and analysis justifying conclusions.

Recommendations

The reviewers' recommendations relate to the following areas, and have been listed in order of suggested priority:

A. Develop and implement comprehensive best practices and resources for hearings involving marginalised parties:

Recommendation 1: The IAD should develop and implement comprehensive best practices and resources applicable to hearings involving marginalised parties particularly those who are self-represented and/or who facing intersecting barriers including addiction, mental health, disability, and trauma. These practices and resources should aim to promote substantive impartiality that is to level the playing field and ensure the fairness of hearing for marginalised litigants. For example, it could consider preparing a job aid or checklists for members for communicating information about the nature of the proceedings, the relevant legal and factual issues, the possible outcomes and the availability of appeal or judicial review routes.

Recommendation 2: The IAD should work with members to develop strategies for effectively managing interactions between participants to a hearing and ensuring civility and equitable treatment.

B. Develop and implement consistent practices in active adjudication:

Recommendation 3: It is recommended that the IAD work with members to develop best practices and training in active adjudication, specifically focusing on how to effectively open and direct hearings. These best practices should include how to provide clear explanations of the legal issues to be determined, the types of evidence that need to be adduced to address the legal issues, the applicable legal tests as determined by statute and binding precedent, and the various functions to discharge through the stages of the hearing.

C. Improve the effectiveness and use of technical support:

Recommendation 4: The IAD should consult with members to understand the reasons behind the underutilization of available technical support during hearings. By identifying and addressing these barriers, the IAD can enhance the effectiveness of technical assistance, ensuring that members can resolve communication issues more efficiently and reduce lost time during hearings.

Management response and action plan

Recommendation 1

The IAD should develop and implement comprehensive best practices and resources applicable to hearings involving marginalised parties particularly those who are self-represented and/or who facing intersecting barriers including addiction, mental health, disability, and trauma. These practices and resources should aim to promote substantive impartiality that is to level the playing field and ensure the fairness of hearing for marginalised litigants. For example, it could consider preparing a job aid or checklists for members for communicating information about the nature of the proceedings, the relevant legal and factual issues, the possible outcomes and the availability of appeal or judicial review routes.

Management response

The IRB partially accepts this recommendation.

The IAD provides self-represented appellants individualized meetings before scheduling a hearing to explain appeal procedures, case details, what to expect at a hearing, and possible outcomes. As a result, IAD members are not expected to repeat this information in a hearing unless there is an evident need to do so. This expectation will remain unchanged.

The IAD has recently provided additional training and resources to its members for managing hearings involving self-represented and marginalized appellants. In Fall 2023, IAD members and Early Resolution Officers received training on the revised Chairperson Guideline 8 – Procedural Accommodations (note that this training occurred at the mid-point of the period in this evaluation). In Spring 2024, it also provided members with training on active adjudication. Finally, in Summer 2024, the IAD reviewed and updated its hearing scripts to aid members in presiding over hearings.

Management action plan

N/A

Lead responsible: IAD

Completion Date: Completed

Recommendation 2

The IAD should work with members to develop strategies for effectively managing interactions between participants to a hearing, and ensuring civility and equitable treatment.

Management response

The IRB accepts this recommendation.

Management action plan

The IAD will conduct open discussions with members to gain insights into the types of problematic behavior encountered during hearings. These sessions will facilitate the sharing of practical strategies to address these issues, taking into account the adversarial nature of IAD proceedings. Additionally, members will be reminded of the procedures for reporting conduct-related matters.

Lead responsible: IAD

Completion Date: Q1 2025-26

Recommendation 3

It is recommended that the IAD work with members to develop best practices and training in active adjudication, specifically focusing on how to effectively open and direct hearings. These best practices should include how to provide clear explanations of the legal issues to be determined, the types of evidence that need to be adduced to address the legal issues, the applicable legal tests as determined by statute and binding precedent, and the various functions to discharge through the stages of the hearing.

Management response

The IRB accepts this recommendation. The IAD notes that, since the assessment period, it has already provided additional training in this area in the spring of 2024 at its National Training Seminar.

Management action plan

Although this recommendation is being marked as completed, the IAD will continually assess if additional training or training aides would be beneficial.

Lead responsible: IAD

Completion Date: Completed

Recommendation 4

The IAD should consult with members to understand the reasons behind the underutilization of available technical support during hearings. By identifying and addressing these barriers, the IAD can enhance the effectiveness of technical assistance, ensuring that members can resolve communication issues more efficiently and reduce lost time during hearings.

Management response

The IRB partially accepts this recommendation.

For context, the IRB has limited technical support for users outside its network. Participants join IAD hearings through various channels (video, dial-in, or calls from the IRB) and connect from different countries. This variety of factors means there is rarely a universal solution, and some hearing time may be lost troubleshooting. The IAD's primary concern is that members address technical issues without compromising the fairness of the hearing.

Management action plan

The IAD will review its troubleshooting procedures with members to ensure they reflect best practices for resolving common technical issues in hearings.

Lead responsible: IAD

Completion Date: Q1 2025-26

Annex A – IAD indicator list