Quality performance in the Refugee Protection Division 2022 to 2023

​​​​This report describes the results of the measurement of quality in decision-making in the Refugee protection Division (RPD).

On this page

• 1. Overview
• 2. Summary of results
• 2.1 Pre-proceeding readiness
• 2.2 Fair and respectful proceedings
• 2.3 Focused proceedings
• 2.4 Reasons state conclusions on determinative issues
• 2.5 Decisions provide findings and analysis necessary to justify conclusions
• 2.6 Reasons are transparent and intelligible
• 2.7 Supplementary questions related to virtual hearings
• 3. Results for specific populations
• 3.1 Consideration for sexual orientation, gender identity and expression, and sex characteristics
• 3.2 Vulnerable persons
• 4. Recommendations
• 5. Management response and action plan
• Annex A – RPD performance indicators
• Annex B – SOGIESC quality review checklist: Performance indicators and rating guide

1. Overview

The study reviewed 80 cases that were finalized between April 1, 2022, and June 30, 2022 (the assessment period) before the Refugee Protection Division (RPD) after an oral hearing. Abandonment hearings, applications for cessation and vacation, and hearings over four hours were excluded. The cases were randomly selected in proportion to region, language, and outcome.

The following charts illustrate the sampling makeup:

Regional office

Language of proceeding

Outcome

Assessment methodology

This qualitative assessment was performed by an independent reviewer, Martine Valois, who is a law professor specializing in administrative tribunals and immigration law. The reviewer examined all necessary evidentiary and administrative materials on file, listened to the complete audio recordings, and assessed these against qualitative indicators in a checklist developed by the Strategic Planning, Accountability and Reporting (SPAR) Directorate and approved by the Deputy Chairperson of the RPD (see Annex A). The checklist assesses thirty-two indicators across seven categories:

1. Pre-proceeding readiness
2. Fair and respectful proceedings
3. Focused proceedings
4. Reasons state conclusions on determinative issues
5. Decisions provide findings and analysis necessary to justify conclusions
6. Reasons are transparent and intelligible
7. ​Supplementary questions related to virtual hearings

Fourteen of the indicators were mandatory for assessment, and eighteen were assessed only when applicable. Each performance indicator is assessed along a 1-3 rating scale and, for supplementary questions related to SOGIESC cases, a categorical yes-no scale was used.

The 1-3 rating scale is as follows:

1=Does not meet expectations: The quality requirement was not met. The evidence showed one or more key instances where the proceeding or reasons would have markedly benefited had this requirement been met. There may have been an effort to apply the requirement, but the level of achievement fell short of expectations.

2=Meets expectations: This is a level of acceptable achievement. On balance, the member satisfied this quality requirement, though there is a margin for minor improvement.

3=Exceeds expectations: This is a level of consistent, above-average achievement. The evidence shows a grasp of the quality requirement and an understanding of its importance to a high-quality proceeding or decision, as the case may be.

Results are also expressed as a percentage of cases that meet expectations, by obtaining a score of 2.0 of higher.

In addition, as part of Gender Based Analysis Plus (GBA+), a second checklist was developed for cases related to Sexual Orientation, Gender Identity and Expression, and Sex Characteristics (SOGIESC). Within the sample, eight cases related to Sexual Orientation, Gender Identity and Expression (SOGIESC) were further analyzed through a particular SOGIESC lens. However, given the small sample size observations are not generalizable.

To mitigate the inherent limitations of qualitative research, detailed performance indicators were provided to the reviewer to help focus the assessment. To ensure quality and consistency in the assessment, a reviewer was selected based on their in-depth knowledge of the Immigration and Refugee Protection Act​ (IRPA), refugee and immigration matters, and administrative law. Moreover, a small sample size limits the inferences that may be made about the broader caseload. Where sample sizes are too small, observations or recommendations may still have been provided, but these are not based on representative findings.

This report aims to provide a perspective to improve the Division's performance overall. The evaluation unit provided the statistics found in the table accompanying each result section as well as the information in 1.0 “Overview”. However, the findings in this report, including all strengths, areas for improvement and recommendations are solely those of the reviewer. Their observations do not lend themselves to firm conclusions on legal matters such as the correct application of the law, the weighing of the evidence, or the fairness of the proceedings from a natural justice perspective. Only a court reviewing the case can arrive at such conclusions.

2. Summary of results

The primary performance target for this assessment is for 80% of cases to meet quality standards. The quality standard is that the case achieves a score of 2.0 or higher in at least 80% of all standard indicators assessed within that case. This target was achieved with 97.5% (78 out of 80 cases), meeting the expectations. In other words, over 97.5% of cases reviewed either met or exceeded the quality standard.

Where an indicator had many cases that did not meet the target, it is addressed in the reviewer's observations following the table (Areas for Improvement).

What we did well

The RPD performed well in the following areas:

• Members were well prepared for hearings. In all cases, members had clearly read the documents on file; they had good knowledge of the facts and main issues. The exchanges between the member and the claimants' counsel ensured that all participants had access to the same documents and evidence.
• In most cases, members clearly showed a level of sensitivity and respect that contributed significantly to improving the quality of the proceedings. Members were helpful to claimants in explaining all the steps of the hearing to them, what the legal issues were, the evidence they had to provide, and how such evidence was assessed and weighed, etc. They ensured that claimants were fully participating in their own hearing, and not merely witnesses providing testimony. 
• In most cases, members' questioning was focused and well organized during the entire hearing.
• In general, decisions were well written and satisfied the criteria of justification, transparency, and intelligibility. Members clearly stated the points to be decided, with their analysis then flowing through the legal framework and their findings of fact to a clear, unambiguous conclusion. Positive and negative decisions refer to both the evidence that supported the conclusions and the evidence that contradicted it, which demonstrated that all the evidence was considered.
• Overall, the cases involving SOGIESC have been handled appropriately. Members were receptive, attentive, and sensitive to the need to give due consideration to the unique characteristics of claimants whose cases involve SOGIESC and refrained from relying on stereotypes. Good practices included specifically asking the question to the claimant as to what their chosen name, pronouns, and preferred terminology for identity were. Members were also sensitive, respectful, and non-confrontational while questioning claimants on intimate matters. Good examples included asking open questions.

What we can improve

The following areas were identified as areas that could be improved. The assessor made recommendations that could improve performance on these indicators:

• Making sure that members convey the same information in all the cases regarding how the hearing is to proceed (order of questioning, making short sentences, waiting for the end of translations, not guessing answers if they did not know, asking to repeat questions if they did not understand, etc.).
• Most members who considered the guidelines mentioned them only at the beginning of the decision. Decisions rarely explain how these guidelines are relevant to the facts and issues of the case, how they are applied, and which paragraph or section of the guidelines were considered. These explanations are especially important when negative decisions are rendered in gender related or SOGIESC cases where absence of credibility findings are made.
• For SOGIESC cases, members should consider intersectional factors such as race, ethnicity, religion, age, and gender. Additional training on when it is relevant and how to consider intersectional factors when making a decision could be helpful in that respect.

2.1 Pre-proceeding readiness

Why measure this

The groundwork for a quality hearing and decision is set when the Registry prepares a timely, organized, and complete case docket and the member assimilates the facts and key issues of the case.

Considerations

Indicator one is applied to all cases.

General observations (and/or best practices)

• Members reviewed the list of consolidated documents with counsel to ensure they had all the documents submitted. When necessary, they took short breaks to read a document that was not part of the record they received. In all cases, members had clearly read the documents on file; they had good knowledge of the facts and main issues. The exchanges between the member and the claimants' counsel ensured that all participants had access to the same documents and evidence.

What we did well

• Indicator 1 – Proceeding readiness:  Making sure that all the necessary documents were filed and approved by the claimant. This discussion between counsel and the member prior to the start of questioning regarding the content of the file proved essential to the smooth running of the proceedings.

What we can improve

• N/A

2.2 Fair and respectful proceedings

Why measure this

Individuals appearing before the IRB expect that they will be treated with sensitivity and respect. Any shortcoming in this regard potentially undermines tribunal integrity and public confidence.

Considerations

Indicators 2 and 3 are applied to all cases while 4-6 are scored on an as-applicable basis.

General observations / ​and or best practices

• Members clearly showed a level of sensitivity and respect that contributed significantly to improving the quality of the proceedings. Members were particularly helpful to claimants in explaining all the steps of the hearing to them, what the legal issues were, the evidence they had to provide, how such evidence was assessed and how it was weighed, etc. They ensured that claimants were fully participating in their own hearing, and not merely witnesses providing testimony. Participants at the hearing were treated with respect, whether they were claimants, interpreters, or counsel. In all cases, members ensured that interpreters and applicants understood each other. In many cases, members and interpreters had already worked on numerous cases together. Hearings ran smoothly, allowing claimants to present their case as fully as possible. I did not witness any case where claimants were prevented or restricted in any way from being heard. Claimants were able to provide evidence without untimely interruptions from members.

What we did well

• The hearings were conducted in a neutral and courteous manner. Participants were treated respectfully. In most cases, members informed the claimants in a detailed manner of the conduct of the proceedings (order of questioning, making short sentences, waiting for the end of translations, not guessing answers if they did not know, asking to repeat questions if they did not understand, etc.). This helped claimants know from the outset how the hearing would proceed and what to expect in the next few hours.
• Members asked claimants and/or counsel whether all the evidence was on the record, whether claimants had anything to add to their testimony, whether counsel had any questions to ask the claimants, and/or whether they had additional representations to make.
• Members asked clarifying questions in cases where they clearly identified what was problematic with the evidence and explained why it was particularly relevant, linking it to the determinative issues of the hearing.
• In all cases, members asked claimants and their counsel if they agreed to waive interpretation for counsel's representations and oral delivery of the reasons for the decision. When counsel or claimants did not, interpretation was provided. This was identified as an issue in the 2020-2021 RPD QMI and was corrected in the present assessment.  

What we can improve

• Encourage members to convey information regarding how the hearing will proceed (order of questioning, making short sentences, waiting for the end of translations, not guessing answers if they did not know, asking to repeat questions if they did not understand, etc.). Cases which scored 3 for indicator 2 may be used as an example of best practices.

2.3 Focused proceedings

Why measure this

Proceedings that are efficient and well managed create conditions for quality outcomes to emerge and support the IRB's efforts to make the most effective use of its resources.

Considerations

Indicators 7 to 9 are assessed against all cases while 10 to 17 are assessed on an as-applicable basis.

General observations and/or best practices

• In most cases, members set out a clear issue agenda at the beginning of the hearing. The most common issue was credibility, and additional issues included identity as a member of a specific group, nexus to the Convention grounds, state protection, and internal flight alternatives (IFA). It is very important that this is done in all cases. The organization of the hearing's time is important to ensure that all the essential elements of the case are covered. Failure to identify determinative issues at the beginning of the hearing in accordance with indicator 7 generally implies that indicator 18 cannot be met unless an issue was subsequently identified during the course of the hearing and dealt with in the reasons.
• In most cases, members' questioning was focused and well organized during the entire hearing.
• In most cases, this has been done; members identified at the outset the main issues that would be addressed. However, it was not done systematically. In some cases, members swore in the claimant and started questioning right away.

What we did well

• When relevant, members clearly indicated that they were satisfied that one or more issues identified at the beginning of the hearing were resolved, and that the remaining part of the hearing would focus on a specific issue. Such indications were particularly relevant to ensure that counsels' questions to claimants were specific to the determinative issues and/or problems with the evidence. With respect to narrowing the issues for final representations, in 13 out of the 14 cases for which it was applicable, members provided clear instructions to counsel that they still had some concerns with respect to a specific issue and/or evidence. Narrowing the issues in such a manner contributed significantly to the hearing's overall efficiency.

What we can improve

• Encourage members to modify their hearing preparation materials to identify determinative issues at the outset of hearings as a matter of procedural fairness and hearing room management.

2.4 Reasons state conclusions on determinative issues

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility, and transparency in a decision of an administrative tribunal. Through indicators 18 to 28 this study applies the Court's requirement in the context of IRB decision-making.

Considerations

These indicators are applied to all cases.

General observations / and or best practices

• This part of the review performed strongest. Decisions were written in a clear manner and canvassed the determinative issues. Conclusions were supported with evidence that members found to be credible and trustworthy.
• Members based their conclusions on all the issues and relevant evidence adduced during the proceedings.

What we did well

• Decisions were clear and intelligible. Positive and negative decisions refer to both the evidence that supported the conclusions and the evidence that contradicted it, which demonstrated that all the evidence has been considered.

What we can improve

• N/A

2.5 Decisions provide findings and analysis necessary to justify conclusions

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility, and transparency in a decision of an administrative tribunal. Through indicators 20 to 28 this study applies the Court's requirement in the context of IRB decision-making.

Considerations

Indicators 20-22 are applied to all cases, while 23-25 are applied on an as applicable basis.

General observations / and or best practices

• This section of the assessment obtained high results. This allows us to conclude that, for the sample evaluated, the members drafted decisions that were justifiable, intelligible, and transparent. Indicators 20, 21 and 22 met or exceeded expectations in almost all cases. The quality of the conclusions improved when members had a well-organized decision with headings identifying the relevant parts and issues of the decision as well as when they support their conclusions with findings of fact supported by clear examples (indicator 21), especially when they refer to the claimant's testimony and how it supports documentary evidence.
• Good examples exceeding expectations for indicators were when members clearly explained why all relevant facts were proven, on a balance of probabilities. In all cases where evidence contrary to the members' decisions was presented, the members referred to the evidence and stated their reasons for rejecting them. In the pursuit of the culture of justification established by the Supreme Court in the Vavilov ruling, and in the spirit of fostering institutional coherence, members should be asked to provide more elaborate reasons for cases where the Chairperson's guidelines are considered. Good examples of adherence to the applicable legislation, regulations, Rules, or Guidelines included members who avoided asking specific questions and details about certain traumatizing events.
• In a few cases, members did not take into account social and cultural contextual factors when rejecting a participant's testimony.

What we did well

• Most decisions presented clear and unequivocal findings of fact, supported by clear examples and credible evidence.
• Most decisions addressed the evidence that ran contrary to the member's decision, and why certain evidence was preferred.

What we can improve

• Most members who considered the guidelines mentioned them only at the beginning of the decision. Decisions rarely explain how these guidelines are relevant to the facts and issues of the case, how they are applied, and which paragraph or section of the guidelines were considered.
• As a best practice, members should be encouraged to explain how they applied the relevant guideline, especially where absence of credibility findings are made.
• Social and cultural factors related to the claimant may explain behaviour that seems unreasonable to the tier of facts. In cases where the claimant's testimony is rejected, members should be encouraged to refer to those factors (social and cultural context) and explain how, if any, they impacted the findings and/or conclusions.

2.6 Reasons are transparent and intelligible

Why measure this

The Supreme Court of Canada set the requirement for justifiability, intelligibility, and transparency in a decision of an administrative tribunal. Through indicators 20 to 28 this study applies the Court's requirement in the context of IRB decision-making.

Considerations

All indicators are applied to all cases.

General observations / and or best practices

• The scores for these indicators allow us to conclude that members wrote clear and intelligible decisions supported by clear findings of fact. In some cases, members exceeded expectations and scored a “3” when they also engaged with the relevant case law to explain their decision. For example, members referred to decisions from the Federal Court, the Federal Court of Appeal and/or the Supreme Court to explain the requirement of a personalized risk for the application of 97(1) IRPA and the notion of a “particular social group” under the Refugee Convention. This practice should be encouraged when applicable, such as when referring to the relevant case law contributes significantly to support or explain a member's conclusion on a determinative or contested issue. Positive decisions tended to be shorter than negative ones, which is understandable, as negative decisions must be substantiated to enable claimants to understand the reasons for why the claim was refused and possibly, to exercise a meaningful right of appeal or judicial review. However, positive decisions were equally justified and transparent.

What we did well

• Decisions were clear, intelligible, and logically sequenced. Findings of fact were supported by clear examples and credible findings. Striking a balance between having a detailed decision supported by many examples, yet ensuring that the decision remains clear and concise is an art. In most cases, this aim was achieved.
• Headings significantly improve the quality of the reasons.

What we can improve

• N/A

2.7 Supplementary questions related to virtual hearings

Why measure this

It is important to ensure that members adhere to guidelines prescribed for adjudicating hearings virtually. This includes ensuring that claimants are afforded accommodations when required, that the technology provides adequate quality for the hearing, that witnesses are participating with the same level of adherence to protocols as an in-person hearings, and that members follow practices in line with regional protocols.

Considerations

All indicators are assessed only if applicable.

General observations / and or best practices

• Virtual hearings took place without major problems, or those which could not be easily overcome. There were sound, video, or technical issues in some cases. The more common problems included echo, background noise (phones or other electronic devices ringing; paper noise; typing noise), sound cutting, image freezing, parties disconnecting, and problems showing a document on the screen. In most cases, members met expectations by acknowledging the problem and finding simple solutions, including asking the parties to repeat themselves and/or rejoin the meeting. 

What we did well

• Members were alert to technical issues that could interfere with the smooth conduct of the hearing. Members asked that new documents, deemed relevant for the hearing, be sent to the registry.

What we can improve

• Claimants are expected to answer questions spontaneously, without having to refer to notes they may have taken before the hearing. The facts on which they testify are supposed to be within their personal knowledge. Claimants are thus not allowed to have notes or documents in front of them, in virtual or in-person hearings. This is important for the assessment of the credibility of their testimony.

3. Results for specific populations

3.1 Consideration for sexual orientation, gender identity and expression, and sex characteristics

Why measure this

For the purposes of Gender Based Analysis Plus (GBA+), which is a priority for the IRB as well as within the Government of Canada as a whole, a second checklist was developed for cases involving sexual orientation, gender identity and expression, and sex characteristics (SOGIESC). The checklist assesses the application of Chairperson’s Guideline 8: Accessibility to IRB Proceedings — Procedural Accommodations and Substantive Considerations .

What was measured

8 SOGIESC cases were identified within the sample, however, not all indicators were applicable to every case. A table of indicators is provided in Annex B. Due to the small number of SOGIESC cases and indicators assessed, findings cannot be generalized to other SOGIESC cases and are included as observations only.

General observations / and or best practices

Overall, the cases involving SOGIESC have been handled appropriately. Members were receptive, attentive, and sensitive to the need to give due consideration to the unique characteristics of claimants whose cases involve SOGIESC and refrained from relying on stereotypes. Good practices included specifically asking the question to the claimant as to what their chosen name, pronouns, and preferred terminology for identity were. Members were also sensitive, respectful, and non-confrontational while questioning claimants on intimate matters. Good examples included asking open questions.

What we did well

• Members avoided relying on stereotypes when making findings of fact.

What we can improve

• Members should always consider laws of general application and intersectional factors such as race, ethnicity, religion, age, and gender. In cases where the basis of the asylum claim is sexual orientation, factors like religion, gender, ethnicity, and age may intersect and add to the complexity of the case.  Intersectionality should be considered, and members should evaluate what impact, if any, it has on the case.  Providing guidance on when it is relevant and how to consider intersectional factors when making a decision for SOGIESC claimants would be helpful.

3.2 Vulnerable persons

Why measure this

Vulnerable person cases were overrepresented in the sample to adequately review the outcomes of this case type and advise the division on areas for improvement and best practices ahead of Chairperson's Guideline 8: Accessibility to IRB Proceedings – Procedural Accommodations and Substantive Considerations coming into effect on October 31, 2023. This guideline replaces ‘Chairperson's Guideline 8: Procedures with Respect to Vulnerable Persons Appearing Before the IRB’, which was in effect during the period of this review.

This guideline’s objective is to enhance access to justice at the IRB by providing guidance on:

• granting procedural accommodations;
• situations where a person’s disability, vulnerability and/or personal characteristics may be relevant to the assessment of the merits of the case
  
• the use of appropriate language; avoiding myths and stereotypes; the principles of trauma-informed adjudication and intersectionality; and the protection of confidential information.

What was measured

Nineteen vulnerable persons cases were identified within the sample. Due to the small number of cases identified, findings cannot be generalized to other vulnerable persons’ cases and are included as observations only.

The Audit and Evaluation Team compared the distribution of scores (by category) for cases involving vulnerable persons to those of the entire sample. The table and graph below display this comparison.

General observations / and or best practices

• Members were particularly sensitive to vulnerable persons and very concerned about providing them with accommodations during the hearing. Even when requests for accommodation were submitted late, members did not hesitate to consider and offer them. This was particularly the case for unaccompanied minors and women whose claims were based on persecution of a sexual nature.

What we did well

• Offering accommodations to vulnerable participants.

What we can improve

• N/A

4. Recommendations

The above results demonstrate that in almost all aspects assessed, the Refugee Protection Division attained very high standards of fairness, transparency, and respect for all hearing participants.  I have thus identified only 3 measures that could be applied to improve a process that is already very efficient and fair. The 3 recommendations are reflected below:

1. Members who are applying Chairperson’s guidelines should mention in their reasons the specific section of these guidelines that is relevant to the issues they have to consider and its impact on the particular facts of the case.
2. Members should make sure that they convey the same information in all cases regarding how the hearing will proceed (order of questioning, making short sentences, waiting for the end of translations, not guessing answers if they did not know, asking to repeat questions if they did not understand, etc.).
3. As a best practice, in SOGIESC cases, members should consider intersectional factors such as race, ethnicity, religion, age, and gender in their evaluation of the well-founded fear of persecution. Additional and/or refresher training should be given to members to assist them in identifying when intersectional factors are relevant in a given case and how it affects the evaluation of the evidence.

5. Management response and action plan

Annex A – RPD performance indicators

Annex B – SOGIESC quality review checklist: Performance indicators and rating guide