Evaluation of the IRB Process for making a complaint about a member - Final report

​​​​​​​​March 2, 2021

Table of Contents

• List of acronyms
• Executive summary
• Design
• Implementation
• Effectiveness and efficiency
• 1.0 Introduction
• 1.1 Evaluation objectives and scope
• 2.0 Overview of the complaints process
• 2.1 Scope
• 2.2 Steps in the complaints process
• 3.0 Methodology
• 3.1 Document, data, and file review
• 3.2 Case studies of comparable organizations with complaints processes
• 3.3 Key informant interviews
• 3.4 Limitations
• 4.0 Findings
• 4.1 Design
• 4.2 Implementation
• 4.3 Effectiveness and efficiency
• 5.0 Conclusions and recommendations
• 5.1 Design
• 5.2 Implementation
• 5.3 Effectiveness and efficiency
• Reference list
• Appendix A – Evaluation Matrix
• Appendix B – Data Collection Instruments

List of acronyms

Executive summary

Introduction

The Immigration and Refugee Board of Canada (IRB) commissioned this independent evaluation to review its process for making a complaint about a member under the 2017 Procedures. The evaluation responds to the Parliamentary Standing Committee on Citizenship and Immigration’s request in September 2018 that the IRB report back to the committee within three years with a review of its complaints process under the 2017 Procedures, with particular consideration to the need for independence in the complaints investigation and adjudication process.

The evaluation’s main objectives are to review the complaints process to consider what is working well and what can be improved. The evaluation considers:

• the design of the complaints process, including whether the 2017 Procedures are clear, accessible, and transparent;
• the implementation and efficiency of the process;
• the credibility, impartiality, and procedural fairness of the process, through which the need for independence is assessed; and
• whether there are improvements that can be made.

The IRB complaints process

In handling the cases before them, the IRB’s members make quasi-judicial decisions and have independence in their decision-making, but they are expected to comply with certain standards of conduct as outlined in the Code of Conduct for Members of the Immigration and Refugee Board of Canada(the Code of Conduct or the Code). The IRB complaints process provides a formal mechanism by which an individual can file a complaint about the conduct of an IRB member. The complaints process was established to promote the integrity of the decision-making process and to help identify systemic issues or trends that require attention.

The complaints process was substantially revised from the previous 2012 Protocol Addressing Member Conduct Issues (the 2012 Protocol) to clarify the process and improve its accessibility and transparency. The new Procedures for Making a Complaint about a Member (the 2017 Procedures) came into effect on December 17, 2017. Under the 2017 Procedures, the complaints process was centralized rather than being handled at the divisional and regional office level. The process is now overseen by the Office of Integrity (OI), which is an independent body from the divisions and reports directly to the Chairperson. The 2017 Procedures were also intended to streamline and simplify the complaints process. In addition, the new process undertook more detailed annual reporting of the results of complaints.

Conclusions and recommendations

Design

Overall, the 2017 Procedures and the design of the new complaints process addressed several of the concerns raised by stakeholders and are considered to be an improvement over the 2012 Protocol. In particular, the new complaints process is considered to be:

• more streamlined and to have more clearly articulated steps;
• more transparent in terms of communications with parties to a complaint and annual reporting of statistics and complaint summaries; and
• more consistent and objective by having the process managed by a central body, the OI, which is independent from the IRB divisions.

Implementation

Accessibility and use: The complaints process is generally considered to be accessible, with the process prominently displayed on the IRB website and an online optional complaint form that is in an easy to use format. However, awareness of the complaints process appears to be an ongoing issue and additional communications and guidance would improve accessibility, particularly for unrepresented complainants, but for other stakeholders as well. In particular, more engagement with internal and external stakeholders about the complaints process could improve accessibility and understanding of the process, including the role of the OI and its position within the IRB.

A total of 43 complaints have been filed in its first two years of operations (December 2017 to December 2019), which is a small number in the context of the size of the IRB (approximately 400 members) and its volume of work (approximately 60,000 claims finalized in 2019). The evaluation cannot assess whether this is due to the quality of IRB members or potential barriers to the complaints process, although it is likely a combination of the two. Key informants noted several potential barriers to filing complaints, but most often raised the fear of negative repercussions for those complainants who appear before the IRB on multiple matters (e.g., representatives of parties to an IRB proceeding, including hearings officers).

Recommendation 1: Promote understanding of the complaints process with outreach activities for both internal and external stakeholders.

Recommendation 2: Consider methods to limit the sharing of the identity of complainants until it is determined that a complaint is within scope and will be investigated.

Clarity and coherence: The scope of the complaints process and its intersection with the Code of Conductrequire clarification. There remains confusion about which Code provisions are within the scope of the complaints process and disagreement about the current scope of the complaints process. The Code includes a number of provisions that are about member conduct but are outside of the complaints process. One of these provisions refers to allegations of bias, which are appealable and, therefore, outside of the complaints process. Key informants consider bias allegations to touch on both behaviour and decision-making. Behaviour underlying claims of bias are thought to be screened out too often by external stakeholders and some IRB key informants. Overall, the evaluation found a desire for greater clarity regarding the criteria used for determining whether to screen complaints into or out of the process.

Recommendation 3: Review the Code of Conduct and its intersection with the 2017 Procedures to clarify the scope of the complaints process in consultation with stakeholders.

Transparency: The IRB has put mechanisms in place that have improved transparency: a clear outline of the steps in the complaints process, expectations regarding communications with parties to a complaint, and extensive annual reporting. Parties generally appreciated the communications from the OI about the progress of their complaint. The evaluation identified some areas of improvement related to the communications with parties:

• the need for greater consistency in terms of when the OI communicates with parties;
• the provision of more clarity about the reasons for decisions (both screening decisions and final decisions); and
• notification to parties regarding when a complaint is communicated to others within the IRB, particularly for complaints held in abeyance.

Regarding transparency of the complaints process overall, the evaluation found the steps in the process to be set out in the 2017 Procedures, yet the process is still considered to be somewhat opaque to stakeholders in terms of how decisions are made at each step, including the criteria used to make decisions (e.g., whether complaint is within scope, whether to use an external investigator, whether there was a breach of the Code, what remedies were selected), what the OI recommendations were and if the Chairperson did not follow them, why not, and what steps, if any, were taken as a result of the complaint.

The annual reporting is a major improvement over the 2012 Protocol, but many external stakeholders are unaware of it and the type of information included.

Recommendation 4: Consider approaches to enhance transparency in terms of improvements in communications with parties, clarification of criteria used at key points in the process, and promotion of annual reporting.

Recommendation 5: To assist with reporting and ensure all variables of interest are captured, consider improvements to the method used to track data on complaints.

Effectiveness and efficiency

Investigations and outcomes: The OI is generally considered to be conducting its job in accordance with the 2017 Procedures and to be respectful in its approach. External investigators are reportedly being used more often, which is hoped to further strengthen the perception of the independence of the process. The main issue identified is the need to clarify what remedial actions and sanctions are available and how they should be applied, including any follow-up.

Recommendation 6: As part of the review of the Code of Conduct and 2017 Procedures, consideration should be given to what remedies and sanctions should be available, the criteria for their use, and what follow-up steps should be in place.

Credibility, impartiality, and procedural fairness: By considering the credibility, impartiality, and procedural fairness of the complaints process, the evaluation assessed whether there is a need for independence. Overall, the complaints process appears to be credible, impartial, and procedurally fair, although more transparency will help build greater confidence in the process. The available data supports this conclusion. In particular, the IRB screens out fewer complaints than complaints processes in other similar adjudicative bodies, and breaches of the Code are found in half of the complaints that proceed to the investigation stage.

Taking stakeholder opinions into account is fundamental to creating and sustaining a credible, objective, and fair process. Stakeholders who questioned the credibility of the complaints process primarily raised issues related to the clarity of the process’s scope. Their concerns, as also noted above under “clarity and coherence” and “transparency,” often centred on the need for a better understanding of what criteria are used to make decisions and better communications about the process.

The other concern raised by stakeholders was related to the decision-makers for screening, investigating, and deciding complaints. In general, the OI was perceived to be an independent body as it is outside of the divisional structure. In addition, external investigators are reportedly being used more often. Given the independence of these actors and their role in providing recommendations to the Chairperson, key informants desired to know what the OI’s and external investigator’s recommendations were in specific complaints. In addition, the role of the Chairperson in the complaints process was also generally not considered to be an issue, but there was, again, a desire for more transparency in terms of the criteria used for decisions, the reasoning used in specific decisions, and whether or not the Chairperson is following the OI’s recommendations or the external investigator’s conclusions.

Recommendation 7: Consider methods to further support the credibility of the complaints process, such as the use of external investigators, and either sharing OI recommendations and the external investigator’s conclusions or indicating whether they were followed.

Efficiency: The complaints process has experienced some issues with timeliness and has not yet set service standards. This is due, in part, to the capacity of the OI, which is being addressed by the expansion of that office. IRB key informants acknowledged the need to set service standards, and external key informants commented that having service standards would enhance the transparency and credibility of the process.

Recommendation 8: Set service standards for timeliness of service.

1.0 Introduction

The Immigration and Refugee Board of Canada (the IRB or the Board) is Canada’s largest independent administrative tribunal.^{Footnote 1} In 2019, the IRB’s approximately 400 decision-makers (members) in its four divisions issued over 72,500 decisions (Immigration and Refugee Board of Canada, 2020b). IRB members make decisions on refugee claims before the Refugee Protection Division (RPD), refugee appeals before the Refugee Appeal Division (RAD), admissibility hearings and detention reviews before the Immigration Division (ID), and immigration appeals before the Immigration Appeal Division (IAD) (Immigration and Refugee Board of Canada, 2020b). The IRB Chairperson is a member of each division and, as chief executive officer of the Board, has supervision over and direction of the work and staff of the Board (Immigration and Refugee Protection Act, 2001, sec. 159).

In handling the cases before them, the IRB’s members make quasi-judicial decisions and have independence in their decision-making, but they are expected to comply with certain standards of conduct as outlined in the Code of Conduct for Members of the Immigration and Refugee Board of Canada (the Code of Conduct or the Code) (Immigration and Refugee Board of Canada, 2019a). To ensure that the provisions in the Code related to the conduct of members are upheld, the IRB established a complaints process whereby any individual—be they any party to IRB proceedings, IRB personnel, an organization, or the public—may file a complaint about any full- or part-time member of the IRB (Immigration and Refugee Board of Canada, n.d., sec. 3.2). The complaint must be about a member’s conduct that is believed to be contrary to the Code of Conduct and cannot be about a member’s decision, which should be handled by appeal to the RAD, IAD, or by application for judicial review at the Federal Court.

The complaints process was substantially revised from the previous 2012 Protocol Addressing Member Conduct Issues (the 2012 Protocol) to clarify the process and improve its accessibility and transparency. The complaints process also became centralized and is now overseen by the Office of Integrity (OI). The new Procedures for Making a Complaint about a Member (the 2017 Procedures) came into effect on December 17, 2017 (Immigration and Refugee Board of Canada, 2017).

1.1 Evaluation objectives and scope

The IRB commissioned an independent evaluation to review its process for making a complaint about a member under the 2017 Procedures. The evaluation responds to the Parliamentary Standing Committee on Citizenship and Immigration’s request in September 2018 that the IRB report back to the committee within three years with a review of its complaints process under the 2017 Procedures, with particular consideration to the need for independence in the complaints investigation and adjudication process (Standing Committee on Citizenship and Immigration, 2018, p. 58).^{Footnote 2}

Consistent with that objective and to provide IRB management with information on what is working well or where the complaints process can be improved, the evaluation considered:

• the design of the complaints process, including whether the 2017 Procedures are clear, accessible, and transparent;
• the implementation and efficiency of the process;
• the credibility, impartiality, and procedural fairness of the process, through which the need for independence is assessed; and
• whether there are improvements that can be made.

The evaluation was primarily formative in nature and considered the implementation of the complaints process (i.e., how the complaints process is being administered and whether its early administration can be improved). The evaluation also captured summative elements related to its effectiveness (i.e., whether the Procedures are clear and the process is fair, accessible, and transparent). The evaluation primarily covered the period from December 17, 2017 to March 31, 2020. For certain lines of evidence, specifically the review of complaints, the evaluation only includes complaints under the 2017 Procedures that were closed as of March 31, 2020.^{Footnote 3} Interviews with key informants occurred from July to October 2020.

2.0 Overview of the complaints process

The IRB complaints process provides a formal mechanism by which an individual can file a complaint about the conduct of an IRB member. The complaints process was established to promote the integrity of the decision-making process and to help identify systemic issues or trends that require attention (Immigration and Refugee Board of Canada, 2020b).

2.1 Scope

Complaints must be related to member conduct, the expectations of which are outlined in the IRB’s Code of Conduct. The scope of the Code of Conduct is broad and covers a range of ethical and professional responsibilities of members. It also addresses not only members' responsibilities to the tribunal and to the parties who appear before them but also to the public. The Chairperson is responsible for administering the Code of Conduct, and any breaches may result in corrective or disciplinary measures.^{Footnote 4} The most recent iteration of the Code of Conduct came into effect on April 9, 2019 (Immigration and Refugee Board of Canada, 2019a). Given the breadth of the Code, not all sections of the Code of Conduct fall under the scope of the complaints process. Specifically, Sections 9-15 of the Code that cover the conduct of members can be the subject of a complaint.

9. “Members shall conduct hearings in a courteous and respectful manner while ensuring that the proceedings are fair, orderly and efficient.
10. Members shall exercise their duties without discrimination. Members must take reasonable measures to accommodate all participants in a proceeding so that they may participate effectively. Members are expected to take into account social and cultural differences and to respect human rights.
11. Members are expected to act honestly and in good faith, in a professional and ethical manner.
12. Members shall conduct themselves with integrity and avoid impropriety, or the appearance of impropriety.
13. Members have a responsibility to perform their duties in a manner that fosters collegiality among members and with staff and to treat them with courtesy and respect. Members are expected to assist their colleagues through the respectful exchange of views, information and opinions.
14. Members shall arrange their private affairs in a manner that will prevent them from being in a conflict of interest, as set out in the applicable legislation, guideline, code, policy or other instrument established for public servants or Governor in Council appointees respectively.
15. Members shall not accept gifts or other advantages, including hospitality or other benefits, as set out in the applicable legislation, guideline, code, policy or other instrument established for public servants or Governor in Council (GIC) appointees respectively” (Immigration and Refugee Board of Canada, 2019a).

IRB members are independent in their decision-making, and their decisions reflect many different factors, including the law, the merits of the case and the credibility of the person concerned in the case, as well as the safety and security of Canada (Immigration and Refugee Board of Canada, 2020b). This concept of members’ adjudicative independence in deciding cases before them is based in a legal principle and is central to the IRB.

2.2 Steps in the complaints process

The following section describes the process for submitting, reviewing, and addressing a complaint, as cited closely from the 2017 Procedures. A diagram that depicts the process is presented after the text descriptions.

Step 1: Who can submit a complaint

Any individual, including a party to IRB proceedings, IRB personnel, an organization, or the public, may file a complaint about any full- or part-time member of the IRB.

Step 2: Submitting a complaint

A complaint about the conduct of a member must be made in writing to the OI (as of October 2020, the Office of the Ombudsperson has assumed the responsibilities of the OI); it can either be mailed by post or sent by email. A complaint should include the following information:

1. the name and contact information of the complainant
2. the name of the member who is the subject of the complaint
3. the IRB file number (if any)
4. a description of the conduct
5. any other information which supports the complaint

Though a complaint form is available online, it does not need to be used (Complaint Form, 2017).

While the IRB does not accept anonymous complaints, the complainant can request that their identity be kept confidential. In these cases, the OI will safeguard the complainant’s anonymity. Should the identity of the complainant need to be revealed to the IRB member so as to ensure a fair and fulsome investigation, the OI will contact the complainant to confirm whether they wish to pursue their complaint or whether they wish the complaint to be closed by the OI.

In general, complaints related to ongoing cases are not dealt with until the proceedings are finalized. In exceptional circumstances, the Chairperson may decide to deal with the complaint prior to the proceedings being finalized. If a complaint about an ongoing proceeding raises allegations that the IRB member is biased in conducting the proceeding, the allegation of bias should be raised with the presiding member at the first reasonable opportunity.

Step 3: Complaint review process

Once a complaint has been received, the OI will send an acknowledgement of receipt to the complainant. They will also send a copy of the complaint to the member who is the subject of the complaint and to the Assistant Deputy Chairperson (ADC) and the Deputy Chairperson (DC) of the Division in which the member works.

The OI will then review the complaint in order to inform their recommendation to the Chairperson as to whether the complaint is within scope of the Procedures and is serious enough to proceed or if it would better be addressed through another process. At any point in the process, the OI is permitted to attempt to informally resolve the complaint, if it is appropriate to do so.

Upon receiving the recommendation of the OI, the Chairperson may follow one of four courses of action:

1. The Chairperson can dismiss the complaint if it is found to be outside of the scope of the Procedures or if it is not serious enough to proceed with an investigation.
2. The Chairperson can refuse to deal with the complaint if it would be better handled by another process.
3. The Chairperson can refer the complaint to the OI for investigation.
4. The Chairperson can, in exceptional cases, refer the complaint to another person, including to an external investigator.

In all cases, the complainant, the member concerned, the ADC, and the DC will all be informed of the screening decision and the reasons behind it. For cases that move forward with an investigation, the parties to a complaint (the complainant and the member) will be kept informed about the complaint process, as appropriate. For cases that are dismissed or refused, the complaint will be closed and the complainant will have no further recourse under the 2017 Procedures.

Should the Chairperson refer the complaint to the OI or an external investigator, an investigation will be conducted and a report produced. As long as fairness is respected, the conduct of investigations can be quite flexible. In some cases, the Chairperson may provide specific direction to the investigator (e.g., timeframes and deliverables). The investigation report is to contain findings of fact, analysis, and conclusions, including whether the Code of Conduct was breached. The report will be provided to the Chairperson for a decision.

Step 4: Final decision of the Chairperson

Upon receiving the investigation report and having ensured that the principle of procedural fairness was applied, the Chairperson must decide whether to accept the conclusions of the report and determine whether the Code of Conduct was breached.

The complainant, the member who is the subject of the complaint, the ADC, and the DC will be informed by the Chairperson of their decision and the reasoning for their decision. If there was a breach of the Code of Conduct, the Chairperson may take any follow-up actions they consider appropriate. Incidents of prior misconduct, the gravity of the conduct, or any other factors and circumstances may be taken into account when determining whether to take follow-up actions.

Once a decision has been made on whether there was a breach of the Code of Conduct, the complaint will be closed and there is no further recourse under these Procedures.

Step 5: Reporting

The IRB publishes an annual report on its website of all the complaints it has received over the past year. This report includes a description of the nature of the complaints, steps taken to address the complaints, and the decisions made, but will not contain information identifying individuals.

IRB procedure for making a complaint against a member

Figure 1

3.0 Methodology

To guide the evaluation, a methodology and evaluation matrix were developed. The evaluation matrix addressed eight main questions, as summarized below (see Appendix A for the complete evaluation matrix).

1. What issues or challenges with the previous complaints process (Protocol Addressing Member Conduct Issues) were the Procedures for Making a Complaint about a Member intended to address?
2. To what extent does the design of the complaints process under the Procedures for Making a Complaint about a Member address the issues with the Protocol and align with complaints processes in judicial or quasi-judicial organizations?
3. To what extent has the complaints process operated as intended?
4. To what extent are the Procedures accessible, clear, coherent, and transparent?
5. To what extent is the complaints process effective in receiving, investigating, and addressing complaints related to member conduct?
6. To what extent is the complaints process efficient at making, investigating, and disposing of complaints?
7. Does the OI have sufficient tools, resources, and capacity to meet demand and support setting service delivery standards?
8. In what ways could the Procedures, the Code of Conduct as it relates to the Procedures, or the operation of the complaints process be improved?

The evaluation used multiple data collection methods, including a document, data, and file review; case studies of other judicial or quasi-judicial complaints processes; and key informant interviews. The data collection instruments used for the case studies and key informant interviews are in Appendix B. The following sections describe each data collection method and any limitations.

3.1 Document, data, and file review

The document review included publicly available IRB documents on the complaints process, such as the 2017 Procedures, the Code of Conduct, the 2012 Protocol, and the two annual reports on the complaints process. In addition, other relevant documents were reviewed, such as the House of Commons Report of the Standing Committee on Citizenship and Immigration Responding to Public Complaints: A Review of the Appointment, Training, and Complaint Processes of the IRB.

As part of the document review, the IRB conducted a case law review related to higher court decisions that provide guidance on member conduct in general or on how best to conduct a member complaints process. The review found no case law providing guidance on this topic.

In addition, administrative data and files kept by the IRB’s OI related to the complaints process were reviewed. Files reviewed included sample correspondence from the OI to parties to a complaint (complainant and member), OI briefing notes and Chairperson decision letters, and samples of investigation reports. The administrative data review involved an analysis of the data tracked by the OI on individual complaints. As noted in Section 1.1, the evaluation focussed on the 32 complaints under the 2017 Procedures filed between December 17, 2017 and March 31, 2020 that were concluded as of March 31, 2020 (decision on complaint, withdrawn, or administratively closed). To put those complaints in context:

• Based on the annual reports, a total of 43 complaints were filed between December 17, 2017 and December 31, 2019 under the 2017 Procedures.
• Just over two thirds (69%) of the concluded complaints related to proceedings in the RPD, with the remaining complaints related to ID proceedings. Neither appeals division had complaints filed against its members under the 2017 Procedures between December 17, 2017 and March 31, 2020. The proportion of complaints that involve RPD members aligns with the volume of cases of that division. Over approximately the same time period (January 2018 to June 2020),^{Footnote 5} almost two thirds (63%) of IRB proceedings that were finalized were RPD cases (Immigration and Refugee Board of Canada, 2018b).

Number of complaints filed and concluded by division and region (December 17, 2017 to March 31, 2020)

Figure 2 Source: Administrative data

3.2 Case studies of comparable organizations with complaints processes

The evaluation included case studies with five selected judicial or quasi-judicial organizations. The case studies were selected based on suggestions during scoping interviews with IRB management that occurred during the design phase of the evaluation as well as a scan for federal tribunals with complaints processes.

• The Canadian Judicial Council (CJC) handles complaints that involve judges in superior courts in Canada.
• Conseil de la justice administrative (CJA) handles complaints that involve members of the Tribunal administratif du Québec.
• The Social Justice Division of Tribunals Ontario’s (TO) (formerly Social Justice Tribunals Ontario) complaints process includes complaints brought against members of any one of the 14 tribunals that constitute TO.
• The Provincial Court of British Columbia (Provincial Court of BC) complaints process handles complaints involving Provincial Court judges, judicial justices, judicial case managers, and justices of the peace.
• The Veterans Review and Appeal Board (VRAB) complaints process involves VRAB members and employees.

Each case study included a review of relevant publicly available documents related to the complaints processes, including relevant codes of conduct and performance reporting, and an interview with a representative of the organization that is involved with the complaints process. One organization preferred not to be interviewed but did respond in writing to a set of questions.

3.3 Key informant interviews

Interviews were conducted with IRB management and staff, external stakeholders, and parties to a complaint (complainants and members who have been subject of a complaint). To enable the evaluation to interview as many individuals as possible, some interviews were conducted with small groups of two to four individuals.

The evaluation used processes to ensure the confidentiality of participants.

• For IRB staff (Coordinating Members and members at large) and external stakeholders who are members of the IRB’s Consultative Committee (IRBCC),^{Footnote 6} the IRB sent an email that explained the evaluation and invited them to participate in an interview by indicating their interest directly to PRA.
• For parties to a complaint, the OI sent an email that explained the evaluation and invited them to participate in an interview by indicating their interest directly to PRA. By having the OI send the invitations, the identity of participants in the complaints process was not disclosed to any other area within the IRB. In addition, to protect the privacy and confidentiality of parties to a complaint, each party was interviewed separately.

As a result of these processes, the IRB does not know who participated in the evaluation among IRB Coordinating Members, members at large, members of the IRBCC, and participants in the complaints process.

Table 1 provides the distribution of interviewees by category of key informant.

Table 1: Key informant interviews

Interviews were coded and analyzed thematically. To indicate the relative frequency of opinions, the following descriptive scale was used to report on some aspects of key informant interview findings.

3.4 Limitations

Each line of evidence had some limitations, as noted below.

Document, file, and data review

In addition to the documentary record of the complaints, the OI maintains administrative data in a spreadsheet. While the administrative data were useful for the evaluation, they were not maintained in a format that easily mapped onto the information in the annual reports and did not contain information on certain variables of interest (e.g., the category of the complainant, whether they are a person appearing before the IRB, counsel, Minister’s counsel, IRB staff, a member of the public, or a representative of an organization). Much of the data were kept in inconsistent and/or text formats. When possible, the evaluation reviewed other documents provided, such as decision letters, to fill any gaps. Looking forward, the OI may want to review and revise how it maintains administrative data on complaints.

While not a limitation of the data collection method, the review of the administrative data and documentary record of the complaints had a circumscribed purpose. It was intended to provide an overview of how complaints have been handled by the IRB and to offer some context for the comments made by key informants related to the complaints process implementation and outcomes. The document and data review was not intended to second-guess the conclusions and recommendations made by either the OI or the external investigator or the decision of the Chairperson on complaints.

Case studies

As noted above, case studies were selected based on suggestions made in the scoping interviews and a scan of the websites of other federal tribunals for complaints processes. The case studies are used to compare the IRB’s complaints process to those used by other judicial or quasi-judicial organizations (i.e., courts or other tribunals).

Finding comparable organizations was challenging because the IRB is unique in many respects. It is the largest independent administrative tribunal in Canada, with both a large volume of cases and number of members. In addition, refugee determination is not generally an adversarial system, and RPD members are expected to engage in direct questioning of claimants to enable them to assess claimants’ credibility, among other issues of fact and law. This approach used by the RPD (the division with the most complaints) opens more possibilities for perceptions of unfairness in behaviour or conduct towards the claimant, and is not used in many judicial and quasi-judicial organizations where the process is adversarial. In addition, each complaints process was different in terms of stages, terminology, and what data were available.

The evaluation undertook any direct comparisons with care. Alignment of processes and data across the case studies to enable comparisons was challenging due to the differences in the organizations and how publicly available information is reported. Any limitations in comparisons are noted when they are presented in the report.

Key informant interviews

Few key informants had experience with or were sufficiently aware of the differences between the 2012 Protocol and the 2017 Procedures to compare the two complaints processes. Therefore, any comparison of the two processes relies largely on a review of the documentation available with key informant opinion noted where relevant.

In addition, some key informants (IRB and external) acknowledged that they had limited knowledge of the 2017 Procedures. This limitation was mitigated by interviewing as many individuals as possible across the three main key informant categories (IRB, external, and parties to a complaint) and ensuring that opinion based on factually incorrect information is not included in the report except as appropriate to demonstrate areas where the IRB can improve communications related to the complaints process.

Another important limitation was the possibility of introducing bias as a result of the approach to selecting participants for the key informant interviews, as well as the voluntary nature of participation in this data collection method. For interviews with IRB staff (Coordinating Members and members at large), external stakeholders on the IRBCC, and parties to a complaint (complainants and members), this risk was mitigated somewhat by taking a census approach (i.e., inviting all IRB members, IRBCC members, and parties to a complaint to participate). While this approach does not eliminate all forms of bias (most notably, self-selection bias among those who choose to be interviewed), it does remove the possibility of introducing bias at the sample development stage. The interviews with key informants also had the possibility of introducing self-reported response bias and strategic response bias. Self-reported response bias occurs when individuals are reporting on their own activities and may want to portray themselves in the best light. Strategic response bias occurs when the participants answer questions with the desire to affect outcomes.

The risk of bias was mitigated by the use of multiple lines of evidence, including objective sources of data, such as documents and administrative data (to the extent that these were available), to arrive at the overall evaluation conclusions. 

4.0 Findings

The findings are presented by three main evaluation themes related to the complaints process’s design, implementation, and effectiveness and efficiency. Responses to the evaluation matrix questions are categorized under the above-mentioned themes. Design: evaluation questions 1 and 2; Implementation: evaluation questions 3 and 4; Effectiveness: evaluation questions 5 and 8; and Efficiency: evaluation questions 6 and 7.

4.1 Design

The IRB designed the 2017 Procedures to address concerns about how the complaints process operated under the 2012 Protocol. External stakeholders had raised several issues with the Protocol both formally and informally, such as the role of member managers in the process and the need for the process to be independent of member managers, the accessibility of the process as information on it was difficult to locate and understand, and the lack of acknowledgement of complaints and service standards on timelines (from interviews and internal IRB documents). As a result of these shortcomings, “some in the counsel community and other interested parties had lost faith in the robustness and transparency of the IRB complaint process” (Immigration and Refugee Board of Canada, 2019b).

Consequently, the IRB undertook a review of the 2012 Protocol in 2017 to improve and strengthen the process. The review resulted in a complete overhaul of the member complaint mechanism. The 2017 Procedures were intended to create a process that is more clear, accessible, and transparent, as well as more efficient and able to administer complaints more consistently. Ultimately, this new process was expected to result in better accountability and oversight of member conduct (Standing Committee on Citizenship and Immigration, 2018). The differences between the two processes are set out below in Table 2 (Immigration and Refugee Board of Canada, n.d.).

Table 2: Comparison of the 2012 Protocol and the 2017 Procedures

As noted in Section 3.4, few key informants had sufficient experience with both processes to offer comparisons. Of the key informants (IRB and external stakeholders) who could compare the previous complaints process to the 2017 Procedures, almost all considered the new process to have addressed many of the concerns raised and to be an improvement over the 2012 Protocol.

• More centralization is considered an improvement by streamlining and clarifying the process. Under the 2012 Protocol, there were potentially multiple levels of review and decision-making that were not clearly set out, nor was it clear where or with whom to file a complaint. Regional managers could handle complaints themselves or they could refer complaints to their divisional head who could in turn reassign the complaint to another member manager for review or refer it to the Chairperson. The Chairperson could then assign it to another divisional head. This convoluted process was thought to be particularly difficult for unrepresented persons appearing before the IRB, who would not know the internal structure of the IRB and, therefore, how to file a complaint or what had occurred with the complaint once filed. The complaints process under the 2017 Procedures has more clearly articulated steps that are overseen by the OI.
• By clarifying the process, the 2017 Procedures also brought more transparency to the complaints process. Under the 2012 Protocol, the process was generally considered more opaque, as, once a complaint was filed, there was not a standardized approach to communications with the parties to the complaint. Parties may not have received an acknowledgement of their complaint or any information on how the complaint was ultimately handled (e.g., whether it was investigated, the decision, and corrective steps taken, if applicable). With the 2017 Procedures, the IRB has more standardized processes, including for acknowledging complaints and otherwise keeping parties to a complaint informed about the process. In addition, information about the complaints process is more visible on the IRB website. The IRB has also instituted more comprehensive annual reporting, which includes statistics of complaints received for the year and detailed case summaries.
• Key informants (primarily IRB and a few external stakeholders) also highlighted that the 2017 Procedures enable a process that has more consistency and objectivity in its decision-making. Under the 2012 Protocol, each division’s regional office could handle similar complaints differently because there was no centralized oversight. In addition, with the first step in addressing complaints under the 2012 Protocol residing with the member’s direct manager, there was greater potential for lack of objectivity due to a closer direct personal relationship. Having a centralized process with OI involvement and oversight was seen as bringing needed third-party involvement in the process, given the OI’s independence from the divisions. In addition, the new process allows for the use of an external investigator in “exceptional cases,” which is determined based on the “seriousness of the alleged conduct and other relevant factors” (Immigration and Refugee Board of Canada, 2017, sec. 5.5 (d)).

Key informants noted that the external stakeholders’ desire to have an independent body external to the IRB review, investigate, and decide complaints was not incorporated into the design of the complaints process under the 2017 Procedures. IRB key informants believe that the OI’s role within the IRB made it a reasonable location for the complaints process given its independence from the divisions, its knowledge of the IRB’s work, and the additional context of the members being public servants and GICs. Another option suggested by external stakeholders was to create an independent body to handle the complaints. This independent body could either be for all federal tribunals, (something akin to the CJC for the judges in superior courts across Canada), which is not something that the IRB can undertake on its own, or it could be for the IRB alone, but this could require new legislation (see Section 4.3.2 for a further discussion).

The remaining sections of the report consider many of the above design features of the 2017 Procedures and complaints process in more detail.

4.2 Implementation

Overall, the evaluation found that the process is working as intended, with some areas for improvement, which are also highlighted in this section.

4.2.1 Accessibility and use

Accessibility

Awareness and understanding of the complaints process are key measures of accessibility. The IRB engaged in consultations prior to adopting the 2017 Procedures, which involved key stakeholders in the development of the new complaints process. The IRB also communicated with stakeholders as part of the initial rollout of the 2017 Procedures.

Methods by which the IRB has made the complaints process more accessible include displaying the complaints process more prominently on its website. A few key informants noted that the 2012 Protocol was difficult to find online and was not promoted, which meant that many potential users of the complaints process were likely not aware that one existed or, even if they knew of the process, would have difficulty finding information about it. In contrast, the links to both the 2017 complaints process (“Making a complaint to the IRB”) and reporting (“Member conduct issues”) are on the IRB home page.^{Footnote 7}

The IRB has also worked to ensure that the content on the website is in plain language so that any potential complainant, including those making a complaint without the assistance of counsel, can understand the process. To reduce barriers to filing a complaint, the online form is optional, and the form itself also allows great leeway in how the complainant can describe the claim.

Awareness of the complaints process appears to be an issue, as a few IRB key informants mentioned that ADCs sometimes still receive complaints directly. The issue raised more often by key informants (IRB and external) was the need for more information about the process to ensure they understand it (e.g., the role of the OI within the IRB, the process’s scope, what is required in a complaint, what is disclosed to a member and when). The desire for more information about the process is discussed further in Section 4.2.3. Based on key informant interviews, more engagement with internal and external stakeholders about the complaints process could improve accessibility and understanding of the process.

Use of the complaints process

Based on the annual reports on member complaints, a total of 43 complaints were filed between December 17, 2017 and December 31, 2019 under the 2017 Procedures (20 in 2018 and 23 in 2019). The volume of complaints filed needs to be considered in context by comparing the number of complaints made to the total number of cases finalized across the IRB. In 2018, there was one complaint for every 2,245 cases finalized (20 complaints filed and 44,897 cases finalized). In 2019, there was one complaint for every 2,783 cases finalized (23 complaints and 64,008 cases finalized) (Immigration and Refugee Board of Canada, 2018b, 2020b). Using the number of IRB members to provide further context, there were approximately 400 IRB members in 2019 and 23 complaints filed (Immigration and Refugee Board of Canada, 2020b, p. 9), which would be approximately one complaint for every 17 members. Unfortunately, comparable data are not available for the case study organizations.^{Footnote 8}

While who can file a complaint is broad—any party to IRB proceedings, IRB personnel, an organization, or the public—the data show that most complaints are made by representatives of persons appearing before the IRB. Based on the annual reports for 2018 and 2019, of the 43 complaints filed in those two years:

• 36 (84%) were filed by representatives of parties to a proceeding;
• 6 (14%) were filed by unrepresented individuals; and
• 1 (2%) was filed by an interpreter.

While unrepresented complainants are the minority at 14%, this is representative of the proportion of unrepresented persons appearing before the IRB, as, during approximately the same time period (December 2017 to March 2019), in 15% of finalized cases before the RPD and ID, the persons appearing before the IRB were unrepresented.^{Footnote 9}

Potential barriers

Some key informants pointed to the number of complaints filed as indicating potential barriers. The barrier to using the complaints process most cited by key informants (external stakeholders and complainants) was the fear of negative repercussions due to making a complaint. This concern arose in two settings. The first was when the complaint was made in an ongoing matter and the complaint was held in abeyance. Some key informants found it unclear when the member would be informed about the complaint and/or assumed that the member would be informed of the complaint before the matter was concluded. As a result, these key informants wanted the member removed from the ongoing case. This concern points to confusion and lack of clarity in the operations of the complaints process. When the member involved is still seized by the case, the complaint is held in abeyance and the member is not notified of the complaint until after the case is finalized. Many key informants (external stakeholders and complainants) emphasized the need for greater clarity on how the complaints process works when a complaint is held in abeyance.

The second setting reflected a more general concern of counsel who appear before the IRB for multiple clients or on multiple matters. They feared potential repercussions in other matters handled by the complainant before the same member. This potential exists with any complaints process absent of reassigning cases so the complainant does not appear before that member again or allowing anonymous complaints. In terms of reassigning all future cases from that complainant to other members, IRB key informants noted that this process was not practicable and would impact the efficiency of the IRB in handling its caseload.

As to anonymous complaints, a few key informants desired that the process allow them, but many (mostly IRB) key informants were concerned about the impact of anonymous complaints on procedural fairness to the member who needs to know the details of the complaint in order to be able to respond to it. Case study organizations are divided on allowing anonymous complaints—three do not allow anonymous complaints (VRAB, Provincial Court of BC, and TO) and two permit them (CJC and CJA). A more limited disclosure of complainants’ identities is one potential method of handling the confidentiality of complainants that would not remove this concern but would mitigate it. Rather than sharing the complaint with the member prior to the screening decision, the complaint and, therefore, the identity of the complainant, could remain undisclosed until the screening decision is made. At that time, complaints that are both within scope and that will be investigated would be shared with the member, while complaints out of scope would not be shared and the identity of the complainant, therefore, would not be disclosed. Another option is to have stakeholder groups act as a clearing house, where they would submit the claim on behalf of counsel or persons appearing before the IRB in order to protect confidentiality.

Many key informants (IRB, external stakeholders, and complainants) expressed concern that the process may still not be easily accessible for complainants who are not counsel. While none of the complainants who volunteered to be interviewed were unrepresented persons appearing before the IRB, complainants who are counsel, external stakeholders, and IRB key informants all commented that it may be difficult for persons appearing before the IRB to bring a complaint on their own behalf. They noted that the primary method for learning about the complaints process and obtaining the form requires access to the Internet, which not all unrepresented persons appearing before the IRB would have. In addition, complainants who were counsel noted that their legal training meant that they understood the level of detail required in the complaint. Because the online form does not provide many instructions, these key informants questioned whether unrepresented complainants would understand the level of detail required to clearly articulate their complaint and to properly frame their allegations so that the complaint would be within the scope of the complaint process.

Some key informants considered the effort required to file a complaint to be a barrier to using the process. Counsel noted that this is uncompensated work and can take hours to put together a sufficiently detailed complaint. While organizations can file complaints, it was noted that likely few do because of the effort that it would involve, particularly because organizations would be more likely to want to address a pattern of behaviour among IRB members, such as asking inappropriate questions that reflect a failure to understand particular types of trauma. These types of complaints could take substantial time to draft and obtain supporting information. Two suggestions were offered for how to potentially address this issue: offer a more informal preliminary complaints process that would take less time for counsel to participate in or provide potential complainants a better understanding of what is required in a complaint, particularly since the OI or an external investigator will be conducting an investigation of complaints found to be within scope.

Some key informants also suggested that the formality of the complaints process may deter some people from making a complaint. They suggested that the complaints process should include an informal approach, such as mediation. The 2017 Procedures do include the possibility of the OI attempting an informal resolution (Immigration and Refugee Board of Canada, 2017, sec. 5.3), but based on the administrative data, that does not appear to have been used, either because the OI did not pursue it or the complainant desired the formal approach.

4.2.2 Clarity and choerence

The desire for greater clarity about the scope of the complaints process as reflected in the Code of Conduct and the 2017 Procedures came from IRB interviewees, external stakeholders, and participants in the complaints process (both complainants and members).

Contributing to lack of clarity about scope appears to be inconsistent or incomplete messaging. As noted in Section 2.1, the Code of Conduct contains numerous provisions that are outside of the scope of the 2017 Procedures for making a complaint about a member. The IRB website now specifies (but did not always) which Code of Conduct provisions (Sections 9-15) can be the subject of a complaint. However, this restriction on the scope of the complaints process is not specified in the Code itself. The relevant sections are grouped together under a heading (“Conduct of Members”), but the entire document purports to be a code of conduct for IRB members, so the fact that only a subset of its provisions can support a complaint about member conduct is not clear from the Code itself. In addition, neither the 2017 Procedures nor the online complaints form specify which Code sections are within scope.

The lack of clarity regarding the scope of the complaints process is evident in the complaints filed. Based on the administrative data, there is confusion about which sections are within scope. The complaint form does not require complainants to indicate the relevant section in the Code of Conduct on which their complaint is based, and, as shown in Figure 3 below, most complaints did not specify a section of the Code. However, when they did, few cited a section that is within the scope of the complaints process. The Code sections most often cited are Sections 22 and 23.^{Footnote 10}

While the OI will review the complaint and determine if it is within scope and proceed accordingly regardless of whether the complainant indicates a Code section or even the appropriate Code section, it is fair to assume that most complainants believe their complaints have merit. As a result, when complaints are screened out of the process for being outside the scope, this creates frustration for complainants and, as will be discussed later in the report, creates credibility issues (see Section 4.3.2).

Code section cited in complaint, frequency, 2018-2019

Figure 3 Source: Administrative data

Interviewees (IRB, external stakeholders, and complainants) noted that the lack of clarity is reflected in the proportion of complaints that are found to be out of scope. Of the 32 complaints filed and concluded between December 17, 2017 and March 31, 2020, 19 (or 59%) did not proceed to an investigation. As shown in Figure 4, the most common reason was that the complaint was out of scope (n=13, or 41% of complaints).

Results of complaints with no investigation (n=19)

Figure 4 Source: Administrative data

All groups interviewed (IRB, external stakeholders, and parties to complaints [complainants and members]) commented that the scope of the complaints process is not clear. Key informants understood that the complaints process covers member conduct and not decisions. They acknowledged the importance of members’ adjudicative independence and that complaints related to legal matters more properly raised by appeal to the RAD or IAD or by application for judicial review at the Federal Court should not be the subject of a complaint related to member conduct. What caused concern among many key informants were the grey areas, and they queried how the complaints process, more specifically the OI, determines in those circumstances whether a situation involves conduct or the member’s decision-making.

Behaviour that creates the apprehension of bias was the major grey area raised by key informants. Under the 2017 Procedures, if a complaint is made about an ongoing proceeding, and the complaint alleges that the IRB member is biased in conducting the proceeding, the allegation of bias should be raised with the presiding member at the first reasonable opportunity (Immigration and Refugee Board of Canada, 2017, sec. 7.1). However, many external stakeholders and some IRB key informants believe there is room for the inclusion of bias in the complaints process when the bias allegations also raises conduct issues.

The Code includes the provision that “members shall conduct themselves in a manner that will not cast doubt on their ability to perform their duties objectively” (Immigration and Refugee Board of Canada, 2019a, sec. 37). This definition of bias involves the perception that the member has prejudged the case, and some key informants (external, IRB, complainants) believe that this type of bias gives rise to situations that straddle conduct and decisions. These key informants noted that this type of bias involves issues that should be raised in the hearing room and then, as needed, on appeal or in Federal Court, but it can also include behaviour that should be addressed through the complaints process. Examples provided included behaviour that indicates unconscious bias or failure to follow some of the new Chairperson Guidelines (e.g., Guideline 9: Proceedings before the IRB Involving Sexual Orientation and Gender Identity and Expression) (Immigration and Refugee Board of Canada, 2018c).

IRB key informants recognized that the treatment of complaints that raised issues of bias had been an issue; if bias was mentioned, it would be screened out too easily. During the second year of the new complaints process, the OI received more assistance from Legal Services so that these determinations are more nuanced, and when bias is reflected in conduct that could be the subject of a complaint, the intention is that the complaint remains in scope. However, the evaluation found a continued concern among both IRB and external stakeholders about how the process is handling complaints that raise issues of bias. Among IRB key informants, there are strong views on both sides, with some members believing that bias should never be included in the complaints process while others desire the more nuanced approach that attempts to separate behaviour that demonstrates bias from legal claims of bias that are appealable. Key informants expressing both positions noted that the IRB needs to clarify the criteria being used by the OI for determining whether a complaint that raises issues of bias is within scope.

Another example of situations where behaviour and decision-making blur involved IRB decisions that included allegedly disrespectful language. Some complainants and external stakeholders noted that sometimes the objectionable language is not appealable and, if it cannot be part of the complaints process, goes unaddressed. This could occur when the language at issue is not relevant to the decision or when the decision is favourable to the complainant’s legal positions. These stakeholders believe that they were not seeking to change a decision and were left without an avenue for raising legitimate concerns about conduct evidenced through oral and/or written statements.

Many of the key informants commenting on scope believe that there is a need to review the Code of Conduct and clarify the scope of the complaints process. For some of these key informants, just because there is the potential for appeal or judicial review of a legal issue does not mean that the underlying behaviour cannot also be the subject of a complaint. Similarly, just because the disrespectful conduct is found in the language of an opinion does not mean that it is by definition outside of scope. As noted above, the OI is taking a more nuanced approach, but from the perspective of some stakeholders, the lack of clarity about how these grey areas are treated creates concerns about transparency, as well as consistency in approach. These key informants desired a review of the Code of Conduct in order to address behaviour related to unconscious bias and cultural insensitivity.

While the evaluation found support for reviewing the Code of Conduct and the scope of the complaints process, it is important to note that a comparison of the scope of the IRB’s complaints process to the case study organizations’ processes shows substantial similarities in approach. All five organizations indicate that the complaints process addresses issues with behaviour and is not to be used to revisit or reverse decisions.^{Footnote 11} The CJC’s Ethical Principles for Judges, which is also used by the Provincial Court of BC, sets the standards by which judges are expected to conduct themselves (Canadian Judicial Council, 2004). Based on available documentation and interviews, complaints within scope relate to the conduct of judges and consider the manner in which judicial services are provided (Canadian Judicial Council, n.d.a; Provincial Court of British Columbia, n.d.). TO and the CJA use similar language (Conseil de la justice administrative, 2020; Tribunals Ontario, n.d.). However, the CJA does provide examples of conduct that includes aspects of decision-making. For example, a complaint can involve claims of an abuse of authority in the conduct of a hearing, such as refusing or granting a postponement without valid reasons or rendering a decision on an issue that has not been submitted to the tribunal by one of the parties (Conseil de la justice administrative, 2020). In interviews, two of the organizations (CJC and TO) specifically addressed the issue of bias, stating that it is not within scope.

The IRB is not unique in receiving criticism about the clarity of the scope of its complaints process. The CJC representative noted that the scope of its process is not perceived to be clear by internal or external stakeholders, especially the distinction between judicial discretion and conduct. Similar to the IRB’s Code of Conduct, the Ethical Principles for Judges is also wide-ranging and includes impartiality of judges as one of its principles, with a section on judicial demeanour, which refers to the importance of the perception of fairness in judicial conduct (Canadian Judicial Council, 2004, p. 33).

4.2.3 Transparency

Transparency of the complaints process has three intersecting dimensions:

• communications about ongoing complaints
• the complaints process in general
• public accountability

The evaluation found that the IRB complaints process has improved transparency in all three areas.

Communications about ongoing complaints

As mentioned in Section 4.1, the 2017 Procedures were intended to address issues with transparency that existed under the 2012 Protocol. Because the Protocol was more informal and decentralized, the communications with the parties to a complaint depended on the management style of the manager handling the complaint. The complaints process under the 2017 Procedures put in place a more standardized approach to communicating with both complainants and members who are the subject of a complaint.

As shown in Figure 5, the nature of the communications with parties to a complaint varies somewhat based on the complaint (e.g., whether the complaint will be held in abeyance, or the complainant has requested that the complaint be dealt with immediately due to exceptional circumstances), but there are standard touchpoints:

• acknowledging receipt of the complaint
• notifying the member of the complaint
• notifying the parties of the screening decision
• if screened in, providing the member with the opportunity to provide a written response to the complaint
• providing both parties with the draft investigation report and an opportunity to provide additional written submissions
• providing the Chairperson’s decision to both parties

In each of the above communications, the OI also notifies the recipient(s) of the next steps in the process.

Communications with parties to a complaint

Figure 5

Based on the interviews with parties to complaints, complainants and members generally appreciated the communications from the OI about the progress of their complaint and the next steps. Other key informants (IRB and external stakeholders) also commented that the process was more transparent than the 2012 Protocol due to the regularity of the communications from the OI regarding the complaints.

Based on interviews with parties to complaints and IRB key informants, some areas of further improvement exist. In particular, the evaluation found issues with the following:

• Consistency of communications: Some parties said they received correspondence from the OI when its handling of the complaint was delayed, while others reported a lengthy wait with no communication from the OI. Based on the information available, it is difficult to know whether the perceived lack of communications from the OI is due, at least in part, to parties not understanding that their complaint is held in abeyance. However, at least some examples included the OI apologizing to the parties for the delay in contacting them.
• Clarity of communications related to decisions: Some parties were dissatisfied with these communications, as they believed that the reasons for the decision were not clearly explained. For example, complaints were screened out for being outside the scope of the complaints process, while parties believed that the complaint was about behaviour and, therefore, member conduct.
• Next steps for matters held in abeyance: Some parties noted that it was unclear when the member would be made aware of the complaint when the case is ongoing. The 2017 Procedures do not clearly state that members are not informed of complaints against them until the member is no longer seized by the case nor do the letters provided to complainants when they are notified that their complaint is being held in abeyance.
• Communications with IRB division management: Some IRB key informants commented that, while the more streamlined and centralized process was an improvement, the 2017 Procedures were unclear on the role of divisional management while a complaint is ongoing. DCs and ADCs are informed that a complaint has been filed, but they sometimes receive little information beyond that, and it was reported that the OI’s level of communication with them was inconsistent. Coordinating Members considered the complaints process opaque once the OI is handling a complaint because, under the Procedures, only the DCs and ADCs are made aware of complaints. Coordinating Members find it difficult as managers when the member who is the subject of the complaint comes to them with questions or has had interactions with the OI that the managers are unaware of. They are also uncertain if there is anything they should do once they know a complaint is filed, such as whether workload assignments should be changed or other actions taken until the complaint has been decided.

The overall complaints process

Based on interview findings, there is an intersection of the transparency of the process and its clarity and coherence. Key informants (IRB and external stakeholders) commented that the IRB has made the steps in the complaints process and who is making the decisions related to the complaint much more transparent with the Procedures, which is a substantial improvement from the 2012 Protocol.

Stakeholders desire the transparency of the process to go further. Among many of the stakeholders interviewed (IRB, external, and parties to a complaint), there is a desire for more clarity on what occurs within each step in the process, such as the criteria used for making decisions. For example, key informants wanted to know the following:

• the criteria used for determining whether a complaint is within scope
• the criteria used for deciding when to refer to an outside investigator
• what information members are told and when are they made aware about complaints
• what the OI recommendations were and if not followed by the Chairperson, why
• what steps, if any, were taken as a result of the complaint

As to the last bullet point, the evaluation found situations where the conduct was found to be inappropriate, which was included in the decision letter to both parties. However, the final decision letter provided to the complainant did not mention if any corrective steps were taken. While these details are in the annual report, complainants should not need to refer to this subsequent document for details about how their complaint was handled.

Another area where transparency could be improved is related to the role and nature of the OI. The rationale for housing the complaints process within the OI, including its relationship to the other areas of the IRB, is not clear to all stakeholders interviewed. In fact, a description of the OI is not easily found on the website, and the description of why it is overseeing the complaints process reads: “As an independent and objective arbiter, the Office also investigates complaints against members to determine if there has been a breach of the Code of Conduct for Members of the IRB and reports the findings directly to the Chairperson of the IRB.” Its role as a body that is “completely independent from IRB divisions and their members and reports directly to the Chairperson” is mentioned on the same page. However, the reasons why the OI should be considered independent for the purposes of handling a complaint against a member is not directly addressed.

Accountability

With the 2017 Procedures, the IRB instituted a more comprehensive annual reporting process. Under the 2012 Protocol, the reporting was minimal and consisted of a few sentences that stated the number of complaints received and their status. The reporting under the 2012 Protocol would not have assisted stakeholders in gaining an understanding of the scope of the complaints process and reasons for decisions.^{Footnote 12}

As noted earlier (see footnote 2), the annual reporting under the 2017 Procedures is intended to address the recommendation in the Standing Committee on Citizenship and Immigration’s report that the IRB report back to the committee on the status of complaints against members by February 2019. In response to the Standing Committee on Citizenship and Immigration’s report, the IRB produced the 2018 annual report and has continued with this reporting commitment as a method of providing transparency. The reporting has evolved between the 2018 and 2019 annual reports by expanding the information provided and providing a more user-friendly format. Both annual reports included case summaries of all complaints finalized during the year. The annual reports also provide information on the steps in the complaints process, its scope, and reasons for decisions, which should assist with improving stakeholders’ understanding of the process and how it operates.

Based on interviews, IRB and external stakeholders who are aware of the annual reports believe that the reports have contributed to the transparency of the process, particularly when compared to previous reporting under the 2012 Protocol. The evaluation findings also pointed to some areas of potential improvement. External stakeholders’ awareness of the annual reports could be improved, as interviews found that many of those interviewed did not know about the annual reports. In addition, the case summaries could potentially be improved for completeness or accuracy. In interviews with parties to a complaint, a few parties noted that, in their opinion, the annual report summaries did not reflect what occurred with their cases or that the information provided to parties by the OI as part of the complaints process was not as complete as the information in the annual report summaries.

When comparing the IRB’s annual reports to the reporting undertaken by the other case study organizations, the IRB’s reporting is the most comprehensive, with a publicly available annual report on the IRB website, statistics on the number of complaints received and outcomes, and case summaries of all complaints finalized during the year, which includes complaints screened out as being outside the scope of the process as well as complaints investigated.^{Footnote 13} Table 3 presents a comparison of the public reporting provided by the case studies, which is described in more detail below.

• Three of the organizations do not currently publish an annual report on their websites. The CJA does not publish an online annual report, although it does provide summaries of complainants that reach the stage of inquiries (which is in the range of two to four per calendar year).^{Footnote 14}
• The two other organizations have online annual reports like the IRB (CJC and the Provincial Court of BC). Their reports provide statistics such as the number screened out and the number of complaints closed, but they have limited statistical information on outcomes, although they do provide short case summaries. The CJC provides case summaries of a sample of complaints received; in its 2017-18 and 2016-17 annual reports, just over 648 complaints were closed and 48 summaries were provided on its website (Canadian Judicial Council, n.d.b, 2016, 2018). The Provincial Court of BC provides summaries for all examinations of complaints performed during the year, which are those complaints that are screened into the complaints process (i.e., are determined to be about conduct), which in 2018-19 was nine out of 370 complaints (Provincial Court of British Columbia, 2019).

Table 3: Comparison of reporting on complaints process - IRB and case study organizations

4.3 Effectiveness and efficiency

4.3.1 Investigations and outcomes^{Footnote 15}

Of the 32 complaints filed and concluded between December 17, 2017 and March 31, 2020, 13 (or 41%) proceeded to an investigation. The investigations were largely conducted by the OI with one conducted by an external investigator. According to interviews, for more recent complaints, external investigators are increasingly being used, but this was not evident yet based on the complaints concluded as of March 31, 2020. The investigations that resulted in findings (one complaint was withdrawn) were evenly divided between findings of a breach of the Code of Conduct or no breach. See Figure 6.

Findings of investigations (n=13)

Figure 6 SourceL Administrative data

Typically, investigations involve the OI or the external investigator reviewing the audio-recording, although in over one third of investigations, the transcript was also reviewed.^{Footnote 16} Key informants noted that transcripts are not always available. As noted in Section 4.2.3, when the draft investigation report is provided to the parties, they are offered the opportunity to provide responses. Figure 7 shows that few members provide written submissions and only one complainant provided additional information.

In general, parties to a complaint reported that the OI had conducted its work with respect and was fair in its treatment of the parties. Those who did not believe that they were treated respectfully or fairly often mentioned insufficient communications from the OI, decision letters that did not clearly state the reasons why a complaint was dismissed, and/or timeliness (see Sections 4.2.3, 4.3.2, and 4.3.3 respectively, for further discussion of these topics).

Evidence used in the investigation (n=13)

Figure 7 SourceL Administrative data

In four of the six cases where the Chairperson found a breach of the Code of Conduct, no other action was deemed to be necessary when the member had acknowledged that their conduct violated the Code. In two other cases, the Chairperson took corrective actions. In one case, the Chairperson found an apology to be warranted, and a verbal reprimand was given. In another, the Chairperson instructed member managers to use the high-level findings and lessons learned in the investigation to determine training needs for their members. See Figure 8. In addition, divisional managers can also examine complaints from a performance lens and take measures in addition to those requested by the Chairperson.

Based on interviews, the type of remedial actions and sanctions is an area of concern. IRB and external stakeholders mentioned that it is unclear what sanctions are available under the Code, which has led to the view among some key informants that the complaints process is more “window dressing” than a process that truly holds members accountable. There is also no process in place for follow-up to ensure that the division managers have taken actions based on the final decision.

Decisions in cases of breach of the Code (n=6)

Figure 8 SourceL Administrative data

4.3.2 Credibility, impartiality, and procedural fairness

This section considers the credibility, impartiality, and procedural fairness of the complaints process, through which the need for independence is assessed.

Credibility

Related to scope

Based on interviews, the credibility of the complaints process is closely connected to the issues of the clarity of scope. All groups interviewed (IRB, external stakeholders, parties to a complaint) indicated some concerns with the credibility of the process.

According to some key informants (IRB, external stakeholders, and complainants), when most complaints are found to be either out of scope or not a breach of a Code provision that is within scope, the credibility of the process is brought into question. In considering this concern with credibility, the evaluation compared the IRB’s proportion of complaints screened out of the process (i.e., found to be out of scope or frivolous) to the case study organizations. The results show that the IRB screens out fewer complaints for being outside of the scope of the process than the other case study organizations for which there is data.^{Footnote 17} It should be noted that the complaints processes of other organizations and the data reported do not precisely align with the IRB’s processes and reporting, but the evaluation has attempted to make the comparisons as accurate as possible.

• Based on the administrative data for completed complaints, over half of the IRB complaints (n=19, 59%) do not proceed to an investigation. However, not all of these are out of scope; some were withdrawn by the complainant, administratively closed because of lack of response from the complainant, or, while not appropriate for the complaints process, were referred to another IRB process.^{Footnote 18} Fewer than half (n=13, 41%) of the 32 completed complaints were screened out for being out of scope (i.e., not about behaviour).
• For the CJC, in its most recent reporting year (2017/18), 359 complaints were opened and 285 (79%) were screened out (i.e., do not involve conduct or are trivial) (Canadian Judicial Council, 2018).
• For the BC Provincial Court, in its most recent reporting year (2018/19), 370 of 379 (98%) complaints were screened out. According to the annual report, most of the complaints screened out were not about conduct (i.e., were essentially an appeal from a judicial decision) (Provincial Court of British Columbia, 2019).

Another indication of the lack of credibility of the complaints process raised by key informants was its low usage. These key informants pointed to this as showing lack of faith in the process. Usage of the complaints process is discussed in Section 4.2.1 and, as shown there, the number of complaints is low, with 43 complaints filed between December 17, 2017 and December 31, 2019. It is difficult to definitively link the level of use of the complaints process to issues with credibility given the available information. A few interviewees mentioned being aware of potential complaints that were not filed due to the level of effort involved, because the potential complaints would be screened out or, even if screened in, the potential remedies are unclear.

As the discussion of the usage of the complaints process shows, credibility cannot be established solely by considering data, as credibility is dependent on perception. Based on interviews, the IRB’s process has some issues with credibility. As discussed in Section 4.2.2, key informants desire more clarity related to the process’s scope and the criteria used for determinations made at different points in the process. In particular, key informants (IRB, external stakeholders, and parties to a complaint) noted various areas that impacted the credibility of the complaints process:

• the lack of clarity of what is within scope
• the need for more transparency related to the criteria used for screening, using an external investigator, and determining whether there was a breach of the Code
• the need for more clarity regarding what remedies are available to hold members accountable for their conduct

Almost all of the complainants interviewed whose complaints were screened out commented that either no reasons were provided or the reasons were not clear. Based on the examples of the decision letters related to screening that were provided to the evaluation, the conclusions are stated but not the reasons why the complaint was viewed as not being about conduct. The screening decision letter also does not make clear that the OI does more than review the complaint but also, in some cases, listens to the audio-recording. Some complainants assumed that the audio-recording was not listened to and questioned the credibility of the process as a result. As one external stakeholder stated when asked what could improve the credibility of the process, “The more detail provided about how the process is handled, the better.”

A few IRB stakeholders (key informants and members to a complaint) expressed concerns with the credibility of the process from a different perspective. Their concerns centred on the perception that the complaints process does not have a mechanism to protect members from frivolous or vexatious complaints beyond screening them out.

IRB and external stakeholders emphasized the importance of transparency in building trust in the process. They noted that the more regularized communications with parties to a complaint and annual reporting under the 2017 Procedures will build trust in the process over time. They also expressed the need for ongoing outreach with stakeholders so that concerns are heard but also to ensure that the process is understood.

Related to the decision-makers

The other area in which credibility was raised by key informants related to the decision-makers for screening, investigating, and deciding complaints. As noted earlier, the change between the 2012 Protocol and the 2017 Procedures to a centralized structure was seen as enabling more consistency in decisions that would be less prone to bias and more objective, as the OI and the Chairperson are more removed from the members than their immediate supervisors.

As with the credibility issues related to the process’s scope, more clarity would improve how the entity responsible for the decision-making is viewed.

• The OI, as described in Section 4.1, is outside of the divisional structure and reports directly to the Chairperson, which is the basis for its greater independence from the divisions than the process under the 2012 Protocol. For many key informants, the role of the OI was less of a concern, although some external stakeholders did not fully understand the rationale for why the OI is the body managing the complaints process. Key informants commented that, for the OI to be seen as independent, they need to know what the OI’s recommendations are, if they are followed by the Chairperson, and if not, why not. That information is not currently provided in the complaint summaries in the annual reports. See also the discussion in Section 4.2.3.
• The role of the Chairperson as the ultimate decision-maker on complaints was generally not as much of an issue for key informants (IRB, external, and parties to a complaint) as the importance of more transparency in terms of how decisions are made – the criteria, reasoning, and whether or not they are following the OI or external investigator’s recommendations.

For some external stakeholders and complainants, any process that is within the IRB is not sufficiently independent and desire an external third party. IRB key informants pointed out issues related to removing the complaints process from the IRB, which they believe were not sufficiently explained to external stakeholders when the 2017 Procedures and the new complaints process was designed and rolled out. In particular, the Chairperson of the IRB is accountable to Parliament and the public for the behaviour of members, which is why it is important for the Chairperson to be involved in the complaints process. In addition, having a third-party body impose discipline for Code violations raises difficulties, as members of the RPD and the ID are public servants appointed in accordance with the Public Service Employment Act (Immigration and Refugee Protection Act, 2001, sec. 169,172). Under federal legislation, only the Chairperson or a delegated manager can make a decision about the discipline of a public servant member. Both the Chairperson and the GIC have legislative authority with respect to the discipline of members who are appointed by the GIC. Therefore, legislative changes would be required to support a third party being the ultimate decision-maker in the complaints process to the point of determining any disciplinary measures to be taken.

The case studies demonstrate that there is no single approach to ensuring credibility, objectivity, impartiality, and fairness in a complaints process. Of the five case studies, three involve processes similar to the IRB’s, with internal decision-makers (TO, BC Provincial Court, and VRAB).

• TO – The investigation and resolution of complaints are handled by the Associate Chair of the tribunal to which the member belongs, though the Executive Chair of TO may decide to perform this function.
• BC Provincial Court – The Chief Judge makes the decision, with the exception of complaints deemed serious enough that potential removal of the judge or justice is at issue and the Chief Judge has opted for an inquiry into the judge’s fitness to perform their duties. In those cases, the judge or justice may choose to have the inquiry conducted by the BC Judicial Council or by a BC Supreme Court judge.
• VRAB – The process is informal and generally handled by the Deputy Chairperson who investigates the complaint and makes recommendations to the Chairperson who then makes the final decision.

Two case study organizations have a process that either involves a separate body or incorporates external stakeholders.

• The CJC – The investigation and resolution of complaints against judges is a separate body with representation of all superior courts. This removes the complaints process from the specific court whose member is the subject of the complaint, but it still involves judges reviewing judges. The structure of the process is similar to the IRB’s in some respects. The Executive Director of the CJC screens the complaint to determine if it is within scope and then the Chairperson reviews the complaint and any other relevant materials including submissions from the parties. At this point, the Chairperson can make a decision, hold the matter in abeyance, or, similar to the BC Provincial Court process, if the complaint may be serious enough to result in the removal of the member, a Review Panel can be struck and decide that an inquiry committee be formed under the Judges Act to hear the matter. At that point, the complaint is outside of the CJC complaints process.
• The CJA – The complaints process includes external decision-makers and uses committees, rather than individual decision-makers. A seven-person review committee meets every two months to review complaints. The committee includes representatives of four of the five tribunals and three members of the public. This committee makes the screening decisions. If the complaint is deemed admissible, it is sent to a three-person inquiry committee that includes one person representing the public (who can be a lawyer or a notary), one peer of the accused judge (i.e., from the same tribunal), and one member of the CJA.

Impartiality

The available data indicate that the IRB’s process is objective and impartial. Breaches of the Code are found in half of the complaints that reach the investigation stage (6 out of 12). Key informants (IRB, external stakeholders, and complainants) raised issue with the proportion of complaints out of scope as indicating favouritism towards IRB members or at least creating that perception. As described above, the IRB screens out a lower proportion of its complaints than the case study organizations, but based on interviews, the IRB’s communications regarding the complaints process could be clearer and more detailed to address the perception of partiality towards members.

The use of external investigators for some complaints is seen as an improvement by a few key informants, but there is a lack of transparency and awareness regarding their use. A few key informants noted that external investigators are still paid by the IRB, which they believe could influence the decisions made by the investigators.

Procedural fairness

Parties to complaints were divided on whether the process is procedurally fair. Those who considered it fair emphasized that the process gave them the opportunity to be heard and considered the process to be generally respectful. For those concerned about the fairness of the process, communications were again a major issue.

• Some parties were concerned with the lack of updates and direct communications from the OI, particularly in those cases where the complaint was screened out.
• Some raised concerns with not getting an opportunity to respond to the other parties’ submissions after the initial complaint.
• Others noted that they did not know when or with whom information regarding the complaint was shared.

4.3.3 Efficiency

The evaluation considered the efficiency of the complaints process based on its timeliness in handling complaints. In general, key informants (IRB, external, and parties to a complaint) considered timeliness to be an issue for the complaints process. IRB key informants pointed out that the OI has a mandate that goes beyond the complaints process, as it also handles issues related to values and ethics in the workplace for IRB staff (e.g., conflicts of interest, workplace harassment, and conflict resolution). In addition, the OI has a small staff consisting of a director and two staff. It is also provided support by lawyers from IRB Legal Services. As a result, the OI has experienced capacity issues which have impacted timeliness.^{Footnote 19}

Timeliness was not as much an issue for the acknowledgement of complaints. In its Second Annual Report (2019), the OI has reported that, on average, it acknowledges complaints within four working days.

Key informants more frequently raised issues with the length of time to resolve the complaint. As shown in Table 4, the average time to decide a complaint not held in abeyance was 114 days from receipt of the complaint, and for complaints held in abeyance, the average time to decide the complaint was 132 days from the date the proceeding was finalized and the member was no longer seized by the matter.

Table 4: Number of days to decision for all complaints filed and concluded between December 17, 2017 to March 31, 2020, n=32

The length of time to resolve complaints had some negative impacts for the parties involved. Complainants noted that they were uncertain what the member or others in the IRB might know about the complaint, which made them uncomfortable during their IRB appearances. Members involved in complaints found waiting for a decision stressful. However, some key informants noted that for the process to be fair, particularly when it goes to the investigation stage, there needs to be time for members to respond to the complaint and both parties to provide submissions after reviewing the draft investigation report.

Similar to other areas of concern for IRB and external stakeholders (e.g., the scope of the complaints process), there was a desire for clearer statements from the IRB about the process. For timeliness, this desire took the form of wanting service standards so parties know how long the process should take. The OI had considered setting service standards earlier but determined that it first needed to gain some experience with the new process. The OI wanted to set service standards that were reasonable based on its workload constraints and that appropriately balanced the need for an expeditious time to decision with requirements of procedural due process.

Two of the case study organizations have set some service standards or performance metrics for timeliness of service in resolving complaints.

• The CJC representative interviewed reported that that the CJC has set as a goal to review 80% of complaints within three to six months (approximately 90 to 180 days) and 90% within nine months (approximately 270 days), which it is currently meeting.
• In its 2019-24 strategic plan, the CJA set service standards with respect to the timeframes for dealing with a complaint. In 2017-2018, inadmissible complaints were dealt with in four months, while admissible complaints were dealt with in 24 months. The CJA has a stated goal of reducing this time for handling complaints by setting new targets for each of the next five years starting with 2019-2020. In that year, the goal is to reduce the time for inadmissible and admissible complaints to three months and 20 months, respectively, and by 2023-24 to two and 12 months, respectively (Conseil de la justice administrative, 2019, p. 4).

These two approaches to service metrics show two different options that the IRB could consider. One approach (CJC) sets the service metric without consideration as to whether the complaint is within scope (screened in or admissible) or outside of scope (screened out or inadmissible). The other approach (CJA) creates separate metrics for cases that are concluded at the screening stage and those that receive further review/investigation and a final decision on whether there was a breach.

A critical part of the IRB complaints process is holding some files in abeyance until the case is finalized to ensure that members will not be influenced by the fact that a complaint was filed against them. Whether or how the above service metrics for the case study organizations take into account complaints that are held in abeyance is not clear. As a result, a direct comparison of the IRB’s current performance related to timeliness cannot be accurately made.

The points below provide the closest comparison possible given the information available for completed complaints between December 17, 2018 and March 31, 2020. Based on these results, the IRB’s complaints process compares favourably.

• When considering files held in abeyance (from time case is finalized until complaint decided) and files not held in abeyance (from receipt of complaint until decided) combined, 65% of complaints are decided within six months (180 days) and 94% are decided within nine months (270 days).
• The average number of days to resolve complaints that were screened as being out of scope (i.e., inadmissible) was 84 days. For complaints that were investigated (i.e., were admissible), the average time to decision was 187 days, which is approximately three months for out of scope complaints and six months for complaints that are investigated. To calculate both numbers, the time the file was held in abeyance, if applicable, was not counted.

5.0 Conclusions and recommendations

5.1 Design

Overall, the 2017 Procedures and the design of the new complaints process addressed several of the concerns raised by stakeholders and are considered to be an improvement over the 2012 Protocol. In particular, the new complaints process is considered to be:

• more streamlined and to have more clearly articulated steps;
• more transparent in terms of communications with parties to a complaint and annual reporting of statistics and complaint summaries; and
• more consistent and objective by having the process managed by a central body, the OI, which is independent from the IRB divisions.

5.2 Implementation

Accessibility and use: The complaints process is generally considered to be accessible, with the process prominently displayed on the IRB website and an online optional complaint form that is in an easy to use format. However, awareness of the complaints process appears to be an ongoing issue, and additional communications and guidance would improve accessibility, particularly for unrepresented complainants, but for other stakeholders as well. In particular, more engagement with internal and external stakeholders about the complaints process could improve accessibility and understanding of the process, including the role of the OI and its position within the IRB.

A total of 43 complaints have been filed in its first two years of operations (December 2017 to December 2019), which is a small number in the context of the size of the IRB (approximately 400 members) and its volume of work (approximately 60,000 claims finalized in 2019). The evaluation cannot assess whether this is due to the quality of IRB members or potential barriers to the complaints process, although it is likely a combination of the two. Key informants noted several potential barriers to filing complaints, but most often raised the fear of negative repercussions for those complainants who appear before the IRB on multiple matters (e.g., representatives of parties to an IRB proceeding, including hearings officers).

Recommendation 1: Promote understanding of the complaints process with outreach activities for both internal and external stakeholders.

Recommendation 2: Consider methods to limit the sharing of the identity of complainants until it is determined that a complaint is within scope and will be investigated.

Clarity and coherence: The scope of the complaints process and its intersection with the Code of Conduct require clarification. There remains confusion about which Code provisions are within the scope of the complaints process and disagreement about the current scope of the complaints process. The Code includes a number of provisions that are about member conduct but are outside of the complaints process. One of these provisions refers to allegations of bias, which are appealable and, therefore, outside of the complaints process. Key informants consider bias allegations to touch on both behaviour and decision-making. Behaviour underlying claims of bias are thought to be screened out too often by external stakeholders and some IRB key informants. Overall, the evaluation found a desire for greater clarity regarding the criteria used for determining whether to screen complaints into or out of the process.

Recommendation 3: Review the Code of Conduct and its intersection with the 2017 Procedures to clarify the scope of the complaints process in consultation with stakeholders.

Transparency: The IRB has put mechanisms in place that have improved transparency: a clear outline of the steps in the complaints process, expectations regarding communications with parties to a complaint, and extensive annual reporting. Parties generally appreciated the communications from the OI about the progress of their complaint. The evaluation identified some areas of improvement related to the communications with parties:

• the need for greater consistency in terms of when the OI communicates with parties;
• the provision of more clarity about the reasons for decisions (both screening decisions and final decisions); and
• notification to parties regarding when a complaint is communicated to others within the IRB, particularly for complaints held in abeyance.

Regarding transparency of the complaints process overall, the evaluation found the steps in the process to be set out in the 2017 Procedures, yet the process is still considered to be somewhat opaque to stakeholders in terms of how decisions are made at each step, including the criteria used to make decisions (e.g., whether complaint is within scope, whether to use an external investigator, whether there was a breach of the Code, what remedies were selected), what the OI recommendations were and if the Chairperson did not follow them, why not, and what steps, if any, were taken as a result of the complaint.

The annual reporting is a major improvement over the 2012 Protocol, but many external stakeholders are unaware of it and the type of information included.

Recommendation 4: Consider approaches to enhance transparency in terms of improvements in communications with parties, clarification of criteria used at key points in the process, and promotion of annual reporting.

Recommendation 5: To assist with reporting and ensure all variables of interest are captured, consider improvements to the method used to track data on complaints.

5.3 Effectiveness and efficiency

Investigations and outcomes: The OI is generally considered to be conducting its job in accordance with the 2017 Procedures and to be respectful in its approach. External investigators are reportedly being used more often, which is hoped to further strengthen the perception of the independence of the process. The main issue identified is the need to clarify what remedial actions and sanctions are available and how they should be applied, including any follow-up.

Recommendation 6: As part of the review of the Code of Conduct and 2017 Procedures, consideration should be given to what remedies and sanctions should be available, the criteria for their use, and what follow-up steps should be in place.

Credibility, impartiality, and procedural fairness: By considering the credibility, impartiality, and procedural fairness of the complaints process, the evaluation assessed whether there is a need for independence. Overall, the complaints process appears to be credible, impartial, and procedurally fair, although more transparency will help build greater confidence in the process. The available data supports this conclusion. In particular, the IRB screens out fewer complaints than complaints processes in other similar adjudicative bodies, and breaches of the Code are found in half of the complaints that proceed to the investigation stage.

Taking stakeholder opinions into account is fundamental to creating and sustaining a credible, objective, and fair process. Stakeholders who questioned the credibility of the complaints process primarily raised issues related to the clarity of the process’s scope. Their concerns, as also noted above under “clarity and coherence” and “transparency,” often centred on the need for a better understanding of what criteria are used to make decisions and better communications about the process.

The other concern raised by stakeholders was related to the decision-makers for screening, investigating, and deciding complaints. In general, the OI was perceived to be an independent body as it is outside of the divisional structure. In addition, external investigators are reportedly being used more often. Given the independence of these actors and their role in providing recommendations to the Chairperson, key informants desired to know what the OI’s and external investigator’s recommendations were in specific complaints. In addition, the role of the Chairperson in the complaints process was also generally not considered to be an issue, but there was, again, a desire for more transparency in terms of the criteria used for decisions, the reasoning used in specific decisions, and whether or not the Chairperson is following the OI’s recommendations or the external investigator’s conclusions.

Recommendation 7: Consider methods to further support the credibility of the complaints process, such as the use of external investigators, and either sharing OI recommendations and the external investigator’s conclusions or indicating whether they were followed.

Efficiency: The complaints process has experienced some issues with timeliness and has not yet set service standards. This is due, in part, to the capacity of the OI, which is being addressed by the expansion of that office. IRB key informants acknowledged the need to set service standards, and external key informants commented that having service standards would enhance the transparency and credibility of the process.

Recommendation 8: Set service standards for timeliness of service.

Reference list

Canadian Judicial Council. (n.d.a). Filing a complaint. Canadian Judicial Council.

Canadian Judicial Council. (n.d.b). The importance of an informed viewpoint. Canadian Judicial Council.

Canadian Judicial Council. (2004). Ethical principles for judges

Canadian Judicial Council. (2016). Annual Report—April 1, 2015 to March 31, 2016.

Canadian Judicial Council. (2018). Annual Report—April 1, 2017 to March 31, 2018.

Complaint Form. (2017). Immigration and Refugee Board of Canada.

Conseil de la justice administrative. (2019). Plan stratégique 2019-2024

Conseil de la justice administrative. (2020). Admissible complaints. Conseil de La Justice Administrative.

Immigration and Refugee Board of Canada. (n.d.). Procedures for Making a Complaint about a Member: Stakeholder Concerns.

Immigration and Refugee Board of Canada. (2008, October 10). Chairperson Guidelines 4: Women Refugee Claimants Fearing Gender-Related Persecution.

Immigration and Refugee Board of Canada. (2016, July 7). Report on the Protocol Addressing Member Conduct Issues: 2015–2016.

Immigration and Refugee Board of Canada. (2017, December 21). Procedures for Making a Complaint about a Member. Immigration and Refugee Board of Canada.

Immigration and Refugee Board of Canada. (2018a, January 5). Member complaints.

Immigration and Refugee Board of Canada. (2018b, February 16). Statistics.

Immigration and Refugee Board of Canada. (2018c, June 21). Chairperson’s Guideline 9: Proceedings Before the IRB Involving Sexual Orientation and Gender Identity and Expression.

Immigration and Refugee Board of Canada. (2019a, May 15). Code of Conduct for Members of the Immigration and Refugee Board of Canada. Immigration and Refugee Board of Canada.

Immigration and Refugee Board of Canada. (2019b, November 28). Report to the House of Commons Standing Committee on Citizenship and Immigration (CIMM) (First annual report on complaints).  

Immigration and Refugee Board of Canada. (2020a, February 3). About the Board. Immigration and Refugee Board of Canada.

Immigration and Refugee Board of Canada. (2020b, April 17). Second Annual Report on Complaints.

Immigration and Refugee Protection Act, S.C. 2001, c. 27 (2001).

Office of the Commissioner for Federal Judicial Affairs Canada. (2020, December 23). Federal Judicial Appointments—Number of Federally Appointed Judges in Canada.

Provincial Court of British Columbia. (n.d.). Complaints | Provincial Court of British Columbia.

Provincial Court of British Columbia. (2019). AnnualReport2018-2019.pdf.

Standing Committee on Citizenship and Immigration. (2018, September). Responding to Public Complaints: A Review of the Appointment, Training and Complaint Processes of the Immigration and Refugee Board [Report of the Standing Committee on Citizenship and Immigration].

Tribunals Ontario. (n.d.). Tribunals Ontario | Complaints Policy and Process.

Appendix A – Evaluation Matrix

Evaluation matrix

The matrix below identifies the issues and questions that the evaluation of the IRB’s complaints process addressed, along with the associated indicators and data sources.

Evaluation matrix for the IRB’s complaints process

Appendix B – Data Collection Instruments

Evaluation of the Immigration and Refugee Board of Canada’s (IRB) Process for Making a Complaint about a Member

IRB Key Informant Interview Guide

The IRB is conducting an evaluation of its Process for Making a Complaint about a Member (the process) that were issued in December 2017. The complaints process is set out in the Procedures and is handled by the IRB’s Office of Integrity, which is an ombudsman-type body within the Board that operates independently from the divisions and reports directly to the Chairperson.

The evaluation responds to the Parliamentary Standing Committee on Citizenship and Immigration’s request that the IRB report back to the committee with a review of its complaints process under the Procedures. In addition, the evaluation will observe requirements under the Treasury Board’s Policy on Results.

The evaluation will consider the design of the complaints process, how the Procedures are being administered, whether the process is fair, clear, accessible, and transparent, and whether there are improvements that can be made. The evaluation will use several lines of enquiry, including interviews with key informants; a document, data, and file review; and case studies of complaints processes used by other judicial or quasi-judicial organizations.

You have been identified as a potential interview participant based on your knowledge and experience within the IRB and with the complaints process.

Interviews are being conducted by PRA Inc., an independent research firm working in collaboration with the IRB on the evaluation. The interview will be by telephone, either one-on-one or in a group. The interview should take about 60 minutes and will be conducted in the official language of your choice.

Your participation in this interview is voluntary. The information you provide will be handled in accordance with applicable privacy laws.

The information gathered through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA unless required by law. With your permission, we will audio-record the interview to ensure the accuracy of the information we gather and report.

We realize that you may not be able to answer all questions; please let us know and we will skip to the next question.

Introduction

1. To begin, please describe your roles and responsibilities within the IRB. What has been your involvement with the process for making a complaint about a member?

Design and implementation

2. What do you consider to be the key features of the complaints process — from making a complaint to implementing decisions? To what extent do those features address issues or challenges that existed with the previous complaints process under the Protocol Addressing Member Conduct Issues (the Protocol)? [Q1 and Q2]
3. The new complaints process has been operating for just over two years. Is it operating as expected or do you think that there are aspects of its operations that reveal design and/or implementation gaps or challenges? [Q3]
4. How has the IRB communicated the new complaints process to its stakeholders? In your response, please consider internal IRB stakeholders and external stakeholders (e.g., claimants, authorized representatives, non-governmental organizations, and media). Do you believe that communication and outreach activities have been successful at building awareness and understanding of the complaints process and the role of the Office of Integrity? Why or why not? [Q4]
5. Are there any barriers to the use of the complaints process by potential complainants? If so, what are they? Are there any steps that the IRB has taken or could take to mitigate any barriers? [Q4 and Q5]
6. How would you describe the degree of clarity and coherence of the process under the Procedures, including how the Procedures are intended to intersect with and enforce provisions in the Code of Conduct for Members of the IRB? [Q4]
1. If you can, please compare the clarity of the process under the Procedures to the Protocol.
2. Do you have any suggestions to improve the clarity of the Procedures and/or the Code of Conduct as it relates to the Procedures?
7. How transparent is the complaints process in relation to both specific complaints and how the IRB and its Office of Integrity handle complaints more generally? [Q4]
1. If you can, please compare the transparency of how complaints are handled under the Procedures to the process under the Protocol.
2. Do you have any suggestions to improve the transparency of the process?
8. How successful do you think the Office of Integrity and the complaints process have been in terms of [Q5 and Q6]:
1. addressing complaints in a timely fashion
2. ensuring procedural fairness for both parties
3. treating parties with respect
4. maintaining objectivity and impartiality in its approach to complaints
5. taking appropriate measures to protect parties’ identities/privacy
6. holding Members accountable for improper conduct
7. determining whether complaints are within the scope of the Procedures

How would you compare the current complaints process to the Protocol regarding any of the above? [Q5]

9. How would you assess the credibility, impartiality, and objectivity of the complaints process from the perspectives of the IRB, its members, complainants, stakeholders, and the public? What factors support or impede its credibility, impartiality, and objectivity? [Q5]
10. Does the complaints process protect the adjudicative independence of the IRB members while still holding them accountable for any misconduct under the Procedures? Please explain why or why not. [Q5]
11. In addition to the complaints process, has the IRB taken any other measures to address issues of Member conduct? Please explain. [Q5]
12. Does the Office of Integrity have sufficient capacity and resources to effectively and efficiently handle the complaints process? If not, what are the gaps or areas in need of attention? [Q6 and Q7]
13. If you could make recommendations to the IRB for improving its complaints process, what would they be and why? [Q8]
14. Do you have any other comments about the complaints process that we haven’t already covered?

External Stakeholders Interview Guide

The IRB is conducting an evaluation of its Process for Making a Complaint about a Member (the process) that were issued in December 2017. The complaints process is set out in the Proc​edures and is handled by the IRB’s Office of Integrity, which is an ombudsman-type body within the Board that operates independently from the divisions and reports directly to the Chairperson.

The evaluation responds to the Parliamentary Standing Committee on Citizenship and Immigration’s request that the IRB report back to the committee with a review of its complaints process under the Procedures. In addition, the evaluation will observe requirements under the Treasury Board’s Policy on Results.

The evaluation will consider the design of the complaints process, how the Procedures are being administered, whether the process is fair, clear, accessible, and transparent, and whether there are improvements that can be made. The evaluation will use several lines of enquiry, including interviews with key informants; a document, data, and file review; and case studies of complaints processes used by other judicial or quasi-judicial organizations.

You have been identified as a potential interview participant based on your knowledge and experience with the IRB and the complaints process.

Interviews are being conducted by PRA Inc., an independent research firm working in collaboration with the IRB on the evaluation. The interview will be by telephone, either one-on-one or in a group. The interview should take about 60 minutes and will be conducted in the official language of your choice.

Your participation in this interview is voluntary. The information you provide will be handled in accordance with applicable privacy laws.

The information gathered through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA unless required by law. With your permission, we will audio-record the interview to ensure the accuracy of the information we gather and report.

We realize that you may not be able to answer all questions; please let us know and we will skip to the next question.

Introduction

1. To begin, please describe your job or organization and how you have been involved with the IRB. Have you had any direct experience with the process for making a complaint about a member?

Design and implementation

2. What do you consider to be the key features of the complaints process under the Procedures —both positive and negative? To what extent do those features address issues or challenges that existed with the previous complaints process under the Protocol Addressing Member Conduct Issues (the Protocol)? [Q1 and Q2]
3. The new complaints process has been operating for just over two years. Is it operating as expected or do you think that there are aspects of its operations that reveal design and/or implementation gaps or challenges? [Q3]
4. How has the IRB communicated the new complaints process to its stakeholders? Do you believe that communication and outreach activities have been successful at building awareness and understanding of the complaints process and the role of the Office of Integrity? Why or why not? [Q4]
5. Are there any barriers to the use of the complaints process by potential complainants? If so, what are they? Are there any steps that the IRB has taken or could take to mitigate any barriers? [Q4 and Q5]
6. How would you describe the degree of clarity and coherence of the process under the Procedures, including how the Procedures are intended to enforce provisions in the Code of Conduct for Members of the IRB? [Q4]
1. If you can, please compare the clarity of the process under the Procedures to the Protocol.
2. Do you have any suggestions to improve the clarity of the Procedures and/or the Code of Conduct as it relates to the Procedures?
7. How transparent is the complaints process in relation to both specific complaints and how the IRB and its Office of Integrity handle complaints more generally? [Q4]
1. If you can, please compare the transparency of how complaints are handled under the Procedures to the process under the Protocol.
2. Do you have any suggestions to improve the transparency of the process?
8. How successful do you think the Office of Integrity and the complaints process have been in terms of [Q5 and Q6]:
1. addressing complaints in a timely fashion
2. ensuring procedural fairness for both parties
3. treating parties with respect
4. maintaining objectivity and impartiality in its approach to complaints
5. taking appropriate measures to protect parties’ identities/privacy
6. holding Members accountable for improper conduct
7. determining whether complaints are within the scope of the Procedures

How would you compare the current complaints process to the Protocol regarding any of the above? [Q5]

9. How would you assess the credibility, impartiality, and objectivity of the complaints process? What factors support or impede its credibility, impartiality, and objectivity? [Q5]
10. Does the complaints process protect adjudicative independence of the IRB members while still holding them accountable for any misconduct under the Procedures? Please explain why or why not. [Q5]
11. In addition to the complaints process, do you think there are other measures that the IRB should take to address issues of Member conduct? Please explain. [Q5]
12. Does the Office of Integrity have sufficient capacity and resources to effectively and efficiently handle the complaints process? If not, what are the gaps or areas in need of attention? [Q6 and Q7]
13. If you could make recommendations to the IRB for improving its complaints process, what would they be and why? [Q8]
14. Do you have any other comments about the complaints process that we haven’t already covered?

Interview Guide for Participants in Complaints Process

The IRB is conducting an evaluation of its Process for Making a Complaint about a Member (the process) that were issued in December 2017. The complaints process is set out in the Procedures and is handled by the IRB’s Office of Integrity, which is an ombudsman-type body within the Board that operates independently from the divisions and reports directly to the Chairperson.

The evaluation responds the Parliamentary Standing Committee on Citizenship and Immigration’s request that the IRB report back to the committee with a review of its complaints process under the Procedures. In addition, the evaluation will observe requirements under the Treasury Board’s Policy on Results.

The evaluation will consider the design of the complaints process, how the Procedures are being administered, whether the process is fair, clear, accessible, and transparent, and whether there are improvements that can be made. The evaluation will use several lines of enquiry, including interviews with key informants; a document, data, and file review; and case studies of complaints processes used by other judicial or quasi-judicial organizations.

You have been identified as a potential interview participant based on your experience with the IRB’s complaints process.

Interviews are being conducted by PRA Inc., an independent research firm working in collaboration with the IRB on the evaluation. The interview will be by telephone. The interview should take about 45 minutes and will be conducted in the official language of your choice.

Your participation in this interview is voluntary. The information you provide will be handled in accordance with applicable privacy laws. 

The information gathered through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA unless required by law. With your permission, we will audio-record the interview to ensure the accuracy of the information we gather and report.

We realize you may not be able to answer all questions; please let us know and we will skip to the next question.

1. To the best of your recollection, what year was the complaint filed? What was the result of the complaint that you were involved in?
2. How did you first become aware of the complaints process? (Probe: IRB website, IRB staff member told them about it, counsel or representative told complainant) [Q4]
1. Did you receive any information from the IRB about the complaints process and the role of the Office of Integrity before you filed the complaint? What type of information did you receive? Who provided it? [Q4]
3. How useful was the information that you received from the IRB in helping you understand the complaints process? How useful was the information in helping you understand the role of the Office of Integrity? What was useful or helpful about the information? How could it be improved? [Q4]
4. Did you have any difficulties or challenges in filing your complaint with the IRB? (Probe: what information to include in complaint, format of complaint, where to file). Please explain. [Q4]
5. What communications did you receive from the Office of Integrity once the complaint was filed? (Probe: Did the Office of Integrity let you know it had received the complaint?) Were you kept informed of the progress on the complaint? Were you interviewed as part of the investigation of the complaint? Were you told about the outcome of the complaint? What information would you have liked to receive from the Office of Integrity that you did not? [Q4]
6. Was the complaint dismissed or not handled by the Office of Integrity? What were the reasons given? Did you understand the reasons?  [Q4]
7. Is it clear what types of concerns are considered within the scope of the Procedures, and what types of concerns (e.g., matters unrelated to member conduct) are properly raised on appeal or on judicial review? Please describe what, if anything, is unclear. [Q4]
8. How successful do you think the Office of Integrity and the complaints process was in terms of [Q5 and Q6]:
1. addressing the complaint in a timely fashion
2. ensuring procedural fairness for both parties
3. treating the parties with respect
4. maintaining objectivity in its approach to complaints
5. taking appropriate measures to protect parties' identities/privacy
6. holding IRB Members accountable for improper conduct
7. determining whether complaints are within the scope of the Procedures

How would you compare the current complaints process to the Protocol regarding any of the above? [Q5]

9. How would you assess the credibility, impartiality, and objectivity of the complaints process? What factors support or impede its credibility, impartiality, and objectivity? [Q5]
10. Does the complaints process protect adjudicative independence of the IRB members while still holding them accountable for any misconduct under the Procedures? Please explain why or why not. [Q5]
11. Tell me if you thought the complaints process did a very good job, an okay job, or a poor job of the following and explain why: [Q5and Q5]
1. keeping you informed about what was happening with your complaint
2. deciding your complaint in a reasonable period of time
3. treating you fairly
4. treating you with respect
5. making you feel heard or understood
6. protecting your privacy so others in the IRB did not know that you had filed a complaint
12. Are there any barriers to the use of the complaints process by potential complainants? If so, what are they? Are there any steps that the IRB has taken or could take to mitigate any barriers? [Q4 and Q5]
13. How would you describe the degree of clarity and coherence of the complaints process under the Procedures, including how the Procedures are intended to enforce provisions in the Code of Conduct for Members of the IRB? [Q4]
1. If you can, please compare the clarity of the process under the Procedures to the Protocol.
2. Do you have any suggestions to improve the clarity of the process under the Procedures and/or the Code of Conduct as it relates to the Procedures?
14. How transparent is the complaints process in relation to both specific complaints and how the IRB and its Office of Integrity handle complaints more generally? [Q4]
1. If you can, please compare the transparency of how complaints are handled under the Procedures to the process under the Protocol.
2. Do you have any suggestions to improve the transparency of the process?
15. In addition to the complaints process, do you think that there are other measures that the IRB should take to address issues of Member conduct? Please explain. [Q5]
16. Based on your experience, does the Office of Integrity have sufficient capacity and resources to effectively and efficiently handle the complaints process? If not, what are the gaps or areas in need of attention? [Q6 and Q7]
17. If you could make recommendations to the IRB for improving its complaints process, what would they be and why? [Q8]
18. What could be better about the complaints process to help other people like yourself who want to make a complaint about an IRB Member's conduct?
19. Do you have any other comments about the complaints process that we haven't already covered?

Interview Guide for Participants in Complaints Process (Members)

The IRB is conducting an evaluation of its Process for Making a Complaint about a Member (the process) that were issued in December 2017. The complaints process is set out in the Procedures and is handled by the IRB's Office of Integrity, which is an ombudsman-type body within the Board that operates independently from the divisions and reports directly to the Chairperson.

The evaluation responds the Parliamentary Standing Committee on Citizenship and Immigration's request that the IRB report back to the committee with a review of its complaints process under the Procedures. In addition, the evaluation will observe requirements under the Treasury Board's Policy on Results.

The evaluation will consider the design of the complaints process, how the Procedures are being administered, whether the process is fair, clear, accessible, and transparent, and whether there are improvements that can be made. The evaluation will use several lines of enquiry, including interviews with key informants; a document, data, and file review; and case studies of complaints processes used by other judicial or quasi-judicial organizations.

You have been identified as a potential interview participant based on your experience with the IRB's complaints process.

Interviews are being conducted by PRA Inc., an independent research firm working in collaboration with the IRB on the evaluation. The interview will be by telephone. The interview should take about 45 minutes and will be conducted in the official language of your choice.

Your participation in this interview is voluntary. The information you provide will be handled in accordance with applicable privacy laws. 

The information gathered through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA unless required by law. With your permission, we will audio-record the interview to ensure the accuracy of the information we gather and report.

We realize you may not be able to answer all questions; please let us know and we will skip to the next question.

1. To the best of your recollection, what year was the complaint filed? What was the result of the complaint that you were involved in?
2. Did you receive any communication from the IRB about the new complaints process and the role of the Office of Integrity prior to the complaint you were involved in? What type of information did you receive and who provided it? [Q4]
3. How useful was the information that you received from the IRB in helping you understand the complaints process? How useful was the information in helping you understand the role of the Office of Integrity? What was useful or helpful about the information? How could it be improved? [Q4]
4. What communications did you receive from the Office of Integrity once the complaint was filed? Were you kept informed of the progress on the complaint? Were you interviewed as part of the investigation of the complaint? Were you told about the outcome of the complaint? What information would you have liked to receive from the Office of Integrity that you did not? [Q4]
5. Is it clear what types of concerns are considered within the scope of the Procedures, and what types of concerns (e.g., matters unrelated to member conduct) are properly raised on appeal or on judicial review? Please describe what, if anything, is unclear. [Q4]
6. How successful do you think the Office of Integrity and the complaints process was in terms of [Q5 and Q6]:
1. addressing the complaint in a timely fashion
2. ensuring procedural fairness for both parties
3. treating the parties with respect
4. maintaining objectivity in its approach to complaints
5. taking appropriate measures to protect parties' identities/privacy
6. holding IRB Members accountable for improper conduct
7. determining whether complaints are within the scope of the Procedures

How would you compare the current complaints process to the Protocol regarding any of the above? [Q5]

7. How would you assess the credibility, impartiality, and objectivity of the complaints process? What factors support or impede its credibility, impartiality, and objectivity? [Q5]
8. Does the complaints process protect adjudicative independence of the IRB members while still holding them accountable for any misconduct under the Procedures? Please explain why or why not. [Q5]
9. How would you describe the degree of clarity and coherence of the complaints process under the Procedures, including how the Procedures are intended to enforce provisions in the Code of Conduct for Members of the IRB? [Q4]
1. If you can, please compare the clarity of the process under the Procedures to the Protocol.
2. Do you have any suggestions to improve the clarity of the process under the Procedures and/or the Code of Conduct as it relates to the Procedures?
10. How transparent is the complaints process in relation to both specific complaints and how the IRB and its Office of Integrity handle complaints more generally? [Q4]
1. If you can, please compare the transparency of how complaints are handled under the Procedures to the process under the Protocol.
2. Do you have any suggestions to improve the transparency of the process?
11. In addition to the complaints process, do you think that there are other measures that the IRB should take to address issues of Member conduct? Please explain. [Q5]
12. Based on your experience, does the Office of Integrity have sufficient capacity and resources to effectively and efficiently handle the complaints process? If not, what are the gaps or areas in need of attention? [Q6 and Q7]
13. If you could make recommendations to the IRB for improving its complaints process, what would they be and why? [Q8]
14. Do you have any other comments about the complaints process that we haven't already covered?

Case Study Interview Guide

The IRB is conducting an evaluation of its Process for Making a Complaint about a Member (the process) that were issued in December 2017. The complaints process is set out in the Procedures and is handled by the IRB’s Office of Integrity, which is an ombudsman-type body within the Board that operates independently from the divisions and reports directly to the Chairperson.

The evaluation responds to the Parliamentary Standing Committee on Citizenship and Immigration’s request that the IRB report back to the committee with a review of its complaints process under the Procedures. In addition, the evaluation will observe requirements under the Treasury Board’s Policy on Results.

The evaluation will consider the design of the complaints process, how the Procedures are being administered, whether the process is fair, clear, accessible, and transparent, and whether there are improvements that can be made. The evaluation will use several lines of enquiry, including interviews with key informants; a document, data, and file review; and case studies of complaints processes used by other judicial or quasi-judicial organizations.

You have been identified as a potential interview participant based on your experience with the complaints process(es) used by other judicial or quasi-judicial organizations.

Interviews are being conducted by PRA Inc., an independent research firm working in collaboration with the IRB on the evaluation. The interview will be by telephone, either one-on-one or in a group. The interview should take about 60 minutes and will be conducted in the official language of your choice.

Your participation in this interview is voluntary. The information you provide will be handled in accordance with applicable privacy laws.

The information gathered through this interview will be summarized in aggregate form. Interview notes will not be shared outside of PRA unless required by law. With your permission, we will audio-record the interview to ensure the accuracy of the information we gather and report.

We realize that you may not be able to answer all questions; please let us know and we will skip to the next question.

Introduction

1. To begin, please describe your organization (e.g., its mandate and scope).
1. What year did your organization first institute a complaints process about the services of its board/tribunal?
2. How long have you been involved with the complaints process used by your organization?

Design and implementation

2. What documentation does your organization have on its complaints process? Does your organization have a code of conduct for its board/tribunal decision-makers? If yes, how does the code of conduct intersect with or inform the complaints process? [Q2]
3. Please describe the key features of the complaints process. The list below is intended as a guide, and if there are other key features, please also note them in your response. [Q2]
1. What is the process’s scope (i.e., what type of complaints or conduct are within or out of scope)?
2. Who can make a complaint (e.g., claimant, counsel, other employee of organization, anonymous)?
3. What are the steps for filing a complaint (e.g., format, timelines)?
4. What are the reasons for dismissing or otherwise not handling a complaint (e.g., out of scope, other) and who makes that determination?
5. Who investigates complaints (is this done by someone internal or external to your organization)?
6. How are complaints handled (e.g., investigation and related activities, communication with complainant and subject of complaint during the process, timelines for the process)?
7. Who is the ultimate decision-maker?
8. What are the types of decisions available?
9. What follow-up is there for decisions?
10. How is the complaints process held accountable to the organization and its stakeholders, including the public (e.g., annual reporting, other)?
4. What features do you think are critical to the credibility, impartiality, and objectivity of the complaints process from the perspectives of the board/tribunal, complainants, stakeholders, and the public? [Q2]
5. How does the complaints process protect the adjudicative independence of the board/tribunal members while still holding them accountable for misconduct? [Q2]
6. Thinking of the key features of the complaints process discussed above, are there any good practices or lessons learned in terms of its design or delivery? [Q8]
7. Has your organization taken any specific steps related to the following that might be instructive to other organizations? [Q4]
1. Reducing barriers to the use of the complaints process by potential complainants
2. Promoting accessibility and clarity of the complaints process
3. Ensuring it is clear what type of conduct or behaviour is within the scope of the complaints process
4. Ensuring that the complaints process is transparent
5. Ensuring that the complaints process is fair, impartial, and objective
6. Addressing complaints in a timely fashion
8. Does your organization have any service standards or other performance metrics by which it assesses the success of its complaints process? What are they? [Q2 and Q8]
9. How has your organization communicated the complaints process to its stakeholders? Are there any good practices or lessons learned in terms of communication and outreach that you can share? [Q4]
10. Do you have any other thoughts to share about your organization’s process for handling complaints?