The effective functioning of a research ethics committee (REC) can be evaluated using self-assessment tools. The Middle East Research Ethics Training Initiative (MERETI) tool can be used by one member, typically the Chair, to score an REC. The consistency of these scores across several members of an REC has never been evaluated. This study examined whether results would be consistent irrespective of who conducts the assessment. One REC’s effective functioning was assessed by several members ( n = 13). The Chair’s scores were compared with scores of other members in relation to their duration of REC membership, research ethics training, gender, and employer’s institutional affiliation to the REC. The Chair’s overall score was higher than the other members’ scores by 11%. No significant differences in scores were obtained in relation to duration of REC membership ( p = .72), interval since last research ethics training ( p = .94), and gender ( p = .27). The MERETI tool is thus consistent irrespective of who performs the assessment.