ORDA - Online Research Data Archive 
    • Login
    View Item 
    •   ORDA Home
    • Chesterfield Royal Hospital NHS Foundation Trust
    • Trustwide Services
    • Research & Development
    • View Item
    •   ORDA Home
    • Chesterfield Royal Hospital NHS Foundation Trust
    • Trustwide Services
    • Research & Development
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    Should candidate scores be adjusted for interviewer stringency or leniency in the multiple mini-interview?

    Thumbnail
    Abstract
    CONTEXT: There are significant levels of variation in candidate multiple mini-interview (MMI) scores caused by interviewer-related factors. Multi-facet Rasch modelling (MFRM) has the capability to both identify these sources of error and partially adjust for them within a measurement model that may be fairer to the candidate. METHODS: Using facets software, a variance components analysis estimated sources of measurement error that were comparable with those produced by generalisability theory. Fair average scores for the effects of the stringency/leniency of interviewers and question difficulty were calculated and adjusted rankings of candidates were modelled. RESULTS: The decisions of 207 interviewers had an acceptable fit to the MFRM model. For one candidate assessed by one interviewer on one MMI question, 19.1% of the variance reflected candidate ability, 8.9% reflected interviewer stringency/leniency, 5.1% reflected interviewer question-specific stringency/leniency and 2.6% reflected question difficulty. If adjustments were made to candidates' raw scores for interviewer stringency/leniency and question difficulty, 11.5% of candidates would see a significant change in their ranking for selection into the programme. Greater interviewer leniency was associated with the number of candidates interviewed. CONCLUSIONS: Interviewers differ in their degree of stringency/leniency and this appears to be a stable characteristic. The MFRM provides a recommendable way of giving a candidate score which adjusts for the stringency/leniency of whichever interviewers the candidate sees and the difficulty of the questions the candidate is asked.
    URI
    https://orda.derbyhospitals.nhs.uk/handle/123456789/1948
    Collections
    • Research & Development [49]
    Date
    2010
    Author
    Crossley, James, GM
    Show full item record

    copyright © 2017  Derby Teaching Hospitals NHS Foundation Trust
    Contact Us | Send Feedback
    Powered by KnowledgeArc
     

     

    Browse

    All of ORDACommunities & CollectionsBy Issue DateAuthorsTitlesSubjectsThis CollectionBy Issue DateAuthorsTitlesSubjects

    Researcher Profiles

    Researchers

    My Account

    Login

    copyright © 2017  Derby Teaching Hospitals NHS Foundation Trust
    Contact Us | Send Feedback
    Powered by KnowledgeArc