Proposed Changes to the NBCE Part III and IV Exams

The Diagnostic Imagining (DIM) case-structured domain/subject has been included in the Part IV administration since its inception; however, developments in computerized testing prompted the National Board of Chiropractic Examiners (NBCE) to consider migrating the DIM domain to the Part III exam. Below you will find the details of this proposal. We are asking for feedback from all stakeholders about this change. After reviewing the proposal, visit https://www.nbce.org/part-iv-project-feedback/ to submit your anonymous feedback. All comments will be reviewed, and an additional communication regarding the status of this project will be sent to all stakeholders in June.

Background

The DIM case-structured domain is currently being administered on the first day of the Part IV exam. A computer lab is set-up at every Part IV test site, and examinees are placed in front of a monitor where they review imaging prior to answering questions on a scantron answer sheet. After successfully administering the Parts I, II, III, and PHT exams on a computer for two years, the NBCE began analyzing the benefit of making the DIM case-structured domain a part of the Part III computerized exam. Here is a summary of our findings:

• Moving the DIM case-structured domain to the Part III would allow the NBCE to eliminate one day of Part IV testing for all examinees. This would be especially beneficial to examinees who have to travel to a Part IV test site.
• Doctor of Chiropractic Programs would no longer need to allocate space on their campus for NBCE exams on Fridays.
• The format of the DIM case-structured domain better aligns with the Part III format, which also incorporates cases that include related imaging. This means that the NBCE could experience efficiencies in exam development, production, and delivery by absorbing the DIM domain into the Part III production cycle.
• The eligibility requirements for taking the Part III and Part IV exam are similar. For Part III, an examinee is eligible once they completely pass Part I and are within 9 months of graduation. For the Part IV exam, an examinee is eligible once they completely pass Part I and are within 6 months of graduation. The NBCE does not believe moving the DIM domain will impact the examinee’s preparedness for the exam content, nor delay them in completing their pre-licensure exams prior to graduation.

Because of the importance of DIM, and our satisfaction with the psychometric performance of the domain, it was decided that we would not make any changes to the case-structured DIM domain format when we move the content to the Part III exam. We will, however, begin referring to the case-structured DIM domain from the Part IV as the Diagnostic Imaging Interpretation (DXI) domain in the Part III exam. Changing the name simply helps differentiate between the domains based on which exam it is associated with.  The DXI domain of the Part III still consists of 20 images, with two questions per image, like it did as the DIM domain of Part IV. Examinees will need 45 minutes of testing time to accommodate this additional content.

The NBCE evaluated the possibility of reducing the content of the current Part III exam to accommodate the additional DXI content, without increasing the overall length of the exam.

Part III Exam Reduction Process

Igor Himelfarb, Ph.D., Director of the NBCE’s Psychometrics and Research Department, performed a similar exam reduction analysis on the Part I and Part II exams in 2018. In the Part I and Part II exam, we were able to reduce the number of items per domain from 90 to 50, and the total testing time was reduced from nearly 11 hours to 5 ½. The analysis showed that reducing the number of items per domain in Parts I and II did not reduce construct validity, the reliability of the scores, or the examinee classification into pass or fail conditions.

We used the same methodologies to evaluate the possible reduction of the current Part III. The current Part III exam consists of two sections, with a total of 110 traditional multiple-choice questions and 10 case vignettes divided evenly between the two sections. The total testing time for the current Part III is 4.5 hours.

The NBCE evaluated the following changes:

• Reduce the traditional multiple-choice questions from 110 to 80. More specifically, the new Part III with DXI will have 70 operational traditional multiple-choice questions, and 10 field test questions. The current Part III has 90 operational and 20 field test questions.
  This change will reduce testing time by approximately 25 minutes.
• Increase the number of case vignettes from 10 to 20, while:

• Reducing the number of extended multiple-choice questions associated with each vignette from 3 to 1. Ultimately, this reduces the number of case-related questions from 30 to 20.

• Reducing the number of response options per case-related question from 8 to 5.
• Reducing the number of correct responses from 3 to 2.

This change will reduce testing time by approximately 20 minutes.

• Add 20 images with two multiple-choice questions per image (the DXI domain, formerly the DIM domain from Part IV).
  This change will increase testing time by 45 minutes.

 

Summary of Proposed Changes to Part III

	Current Part III	Proposed Part III with DXI
Number of items	· 110 traditional multiple-choice questions. · 10 case vignettes with three extended multiple-choice questions.	· 80 traditional multiple-choice questions. · 20 case vignettes with one extended multiple-choice question. · 20 images with two multiple-choice questions per image.
Testing Time (includes instructions and breaks)	4.5 hours	4.5 hours

 

The Psychometrics and Research Department, under the direction of Dr. Igor Himelfarb, was tasked with evaluating the Part III historical data to determine if these changes were feasible. Decision consistency studies were performed on both the traditional multiple-choice questions and the case vignettes using two different approaches: classical test theory (CTT) and item response theory (IRT). An approach based on Bayesian methodology was also used to analyze reductions to the traditional multiple-choice questions. Data from four Part III administrations was used for the decision consistency studies. All results from the decision consistency studies supported the proposed reduction of the number of items in Part III.

• The accuracy of classification into pass/fail status between the 90 and the 70 operational traditional multiple-choice questions was consistently above 90%.
• The accuracy of classification into pass/fail status between 10 case vignettes with three extended multiple-choice questions and 20 case vignettes with one extended multiple-choice question was consistently above 80%, which was expected due to changes in the exam format, and is acceptable classification reliability.

The complete analysis of the Part III exam reduction has been documented by the Psychometrics and Research department. Based on the results of this multi-faceted analysis, and our previous success in reducing the Part I and Part II exam, we are confident that the proposed changes will only add to the reliability and integrity of the Part III exam.

To review the NBCE’s publications relating to exam reductions, or other exam-related topics, please visit https://www.nbce.org/nbce-publications/

 

Proposed Timeline

In order to maximize the anticipated efficiencies associated with moving the DIM case- structured domain to the Part III, we have to carefully plan the rollout of the new exams to align with our internal publishing and administration schedules. For this reason, the NBCE is proposing to deliver the first Part III exam with the DXI domain during the October 16-17, 2021 administration. We would then deliver the first Part IV without DIM during the May 2022 administration.

While planning this transition, we realize the importance of ensuring that every examinee has successfully demonstrated competency in Diagnostic Imaging Interpretation only once. This means that we will continue to offer a version of the Part III without the DXI domain for examinees who have previously passed DIM during a Part IV administration. Conversely, we will also continue to offer the DIM exam during the Part IV administration for examinees who passed Part III when the DXI domain was not included. Determining which version of the Part III or the Part IV exam an examinee is eligible to take will be done automatically by the NBCE’s application system, ensuring that every examinee receives the correct version of the exam based on their exam history.

Questions

Isn’t there Diagnostic Imaging (DIM) content in multiple NBCE exams (Part II, Part III, and Part IV)? Will all DIM content now be in the Part III exam?

Diagnostic Imagining (DIM) is a domain in the Part II exam, and it will not be impacted by this proposed change. The DXI domain focuses on radiological interpretation, whereas the Part II DIM tests on radiology methods, safety, indications, physics of radiology, etc., and its content does not typically overlap with the DXI domain. This proposal is not meant to consolidate all DIM content to the Part III exam. It is specifically referencing the migration of the DIM case-structured domain of the Part IV exam to the Part III exam. In fact, you will likely still see DIM content in the Part IV exam after this change. It will simply be incorporated into the stations.

How will this change impact scoring?

The scoring processes for the Part III and the Part IV will remain unchanged. The passing score for the Part III and Part IV remains 375. We will monitor the difficulty of the exams and ensure equivalence through statistical equating procedures.

How will this change impact score reporting to Doctor of Chiropractic Programs?

After the October 2021 Part III exam, you will start to see the DXI domain, and associated sub-domains, in the Part III score reports and college reports. You will still see the DIM domain in the Part IV score and college reports for November 2021 and beyond, until all examinees have cycled through the system and we no longer have to administer the DIM exam with the Part IV.

As part of our desire to engage stakeholders in this project, we have formed an Advisory Board to consult the NBCE on these proposed changes. Five members of the Advisory Board represent the Doctor of Chiropractic Programs. With their assistance, we will consider additional changes to score reporting to assist Programs in navigating this change. If you have a specific suggestion or concern regarding score reporting, please submit your feedback here: https://www.nbce.org/part-iv-project-feedback/

How will this change impact transcripts?

There will be no visible change to examinee transcripts. As long as an examinee receives a passing score in Part III and Part IV, State Boards can be assured that they have demonstrated at least the minimum competency required to be considered for licensure. As all states require a passing score in Part III and Part IV for licensure, we do not see the need to add this additional information to transcripts at this time.

Will this change impact exam fees?

No. There will not be a change to exam fees as a result of this change. The NBCE’s Board of Directors evaluate exam fees at the end of every year, and can potentially change exam fees if they determine it is necessary based on other financial determinants.

Provide Feedback

Thank you for taking the time to review the NBCE’s proposal to move the Part IV case-structured DIM domain to the Part III exam. We would like to hear from you prior to planning our next steps. Please visit https://www.nbce.org/part-iv-project-feedback/ to submit your questions, concerns, or support for this change. We will be accepting feedback until May 24. After that, we will close the public comment period and evaluate all submissions with the Part IV Advancement and Development project Advisory Board. We plan to send an update about this project to all stakeholders in June.

For more information about the initiatives and goals of the Part IV Advancement and Development project, and to see the Advisory Board members, please visit https://www.nbce.org/pad.