Currently Updated As of 3:30 PM MT 4/27/2021:
Planned Changes to the Part III & IV Exams - posted 6/14/21
At the end of April, the NBCE sent a proposal about moving the Friday Diagnostic Imaging (DIM) domain from the Part IV exam to the Part III exam to over 2,000 State Board members, Chiropractic College faculty and staff, and current NBCE examinees. To review the proposal, click here: https://www.nbce.org/pad-updates/. We also opened a public comment period to collect feedback about the proposal, and its impact on our stakeholders. We received 155 comments, which represents an 8% response rate. After reviewing the feedback, and consulting with the Part IV Advancement and Development Advisory Board, it was clear that this change is supported by our stakeholders, and is seen as a benefit to the examinees. For more detail about the feedback we received, visit https://www.nbce.org/pad-research/.
The NBCE will be updating our websites, and sending targeted communications about the changes to the Part III and the Part IV exams throughout the month of June 2021. Here is an overview of the changes that will be made starting as of October 2021.
Will examinees have to pass the DIM domain twice?
This was the most common question/concern we received during the public comment period. The answer is no, an examinee is only required to pass either the DIM domain of Part IV or the equivalent DXI domain of Part III. Under no circumstances will they be required to pass both. We will ensure this by monitoring three tracks of examinees.
- Track 1: Examinees who passed the DIM domain of Part IV. If these examinees have not successfully completed the Part III exam, they will only receive a version of the Part III exam without the DXI domain.
- Track 2: Examinees who passed the DXI domain of Part III. These examinees will only be given the version of Part IV without the DIM domain.
- Track 3: Examinees signed up to take Part III in October 2021, and Part IV in November 2021. These examinees will be given the version of Part III without the DXI domain, and the version of Part IV with the DIM domain.
As examinees sign up for Part III or Part IV exams during this transition period, they will receive additional detailed information about which version of the exam they will be taking.
We have created a website you can access at any point to receive the latest news about the Part IV Advancement and Development project. You can access the website here: https://www.nbce.org/pad/. On the contact information page, you can join our mailing list to receive project-related updates and information.
Thank you for taking the time to learn about this change. Please keep an eye out for more detail about these changes, and how they will impact you, throughout the rest of June. If you have immediate questions, or require additional information, please contact us at email@example.com.
Proposed Changes to the NBCE Part III & IV Exams - posted 4/27/21
Proposed Changes to the NBCE Part III and IV Exams
The Diagnostic Imagining (DIM) case-structured domain/subject has been included in the Part IV administration since its inception; however, developments in computerized testing prompted the National Board of Chiropractic Examiners (NBCE) to consider migrating the DIM domain to the Part III exam. Below you will find the details of this proposal. We are asking for feedback from all stakeholders about this change. After reviewing the proposal, visit https://www.nbce.org/part-iv-project-feedback/ to submit your anonymous feedback. All comments will be reviewed, and an additional communication regarding the status of this project will be sent to all stakeholders in June.
The DIM case-structured domain is currently being administered on the first day of the Part IV exam. A computer lab is set-up at every Part IV test site, and examinees are placed in front of a monitor where they review imaging prior to answering questions on a scantron answer sheet. After successfully administering the Parts I, II, III, and PHT exams on a computer for two years, the NBCE began analyzing the benefit of making the DIM case-structured domain a part of the Part III computerized exam. Here is a summary of our findings:
- Moving the DIM case-structured domain to the Part III would allow the NBCE to eliminate one day of Part IV testing for all examinees. This would be especially beneficial to examinees who have to travel to a Part IV test site.
- Doctor of Chiropractic Programs would no longer need to allocate space on their campus for NBCE exams on Fridays.
- The format of the DIM case-structured domain better aligns with the Part III format, which also incorporates cases that include related imaging. This means that the NBCE could experience efficiencies in exam development, production, and delivery by absorbing the DIM domain into the Part III production cycle.
- The eligibility requirements for taking the Part III and Part IV exam are similar. For Part III, an examinee is eligible once they completely pass Part I and are within 9 months of graduation. For the Part IV exam, an examinee is eligible once they completely pass Part I and are within 6 months of graduation. The NBCE does not believe moving the DIM domain will impact the examinee’s preparedness for the exam content, nor delay them in completing their pre-licensure exams prior to graduation.
Because of the importance of DIM, and our satisfaction with the psychometric performance of the domain, it was decided that we would not make any changes to the case-structured DIM domain format when we move the content to the Part III exam. We will, however, begin referring to the case-structured DIM domain from the Part IV as the Diagnostic Imaging Interpretation (DXI) domain in the Part III exam. Changing the name simply helps differentiate between the domains based on which exam it is associated with. The DXI domain of the Part III still consists of 20 images, with two questions per image, like it did as the DIM domain of Part IV. Examinees will need 45 minutes of testing time to accommodate this additional content.
The NBCE evaluated the possibility of reducing the content of the current Part III exam to accommodate the additional DXI content, without increasing the overall length of the exam.
Part III Exam Reduction Process
Igor Himelfarb, Ph.D., Director of the NBCE’s Psychometrics and Research Department, performed a similar exam reduction analysis on the Part I and Part II exams in 2018. In the Part I and Part II exam, we were able to reduce the number of items per domain from 90 to 50, and the total testing time was reduced from nearly 11 hours to 5 ½. The analysis showed that reducing the number of items per domain in Parts I and II did not reduce construct validity, the reliability of the scores, or the examinee classification into pass or fail conditions.
We used the same methodologies to evaluate the possible reduction of the current Part III. The current Part III exam consists of two sections, with a total of 110 traditional multiple-choice questions and 10 case vignettes divided evenly between the two sections. The total testing time for the current Part III is 4.5 hours.
The NBCE evaluated the following changes:
- Reduce the traditional multiple-choice questions from 110 to 80. More specifically, the new Part III with DXI will have 70 operational traditional multiple-choice questions, and 10 field test questions. The current Part III has 90 operational and 20 field test questions.
This change will reduce testing time by approximately 25 minutes.
- Increase the number of case vignettes from 10 to 20, while:
- Reducing the number of extended multiple-choice questions associated with each vignette from 3 to 1. Ultimately, this reduces the number of case-related questions from 30 to 20.
- Reducing the number of response options per case-related question from 8 to 5.
- Reducing the number of correct responses from 3 to 2.
This change will reduce testing time by approximately 20 minutes.
- Add 20 images with two multiple-choice questions per image (the DXI domain, formerly the DIM domain from Part IV).
This change will increase testing time by 45 minutes.
Summary of Proposed Changes to Part III
|Current Part III||Proposed Part III with DXI|
|Number of items||
· 110 traditional multiple-choice questions.
· 10 case vignettes with three extended multiple-choice questions.
· 80 traditional multiple-choice questions.
· 20 case vignettes with one extended multiple-choice question.
· 20 images with two multiple-choice questions per image.
|Testing Time (includes instructions and breaks)||4.5 hours||4.5 hours|
The Psychometrics and Research Department, under the direction of Dr. Igor Himelfarb, was tasked with evaluating the Part III historical data to determine if these changes were feasible. Decision consistency studies were performed on both the traditional multiple-choice questions and the case vignettes using two different approaches: classical test theory (CTT) and item response theory (IRT). An approach based on Bayesian methodology was also used to analyze reductions to the traditional multiple-choice questions. Data from four Part III administrations was used for the decision consistency studies. All results from the decision consistency studies supported the proposed reduction of the number of items in Part III.
- The accuracy of classification into pass/fail status between the 90 and the 70 operational traditional multiple-choice questions was consistently above 90%.
- The accuracy of classification into pass/fail status between 10 case vignettes with three extended multiple-choice questions and 20 case vignettes with one extended multiple-choice question was consistently above 80%, which was expected due to changes in the exam format, and is acceptable classification reliability.
The complete analysis of the Part III exam reduction has been documented by the Psychometrics and Research department. Based on the results of this multi-faceted analysis, and our previous success in reducing the Part I and Part II exam, we are confident that the proposed changes will only add to the reliability and integrity of the Part III exam.
To review the NBCE’s publications relating to exam reductions, or other exam-related topics, please visit https://www.nbce.org/nbce-publications/
In order to maximize the anticipated efficiencies associated with moving the DIM case- structured domain to the Part III, we have to carefully plan the rollout of the new exams to align with our internal publishing and administration schedules. For this reason, the NBCE is proposing to deliver the first Part III exam with the DXI domain during the October 16-17, 2021 administration. We would then deliver the first Part IV without DIM during the May 2022 administration.
While planning this transition, we realize the importance of ensuring that every examinee has successfully demonstrated competency in Diagnostic Imaging Interpretation only once. This means that we will continue to offer a version of the Part III without the DXI domain for examinees who have previously passed DIM during a Part IV administration. Conversely, we will also continue to offer the DIM exam during the Part IV administration for examinees who passed Part III when the DXI domain was not included. Determining which version of the Part III or the Part IV exam an examinee is eligible to take will be done automatically by the NBCE’s application system, ensuring that every examinee receives the correct version of the exam based on their exam history.
Isn’t there Diagnostic Imaging (DIM) content in multiple NBCE exams (Part II, Part III, and Part IV)? Will all DIM content now be in the Part III exam?
Diagnostic Imagining (DIM) is a domain in the Part II exam, and it will not be impacted by this proposed change. The DXI domain focuses on radiological interpretation, whereas the Part II DIM tests on radiology methods, safety, indications, physics of radiology, etc., and its content does not typically overlap with the DXI domain. This proposal is not meant to consolidate all DIM content to the Part III exam. It is specifically referencing the migration of the DIM case-structured domain of the Part IV exam to the Part III exam. In fact, you will likely still see DIM content in the Part IV exam after this change. It will simply be incorporated into the stations.
How will this change impact scoring?
The scoring processes for the Part III and the Part IV will remain unchanged. The passing score for the Part III and Part IV remains 375. We will monitor the difficulty of the exams and ensure equivalence through statistical equating procedures.
How will this change impact score reporting to Doctor of Chiropractic Programs?
After the October 2021 Part III exam, you will start to see the DXI domain, and associated sub-domains, in the Part III score reports and college reports. You will still see the DIM domain in the Part IV score and college reports for November 2021 and beyond, until all examinees have cycled through the system and we no longer have to administer the DIM exam with the Part IV.
As part of our desire to engage stakeholders in this project, we have formed an Advisory Board to consult the NBCE on these proposed changes. Five members of the Advisory Board represent the Doctor of Chiropractic Programs. With their assistance, we will consider additional changes to score reporting to assist Programs in navigating this change. If you have a specific suggestion or concern regarding score reporting, please submit your feedback here: https://www.nbce.org/part-iv-project-feedback/
How will this change impact transcripts?
There will be no visible change to examinee transcripts. As long as an examinee receives a passing score in Part III and Part IV, State Boards can be assured that they have demonstrated at least the minimum competency required to be considered for licensure. As all states require a passing score in Part III and Part IV for licensure, we do not see the need to add this additional information to transcripts at this time.
Will this change impact exam fees?
No. There will not be a change to exam fees as a result of this change. The NBCE’s Board of Directors evaluate exam fees at the end of every year, and can potentially change exam fees if they determine it is necessary based on other financial determinants.
Thank you for taking the time to review the NBCE’s proposal to move the Part IV case-structured DIM domain to the Part III exam. We would like to hear from you prior to planning our next steps. Please visit https://www.nbce.org/part-iv-project-feedback/ to submit your questions, concerns, or support for this change. We will be accepting feedback until May 24. After that, we will close the public comment period and evaluate all submissions with the Part IV Advancement and Development project Advisory Board. We plan to send an update about this project to all stakeholders in June.
For more information about the initiatives and goals of the Part IV Advancement and Development project, and to see the Advisory Board members, please visit https://www.nbce.org/pad.
Introducing the Part IV Advancement & Development Project - posted 3/1/21
Introducing the Part IV Advancement and Development Project
In 2019, after a year of successful computer-based exam administrations, the NBCE decided to turn its focus to evaluating the current state of the Part IV exam. We performed a preliminary analysis of our current Part IV in comparison to Objective Structured Clinical Examinations (OSCE) given to other healthcare professionals. The NBCE’s Board of Directors saw a need to further investigate best practices in the field of OSCE testing and formed a project team with the following objectives:
- Perform an evaluation of all OSCE exam trends by researching a benchmark group.
- Engage stakeholders to gather feedback about the current state of the NBCE’s Part IV exam and gauge interest in improvements.
- Perform a psychometric analysis of the current Part IV exam to identify areas of strength and areas that could be improved.
In 2020, the NBCE connected with other testing organizations. We extensively researched other OSCEs, and created preliminary budgets to evaluate the cost of potential changes. We sent a survey to examinees to get their opinion about the current state of the Part IV exam (a summary of the results can be found here), and held a focus group with state board members about potential changes to the exam. Ultimately, we developed a list of “trends”, which are potential improvements to the NBCE’s Part IV exam that were discovered through our benchmark analysis. When evaluating potential improvements to the NBCE’s Part IV exam, we used the information gathered to evaluate the trends based on the following criteria:
- How well is the trend represented in the benchmark group (considered a best practice)?
- Will it impact the validity or reliability of the Part IV exam?
- Is it financially feasible to implement the trend?
- Will it impact the standardization of the exam?
- How will it impact the NBCE’s stakeholders?
- How feasible is implementation?
Through the evaluation of these criteria, we were able to identify eight potential changes that could improve the NBCE’s Part IV exam and should be evaluated for implementation.
- Comprehensive Examinee-Guided Station Format: Combining the single-point stations (Case history, Ortho/Neuro, Physical exam, and Technique) into one comprehensive station that better mimics a real-world patient encounter.
- Adding a Documentation Component: Examinees would be required to create a patient note (after the patient-encounter station) documenting their findings, assessment, and plan for that patient.
- More External Contribution to Item Development: Develop a method for collecting exam content from a wider variety of people within the chiropractic community (academics, practicing DCs, and state board members).
- Adding Video Recording to Stations: Requiring patient-encounter rooms to be set up with video recording equipment that can accurately capture the examinee’s performance for remote review.
- Moving DIM to Written Exams: Since DIM is a Computer-Based Test (CBT), we could incorporate the content into the Part III exam, eliminating the Friday testing day.
- Eliminating Cross Country Sequestration: Removing the sequestration requirement, which currently requires us to hold examinees until the exam is completed across the country.
- Patient Grading Component: Standardized patients rate the examinee on the “humanistic” components of the station.
- Paperless Exam: Adding computerized testing equipment for the examinees and evaluators to eliminate scantron answer sheets.
As we kickoff this project, and enter the Discovery and Design phase, we plan on continuing to research these best practices while gathering stakeholder feedback about the impact of these changes. We plan on forming an Advisory Board, made up of members from Chiropractic education and the regulatory agencies who use NBCE’s exam to assist them in making licensing decisions. In order to stay informed of important project updates, please join our mailing list (link found on the project homepage). If you have any questions or comments about the Part IV Advancement and Development project, please reach out to us at firstname.lastname@example.org.
ENSURING PROFESSIONAL COMPETENCY