Test Results & Scoring

All CELPIP Reading and Listening questions are in the multiple choice or similar format. All the answers to CELPIP Reading and Listening questions are scored dichotomously: a response is either correct or incorrect. Questions that are left blank are scored as incorrect. All the scoring is done by computer.

The writing and speaking components of the CELPIP-General Test are scored by qualified raters trained to apply consistent criteria to assess test taker performances based on standard scoring rubrics. Raters receive ongoing training and regular monitoring. Paragon uses rater agreement statistics to determine the quality of ratings; for a given test taker, a rater agrees with the other raters of this test taker if their rating is sufficiently close to that of the other raters (i.e., consensus).

Raters receive ongoing training to ensure that the scoring criteria are consistently and systematically applied by all raters, and to minimize potential bias introduced by human judgment.

1. Initial Rater Training

All raters attend an initial training program to guide them through Paragon’s rating approach. After completing a training manual, exercises, and rating samples, trainees engage in a certification process during which they rate 3-6 certification sets. In order to certify, trainees must achieve a minimum 80% agreement with the official score assigned to each performance in at least three consecutive sets. Only certified raters can start operational rating.

2. Operational Rater Training

To maintain a shared perspective on relevant rating principles and criteria, all operational raters receive ongoing in-service training and monitoring, including:

• Weekly feedback on their agreement with other raters
• Weekly sample performances rated by expert raters
• Biweekly in-depth training materials in the form of communications hosted online, including a range of sample performances rated and justified by expert raters
• Detailed notes from rater seminars, in which challenging responses are discussed and rated by expert raters.

3. Rater Monitoring

Rater performance analysis is conducted monthly to monitor the reliability of the rater pool and to identify raters who have unsatisfactory rater agreement. Underperforming raters receive personalized feedback on rating samples that demonstrate a significant discrepancy between their ratings and benchmark ratings Additional samples are provided upon request by a rater. Once identified as underperforming, a rater must demonstrate improvement within 8 weeks. If an underperforming rater does not demonstrate an improvement to meet Paragon’s rating standards within that period, Paragon may terminate their rating contract.

All tests are randomly assigned to raters by an online system. Test taker anonymity is maintained at all times. Each test taker’s performance, i.e. a test taker’s responses to all tasks in the component, is assessed by multiple raters. Each CELPIP speaking performance is rated by a minimum of three speaking raters, and each CELPIP writing performance is rated by a minimum of four writing raters. Raters work independently of one another, and have no knowledge of the ratings assigned by other raters.

1. Rating criteria

The rating dimensions that have been developed for the writing and speaking component are listed above on this page in the Performance Standards section:

Speaking: Content/Coherence, Vocabulary, Listenability, and Task Fulfillment

Writing: Content/Coherence, Vocabulary, Readability, and Task Fulfillment

Each dimension is divided into five performance levels. Performance descriptors are provided for each level in each dimension. Raters assign a level in each dimension by identifying tangible evidence in the test taker’s performance that matches the descriptors in the rating scale.

2. Benchmarking

When the ratings of a test taker’s performance are complete, they are inspected for agreement. If the ratings are in disagreement, a benchmark rater is automatically assigned to assess the performance. All benchmark raters are experienced raters who have demonstrated consistent accuracy and reliability in rating. Benchmark raters have no knowledge of the initial ratings.

3. How is the final score determined?

The Speaking and Writing component scores are derived from the dimensional ratings assigned by the raters. These scores are then transformed into a CELPIP level. The transformation rules have been established by English language experts who participated in a standard setting exercise. Standard setting is an extensive, research-based process. Language experts work with testing professionals to identify what language learners need to be able to do at each performance level, such as CLB 8. The experts then analyze the test in detail and determine what level of performance a test taker needs to demonstrate for each CELPIP level. This process has established a defensible link between each Speaking and Writing component score and its corresponding CELPIP level.

New items are constantly being written. Before they can be used as scored items, they are pre-tested to ensure that they are equivalent in quality to existing items. Paragon includes some new items in every test. These items look the same as the scored items but they are not used to calculate your score. Paragon does not tell the test taker which questions will be unscored because it is important that test takers try their best for every item. This ensures that the data collected on the new items can be used to evaluate their quality. Only questions that have performed well will be used as scored items in the future.

While each test form contains different questions, each test form is constructed following explicit guidelines regarding content and difficulty. Paragon’s pre-testing and form creation procedures ensure that the different forms have approximately the same difficulty. However, the items are not identical from administration to administration, and this means that it is possible for there to be small differences in difficulty. It would be unfair to test takers if final test scores did not correct for these small differences. The process of score equating removes even this slight variation.

Score equating is the process of correcting final scores for slight variations in difficulty between test forms. For example, if one test taker correctly answers 30 questions on a relatively easy form and another test taker answers 30 questions on a more difficult form, equating will correct for the disparity in form difficulty. It is of utmost importance that the score reported for either of the test forms is comparable. We need to make sure that the final score reflects your underlying language proficiency and is not dependent on the difficulty of the questions you or someone else received.

The goal of any test is to provide a fair and accurate assessment of each test taker, regardless of the specific questions presented during the test. Though CELPIP tests are assembled following guidelines for content and difficulty, it is still possible for test forms to vary slightly in difficulty. Since a raw score is merely the sum of the questions that a test taker has correctly responded to, it cannot account for these slight variations in difficulty. Consequently, a raw score of 30 will not have the same meaning across different forms of a test. This means that different test takers’ raw scores could be hard to interpret and compare.

In order to account for differences between test forms, Paragon transforms test takers’ raw scores into a scaled score. Scaled scores adjust raw scores in a consistent way so that test takers’ scores on different test forms can be compared.

After the Reading and Listening scaled scores have been determined, they are transformed into a CELPIP level. The transformation rules were established by English language experts who participated in a standard setting exercise. Standard setting is an extensive, research-based process. Language experts work with testing professionals to identify what language learners need to be able to do at each performance level, such as CLB 8. The experts then analyze the test in detail and determine what level of performance a test taker needs to demonstrate for each CELPIP level. This process has established a defensible link between each Reading and Listening scaled score and its corresponding CELPIP level.

There are many ways to measure reliability of a test. One good measure of reliability is Cronbach’s alpha, which specifically measures the internal consistency of a test form. The result of this statistical measure can range from -1 to +1 where -1 indicates a complete lack of internal consistency and +1 indicates perfect consistency. A result of 0.80 or higher is considered to be excellent. For both the CELPIP Reading and Listening components, the test forms have an average Cronbach’s alpha of 0.88. This indicates that CELPIP Reading and Listening test forms demonstrate excellent internal consistency.

There are two ways to request Express Rating:

1. During your CELPIP Test Registration, or
2. If you have already registered for a CELPIP Test, through your CELPIP Account.

Please note:

• Express Rating requests and payments must be made at least 24 hours before your scheduled test date.
• There is limited availability of the Express Rating option.
• Express Rating is available for all test sittings except those held on Sunday after 11:00 a.m. (Local Time)

Cancellation Policy: You may request a cancellation of your Express Rating. If your cancellation request is received at least nine (9) calendar days before your scheduled test date, you will be eligible for a full refund of the Express Rating fee. No cancellations or refunds will be granted for requests received less than 9 calendar days before the scheduled test date.

Cancellation by Paragon: If technical problems occur during Express Rating that delay the availability of your test scores online, you may contact us to receive a full refund of your Express Rating fee

Transfer Policy: If you purchase Express Rating and decide to transfer your test to a later date, Express Rating may not be available for the test date you are transferring into. If Express Rating is not available, you can request a full refund of your Express Rating fee or you can request a transfer to a different test date for which Express Rating is available.