There are many reasons why awarding bodies and exam boards are making the switch to e-assessment. If I were a salesman for one of the many suppliers in the industry here are 5 arguments I would be putting to my prospective clients for consideration:
- Candidate’s Expectations.
Any organisation, whether it is a commercial business or a not-for-profit, should be focussed on providing the best service they can to their candidates which is why I believe this has to be the primary reason for adopting e-assessment. Candidates learn differently nowadays with the majority of people using laptops, tablets and smart phones to consume education so it is expected that the method of assessment is also delivered electronically. To deliver the training in new and innovative ways and to then ask somebody to sit a three hour written exam using a pen and paper surely is not an effective way of truly measuring somebody’s comprehension. This is more so the case when the candidate goes on to work in their field using the similar technology to that on which they learnt, meaning only the assessment part of the process was undertaken in this manner. Also with the fast paced lifestyle that most of us live candidates are expecting to receive instant results, admittedly this is not always the case even with e-assessment as essay based examinations usually need human marking, but for most forms of assessment candidates are able to sit the exam and have the results handed to them as they leave.
2. Data, Data, Data. The world is obsessed with data, whether it is supermarkets monitoring how many tins of beans you buy so they can remind you next time you don’t buy any, or whether it is gathering telemetry from a racing car to maximise performance, data is being collected and used on a massive scale. The world of assessment is no different. Item data has been collected for many years, it certainly isn’t something new as a result of computer based testing, however the range of data now available is greater than ever. It does depend on the systems you use of course but as well as gathering statistics to inform ‘Item Difficulty’, ‘Item Discrimination’, ‘KR20’ etc, which I repeat was available from paper and pen delivered assessment (although not automated), we are able to monitor other things - i.e. how long a candidate spent on each Item, whether a candidate selected a particular Response before going back and changing it later on (perhaps indicating an Item later in the exam may be giving the answer), feedback gathered from the candidate on each individual Item can highlight potential issues, the list goes on. With all of this data we can start to get a real picture of how the Items are performing, and how the exam is performing as a whole, to ensure the assessment is fair, valid and reliable, a mantra uttered by psyshometricians the world over. It also provides the exam owner with the opportunity to more easily drill down into the data to compare candidate’s performance across different gender, sector, geography, age, religion, culture giving them the information they need to reach out to new markets or to improve training materials, for example.
3. Item Types. When you talk to somebody about e-assessment, 9 times out of 10 the assumption is that the Item is Multiple Choice - the ‘Stem’ (the question being asked) and a number of ‘Distractors’ (the possible answers). In the early days this assumption would probably have been correct but the evolution of technology has meant that a varying range of innovative and engaging Item types can now be delivered, even on a modest spec machine. ‘Hot Spot’ Items allow the candidate to select an area of an image/graph/diagram/text - ‘X’ marks the spot so-to-speak, video and audio can be delivered to accompany the question giving greater context to the subject, fully working balance sheets can be completed and marked for financial qualifications, flash based simulations allow you to look around a mouth before you identify and extract the rotten tooth. This list is by no means exhaustive and the possibilities are limited only by imagination and, without doubt, budget!
4. Security. What is the most valuable asset an awarding body owns? The Item Bank. Sometimes costing hundreds of £1000s to develop it is crucial that the security of the Item Bank is maintained. Memorising Items to either post in forums or sell online (‘Item Farming’) is big business - some unscrupulous website owners sell the answers to known exam Forms for anything ranging from £50 to £2000 and more. This problem is unlikely to ever be eradicated entirely but e-assessment does give the exam owner some weapons to fight back with. Randomised Items, including randomising the ‘Distractors’, makes it harder for people to memorise the Items in a particular order, combined with ‘Linear On The Fly’ (LOFT) or ‘Adaptive’ (CAT) exams, where Items are pulled from the Item Bank as they are delivered, means that somebody wanting to gain an advantage through this method would need to memorise an entire Item Bank, not just one set of Items. Fraud indicators provided by the increased data feedback also provides both a deterrent and a means to detect people up to no good. ‘Proxy Testers’ (people taking exams on behalf of others for money) have been caught, and in some cases prosecuted and imprisoned, because the test delivery software detected suspicious patterns such as irregular high pass rates in certain centres or regions, quick response times, consistent correct and incorrect answers. These detection indicators used to protect an Item Bank are fairly high tech solutions and not every provider is able to cater for this, however, one method of protection every provider can offer when delivering e-assessment is encryption - simple but effective.
5. Efficiencies. One thing suppliers may tell you is that e-assessment will cost you less than delivering paper and pen exams. In my experience, particularly in a high stakes environment where the exam is being sat in a Test Centre with invigilators, this is unlikely to ring true, although I do see prices starting to come down as more competitors enter the market. However, e-assessment does bring many efficiencies which might just be enough to warrant any extra cost. No more shipping boxes of exam papers to far flung countries. As well as the saving on shipping it also removes a huge risk of papers getting lost or stolen which would cost a large sum of money in re-developing an item bank. Also if they were lost or stolen after the exam has taken place a lot of time, money and probably compensation arranging another exam session would be required. E-assessment also removes a lot of the human marking element from the assessment process. As I mentioned earlier there will always need to be an element of human marking for particular forms of assessment, even Multiple Choice where moderation is required, but generally speaking the computer can mark the exams for you meaning less work for the exam owner and a happier candidate (depending on the result of course!). Reporting, including operational and Item statistics/measurement, has been made quicker and easier, and in some cases completely automated, through the collection of data from the test delivery software. Some people might argue that the machines are replacing humans but I haven’t seen that yet - they are merely allowing us to be more efficient, freeing up some time so that we can do our job with a little less pressure which has to be a great outcome for the individual, their employer and the candidates they are serving.
