2 Responses

1. Step by Step Approach
   I know from my own research in this area, that the testing theorists are often quick to advise against using mutliple choice testing for anything other than knowledge recall, but I believe it can be used to test understanding, which I define as being able to use the knowledge to achieve an end result through the application of discrete pieces of that knowledge and capability, which I define as the ability to solve a problem, using a broader range of acquired knowledge.

   So for instance, at a knowledge recall level the question could go something like this:

   (Q) What is the correct sequence of steps to calculate the total of a range of cells?

   At the understanding level, the question would just state:

   (Q) What is the average value of the numbers in range A1..A99?

   Here we require the person to use the steps they’ve learned to calculate the results.

   At the capability level, you could ask?

   (Q) Given this spreadsheet of stock level figures by three product lines at ten warehouses, calculate the cost of insuring the stock in these warehouses at a rate of 15p per unit of item A, 20p per unit of item B and 25p per unit of item C.

   Here we need the person to think about what needs to be done, e.g. add the columns for each line, then multiply the totals by the respective insurance rate, then actually perform the necessary steps.

   This is the sort of example that helps me to differentiate between what it is I’m testing.

2. Processes, not Facts
   Angela

   I have to agree with the comment that multiple choice tests are better at testing “fact recall” rather than understanding.

   It also isn’t clear what level of IT competence you want to test.

   HOWEVER, since you obviously have your reasons for wanting to use an MCT in this way, I’d suggest that you ask process questions rather than simple fact questions. So, AVOID:

   Q. The thing that looks like a TV but can’t get ‘Racing on 4’ is:

   1. A keyboard
   2. A two-way mirror
   3. A monitor

   and USE questions like:

   Q. In order to deal with situation xyz I would:

   1. Brief description of Process 1
   2. Brief description of Process 2
   3. Brief description of Process 3

   where the three (four, five?) descriptions are similar but vary on some key point.

   Also make sure that the answers don’t all vary on the same point.
   So description 1 and description 2 might be the same except at step 3; description 1 and description 3 would be the same except at step 2. And so on.