Psychometrics: a users guide

TrainingZone

Psychometrics: a users guide

By TrainingZone

In this article Helen Bouchami of Helen Bouchami Consulting examines the use of psychometrics in organisations. It includes a brief introduction to psychometrics and some good practice guidelines intended for HR managers and other organisational users who have not had access to training in the use of these tests. The second part of the article looks specifically how psychometrics can be used in development.

What do we mean by psychometric tests?

Introduction

Psychometric tests are developed to give as objective an assessment as possible of a given human characteristic. Test developers usually spend considerable time researching and testing the questions and checking out how well the tests work in practice. In particular, tests should achieve a good degree of reliability (measures what it measures consistently) and validity (measures what it claims to measure). For example, you step onto a weighing scale that tells you that you weigh 5 pounds. Since you are an adult, you know that this is not your real weight, yet every time you step on, the scale still reads 5 pounds. The scale would be reliable (measures consistently) but not valid (measures what it is designed to measure)
Another feature of psychometric tests is that care is taken to ensure that they are administered in a standardised way, so that factors such as the way the test is explained do not influence the outcome.

What are the main types of test?

Broadly there are four types of test:

Attainment tests: these measure what you are able to do after a period of instruction eg. driving test
Ability tests: measure a particular ability eg. numerical reasoning
Interest inventories
Personality inventories

The first two measure the best performance an individual is capable of – there are right and wrong answers. Interest and Personality inventories ask about how people typically behave or what they prefer to do in a given situation. There are no right or wrong answers.

Good Practice Guidelines

Valuable as they are in measuring aptitudes, abilities, interests and behaviour preferences, psychometric instruments must be used with care, in their selection, their administration, their interpretation and the safeguarding of data.

Use a qualified practitioner

Psychometric tests are only made available to those trained in their use. HR managers may balk at the cost of bringing in consultants and buying restricted materials. Some find ways around this, but the use of test can be an ethical and even legal minefield for the unwary. Also, test results are only reliable if the test has been administered and interpreted properly. In the UK, practitioners should ideally be registered with the British Psychological Society. BPS Level A, means the practitioner is qualified to use ability and interest inventories. BPS Level B means they have a further qualification in the use of personality inventories.

Use psychometrics for appropriate purposes

Be clear about what information will be gained from the test, and how it will contribute to the process you have in mind – selection, development, team effectiveness etc. Choose tests that are relevant for their intended purpose – ask your practitioner to explain their choice of tests. For example, I have come across Belbin, designed to look at optimal team composition and not really a psychometric test anyway, used for individual personality assessment.

Choose tests that have been thoroughly researched and tested on a broad population. Never use tests as the sole basis for a decision eg. entry to a fast track programme or selection for a job. Avoid tests that may introduce cultural bias or discriminate on the basis of age, sex, race, religion or language.

Test Administration

Gain the informed consent of those completing the tests. Inform them in advance of the purpose and nature of the test, the feedback they can expect to receive and who will have access to their data. Ensure that test administration is conducted as recommended in the Test manual to ensure that the resulting data can be relied upon. Example: It is easy to overlook these details, and to be honest, when I went through my training I thought administration was rather boring and trivial compared to interpretation. That was before I personally experienced poor test administration.

I was under consideration for a role with a major consultancy. I was asked to complete some psychometrics including a numerical reasoning test. The administrator handed me the test remarking ‘ you’ll probably find most of the questions a bit Mickey Mouse. I’m supposed to give you a calculator – it’s there if you want it but you probably won’t need it.’ Needless to say, she was more used to recruiting accountants than management developers. As I looked at the questions and found them none too straightforward I felt my stress levels rising. It brought back all the nightmares I had ever had about not being able to answer the questions in exams! Not the ideal state to complete the test.

Feedback

It may not always be possible to give individual face-to-face feedback to all those completing the tests, but this is something to incorporate wherever possible. Feedback should include an explanation of the test results, and how they relate to other available data about the individual. Test results should be related to the purpose for which the test was completed. Maintain the confidentiality of the test results. Normally they will be regarded as confidential to the individual and will not be disclosed to any other person, including the individual’s manager or the company HR manager, except where this was intended and consented to by the individual at the outset.

Access to test data

There may be specific legal considerations regarding storage of, and access to, test data. For example in the UK the Data Protection Act applies. Generally though, you should:

Inform individuals before completing the test, who will have access to the test data and any reports based upon them
Tell them where, and for how long their data will be stored
Never provide actual scores to anyone who is not trained and qualified in their interpretation
Use test results only for the purpose for which consent has been given

How can psychometrics help development?

Introduction

In some ways using psychometrics in development is simpler than for recruitment. Respondents are usually willing participants and direct beneficiaries of any insights the interpretation brings. However, because development applications make most use of Personality and Interest Inventories there are additional requirements made of the practitioner. All too often questionnaires are administered, scored and ‘interpreted’ without any real purpose in mind, Because everyone’s favourite subject is themselves, it can be an easy win on a course – very interesting to know that you are an ENFJ or that you are cautious about taking on new responsibilities, but ultimately, so what?

The skill of the practitioner is to help the individual to make sense of the scores in the light of their own experience and to relate them to a real and immediate purpose. It follows that the practitioner needs not only to understand how to interpret paper questionnaires, but also to have a good practical understanding of how people develop both personally and within organisations, how careers are shaped and how people relate to one another. The practitioner’s responsibility needs to include helping the individual to develop a plan of action to achieve their particular goal – whether it be flexing their leadership style to meet the needs of the people they manage, improving a key relationship or re-directing their career.
This is why HBC makes use of psychometrics only in the context of other services, especially executive coaching, career counselling and team-effectiveness workshops.
The particular instruments used, alone or in combination, include Myers Briggs Type Indicator, FIRO-B, the STRONG Interest Inventory and CPI. Other questionnaires used, although they are not strictly-speaking psychometrics, include Belbin Team roles inventory, Career Anchors and Honey and Mumford’s Learning Styles questionnaire.

Self-awareness

Because executive coaching relationships require great discretion, it is not always possible to observe the executive in action, nor to interview other people about their perceptions. The coach relies on self-reporting, yet if the individual is unaware of some of their behaviour or of its impact on others, or if they resist acknowledging it, then the coach needs another way in if they are to be of any help. Psychometrics can help identify patterns and provide the coach with hypotheses and areas to probe.
Example: In a recent coaching relationship the executive complained that team members took too little responsibility and referred everything back to him. He felt overworked and stressed and that the people-management aspect of his job was draining his energy. One instrument we used showed him to have very high control needs – typical of this pattern is a tendency to insist that others take responsibility, only to reject their efforts if the results are not exactly what the manager wanted. Without assuming that this was necessarily the case, it provided an avenue to explore. Once he had understood how his own behaviour was contributing to the situation, he had some choices about what to do.

Relationships with others

Once we understand who we are and the ways that make up our distinctive approach to the world, we also understand the ways in which we are different from others. This understanding can help not just in valuing the contribution of other people, but also in developing practical strategies for working with them more effectively.
Example: A manager was talking to me about one of his direct reports. ‘John says I don’t show him enough appreciation, but I’m constantly praising his work. I don’t know what more I can do!’ We knew from John’s Myers Briggs Type, that appreciation was particularly important to him and this also gave us some clues about how he would like that appreciation to be expressed.

I asked the manager to describe some recent feedback. He told me: ‘Just last week I sent him a mail saying that his report was excellent and that the management team had been impressed when I presented it to them.’ The most important aspect of appreciation for someone with John’s profile is that it should be personal. So, with the above example, what John would have preferred was a face-to-face conversation, along the lines:

‘John, I wanted to tell you that I appreciate the hard work you put into the report. It must have taken you a lot of time to research it so thoroughly and I can see you took care over the organisation and presentation. I felt very confident presenting it to the management team. Thank you for your support’

Because the manager did not share this need for frequent personal appreciation, it was difficult for him to understand John. Exploring Myers Briggs helped him to understand that his employee was not simply being perverse and demanding, just different. It enabled him to develop an effective strategy for a more productive relationship.

Team effectiveness

Much of what is written about teamwork promotes the value of diversity. By having a range of skills and perspectives within the team, there is a broader repertoire to draw upon. What is less often acknowledged is that difference is actually quite difficult to deal with. Imagine that you are a high energy, enthusiastic and creative individual. What you most need is a cooler, more analytical type who will check the details and ensure that your exciting ideas have a chance of working. However, what you most enjoy is more likely to be another creative individual with whom you can bounce ideas around. It is all too easy to personalise these differences, and to feel that the individual who always picks holes in your ideas is somehow against you, when in reality, they are just being themselves.
In other cases conflict can arise because two people are too alike. For example, a very dominant individual may get on better with those who readily accept their direction than with someone who has a similarly strong need to control.

Sometimes a dominant team culture can discourage the expression of different perspectives, and individuals with a potentially valuable contribution to make are sidelined.
Psychometrics can help a team to acknowledge the range of contributions needed to be effective. The group can discuss how to make best use of the different talents within the team.

Example: The creative individual described above may be a great asset to a team when they are trying to develop options and ideas. However, their efforts will be less helpful if they constantly throw in yet more ideas as the team is trying to reach a conclusion. With the sort of insights that an exploration using psychometrics can bring, such an individual can learn when to temper their enthusiasm and let someone else take the lead.

Career Choice

Individuals may seek career counselling at almost any stage:

Before starting out, or even to guide the choice of exam subjects or university courses
A couple of jobs into a career – ‘am I on the right track?’ questions arise
After a crisis such as redundancy
When individuals finds themselves in jobs that don't really satisfy them
In middle life when financial commitments (children, mortgage) are less pressing an individual may change career to take more of a risk, or seek greater quality of life

The general premise is that the better the fit between who you are and what you do, the more satisfied and fulfilled you will be, and usually, the better you will succeed (whatever success means to you). Yet people often find it difficult to articulate what they are good at, or what skills learned in one job can be transferred elsewhere. Finding the right career can be a matter of trial and error. The problem with experience, as Vernon Law pointed out is ‘it gives you the test before presenting the lesson.’ Psychometrics don’t prescribe career choice, but they do help clarify what’s important so that the individual can make informed choices about roles that are likely to satisfy them.

Example: To give a personal example, I thoroughly enjoy my consulting work but one thing I miss is the sense of belonging to an organisation. Also, because clients (reasonably enough) expect consultants to already be experts in their chosen field, there is rarely the opportunity to try something completely new. I had tried several ways of meeting these needs, none of which had really worked. I completed the Strong Interest inventory as part of my training in psychometrics. As well as confirming my general career choice, the results revealed a particular interest in medical matters. I was only vaguely aware of this – I hadn’t thought of myself as different from anyone else in this respect.

Too late to go back and re-train as a brain surgeon, maybe, but I am now a Non-Executive Director of the Royal Berkshire Ambulance Service, a part-time role that has added a very satisfying element to my career.

Helen Bouchami is an independent consultant offering services in HRD strategy, Executive Coaching, career counselling, customised learning design and team effectiveness. Helen can be contacted on +44 (0)118 988 5795 or e-mail [email protected] - or you can visit her website at www.bouchami.com.