By Dr Graham Tyler, Registered Business Psychologist,
Chartered Scientist and Executive Director at PsyAsia International: www.psyasia.com
Absolute objectivity
Psychometric tests are tools designed to assess psychological attributes such as aptitude and personality in an objective and quantitative manner. There has been over sixty years of research in this field and it is well documented that—psychometric tests, when used properly by trained individuals, add significantly to the ability of HR to select the best candidate for the job.
Organisations which have adopted more ‘scientific’ human resource processes usually incorporate psychometrics into their approach. Psychometrics is probably the only aspect of the selection process that allows for absolutely objective assessment of candidates. Moreover, they allow HR to assess large numbers of candidates in very short periods of time, which would be immaterial if the test did not do its job properly. However, research has shown that the ability of tests to predict successful hires often exceeds that of traditional interviews. Generally, aptitude tests have been shown to be more predictive of success than personality tests, although there are a couple of notable personality tools that are also on par with aptitude tests.
There are numerous tests currently on the market and many distributors in Hong Kong—so how can HR be certain they are choosing the right test and choosing to work with the right distributor? Two videos are avialable giving advice to those in HR on the right and wrong questions to ask psychometric test distributors and can be viewed via the follwoing link:
The right and wrong questions to ask psychometric test distributors
Professional understanding
Given that psychometric tests measure psychological characteristics, HR are advised to try and work with registered business psychologists, who will have the benefit of years of training in the psychology of people at work. Doctoral psychologists have the same level and years of training and experience as medical doctors, and can only re-register each year if they have undertaken significant professional development. In this respect, professionally qualified psychologists will have a much better understanding of psychometric tests. Some test distributors may have only undergone a few days of training in the tests they sell and will lack a proper understanding of the science and metrics beneath the tests, and are not accountable to anybody for their competence.
Which test?
As for choice of test, this poses a perennial problem for those without training in psychometrics. Many, new to the field, will ask which companies use a particular test—which would then require the distributor to break commercial confidence. Alternatively, HR may come pre-armed with the name of the test they want to use because a friend’s organisation uses that particular test. In fact, such recommendations should have little bearing on test choice, as tests have been designed for different situations and different people. A friend may have chosen their test when there was less market choice, and it may no longer be the best one currently available. So HR is advised to survey all tests currently available and select one that best suits their individual requirements bearing in mind the following key concerns.
How many questions?
One of the most frequently asked questions regarding personality tests is, “How many questions are there in the test.” When we reply with, “200” there is usually a gasp—it should be noted that the reliability, and therefore the validity, of tests generally increases with the number of questions—provided the test is not ridiculously long. Most good tests, which assess multiple relevant aspects of personality for work, do so with around 200 questions. Personality tests cannot do their job well with the 20-50 questions that many in HR may expect. This is one reason why some test distributors do not offer free trials, as a test cannot be evaluated on the basis of the questions alone. Trial users may also be discouraged by long questionnaires and repetition of questions—when in fact this results in higher reliability and a better measure of candidate consistency and attention.
Locally or internationally developed?
There has been a movement that supports Chinese Personality tests that are ‘designed by the Chinese for the Chinese’. The idea may sound sensible and quite seductive, however, research published in the British Psychological Society’s Selection & Development Review, entitled: The Chinese challenge to the Big-5, indicated that locally-developed tests lacked the rigour of internationally-developed ones and had unacceptable reliability levels. Additionally, they had no incremental validity over internationally developed tests. Traditionally, Chinese culture may place greater importance on certain aspects of personality, such as ‘face’, than other cultures, however, our research did not demonstrate that this had any utility in helping to predict candidate performance at work in Hong Kong companies.
Overcoming inertia
If psychometric tests are so useful, why does every organisation not use them? Firstly, cost—and many in procurement may not consider the return on investment. Every aspect of the selection process incurs costs. However, as capital invested in psychometric tests is usually paid to a vendor rather than to in-house staff, this often acts as a further deterrent to purchasing them in the first place. HR can help by explaining the tangible ROI benefits of psychometric tests to those in procurement so they are better aware that using good tests results in superior prediction of talent performance, which will lead to higher retention rates, lower absenteeism, a happier workforce and ultimately a more successful organisation.
Test administrator training essential
As with any new procedure, adopting psychometric tests requires planning, a company policy on test use, fairness, competence in testing, etc. and this all takes time and can be off-putting. HR is often surprised by the need for training test administrators. Proper training is, however, extremely important so that the HR staff administering tests and making decisions based the results know how to use and analyse them properly. Less reputable test suppliers may succumb to the reticence of organisations to train staff by offering tests that require little or no training, however, this is likely to fuel misunderstandings, misinterpretation and ill-informed decisions. For example, if a candidate scores low on an empathy scale—without proper understanding of the test—some may assume they have little empathy. In reality, the scale always means more than its label and the greatest validity can only be obtained from interpreting interactions of scores among multiple scales. Furthermore, interpretation in selection varies significantly depending on whether tests are used with ‘forced-choice questions’ or ‘rating scales’.
As in any assessment method, there is a degree of error in psychometric test scores and this must also be taken into account during interpretation. Quality psychometric training courses will help educate administrators how to do this. The best training courses also teach the right questions to ask distributors and how to evaluate the response—from asking for reliability and validity figures, to understanding the rationale behind the test itself and the test publisher’s background.
Training bloopers
Non-psychologists, who run training programmes, may lack a comprehensive understanding of the field. One trainer for a well-known personality type tool told his class that criterion-related validity was most important for his test—the reality being that construct validity is. The manager at a profiling company in Hong Kong told an enquirer that his questions were getting too complicated and most clients don’t get that technical. The client was only asking the questions they were taught on a six-day British Psychological Society course. One supplier defined test validity as being ‘how accurately the candidate believes the test represents them’ and claimed their tests average 95% validity. In fact, there are many types of technical validity and this is not recognised as one of them. So, HR should be cautious with validity claims that seem too good to be true—check exactly how the supplier defines the term ‘validity’. More credible suppliers would provide evidence that the test significantly predicts meaningful workplace performance variables. This evidence should come from a sizable sample of employees rather than students in universities.
Top 5 questions to ask suppliers
- What is the rationale or theory behind the test—is it a validated theory?
- May I see evidence of the test’s reliability? Look for internal consistency and test—retest reliabilities of 0.7+ for personality tests and 0.8+ for aptitude tests. Reliability is a precursor to validity.
- May I see evidence of the test’s validity? Look for either criterion-related validity or construct-related validity depending on how you will be using the test. You need to see that the test predicts something meaningful or accurately assesses your construct of interest.*
- What training is required to use the test?
- Do you employ experts in psychometrics, such as business psychologists, rather than test salespeople who are not experts in psychometrics or psychology?
*Notes
Criterion-related validity is evidence that the test actually predicts what it says it does, for example, leadership potential or performance on various competencies.
Construct-related validity is evidence that the test really does assess the construct it was designed to measure, for example, numerical reasoning or conscientiousness.
Conclusion
In summary, psychometric tests have an established record of being able to predict the skills and performance of potential hires and new recruits. However, for them to be effective, it is crucial that HR first equips itself with the right test. In order to choose the right test, HR needs to ask the right questions to suppliers so they can gain a proper understanding of the technical properties of the test, rather than simply relying on colleague recommendations. Psychometric tests are here to stay and both test vendors and HR have a responsibility to ensure competent and ethical test use and thus accurate and fair prediction of talent and performance in Hong Kong’s workplaces.
For more details visit: www.psychometricassessment.com
Paul Arkwright
Publisher
