Two Surveys Are Better Than One

Warren Heaps – Birches Group LLC

If you work in international HR, you know how important market data is for the management of your international operations.  Finding reliable data in all of your countries is undoubtedly a challenge.  Each country has different survey suppliers, different employers participating in the surveys, and variable levels of quality.  So naturally, when you find a survey you trust, the tendency is to stick with it.  But is one survey enough?  I don’t think so.

Two Surveys Provide A Balanced View of the Market
Typically, your “go-to” survey has several characteristics:

  • It reflects the market in which you compete – the participants are the ones you are most often look for, usually from the same sector.  It’s the survey “everyone” is in.
  • It’s the one that “corporate” designates (and pays for)sometimes it’s not the best, but you have to follow headquarters instructions, and besides, it doesn’t hit your budget.
  • It’s the survey you’re used to  – why bother exploring another option when you’ve finally mastered this one?
  • The survey results are OK – you’ve never had a problem using the survey, except a few times when some of the data was a bit suspect, but overall, it’s fine.

These are some of the typical reasons why companies participate and rely on surveys year after year.  But these same reasons often stand in the way of exploring additional options.

They say two heads are better than one.  While I have yet to meet a two-headed individual, I have worked with a lot with surveys, and I can attest to the fact that two surveys are better than one!

There are many reasons for this — here’s just a few:

  • Some key employers might be missing – by participating in another survey, you will get an alternative group of employers and therefore, a broader view of the market.  If your main survey is a sector-based one, try a general market survey to complement your sector data.
  • Survey methodologies vary – and this means the results from another survey could be different.  Different could be good – it will challenge you to dig deeper and try to understand the differences.  Perhaps the alternative survey provides more details in certain areas, such as non-salary allowances and benefits in-kind, or uses a unique valuation for benefits.
  • The results might actually be aligned – maybe the data from the two surveys matches closely.  That reinforces the conclusions you may have already reached and raises the confidence in both surveys as a result.
  • The results might be totally different – thus allowing you to explore the full extent of the market options.  Is it the employer group that causes the difference, or the methodology?  How do know which survey is right?

The answers to these dilemmas, of course, vary according to your unique situation.  Which country?  Which sector?  How great are the differences? You have to explore to find the answer for your company.

So Look Beyond Your Old Reliable Survey
Challenge yourself to participate in at least two surveys.  Make sure that they focus on different aspects of the market and bring a unique perspective. Analyze the data carefully and apply your judgement to incorporate data from both perspectives.  Soon you will find you have a better understanding of the market.

If you need a second survey source for your international locations, be sure to check out Birches Group surveys.  We cover 148 developing markets, more than any other global provider, and together with Aon Hewitt, our marketing partner, we cover the world.

More About Warren

Warren Heaps

Warren on LinkedIn

Developing Markets Compensation and Benefits Group on LinkedIn

Email Warren

9 responses to “Two Surveys Are Better Than One

  1. Pingback: Two Surveys Are Better Than One - International HR Forum - Member Blogs - HR Blogs - HR Space from Personnel Today and Xpert HR

  2. As a fellow professional provider I totally agree with Warren that a company should use at least two as whilst everyone works to produce usable information in a comprehensive format the emphasis can be different between consultancies. But how many more? You could end up with paralysis by analysis.

  3. I like the article Warren; good advice to look outside the confines of the structures (sometimes strictures) that we have built. I speak from professional experience that you and your team provide excellent, relevant data and inputs.

  4. Totally agree Warren !
    Two is better than one for all the points you mention. I see 2 other advantages of using different surveys :
    * They may not cover the same jobs, giving you more data points for your employee base. For example you may find some industry-specific roles in one survey and other in the second survey.
    * The matching exercise, though long and sometimes cumbersome, provides an opportunity for you to dig deeper into the roles that exist in your organisation. That by itself makes you a better HR business partner or a better Compensation specialist.

  5. I agree with the general principles. And if you really want to go for the jugular, and are operating in a survey-rich market, try three (3) surveys per benchmark job. This is especially if you want to match current modal practices in market-referenced job pricing (‘market pricing’). This allows you to achieve greater balance in the three simplified steps that I consider important to pricing JOBS (I always have trouble with the notion of ‘market pricing’ versus job pricing: Descriptions, Matching, Pricing (DMaP for short). (Simplified, but the devil is in the details).
    1. Descriptions: Job Descriptions are central to any survey work. Your job descriptions, at least for those jobs you consider as benchmark jobs, need to be current and comprehensive, among other things, relative to the Level of Work, and Nature of Work. These make it easier to determine the required qualifications (KSAOs). If you start with a focus on benchmark jobs, at least you will have a starting line of well-documented jobs that anchor the rest. This works even better when you sample each job family in your organization to ensure that you have one or more benchmark jobs for each job family (the more grade levels you have the higher the number of benchmark jobs you will need for practical purposes).

    2. Matching Jobs to Survey Benchmark Jobs: We all know that job titles are often misleading as basis for matching jobs. So we have to look at job content. But pay surveys are notorious for being regrettably brief in their descriptions of benchmark jobs. Accordingly, the process of determining the degree of match between each selected (not all) internal / company job and the market jobs from survey sources is a delicate matter of considerable care and skill. This is so when you need to specify the degree of match, and rule out those survey jobs that do NOT match. A rule of thumb is that a match of about 80% of more is worth including in your process. Anything less becomes somewhat suspect. But then, that depends on the abundance of surveys in your market for your jobs of interest.

    3. Pricing Jobs: Assuming then that you have three benchmark jobs from three survey sources that reasonably match your selected (not all) internal / company job, the business of pricing your job becomes easier, when you apply the degree of match as weights in determining the composite price fro each key quartile (say, 25th, median, and 75th). Again this assumes that your survey sources provide these measures. Once a well distributed set of benchmark jobs are priced, you have opportunity to slot in the rest. Again, the devil is in the details.

    Rewards in the Jungle:
    All these procedures are fine, except when you are in a rewards ‘jungle’: defined here to mean a relatively undeveloped market relative to the availability of conveniences, access, good road maps, and optional routes relative to good quality pay surveys.

    • EK –

      Thanks for your very comprehensive comments. I agree for market pricing the steps outlined are the right ones. And yes, in survey-rich markets, three is even better than two surveys.

      But as you mentioned, rewards in developing world are quite different. Birches Group surveys are unique in our approach, since we survey only in developing countries. For example, we interview every participating employer and our analysts do the job matching, not the client. This is easier for clients and more accurate and consistent for the survey. After all, who are the most expert in applying a survey job evaluation method than the employees who work for the company that designed it? We are gobsmacked by the fact that all the “majors” rely on clients to master the consultants tools, instead of the consultant applying them for the client, as a service provider should (e.g., provide service, right?).

      You might also want to check out these prior posts on the site that touch on issues specific to developing country market surveys:

      Thanks again for your informative contribution.


  6. Pingback: Compensation Insider - Share the knowledge - June 2012 |