Missteps Make for Better Analysis

Oops. By Malcolm Slaney
Oops.  Courtesy of Malcolm Slaney.

A major voice in people analytics just advocated for the professionalization of my field.  An April 27, 2017 blog post by Max Blumberg and Mark Lawrence suggests that workforce analytics regulate itself under a professional association.  The authors have a good point.  The explosion of alleged experts in my field is making things really confusing for lay audiences.  We have no idea if someone claiming to have expertise is truly knowledgeable.  There is a gold-rush mentality in workforce analytics, and we can barely distinguish those on the cutting edge from the outright con-artists.  Bad experiences and false starts are causing skepticism.

I agree with this assessment of the current state of affairs.  I decline the vast majority of conferences, webinars, and software on offer.  Being strong at workforce analytics turns on having daily exposure to the data itself.  I have yet to hear a provider offer something more interesting than that thing we just figured out last week, by ourselves, with in-house staff using excel.

However, I have to disagree with the proposal that the field should be regulated.  You see, the main opportunity is to democratize the skill set and bolster the overall number of people who read the data and create simple calculations.  If you can get one-tenth of a human resources team to tool-up with a small amount of learning and experimentation with the data, that’s a huge boost in organizational capacity.  There is one specialist for every five or 10 people in the earliest steps of the learning curve.  Tinkerers and new entrants are half of the equation, and sometimes they are the most important half.

There is another problem.  We don’t yet know what excellence in workforce analytics looks like.  Sure, getting the attention of the c-suite, saving money, having clean data, and making your findings presentable are really obvious signs that you know this stuff.  But mysteries abound.  The information is disruptive to those with power, so how shall we deal with the office politics?  The data improves every day, so how do we maintain composure while discussing last-year’s erroneous data.  We’re supposed to align to strategy, but strategy and leadership change is constant.  And how are we to negotiate the boundaries between the professions when accounting has their own cost model, and marketing researchers are experts in employee surveys?

The mystery, confusion, emotional drama, flashes of growth and pride all bring the field to life.  Workforce analytics is a mosh pit.  Our outputs are a meal thrown together from what is leftover in the fridge.  Our first attempt at everything looks like a Pinterest fail.

Let’s keep it messy.  We’re more honest that way.  Besides, we work harder when we’re having fun.

Share Your Data, Share Your Power

Sharing, by Andrii Zymohliad
Sharing.  Courtesy of Andrii Zymohliad.

I looked up the phrase “information is power” to figure out where it came from.  It turns out, it comes from everywhere.  The phrase is actually part of several longer and more complex quotes.  It is a call to freedom, such as Kofi Annan’s statement that “Education is the premise of progress, in every society, in every family.”  By contrast, the control of information has a coercive impact which we must overcome.  Robin Morgan says “the secreting or hoarding of knowledge or information may be an act of tyranny camouflaged as humility.”  Both themes ring true with workplace analytics.  First you need to take full advantage of information for your own empowerment.  Second, you are obliged to use that power justly.  The smart money is on the sharing of information, and the sharing of power, in a high-functioning modern workplace.

Millennials: a Shiny Face on All Behaviour

Untitled Photo Courtesy of Bina. (=)
Untitled photo courtesy of Bina.

How much can we talk about people without talking about people data?  Not very much, it appears.  Those dealing with employees of all types must know more about their hearts and souls than ever before.  And if you make one false move with a data point, your most brilliant philosophical insights can be taken sideways.

In December 2016, author Simon Sinek was interviewed on Inside Quest on the topic of Millennials.  I am a big fan of Sinek, having changed my approach to work based on his influential TED talk on how to Start With Why. The Inside Quest interview (20 minutes long) is also great because it covers many key topics.

Sinek posted a follow-up video days later to clarify much of what he had to say.  There was a dramatic change in body language.  In the first video he seemed calm and knowledgeable.  However, in the follow-up video (from what appears to be his dining-room) he is a little sheepish, making clarifications, imploring people to keep the conversation alive with constructive criticism.  The first interview had gone a tad viral and he got a lot of feedback.

During the Inside Quest interview he made piercing social criticism and attributed a lot of what was happening in society to the experience and context of millennials.  In what should be described as “a good problem to have,” he understated the importance of his critique.  You see, the things he said were true for many of us regardless of generation.

His critique?  We must learn to wait.  We must put time and years into our greatest accomplishments.  We are lonely because we are embarrassed to talk about our disappointments and frustrations.  We need to talk through our difficulties.  We must aspire to engage in sincere conversations.  We must help others.  Look up from your phone and be human.

In my opinion these are all massive issues for workplace culture.  Managers are struggling to learn how to compel their staff to work hard without being coercive or demeaning.  Everyone who takes benefits costs seriously is now hyper-sensitive to whether employees can talk openly about mental health and wellbeing.  Executives worried about people quitting are stumbling onto growing evidence that people want to thrive and grow.  And still, the dream persists that we can all succeed.

I think that these topics entered the mainstream concurrent with the rise of the millennial workforce, not necessarily because of them.  The analytics that identify turnover trends happened largely because of emerging technology; the de-stigmatization of mental illness was popularized by baby-boomer medical professionals; smart phones have been improving for decades; and teachers have been pushing anti-bullying efforts for some time.  These things came sharply into focus when millennials first started to speak their minds in the workplace.

Based on his dining-room talk, it appears that Sinek’s feedback came from many non-millennials who want in on the broader discussion.  This is important from a social perspective.  But the social perspective is the flip-side of a data issue.  That is because he got tripped up by a data-labelling error.  You see, he casually referred to millennials has having been born approximately 1984 and after.  He didn’t specify a 20-year generational cohort.  He left it open-ended, like there was an unlimited supply of this generation being born every day.  This is problematic because we need good definitions to determine if there are clear differences between clear categories.  If the definition is muddy, then the identification of differences will be muddy as well.

I have had the pleasure of working with clearly defined data where I described millennials as those born from 1976 to 1995.  By getting specific about date of birth, you will find that each year you look at the data the findings can shift.  Age and generation are not the same things, and if you look at the two separately you might find, for example, that millennials as a generation do not have different quit rates.  Or you might find that concerns about career advancement are widespread (more on that in a future post).

For me this is an excellent example of how workplace analytics and workplace culture are never that far from one another.  To love humans is to wish the very best for them and their data.

New Technology Not Entirely Helpful

smartphones. by Sam Churchill
Smartphones.  Photo courtesy of Sam Churchill.

Josh Bersin of Bersin Deloitte is expressing some skepticism about whether new technologies are actually delivering productivity improvements.  Allah D. Wright contributed this article for shrm.org describing Bersin’s comments at a speech in Hyderabad, India on April 20, 2017. Bersin spoke of the long-term impacts of technology on economic transformation.  He noted that historically technology has had very large impacts over time.  However, there are some downsides of new technology.  The average US worker spends 25% of their time reading or answering email.  The average mobile phone user checks their device 150 times per day.  Work is getting harder, with 40% of the US population believing that it is impossible to balance work and life.  Bersin asserts that it is not HR’s job to cause technology to succeed, but rather to pay attention to the way technology changes the way we work.

In my opinion, the major shift in the past decade has been the flood of incoming information.  The new emerging skill is the careful determination of what incoming information is useful.  Decision-making about which information is valuable needs to be diffused, or employees will simply be flooded.  This means that the new skill sets will be the assessment of information for relevance, taking pauses to reflect between waves of new developments, and the more cautious and deliberate composition of our own outgoing communications.  After all, if you’re just passing along high-volume spam and memes, you may be replaceable by artificial intelligence.

Excellence Isn’t What You Think It Is

DSC08778 by Yasunobo Hiraoka, (=)
DSC08778, courtesy of Yasunobo Hiraoka.

In human resources you might spend a lot of time talking about employee performance and what it means to be excellent, average, and under-performing.  But your conversations about performance might be trapped in a decades-long mathematical error which skews your subjective judgements.  It’s worth exploring assumptions about the bell curve performance distribution, if only to be a little wiser when you “use your words.”

You will often hear that high-performing employees are three times as productive as an average performer.  I looked into it, and it’s not true.  The differences in performance are far greater, in some cases six-to-one or more.  I tracked down a good academic article by two Organizational Behaviour academics, tucked behind a paywall.  The paper is “The Best and the Rest: Revisiting the Norm of Normality of Individual Performance.”  Ernest O’Boyle Jr. and Herman Aguinis.  Personnel Psychology 2012, 65, 79-119.

The authors note there is a decades-long consensus that employee performance is distributed on a bell curve.  However, this consensus might be way off base.  Through a series of studies, they show that the distribution of performance more closely resembles a power-law distribution.  Here’s what the two distribution curves look like side-by-side.



The blue diagram should be familiar to most people as “the bell curve.”  There are a bunch of proper names for it, but the features are well-known.  The largest number of people is really close to the peak in the middle, which in this case is the average performance score.  A bunch of people are a little to the left or little to the right of the average, and those are your below-average and above-average performers.  Then there are tails on either side of the curve; those are the rare low and high performers who are often about to get terminated or promoted.

There’s another way to look at it.  The rose-colored diagram is called a “power-law” distribution.  (Note that in this case the axes are reversed so that performance runs up-down and percentage runs left-right).  This diagram can reflect the likelihood that you will buy a rock album, with you and millions of others buying Beatles and Broken Bells on the far left of the curve, and dozen people buying albums from your own band on the far right.  The important thing to notice about the power-law distribution is that there’s a lot of activity on the far-left side of the diagram where the high performers are satisfying customers.

Researchers tested employee performance in a number of fields and found that performance more closely resembles the rose-colored diagram, the power-law distribution.

The research isn’t supposed to turn out that way, according to mainstream thinking.  The authors get into why we assume that performance fits this bell curve, and it’s not flattering to the legacy of social scientists.  Their zinger is that this is a “received doctrine” passed down from one decade to the next.  Yet if you trace the doctrine back to earlier sources, nobody can name that one study where they proved that the bell curve made sense in the first place.  Rather, there is lots of evidence of people fudging their data and throwing out the “outliers” to get their model to fit the doctrine.

The areas of performance that they studied were entertainers, university professors, politicians, and athletes.  There’s a small vulnerability (which they acknowledge) that these industries might not reflect all sectors.  In my opinion these are all star-system fields with a winner-take-all rewards system, and that system isn’t true of all types of work.  But on the upside, they have chosen fields with lots of data, and where our personal perceptions match the data.  We have heard of Sinatra, Einstein, Reagan, and Babe Ruth.  We understand that people just below the top ranking in these fields are barely known.

According to the math, the power-law distribution implies that top performers deliver the goods to a far greater extreme than originally thought.  People who perform at two standard deviations above the mean – a common measure for high performance – would be four times as productive under the bell curve.  Looking at actual performance, which more closely matches the power-law distribution, the correct multiple is seven times as productive.  At the top one-tenth of one percent of performance, the bell curve says they are six times as productive but power-law says they are twenty-five times as productive.

What about people who are below-average?  With the bell curve, the below-average people are one-half of the population because the average cuts the distribution in half.  It’s almost like a democracy.  But because superstars deliver so much more under the power-law distribution, it skews the average and creates a larger pool of people who are below average.  With a power-law distribution, below-average performers are 66% of academics, 83% of actors, 68% of politicians, and 71% of professional basketball players.

This research has many implications.  For example, those who have excelled might want to keep all of the gains for themselves, opening a controversy about the distribution of the spoils.  However the authors flag that excellence does not exist in a vacuum and all of these people are surrounded by support systems that cause them to be great.  Perhaps the gains from high performance need to be re-invested into these support systems to sustain excellence over the long-term.  Some superstars also engage in anti-social behaviours because of fawning admirers and their employer’s reluctance to terminate.  These behaviours make great gossipy television, but it doesn’t look good during lawsuits.  It is also unclear what traits cause these people to be superstars and whether these traits can be developed. Could we choose to create superstars then cast them aside every few years and start again?  Isn’t that what boy bands are all about?

I have worked in a couple of fields with a number of employers, and I have experienced diverse feedback about my own performance.  Let’s give others the benefit of the doubt and assume that the feedback is accurate.  Could it be that each of us is exceptional at one or two things, and mediocre at the rest?  Does it sound about right that we are more exceptional in some environments than others?  There is over a thousand professions on this earth and there are many workplaces.  How do you find that one skill and that one workplace where you rock the world?  …but enough about me and you.

If you work in human resources, how do you help other employees find that one thing?  If you’re in public policy, how do you organize a modern economy so that more people can find that perfect fit?  Would you ear-tag individual employees by type, place them into known jobs, prescribe what they ought to learn, and judge them against the average?  Or would you assess employees for their past moments of magic, foster intrinsic motivation, cultivate them to experience bursts of growth, build the work around their talents, and encourage them migrate into roles that are best for them?  As you can see, core mathematical assumptions have a big impact on how we talk to one another as humans.

Beating Bias with Blindness

Blind Justice. By Tim Green.
Blind Justice.  Photo courtesy of Tim Green.

Managers and human resource professionals are supposed to have non-discriminatory hiring practices.  Yet we are only in the early days of seeing job applicants neutrally.  There are several new (and not-so-new) methods for considering applicants fairly.  There is also the possibility of using good math to prove and reduce bias.

Canada’s federal public service announced on April 20, 2017 that it is starting a pilot project to recruit job applicants on a name-blind basis.  The minister responsible said “research has shown that English-speaking employers are 40 per cent more likely to pick candidates with an English or anglicized name…”  At the end of the pilot they will analyze the two sets of candidate shortlists, both name-blind and traditional-method.  The results of the experiment will be ready in October, for possible roll-out to the entire public service.

What is worth noting is that the Canadian government is running a formal experiment for a limited time.  This raises hope that the eventual course of action will be determined by evidence, not speculation.  They will measure the discrimination before attempting to remedy it, which could bolster support.  The approach also implies the pilot has permission to fail.  After all, they might find something totally different from what they expected.  But that kind of thing that happens when you care about science.

Of course this pilot addresses only one part of the discrimination puzzle.  I would speculate that résumés that still indicate the year and city in which a degree is attained will tip-off employers about age and ethnicity.  An obvious next phase of analysis is to block-out the graduation date and the name of the University.  After all, you only need to know if they finished their degree, plus the degree’s level and academic major, and a broad sense of the school ranking (i.e. top-100, top-400).

Job applications also reveal writing style, which should be good.  But there are differences between the sexes in the use of words.  In the book The Secret Life of Pronouns by James W. Pennebaker the author reveals the findings of high-volume statistical analyses revealing (amongst other things) that men make bold pronouncements without referring to themselves in first-person.  Women, by contrast, attribute their story to themselves, which is more clear, social, and modest.  I personally think that confidence, and willingness to boast, are unreliable indicators of competence.

In classical music, blind auditions are now commonly used to select new hires onto symphony orchestras.  They’ve been doing this for years.  The musicians submit recordings of their auditions and provide live performances behind a physical screen.  I have heard that judges gossip “you can tell” if the candidate is a man yet when the winner steps out from behind the screen it is often a woman.  In this not-so-new paper from 2000, authors Claudio Goldin and Cecilia Rouse conducted an analysis of 7,065 individuals and 588 audition-rounds to see what impact blind auditions had.  They identified that the blind auditions work.

When you’re fighting the man, words are important.  When you’re putting change into effect, math is importanter.