First time here? Get updates by RSS feed or follow me on Twitter.
Close« June 2007 | Main | August 2007 »
When was the last time you went to a conference and really learned something? At Business of Software 2007 you'll learn the timeless, eternal truths of what drives success in the software business. No fads, no sales pitches, no bull.
The early bird discount for Business of Software 2007 will end on August 15th. To hear Guy Kawasaki, Joel Spolsky, Tim Lister, Eric Sink and more great speakers for the special 2-day price of only $1195 you need to book soon.
The software industry is facing a plague in an area that affects just about every company’s business strategy: PR. The sickness has been with us for more than a decade, but a variety of factors have ignited an epidemic over the last few years.
Software companies of all sizes should care about this for a two reasons: They could be wasting thousands or even millions of dollars that could be spent more profitably elsewhere. (2) They are missing opportunities that could differentiate them from competitors, help them build loyal customer bases, and ensure long-term success.
The decline of traditional PR can be traced to how it is defined and practiced. PR is most often defined in terms of persuasion and influence. In practice, that has often meant inundating audiences with messages that are thinly disguised propaganda. It presumes a certain naivety about the audience, and faith that PR practitioners will be so good at persuasion and influence that those who receive their messages won’t know what hit them.
This type of approach probably never was as effective as PR practitioners and their clients believed it to be, but it is even more suspect in the freewheeling world of participatory journalism and the blogosphere, where scents of advocacy or self-interest are readily detected, and rebuked with an unprecedented vehemence.
Simultaneous with the rise of online forums and blogs has come another affliction: the rapid decline of print publications and news-oriented web sites, many of which were the depositories for the meat and potatoes of traditional PR – reams of press releases, 2,000-word white papers, product-promoting case studies, canned testimonials, and advertising-for-editorial trade-offs.
Forcing the change
The massive ground shifts of the past few years have PR professionals worried about the fate of their industry, but most are still in denial. As long as the checks keep flowing, things are presumed to be OK. So, we’re at the point where all reform must take place: The companies that are paying PR practitioners must force the change.
As with most change, this one must start at a basic level. That means changing the terms that define PR from persuasion and influence to information and integrity – words not normally used in the same sentence with PR.
PR programs based on information and integrity can be a scary notion for both the PR profession and its clients. From the PR profession’s side, it means, at best, retooling many of its entrenched methodologies, and retraining or replacing people. At worst, it means massive business losses and trying to start again from scratch.
There’s an equally daunting challenge for software companies: to make this form of PR work, they must deal honestly and openly with their customers, the public, and the media in all of its permutations. This takes a company that is secure in its skin; one confident enough in its competitive assets and way of doing business that it can deliver information and let the user community do the persuading and influencing. For companies that can operate in this manner, there is the rare opportunity of not just winning customers, but gaining long-term, vocal advocates.
The people to make it happen
As with all change, the most important element is finding the right people to make it happen. Most likely, these will be people already inside your company, new hires not encumbered by old PR ways, an individual (perhaps an ex-journalist) who knows your industry, or a small agency attuned to your business. The solution is unlikely to be found in large agencies, because their structures don’t normally accommodate the individualized attention a company needs in the new PR environment.
The people you’re looking for should possess an eclectic group of skills:
What will they do?
So, what will these people work on if their time isn’t totally absorbed with traditional PR activities? They’ll be busy doing these types of things:
Throw out the old, bring in the new
Any company moving away from traditional PR will be forsaking some time-worn methods of measuring effectiveness. Instead of hundreds or thousands of press clippings a year – the vast majority of which are meaningless to your market – you might have 30 or 40 well-placed articles. You won’t need three-page activity reports each month that document every phone call and transaction; an informal reporting will do, and results should be obvious.
In place of the traditional PR paper trail, your new approach can give your company some things it probably has never had before: positive exposure in the most important media outlets; rich information exchanges with buyers and users; a sense of loyalty to your company and its brands; an opt-in list of people who actually want to receive your e-mails; concrete testimonials in a true voice of appreciation; and a sense of trust that will provide a bridge for introducing new products and technologies into the market.
As you read this, there are raging discussions throughout the Internet about whether or not PR is dead and how it should be conducted in the future. There are companies bewildered, frustrated and angry with their PR people. And, there are the trailblazers who are doing with PR what innovative companies always do: throwing out convention and blazing new paths that distinguish them from the competition.
###
Bob Cramblitt is principal of Cramblitt & Company, based in Cary, N.C. He has more than 20 years of experience in the software industry as a writer, editor, and marketing/communications and PR specialist.
A musician arrives in a small Mexican town. He’s mistaken for a guitar-wielding killer. Assassins try to track him down and kill him. That’s the plot of El Mariachi, Robert Rodriguez’s 1992 film. It lacks some polish, but it’s a raw, gripping action film. It’s a better film than, say, Miami Vice.
Rodriguez spent $7,000 making El Mariachi. That’s less than Columbia TriStar spent on El Mariachi's logo, and less than Michael Mann spent on Jamie Foxx’s private jet for Miami Vice.
So how did he do it? Rodriguez was both the editor and the photographer: he could make on-the-spot decisions about what to do. He used a single, hand-held camera. He used squibs made from condoms filled with corn syrup and food colouring. For the shoot-out scenes he borrowed a couple of guns from local policemen and gave water pistols to the rest of the cast. Local newscasters complained about people running down the street waving guns, so he put them in the film.
He still had a script though. Most importantly, he had a clear idea of what the film was going to be. He refused to get sucked into a spending spiral. He couldn’t: he had no money. In his own words, the key to making a low budget film is “You refuse to spend. Think of a creative way to get around your problem.”
Bill Buxton likens writing software to producing a film. What can we learn about software from Rodriguez? It tells us about the curse of plenty. If you have plenty of time, money and people then you use them. And the extra output isn’t always worth the extra input. Microsoft is a good example. They’ve got a lot of money, a lot of people, and a lot of time but they’d write better software if they didn’t. Last year, Moishe Lettvin wrote about how he spent a year on a feature that should have taken a week.
A couple of years ago, as Microsoft was desperately trying to get Visual Studio Team System out of the door, I asked the man in charge how he’d ship it faster. At the time, he had a team of 250 people, all working on this project. He said - and I don’t think he was joking - that he’d get rid of half of them. He was right. That would have been a good idea.
I see the curse of plenty at Red Gate too. We’ve just launched a new version of SQL Prompt, our code completion tool for SQL Server. We want a splash screen for the installer. We’ve already got a logo, so normally a developer would just stick it into Paint.net, add a bit of text and use that. Now, however, we’ve got an in-house graphic designer so we ask him. He’s a fantastic designer, with lots of great ideas. Obviously he wants to do the best splash-screen possible, with alpha-blending so we aren’t stuck with a boring rectangle. The installer is written in C++, so that adds complexity. Eventually, reason wins and we just use the logo with some text. Boring, but does it really matter?
Writing software, like making films, is a compromise. And that’s the way it must be. Unless, of course, you can afford to never ship. As Robert Rodriguez says, “It’s such a big production when you make a big production out of it. Don’t.”
Next time you’re in a similar situation, ask yourself “What would Robert Rodriguez do?”
Rob Malda, aka CmdrTaco and founder of Slashdot, kindly agreed to be interviewed as part of my series on influential and interesting people in the software industry. We talked about Slashdot, Digg and conspiracy theories.
Davidson: Social bookmarking sites are very fashionable at the moment. How much of a threat are they to Slashdot?
CmdrTaco: I think that we provide a good balance between the wisdom of a crowd and the tyranny of a mob. Some people may be ok with what the social bookmark websites provide. When I use them, I see far too much garbage mixed in with the quality. Developing a system where everyone participates is very difficult because everything goes to the middle.
Davidson: Yet the firehose clearly brings some kind of social element to the editorial process. How does the firehose influence you? Do you ever post / withhold stories against your better judgement because of what the firehose is telling you?
CmdrTaco: I use the firehose as a tool. It can influence my decision making process, but it's only part of the big picture. A stupid story is still stupid even if 100 people think it's great. Likewise many stories which are valid news are not appropriate for Slashdot. My role is to maintain that continuity.
Davidson: I agree with you about the garbage. For me, sites with strong editorial focus such as Slashdot and Boing Boing are far more interesting than, say, Digg. However, it's impossible to deny Digg's popularity. On the principle that competition can be good both for customers and producers (it reduces complacency and forces you to sharpen your game), has this changed your behaviour?
CmdrTaco: "Popularity" is only one measure of success. Personally I find Digg's story selection to be shaky at best. Reddit does a much better job, and it is less popular. But both sites are struggling with their growth. Their value was the focused nature of their community. As each grew, that focus diluted, and with it the quality of their story selection. I like the speed that those sites can operate with. I like the large volume of content that they can work through. But I like the quality control measures that I can impose. That's what the Firehose stuff we're working on is for.
Davidson: As a founder of something as visibly successful as Slashdot you've got to deal with others bitching and sniping about your opinions and decisions. Is there anything in particular that irritates you the most?
CmdrTaco: One of my biggest pet peeves is the conspiracy theory. Anything that I do can be interpreted as a conspiracy. Anything that happens on Slashdot can be interpreted as a conspiracy. Honest bugs in our system are interpreted as a concerted effort to oppress someone. Lack of certain features is interpreted as malicious when the truth is that we don't have much developer time. I've long ago had to give up on addressing every one of these ridiculous theories in the forum, but they really are a type of virus. People love to assume the worst and it can really be frustrating when you're really trying to do your best.
Davidson: Do you think that's human nature, or is it exacerbated by the slightly cynical nature of your readers? Or maybe that's a gross over-generalization?
I think it's largely human nature, but generally speaking Slashdot readers are just conspiracy oriented to begin with, so I'm never surprised when that trait is directed back at me. But it still hurts when someone thinks your best efforts are of malicious intent.
Davidson: Do you think participatory journalism will one day make traditional media outlets obsolete?
CmdrTaco: Nope. Traditional media still provides a trust network that participatory journalism lacks. Until we figure out that problem, the two will exist side by side.
Davidson: Do you think participatory journalism will ever earn that trust? Or does it require a professionalism (professional writing, editing, fact-checking etc.) that only an institution can provide?
CmdrTaco: I don't know if it will earn "That" trust but it will earn "Some" trust. My level of trust for Fox News is different than my trust for Encylopedia Britannica. Each is flawed in some ways, just like "Participatory Journalism".
The real issue here is trust. "Fact Checking" and "Editing" is not something that "Only an institution can provide". Wikipedia can do that for example. So can a responsive blogger and a discussion forum.
The traditional role of mainstream publications was that of distribution, but the internet removes that. But another less understood role of those publications is to provide a sort of trustability rating. "Bob writes for the Wall Street Journal, therefore I trust him more than a pamphlet I was handed by an unshaven man on the street." There are many ways to define trust... for example having 10,000 MySpace friends. Or perhaps just knowing that your friend vouches for a guy. All of these things mix together to provide trust-ratings.
Davidson: Does Slashdot provide all the news that you personally need? Or do you ever kick off your slippers, lie back and read a newspaper?
CmdrTaco: I still read a bit of mainstream news, as well as "Silly" news. Slashdot covers tech news extremely well. And when you read the full, uncut submissions that come from our readers (which are available at http://slashdot.org/firehose.pl) you really get a surprisingly diverse sampling of subject matter and perspective. But if you want silly news, comics, or just a good top 5 international news headlines thing, you need to go elsewhere.
Davidson: If you didn't run slashdot, what would you do?
CmdrTaco: I'd probably just be doing web development for a mid-sized company.
Davidson: One last question. Would you prefer to be stuck on a plane for 8 hours with Steve Jobs, Richard Stallman or a bunch of Slashdot readers?
CmdrTaco: To be fair, I'm sure both of them have read their share of Slashdot and I wear headphones on planes :) But I guess it would depend on which Slashdot readers - I've met some great ones, and man have I met some freaks!
If you enjoyed this interview, then read my interview with Robert X. Cringely where he talks about the Web 2.0 Pillsbury bake-off
Robert X Cringely is a sex symbol, airplane enthusiast and intrepid adventurer. He's the host of the hit PBS-TV series "Electric Money," and author of Accidental Empires: How the Boys of Silicon Valley Make Their Millions, Battle Foreign Competition, and Still Can't Get a Date, and the popular column I, Cringely. I spoke with Cringely about the Web 2.0 "Pillsbury Bake-off," the ruthlessness of Microsoft, user-generated video ("the CB radio of our decade") and his own unemployability.
Davidson: Which software company would you hate to compete against? What makes you single them out?
Cringely: Microsoft of course. They have the deepest of pockets, unlimited ambition, and they are willing to lose money for years and years just to make sure that you don't make any money, either. And they are mean, REALLY mean.
Davidson: Why do you think Microsoft is mean? Are you implying some kind of malicious intent rather than just ruthlessness?
Cringely: Maybe "mean" is the wrong word to use for Microsoft. "Ruthless" is good. The company is built in the image of Bill Gates and Bill is a guy who gets caught-up in the game of business and doesn't typically see its personal cost. To use what might seem to be an obscure example, just look at all the various partnerships and industry consortia that Microsoft has announced through the years that never produced a product or even a usable specification. There have been literally dozens of these operations that are intended solely to freeze the competition until Microsoft can figure what the heck it actually wants to do. To Microsoft its a PR exercise that helps them compete but to customers it is just a damned lie. That's ruthless. There are plenty of other examples I can give but you get the point. I represent the concerns of users, not vendors, and Microsoft doesn't really care about users.
Davidson: Do you think they're meaner than, say, Google or Apple?
Cringely: In comparison Google and Apple are, well, different. Apple cares very much about users as a group, but very little about users as individuals. If you disagree with Steve Jobs you are just plain wrong, but as a class users deserve cool stuff (as opposed to Microsoft's view that users are simply a burden). Google can't get past the algorithm. To Google there are no users, just electrons with intent. Google would rather users not exist, but unlike Microsoft I don't believe Google actually resents users, at least not yet.
Davidson: Who do you reckon will dominate in five years' time: Google, Microsoft or somebody else (possibly unknown now)?
Cringely: Five years from now Microsoft will still be a cash machine but Google will be the big bully and by that time they'll probably be in competition with Apple, too.
Davidson: What companies or technologies do you think are full of hot air and why?
Cringely: Web 2.0 is cruising for a bruising not because it isn't clever or even a whole new market segment, but because too few companies realize that Web 2.0 is more like the Pillsbury Bake-Off than it is a business. This requires some explaining because I know it will piss off some folks, but the problem with Web 2.0 BUSINESSES is that they are usually built on a recipe for mixing together technologies from other companies. That's fine, but then how do you protect your recipe? You can do it through market share, but that just means you have to be the first or second serious player in your category, which is why we have MySpace, Facebook, and..... almost nobody. Yet every VC in Silicon Valley will hear this week a pitch for some startup that calls itself "better than Facebook." Sheesh.
Davidson: What is the most over-hyped bit of tech at the moment?
Cringely: User-generated video is the CB radio of our decade and, like the CB craze of the 1970s, this one will decline. I'm not at all saying there is no value to user-generated video, but it won't replace television.
Davidson: Are we in a tech bubble?
Cringely: No. Tech companies today have to make a profit and the IPO market stinks. There have been some lucrative buy-outs and a couple big IPOs, but this is nothing like what we saw in the late 1990s. Not even close. If you are making a profit, it isn't a bubble.
Davidson: What is the single biggest mistake (or mistakes) that software companies consistently make?
Cringely: They assume that having the best technology is good enough. Yes, technology is important, but not as important as good marketing, proper product positioning, and simply making sure that more money comes in than goes out. Silly stuff like that.
Davidson: Who do you consider the best leader of a software company you have seen and why?
Cringely: It varies from era to era and no leader is perfect, but my favorite software company for pure execution is Intuit, which consistently invents new businesses that attract huge competitors yet the company still maintains dominant market share.
Davidson: If you could have worked for any company in the last 20 years which would it be and why?
Cringely: I am unemployable, having been fired by every company I have ever worked for. I am not making this up. But if I were employable, I suppose I would want to be at some software company that would have made me rich while not making me bored. In the 1990s I would have wanted to work for Netscape, which made a lot of people rich when AOL overpaid for the company. In this decade I suppose I would have picked YouTube simply because it grew so fast and sold out for a good price in a heady rush. Nothing like zero to $1.6 billion in 15 months. I'm not sure we'll see that again.
I haven't yet persuaded Robert X. Cringely to speak at Business of Software 2007. If you think I should, then post your comment here.
For more information about the conference, and to get your free e-book, go to Business of Software
Ted Nelson is a computing genius. Although he denies it, he predicted and described the world wide web in the 1960s. He's the guy who coined the term "hypertext" in 1963.
Incidentally, he's also responsible for possibly the world's longest software gestation. Project Xanadu was started in 1960. A prototype was demonstrated in 1972 and something finally shipped earlier on this year.
In his 1974 book Dream Machines there's a section called "How to Learn Anything". It's an interesting read [all typos are my own]:
A lot of people are afraid to ask questions because they're afraid of looking dumb. But the dumb thing is not asking questions.
If I ever get my school, the one course will be
HOW TO LEARN ANYTHING
As far as I can tell, these are the techniques used by bright people who want to learn something other than by taking courses in it. It's the way PhD's pick up a second field; it's the way journalists and "geniuses" operate; it brings the general understandings of a field that children of eminent people in that field get as a birthright; it's the way anybody can learn anything, if he has the nerve.
1. DECIDE WHAT YOU WANT TO LEARN. But you can't know *exactly*, because of course you don't know exactly how any field is structured until you know all about it.
2. READ EVERYTHING YOU CAN ON IT, especially what you enjoy, since that way you can read more of it and faster.
3. GRAB FOR INSIGHTS. Regardless of points others are trying to make, when you recognize an insight that has meaning for you, make it your own. It may have to do with the shape of molecules, or the personality of a specific emperor, or the quirks of a Great Man in the Field. Its importance is not how central it is, but how clear and interesting and memorable to you. REMEMBER IT. Then go for another.
4. TIE INSIGHTS TOGETHER. Soon you will have your *own* string of insights in a field, like the string of lights around a Christmas tree.
5. CONCENTRATE ON MAGAZINES, NOT BOOKS. Magazines have far more insights per inch of text, and can be read much faster. But when a book really speaks to you, lavish attention on it.
6. FIND TOUR OWN SPECIAL TOPICS AND PURSUE THEM.
7. GO TO CONVENTIONS. For some reason, conventions are a splendid concentrated way to learn things; talking to people helps. Don't think you have to be anybody special to go to a convention; just plunk down your money. But you have to have a handle. Calling yourself a Consultant is good; "Student" is perfectly honorable.
8. "FIND YOUR MAN." Somewhere in the world is someone who will answer your questions extraordinarily well. If you find him, dog him. He may be a janitor or a teenage kid; no matter. Follow him with your begging bowl, if that's what he wants, or take him to expensive restaurants, or whatever.
9. KEEP IMPROVING YOUR QUESTIONS. Probably in your head there are questions that don't seem to line up what you're hearing. Don't assume that you don't understand; keep adjusting the questions till you can get an answer that relates to what you wanted.
10. YOUR FIELD IS BOUNDED WHERE YOU WANT IT TO BE. Just because others group and stereotype things in conventional ways does not mean they are necessarily right. Intellectual subjects are connected every whichway; your field is what you think it is. (Again, this is one of the things that will give you insights and keep you motivated; but it will get you into trouble if you try to go for degrees.)
***
There are limitations. This doesn't give you lab experience, and you will continually have to be making up for gaps. But for alertness and the ability to use his mind, give me the man who's learned this way, rather than been blinkered and clichéd to death within the educational system.
BIBLIOGRAPHY
Wilmar Shiras, Children of the Atom. Science-Fiction about what a school could be like where kids *really* used their minds. I've always been sure it was possible; the R.E.S.I.S.T.O.R.S. (see p. 47) made me surer.
(c) Ted Nelson, 1974
It's a fantastic book. You should do whatever you can to get hold of a copy.
Tim Lister is a principal of Atlantic Systems Guild and co-author of the acclaimed books, Waltzing with Bears: Managing Software Project Risk, and Peopleware: Productive Projects and Teams. Freelance writer Bob Cramblitt spoke with Lister about the patterns that help determine software success or failure.
Cramblitt: You’re working on a new book. Tell us a bit about it.
Lister: Actually I’m working on it with the five other principals of Atlantic Systems Guild. The book is tentatively titled Project Patterns: From Adrenalin Junkies to Template Zombies. It’s a series of nearly 100 short essays on project patterns for good and bad. If they are healthy patterns, we talk about how you promote them within the organization. If they are dysfunctional, we tell you how to stop them.
Cramblitt: Let’s start with the bad ones. What are three things that project teams can do to ensure failure?
Lister: One is what we call Brownian Motion, based on the physics term defining a constant state of random motion. In our definition, it means loading a project with people when you still haven’t decided what you are actually going to do.
Another pattern is project sluts – organizations that can’t say no; they have far too many projects for the staff they have, and things never get done.
We also see a pattern called dead fish. This is a project that is doomed from the start because the schedule is outrageously unrealistic.
Cramblitt: What are some good patterns?
Lister: One of the best is strawman: building low-fidelity, low-cost prototypes for projects even if you know the approach isn’t right. Strawman projects help you discover where you are going wrong early on, and help people articulate what they want and need.
Seasons for change is a pattern that specifies that changes will be made only during certain windows of time, avoiding problems with constant tinkering.
Testing before testing describes a pattern of doing quality assurance in parallel with the project, instead of waiting until the end. This helps everyone understand project specs in the early stages of the project.
Safety valve is another one. People involved in projects need to blow off steam and do goofy things like jump up in the afternoon and go bowling. There are other things related to this that are important too, such as sharing food, and setting up opportunities for people to get to know one another outside of work. My dad once said, “it’s hard to hate someone when you know their name.”
Cramblitt: Has project management improved in the 20 years since you co-wrote Peopleware?
Lister: Yes, I think so. More organizations realize that system building is a deeply human activity, rather than a high-tech, engineering effort. People are looking at bigger projects differently. Good organizations aren’t doing things in a ballistic way – where you aim a gun, shoot, and the bullet goes where it goes. The new approach is more like driving a car: you drive a bit and if don’t like where you are going, you steer in a different direction.
I think the agile development people have a tiger by the tail. They don’t attempt to build a perfect spec that will be the bible for the next two years of development. Instead, the approach is to spec a little, build a little, run it by people and get some feedback, then get another cycle going, continuing until you hit the moving target. Larger projects are much riskier, and agile development is a way to minimize risk.
Cramblitt: What do you think about the reliance on best practices?
Lister: I get chills when I hear that phrase. From my point of view there are some pretty good practices, but no best practices because that implies generic software development. All projects are related to the domain they’re in. A best practice for defibrillator software is not the best practice in another domain. I’d like people to think about patterns – abstracting their work and recognizing the patterns they’re in, good and bad, and making informed decisions to promote those patterns or replace them.
Tim Lister will talk about project sluts, strawmen, safety valves and much more at the Business of Software 2007 conference, October 29-30 in San Jose.
Sign up for the newsletter and get a free eBook of 'Eric Sink on the Business of Software'
He’ll be joined by other software luminaries such as Guy Kawasaki, Joel Spolsky, Rick Chapman, and Jeffrey Pfeffer. For more information and to register, visit www.businessofsoftware.org.
What do ringtone sellers, third party shareware sites and print magazines have in common? They're all struggling for survival, and will stop at nothing to extend their miserable existence by even a fraction.
Monstermob makes its money by selling mobile phone ring tones. Actually, it makes its money by scamming unsuspecting customers. You buy a single ringtone but they sign you up for a monthly subscription. It works on the principle of stealing a little money from a lot of slightly lazy people with better things to do than claw back a few dollars. If you need to trick your customers like this you’re doomed. It’s either dishonest, or a last-minute stand against a broken business model. Indeed, Monstermob’s shares are currently at a record low.
Surely we’d never see a similar thing in the software industry? Think again. A lot of software companies sell their software through third parties such as Digital River. One of these third party ecommerce providers - SWREG - has started asking an additional question at the end of each sale. If you’re not careful you will be signed up to a $10 / month subscription. This is simply dishonest, and clearly not in anybody’s long term interests. It’s a sign that their business model can no longer work. With the arrival of Google checkout, software vendors can now sell their software themselves for a commission rate of effectively 0%. There is no room for third parties such as SWREG. The only way they can survive at all in the short term is to stiff their customers and partners.
I had a similar experience with SQL Server Magazine. I couldn’t find my paper subscription so I downloaded an article from their web site. I was led to understand that I was buying a single article but was signed up for a monthly subscription. This behaviour is so obviously wrong that it can only be a desperate attempt to stave off the inevitable death that most print magazines face.
Although I can’t condone this behaviour, I can understand it. Good people can do bad things. If your company were doomed, what would you do?
The odds are you use psychometric tests as part of your interview process. If you don’t, experts probably tell you should. Unless you understand their assumptions and interpret them correctly then you will hire the wrong people. Here’s why.
Firstly, these tests are based on a number of assumptions. They assume that, fundamentally, we have a constant personality that predicts our behaviour. This might seem like common sense, but it’s good to question common sense occasionally. Do you behave the same when relaxed and when stressed? With strangers and with friends? At work and at home?
The tests also assume that we can all be measured against similar yardsticks. We are all points on a graph with axes corresponding to universal traits.
You need to be aware of how these tests are developed. Researchers ask thousands of people to describe themselves and others and then examine the words used. Assuming that words used correspond to characteristics, they search for correlations between words. If there are strong correlations, then one of the characteristics is eliminated. For example, if ‘nice’ always corresponds to ‘kind’ then both describe the same trait so one is discarded. Of course, there’s never a perfect correlation so information is lost.
Personality is therefore distilled down to between 3 and 16 factors. This results in gross over-simplification. Carry out the same exercise with physical characteristics and you’d end up describing everybody in terms of their height, weight and skin colour. That broad-brush picture would give us some information, but it’s the lost nuances that are important.
You should note that these tests ignore some realms of personality. Some of the missing traits are important to the workplace, some are not. These tests rarely tell you if you’re an asshole, evil, gullible, attractive, intelligent, a hard worker, promiscuous or religious. Although not always relevant to work, these are important characteristics that people have.
The last point is more subtle and is about interpreting the results. When evaluating the world, our minds use certain heuristics and take certain short cuts. Normally, this works well but it can introduce some biases. One of the most notorious is that we tend to ignore base rates. For example, say a doctor tests you for a disease. The disease affects one person in ten thousand and the test is 99% accurate. You test positive. How likely is it that you have the disease? Take a moment to think about it.
Most people – and this includes doctors and statisticians – will say 99%. This is the wrong answer. This becomes clear if you think in concrete rather than abstract terms. Take ten thousand people. One person is ill. The test is 99% accurate so 1% of people will have false positives. That means that 100 people test positive. Since only one of those people is actually ill, the odds of you being ill given a positive test result is actually only 1% (ignoring the negligible effect of false negatives). The important point here is that the base rate is important. The underlying incidence of the disease massively affects the probabilities.
Your mind can take you down a similar blind alley with psychometric test results. You can see this from the diagram below. Say you’re trying to hire a developer. Obviously you want to hire an outstanding one, and they are rare (the small circle). You’ve noticed that the outstanding developers you know almost all share similar traits (the red area) so you decide to test for that trait.
You find a developer with the required trait. This group is represented by the large circle. However, because many average developers may also have the required trait, you are much more likely to have picked somebody in the blue area (average developer with the trait) than the red area (outstanding with the trait). You have been misled by the base rate – the number of developers in the general developer population with the trait. In other words, if you hire on the basis of the test results, you will likely hire the wrong person.
In conclusion, by all means use psychometric tests but understand their assumptions and interpret them carefully.
We're giving away a free, signed copy of Rick Chapman's In Search of Stupidity: Over Twenty Years of High Tech Marketing Disasters to the first 100 people who register for the Business of Software 2007 conference. This an awesome book, with 44 five star reviews on amazon.
Rick is speaking at the conference. He's definitely worth listening to.
You can find out more at http://www.businessofsoftware.org/register.html
Joint CEO of Red Gate Software and organiser of the Business of Software conference. Read More.
Boston
October 4th-6th
Subscribe to the RSS feed
