Comment Re:Well, I wish him well (Score 1) 26
Ditto! The irony of not researching the viability of your software platform, which is designed to assess the viability of other software platforms, is a little too much.
I generally exercise some degree of distrust towards computer manufacturer recommendations when my product is no longer under warranty and their legal team likely has them relatively well protected against your situation, but I'd definitely name names. Send a note to the Consumerist, find a few execs and contact them directly. It may be legal, but it's a dishonest approach for those companies to take. It doesn't cost you much time and energy to bring unwanted attention to the companies and that attention is sometimes enough to suddenly get your components replaced. It won't cause systematic change, but at least you're better off.
Not one to miss an opportunity for a car analogy: if a critical recall fix bricked your ride, I think most everyone would agree it is the manufacturer's responsibility to make things right even if the vehicle is out of warranty. Of course, there's obviously more regulation involved and a more direct correlation to physical safety in the case of cars (i.e., you are putting yourself at risk of bodily harm if you choose to disregard the recall fix).
What are they talking about? Drones aren't murderous, they're just full of angst!
I signed up for some term life insurance a few years back and the contract explicitly permitted suicide. That surprised me but I confirmed it with the agent. I image this doesn't apply to everyone (maybe it's not allowed in whole life plans), but I have a standard plan from a major carrier.
Unfortunately my wife also read the fine print so now I have to watch my back...
This may be overly cynical, but I think it's the lesser of two evils. Historically, we've done an awful job of matching people to jobs. It's depressing to learn what really gets people hired and fired. As an I/O psychologist, I cringe whenever an under qualified applicant gets a job just because they know somebody. So as a field, in general, our goal is to make the process more scientific. As a field I/O is generally pretty positive, but it can definitely be used for evil. The strongest force stopping those folks are probably the equal opportunity laws and regulations.
If you want creepy though, the screening piece is just the beginning. Someday careers could be like your Netflix queue. Based on your past jobs, your performance at those jobs, your ratings of those jobs, your knowledge, skills, aptitudes, and interests, and other factors, your career queue could literally suggest new jobs for you and even line up start dates. That's terrifying for many of us, but for others that peace of mind would be a godsend. If it was mandated it's definitely Brave New World. But if it's voluntary? Then it's not so bad. You don't have to watch the movies Netflix suggests... But how would you ever break out and try something new? I think a poor implementation would be linear, single-career (which we know isn't realistic). A smart implementation which could actually enrich a lot of lives, would encourage career jumps every so many years to help keep people engaged and motivated.
Sure, at the end of the day the employer pays our bills. That said, the highly inquisitive employee may not do so well in a customer service roll, but they can often make a great technical support agent. When an employer is trying to fill three positions and we can tell them which one an applicant is most likely to succeed in, we're helping the employer but also indirectly helping the employee. Many applicants for these types of jobs are taking the shotgun approach and apply to everything they find. They just want any job, so if we can match them with one where're they'll tend to do better, it's a least a small improvement over the old way of doing things.
What worries me more is somebody who scores poorly across our entire assessment. Somebody who you might say "does not play well with others". Personality isn't quite as static as pop psychology would have us believe, but that doesn't mean it's easy to change. How do we as a society engage with somebody who wants to work (they're applying for a job) but doesn't have a good attitude for it? Are there really that many entry level jobs to go around for the poor scoring applicants?
You raise very valid concerns. Faking is addressed through two main approaches: different scoring patterns for different jobs and adaptive tests (the content you see today may not be there tomorrow). Many of our clients are trying to fill two or three positions at the same time. That could mean customer service, sales, and tech support at a call center, or cashier and back room at a retailer. A lot of skills and aptitudes are important to both positions, but there will be some differences and even different weighting within the same trait. So a question often has multiple "correct" responses, but one is more likely to point towards sales and the other customer service. In these cases, faking doesn't really help an applicant. Adaptive tests also include a concept called "item exposure" which is simply the number of times a question has appeared in a test. Item exposure becomes part of any adaptive testing algorithm to help determine which question to present next - the more times it's been exposed (thus more susceptible to cheating), the less often it will appear.
Absolutely a concern. Evolv (and any assessment vendor worth their salt) checks this to confirm their products don't cause adverse impact on age, ethnicity, or gender. With that in mind, there's data we've looked at but cannot use. For instance the distance from work variable -- yes it's reasonable to think that people who live further away are more likely to quit, but that data is often tied to socioeconomic status, which is often tied to ethnicity. Poorer people typically don't live next to employers. So we're aware of the correlation but we can't use it.
The big data approach works best in hourly positions where there's plenty of feedback for machine learning. When the inputs are fuzzy, it's a less useful solution. When companies do a poor job measuring performance to begin with, no amount of processing power will save them.
You're absolutely correct, I do think this is the best currently available method for hiring hourly workers. The assessment itself is just one component. For instance, call center applicants also do a mini voice audition (clarity, tone, etc), take a typing test, and complete a behavioral descriptive interview. There's always a human element in the process, we're just trying to make sure the interviewers are asking consistent, job-relevant questions.
I completely agree that people can learn to be good at their job! Our assessments don't quiz people on knowledge they'll pick up in training or on the job. We don't pay much attention to resumes because great hires can come from any background, regardless of whether they have a specific skill set, and be a great fit for the role. When we start working with a new client, this is often a paradigm shift. But we do look at some relevant skills. If two people apply for an email support position and one of them has better typing WPM than the other, it only makes sense to hire the better typist even though the slower typist could learn.
At my previous Fortune 50 employer, top level executives absolutely did take tests like these. The analytics weren't as sophisticated as what we do at Evolv, but the same basic concepts applied. In either case though, these are never the single deciding factor for C-suite or front line positions.
I wouldn't worry about that, employment laws are well entrenched. IANAL (though I am an industrial/organizational psychologist at Evolv), but the employment laws are pretty clear when it comes to discriminating against protected classes. We've also known for years that intelligence is the single best predictor of performance across all job types, but as an industry we can't really use it because intelligence tests tend to discriminate. That's why you see so many personality-style tests.
There are a lot of specific questions employers cannot ask (personal, disability, some criminal history) as well as protected classes which cannot be arbitrarily discriminated against. Protected classes include ethnicity/race, gender, and age (people over 40). We're constantly checking our assessments to ensure they do not discriminate against women, any ethnicity, or older applicants.
Things get trickier when you add the notion of job relevance. IF you are using a screening tool that discriminates, it MUST be job relevant. You cannot disproportionately screen out women who can't lift xx lbs over their head from a firefighting job if that's not something a firefighter actually has to do on the job. You CAN disproportionately screen out blind people for the job of fire truck driver because vision is obviously job relevant.
Most employers try to stick with screening tools that are job relevant AND don't discriminate. Rarely do you see an idiot employer who screens based on some illegal, discriminatory criteria. More commonly it's an edge case, where a tool is job relevant but is discriminatory against some protected class (and isn't as obvious as the blind driver). Somebody sues and it gets settled out of court. It's just safer legally to not discriminate at all, job relevancy or no. At Evolv we include job relevant questions (i.e., call simulations for call center applicants; retail simulations for retail applicants) and situational judgment tests AND check to make sure they don't discriminate against a protected group. We could use job relevant questions that do discriminate since it would be legal, but it's just not worth the risk so you typically don't see it done.
I'm an industrial/organizational psychologist at Evolv. I help build assessment content and I work closely with our predictive algorithms. A few clarifications from the WSJ article & responses to
Yes, creativity and empathy are important for some positions, even in call centers! We're not looking for hateful drones who will hang up on you when you call in. In addition to staying longer, our recommended hires perform better as well. That means increases in both customer satisfaction and efficiency (we call it "average handle time"). But it's a curvilinear relationship - somebody who is too inquisitive is going to tend to waste your valuable time (and their employer's) while trying to resolve your issue. There's a balance.
Most test vendors put a test in place and walk away. At Evolv we take all the post-hire data from our clients and continually feed it back into our algorithms. The content, scoring, and weighting adjust over time to be more predictive.
At Evolv, we don't pair obvious responses when we create questions. So no "I like to steal office supplies" vs "I always show up to work on time" questions. Coupled with the continual refresh & validation of the content, there is no "answer key" that will get you a job. One of the neat things about this approach that we've found is that people applying to entry level positions often don't know what they're good at. Either they've bounced around a few jobs or they're just out of high school. So when somebody applies to a call center job that's hiring for both customer service and sales positions, and we can recommend the position for which they're likely to be "fitter, happier, and more productive"... that's kind of cool. Their employer will make more money off a more stable employee, and the employee ends up doing something they will enjoy just a little bit more. I know some folks will see it from the Radiohead point of view, as creepy (and I respect that), but we think it's better than dumping somebody into a position they're not going to enjoy just because they had the right keywords on their resume or they BS'd their way through an interview.
Science & statistics help eliminate some crazy gut-based hiring decisions. Some hiring managers want to ask call center applicants what they'll be doing in 10 years with an expected response of "I'll be working at this call center". But let's be realistic - while some people enjoy them and thrive, call center jobs are typically not where you plan to be in 10 years. We've also found that resume experience for entry level positions is less important than basic skills and attitude. It's easy to look at that and say "duh" but you'd be surprised how many people hiring & screening for these roles want to exclude applicants who don't have prior experience. So we can cut things out of the interview and hiring process that just don't mean anything.
Evolv doesn't just do employment screening. We periodically follow up with people after they're hired. We find out what information wasn't communicated well during the hiring process, get their feedback on how their training is going, their thoughts on their supervisor, that sort of thing. We feed all of this back in to improve the process. In some cases, that means identifying the trainers whose students perform poorly when they start working. Other times it could be flagging a tenured stellar performer whose numbers are starting to dip for a new position to help reinvigorate them. We strive to improve profitability across the workforce, and do so in an employee-friendly way.
Last but not least, we're still expanding through Xerox, so if you've called their customer service and had a bad experience it must not have been one of our hires. Joking aside, agents are people too, and even our top recommendations have a bad day. We're working hard to to make it better though!
Hope that helps! Yes, there definitely are risks with employment testing, but we try to avoid them and build solutions that make everybody's life a little better.
Cheers,
Tim
This. I definitely consider myself on the noob end of the Pi experience spectrum, but it didn't take me long to figure out that all the advice online recommending the use of a solid power supply (> 1A) was *gasp* correct. I started out with a minimum spec power supply when mine first arrived and was experiencing issues. I've since swapped out for beefier power sources and had two Pi's running RaspBMC for a while now, one on ethernet and the other on wifi, with no USB hubs. I don't have a bazillion devices plugged in either, so YMMV.
Sure, the USB wifi dongle was a bitch to get running at first, but for a $35 toy I can velcro to an unused LCD and deliver streaming media throughout the house, I'm happy.
The questions posted are stuff an interview gets anyway, because every job application has a form to fill asking for race, religion, etc. It supposedly is optional, but in reality, if an applicant bins that form, their resume gets binned.
Um, no. My company provides job application assessments and applicant tracking systems - we do collect the information for legal compliance but the interviewer at the client site never sees it. The only way this could really happen would be if you submitted a paper application (who here does that?) and they used that information for hiring, which is illegal. Otherwise, the people doing the hiring & screening do not see that information.
As for the interview questions you listed, I agree they are generally awful.
Understanding is always the understanding of a smaller problem in relation to a bigger problem. -- P.D. Ouspensky
