A prospective employer invited me to do a HackerRank test. Here's my proposed alternative.

no-hackerrank.md

Well, that was unexpected. In the following, I’m trying to follow Jon Evans’ advice from “The Terrible Technical Interview”.

---

To: [email protected]
From: Ahmed Fasih
Subject: Re: Programming Test Invitation

Hi there! Thanks for offering to let me take a HackerRank test for ABC, I appreciate the vote of confidence.

I'd never heard of HackerRank, but after you wrote two other employers sent me their own HackerRank tests. Having worked on those tests first (I considered them practice, for the real thing with ABC :), I'd like to check if you have flexibility in finding an alternative way to evaluate my basic coding chops.

This is because, as functional programmer and author Paul Chiusano says, "Programming is all about managing complexity" [1], but HackerRank is quite bad at measuring my ability to manage complexity. It asks for small algorithmic coding puzzles to be done in unnatural conditions including (1) time limits, (2) forbidding research on Wikipedia or StackOverflow, (3) forbidding collaboration, and (4) forbidding the use of libraries (Python and JavaScript e.g. are so different when confined to their vanilla languages without Numpy/Pandas or lodash/npm packages).

I'm hoping ABC's recruitment policy is flexible enough to let me offer alternative, or at least parallel, routes to quantifying my skill in coding—skills in managing complexity, selecting libraries, and extending existing code, not just solving algorithmic puzzles. I have written a number of open-source projects that I'd love to spend two hours adding features and squashing bugs:

• Ebisu is a Python (and JavaScript) library I wrote to implement a Bayesian estimation problem (for scheduling quizzes in spaced-repetition software): I have a detailed writeup on what it does at https://fasiha.github.io/ebisu/ and the source code lives at https://github.com/fasiha/ebisu

• Mudder.js is a JavaScript library I wrote that implements simple arithmetic in arbitrarily-high-base numbering systems to lexicographically-subdivide string space, which is useful in NoSQL databases for use as keys: a detailed writeup is included in the repo at https://github.com/fasiha/mudderjs

• The Texture Shaded Globe lets you interactively visualize the world's elevation and terrain after it's been texture-shaded, which is a fractional-Laplacian frequency-domain operation that I wrote in Python to work on ~100 gigabytes of data: the app is at https://fasiha.github.io/texshade-cesium-viewer/ and a description of it, with links to specific views, is at https://fasiha.github.io/post/texshade/

• KanjiBreak is a webapp I wrote in JavaScript and Elm to collaborate with my friends and family who were also learning Chinese/Japanese characters. We are using it to make a character dependency graph: the app is at https://kanjibreak.glitch.me and includes a detailed "Help" section (it may take a few seconds to load the first time, since Glitch is a free resource that spins down inactive servers).

These are just four projects I picked as I scrolled through my list of recently-committed GitHub projects [2] that I think showcase not just skill in programming but also in math and design.

Would ABC be willing to work with me to define a better way to check my technical qualifications by choosing one of these projects (or any other project of mine!) to perhaps conduct a code review and/or add an enhancement that you would like to see? I think that would be a much more interesting and useful way to spend two hours, rather than implementing cute little algorithms inside an isolated environment like HackerRank.

I'm hoping that, if the HackerRank test turns out to be an absolute requirement for ABC, that we can do something like the above in addition to HackerRank.

Thanks for considering it! I'd love to work for ABC, and I hope I'm not being too forward or presumptuous by sending you this long email.

Best regards,

Ahmed

[1] See http://pchiusano.github.io/2017-01-20/why-not-haskell.html
[2] At https://github.com/fasiha?tab=repositories

Let me bring an employers view into the discussion.

It is correct that this is an unnatural step testing basic things not often used in practice. They are meant to be simple and easy, but are also often more academic than practical. The interview checking all the things mentioned in this post - architectural approach, dealing with complexity, etc - usually comes as a next step.

Online coding tests, however, simply set a minimum bar for entrance. Although the bar is fairly low, this filters out an unexpectedly large amount of candidates at a early stage. The purpose for the company is simply an early low-effort filtering step to save time.

I do agree they can be frustrating, having recently gone through multiple interviews myself. However, having been on the interviewer side, we have often spent multiple hours on candidates who seem perfect in a face to face interview and on paper, but then completely fall apart when faced with a simple coding challenge. After introducing a simple Codility test, our pool of candidates is of much higher quality, and we can avoid spending days on candidates who claim to have 3 years of Java experience but can't implement FizzBuzz. (A quick note: We could still hire a candidate like this, if we see otherwise potential and are looking for a junior position with no required work experience. I assume this is for a position requiring some work life experience.)

The only downside to HackerRank tests is some candidates may not be quick thinkers, but are otherwise excellent engineers, and can randomly fail these tests due to the pressure of the time limit. This is unfortunate, but from a hiring standpoint, I would rather risk to miss hiring an excellent engineer, than risk to hire a bad coworker.

A word of note on time limits: These are actually advantageous from a diversity perspective - people who have children and other responsibilities can have problems dedicating more than an hour time slot. Therefore restricting the time available makes the playing field even, regardless of if you are a parent, or young free and freshly graduated.

What does this mean for you as a candidate? Assuming you are competent, it means having to dedicate an extra hour or two per application for a trivial task. It is something I would prefer not to, but it is better than not getting an interview at all because some other person with no skills knew how to flair up his CV and bullshit his way through an interview.

Oh and a final word of warning: If a company gives you a online coding test, and then checks your skill in no other way, you may want to reconsider...

I'll second what @svetob wrote above: online, time-limited tests are typically used as a sieve, nothing more. You don't have to do great, you have to do something. This way whoever is recruiting can see if there's at least a kernel of truth in what your CV or GitHub portfolio claim. Pass on the offer and don't bother with this company only if there's no additional technical interview. But that's unlikely. I may be a bad person but I'd roll my eyes if I got this email. You may be great but that (ultimately) should be reflected in your role and pay. Being great doesn't necessarily mean that you should be treat differently throughout a recruitment process. In fact I've seen people being treated preferentially during the recruitment who later turned out to be mediocre at best (I guess I'd consider myself at some point in this category).

These tests are really good indicators for whether or not some other programmer who already works there will get annoyed with your style, spacing, and variable name selection choices, but not much else.

I would rather risk to miss hiring an excellent engineer, than risk to hire a bad coworker.

Agreed, since a bad hire means you not only incurred expense for that hire, but also missed the excellent engineer. But canned tests like this are a weak filter at best, and are going to probably select for other attributes that you may not value or possibly find negative, like being highly skilled at faking it.

I've had pretty good success at picking one or two really difficult problems I've encountered and presenting them to candidates. They rarely solve these problems on the spot, but I have them talk through their thought process. And sometimes they come back post-interview with a solution on their own initiative. This provides me direct evidence of characteristics I value in a hire.

Solve this novel algorithm and data structure problem in a Google Docs page live while on the phone is how Google does early round screening. Your body of work or ability to write prose is likely immaterial to the astrophysics Ph.D who works on the gmail team on the other end of the phone line.

But canned tests like this are a weak filter at best, and are going to probably select for other attributes that you may not value or possibly find negative, like being highly skilled at faking it.

They are weak indeed, but even this weak test can filter out some 50-90% of applicants depending on company/position. If you can 'fake' your way through a coding interview and then an onsite coding challenge - great! If you can also 'fake' your way through providing value to the company then when can you start? :)

Having recently spoken to a few companies that sent over one of these tests I can attest to the massive waste of time they are, specially since after I finished the test for each they scheduled an on site or requested times for an on site and then all three ended up hiring someone before they interviewed me.

As someone that has a job, side projects and a life that contains time consuming hobbies. After coding all week if I'm spending time on these academic tests that prove nothing. I would much rather give them my GitHub than spend time doing hacker rank tests, I have since then refused any take home tests since the companies have no respect for my time, I don't see a reason to respect theirs.

Sure it is a bar, but that is only useful when u are dealing with fresh grads, after like 10years of working in the industry they hold little value since we may not remember every algorithm out there but we can code it up once we know of it.

Expecting someone to know how to do matrix multiplication using the strassen algorithm without looking it up is kinda silly since, 1) we are engineers not mathematicians. 2) we can look up when that algorithm wants us to do and implement it without needing to copy paste code. Point being, take home tests will not get u anywhere as an employer or prospective candidate.

Honestly, if someone used documentation over stackoverflow they get bonus points with me.

From my experience the best way to test a candidate is to give a home coding challenge and then work with him/her to extend it/explain it during a pair programming session. Solving algorithms may be a suitable task for a junior developer or a graduate, but not for someone with 7-8 years of experience. I just don't think that your whole career can be judged on the basis of 20 lines of code (or however long the algorithm is). From what I have experienced very few companies are actually willing to proceed with you if you don't actually solve the problem, nevermind how well you approached it.

I’d say take-homes can be argued to serve two purposes:

• Filter our the very vast majority of applicants who pretend to have solid experience building software but actually don’t, and tend to happen at all positions. (I recently phone-screened an architect / lead eng guy who kicked butt on the phone so we brought him in directly, but it ended up showing during on-sites that he never actually built anything himself.) As a matter of respect, I agree that those take-homes must be very short (they still filter a lot of people out that way anyway), and you want to move people forward if they fulfilled the requirements, regardless of style or whatever biased metrics (you’ll talk about those concerns when you meet them, assume they have good reasons to have written that code that way). Also, they’re take-homes and not live-coding in order to provide a stress-free environment at the entrance of the funnel.
• Another purpose they serve is to provide the exact same process to all candidates who walk in, in an attempt to unbiased the process as much as possible, and provide equal opportunity as much as possible. I realize it’s hard to see when at the receiving end of the take-home, but it’s likely this is one of the priorities at the giving end of it; and therefore saying “I’m too good for this” can come across as “I’m too good for equal opportunity practices”. I’d even say that it can be used by some employers to check for inclusive behavior as well, or at least basic knowledge of D&I practices.

So, really, it’s true that when experienced, a given engineer is not really doing it for themselves; rather, they’re helping out the employer (spending an hour or less so the employer won’t have to spend dozens and dozens of hours on coding sessions with people trying to fraud their way in), and they’re helping the community (solving a simple challenge so that anyone who can solve that simple challenge gets an equal shot at interviewing face-to-face, regardless of CV or GitHub account).

Do it with kindness in mind. ;)

People spends countless hour to argue whether good at running proving that you are a good football player? The answer is no, but it is an indicator that's you have good speed and strength and stuff and it does relate somehow. That's why to test a football player, you need to have multiple type of tests: running, passing ... and yes, playing in a game etc.

Similarly, if you try to say a simple screening test should test all of your abilities and should say whether you are a good software engineer? We are all engineers, and we know that it's a hard problem to measure a good engineer, and to solve a hard problem, you need to break the problem down into different components and test it in multiple rounds, that's why we have screening, phone and many onsite rounds, each will have its own purpose. So if you try to come from the angle that this silly test is not telling that you are a good engineer, yeah, you are right, so? It is just one part in a big system, please, be an engineer, not a complainer! Thank you.

I don't see anything wrong with the coding test approach. It sets a minimum bar for prospective employees. Especially when they also take a look at your syntax and code after you pass the test to determine if you're worthy of the second round (the way you write code reveals a lot about your abilities as a programmer).

It's nice and courageous and all to send this to the HR contact, but if I were HR (I'm not), I wouldn't care that much. When all is said and done, if I'm a representative of a prestigious company, I probably have 10,000 other applicants for the position who are at least as qualified as you are, and who have managed to pass the test with flying colors.

Key assumption here is "HR of a prestigious company" assumption. Really wouldn't give a flying f*** in that case.

I have been programming since 1980, and have a PhD in computer science, a long resume working at some very fine institutions.
Capital One just wanted to administer this test to me.
Can you imaging going to a surgeon and asking them if they knew the heart was on the left?
Asking your plumber if they knew what a wrench was?
How can you be expected to be treated as a professional if you respond to such insulting questions?
Just walk away from places that have no clue and administer this test.

As a programmer working with the greatest teammates, I believe only very few of them, if any at all, can answer moderate level of algorithm tests on Hackerrank website. But I would work with them anytime or hire them in a heartbeat if I were an employer.

They have been dealing with higher level problems on daily basis rather than writing code to find out how many palindromes with maximum 4-letter vowels are there from a given string. The problems for them usually involve figuring out how to finish a task that comes with unrealistic time frame, implementing a ground breaking feature that can fit in with the legacy code in backward-compatibility manner, to find a bug in time-attack fashion from the production code written by someone else who already left the company, to convince the product owner that his "beautiful UI" can be improved, to communicate with other teams located across the globe, juggling between Java, C#, JavaScript, and Kotlin codes from multiple projects/micro services and confused with the syntax and data types, trying to be patient with scrum processes, meetings, and corporate processes, being lost in thousands of JIRA tasks and confluence pages, etc...

The simplistic idea of a good programmer being able to come up with a good algorithm in few minutes without looking up any reference doesn't fit with the current enterprise software industry. Most of the enterprise application programmers have never had to write their own sorting algorithm after they have left the school. If they had to take these tests, they have to invest not just 1 or 2 hours to take the test, but there also is a amount of time needed to refresh their memory about what they learned many years ago. Besides, not all of them are singles, they have families, and their own commitments. Time is a extremely valuable commodity.

Having said that, these tests are useful if the employer is looking for someone with excellent algorithms skills and the job requires writing brilliant ones, or the employer is looking for perfect or near-perfect programmers who are good with both algorithm and real life problems.

In any case, It would be great to have hiring manager or an senior employee from the hiring team taken the test to see if it really fits the position they are hiring for, instead of letting HR make a decision whether to use those tests. For me personally, I would prefer to talk to someone about solving real world problems with enough technical questions- those not from textbook - sprinkled on.

Programming tests are not the perfect approach, just best effort which is good to set a minimum bar and that bar needs to be set by your average team member... Give the same test internally in your company and then expect the new candidate to be slightly better than average results so that your team improves over time. Don't hire a p***y or it will make your skilled people quit at the speed of light. Skilled engineers like to be around skilled engineers, if they perceive the environment to be "low standard", they will quit.... This is not to make HR life easier, but to raise the standard of your team, you get 100 candidates which are better than the average you've got already, then you screen all of them. HR laziness through filtering will not help to find the gems. If you want to find the best gems, you will need to dig a lot of dirt, one way or another, you must put a lot of work. Make your own tests, don't rely on external sources, candidates can easily prepare on those. It has to be a problem made by the best team member which will be the only one not trialing the test, and it must be unique. Then expect above average results given the other team members results.

I could not agree more

Hackerrank has med it worse, because its no longer pseudocode, it has to run. So now it's not 45 minutes of sketching out a solution, its 45 minutes of sketching it out and getting it to run.

The last good interview I had, they gave me a programming task, not a huge one, but a small task. I wrote and deployed it in the cloud so I didn't have to send along build instructions.

I have read this thread and see all the comments in here (until now) and I agree with @scottralph,
And in my opinion : to get/hire a best programmer can not determined from hackerranktest, that is just showing how weak the HR to look and find best candidates to fill the programmer position in that company. Many ways can be done to filter best candidates to fill that position and they (HR) only use the easiest way to get the best result