what-hiring-should-look-like.md

What Hiring Should Look Like

This is definitely not the first time I've written about this topic, but I haven't written formally about it in quite awhile. So I want to revisit why I think technical-position interviewing is so poorly designed, and lay out what I think would be a better process.

I'm just one guy, with a bunch of strong opinions and a bunch of flaws. So take these suggestions with a grain of salt. I'm sure there's a lot of talented, passionate folks with other thoughts, and some are probably a lot more interesting and useful than my own.

But at the same time, I hope you'll set aside the assumptions and status quo of how interviewing is always done. Just because you were hired a certain way, and even if you liked it, doesn't mean that it's a good interview process to repeat.

If you're happy with the way technical interviewing currently works at your company, fine. Just stop, don't read any further. I'm not going to spend any effort trying to convince you otherwise.

The rest of this post will assume that you agree with me, that's the whole current status quo process is broken from the ground up, and that it needs to be completely re-designed. I'm going to offer thoughts on what I think it should look like.

Technical Screen

Off the top, let's call out that every technical-position interview is necessarily going to need several parts to it. You can't have a single 15 minute conversation with any person and know (and them know!) that everything is a perfect fit and hire them on the spot. That's a terrible and unattainable process.

One of those steps is the technical screen. Why does this step exist? Why is it necessary?

The reason for the "screening call" is to weed out, in an early pass with very little expense on either side, those who are definitely not a good fit (technically, especially), perhaps either because they're too inexperienced, or (worse) they are misrepresenting themselves.

I guess it's also possible that screening eliminates those who are over-qualified. But for this post, I'm going to assume that it's a good problem if you find someone over-qualified, because at least you now have a future pool for other positions you may someday need to fill.

Somewhere along the way, the screening call also became more of a way to fully vet technical skills. Let's all admit that while vetting technical skills is important, that's separate from weeding out the lower end that isn't up to the necessary level. Using the same mechanism for both is part of what makes technical interviewing so broken.

Let's also talk about this "lower threshold" for a moment. My assertion is that most companies think they know what the lower threshold is, but they are actually really bad at that.

If your job posting title is "Junior Developer - JS", what exactly is the lowest threshold that you should screen people out who fall below?

You may think this is a trivial concern. Let's just say that they need "at least 12 months experience with JS", but that's not even close to sufficient of a standard. "12 months experience" can be radically different depending on the job, the code base, the team around you, etc.

Basically, I think I've come to the conculsion that most screening is bullshit and should be tossed out.

I think the process I will suggest in a moment will naturally handle the false-positive signals you're so worried about.

Technical Vetting

Vetting a candidate to see what their technical skills are -- problem solving, coding style, algorithmic thinking, etc -- is important, no doubt.

But coding quiz type interviews are the worst possible way to do this vetting. Why? Because the noise-to-signal ratio is so poor that you can't actually get any useful information. Worse, you probably don't even realize all the bias factors muddying the signal, so you make really poor decisions.

Also, ability to code with algorithms should only be a tiny part of what you should be trying to vet a candidate.

Anyone who's ever spent much time as a developer knows that coding is a part of the job, but there's a ton of other stuff that I would argue is every bit as important as the code, probably more. For example: documentation, writing good tests, written communication (in PRs/issues), ability to take constructive feedback on code, and much, much more.

All of those things are far more indicative of how well you'll do as a developer on most teams than whether you're someone who likes to write for (let i = 0; ..) or list.forEach(..).

If you focus on vetting all these things, you'll actually end up vetting their coding skills at the same time (for free!). But if you focus only on vetting their coding skills, you miss out vetting all this other stuff.

One reason people don't focus on this sort of vetting, and instead obsess about vetting coding skills, is because it makes them feel better if they can design gotcha quiz questions that trip people up on lack of some esoteric corner-case knowledge. They substitute that poor signal for "has been around coding long enough".

Another reason they do it is because of the bullshit of "meritocracy", the idea that tech and ideas are the most important thing. This is complete horseshit. Anyone who still doesn't realize that this job is all about people, who thinks this is still about operating computers, has no business being any where near the design or implementation of a hiring process (or managing developers).

Humans over code. I make no apologies for that stance. And if you don't agree, we have nothing else left to talk about. Move along.

So, we have to vet candidates for a variety of technical skills, only one small part of that being coding. Is it really that much of a surprise that we do so poorly in hiring and placing individuals when the process we use for that only focuses on one tiny part of it?

The Ideal Tech Interview

Here's my concept for the ideal tech interview. By the way, I'm going to say throughout "I ..." as if I'm the only one conducting all these steps. But that's not necessary. Other team members can absolutely help with different parts, and we can collect all that feedback together at the end.

First, my strong preference is that a candidate be able to demonstrate what they've done over a period of time, ideally at least 3-6 months. I don't need a picture of you over the course of 45 minutes or even 4 hours. That's not even close to a large enough time sample.

One great way this can be done is through someone's OSS work (on github, etc). That's by far not the only way to get this information -- I'll cover alternatives in a minute -- but it's in my mind a really good option if the person has done so.

What I want then is for them to submit links to 1-3 samples of recent (less than 2 years old) OSS work they've done, with some brief descriptions of what they contributed (especially if others participated, too).

What I'm looking for in my quick review (say, 15-20 minutes max) is:

• Can I see some samples of code they wrote?

• Can I see any documentation? How easy it to understand the project from the docs?

• Are there tests, and what do the tests look like? Are they insightful and well designed, or naive?

• Any open issues filed (by others)? How has this person triaged comments/questions from others?

• Any PRs opened? How did this person handle when someone wanted to change their code? Did they feel threatened? Did they respond cogently and respectfully?

After I've reviewed this material, if I feel I've seen enough that I can ask some questions, I'll want to schedule a call with the person, say for 30-60 minutes. On this call, I want that person to walk me through their submitted links. I'll hopefully have a bunch of questions I can probe with, but I'll want them to convince me that what I'm seeing is a good representation of how that person performed as a developer over the span of some time.

Alternative To OSS Portfolio

If I'm interviewing a person who's well known publicly -- perhaps they've run some high profile OSS projects, or spoken at conferences, or written one or more books, or whatever -- then I would substitute review of some of those materials for at least some of the OSS portfolio submissions.

I'd still probably want to see at least one example to review, but depending on the body of public reputation/track record the person has, and the level of the position (highly specialist) we're hiring for, I wouldn't expect that person to re-prove all their technical chops. That part should be pretty obvious.

Let's say the person has no public resources to demonstrate their developer credentials.

My backup plan, which by the way any candidate could choose to participate in if they didn't want to use their public work, would be this:

1. I would clone a private github repo with a half dozen code files and resources in it, maybe something on the lines of a TodoMVC level of complexity.

2. I'd give access to the candidate, and give them a day or two to take a look at it at their own pace.

3. I'd ask them to send me a list of 1-3 observations or questions they have from looking at the repo.

4. Then I'd tell them that I would be assigning some small tasks to them over the course of the next few days. I'd set the expectation that they're going to spend, at most, 15-30 minutes on each task, and that total they may need to spend 60-90 minutes across all the tasks. I'd give them a window of 3-5 days to complete these, and I would deliberately spread them out.

5. One task might be to file a PR with a change to the code. I'd ask them to review the PR and make any comments they feel are necessary, and to ask any clarifying questions.

6. Another task might be to ask for a newly added feature to have some documentation written. Another might be to write tests for a segment of the code. Another might be to conduct a PR review of their code they added.

7. I might, in the persona of a customer, file a bug report (with very little information!), and ask the candidate to triage this report. I'd want them to politely ask for more information and follow-up to understand the bug report.

8. I'd iterate with the candidate on these kinds of tasks for a few days, never more than 15-30 minutes at a time, and let them tackle at their own pace (quickly or slowly), so as to not burden them too much.

9. Over the course of this interaction, I'd be gathering a bunch of observations about how they behave in these various scenarios. This is super valuable info. My goal is to collect over the course of these 60-90 minutes of interactions the same kind of insight as I could have gained if they had submitted the 1-3 OSS project links.

At the end, I'd schedule a short (30min max) feedback call, for the candidate to provide feedback to me, ask questions, etc, and for me to do the same with them, get any clarifications I may still be curious about.

Next Steps

By the end of either a review session of existing OSS work, or the private re-creation of the whole software lifecycle process in the private repository, I would think I have a very solid idea of their technical skills.

In other words, at this point, they've been fully vetted technically. I've seen everything technical that I need to make my decision.

If they want to keep moving forward, and I like what I've seen, then we move onto the next stage: the in-person (or, in these COVID days, the group online) interview.

I would schedule a half day (at most) with the candidate. Here's how we'd break down that time:

1. (30 min) To start, I'd ask them to attend my team's standup (meeting/call). They don't have to speak, just want them to observe. Then I'd ask them to provide me 5-10 minutes of feedback on what they observed. What could they tell about what we're working on, where our blockers are, any potential communication gaps, etc?

2. (60 min) Next, I'd have them pair with a senior member of the team for about 30-45 minutes. Their goal should be to ask questions and watch as that team member worked on some task. Then I'd ask them to come back and give me the same sort of observation feedback.

3. (60 min) Next, I'd have them attend a team/group meeting. Again, no need to present, just observe. And ask questions if appropriate. Come back and give me any feedback.

4. (60-90 min) Lastly, I'd ask them to teach me something (they would be prepared for this in advance of the interview). I'd want to see how carefully they pay attention to me and the questions I ask as they explain some skill. This could be tech related, or not. Their choice. My objective is to see how well they communicate and share knowledge with others.

   Note: If they can't come up with anything suitable to teach me, I'd ask them (in advance of the interview) to take a quick 30-60min course on something, and then re-teach me what they learned.

At the end, I'd have one final short feedback session where I give them a chance to ask me any final questions, and provide any other observations they made while being embedded with the team for the (half) day.

Final Steps

After the interview, I'd gather with any others who participated or observed the candidate, and collect their feedback. We'd discuss the pros/cons of this candidate.

Here's an important part: whether we decide to move forward or not, I would expect to need to provide some amount of concrete written feedback for them. I'd probably want to be able to provide at least 3-4 paragraphs, like half a page or a medium-length email, amount. I'd include quotes from other team members where appropriate.

My belief is that the candidate deserves this feedback, if they went through this whole interview process. Even if we're going to give them a job offer, I want to make sure they get feedback on the good stuff and on the constructive criticisms.

Take Stock

Let's step back and consider what I'm suggesting:

1. The initial vetting of a candidate can take somewhere from 45 minutes up to 120 minutes. It might be on one single call, or it may be spread over ~ 3-7 days.

2. If the vetting passed, we'd invest up to 4 hours of interaction with me and our team.

3. In other words, the candidate has to be willing to give 5-6 total hours over the whole process. Most companies already require at least that much anyway. But in this way, some of that time is spread out, to make it easier for both the company and the candidate to fit it into their broader life/schedule.

Is this really an impossibly high bar? I don't think so. I think it's at least as much as a company should be willing to spend vetting and hiring a candidate.

And yes, if you're going to hire, you probably need to run ideally 3-10 candidates through some or all of this process, so each job opening might take upwards of 40-50 hours of company labor. But again, that's already happening at most teams, when you add up all the time recruiters, managers, and interviewers spend interacting with the candidate.

So I'm really not expanding the scope of interviewing burden, but re-purposing that time to be more well spent, focusing on signals rather than noise.

Wrapping Up

There's a lot more details in between the lines of all these suggested ideas. And it's just a rough sketch, not a precise formula.

I suspect some readers will reject everything here entirely. That's fine. But I hope a few more of you take a step back and re-consider if your processes are actually asking the right questions of candidates.

Wouldn't you rather know how they're going to do when you file a bug against their code, than spending time checking to see what style of for loop/iteration they prefer?

Thanks for sharing.

(30 min) To start, I'd ask them to attend my team's standup (meeting/call). They don't have to speak, just want them to observe. Then I'd ask them to provide me 5-10 minutes of feedback on what they observed. What could they tell about what we're working on, where our blockers are, any potential communication gaps, etc?

A lot of the companies do not like the idea of having an external presence in internal processes where they can get to know more than what the company would like them to. Any alternate techniques?

@pushkar100

A lot of the companies do not like the idea of having an external presence in internal processes where they can get to know more than what the company would like them to.

I don't think this is a healthy mindset to have, because it means that you have to treat every candidate at arm's length, which means that you never get to judge the candidate in the real context of your workplace. How could you possibly tell if they'll be a good fit if you insist on keeping them from the situations they need to fit in?

Moreover, candidates can't possibly judge if they want to work there if you create a fake/arms-length environment to interview them in. They need and want to see what the job environment/situations would look like, so that they can make an accurate assessment.

Remember, you shouldn't be at this step with a candidate unless you're pretty sure there's a strong chance of a fit. That should limited the number of people filing through a bit. It's also quite common (IME) for companies to ask candidates to sign limited NDAs during interviewing so as to protect against exfiltration of any sensitive info.

Any alternate techniques?

The only thing I can think of is creating a "fake meeting" where you pretend to be doing these things, but make sure you're not actually talking about real details. I think that's a bad idea, but it's probably better than not doing anything to create the scenarios you expect you and the hire to be able match and fit in.

@getify Thank you for the explanation

This is great. @getify Say if you are hiring a junior developer (most likely for their first job), what would you change?

They might have an active Github profile of personal(individual contributor) projects but not necessarily contributed to any popular OSS. In that case, they won't have PRs/comments raised because they might be pushing directly to master. What do you think should be a good approach there?

@prank7 see this thread, it might offer some insight into my perspective there: https://twitter.com/getify/status/1294641351737380864

And...

https://twitter.com/getify/status/1294680109358764032

https://twitter.com/getify/status/1294684866806677504

I wouldn't put too much emphasis on OSS... I'd switch "The Ideal Tech Interview" and "Alternative To OSS Portfolio" around in preference order.

There's a lot of talk of privilege these days, and you have to recognise that someone working on OSS is privileged in some ways. They might not have an extensive Git repo because they have a family at home, or perhaps they just want to do a job and then come home and get drunk/listen to paul oakenfold/do a hobby/see friends/see family/anything else not related to coding.

If they don't have an OSS portfolio or github account, you shouldn't ask why not... I would in fact avoid asking about a git or OSS stuff unless they bring it up, though perfectly fine to ask "tell me about your hobbies".

I also errr on the side of coding homework. Again because of the above, but also it's not a real reflection of how they might code on a job/in your team and the standards your team has (via eslint or whatever). Though I quite like your ideas in "Alternative To OSS Portfolio", but I suspect this is really becoming a bit too long winded.

I think there's a difference between hiring a junior (no experience) and hiring someone who has been working in tech since uni (let's say 22) and might be in their 30+ years... I'd love to see data on teams that invest what you've suggested and more into their hiring process vs teams that might keep it quite simple sticking with a tech screen/vet phone call of 30 mins then an hour or so interview process.

Hiring in tech is broken. I don't think the above fixes it, but there are some good ideas there. but please just be aware, not everyone dedicates time outside of work to their day job. for some coding is a day job, it might not be for you, but the idea of being able to learn and code outside of your 9-5 is a privilege.

@JaredCE

I'd switch "The Ideal Tech Interview" and "Alternative To OSS Portfolio" around in preference order.

For those who have the OSS experience and can demonstrate it adequately with links to their public work (again, not just code, but all the other important signals), I think it would be irritating to be asked to re-create that in the private repo. I don't see why suggesting both paths and letting the candidate choose is a problem.

I don't think it has to be worded in such a way to make someone feel bad if they don't have (or don't want to use) any OSS work.

Thank you @getify for speaking your mind . We all know hiring is broken in various ways but we need to figure out how to fix that one day at a time. I have given a serious thought about this today and from tomorrow onwards I have changed the interview processes taking a lot of inspiration from here . This process will start for my own team but I will try to push this as an engineering culture in our organisation .

@getify I totally agree on OSS part that it makes the judgement transparent but again asking around that projects might not be best for organisation having no project as such or the interviewer has not worked on that framework or technology specifically. Such challenges are always there.
2. Next thing I personally try it that, I ask candidates between the lines out of 3-4 min. introduction he gives and try to dig deeper with him as why he took that approach, cross question around that and checks if he has good quantitative analysis and how he responds to such questions.
3. Next is giving a very basic problem that requires 30-50 lines of code so that I can understand how he writes code, for-of or foreach doesn't matter but variable initialisations, naming conventions, how readable code he writes, how modular and error cases he can think of definitely counts I think.

Gold! Thank you for sharing this!

One task might be to file a PR with a change to the code. I'd ask them to review the PR and make any comments they feel are necessary, and to ask any clarifying questions.

perhaps, if you are starting with a known repo, you could write and send a PR to them, ask them to critique someone else's PR rather than reviewing their own PR.

I feel like I might run the risk of saying "this PR is a flawless sample of exemplary work, and we should definitely hire this person immediately." if I was asked to send and subsequently review my own pull request.

I have one question. Any production-grade code is the organization's propriety. So, legally, no one shares code snippets with anyone. In that case, I see a problem in this approach. Let's consider a happy path as well. Say someone has agreed to show the codebase and take the interviewer through the code. How does an interviewer get to know whether the code snippets were written by the interviewee or not? Git blame is an option but that can tweaked. That's the reason, I do not find this practical, or I failed to grasp the proposal fully.

Quite an involved process, it does reflect real life more. BUT, I would like to see it phrased for the company that's being pitched at. Because from my perspective, from what I see, the company is trying to make the best decision, with the least amount of spending, effort.