Technical Interviews Have Issues But We Can Make Them Better (A Deep Dive With Ted Tencza)
Jump to:
If you think technical interviews could use improvement, you're not alone. Ted Tencza, founder of Code Purple Consulting (you can hire them!) and a seasoned expert in software development and leadership, knows all about the pitfalls of bad interviewing practices. We had a chance to pick his brain, and his tips are pure gold for anyone looking to make better hires.
Let’s set up the scene
Could you start by introducing yourself and sharing your experience with technical interviews?
I started working on becoming a better interviewer when I first became a manager 20 years ago. I was thrown into the deep end and quickly saw how flawed the interview process was from both sides of the table and was determined to make it better. The tipping point came when I was interviewing for an Engineering Manager role, and the current EMs admitted at the start of the interview that they didn’t interview much, and were unsure what they were doing. I ended up running the interview myself and was hired. From there, I worked to improve the process within my control, then organisation-wide, and now I am on a mission to solve them systematically and multi-organisation wide.
From your experience, what are the most common problems with technical interviews?
Almost all problems stem from a lack of knowledge and training. Many companies assume that a few years of coding experience makes someone a good interviewer. Without training, they rely on their own experience, which perpetuates the bad practices inherent in the tech interview process. Some rely on copy-paste solutions, which rarely work in new contexts—think Google's lengthy hiring cycles and brain-teaser questions.
When coding tests go bad
What are some common pitfalls when setting up and running coding tests, both from the candidates' perspective and the interviewer's assessment?
Lack of clarity, for both the interviewer and the candidate. The questions can be irrelevant, convoluted, and/or favour a very specific cohort of applicants. There's also inconsistency in what's considered a passing answer, with multiple solutions often being possible, and the candidate has to rely on luck that their correct solution is the specific one the interviewer is looking for.
How can we make technical interviews less stressful?
First, don’t surprise the candidate. Clearly outline what will happen in the interview. Telling them that they will “debug an issue in an API” for instance, will not give them any advantage in solving the problem. Tell them if they can use Google or Stack Overflow, and that they can ask clarifying questions. Ease the candidate into the test to help calm nerves. Make the first one or two tasks fairly short and trivial, so the candidate gets a few wins up front and to build confidence, before moving on to more challenging questions.
What tips can you offer to interviewees for tackling coding tests?
As a candidate, it is (or should be) ok to ask questions. Ask what the technical test entails - is it writing something from scratch, debugging, system design? What hardware will be used? Are you expected to bring your own, or will a machine be provided? What IDE will be used, and can you use your preferred IDE? What resources (i.e., Stack Overflow) are available? Remember, the company needs you, so it’s a two-way street.
Some suggest avoiding coding tests altogether, instead evaluating candidates based on their GitHub profiles or prepackaged take-home tests. What are your thoughts on this approach?
While appealing, this approach limits candidates to a specific demographic. Not all great developers have active GitHub projects; some are busy with families or other hobbies. Take-home tests also favour candidates without other responsibilities. They require effort from the candidate with little value in return (effort/value asymmetry). That works for FAANG companies who are swimming in candidates, but a) it means even FAANG companies are losing out on good candidates who cannot be bothered to give effort for no value, and b) 99% of companies are not FAANG companies.
Structuring the interview process
Do you have any advice on structuring the interview process, including the number of interviews? Are there any issues with conducting technical interviews first?
It depends on the role. If specific skills are needed, more interviews may be necessary. Larger organisations might require more interviews due to more stakeholders.
I see three issues with having the technical interview first:
- Value/effort asymmetry - the candidate puts in more effort before knowing the value of the company/role, which could deter some really good candidates.
- Smaller pool of interviewers - Interviewers need a deep understanding of the technology. A .NET developer will struggle to know if a Node.js candidate is a fully qualified and effective programmer. This leads to the interviewer pool being overwhelmed with conducting interviews on top of their other responsibilities.
- Stress - Coding tests are stressful. It is easier to answer questions about previous experience and roles than to code with someone looking over your shoulder. Having one successful interview with a company already completed will help the candidate feel more relaxed and allow for a more accurate assessment of their skills.
Who should be on the technical interview panel and how do I choose the right people for the task?
The most important characteristic is that the interviewer wants to be there. Someone who thinks interviewing is a waste of time will show that belief in their conduct during the interview. They will also be unreceptive to improving their interviewing skills, provide limited and poor feedback, and make poor pass/fail decisions.
For the technical interview, the interviewer should be well-versed in the technology under discussion if possible. They should be at the same level as the role or higher, if feasible. Finally, they should be trained on how to interview, how to sell the company to the candidate, and how to handle various candidate personalities (nervous, shy, soft-spoken) that are poor interviewing traits but wouldn’t impact their ability to perform the role at a high level.
Can you talk about the benefits of having different levels of interviewers and using them effectively in the interview process?
Given that interviewing is a different skill from software development, it makes sense to treat it as such. Evaluate your interviewers based on their interviewing skills and experience, and rate them accordingly. At a previous company, I formalised this process, and we had junior, mid, and senior interviewers, independent of the software developer job title. Work with your TA/P&C team to understand which interviewers are at what level and assign them accordingly. Don’t assign two junior interviewers to the same panel; pair a junior with a senior so the junior can grow. Task the seniors with mentoring junior interviewers. If you have a high-value candidate, assign your most skilled interviewers to the panel.
Let's discuss candidate care. What does it entail, and why is it important?
Candidate care is simply treating candidates (and their valuable time) with respect. It is important for two reasons. First, interviewing is a two-way street; if a company treats a candidate poorly, the candidate may withdraw from the process or refuse an offer. Second, it provides the best possible environment to successfully and correctly evaluate a candidate.
It entails several things. Be prompt with feedback and correspondence. Provide candidates with appropriate information about the process (how many interviews, what type, how long, etc.). Be on time for interviews; don’t make a candidate wait. End interviews on time. For in-person interviews, ensure a candidate has a glass of water, knows where the bathrooms are, and is escorted while in the building.
You often recommend using a Candidate Briefing Package. Can you walk us through what that includes and why it's beneficial?
A Candidate Briefing Package provides candidates with appropriate information about the process (how many interviews, what type, how long, etc.). It sets expectations for the overall duration of the process and gives tips on how to succeed in the interview. It is beneficial because it shows respect for the candidate, reduces nervousness, ensures standardisation across the interview process, and provides an environment where the candidate can showcase their best possible self.
Any tips on gathering and utalising feedback in the interview process?
Most importantly, don’t do anything else during the interview except interview the candidate. Don’t check emails, engage in Slack conversations, etc. Pay attention to what the candidate is saying, rather than trying to think of the next question you are going to ask.
Having a note-taking template has always helped me—it reminds me of the types of questions I want to ask and allows me to record the candidate's answers. Editorialise as you take notes. If it’s a great answer, note that down. If it is a rambling response that doesn’t address the question, note that down. If the answer is a red flag, definitely note it down.
Write up feedback as soon after the interview as possible, ideally before speaking to the next candidate. The longer the time and the more candidates between the interview and the feedback, the less accurate it will be. Finally, make sure the feedback is more than just writing down the candidate’s answers; it should also reflect the interviewer's judgment on the quality of those answers.
Honing interview skills
How can we evaluate our company's interviewers so we can make them better? And how do we convince people to want to get better at interviewing?
Look at the candidate pipeline. Are you rejecting a high percentage? If so, are they being rejected for the right reasons? Look at the interview feedback. Does it justify decisions or seem to be based on gut instinct? Does feedback show evidence of bias?
To convince people, point out that poor interviewing leads to poor hires. And then you have to work with those hires. To me, that is motivation enough to want to be an effective interviewer.
What can we do to improve our interviewing skills?
Provide training to the interviewers. Treat it as a learnable (and improvable) skill.
What are some biases we can watch out for when interviewing?
“Not technical enough” without evidence, especially when applied to URM candidates. Rejecting URM candidates at significantly higher rates. Non-standard questions that do not allow for apples-to-apples comparisons between candidates.
What advice can you offer to help people properly evaluate a candidate?
Understand the role. We used to do internal “success profiles” for roles. These are not JDs but rather benchmarks. Would they need to learn a new language within the next year? Would they need certifications? Would they need to have an immediate understanding of the domain?
Then ask candidates questions to understand if they would be successful. This way, you don’t waste time evaluating irrelevant criteria.
What are some of the most useful behavioural interview questions? And what are some of the least useful? (especially as it relates to technical roles)
Most useful questions deal with the candidate's problem-solving and decision-making ability. Do they know, and can they articulate, the reasons they made certain decisions? If they delayed a project by a week to make it performant, can they explain why, and what trade-offs did they consider? Finally, ask about things that went wrong. Do they understand why it went wrong, and what lessons did they learn?
The least useful questions ask about processes, procedures, or project decisions not created or influenced by the candidate. For instance, asking questions like “how does your company do agile” is useless unless the candidate instituted the practice and will need to do so again in the new role.
Best practices and takeaways
To wrap it all up, what can companies do to establish better interviewing practices?
Hire me. In all seriousness, treat it as its own thing, not as an afterthought. Have someone competent own the process and be responsible for ensuring it functions properly. Don’t just assume the developers doing the interviews are doing it correctly.
What are your top takeaways for conducting better technical interviews?
Have skilled interviewers ask relevant questions.
Conduct periodic reviews of the process. Are the questions still relevant? Are the number, order, and types of interviews the best they can be? Is feedback from candidates showing a problem?
Train, train, train the interviewers.
How can you help companies stay on top of their interviewing and hiring game?
I can audit the process from top to bottom and provide feedback on every aspect we touched on above. I can partner with the TA team to improve outcomes. I can train interviewers and provide coaching to them. Finally, I can train and coach the person who is responsible for the interviewing process (or recommend that there be one person in this role).
Anything else you’d like to add?
It doesn’t end with the offer. Make sure you have your technical onboarding process in good shape, or you might lose the candidate/employee after spending all that time and effort to get them to the accepted offer stage.
About Ted Tencza
Ted has over 25 years of experience in Software Development, the last thirteen of which have been in leadership roles. Currently serving as the founder of Code Purple Consulting, he has run multiple teams at Atlassian, Bigcommerce, finder.com and Prospa, with experience in both operational and development projects. He enjoys focusing specifically on the SaaS offerings, as well as working on improving the recruiting, hiring, and onboarding of new developers. Learn more about working with Ted here.
Read our newsletter interview with Ted here.
Tips for how to run a good interview
Hiring tips from our friends in tech
Resume tips from a tech recruiter