The letter A styled as Alchemists logo. lchemists
Avatar of Brooke wrapped in a grey scarf and wearing a black jacket with a red zipper against a slightly blurred background of fall colors of mainly orange leaves. Brooke Kuhlmann
Published October 11, 2020 Updated March 22, 2024
Cover by Leonardo AI
Interviewing

Interviewing, especially within the software engineering industry, has been grievously broken for years, even decades. Having been both in charge of evaluating talent as well as marketing myself as a potential hire, I’ve built up experience worth sharing. Interviewing is a constant evolution and fully expect even this article to evolve in years to come.

Much of this article is geared towards hiring distributed talent. There are two major sections to this article are focused on interviewing candidates and interviewing organizations. No matter which side of the table you are on — and in your career you’re likely to be on both — this article has you covered.

History

I wish I had better statistics to report but the software industry is getting worse with absurd interview processes. We need to band together, form unions, and fight against these terrible practices.

Speed Hiring

Before we begin, I’d like to mention my Speed Hiring article which collapses down a lot of what is explained in this article to it’s core essence. You might enjoy speed hiring as a fast and effective way to find talent or employment if you don’t have a lot of time on your hands. That said, taking the time to read through this article will detail how to think about interviewing in a much more equitable and empowering manner.

Candidates

The following is my process for interviewing candidates, which has been compiled over several years of experience and also shows how this process can be leveraged as a marketing tool. Interviewing doesn’t have to be mundane. In fact, your interviewing process can also be a way the industry looks to you for inspiration and how you can be a magnet for talent, meaning they come to you rather than you searching for them.

General

The following are good rules of thumb to interviewing. This list can be a guiding factor to follow when we discuss the five stages of the interview process later.

  • Prefer writers

    • Writers are clear thinkers and good communicators who don’t let poor ideas live long. They are highly treasured.

    • Introductory emails, published articles, open source documentation, and/or Git commits are a great indicator of who these people might be.

  • Prefer portfolios over resumes

    • Check for digital identity. A professional avatar, full name, and proper email (or any contact info in general) makes a huge first impression.

    • Read what they are saying and how they express themselves online. This can include social media, but hopefully they also write long form via a syndicated feed. Syndicated feeds are a great way to see how they communicate, express thoughts/ideas, what their experiences have been, and what they are passionate about.

    • Read through their open source code. You want to spend a good portion of time on this endeavor since you can get a sense of their engineering prowess:

      • Git Etiquette - Do they use a Git Rebase Workflow and can they document how the implementation was architected through structured commits?

      • Documentation - Does the project have a README complete with setup, usage, configuration, tests, changes, etc?

      • Architecture - Is the implementation well architected, easy to navigate, has passing tests, of high code quality, and works as described? Even better if built with tools like Rubysmith or Gemsmith which establish these practices from the start.

      • Maintenance - Is the implementation well maintained or can it be easily maintained? There should be a history of documented changes and versioned releases.

    • When an interviewee doesn’t have an online presence, ask if it’s possible to learn about their work in other ways. Perhaps their work is private and you can get legal, temporary, access to review and discuss further. The latter case shouldn’t diminish the value of the applicant.

  • Use resumes as a backup to portfolios

    • Useful when the interviewee doesn’t have an online presence or publicly available work.

    • When the interview process is almost to an end and you’d like a chronological view of their career journey.

  • Avoid puzzles and esoteric challenges. Using complicated, even esoteric code challenges often feels like a way to diminish the interviewee and/or demonstrate the interviewer’s so-thought superior knowledge. Such practices don’t make you look smart, but will instead hinder your efforts at diversity and give you a bad reputation in the community.

  • Avoid reference checks. In research conducted by Katy Putsavage, she found letters of reference for women tend to raise doubts and emphasize the collaborative nature of women’s work while letters to men are more likely to denote individual brilliance.

  • Once the initial impression of the interviewee is formed, spend the rest of the interview asking questions that can change that impression. Keep it fluid by digging deeper where there is interest and letting the interviewee do the same.

  • Take copious notes during the interview process in order to capture a detailed analysis of the interviewee later. Always inform the interviewee, when taking notes, you are doing so and is the reason why they see you, occasionally, looking away or typing on your keyboard. Recording each interview session can be a great way to capture notes and re-review what was discussed but only if the interviewee gives consent.

  • Strive to hire the best and brightest but realize not everyone is a superstar. The interviewee should be able to augment the team, fill-in gaps, get things done, and have the potential for further growth (both personally and for the team).

  • Exit an interview with a clear sense of whether the interviewee will strengthen the team, inspire change/new ideas, and improve the overall success of the organization.

Team

Assemble a stellar interview team and train each member of the team in the art of interviewing, never bring them in cold. Each interviewer should have completed all of the Havard Implicit Bias Tests, have a clear understanding of what their biases are, and how to correct them. These tests should be conducted prior to each interview because research shows implicit bias is reduced when training is done on-demand.

Prepare for every interview by thoroughly researching their portfolio, resume, and/or code samples. Include all team members in this process as much as possible. If you need a study group to build a common understanding of the interviewee, do it. This effort is important to show you truly care about the interviewee and their past experiences. It’s the first step in establishing good rapport and trust. The goodwill makes the interview experience a positive one by appreciating and seeing them as an individual instead of a number.

Ensure every team member can provide comprehensive/constructive feedback of each interviewee and everyone’s expectation of the interviewee is the same in terms of having a positive experience, qualities desired, interview goals, etc.

Time

The time required for processing an applicant through all stages of the interview process must be efficient and fast. Otherwise, you’ll risk hurting your reputation. The best applicants are only on the market for a short time — if at all — so do the same by letting the community know that when you post a job opening people take notice and rush to apply immediately.

Stages

The following details the stages of the interview process, each is described in order of importance. These stages are designed to filter out candidates that might not be fully qualified yet, thereby saving valuable time for both the team and the candidate. Reapplying later, after the necessary skills have been acquired, should be encouraged and welcomed.

Prior to conducting each stage of the interview process, ensure each candidate has an outline of the dates and times in which the each stage will be conducted. Included in the outline, should be the list of questions to be asked because this provides support to folks with learning differences and allows time to organize their thoughts.

I: Application

Use a single page to describe the hiring process. It should be easy to read and use so the interviewee is able to get started quickly with minimal resistance.

Overview

Start with why the organization is special and why people would be excited to work with you.

Expectations

Define clear expectations. Roles, responsibilities, avenues of growth, etc. are all welcome here. A bullet list is nice for this but feel free to be creative.

Benefits

Define all the benefits you offer and add links for further details, if necessary. Example:

  • Salary.

  • Paid time off (including sick time).

  • Paid holidays.

  • Paid parental leave.

  • Paid learning (schooling, tutorials, subscriptions, books, etc.).

  • Paid conference time and expenses.

  • Paid equipment/accessories.

  • Health, vision, and dental insurance.

  • 401(k) and retirement.

  • Short/long term disability.

  • Anything else of importance.

Form

The form should be the last section of the page so the applicant can quickly fill in details about themselves. Like any positive user experience, ensure the interviewee has little to do in terms of data entry. You want the experience to feel as if they did nothing at all to apply. Example:

  • Name (required) - The interviewee’s full name.

  • Email (required) - The interviewee’s primary email address.

  • Location (required) - The interviewee’s primary location.

  • Portfolio (required) - Can be single or multiple links (i.e. web site, open source, code sample, etc).

  • Salary (required) - Clarifies the range of what you can and can’t afford.

  • Notes (optional) - Allow room for random thoughts, additional explanation of above details, etc. Leaving this blank is perfectly fine too.

After the application has been submitted, ensure an automated message is immediately sent to the applicant so they can rest assured their application was properly received and is being processed. This message should include an estimated response time so the applicant has a rough idea of when to expect further details.

Filter

Making the process near effortless, means you’ll have a high volume of applicants worth filtering. Keep the process simple by iterating over each applicant in chronological order and filter those you want to accept and decline.

When accepting an applicant, make sure to congratulate them for making it to Stage II. You’ll want to include a date, time, and meeting agenda for next stage of the interview process.

When declining an applicant, make sure to immediately message them and explain why they didn’t meet the position’s criteria. Use this as an opportunity to help them grow and level up. This level of care has the added effect of making the experience a positive one while still showing the applicant they’re valued. So many companies don’t even bother to respond or, worse, leave you with a one-line, unsympathetic message. Even when declining, you have an opportunity to make a lasting positive impression on the applicant, which can pay off in years to come in interesting and surprising ways.

II: First Impression

Assuming the application is enticing, meet the interviewee for the first time. Do this via video chat or at a public place (like at a tea shop). The former is preferred as it is convenient, less time consuming for everyone involved, and properly reflects the remote working environment.

Start by being inquisitive about them. For instance, ask them how their day is going, did they have a good weekend, recent events in world news, etc. and then move into more job-specific questions. Focus on being kind, a good listener, non-judgmental, and creating a safe space for discussion.

Watch body language. If the interviewee is nervous or anxious, determine the cause and figure out what will make them most comfortable. It’s important the interviewee sense the interview — and potential work environment — is a safe space for them to be who they are.

The following is a list of questions that can yield a great first impression:

  • Salary

    • Confirm salary expectations as provided in Step I in terms of range and exact figures to ensure everyone is on the same page.

  • Purpose

    • What got you into software development?

    • What are you career goals?

    • Why are you interested in working with us?

  • Mastery

    • What are your strengths and/or superpowers?

    • What are your weaknesses and/or aspects of your career you are not interested in?

    • What methodologies and tools do you use to manage your workflow?

    • Can you explain your project maintenance practice?

    • When do you consider your work to be finished?

  • Autonomy

    • How long have you been doing distributed work?

    • How do you handle distractions?

    • What is your preferred way to work and get things done?

  • Communication

    • What is the best way to collaborate on a project?

    • When in the middle of an iteration with an unexpected discovery or major change in functionality, how do you explain this to the rest of the team?

    • How do you handle disagreements with your proposal, methodology, and/or implementation yet ensure your opinion is heard?

  • Learning

    • What are some of your favorite software engineering news feeds, books, podcasts, etc?

    • What have you read recently that has excited you or caused you to learn something new?

    • What is on your list of things to learn next?

    • What are you building or have recently built that is most exciting to you?

    • What have been the biggest influences on your life and why were they so impactful?

  • Leadership

    • Can you explain when you’ve had to take ownership of a situation/project?

    • Can you explain when you’ve had to recover from a difficult situation?

    • How have you handled mistakes and/or failures with software?

    • How have you handled mistakes and/or weakness of character in others?

    • Have you mentored/taught others and do you have a mentor yourself?

  • Ethics

    • How do you honor the contributions of others and why is that important?

    • How have you ever been in a situation in which you confronted unfair, biased, or cruel behavior in a work environment?

    • How have you confronted illegal or unethical situations in the workplace (if any)?

  • Security

    • What steps do you take to be secure?

  • Technical

    • What is the difference between Git rebase and merge workflows?

    • What is the difference between composition and inheritance?

    • What is a monad?

    • What are test doubles and the differences between them?

    • What is SOLID design?

    • What is TRUE design?

III: Collaboration

There are a couple of paths to follow here and it depends on the individual and their situation. The following details both with the first being the preferred approach.

One Week Iteration

When an interviewee has a strong portfolio such as a syndicated feed, social presence, open source code, etc., it would be insulting to force the interviewee to do a code challenge or spend more of their time proving themselves to you. They are impressive enough and it’s important to recognize that. In this situation, bring the interviewee in for a weekly iteration with your team and pay them for it. Repeat: When you pay them for their time, you respect they are doing real work and shows you value their time.

Regardless of whether you perform weekly iterations, schedule your week as follows:

  • Monday: Plan out the week with the interviewee and the team. Encourage the interviewee to ask questions and pick up tasks of interest.

  • Tuesday: Share progress and/or pick up new tasks.

  • Wednesday: Share progress and/or pick up new tasks.

  • Thursday: Share progress and/or pick up new tasks.

  • Friday:

    • Share progress and/or pick up new tasks.

    • Have the interviewee, along with the rest of the team, demo and explain completed features and/or bugs fixed.

    • Schedule time for a formal review of interviewee’s work with the team. Allow the interviewee to provide feedback on experiences with working in the code base, the business space, team/organization culture, and anything else the interviewee might have observed.

    • End the day early by extending an offer to the interviewee or providing feedback on what they could improve upon so they can try again later after they’ve gained more experience. Either way, end on a positive note because if it’s not a good fit now then it might turn into something more fruitful in the future.

Doing an iteration like this is valuable as it provides a two-way discourse between you and the interviewee so you can learn:

  • Their Git commit etiquette.

  • Their design process.

  • Their implementation style.

  • How they ask questions.

  • How they handle being stuck and the time to resolution.

  • How they communicate/pair with others.

  • How they handle code reviews and resolve feedback.

  • How they deliver quality solutions.

  • How they manage a bug/feature, implement, deliver, and handle any unforeseen production issues/customer feedback.

  • …​etc…​

At the same time, the interviewee is learning about the business, code base, team dynamics, culture, etc. All of this is important as it allows both the team and the interviewee to determine if it’s a good long term fit. It’s important both parties determine this quickly rather than several weeks/months later when so much time and effort has been invested in the interviewee.

Lastly, don’t forget to have time to hang out with the interviewee during the week casually or while working together!

One Month Iteration

In situations where the interviewee can’t schedule an entire week, which is definitely tough for some, have them contribute when they are able to fit time into their schedule. Be flexible and considerate of the interviewee’s schedule as they might be entertaining other offers, traveling, handling currently employed tasks, etc. Spreading this work over a month can be a great way to accommodate the interviewee while you both get to know each other. Plus, this encourages — and is typical of — a distributed work environment. 🎉

Again, and this is important, pay them for their time and be respectful of their effort. All of this goes a long way in showing you care about the interviewee.

IV: Review

After all stages of the interview process has been completed, it’s time to gather as a team and discuss notes. Breaking the notes down by key areas of interest can be very helpful here. Example:

  • Drive

    • Autonomy - Can be given a task, owns it, and carries it through to completion while managing time and technique along the way.

    • Mastery - Has deep knowledge of current skillset and tools. Is able to dive deep and gain thorough knowledge and experience of a subject by being constantly curious and reexamining existing standards in new and interesting ways.

    • Purpose - Has the passion and discipline to focus on things that matter most. Remains engaged and has the endurance to see it through.

  • Professional - Is humble, composed in heated situations, can take critical feedback, maintains an optimistic outlook, and appreciates excellence.

  • Quality - Cares deeply about the quality and craft of the work they are doing. If they see something broken, they fix it.

  • Communication - Is a good listener and is able to communicate clearly and effectively either verbally or as written text.

  • Kindness - Shines with optimism and joy either through the look in their eyes, the laugh lines in their face, and/or way in which they carry themselves. Can listen and feel what others are experiencing; includes and appreciates the work of others; demonstrates respect, and does unto others as they believe they would do unto themselves.

  • Integrity - Knows when to do the right thing when people are not looking or even if something is not in their best interest.

  • Ethical - Has a strong moral compass, defends any who may be marginalized, avoids technologies/services that may be abusive and/or legally compromising, and can navigate an ethical course of action.

  • Organization - Can keep the task at hand organized in terms of requirements, deliverables, schedules, notes, interrupts, etc.

  • Analytical - Can figure out solutions to hard problems based on previous experience, searching for answers, reading up on new solutions, and/or asking for help from others.

  • Self Awareness - We all have different motivations, sleeping patterns, levels of energy, etc. Each day is different too. Knowing their mood is important because of the downstream effects it can have on others.

  • Growth - Growth is the hardest and, sometimes, most difficult to measure but shouldn’t be overlooked. When an interviewee is low scoring in terms of current skills, tools, breadth of knowledge, and generally rough around all edges but has the passion, eagerness to learn, and high level of curiosity, you might have stumbled upon a diamond in the rough. It could be worth investing in the interviewee (perhaps at a lower salary). That investment depends on whether you have capacity to absorb the individual and have mentors on your staff to polish those rough edges adequately. Keep in mind that growth can occur at any age and/or experience level. Due to the nature of this particular quality, it can have a strong influence over other scores.

This can be further automated by using an internal application or spreadsheet. Each can be graded on a scale of -1 to 1 where -1 is negative, 0 is neutral, and 1 is positive. Allow space for each team member to record and explain notes for each score as well.

V: Resolution

Once review and final analysis of the interviewee is concluded, it is time to either accept or decline the interviewee.

Accept

Upon acceptance, email the interviewee with getting started instructions:

  • Explain (or reiterate) the position, salary, role, and responsibilities.

  • Provide links for setting up payroll, health benefits, and anything else which might be necessary.

Decline

Upon declining the interviewee, respond quickly explaining why they were declined in addition to what you liked about them. When explaining disqualifying criteria, make sure to provide resources on how they can improve. Though they didn’t make it through the door this time, that doesn’t mean they could acquire the skills to try again. Remember, interviewees go through a lot of effort during the interview process so being able to decline them while being genuine and honest will go a long way. Careers are long, so leave interviewees with a good impression of you and the organization. Interviewees won’t forget this. Planting this kind of good will as a first impression will go a long way and, who knows, it could pay dividends in the future in surprising and interesting ways.

Organizations

Now that we have covered interviewing potential candidates, what do you do when the situation in reversed? Next, we’ll dive into what to do when looking for new opportunities so you can land on solid ground.

Be Current

Keep your online portfolio such as your web site, syndicated feeds, open source work, resume, etc. up-to-date. Spellcheck everything you write, avoid buzzwords, and speak/write clearly.

Be Prepared

First impressions are critical so research the organization you are interested in and personalize your introductory message accordingly. If possible, register and use the product/service prior to interviewing so you have more of a sense of what you might be working on.

The dress code for most interviews these days is fairly casual. Still, double check you know what the dress code is before appearing via video chat or in person. When appearing via video chat, ensure your environment is well lit, organized, and clean. If unable to use a live setting, enable a digital background to ensure you remain presentable.

Know the date, time, and length of each interview and double check the details. You’d be surprised how many times an organization, and especially, a recruiter can mess this up — which, by the way, should be a warning sign to you as well. With video chats, ensure the web link works prior to the interview. You’ll want to double check your video and audio too. In some cases, you might even have to download software or install a browser extension.

Know everyone involved in the interview. Write down the names/roles and keep them with you the entire time. I like to use a note taking application, like Obsidian, when interviewing using the following format:

# Format
<name> | <site> | <email> | <phone>
<notes>

# Example
Sandi Metz | https://sandimetz.com | example@sandimetz.com | 111-111-1111
A joy to talk to and well versed in the craft. I could learn a lot from working with her!

Finally, always respond with a follow-up email after the first interview to thank the team, share additional thoughts, or ask follow-up questions.

Be Honest

You’ll earn more respect by stating you don’t know the answer and will avoid future questioning of deeper knowledge that can’t be supported.

Be Humble

If asked to rate/estimate something, answer with a response lower than you might think. It is better to set a realistic expectation with room to surprise rather than a lofty goal that can’t be reached/delivered.

Be Positive

Maintain an enthusiastic outlook, even under the duress of questioning, feedback, and/or pairing. Such patience is difficult, I know, especially if those you are working with are relentless, unsympathetic, or even tone-deaf. You’ll most likely not want to continue working with such individual/individuals but that doesn’t mean you can’t use the situation to get better at your diplomatic skills. 🚀

Be Inquisitive

Do your homework and ask lots of questions. Everyone loves to talk about themselves, so use that to your advantage. Here is a checklist that might be of interest:

  • Financials:

    • What is the salary range for this position?

    • What has revenue looked like over the past couple of years?

    • What does the compensation package look like (i.e. health, bonuses, stocks, equity, etc.)?

    • What does the skill ladder look like and is there a clear path for growth?

  • Work/Life Balance:

    • Are the founders/leaders remote?

    • Are employees remote and treated as first-class citizens?

    • What does a typical week look like?

    • What does a typical day look like?

    • What hours are required (flexible, strict, and/or implied overtime)?

    • Do all leaders adhere to and exemplify reasonable work hours?

  • Inclusivity:

    • What is the ethnic range of leadership and engineering?

    • What is the female to male ratio of leadership and engineering?

    • What is the ratio of novice to experienced engineers?

    • What is the age range of the staff?

  • Communication:

    • Do you practice Readme Driven Development including proactive upkeep?

    • Do you use a Git Rebase Workflow?

    • Do you record architectural decisions and related discussions for long tail thinking?

    • How is group chat used and are notifications rarely used?

    • How is email used?

    • How are meetings scheduled in terms of frequency and do they have an agenda with action items?

  • Team:

    • Is mental health a top priority and openly talked about?

    • Are vulnerable people embraced, treated fairly, and respected?

    • What is the organizational structure and is there a diagram available?

    • Are consultants currently on the team (or have been in the past)?

    • Has there ever been layoffs and how many if so?

    • What is the retention rate of staff in general?

    • Is there an on-call rotation and why is it needed?

    • How is collaboration/pairing performed?

    • How are code reviews performed?

    • How are retrospectives performed?

    • How is organizational debt addressed?

    • How are new members onboarded?

  • Stack:

    • What languages/frameworks are used?

    • What servers are used?

    • What databases are used?

    • What third-party services are integrated?

    • What deployment strategy is used and is it fully automated?

    • What exception monitoring is used and do you practice Inbox Zero?

    • What metrics are used?

    • What issue tracker is used and is it well organized?

    • Do you use home-grown solutions or pay for existing solutions?

    • Are all dependencies on the latest version and maintained on a weekly basis?

    • What onboarding scripts are used and are they maintained?

  • Outliers:

    • Is there a strong focus on individual growth such as mentorship/coaching, speaking, leadership, new skills, etc.?

    • What is the last change you implemented — based on employee feedback — and the results of that impact?

    • Does everyone use the software/product developed?

    • What impact and primary clear win would I have on the team, if hired?

    • What is the biggest question you have wanted to ask me but haven’t yet?

Be Polite

  • Avoid asking how you did at the end of an interview. Receiving further inquiries will be your answer.

  • Avoid badmouthing your former employer. Speak positively as to why you are moving on and the lessons learned from that experience.

Be Growing

Each interview is a learning experience. Consider interviewing once a year to keep your skills sharp and a pulse on the ever changing landscape.

Conclusion

Pericles once said:

What you leave behind is not what is engraved in stone monuments, but what is woven into the lives of others.

Whether you are interviewing future prospects or looking to land new work yourself, may this guide to interviewing be an inspiration to you and the people you influence. I’ve implemented and performed the above many times over the years and strive to promote this practice wherever I go. Together, we can make the industry a better place to live and work!