With a remote-first team, Ad Hoc has some interesting challenges when it comes to recruiting engineers.
One of those challenges is finding a way to evaluate technical ability without an in-person technical interview. We do this by issuing homework assignments to candidates to be completed on their own time, and we’ve had great results with this approach. This blog post will highlight some of the benefits of our homework-based evaluation process, both for our company as well as prospective candidates.
Mapping evaluation to the work
In our collective experience, reversing a linked-list on a whiteboard is not indicative of the work done on a daily basis by a software engineer. Our homework problems are extracted from real challenges engineers face and are designed to evaluate skills that are needed to get the job done, day-in, day-out. As a remote-first company, the ability to work independently and reach out for help when needed is paramount. We operate our chat application as a way to provide that help in a way that is similar to how one of our engineers would ask their teammates for help on a problem. This also gives folks an opportunity to use any reference materials they may have, which allows candidates to do their best work in a much lower-pressure situation than a whiteboard evaluation.
Removing bias
All submitted homework is graded blind, with individual submissions assigned a random number as a reference for grading. We also have a requirement that candidates remove any identifying information from their submission to prevent accidental unblinding. This lets us grade without any preconceived notions of abilities, and ensures the first evaluation of a candidate is based solely on the solution delivered. This means that everyone starts on the same footing in the process, regardless of whether a given candidate has a Computer Science degree, is a boot camp graduate, or is wholly self-taught.
Refocusing the interview
Using homework as a technical filter allows us to to take technical evaluation out of the interview; this leaves space for interviewers to get a better sense for how a candidate would fit on our team. What technologies excite them? How do they handle stressful situations? Given engineers do our technical interviews, it also leaves a lot of space for candidates to ask questions to get a feel for what day-to-day life is like at Ad Hoc. I’ve been able to field great questions during interviews on topics like allocating story points and how we make technical decisions because I’m not spending time on trying to screen someone in or out technically.
Democratization of evaluation
Engineers at Ad Hoc are eligible to grade homework on the first day on the job and are expected to grade homework after they’ve gotten settled in. This means that everyone has a stake in ensuring the quality of their future teammates. We also feel that code review is one of the most important skills in an engineer’s toolbelt, and evaluating homework submissions is a great opportunity to hone those skills.
Minimizing continuous blocks of time
We don’t set a hard deadline on folks completing the assignments. We recognize that we’re asking folks to invest time into our process, but we feel like our homework compares favorably to extensive on-site interviews or other evaluation techniques, especially for candidates who have responsibilities outside of their work life. The homework can be completed in small blocks of time as candidates can fit it in.
Want to give our homework a shot (and maybe grade some yourself someday)? Apply to one of our open positions, then if one of our recruiters thinks you’re well-qualified, they’ll invite you to complete one of our homework challenges. Thinking about using homework to evaluate candidates at your company? We’ve licensed our assignments with V2 of the GPL so feel free to take inspiration from our problems! (But no, you can’t have the answers.)
