Using an open source contribution to skip a take home challenge

TL;DR

Adam Mikulasev and I made a list of active open source projects that need help. If you’re asked to create code for an interview process, this feels like it could be better than a take home.

👉 Active JavaScript projects: https://airtable.com/shrVw5qxaFJZSBgie/
👉 Active Ruby projects: https://airtable.com/shrJ7XfBanoxqS9pn/

Onto the article

My ideal world is one where take home challenges are rare. Even with good takehomes there’s significant value asymmetry. Employers ask for hours of free time in exchange for a 👍/👎.

They exist because employers want to predict performance and most code is private so there’s little observable work. Public code is usually open source, and only highly privileged folk have spare time to create that on their own.

If we must create code for job interviews, can it be code that supports the open source community that gives us so much when we’re gainfully employed?

Also, wouldn’t a pull request be a more accurate audition for the job than a toy project that lives in a silo? Let that one pull request be used as a technical conversation starter for multiple employers.

Thinking about developer job search experience today

Reasonable demands from multiple employers adds up to unreasonable demands on the candidate.

Picture Jess, a senior engineer who applied to 5 companies. After completing 5 phone screens over a week, she’s left with 3x 1 hour takehome assessments, 1 pairing interview, and decided 1 company isn’t a fit.

Each 1 hour challenge needs 2 hours by the time you get in the zone, complete the challenge, clean it up and submit.

The results are often inconsistent, which means each employer sees a different candidate. Say Jess completes the first takehome with gusto but made a mistake. The second takehome is best, and the third was phoned in because her week was compressed.

Sidenote: the company skipping to a pairing interview has a huge advantage. It can be booked shortly after the phone screen so their process maintains momentum. The other 3 companies have paused their process until Jess completes their homework. I still think it’s nuts we spend all this effort getting developers into our hiring funnel, only to send them away with homework.

Where open source contributions come in

Most takehomes we see are contrived problems. An example is Toy Robot, which has a book on how to solve it. You’re trying to predict how someone will be in the job. Someone who successfully solves toy puzzles won’t necessarily make a great employee at your company.

If someone could show how they:

• Refactor someone else’s code.
• Interpret a coding style and follow it (or not).
• Write about what changed and why it’s an improvement.

Wouldn’t that be a stronger signal? You get that with a small open source contribution. It can be a great technical interview conversation starter.

Other benefits:

• It’s easier to get started on a refactoring exercise than it is to start from new.
• It’s harder to cheat because each project is unique. And if someone uses ChatGPT or Github Copilot to solve a real problem, isn’t that something a good employee might do?

Downsides

Here are the primary downsides that I can think of, along with potential solutions.

1: It’s harder to apply an assessment rubric because each problem is unique.

Whatever the code is, it should just be a conversation starter. And it’s not just about the code - you’re observing how someone interprets a problem, proposes a solution and accepts feedback. Rubrics can measure those behaviours, even when the problems solved differ.


2. It’s not clear which employers will use this in their recruiting process.

Lookahead.bio is being built to help guide you here. We’re creating a marketplace where employers share what evidence they accept in their interview process.

That helps developers know what to spend their time on, and creates reusability. 2 hours spent contributing to an open source project is time well spent if you know most of the employers you’re interested in would use it as part of their technical interview process.


3. Finding the right project to contribute to is hard.

The paradox of choice is real. We have narrowed things down somewhat.

👉 Active Javascript projects and Ruby projects.

Adam created this list. He has helped 40 programmers close 39 issues in open source projects, including the Brave browser, Vue Storefront, pandas, Focalboard, bentoML, Weaviate, Wordpress and freeCodeCamp.

Using GitHub’s API, the list has the top ~1000 most popular (starred) projects for Ruby and Javascript.

Together we tinkered with filters to see what provides the right signal/noise ratio. We ended up looking at three things:

1. Opportunity to get involved (12+ open issues)
2. Momentum in the past month (4+ new issues opened, 4+ issues closed and 4+ pull requests merged)
3. Collaboration in the past month (4+ human contributors)

What do you think?

I’m genuinely interested in experience shares. I’m still learning and will always be trying to improve hiring: steve@lookahead.com.au.

On that note, a plug for our new thing: lookahead.bio. It’s a place where developers can provide evidence of their work and use it for every company they apply to. We say it’s DRY principles, for your job search. If you are interested in hiring or being hired, I’d love to hear from you.