Plagiarism: The Overlooked Bug in Technical Assessments

 

Most companies struggle to attract and hire technical talent. While much of this is due to a shortage of capable coders, software developers also possess a unique skill set that is difficult for recruiting teams to screen.

Very few recruiters know how to code. To adequately gauge candidates’ abilities, most technical recruiting teams are left with one of two options:

1. Involve hiring managers early in the hiring process.
2. Leverage some sort of technical skills assessment. 

The first option is unrealistic. Technical hiring managers’ time is expensive. You shouldn’t expect them to spend hours reviewing resumes or conducting phone screens when they could be managing teams and shipping product.

This is why many organizations turn to the technical assessment. Technical assessments - usually in the form of coding challenges - are designed to test the skills and competencies required of capable developers. That said, the proliferation of coding challenges (and the lucrative nature of programming jobs) has lead to an increase in the number of candidates who attempt to game the system.

Plagiarism in Technical Assessments

Computer programmers are constantly solving a vast array of technical problems. Often, there are multitudes of solutions posted online for any given programming problem and no particular solution is the best for all potential applications of that code.

During real-world software development work, it is expected to do research on existing solutions before tackling the task at hand. It is absolutely normal for a software developer to use others’ solutions that are readily available online when considering a problem. Doing so both reduces development time and increases reliability since solutions do not need to be developed from scratch.

Unfortunately, this approach is not helpful when the goal is to understand a candidate’s coding proficiency and problem-solving skill in a technical assessment. 

Plagiarism in this context can actually be counterproductive since it is impossible to distinguish between candidates who understand how the code works and candidates who simply copied someone else’s solution.

This puts hiring teams in a pickle. If the assessment can’t distinguish between capable candidates who know what their code does and those who copied it without understanding it, the burden of screening candidates gets pushed back to hiring managers. Again, technical hiring managers’ time is some of the most expensive time in the organization.

Learn the best practices for screening technical talent in our webinar - How to Crack the Technical Hiring Code

5 Ways to Combat Plagiarism in Recruiting

So if plagiarism can have such a negative impact on hiring manager time, how can we go about combatting it? These are five methods you can use - both in and outside of the technical assessment - to deter and detect it.

1) Establish What Qualifies as Plagiarism

As mentioned above, copying and re-using code is an expected and encouraged practice in software development. When it comes to hiring, most managers have different ideas about what constitutes plagiarism. Even in the same organization, different hiring managers may have radically different ideas around copying code in the assessment process.

If your organization is using technical assessments, it is critical for recruiters to sit down with individual hiring managers and discuss their views on plagiarism.

2) Set Expectations With Candidates

Once recruiters have established the hiring manager’s perspective, that information needs to be passed to the candidate before they start their assessment. That hiring manager’s definition of plagiarism shouldn’t be assumed as universal.

If you don’t establish the parameters for what constitutes “cheating” during the assessment, you’re not being fair to the candidate and may pass over candidates who perfectly understand the code they copied.

For example, you might tell candidates that you expect them to build their solution within the assessment platform (compared to in an external development environment), or discourage copying and pasting, if these are the criteria you decided on with hiring managers.

This is also a good opportunity to establish some ground rules. If you tell a candidate that you’re more interested in how they solve a problem, versus a “perfect” answer, they’ll relax and be less likely to plagiarize that perfect answer.

3) Use Software Similarity Scoring to Compare Submitted Code

Some technical assessments automatically compare a candidate’s submitted code to code previously submitted by other candidates. HireVue uses Measure of Software Similarity Scoring (MOSS) to flag likely plagiarists.

High code similarity is an indication of suspicious activity. That said, it should not be used in isolation. Recruiters should, as a rule of thumb, assume benign intent. It’s a starting point, a flag that a recruiter - possibly in conjunction with a hiring manager - should look closer at a candidate’s solution.

4) Evaluate How a Candidate Codes their Solution

Technical assessments should also provide an easily digestible timeline of a candidate’s activity while they completed the challenge. For example:

• Time spent in another browser window (loss of browser focus);
• When content is pasted;
• When code is tested successfully;
• When code is tested unsuccessfully
• When candidates are inactive.

With this data in hand, recruiters can often deduce if candidates are plagiarizing. A typical plagiarism pattern looks like this:

1. Candidate does nothing at the start (inactive - reading the challenge details).
2. Browser focus goes away for a period of time (searching for an answer).
3. Candidate pastes a large amount of code.
4. Candidate quickly passes tests without editing the code.

Again, this method is not entirely foolproof. While the pattern above is common with plagiarists, it is also the pattern we would expect if a knowledgeable candidate simply coded their response in another window or development environment.

This is where the final, and strongest, method of detecting plagiarism comes into play.

5) Validate Understanding With Follow-Up Video Interview Questions

The most powerful method of validating candidate understanding is simply to ask them questions about their solution. On-demand video interview questions delivered as a follow-up to the coding challenge turn the challenge into a conversation piece.

If a candidate can walk you through their thought process, the choices they made, the approaches they took, and the solution alternatives they considered, chances are they constructed the solution themselves.

If a candidate copied their solution without understanding it, their response usually makes this  very, very obvious.

Plagiarism: Means Different Things to Different Managers

Plagiarism can be a touchy subject in technical hiring. Ultimately, the best way to deal with plagiarism involves both out-of-assessment communication (setting expectations with hiring managers and candidates) and in-assessment features (similarity score, activity timelines, video interview questions, etc).

At HireVue, our technical assessment solution doesn’t attempt to define what plagiarism is. Instead, we give you the tools to look at how a candidate solved their solution, and leave it up to you to decide if a candidate’s solution constitutes plagiarism. We try to empower hiring teams to understand what really matters: if a candidate is capable.

This article originally appeared on RecruitingDaily. 

About the Author:

Scott Keller is HireVue’s CodeVue Product Champion. As product champion for the CodeVue technical hiring solution, he consults Recruiters, Technical Hiring Managers, and HireVue’s Product Engineering Group on how to recognize and retain talented software developers. Prior to working at HireVue, Scott was Senior Architect and Consultant at Navitaire, where he worked with development teams across the globe on building custom back office solutions, earned data processing patents in 3 countries, and invented the Temporal Tables model of the ANSI-SQL 2011 standard. Find him on LinkedIn.