Hire reliable agency staff by scoring a paid work sample

What to take away

One paid task, built with gaps on purpose, tells you more than any interview. Give the same task to every finalist and score it the same way each time.
Fix the scoring columns before the first submission arrives. Deciding what matters after meeting somebody is how agencies hire people they liked.
Raising the item that should be escalated is the strongest sign of reliability in the sample, and checking for it costs nothing.
The reference call is worth more than the interview, and only if you ask questions that cannot be answered with a compliment.

Build one task with gaps in it

Make the task a message from a fictional client with one missing detail, one ambiguity and one item that should be escalated rather than decided. Reliability in this work is mostly about what somebody does when instructions are incomplete, so the task has to be incomplete on purpose.

Keep it modest in size, so a good answer fits inside the paid time. One example, close to a real inbox:

Hi. Can you pull the March onboarding batch together and send it over before my Thursday review? It is all in the shared drive somewhere. Also one client has asked for a refund on the setup fee, so just sort that out if it comes up. Thanks.

Here the missing detail is which batch and where the files sit, the ambiguity is which Thursday, and the refund is the item to raise rather than decide. Write that answer key before you send the task, so scoring does not drift toward whatever the first candidate happened to do.

Pay for it. Unpaid work samples select for people who can afford to work for free, and they attract the wrong candidates in both directions. A paid task also lets every finalist start on the same terms.

What to read for in each submission

Read each submission against five questions and write the answers down before you form an overall impression. The impression is the part that favours the candidate who happened to write warmly.

Did they ask about the missing detail before starting, or fill the gap silently?
Did they name the ambiguity and say which reading they took?
Did they write in a form you could pass to a client without rewriting it?
Did they finish inside the paid time, and say so if they did not?
Did they raise the escalation item with a recommendation instead of deciding it?

Work sample: what to watch for

Noticed the missing detail
Asked one precise question
Escalated what should be escalated
Writing clear for light review
Delivered by the promised time

The last item is the single strongest predictor of reliability, and it is free. A candidate who raises the refund with a recommendation has shown the habit you will rely on when you are not in the room.

Score four columns before anyone discusses the candidate

Score comprehension, communication, judgment and dependability one at a time, each from evidence you can point to. Do it alone and before talking to anyone, because group discussion first produces consensus around whoever was most personable.

Column	Question to answer	Where the evidence sits
Comprehension	Did they grasp what was not said?	The work sample
Communication	Is the length and tone usable as sent?	The sample and the emails around it
Judgment	Did they decide only what was theirs to decide?	The escalation item
Dependability	Did they do what they said, when they said?	The whole process, start to finish

Four-column candidate scorecard

Comprehension

Measures: Understood request
Evidence: The work sample
Score timing: Before discussion

Communication

Measures: Wrote clearly
Evidence: Sample and emails
Score timing: Before discussion

Judgment

Measures: Decided vs raised
Evidence: The escalation item
Score timing: Before discussion

Dependability

Measures: Did what they said
Evidence: Whole process
Score timing: Before discussion

Software familiarity does not get a column. It is the easiest thing to teach and the most commonly over-weighted, and the new-hire checklist is where that teaching belongs once the person has started.

Keep the interview to confirmation

One conversation, half an hour, three questions. The sample is already your evidence, so the interview only checks whether the good decisions were deliberate.

Walk me through the sample. Why did you ask that question?
Tell me about a time you did not have enough information to do something properly. What did you do?
What would make you leave a job like this within a year?

The first checks intent. The second listens for whether they went and found out. The third tells you what to avoid, and honest candidates answer it honestly.

Do not use the interview to re-test the sample. You already have that evidence.

Reference calls that produce answers

Most reference calls produce compliments because most questions invite them. Three questions do not, and each one asks about something that happened rather than something the referee feels.

Tell me about a time their work reached a client with an error in it. What happened next?
How often did you have to follow up on something they had said was done?
Would you hire them again for the same role, and what would you do differently?

Three reference questions that work

What did they need most support with?
How did they handle an unclear instruction?
Would you have them back, and in what role?

Ask for a referee who supervised them, not a colleague. And ask the same three questions every time, so answers are comparable.

Keep the process the same for every finalist

Your obligations as an employer apply to every finalist. The EEOC states that it is illegal to discriminate against a job applicant on protected grounds, and application and hiring are among the areas it names.

One task, one answer key, one set of columns and one set of reference questions give you a record that you judged the work, not the person.

The EEOC's guidance on background checks tells employers to make sure they are treating everyone equally. Asking every referee the same three questions is the simplest way to do that in this process.

From the decision to day one

Carry the scores into the first ninety days, because the lowest column is where the supervisor should look first. Before the offer, confirm the actual hours and the time zone expectation in writing, so nobody learns them after starting.

Days 1 to 30shadow the person who holds the work now, learn the client's routines, and record each step while it is fresh.
Days 31 to 60run the work end to end with a reviewer checking the output, and prove the written steps by having someone else follow them.
Days 61 to 90hold the work without a reviewer, and cover a full week of the account owner's absence from the written steps alone.
By day 90the written steps are complete enough that a second person can take the task over from them.

First ninety days onboarding

Week one
Standards, shadowing, no client contact
Weeks two to four
One account, everything reviewed
Weeks five to eight
Second account, sampled review
Weeks nine to twelve
Cover somebody's absence

If they cannot cover an absence by the end of the third month, the problem is usually your documentation rather than the hire. The wider training system behind this sequence is set out in the hiring and training guide.

Common questions

What if the best candidate wants more than I planned?

Recompute your cost of a delivered hour before deciding. If the price the market pays for the work does not support the pay the market demands, the answer is in your price, not in the candidate. The labour cost breakdown shows what loads that figure.

Why pay for a sample every candidate may not finish?

Unpaid samples select for people who can afford to work for free. Payment keeps the pool wider and lets you hold every finalist to the same standard of finishing inside the paid time.

Does a hire change what I should charge?

Your acceptance rate decides what you must actually offer, and two hires will tell you. That cost feeds the arithmetic in the pricing and profit guide.