Data labeling outsourcing
Data labeling outsourcing is paying another company to label the photos, video, text or sound you have, rather than hiring staff of your own for it. Those labeled examples are what AI software (an AI model) learns from. Using your own people usually costs far more than paying an outside company.
Three outsourcing options
- A full-service company (a managed service): it hires, trains and checks the workers, and you get the finished labels. Our comparison covers fifteen of these companies.
- A crowd work website: many part-time workers online each take on small tasks. For simple labels it's cheap and fast, but you have to review more of what comes back.
- A team just for you: a fixed group of the outside company's staff works on your project and nothing else, with payment usually by the month. The smallest team at Mindy Support has about five people.
Keeping the work in-house
When your files can't leave the company, your staff can label them using free labeling programs like CVAT or Label Studio. The cost is usually higher, since salaries, software and checking all fall on you.
Try a pilot before signing anything
A pilot is a small test job. Send each company on your list the same sample, and put a few of your trickiest files in it. For a first test, the makers of the labeling tool CVAT suggest sending 50 to 100 pictures, or 1 or 2 videos. Label Your Data and CloudFactory will label a sample free. Judge the labels against answers you made yourself rather than looking only at the cost.
Contract terms to include
- How many labels must be correct (for example 95 out of 100), and how that will be counted.
- What portion of the labels a second person checks.
- The place your data is held, and which people may open it.
- Whether your files are deleted or sent back when the job ends.
- Whether outside experts have checked how the company keeps files safe, and whether you can read their report. The two checks you'll see most are called SOC 2 and ISO 27001.
Checking the quality of labels
The simplest check is a test set: slip a few files you labeled yourself into the batch, without telling the labelers which ones they are. The number they get right shows their accuracy on work like yours.
Another check: two labelers get the same files, and you see how often they agree. Frequent disagreement usually means your label guide (the document explaining each label) is unclear; a clearer guide fixes most of the problem.
Outsourcing labeling: common questions
Does outsourcing data labeling cost less?
Usually. One labeling company, Habile Data, gives this example: a team in another country charged $40 to $120 to draw boxes around 1,000 objects in pictures. Doing the same with your own staff cost $150 to $400, once salaries, software and checking were added up.
What size should a labeling pilot be?
Large enough that problems show up. Companies ask for 50 to 100 images as a first test; larger paid pilots cover 500 to 2,000 files each.
Will our files be safe with an outside company?
Yes, if you check first. Find out which independent security audits the firm has passed, and have the contract state where your files stay and when they're deleted. The comparison of companies shows which audits each company mentions on its website.
How is outsourced labeling work priced?
Easy-to-count work, like drawing a box around every car in a picture, is priced per label. For adding a few describing words (tags) to each file, you pay per file; for typing out recorded speech, you pay per minute of sound. When the work can't be counted easily, such as people judging the answers an AI chat program gives, you pay by the hour. The pricing guide has published prices for each one.
Send one brief, hear from three vendors
Describe your data once and we pass it to three annotation vendors that label that kind of data, so you can compare quotes side by side.
- One short form instead of three sales calls
- Matched on data type, volume and security needs
- You choose who to talk to