://BacklinkURLs

For agencies

How to vet links at scale: triage hundreds of sites cheaply

Vetting links at scale

Vetting one site carefully takes time. A seller sends a spreadsheet of 800. Nobody is going to do 800 careful reviews, so what usually happens is that the list gets sorted by DR and the top 50 are bought.

There is a better way to cut 800 down to 50, and it costs very little. This guide is the order of operations, with what each stage removed when we ran it.

The principle: cheapest check first

Vetting links at scaleVetting links at scale
Each stage is more expensive per site and sees fewer sites.

Every check has a cost per site, in money or minutes. Run them in rising order of cost, and remove a site at its first fail. A site that redirects to a parked page should never reach the stage where you pay for its traffic history.

The stages

StageChecksCost per siteCan be automated
1. Clean the listRemove duplicates, subdomains and sites you have already rejected.NoneYes
2. Does it loadFetch the homepage. Note errors and redirects to another domain.NoneYes
3. What is itBlog or not, on your subject or not, from the title and recent post titles.NoneMostly
4. Gray topicsSearch every post address in the sitemap for a word list.NoneYes
5. Content samplePosting dates, post length, outbound links in recent posts.NoneMostly
6. Traffic24 months of estimated visits and keywords, in bulk.Paid, smallYes
7. SellerPrice and terms by email.TimePartly
8. JudgmentWould this client be happy to appear here?TimeNo

What each stage removed for us

The saving is plain from the numbers. Had we bought traffic data for every candidate, more than 98 of every 100 lookups would have been spent on sites a free check removes. Your lists will be cleaner than our raw sweep, so your free stages will remove a smaller share, and the order still holds. The full report has every figure.

What to automate and what not to

Bulk link vettingBulk link vetting
Machines for the left column, people for the right.
  • Automate fetching. Status codes, redirects, titles, sitemaps, feed dates. A simple script does thousands an hour.
  • Automate word matching. Gray-topic terms against sitemap addresses. Expect false alarms and review the matches.
  • Automate the traffic rules. Under 500 visits, under 100 keywords, under 30% of peak are arithmetic.
  • Do not automate the last look. Whether a site suits a particular client is a human call.
  • Do not automate away the seller. Terms need reading.
  • Only send survivors. Everything that failed a free stage stays out.
  • Use bulk lookups. Most data providers price a bulk request far below single lookups.
  • Cache results. Traffic history does not change day to day. Keep each result for a couple of months.
  • Set a spending ceiling per month and check it before each batch.
  • Pull once, use often. The same 24 months serve the level, retention, direction and season checks.

The rejected list is an asset

Sellers recycle the same sites across lists, and different sellers offer the same domains. Keep one list of every site you have rejected, with the reason and the date. Stage 1 removes them before anyone spends a second on them.

KeepWhy
DomainThe key for matching.
Stage it failedSo you know how cheaply it can be re-tested.
ReasonGray topics rarely reverse. Low traffic sometimes does.
DateRe-test traffic fails after a year. Never re-test gray-topic fails.

Fitting it into a team

Link vetting triageLink vetting triage
A weekly rhythm for a list of several hundred sites.

Stages 1 to 6 are a job for one person with a script, or a tool, in a few hours. Stages 7 and 8 go to link builders, who now start from a short list worth their time. Sign-off follows the QA process, and what survives feeds the client report.

How to check this yourself

  1. Start your rejected list today, with whatever you have turned down before.
  2. Take the next seller list and remove duplicates and known rejects.
  3. Fetch every homepage and drop errors and off-domain redirects.
  4. Classify by title and recent posts. Drop what is not a blog on your subject.
  5. Run the sitemap word search. Review the matches and drop the real ones.
  6. Send the survivors for bulk traffic data and apply the three traffic lines.
  7. Hand the short list to a person for seller contact and the final look.

Questions buyers ask

What is the cheapest way to vet a large list of sites?
Run free checks first and remove a site at its first fail: does it load, is it a blog on your subject, does its sitemap contain gray topics. Pay for traffic data only on what survives.
Can link vetting be fully automated?
The mechanical checks can. Reviewing gray-topic matches, reading seller terms and judging fit for a client still need a person.
How many sites survive vetting?
It depends on the source. From our broad sweep, 1.8% passed the free checks and about a third of those tested then passed on traffic. A curated seller list will do better and still lose most of its sites.

Sources

  1. BacklinkURLs methodology

Last updated Methodology