How to create a huge campaign based on competitors’ keywords without manual labor and collect huge amounts of data?
For example, I will take the database of competitors collected according to the method described in my other article — «How to spy on competitors in Google Ads«.
The advantages of this approach will be:
- The fact that you will get all the real competitors in search advertising, and not just those that you can think of\remember.
- The fact that you will have data on the extent to which your queries intersect with competitors ‘ queries will allow you to select the most relevant ones.
So, the data is in my BigQuery. In the tables we are interested in these fields:
| Field name | Type | Mode |
|---|---|---|
| domain | STRING | NULLABLE |
| keyword | STRING | NULLABLE |
This is enough for us to create a campaign structure.
You need to enable the BigQuery service in Advanced Apis. We will get the data for the script using the following function:
function get_competitors() {
var project_name = '**********', // Project name
dataset_name = '**********', // dataset name
tables_mask = 'AU_IS_*';
var queryRequest = BigQuery.newQueryRequest();
queryRequest.query = 'SELECT COUNT(keyword) AS keys, domain ' +
'FROM `' + project_name + '.' + dataset_name + '.AU_IS_*` ' +
// enter your dates
'WHERE SUBSTR(_TABLE_SUFFIX,-8) BETWEEN "20210121" AND "20210821" ' +
'GROUP BY domain ' +
'ORDER BY keys DESC ' +
'LIMIT 500'; // top-how many competitors do we want to select?
queryRequest.useLegacySql = false;
var query = BigQuery.Jobs.query(queryRequest, project_name);
var values = [];
if (query.jobComplete) {
for (var i = 0; i < query.rows.length; i++) {
var row = query.rows[i];
var domain = row.f[1].v;
if (domain.indexOf('.') > -1) {
values.push(domain);
}
}
}
return values;
}
This function returns competitors’ domains in descending order of search query coverage. It is logical to assume that the more a competitor covers your queries, the more relevant they are to you, and the more interesting their queries are to you.
If you are starting a campaign from scratch and you don’t have a similar data history, you can emulate it, for example, by using the function of collecting competitor ads in Key Collector, or any other similar service.
Next, go to creating the campaign structure. All you need to do in advance is create an empty campaign, which we will fill with groups using the following function:
function buildGroups() {
var campaignSelector = AdsApp
.campaigns()
.withCondition('Name CONTAINS_IGNORE_CASE "Competitors"'); // select this campaign
var campaignIterator = campaignSelector.get();
while (campaignIterator.hasNext()) {
var campaign = campaignIterator.next();
var competitors = get_competitors(); // get competitors' domains from BigQuery
for (var i = 0; i < competitors.length; i++) {
var adGroupSelector = campaign
.adGroups()
.withCondition('Name CONTAINS_IGNORE_CASE "' + competitors[i] + '"');
var adGroupIterator = adGroupSelector.get();
if (!adGroupIterator.hasNext()) {
var adGroupBuilder = campaign.newAdGroupBuilder();
var adGroupOperation = adGroupBuilder
.withName(competitors[i])
.withStatus("PAUSED")
.build();
// create a ad group by the competitor's domain, if there was no such group in the campaign before
}
}
}
}
You can run this function regularly — It will add groups if new competitors ‘ entries appear in the database.
Now the most interesting part is that you need to fill your groups with your competitors ‘ keywords.
To do this, I took data from the SimilarWeb API service. They have a suitable method (I use Desktop Non-Branded Keywords), plus their data seemed suitable to me both qualitatively and quantitatively.
In principle, you can take any service that provides search query data for a domain.
Here is a function for getting keywords by domain by software:
function get_keys(domain) {
var apiKey = '1234567890qwertyuiop'; // API key
var MILLIS_PER_DAY = 1000 * 60 * 60 * 24,
now = new Date(),
fromDate = new Date(now.getTime() - 365 * MILLIS_PER_DAY),
toDate = new Date(now.getTime() - 31 * MILLIS_PER_DAY),
nowDate = new Date(now.getTime()),
timeZone = AdsApp.currentAccount().getTimeZone(),
fromformatDate = Utilities.formatDate(fromDate, timeZone, 'yyyy-MM'),
toformatDate = Utilities.formatDate(toDate, timeZone, 'yyyy-MM');
var url = 'https://api.similarweb.com/v1/website/' + domain +
'/traffic-sources/nonbranded-search?api_key=' + apiKey +
'&start_date=' + fromformatDate +
'&end_date=' + toformatDate +
'&country=world&limit=10000&main_domain_only=false&format=json';
// Logger.log(url);
var options = {
muteHttpExceptions: true,
method: 'GET',
contentType: 'application/json'
};
var response = UrlFetchApp.fetch(url, options);
var json = JSON.parse(response);
if (!!json.meta.status) {
if (json.meta.status == 'Error') {
return null
}
if (json.meta.status == 'Success') {
var keysArr = [];
for (var k = 0; k < json.search.length; k++) {
var key = json.search[k].search_term;
if (key.split(' ').length < 5) {
// I decided to take only phrases shorter than 5 words
keysArr.push(key);
}
}
return keysArr;
}
}
}
At this stage, I take only the original text of the phrase, for the sake of the speed of the script. It would be possible to create phrase and exact matches at once, but then the time for adding keys to the group would be tripled. It is more correct to fill in one option, and only then create the necessary match types from it.
So, we have a function for getting keys by domain — let’s put them in groups.
Here is the function for filling in groups:
function buildKeys() {
var campaignSelector = AdsApp
.campaigns()
.withCondition('Name CONTAINS_IGNORE_CASE "Competitors"');
var campaignIterator = campaignSelector.get();
while (campaignIterator.hasNext()) {
var campaign = campaignIterator.next();
var adGroupSelector = campaign
.adGroups()
.withCondition('Status = PAUSED')
.withCondition('Name DOES_NOT_CONTAIN "_"')
.orderBy('Name ASC');
// Take only ad groups that are still stopped and not processed
// When running the script regularly, this is convenient
// as soon as you start the ad group - the data stops updating
var adGroupIterator = adGroupSelector.get();
while (adGroupIterator.hasNext()) {
var adGroup = adGroupIterator.next(),
adGroupName = adGroup.getName();
adGroup.setName('_' + adGroupName);
// mark the group as processed
var domain = adGroupName;
Logger.log(domain);
var keywordSelector = adGroup
.keywords();
var keywordIterator = keywordSelector.get();
if (!keywordIterator.hasNext()) {
var keys = get_keys(domain);
// get phrases
if ((!!keys) && (keys != null) && (keys.length > +0)) {
for (var n = 0; n < keys.length; n++) {
try {
var keywordOperation = adGroup.newKeywordBuilder()
.withText(keys[n])
.build();
// create phrase
} catch (e) {
Logger.log(e);
}
}
}
}
}
}
}
There are some special features here:
- The script marks the group as processed, even if it was not possible to get keywords for the domain. This is necessary for cases when the competitor’s domain is still new and the spy service has not yet collected data on it.
- For maximum speed, keys are added without any transformation or verification of the success of the add operation.
I should explain why this is the second time I’ve mentioned speed. The fact is that you can get a lot of keywords for large competitors. So much so that adding phrases to just one group will not have time to work out in the allotted 30 minutes.
With the above settings, in about 3 days of auto-running the script, I got a campaign of 500 groups and ~800,000 phrases.
In the future, you can view keywords, duplicate them in the appropriate match types, and add ads to groups and start the group. Or just keep the campaign on pause, and gradually research to take away successful phrases from there to your work campaigns.
