Algorithm driven Online Ad Optimization

Those who have done or have been doing SEM since it became popular last decade, would understand the importance of Pay Per Click Bid Optimization. How allocation of SEM budgets and CPC bidding tactics had to be applied in near real time across hundreds of thousands of Keywords, to maintain aggressive CPAs even while trying to meet Conversion targets. During initial years of SEM, which was specifically second half of last decade(2004-2010): 

• Competition was not tight 
• CPC prices were lower and
• Generating Leads was much easier.  

Now in 2013, all these 3 points have changed and 

• Competition has become bigger both in breadth and depth.
• CPC prices have increased, skyrocketed in some Industry Verticals.
• Generating qualified Leads has become even more difficult.

So manual intuitive bidding tactics or using Off-the-shelf bidding software is no longer useful.

Also other forms of Online Marketing like Online Display Ads and Email Marketing have seen huge improvements in Conversion Rates (%) over the years, due to application of successful Ad Re-Targeting algorithms. Just few years ago, SEM & SEO were the undisputed champions of Online Marketing. However now Algorithm driven Re-Targeted Online Display Ads are increasingly eating away Conversions from SEM & SEO. Google itself is promoting usage of Re-Targeted Online Display Ads to its Adwords accounts, offering multiple Ad formats apart from mainly Search Ads few years ago. 

Also new Online Channels like Twitter and Facebook have become more popular, though they are yet to prove their impact in Online Advertising, as their Online Ad share in still minuscule compared to Google Search Ads or Display Ads or even Email Marketing.

To summarize the above points

1. Changing Economic Cycles
2. Increasingly intense competition for generating qualified Leads or Customers Online
3. Ever rising CPC prices in SEM
4. Re-Birth of Online Display Ads and even Email Marketing as very reliable high Conversion Online Channels (both these had been written off few years ago) and 
5. Finally proliferation of new Online Channels like Twitter or Facebook etc.

.......Have all increased the sophistication of Online Marketing. It is no longer efficient or effective to manage Online Ad Campaigns by throwing some internal teams and some external agencies. 

Welcome to the world of Algorithm driven Online Ad Optimization. The need of today, is to develop Algorithms which can process massive volumes of Online Marketing content and data in real time to effectively allocate Online Ad budgets across Online Channels, identify right mix and delivery of targeted Online Ads at right time to prospects to meet Conversion targets and maintain ever more aggressive CPAs.

Just like in Financial Services sector, where savvy Hedge Funds use sophisticated Algorithms to allocate Clients budgets optimally across stock or asset portfolios, even Online Ad sector needs to create sophisticated Algorithms to allocate Account budgets optimally across Online Channels or Ad portfolios. It's just the same.

Signature: Roopkumar T.V.

Google's Online Ad Re-Targeting based on Post-Search Behavior of Users

What is similar between the below 4 screenshots of my web sessions :

• In the 1^st browser window, I am searching in YouTube for Euro Cup top 10 goals.
• In the 2^nd  window, I am watching a video on Euro Cup top 10 goals in YouTube.

Then I shut down my PC, and after an hour again start my PC, then open my web browser and start browsing some newspapers online.

• In the 3^rd window, I am reading an article in Eenadu, very popular Telugu language newspaper online.
• In the 4^th window, just out of curiosity I open a popular Tamil language daily magazine online, Dinamalar.com

If we notice all these web browsing sessions carefully, something is very strange. Every website, I had opened that evening, were displaying a particular Ad on “San Francisco Car Rentals”. This was strange.

I was also in Bangalore, India on that day.

What’s wrong here -

I am in Bangalore, then why should I get an online Display Ad on “San Francisco Car Rentals” on almost every website or YouTube video that I browsed to or watched on that day.  Definitely car rentals in San Francisco are not relevant for a Bangalore, India based browser who is hundreds of thousands of miles faraway.

Now I open DeccanHerald.com, very popular English language newspaper among old timers of Bangalore, and we can see a similar Display Ad “Car Hire SFO Airport” even on this website.

Definitely Ads on “San Francisco Car Rentals” or “SFO Airport Car Hire” are not relevant when I am watching a

1. YouTube video on Euro Cup soccer top 10 goals. At least if Display Ads for Bus timings in Paris or Amsterdam was displayed while watching this video on European soccer we can understand, but what is a far away San Francisco, USA Car Rentals company’s online Ad doing on a YouTube video about Euro Cup soccer top 10 goals.
2. India speaks many different languages, fine – but every language newspaper online whether Telugu, Tamil or English was displaying this Ad by this San Francisco, USA Car Rentals company. Again this online Ad is not relevant on these newspaper websites.

So what’s wrong here – If YouTube alone is not doing a good job of mapping Ad relevancy of its videos, we can understand; but how can many other websites in different languages belonging to totally different media companies also do the same mistake of showing the same irrelevant Ad? 

Actually nothing is wrong here. Let me give a clue, all these different websites that I had visited, from YouTube to regional language news websites were opened using the same web browser Google Chrome.

Welcome to the home of Ad Re-Targeting on Google’s Online Advertising platform.  

Actually on that same day, early in the morning I had searched for "San Francisco Car Rentals" in Google.com using Google Chrome web browser, since I was planning a business trip 2 weeks later to San Francisco. I had clicked on a Google Search Ad of a Car Rentals company, browsed to its website and viewed different models of cars and after selecting a model, decided to book it. Then after clicking on “Book this car” submission, on this Car Rentals website had decided to abandon this transaction midway without actually booking, as I wanted to check out other car hire deals on other Car Rental websites and also wanted to book closer to my trip date.

Google had tracked this entire activity of mine done early in the morning on that day, from the time I had searched for "San Francisco Car Rentals"  till I had abandoned the booking on that Car Rentals website, and everything in between. I am not sure, but Google might have even tracked the models of cars I had viewed on this Car Rentals website, the dates selected for car hire and even the car pickup location. I am sure about the last one, since later in that day I had seen an Ad on DeccanHerald.com on “Car Hire SFO Airport”. Therefore for Google’s Online Ad algorithms, this was a strong user intent towards a conversion, as I had tried to almost book a rental car before abandoning midway.

Hence these algorithms have decided to Re-Target me with online Display Ads on “San Francisco Car Rentals“ every time I was browsing through other websites or watching YouTube videos using Google Chrome browser. If I used a different web browser the next day or next week, I wouldn't see these Re-Targeted ads. As long as I am using Google’s Chrome web browser – for the next few days perhaps for few weeks I am going to frequently see this SFO Car Rental Ad in almost all my web browsing sessions, as I have become a good qualified lead for SFO car rentals.

I had even searched for "Hotels in San Francisco", on that same morning but did not try to book any hotel (as my hotel booking has to be done through my Employer’s Contractor only) and only wanted to check out the prices. For Google’s Online Ad algorithms, this was not a strong user intent towards a conversion – since I had not tried to book a hotel on the Hotel Booking website after clicking through the Google Search Ad. Therefore in case of San Francisco hotel booking, these algorithms have decided not to Re-Target me with online Ads as I am not a good prospect for SFO hotel booking.

Using the Visitor Identification and Browse behavior (including the Keywords searched in Google.com and the specific intent expressed by users on various websites visited post-search) Google’s powerful Online Ad algorithms are capable to Re-Target Ads to Visitors through the cookies stored in Google Chrome browsers. These algorithms typically execute on Big Data Analytics Architecture as discussed previously.

Signature: Roopkumar T.V.

Big Data Analytics solutions for Online Marketing - Use Case 1

A sample Online Marketing application deployed in the Big Data Architecture, is shown below.

Online users search for products, services, topics of interest etc. not only in Google and other search engines, but also more importantly on site itself (For example, in eCommerce site Amazon.com, search is the top product finding method used by site visitors). Facilitating searchers by providing relevant search results is something online search providers like Google, Bing and also site search providers continuously optimize and calibrate.

From an Online Marketing perspective, once the searchers click through the search results and arrive at the website (if coming through external search like Google) or arrive at the product or topic page they were searching internally on the site, that page of arrival from a search result, called as landing page in Online Marketing terminology, is very important for:

• Improving Conversion Rate (%) of the site.
• Traffic dispersion to subsequent stages of the site.
• Improving site engagement for the users 

As already discussed in a previous post, delivering dynamic and search relevant landing pages is very important, particularly for large websites like eCommerce stores, Music & Movie download sites, Travel websites etc.  While delivering keyword or search relevant landing pages dynamically across thousands of keywords, perhaps across hundreds of thousands of keywords for large websites, itself is a big challenge; even bigger challenge is to deliver these dynamic, search relevant landing pages targeted to each of different user segments. As already discussed previously, luckily Big Data Analytics solutions are available now to solve these Big Data challenges in Online Marketing.

Large websites generate and also need to process, huge volumes of different varieties of data as below:

• Website clickstream data collected through Web Analytics applications like Omniture and from webserver logs.
• The website content such as product content, marketing content, navigation etc. in various formats like text, images, videos etc. which is available in the web content management systems.
• External web content typically collected by web crawlers, which includes content such as
• Product content from competitor websites
• Marketing collaterals from external industry websites etc.
• User generated content such as product reviews, user survey feedback, social media posts, online discussions, tweets, blog posts, online comments, Wiki articles etc.

Most of the above varieties of data are unstructured or semi-structured, and hence cannot be collected and processed in traditional RDBMS databases like Oracle or MySQL.

For large websites, it is not just important to collect large volumes of variety of data as shown above, but it is also important to handle the velocity at which all these data is getting generated online, particularly clickstream data and user generated content.

This is where Big Data Analytics solutions come in. In this above example, a typical Architecture to support Big Data Analytics is solutioned using open source Apache Hadoop framework.  In an Hadoop architecture - big volumes, variety and velocity of online data are collected and then stored in HDFS file system. Hadoop architecture also provides RDBMS like databases such as HBase, for storing big data in traditional style, particularly useful for beginners and new users of these Big Data Architectures. As we can see in this example, a big data landing zone is set up on a Hadoop cluster to collect big data, which is then stored in HDFS file system.

Using Map-Reduce programming method, Online Marketing Analysts or Big Data Scientists or Analysts develop and deploy various algorithms on a Hadoop cluster for performing Big Data Analytics. These algorithms can be implemented in standard Core Java programming language which is the core programming language used for executing various services for collecting, storing and analyses of big data in a Hadoop architecture.  Additional programming languages like Pig, Hive, Python or R can be used to implement the same algorithms with less number of lines of code to be deployed. However code written in any of these additional languages would still be compiled into Core Java code by Java Compilers for execution on Big Data Hadoop Architectures.

Some of the use cases of Online Marketing Algorithms which can be implemented on Hadoop Architecture for deriving Analytics are shown in the same example. All these algorithms are deployed using the Map-Reduce programming method.

• Keyword Research: Counting the number of occurrences in content and search for hundreds of thousands of keywords across the diverse variety of data collected into Hadoop and stored in HDFS. This algorithm would help identify top keywords by volume, and also the long tail of hundreds of thousands of keywords searched by users. Even new hidden gems among keywords can be discovered using this algorithm to deploy in SEM/SEO campaigns.
• Content Classifications / Themes: Classify the user generate content and also web content into specific themes. Due to huge processing capabilities of Hadoop Architecture, huge volumes of content can be processed and classified into dozens of major themes and hundreds of sub themes.
• User Segmentation: Individual user behavior available in web clickstream data is combined with online user generated content and further combined with user targeted content available in web content management systems to generate dozens of user segments, both major & minor segments. Further this algorithm would identify the top keywords and right content themes targeted for each of the dozens of user segments, by combining the output from other algorithms used for Keyword Research and Content Classifications.

Also, since the Hadoop Architecture is running on clusters of computers, all the above algorithms can not only process huge voluminous amounts and varieties of data, but can handle data in motion which keeps coming into the Hadoop Big Data landing zone in near real time. This would enable the Online Marketing Campaigns to be tweaked in near real time to derive better ROIs from Online Marketing spends.  In the example illustrated above, the output from the 3 algorithms running in parallel, is dynamic Keyword Relevant Content Rich User Targeted Landing Pages generated in near real time, for hundreds of thousands of keywords, across dozens of content themes and targeted across dozens of user segments. This output would be integrated with eCommerce platforms or Web Content Management Systems or with Web Portals for creation, production & delivery of Keyword Relevant Content Rich User Targeted Landing Pages in near real time.


Signature: Roopkumar T.V.

Dynamic and Search Keyword Relevant Landing Pages

One of the biggest challenges faced by Large Dynamic websites like eCommerce Stores, Music or Movie download websites, Travel & Hospitality websites etc. is that they don't always deliver the most relevant landing pages to the Visitors who arrived at their website from Google and other search engines.

Recalling a much simpler earlier post on this topic, the disadvantages of not delivering the most relevant landing pages to users who arrived from Google or other search engines would include

• Lost sales or lead generation opportunity
• Lost opportunity to build engaging long term customer relationships or customer loyalty
• Bad reputation and negative feedback, even negative reviews
• Lost investments on the website

A landing page is the first touch point on the website for users coming from search engines. Users will not spend more then 10 seconds on a website, which has a irrelevant landing page. Capturing the users interest in the first 10 seconds is very important, and this is only possible by delivering the most relevant content to Visitors consistently.

The large dynamically changing websites would be searched and found in search engines like Google, across thousands of keywords. The top searched keywords would keep changing for large dynamic websites each month, or perhaps even each week. Also there would be a long tail of thousands of keywords, in some cases hundreds of thousands of long tail keywords for large websites. Hence delivering Search Keyword Relevant Landing pages across thousands, perhaps hundreds of thousands of keywords is always a challenge for large dynamic websites.

Good news is that solutions are now available to help large dynamic websites, to always deliver Search Keyword Relevant Landing pages across hundreds of thousands of keywords. All these solutions leverage the Big Data Analytics Platforms. 

Big Data Analytics Solutions would help organizations to always deliver Dynamic and Search Keyword Relevant Landing Pages across hundreds of thousands of keywords always and for every search.

Big Data Analytics Platforms benefit Organizations in discovering potential keywords for SEM and SEO by 

1. Scanning or crawling all their content found anywhere on the internet, discovering hundreds of thousands of potential keywords for which their content (websites, images, videos, mobile apps, social apps, pages, blogs, Facebook fan pages etc.) could be discovered in Google search. 
2. Scanning or crawling the content of direct competitors to discover additional potential thousands of keywords.
3. Scanning similar content across other websites, blogs, social media, mobile or social apps, images, videos etc. to discover the long tail of potentially hundreds of thousands of keywords.

By integrating these Big Data Analytics Platforms to their Web Portals, E-Commerce Platforms, Content Management Systems, Business Process Management systems etc. - Organizations can deliver Dynamic and Search Keyword Relevant Landing Pages across hundreds of thousands of keywords always and for every search. This methodology would be discussed in more detail in future posts.

Signature: Roopkumar T.V.

Online Multi-Channel Campaign Attribution

While I prefer to write my own posts, based on my own experiences - This post is credited to the official Google Analytics website.  This post and the embedded video from Google Analytics team provides an overview of online multi-channel campaign attribution in most simple and effective way then anywhere on the web.

Original link to Google Analytics site on online Multi-Channel Campaign Attribution.

Signature: Roopkumar T.V.

Relevant Landing Pages for Search keywords

I will start my first post with a simple and effective method to improve the user experience for searchers who come to your website from Paid search campaigns using search engines like Google. Give the users the most relevant landing page.

If the user searched using a generic keyword or brand keyword, it would mean the user is still in awareness stage of purchase life-cycle. Send these users to home page.

If the user searched using specific keywords, related to specific categories in the content or product hierarchy of the site, direct them to relevant category pages. Users are mostly at a selection stage of purchase life-cycle.

If the user searched using very specific product names or for very specific content, it would mean the user has reached the buy stage of purchase life-cycle. Direct these users to most specific inner landing pages in the site serving the very specific content or products searched by these users.

Following the above points, would ensure that the first step in the digital user experience of a user coming from search engines is taken care to

1. Reduce bounce rates of landing pages
2. Increase next page traffic dispersion to subsequent pages from the landing pages.

Signature: Roopkumar T.V.