e-commerce insights and solutions

apps

Cart Bots

gallery/robot-916284_960_720

Going live with a new application is always an adventure, and one thing we did not expect with Cart Insights was the high volume of bot scripts running visiting our customer's Shopify e-commerce shops.  We have been monitoring the webhooks in production for a few months, and have deployed several filtering tactics to remove the bot cart activity from our customer's cart reporting.  Here are some details that we hope will inform and answer your questions:

 

What are bots?

 

There are all sorts of automatic scripts and programs that visit websites and perform various functions, and these are usually known as robots or bots.  Some bots are used by search engines such as Google to scan the contents of a website, check links, check for content updates, categorize and rank.  Other bots are used to scrape or read data for various purposes such as collecting statistics or creating databases of data for travel websites, statistical charts, background checks, etc.  And in the context of e-commerce, there are of course all types of specialized bots.

 

Here are some pertinent links for further reading:

 

https://www.akamai.com/us/en/multimedia/documents/white-paper/akamai-scrapers-and-bot-series-managing-professional-bots-white-paper.pdf
 

https://datadome.co/bot-management-protection/top-5-bot-threats-for-the-online-retail-industry/

 

https://www.shieldsquare.com/this-is-what-happens-when-bots-influence-cart-abandonment-in-ecommerce-infographic/

 

https://www.digitalcommerce360.com/2016/12/16/how-hoarder-bots-steal-sales-online-retailers/

 

 

Why are bots visiting Shopify e-commerce websites and adding cart items?

 

Most Shopify e-commerce websites are not going to be the targets of a Denial-of-Inventory attack, and they are not likely to be visited by a custom bot service for the purposes of trying to purchase an in-demand concert ticket or a limited-edition sneaker.  Since we have observed that bots are targeting only specific Shopify websites, we believe those bots are being set up and deployed to generate clicks, to generate website traffic for referral links, and to generate shopping cart activity in order to superficially inflate the efficacy of various paid e-commerce marketing, traffic referral, and review services and apps.

 

Whether a Shopify site will have bot activity is going to depend on where they have links placed, where their traffic is coming from, the type of products sold, and the methods they are using to generate clicks and traffic.  Since we do not have visibility to something like Google Analytics for our customers, we simply cannot be sure.

 

 

gallery/dyno load

What Can Be Done about Bots?

 

It is going to be the burden of Shopify to manage the traffic and increased load on their customer shops, and we are certain they have tools in place to manage and monitor traffic at the packet level.  We have seen certain customer websites where a spike in bot activity is following by a significant reduction and relative calm, but it's not possible to say whether the bots have moved on or whether traffic management has been deployed by Shopify's team in response with filtering tactics using rate controls, behavior and time analysis to identify and manage bot traffic.

 

Shopify website owners and managers who suspect bot activity are advised to monitor page load times, for example, in case bot traffic is impacting the customer experience and in turn reducing conversion rates.  Those websites that do reduce inventory upon cart adds should be especially cautious to ensure that bot activity is not causing inventory to show as unavailable to actual customers.

 

Traffic Quality and Marketing Strategies

 

As e-commerce has matured and gained significant market share, the services, platforms, and agencies that promise to deliver better search ranking, more traffic and increased sales have also proliferated.  And with these ever-growing options, an entire landscape of bots and scripts has been created to attempt to increase and inflate the statistics of traffic and clicks that are used to prove or validate the efficacy of a given service, or to damage a competitor.

 

Traffic and clicks is of course a costly expenditure for an e-commerce operation, and most marketing professionials will observe if a specific service is effective by tracking converstion rates.  We would also advise the tracking of the entire funnel to identify those services that may be drawing bots towards a website.  Additionally such bot traffic may negatively impact organic traffic by reducing a website's search ranking.  

 

How do Cart Bots affect the Cart Insights app?

 

Since Cart Insights uses webhook subscriptions to receive website activity including cart and checkout creation and update webhooks, we have a limited amount of data sourced from the webhook JSON available in order to logically filter out bot activity in a shop, such as tokens and timestamps.  However these are not always effective as each bot script has its own behavior and some are more intelligent than orders, for example with regard to timing.  To some extent we are dependant on Shopify to hopefully improve their own filtering technologies so that only legitimate webhooks are generated and sent to subscribers, or to add additional webhook fields that will allow us to improve our filtering logic.