Marketers often rely on open-and click rates for quick campaign performance evaluations. Yet these metrics have long been known to be inaccurate. We often believe that things on the internet improve over time. That is not the case with email metrics – they are getting less accurate. Why? Should you be worried and should we abandon these metrics altogether?
About the author: Jakub Olexa is Founder & CEO of Mailkit (Sponsor of the 2024 Email Marketing Automation Summit).
The good old times: accurate open rates and clicks
Opens and clicks, or to be more accurate open rates and click rates, have long been the most important metrics for all email marketers. A single number tells how well a campaign has been performing. A single number to judge the engagement by. The underlying interactions have been the go-to data for recipient targeting, marketing automation as well as list cleaning operations. No matter how inaccurate, there are many use cases for the interaction data.
Over the past couple of years, we have seen these KPIs growing until everything broke – Apple announced its Mail Privacy Protection (Apple MPP for short). That was the moment when open rates skyrocketed. Marketers could no longer trust their data.
But the problem started long before MPP – the steady growth of open rates was one of the symptoms.
For the longest time, the tracking of opens was considered as accurate as it could be. We knew about the inaccuracies caused by tracking images not being loaded, email clients blocking images for security reasons, etc. The overall consensus in the industry was that while inaccurate, the data represented real interactions and therefore could be trusted.
Open and click patterns didn’t add up
Unlike opens, clicks were considered highly accurate. Clicks could not be easily intercepted by email clients (as always, there are exceptions) and provided firm data. Over time, we have started to see more and more emails being opened and links clicked with patterns that did not add up. The opens and clicks occurred at extremely high speed – messages opened immediately after delivery, and multiple links from a single mail clicked in a single second. These were clearly “Non-human interactions”!
We had so many questions… Did Rossum’s Universal Robots* escape to conquer the world of email?! How do we stop them?! How do we make the analytics accurate again?
We soon found out that these Robots opening emails and clicking the links did not come to conquer the email. They came to protect it – from the evil Robots! (Oh yes, there is such a thing as evil Robots – spam, phishing and malware are often sent and controlled by these evil Robots). By opening the messages and following the links inside, they were able to identify phishing attacks and spam campaigns faster and better than before, protecting individual recipients.
The challenge here is twofold – on one hand you have the Non-Human opens which skew the open-rate metrics marketers heavily rely on. On the other hand you have the Non-Human clicks that drive traffic but also automations.
The common denominator: Apple’s Mail Privacy Protection
The industry was fairly quick to identify the most common denominator for Non-Human Opens – no surprise, it was Apple’s MPP. Our data shows that 99% of all Non-Human opens are attributed to Apple. More importantly, over 50% of all email opens in Europe were Non-Human at the end of 2023. The percentage is even higher in the US, where Apple products have larger market share.
While easy to detect, the industry response was not that straightforward. Although the technical understanding is in place, the business priorities are often quite different. Detecting and removing Non-Human opens from reports was not a priority for most platforms. The resulting drop in open-rates would put extra pressure on support staff and could scare away some customers. Businesswise there is only one argument to do the detection – data quality. An argument that is hard to translate to sales and therefore most platforms put this feature in the backlog.
Deciphering Non-Human Interactions
It’s safe to say that if you are an email marketer doing B2C in Europe and your platform doesn’t remove Non-Human interactions from their reports you can divide your open rates by 2. Before you do, check if your platform offers any detection of non-human interaction. If it does there is usually the option to hide/discount such interactions.
Non-Human clicks are a different beast though. These are performed by anti-spam filters, security vendors, etc. to identify abusive links. The impact of such scans vary a lot. In B2C space you can expect around 3-5% of all the clicks to be non-human. In B2B space we have encountered cases where 100% of clicks were caused by scanners. This is easily explained by the fact that each industry segment uses different anti-spam methods.
In B2C most mailboxes are operated by large freemail providers like Google, Yahoo and Microsoft. These systems are built to identify clusters of messages (bulk mail) and scan only a sample group of the messages. Results of the checks on the sample group are then applied to the whole cluster of messages – be it 10 or 10 million messages. Only the links from the sample group will be followed for security check and only those will show up in the reports as clicked.
Anti-spam systems in the B2B space usually can’t reliably cluster messages as even the largest enterprises count their mailboxes in thousands. As a result each message has to be scanned individually. Each message being scanned means links in each message may be subject to being scanned for abusive content. Every link in every message is “clicked” before the message gets delivered to its recipient.
All of these clicks (but opens as well) may trigger automations. Automations that will result in more emails to be sent. This is where the detection becomes extremely complicated. Security scanners are working extra hard to stay undetected – not because they want to mess with click rates but because they need to. They must stay undetected to protect the recipient’s mailboxes. It’s not only about this one campaign, but all the future campaigns and automations.
The end goal: finding a balance of data quality
The goal for us as ESPs is to find the right balance of data quality. We can’t detect all non-human interactions but we should do our best, as these interactions not only affect the reports but also our customer’s costs. Data quality becomes a very strong business case.
Author
Jakub Olexa
Founder & CEO | Mailkit