Gmail is Prefetching Images, Leading to Marginally Inflated Opens

The recent changes with Apple Mail Privacy Protection had us wondering – where else is prefetching happening? While false opens come as no huge surprise, we have additional details around the limited set of circumstances in which Gmail is prefetching images in emails sent to Gmail users. 

The Gmail prefetch opens occur in the following circumstances:

1. A Gmail recipient is logged into and has an active session open to the Gmail app (either web or mobile app).

2. An email is sent to the Gmail recipient while their session is active/open.

3. Gmail prefetches all images immediately before the UI displays the email.

4. This image prefetch is in addition to (and different from) Google Image Cache opens, which occurs when the user opens the email.

The image prefetch only occurs when the user is logged into the Gmail application, comes from a Google IP address, and is requested using the following user-agent string:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246 Mozilla/5.0

In investigating billions of open events, we can confidently say that these opens are false opens and do not indicate an actual user open event. These open events are independent and distinct from user-initiated open events triggered by Google Image Cache.

Gmail Prefetch Impacts

What are the implications of the false opens at Gmail? Thankfully, they are minor and nowhere near the Apple Mail Privacy Protection scale. 

In looking at over 9.8B Gmail recipient open events in December 2021, for most senders, we saw that false opens accounted for between 1-6% of open events. What this means is your open rate may be inflated by up to 2 percentage points. For example: If you currently have an overall open rate of 20% at Gmail, your correct open rate would be closer to 18%.  

Your specific false open rate may be significantly higher or lower than what we are reporting above. Because, the false opens are triggered based upon when users are using the Gmail application, your specific audience’s behavior and use-cases are the primary factors in how much you will be impacted by this anomaly.  

How to Detect and Ignore Gmail Prefetched Opens

For SparkPost senders, we have you covered. We have already updated our events API and event webhooks to automatically identify these Gmail Prefetch events using the newly introduced is_prefetched flag. We are also actively working to add the ability to distinguish prefetched and proxy opens in our Analytics Report UI and Metrics API. Stay tuned for future updates regarding the Report UI enhancements.

For others, detecting Gmail prefetch opens is still relatively straightforward. For each open event, you will want to ignore (or uniquely tag) any open event that matches the following user agent string:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246 Mozilla/5.0

We have been able to confirm that this string is unique to Google’s Prefetch Bot. 

Gmail Prefetch Detailed Analysis

As detailed above, Gmail prefetching only occurs in a limited set of circumstances. Prefetching does not occur with other mail clients. Instead, this behavior is specific to when a Gmail user has the Gmail app open within their web browser or is actively using the mobile app. Our best guess is that it’s a security scan before showing the email to the user in their browser.

The full request headers for the image request are detailed below. A few things you will notice:

• The referer is set to http://mail.google.com. Interestingly, even though the user is on https://, Gmail still sets the referer to the http:// protocol when making the request. 

• The request comes from Gmail’s servers and not the user’s browser. The client IP always resolves to a Google-owned IP space.

• Unlike Google Image Cache, the user-agent string does not identify that the request is coming from one of Google’s bots. Instead, the user-agent string looks like an actual user image request. However, we have confirmed that this user-agent string does identify the Google prefetch bot.

• The open request happens within seconds of the email delivery. Further, the request occurs before the email appears in the user’s Gmail interface. This behavior leads us to believe the request is for security purposes.

• The prefetch seems to only happen once per unread Gmail email thread. In our extensive testing, once a message was read by the user, any future emails that went into that thread group did not initiate a prefetch request.

• This prefetch is separate from Google Image Cache. Our testing indicates that even after the image is prefetched, a separate Google Image Cache request is made when the user opens the email.

• If a user has the Gmail mobile app open, the prefetch will continue to happen for a short period of time, even after closing the mobile app.

Here is an example of what the request headers will look like when an image is requested from the Google Prefetch Bot:

  headers: {

    host: ‘{redacted}.m.pipedream.net’,

    ‘x-amzn-trace-id’: ‘Root={redacted}’,

    ‘accept-language’: ‘en-US’,

    referer: ‘http://mail.google.com/’,

    accept: ‘image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8’,

    from: ”,

    ‘user-agent’: ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246 Mozilla/5.0’,

    ‘accept-encoding’: ‘gzip, deflate, br’

   },

   body: {},

   inferred_body_type: ‘FORM’,

   method: ‘GET’,

   url: ‘https://{redacted}.m.pipedream.net/header-1641a1.gif’,

   client_ip: ‘66.249.92.1’,

   query: {}

As is the case with Apple’s Mail Privacy Protection, senders should treat all open events with care. Opens are just one, and often not the best one, of the many engagement metrics that senders should be monitoring and including when making determinations about user engagement.