Comparing the Performance of the Top 6 Analytics SDKs for iOS
Comparing the Performance of the Top 6 Analytics SDKs for iOS
October 4, 2023 by Brian Capps
Estimated read time: 5-10 minutes

The following is a guest post from Brian Capps at Lickability, a premier mobile software studio.

“If you can’t measure it, you can’t manage it.”
- Peter Drucker

Emerge Tools can measure a lot about your app — size, performance, UI changes, and more. However, only you can measure what success looks like for your product. Number of users, retention, conversion: these success metrics are often critical to your business and can be measured using analytics. But with a head-spinning number of analytics libraries available, which one should you choose for your app?

appFigures has analyzed millions of apps in the App Store and provides a handy list of the most popular analytics libraries. It's easy to find a list of features for each library and why you might want to use it. But how does adding an SDK affect your app? What type of overhead do these libraries bring?

We can’t choose an analytics library for you, but we can compare the size and performance implications of different SDKs to your app on iOS. Let’s see how they all stack up!

The Comparison

This post will compare the top 6 analytics SDKs on iOS. Every SDK will be measured against a newly created Xcode project with the default iOS app template (base app). This simple, single-view app is a mere 118.5kb in size with a clean startup path.

We’ll add each library as a dependency through Swift Package Manager and initially configure it as directed (generally calling a setup method from the app delegate). Then we’ll compare the size and startup performance to the base app.

Startup performance comparisons will be done using Emerge's Performance Analysis. Some notes on how it works:

  • Testing is done on physical devices (iPhone SE 2020)
  • For this comparison, startup time is defined as process start to didFinishLaunching (Emerge users can customize markers)
  • The primary metric is the relative change in startup time between the two builds. While we often think of startup in absolute time, the percentage change is more apt for comparisons
  • To come to a statistically significant conclusion, testing is done in a controlled environment and consists of numerous iterations of both the head and base builds (below image)
  • You can refer to our docs for full details on how testing is done
Flame chart of app launch initializing Firebase
Test iterations for Firebase. Iterations are run on a head and base build until a statistically significant conclusion can be reached

Links to our complete analysis are included for each SDK. The following are the 6 most popular analytics SDKs from most to least popular.

Firebase logo

Firebase

Size Impact: +1.6 MB
|
Startup Impact: +30.2% (27 ms)

Firebase from Google is by far the most popular analytics library in use on the App Store. This makes sense, with how many features Firebase offers as an all-service app development platform (database, push notifications, A/B testing, etc.). Our focus is testing the implications of only adding the FirebaseAnalytics product from the package. We are using the latest version, 10.14.0, and here are the results of the analyses:

Flame chart of app launch initializing Firebase
Flame chart of app launch initializing Firebase
Detailed size diff when adding the Firebase SDK
Detailed size diff when adding the Firebase SDK

Note: While appFigures lists Fabric and Google Analytics as popular SDKs, Google has rolled both products into Firebase Analytics, so they have not been included.

Facebook Analytics logo

Facebook Analytics

Size Impact: +1.4 MB
|
Startup Impact: +18.7% (17 ms)

The next most popular SDK is Facebook Analytics. This popularity is also no surprise: Meta and Google are the largest companies on this list, and Facebook has long been a presence in the iOS community (anyone remember Three20?). To set this up, we added the Facebook SDK version 14.1.0 with only the FacebookCore package product for analytics. Here’s what we found:

Flame chart of app launch initializing Facebook
Flame chart of app launch initializing Facebook
Detailed size diff when adding the FB SDK
Detailed size diff when adding the FB SDK

Looking at the flame chart, we can see Facebook accessing the Keychain on the main thread. While these are quite fast, best practice is to put these calls in the background.

Spans showing keychain access on the main thread
Keychain access on the main thread
Flurry logo

Flurry

Size Impact: +1.0 MB
|
Startup Impact: +10.8% (10 ms)

Like many of the top analytics SDKs, Flurry has been around for a long time. The configuration instructions show their age, referencing Objective-C code by default with no mention of SwiftUI or any newer technologies. After some initial trouble with website load issues, we installed version 12.4.0 and got to work on the comparison. You can see it below:

Flame chart of app launch initializing Flurry
Flame chart of app launch initializing Flurry
Detailed size diff when adding the Flurry SDK
Detailed size diff when adding the Flurry SDK
Amplitude logo

Amplitude

Size Impact: +468.5 kB
|
Startup Impact: +23.6% (21 ms)

Amplitude is an analytics platform with detailed insight and experimentation capabilities. Founded in 2011, it’s the newest company on the list. They offer several iOS SDK options, but for this comparison, we looked at the newer, purely Swift beta iOS SDK, version 0.4.14:

Flame chart of app launch initializing Amplitude
Flame chart of app launch initializing Amplitude
Detailed size diff when adding the Amplitude SDK
Detailed size diff when adding the Amplitude SDK
Segment logo

Segment

Size Impact: +718.8 kB
|
Startup Impact: +14.6% (13 ms)

Segment is another platform for analytics and customer data, with a focus on data insight. They were acquired in 2020 and are now a division of Twilio. They offer several iOS SDKs, but we chose their Swift SDK for this measurement. Here’s what we saw on Analytics-Swift version 1.4.7:

Flame chart of app launch initializing Segment
Flame chart of app launch initializing Segment
Detailed size diff when adding the Segment SDK
Detailed size diff when adding the Segment SDK

Part of Emerge's analysis identifies protocols without conformances — dead code. Of all the SDKs, Segment SDK had the most of this type of dead code, but not a significant amount.

Screenshot of Emerge Insight showing protocols without conformances in Segment SDK
Protocols without conformances in Segment SDK
Mixpanel logo

Mixpanel

Size Impact: +422.2 kB
|
Startup Impact: +31.0% (27 ms)

Finally, we have Mixpanel, our last (but certainly not least!) analytics provider. Mixpanel has been a staple of iOS analytics for quite a while, and they offer similar features to Segment and Amplitude. We selected the Swift SDK and tested the latest version, 4.1.4:

Flame chart of app launch initializing Mixpanel
Flame chart of app launch initializing Mixpanel
Detailed size diff when adding the Mixpanel SDK
Detailed size diff when adding the Mixpanel SDK

Results

So how did they all do? Let’s see all the results in a table for easier comparison:

SDKApp Size IncreaseStartup Time Increase (Relative)
Firebase Analytics1.6 MB+30.2%
Facebook Analytics1.4 MB+18.7%
Flurry1.0 MB+10.8%
Amplitude468.5 kB+23.6%
Segment718.8 kB+14.6%
Mixpanel422.2 kB+31.0%

As you can see, there are significant differences in app size and startup time between the SDKs. The most popular SDKs with some of the oldest repositories and many features add the most to your app size, while some of the more focused, less popular entrants are more compact and nimble. This may have been expected, but now we have the data to dig in more.

Firebase, with many years of development and shared core code with its myriad other features, tops the list as the most expensive in size and second in startup time. On the other end of the spectrum, Mixpanel's SDK is nearly 75% smaller than Firebase, while still adding just as much time to your app’s startup. All the others fall somewhere in between, with a few surprises like Flurry’s minimal impact on startup.

Increased size and runtime can affect your bottom line in many ways (and these SDKs will help you confirm that 😉). Aside from impacting the user, adding a megabyte of app size can have the equivalent CO2 emissions for 5 round-trip flights from London to LAX for a globally-scaled app, so it's important to understand what's going into your app.

More than Metrics

While these metrics provide helpful information, they can’t tell you everything. There may be features that your product team needs that only some SDKs offer, cost considerations, and many more factors. For example, Firebase has many other features that are incredibly useful for mobile developers (along with it being mostly free!), and that may be a much greater consideration than any app size or startup cost.

The analytics SDK you choose for your app will be personal for you and your business. Hopefully, these metrics provide helpful context for your team to decide. If you'd like to evaluate the performance implications of a specific SDK, you can create an Emerge Tools account to see our analysis!

You can reach out to Lickability for help implementing your Emerge Tools analysis recommendations!


Share on Twitter

Sign up for our newsletter 🛸

Never miss a post or product update



2024 © Emerge Tools, Inc. All rights reserved.