Files
purchases-ios/Sources/Networking/RemoteConfigSourceProvider.swift
Toni RicoandClaude Fable 5 8925524293 feat(remote-config): health-check-gated API source failover in HTTPClient (#7294)
* feat(remote-config): health-check-gated API source failover in HTTPClient

Wires the API source failover into the request pipeline, still behind the
usesRemoteConfigAPISources dangerous setting (internal and defaulting to
false, so released behavior is unchanged):

- Requests resolve their API source once (in start) and carry the handle,
  so a failure can report the exact source unhealthy and ETag/backoff
  retries keep targeting the same host.
- On a failure that points at the host, the source is health-checked
  before failing over: a healthy source surfaces the original error, an
  unhealthy one is reported and the request transparently retries on the
  next source (capped at 5 source attempts per request). The static
  per-endpoint fallback hosts stay the last resort.
- Unlike Android, device-connectivity errors (reusing
  NetworkError.isAllowedToRetryWithFallbackHost) skip the health check
  entirely and surface the original error: switching hosts can't fix a
  device that is offline.
- Both Backend HTTPClients share one APISourceFailover, so they walk one
  source list and share one health-check cache.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(remote-config): end-to-end API source failover integration tests

Port of purchases-android's BackendAPISourceFailoverIntegrationTest: real
Backend.getCustomerInfo through the production wiring (real HTTPClients,
shared APISourceFailover, real SourceHealthChecker probing over the URL
loading system) and a real RemoteConfigSourceProvider fed by an in-memory
sources topic, with OHHTTPStubs standing in for the source hosts and
recording exact per-host request order. Adds the iOS-only
device-connectivity scenario and a dangerous-setting-off wiring check on
top of Android's seven scenarios.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: model the request's API source as a single state enum

Replaces the three loosely coupled Request fields (apiSource,
apiSourceResolved, apiSourceAttemptCount) with one APISourceState enum
(unresolved / noSource / source(_, attemptCount:)), making inconsistent
combinations unrepresentable. No behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: adopt the narrowed APISourceFailover initializer

Passes just the usesRemoteConfigAPISources flag at the two construction
sites, following the signature change in the base branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: clarify why static fallback still runs when the source is healthy

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(remote-config): raise the API failover restart interval to 2 hours

The api source list now returns to the primary source 2 hours after it
first failed over instead of 10 minutes. Timer tests derive their time
values from the constant so future changes are single-line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(remote-config): cover shared failover and probe-in-flight ordering

Addresses review feedback:
- Both HTTPClient lanes share one APISourceFailover: a failover on the
  main lane advances the source list the remote-config lane resolves.
- A request enqueued while a health check is in flight stays queued
  until the probe resolves and runs after the front-queued failover
  retry.
- Bind the source and attempt count in one pattern match at the
  failover guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: drop the APISourceState.attemptCount accessor

Its only caller pattern-matches instead, surfacing the "must already
target a source" assumption at the one place it matters rather than
papering over impossible states with a 0 default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-28 17:25:33 +02:00

363 lines
14 KiB
Swift

//
// RemoteConfigSourceProvider.swift
// RevenueCat
//
// Created by Antonio Pallares on 26/06/2026.
// Copyright © 2026 RevenueCat, Inc. All rights reserved.
import Foundation
/// A remote config source: a URL plus the metadata used to order sources.
struct RemoteConfigSource: WeightedSource {
/// A plain URL or a URL format with placeholders (e.g. `{blob_ref}`), to be resolved by the caller.
let url: String
let priority: Int
let weight: Int
}
/// A source handed out by a `RemoteConfigSourceProvider`, tagged with its purpose (api or blob).
/// Report it back via `reportUnhealthy(_:)` to fall back to the next source. The opaque `token`
/// is its identity: a report is ignored once the provider has moved past it (via fallback or
/// `restart(for:)`), so stale or concurrent reports can't advance the order more than once.
struct RemoteConfigSourceHandle {
/// What the source is used for: calling the config api or downloading a blob.
enum Purpose {
case api
case blob
}
let purpose: Purpose
let source: RemoteConfigSource
/// Identifies the handout that produced this handle. Opaque to callers; only meaningful to the
/// provider, which uses it to discard stale reports.
fileprivate let token: Int
/// A plain URL or a URL format with placeholders, to be resolved by the caller.
var url: String { self.source.url }
fileprivate init(purpose: Purpose, source: RemoteConfigSource, token: Int) {
self.purpose = purpose
self.source = source
self.token = token
}
}
protocol RemoteConfigSourceProviderType: AnyObject {
/// The current healthy source for `purpose`, or `nil` once all of its sources are reported unhealthy.
func getCurrent(for purpose: RemoteConfigSourceHandle.Purpose) -> RemoteConfigSourceHandle?
/// The current healthy API base source, or `nil` once every API source has been reported unhealthy.
func currentAPISource() -> RemoteConfigSourceHandle?
/// Falls back to the next source for the handle's purpose. No-op if `handle` is no longer current.
func reportUnhealthy(_ handle: RemoteConfigSourceHandle)
/// Rewinds the given purpose to its first source, e.g. to start fresh on a new fetch cycle.
func restart(for purpose: RemoteConfigSourceHandle.Purpose)
/// Rewinds the given purpose only if every known source for it has already been exhausted.
@discardableResult
func restartIfExhausted(for purpose: RemoteConfigSourceHandle.Purpose) -> Bool
}
extension RemoteConfigSourceProviderType {
func currentAPISource() -> RemoteConfigSourceHandle? {
return self.getCurrent(for: .api)
}
}
/// The address book for remote config: hands out the current healthy api and blob sources and
/// falls back to the next one when a source is reported unhealthy. Each purpose fails over
/// independently.
///
/// Reads the `sources` topic lazily from the topic store and rebuilds its ordered lists only when
/// that topic changes: an unchanged topic keeps failover progress, while a changed one restarts both
/// lists from the top. While the topic has no usable api sources, it falls back to an embedded
/// default so the SDK can reach the config api before any config is fetched. Blob sources have no
/// embedded default: they are only useful alongside a fetched config, which carries its own. Sources
/// are deduped by url and ordered via `WeightedSourceSelector`.
///
/// The api list additionally restarts on its own once `apiFailoverRestartInterval` has elapsed since
/// it first advanced past the top, so the SDK periodically returns to the primary source (or re-arms
/// an exhausted list) instead of sticking on a backup forever. During a persistent outage this
/// retries the primary at most once per interval. Blob failover has no such timer: it is restarted
/// per fetch cycle by its callers.
///
/// - Note: Thread-safe.
final class RemoteConfigSourceProvider: RemoteConfigSourceProviderType {
private static let apiItem = "api"
private static let blobItem = "blob"
private static let sourcesKey = "sources"
private static let urlKey = "url"
private static let urlFormatKey = "url_format"
private static let priorityKey = "priority"
private static let weightKey = "weight"
// Embedded api defaults used until a `sources` topic is fetched, so the SDK can always reach
// config. Their very high `priority` numbers keep them below anything a fetched topic provides
// (lower number wins), so they only act as a fallback.
private static let defaultAPISources = [
RemoteConfigSource(url: "https://api.revenuecat.com/", priority: 100_000, weight: 1),
RemoteConfigSource(url: "https://api.rc-backup.com/", priority: 100_001, weight: 1)
]
// Visible for tests.
static let apiFailoverRestartInterval: TimeInterval = 2 * 60 * 60
private let topicStore: RemoteConfigTopicStoreType?
private let randomizer: WeightedSourceRandomizer?
private let dateProvider: DateProvider
private let lock = Lock()
/// Topic the current failovers were built from. `nil` means there is no sources topic (absent, or
/// none seen yet), in which case the failovers hold the embedded defaults.
private var sourcesTopic: RemoteConfiguration.ConfigTopic?
private var api: SourceFailover
private var blob: SourceFailover
/// When the api failover first advanced past the top of its list, or `nil` while it sits at the
/// top. Stamped on the first advance (not the latest) so a flapping source can't postpone the
/// periodic return to the primary indefinitely.
private var apiFailoverStartedAt: Date?
init(
topicStore: RemoteConfigTopicStoreType?,
randomizer: WeightedSourceRandomizer? = nil,
dateProvider: DateProvider = DateProvider()
) {
self.topicStore = topicStore
self.randomizer = randomizer
self.dateProvider = dateProvider
self.api = SourceFailover(
purpose: .api,
sources: Self.dedupe(Self.sources(from: nil, for: .api)),
randomizer: randomizer
)
self.blob = SourceFailover(
purpose: .blob,
sources: Self.dedupe(Self.sources(from: nil, for: .blob)),
randomizer: randomizer
)
}
func getCurrent(for purpose: RemoteConfigSourceHandle.Purpose) -> RemoteConfigSourceHandle? {
return self.lock.perform {
self.rebuildIfNeeded()
self.restartAPIIfExpired()
return self.failover(for: purpose).current
}
}
func reportUnhealthy(_ handle: RemoteConfigSourceHandle) {
self.lock.perform {
// Rebuild happened, no need to report unhealthy
if self.rebuildIfNeeded() { return }
self.restartAPIIfExpired()
let advanced = self.failover(for: handle.purpose).reportUnhealthy(handle)
if advanced, handle.purpose == .api, self.apiFailoverStartedAt == nil {
self.apiFailoverStartedAt = self.dateProvider.now()
}
}
}
func restart(for purpose: RemoteConfigSourceHandle.Purpose) {
self.lock.perform {
// Rebuild happened, no need to restart
if !self.rebuildIfNeeded() {
self.failover(for: purpose).restart()
if purpose == .api {
self.apiFailoverStartedAt = nil
}
}
}
}
@discardableResult
func restartIfExhausted(for purpose: RemoteConfigSourceHandle.Purpose) -> Bool {
return self.lock.perform {
if self.rebuildIfNeeded() {
return true
}
self.restartAPIIfExpired()
let failover = self.failover(for: purpose)
guard failover.current == nil else { return false }
failover.restart()
if purpose == .api {
self.apiFailoverStartedAt = nil
}
return true
}
}
/// Restarts the api failover once `apiFailoverRestartInterval` has elapsed since it first
/// advanced, so a backup (or an exhausted list) periodically retries the primary source. The
/// restart bumps the token, so handles from before it can no longer advance the list. Callers
/// must hold `lock`.
private func restartAPIIfExpired() {
guard let startedAt = self.apiFailoverStartedAt,
self.dateProvider.now().timeIntervalSince(startedAt) >= Self.apiFailoverRestartInterval else {
return
}
self.api.restart()
self.apiFailoverStartedAt = nil
}
private func failover(for purpose: RemoteConfigSourceHandle.Purpose) -> SourceFailover {
switch purpose {
case .api: return self.api
case .blob: return self.blob
}
}
/// Rebuilds both failovers from the latest `sources` topic when it changed, returning whether a
/// rebuild happened. Callers must hold `lock`.
@discardableResult
private func rebuildIfNeeded() -> Bool {
let topic = self.topicStore?.topic(.sources)
guard topic != self.sourcesTopic else { return false }
// Seed the new generation past any token the previous one could have handed out, so reports
// left over from before the rebuild are ignored instead of advancing the freshly-restarted list.
let nextToken = max(self.api.currentToken, self.blob.currentToken) + 1
self.api = SourceFailover(
purpose: .api,
sources: Self.dedupe(Self.sources(from: topic, for: .api)),
randomizer: self.randomizer,
initialToken: nextToken
)
self.blob = SourceFailover(
purpose: .blob,
sources: Self.dedupe(Self.sources(from: topic, for: .blob)),
randomizer: self.randomizer,
initialToken: nextToken
)
self.sourcesTopic = topic
self.apiFailoverStartedAt = nil
return true
}
/// The sources for `purpose`, parsed from the `sources` `topic`. Api falls back to the embedded
/// default while the topic has no usable api sources; blob has no default, so it can be empty.
private static func sources(
from topic: RemoteConfiguration.ConfigTopic?,
for purpose: RemoteConfigSourceHandle.Purpose
) -> [RemoteConfigSource] {
switch purpose {
case .api:
let parsed = topic.map { Self.parseSources($0, item: Self.apiItem, urlKey: Self.urlKey) } ?? []
return parsed.isEmpty ? Self.defaultAPISources : parsed
case .blob:
return topic.map { Self.parseSources($0, item: Self.blobItem, urlKey: Self.urlFormatKey) } ?? []
}
}
/// Extracts the source list from the `sources` topic item `item` (`api` or `blob`), reading each
/// entry's url from `urlKey` (`url` for api, `url_format` for blob). Malformed entries are skipped.
private static func parseSources(
_ topic: RemoteConfiguration.ConfigTopic,
item: String,
urlKey: String
) -> [RemoteConfigSource] {
guard case .array(let entries)? = topic[item]?.content[Self.sourcesKey] else {
return []
}
return entries.compactMap { element in
guard case .object(let object) = element,
case .string(let url)? = object[urlKey],
case .int(let priority)? = object[Self.priorityKey],
case .int(let weight)? = object[Self.weightKey] else {
return nil
}
return RemoteConfigSource(url: url, priority: priority, weight: weight)
}
}
/// Collapses duplicate urls to the occurrence with the highest priority (i.e. the lowest
/// `priority` number, tie-broken by weight), keeping first-seen order. Done once per rebuild so
/// reads never need to re-dedupe.
private static func dedupe(_ sources: [RemoteConfigSource]) -> [RemoteConfigSource] {
var bestByURL: [String: RemoteConfigSource] = [:]
var order: [String] = []
for source in sources {
guard let existing = bestByURL[source.url] else {
bestByURL[source.url] = source
order.append(source.url)
continue
}
if source.priority != existing.priority || source.weight != existing.weight {
Logger.warn(Strings.remoteConfig.duplicateSourceURL(source.url))
}
if source.priority < existing.priority ||
(source.priority == existing.priority && source.weight > existing.weight) {
bestByURL[source.url] = source
}
}
return order.compactMap { bestByURL[$0] }
}
}
/// Walks a single list of sources in fallback order. Every handout is stamped with the current
/// `token`, which is bumped whenever the position changes (fallback or `restart()`). A report only
/// advances if its handle still carries the current token, so stale or concurrent reports - and
/// reports left over from before a `restart()` - are ignored.
///
/// - Note: Not thread-safe on its own: `RemoteConfigSourceProvider` serializes every access under its
/// own lock.
private final class SourceFailover {
private let purpose: RemoteConfigSourceHandle.Purpose
private let selector: WeightedSourceSelector<RemoteConfigSource>
private var token: Int
init(
purpose: RemoteConfigSourceHandle.Purpose,
sources: [RemoteConfigSource],
randomizer: WeightedSourceRandomizer?,
initialToken: Int = 0
) {
self.purpose = purpose
self.selector = WeightedSourceSelector(sources: sources, randomizer: randomizer)
self.token = initialToken
}
/// The token a handle handed out right now would carry. Used to seed the next generation on rebuild.
var currentToken: Int {
return self.token
}
var current: RemoteConfigSourceHandle? {
return self.selector.current.map {
RemoteConfigSourceHandle(purpose: self.purpose, source: $0, token: self.token)
}
}
/// Advances past the handle's source, returning whether it did (stale reports are ignored).
@discardableResult
func reportUnhealthy(_ handle: RemoteConfigSourceHandle) -> Bool {
guard handle.token == self.token else { return false }
self.selector.advance()
self.token += 1
return true
}
func restart() {
self.selector.reset()
self.token += 1
}
}