Files
Andrew Heard bc9f9ea1da feat(ai): add FoundationModelsFake for GLM tests (#16995)
Add FoundationModelsFake, a nested Swift package that fakes Apple's
Foundation Models framework, so that GeminiLanguageModel (GLM) can build
and run its unit tests on Linux and on OS versions without the real
framework. It's used only for development and testing.

- GLM depends on the fake through a `FoundationModelsFake` package
  trait, which is on by default where Darwin isn't available (e.g.,
  Linux). With the trait off, the fake isn't built. The root
  `Package.swift` and podspecs can't reach it.
- The fake mirrors Apple's public API and availability in the 27.2
  SDKs, including `@Generable` macro expansions. It omits
  PrivateCloudComputeLanguageModel, OCRTool, BarcodeReaderTool, the
  adapter and feedback APIs, and deprecated APIs.
  SystemLanguageModel is always unavailable. Below Swift 6.4 it
  compiles to an empty module.
- Decoding rejects transcript images with `file:` URLs and schemas
  whose references never resolve, instead of reading local files or
  overflowing the stack.
- Without ImageIO, GLM passes PNG, JPEG, WebP, HEIC, and HEIF images
  through to Gemini after checking their signatures, and rejects
  images that would need re-encoding.
- Add CI jobs that run both packages' tests against the fake: one on
  Linux in a digest-pinned `swift:6.4.2` container, and one on macOS
  with Xcode 27.2. Skip `sdk.ai` for changes that only touch the fake.
2026-11-05 14:13:50 -05:00
..

GeminiLanguageModel Integration Tests

End-to-end integration tests validating GeminiLanguageModel sessions against remote Gemini and Firebase AI Logic backends.

Quick Start

# Run integration tests (using environment variables or GoogleService-Info.plist)
swift test --filter IntegrationTests

# Run a specific integration test suite
swift test --filter ToolCallingIntegrationTests

# Run all unit tests (fast, skips all integration tests)
swift test --skip IntegrationTests

Running Integration Tests

Integration tests in this directory connect to Google Cloud or Firebase endpoints when credentials are configured:

  1. Gemini Developer API: Set GEMINI_API_KEY or GOOGLE_API_KEY in your environment.
  2. Firebase AI Logic: Provide credentials via a configuration file or individual variables:
    • Set FIREBASE_PLIST_PATH pointing to a valid GoogleService-Info.plist file, along with AppCheckDebugToken.
    • Alternatively, supply FIREBASE_PROJECT_ID, FIREBASE_APP_ID, FIREBASE_API_KEY, and AppCheckDebugToken.

Automatic Skipping

If credentials are not present in the environment, integration tests marked with .requireIntegrationTestingBackend are automatically and cleanly skipped without failing test execution.

Supported Backends

All integration tests are parameterized across IntegrationTestingBackend.availableBackends:

  • .developerAPI: Direct Gemini Developer API (generativelanguage.googleapis.com).
  • .firebaseAILogicDeveloperAPI: Firebase AI Logic proxy to Gemini Developer API (firebasevertexai.googleapis.com).
  • .firebaseAILogicAgentPlatform(location:): Firebase AI Logic proxy to Gemini Enterprise Agent Platform (defaulting to location: "global").

Files in this Directory

  • BasicContentGenerationIntegrationTests.swift: Parameterized integration tests for single-turn prompt response, multi-turn chat sessions, and streaming chunk responses.
  • GuidedGenerationIntegrationTests.swift: Parameterized integration tests for guided generation (structured outputs) using @Generable types, including single-turn and streaming generation, enum classification, and rich multi-type recursive hierarchies.
  • ToolCallingIntegrationTests.swift: Parameterized integration tests for tool calling (function calling) using FoundationModels Tool definitions, covering single-turn tool calls, sequential tool calls, parallel tool calls, tool calling mode configuration, parameterless tools with empty arguments, and thought signatures recorded as reasoning entries.
  • ImageUnderstandingIntegrationTests.swift: Parameterized integration tests for image input (vision) using images generated in code, covering single and multiple images, @Generable classification, image orientation, PNG and HEIC file URLs, streaming, and images in multi-turn history. These tests use the default sampling mode with tightly constrained prompts, since many Gemini 3 models don't support greedy sampling.
  • IntegrationTestingBackend+GeminiLanguageModel.swift: Convenience extension providing backend.makeModel() to instantiate a pre-configured GeminiLanguageModel.

Writing New Integration Tests

To add a new integration test suite, parameterize on IntegrationTestingBackend:

@Suite("My New Feature Integration Tests", .requireFoundationModels)
struct MyNewFeatureIntegrationTests {
  @Test(
    .tags(.integration),
    .requireIntegrationTestingBackend,
    arguments: IntegrationTestingBackend.availableBackends
  )
  @available(macOS 27.0, iOS 27.0, watchOS 27.0, visionOS 27.0, *)
  func testFeature(backend: IntegrationTestingBackend) async throws {
    let model = try await backend.makeModel()
    let session = LanguageModelSession(model: model)
    // Execute test assertions...
  }
}