Add FoundationModelsFake, a nested Swift package that fakes Apple's
Foundation Models framework, so that GeminiLanguageModel (GLM) can build
and run its unit tests on Linux and on OS versions without the real
framework. It's used only for development and testing.
- GLM depends on the fake through a `FoundationModelsFake` package
trait, which is on by default where Darwin isn't available (e.g.,
Linux). With the trait off, the fake isn't built. The root
`Package.swift` and podspecs can't reach it.
- The fake mirrors Apple's public API and availability in the 27.2
SDKs, including `@Generable` macro expansions. It omits
PrivateCloudComputeLanguageModel, OCRTool, BarcodeReaderTool, the
adapter and feedback APIs, and deprecated APIs.
SystemLanguageModel is always unavailable. Below Swift 6.4 it
compiles to an empty module.
- Decoding rejects transcript images with `file:` URLs and schemas
whose references never resolve, instead of reading local files or
overflowing the stack.
- Without ImageIO, GLM passes PNG, JPEG, WebP, HEIC, and HEIF images
through to Gemini after checking their signatures, and rejects
images that would need re-encoding.
- Add CI jobs that run both packages' tests against the fake: one on
Linux in a digest-pinned `swift:6.4.2` container, and one on macOS
with Xcode 27.2. Skip `sdk.ai` for changes that only touch the fake.
Declare the vision capability in GeminiLanguageModel and translate
prompt image attachments into Gemini inlineData parts. Previously,
any attachment segment in a transcript threw an error.
Key changes and implementation details:
- Pass-through optimization: PNG, JPEG, WebP, HEIC, and HEIF file URLs
that are already upright are sent unchanged without re-encoding.
Multi-image files are only passed through if the primary image is
the first image.
- Re-encoding and normalization: All other image sources (CGImage,
CIImage, CVPixelBuffer, other formats, or rotated files) are rotated
to display orientation and encoded as JPEG (quality 0.9, matching
Transcript Codable conformance). Grayscale, extended-range, and HDR
images are converted to sRGB, compositing transparency over black.
- Platform portability: Re-encoding relies solely on CoreGraphics,
ImageIO, and UniformTypeIdentifiers to maintain watchOS support.
- Scoping and validation: Image attachments are supported only in
prompt entries. Non-file URLs are rejected by scheme to avoid
leaking tokens or dumping long payloads.
- Ordering: Prompt segments retain their order, with adjacent text runs
merged into single text parts.
- Tests: Add extensive unit and integration tests covering formats,
orientations, color spaces, and backend image understanding.
Replace `#if canImport(FoundationModels, _version: 2.1)` with
`#if compiler(>=6.4.2)` around the `Transcript.Entry.data` case in
`GeminiTranscriptTranslator` and its test. Xcode 27.2 beta 3 bumped the
Swift compiler to 6.4.2, so the condition no longer depends on the
underscored `_version` parameter of `canImport`.
Earlier Xcode 27.2 betas report Swift 6.4 and skip the case, which is
acceptable since beta 3 supersedes them.
The Xcode 27.2 SDK (FoundationModels 2.1) adds a `Transcript.Entry.data`
case, which triggers a "switch must be exhaustive" warning in
`GeminiTranscriptTranslator`. Handle the new case by throwing
`unsupportedTranscriptContent`, gated with `canImport(FoundationModels,
_version: 2.1)` so only the one case is conditionally compiled and
Xcode 27.0 and 27.1 continue to build.
Add a unit test for the new case. It requires macOS 27.2 or later at
runtime, so it is skipped on earlier OS versions.
Add a `unit-tests` job to the GeminiLanguageModel workflow that runs
`swift test --skip IntegrationTests` on Xcode 27.0 and 27.2, as well as
on Xcode 26.6 (macOS 26) and Xcode 26.2 (macOS 15). The FoundationModels
code is compiled out on Xcode 26, so those jobs verify that the package
still builds and its remaining tests pass with older toolchains.
Update the test READMEs to use `--skip IntegrationTests`, since
`--skip Integration` also skips the `IntegrationTestingBackendTests`
unit tests.
Gemini 3.x models return the thought signature of a text response on a
trailing empty text part. The executor records this as a signature-only
`.reasoning` transcript entry, which `GeminiTranscriptTranslator` then
round-tripped on the next turn as a stand-alone `Part` containing only
`thought` and `thoughtSignature` and no `data`.
The Gemini Developer API tolerates a part with no data, but the Gemini
Enterprise Agent Platform rejects it nondeterministically with
`400 INVALID_ARGUMENT` ("Request contains an invalid argument."),
making any multi-turn session that follows a completed text response
flaky on that backend (e.g. `sessionRespondSequentialToolCalls`).
Never emit a signature-only reasoning entry as a data-less part:
- A signature preceding a response is attached to the response's text
part.
- A signature following a response is attached to the last part of the
current model turn, mirroring how the server originally sent it.
- If there is no model part to attach to, fall back to a thought part
with empty text so the `data` oneof is always populated.
Replace the unit test that asserted the data-less shape with coverage
for all three cases.
Add function calling (tool calling) support to GeminiLanguageModel
conforming to Apple's FoundationModels framework.
* Add .toolCalling capability to GeminiLanguageModel.
* Translate FoundationModels ToolDefinition into Gemini Tool and
FunctionDeclaration using OpenAPI parametersJsonSchema.
* Map GenerationOptions.ToolCallingMode (.allowed, .required,
.disallowed) to Gemini FunctionCallingConfig.Mode.
* Stream function calls in GeminiLanguageModelExecutor, forwarding
JSON-encoded arguments or empty fallback "{}" into channel actions.
* Translate Transcript tool calls and tool outputs into Gemini Content
parts, preserving thought signatures from reasoning models and
flushing buffered reasoning thoughts cleanly.
* Unpack structured tool responses and wrap scalar text tool outputs
under the result field.
* Support parameterless tools with empty arguments across request
schema translation, model response handling, and transcript history.
* Add unit tests across GeminiLanguageModel, GeminiRequestTranslator,
and GeminiTranscriptTranslator.
* Add end-to-end integration tests covering single-turn, sequential,
parallel, disallowed, parameterless, and reasoning tool calling
across all supported backends.
Establishes the integration testing foundation and parameterized test
suites for `GeminiLanguageModel`, enabling fast local execution via
`swift test` and automated CI verification on iOS Simulator.
Key Changes:
- `IntegrationTestingBackend`: Defines canonical backend configurations
(`.developerAPI`, `.firebaseAILogicDeveloperAPI`, and
`.firebaseAILogicAgentPlatform`), dynamically resolving endpoints,
model resource paths, and auth headers from available credentials.
- `IntegrationTestEnvironment`: Centralizes credential discovery
(`GEMINI_API_KEY`, `GOOGLE_API_KEY`, and `FIREBASE_PLIST_PATH`) into
an immutable, thread-safe configuration struct with cached process
access to avoid redundant disk I/O.
- `AppCheckDebugClient`: Implements a lightweight client to exchange
debug tokens for App Check tokens without Keychain dependencies,
deduplicating token requests across test runs with an in-memory cache.
- Parameterized Tests: Adds `BasicContentGenerationIntegrationTests`
running against all available backends, tagged with `@Tag.integration`
and gated by condition traits to cleanly skip when credentials are
missing.
- CI Workflow: Adds `.github/workflows/sdk.geminilanguagemodel.yml` to
run integration tests against Firebase AI Logic on iOS Simulator
using decrypted repo secrets.
Implemented `GeminiLanguageModel`, providing an adapter for Google's
Gemini models conforming to Apple's `FoundationModels.LanguageModel`
protocol. This PR includes text to text generation capabilities; other
modalities and Gemini features will be handled in follow-up PRs.
**Note:** The `GeminiLanguageModel` implementation is guarded by the
environment variable `GEMINI_LANGUAGE_MODEL`. By default none of the
`GeminiLanguageModel` implementation is included.
Key additions:
- `GeminiLanguageModel`: Implements `LanguageModel` protocol with
reasoning capabilities, configuring execution via `ModelResource`
and `EndpointConfiguration`.
- `GeminiLanguageModelExecutor`: Converts prompt instructions and
history into Gemini streaming requests, yielding responses as
`LanguageModelSession.ResponseStream`.
- `GeminiTranscriptTranslator`: Translates Apple `Transcript` entries
and segments into Gemini `Content` structures while preserving thought
signatures and validating supported transcript features.
- `GeminiLanguageModel.Error` and `GeminiErrorMapper`: Translates Gemini
API, HTTP, and network errors into standard `LanguageModelError` cases
(`rateLimited`, `timeout`, `guardrailViolation`, `refusal`, and
`unsupportedLanguageOrLocale`) or `GeminiLanguageModel.Error` cases
(`serviceUnavailable`, `networkFailure`, `modelNotFound`, `apiError`).
The `APIError` payload is designed for seamless forward compatibility
with both GenerateContent and the upcoming Interactions API.
- Testing infrastructure: Adds `ConditionTrait.requireFoundationModels`
to conditionally run Foundation Models tests only on supported
platforms (macOS 27+, iOS 27+, watchOS 27+, visionOS 27+), along with
shared test constants, mock fixtures, and comprehensive unit and
integration test suites.
Andrew Heard