Add SpeechMetadata (public preview) so requests to Gemini
text-to-speech models can attach turn-level instructions to a
TextPart:
- speaker assigns the turn to a voice configured in
MultiSpeakerVoiceConfig. Multi-speaker requests require it on
every part.
- style describes sustained delivery, such as emotion, pacing, or
volume, so it isn't read aloud as part of the transcript.
Momentary sounds and pauses stay inline as vocal tags, such as
<laugh> or <short pause>.
Set metadata with the new TextPart.init(_:speechMetadata:). The
existing init(_:) is kept so references like
[String].map(TextPart.init) still compile. TextPart.speechMetadata
and the SpeechMetadata properties are internal while the API is in
preview.
InternalPart encodes and decodes speechMetadata on the wire. History
doesn't merge streamed text parts that carry speech metadata, so
speaker turns stay separate.
Document the multi-speaker requirements on SpeechConfig,
MultiSpeakerVoiceConfig, and SpeakerVoiceConfig.
Replace the gemini-3.1-flash-tts-preview integration tests with a
TextToSpeechTests suite covering gemini-3.8-flash-tts and
gemini-3.8-flash-lite-tts in unary and streaming, single- and
multi-speaker requests.
Revert commit dd2cfa8df1 (#16685) to undo
the renaming from Gemini Enterprise Agent Platform to Gemini Enterprise
API.
- Restore Backend.agentPlatform(location:) as the primary public API
and remove Backend.enterprise(location:).
- Revert APIConfig.Service.enterprise to .agentPlatform and restore
agentPlatformStagingBypassProxy.
- Revert documentation comments, DocC topics, and guides back to
Agent Platform Gemini API and Gemini Enterprise Agent Platform.
- Clean up leftover Vertex AI symbols to align with Agent Platform and
FirebaseAILogic (isFirebaseAILogicServiceDisabledError and
defaultAgentPlatformAPIConfig).
- Re-align tests and conflict resolutions from #16689, #16693, #16676,
and #16756.
Revert the General Availability (GA) designation for Server Prompt
Templates in docstrings and CHANGELOG.md. Multi-turn conversations
(`TemplateChat`) and function calling (`TemplateTool`) remain public,
annotated with Public Preview warnings.
Key changes:
- Add Public Preview annotations and warnings across the Server Prompt
Templates public API surface (TemplateGenerativeModel, TemplateChat,
TemplateTool, and TemplateToolConfig).
- Update CHANGELOG.md to reflect Public Preview status.
Nest `Tool` inside `GenerativeModel` as `GenerativeModel.Tool` to avoid
naming conflicts with Apple's `FoundationModels.Tool` protocol when both
modules are imported in the same file.
Moving `Tool` into `GenerativeModel` allows developers to adopt Apple's
Foundation Models framework without ambiguous symbol errors for
unqualified `Tool`. Call sites passing tools via leading-dot syntax
(e.g., `tools: [.googleSearch()]`) continue to work without changes.
- Nest `Tool` inside `GenerativeModel` and remove top-level `Tool`.
- Move static conveniences (`.googleSearch()`, `.googleMaps()`,
`.functionDeclarations()`, `.urlContext()`, `.codeExecution()`) to an
extension on `GenerativeModel.Tool`.
- Update `FirebaseAI.generativeModel`, `FirebaseAI.liveModel`,
`LiveGenerativeModel`, and internal request types to use
`GenerativeModel.Tool`.
- Remove leftover internal `isFoundationModeCompatible` property.
- Update DocC links and unit/integration tests across the module.
BREAKING CHANGE: The top-level `Tool` type in `FirebaseAILogic` has been
removed and nested inside `GenerativeModel` as `GenerativeModel.Tool`.
Call sites explicitly typed as `Tool` must update to
`GenerativeModel.Tool`. Calls using leading-dot syntax continue to
compile unmodified.
Clean up the interim integration with Apple's Foundation Models
framework in Firebase AI Logic for Firebase 13.
Apple now natively supports third-party providers in the Foundation
Models framework via LanguageModel, which is implemented directly by
GeminiLanguageModel. The custom GenerativeModelSession,
SystemLanguageModel, HybridModel, custom GenerationSchema, and related
bridging types and protocols are now obsolete.
- Remove GenerativeModelSession, SystemLanguageModel, HybridModel,
GeminiModel, and related session protocols and provider types.
- Remove custom GenerationSchema, GeneratedContent, and related
structured output protocols.
- Remove internal responseJSONSchema support from GenerationConfig and
associated test permutations in FirebaseAILogic and TestApp.
- Remove IS_FOUNDATION_MODELS_SUPPORTED_PLATFORM build settings from
Package.swift and FirebaseAILogic.podspec.
BREAKING CHANGE: GenerativeModelSession, SystemLanguageModel,
HybridModel, GeminiModel, and associated interim integration types and
protocols have been removed. Use GeminiLanguageModel with Apple's
Foundation Models framework instead.
Promote Server Prompt Templates to General Availability (GA) in the
FirebaseAILogic Swift SDK, enabling developers to decouple prompt
engineering, schemas, and model configs from client release cycles.
Key changes:
- Expose TemplateGenerativeModel publicly via FirebaseAI's
templateGenerativeModel(tools:toolConfig:requestOptions:).
- Introduce TemplateChat and TemplateGenerativeModel.startChat for
multi-turn conversations with templates containing {{history}},
supporting empty turns to run template prompts.
- Add TemplateTool with functionDeclarations and googleMaps, as well
as TemplateToolConfig for Grounding with Google Maps.
- Add Schema.toJSONSchema() to convert Schema instances to JSON Schema
Draft 7/2020-12 for function parameter declarations.
- Add TemplateInput conversion supporting primitives, arrays, and
dictionaries, with NSNumber boolean disambiguation and rich errors.
- Sanitize and percent-encode template IDs in request URLs to prevent
path and method injection.
- Add comprehensive unit test coverage and end-to-end integration
tests covering generation, streaming, chat, and function calling.
Implemented `GeminiLanguageModel`, providing an adapter for Google's
Gemini models conforming to Apple's `FoundationModels.LanguageModel`
protocol. This PR includes text to text generation capabilities; other
modalities and Gemini features will be handled in follow-up PRs.
**Note:** The `GeminiLanguageModel` implementation is guarded by the
environment variable `GEMINI_LANGUAGE_MODEL`. By default none of the
`GeminiLanguageModel` implementation is included.
Key additions:
- `GeminiLanguageModel`: Implements `LanguageModel` protocol with
reasoning capabilities, configuring execution via `ModelResource`
and `EndpointConfiguration`.
- `GeminiLanguageModelExecutor`: Converts prompt instructions and
history into Gemini streaming requests, yielding responses as
`LanguageModelSession.ResponseStream`.
- `GeminiTranscriptTranslator`: Translates Apple `Transcript` entries
and segments into Gemini `Content` structures while preserving thought
signatures and validating supported transcript features.
- `GeminiLanguageModel.Error` and `GeminiErrorMapper`: Translates Gemini
API, HTTP, and network errors into standard `LanguageModelError` cases
(`rateLimited`, `timeout`, `guardrailViolation`, `refusal`, and
`unsupportedLanguageOrLocale`) or `GeminiLanguageModel.Error` cases
(`serviceUnavailable`, `networkFailure`, `modelNotFound`, `apiError`).
The `APIError` payload is designed for seamless forward compatibility
with both GenerateContent and the upcoming Interactions API.
- Testing infrastructure: Adds `ConditionTrait.requireFoundationModels`
to conditionally run Foundation Models tests only on supported
platforms (macOS 27+, iOS 27+, watchOS 27+, visionOS 27+), along with
shared test constants, mock fixtures, and comprehensive unit and
integration test suites.
Andrew Heard