Commit Graph
92 Commits
Author SHA1 Message Date
Gil 8770828dd8 Refactor usages of DatabaseId to avoid raw pointers (#3041)
* Remove stale references to DatabaseId
* Have Firestore take the DatabaseId directly
* Allow tests to default the database implicitly
* Have FSTSerializerBeta take DatabaseId by value
* Have FSTUserDataConverter take DatabaseId by value
* Make FIRFirestore.databaseID a const reference
* Make FSTFirestoreClient.databaseID a const reference
* Have FSTReferenceValue take DatabaseId by value
* Have DatabaseInfo take DatabaseId by value
* Have Serializer take DatabaseId by value
* Fix C++ ReferenceValue
  * Take DatabaseId by value
  * Reverse order to match Android
  * Rename `reference` to `key`
2019-05-21 20:28:51 -07:00
Gil c2a3d04e6d Port FSTDocumentSet to C++ (#2608)
* Replace FSTDocumentSet's guts with immutable::SortedSet.

* Port FSTDocumentSet to C++ DocumentSet.

* Replace implementation of FSTDocumentSet with C++ DocumentSet

* Update all usages of FSTDocumentSet

* Remove FSTDocumentSet

* Add Objective-C++ guard

* Use absl::c_equal instead of hand-rolling it.

* Split SortedContainer out of SortedMapBase

... and use it

* Fix NSNotFound -> npos

* Remove = default constructors that aren't true.

These are implicitly deleted because DocumentSet is a member and isn't
default constructible.

* Fix usage of FSTQuery.limit
2019-03-25 11:04:21 -07:00
Gil e4dcc96e52 Eliminate Objective-C BOOL in Firestore/core (#2574)
* Add objc::EqualTo and objc::Hash

These make it possible to use std::unordered_map with Objective-C object
keys.

* Use std::unordered_map in MemoryQueryCache

* Fix objective-C library detection

This ensures that .mm files are compiled with -fobjc-arc (among other
things).

* Remove BOOL in FSTDocument to eliminate static_cast<bool>
2019-03-18 17:28:46 -07:00
Michael Lehenbauer bd0b8e9449 Collection Group Query Internals (#2378)
Includes the new index and all internals for collection group queries, but the public API will not be exposed until backend support is ready.
2019-03-08 13:37:00 -08:00
Sebastian Schmidt 657f53a9b8 Adding FieldValue.increment() (#2075) 2019-03-06 14:51:30 -08:00
Michael Lehenbauer 61a7fc1571 Improved query performance in the face of subcollections. (#2471)
Ports an optimization from Android where we skip parsing documents that are in
subcollections beneath the collection we're querying over.
2019-03-05 13:21:37 -08:00
Konstantin Varlamov a22b4cd08e Pass FSTMutations using a vector (#2357) 2019-02-07 22:03:17 -05:00
Konstantin Varlamov 9b3654b156 C++ migration: port write stream-related part of FSTRemoteStore (#2335) 2019-02-06 20:23:29 -05:00
Konstantin Varlamov 456e8eb0f2 C++ migration: port watch stream-related part of FSTRemoteStore (#2331) 2019-02-03 18:01:52 -05:00
Greg Soltis 388f267d6e Remove FSTMutationQueue (#2319)
* Remove FSTMutationQueue
2019-01-29 16:01:59 -08:00
Konstantin Varlamov 8626e31feb C++ migration: port FSTRemoteEvent (#2320) 2019-01-29 18:55:25 -05:00
Greg Soltis d8e3704360 Add MutationQueue interface (#2317)
* Implement mutation queue interface and port tests
2019-01-28 10:21:20 -08:00
Konstantin Varlamov 9789c614ea C++ migration: port FSTWatchChangeAggregator and FSTTargetState (#2313) 2019-01-27 16:45:15 -05:00
Greg Soltis a98e1ddb9c Implement FSTLevelDBMutationQueue in C++ (#2310)
* Port leveldb mutation queue to C++
2019-01-25 13:27:58 -08:00
Konstantin Varlamov faad116a5b C++ migration: eliminate FSTBoxedTargetId (#2301) 2019-01-24 17:01:31 -05:00
Konstantin Varlamov acb222026a C++ migration: port FSTWatchChange. (#2300) 2019-01-23 17:26:49 -05:00
Konstantin Varlamov a52abec999 Stop persisting last acknowledged batch ID (#2243)
After #2029, acknowledged batches are always immediately removed, so persisting the last acknowledged batch ID is no longer necessary.
2019-01-11 12:43:07 -05:00
Paul Beusterien 7c53489e03 Travis to Xcode 10.1 and clang-format to 8.0.0 (tags/google/stable/2018-08-24) (#2222)
* Travis to Xcode 10.1
* Update to clang-format 8
* Update clang-format homebrew link
* Work around space in filename style.sh issue
2019-01-02 09:27:31 -08:00
Greg Soltis 900ca3b83d Implement QueryCache interface and port QueryCache tests (#2209)
* Implement QueryCache interface and port tests

* Port production usages of QueryCache (#2211)

* Remove FSTQueryCache and implementations

* Switch size() to size_t
2018-12-21 22:08:34 -08:00
Greg Soltis 466880b094 Continue work on ReferenceSet (#2213)
* Migrate FSTDocumentReference to C++

* Change SortedSet template parameter ordering

Makes it easier to specify a comparator without specifying what the
empty member of the underlying map is.

* Migrate MemoryMutationQueue to C++ references by key

* migrate.py

* CMake

* Finish porting ReferenceSet

* Swap reference set implementation

* Port MemoryQueryCache to use ported ReferenceSet

* Port FSTReferenceSetTest

* Port usage for limbo document refs

* Port LRU and LocalStore usages

* Remove FSTReferenceSet and FSTDocumentReference

* Style

* Add newline
2018-12-21 21:03:13 -08:00
Gil 6af931d4e8 Migrate Firestore to the v1 protocol (#2200)
* Use python executable directly

  * python2 is not guaranteed to exist
  * scripts aren't directly executable on Windows

* Add Firestore v1 protos

* Point cmake at Firestore v1 protos

* Regenerate protobuf-related sources

* Make local protos refer to v1 protos

* fixup! Regenerate protobuf-related sources

* Remove v1beta1 protos

* s/v1beta1/v1/g in source.

* s/v1beta1/v1/ in the Xcode project

* Remove stale bug comments.

This was fixed by adding an explicit FieldPath API rather than exposing
escaping to the end user.
2018-12-21 11:28:48 -08:00
Greg Soltis ab05b96810 Port FSTRemoteDocumentCache (#2196)
* Remove FSTRemoteDocumentCache
2018-12-18 13:24:35 -08:00
Greg Soltis e226001f97 Port FSTRemoteDocumentCacheTest to use C++ interface (#2194) 2018-12-17 13:24:49 -08:00
Greg Soltis 598bc55d29 Remove start from persistence interface (#2173)
* Remove start from persistence interface, switch FSTLevelDB to use a factory method that returns Status
2018-12-14 15:49:15 -08:00
Konstantin Varlamov 9f6ed13914 Port performance optimizations to speed up reading large collections from Android (#2140)
Straightforward port of firebase/firebase-android-sdk#123.
2018-12-10 14:39:22 -08:00
Konstantin Varlamov 673b08592a C++: replace FSTMaybeDocumentDictionary with a C++ equivalent (#2139)
Also eliminate most usages of `FSTDocumentKey`, remove most methods from the Objective-C class and make it just a wrapper over `DocumentKey`. The only usage that cannot be directly replaced by C++ `DocumentKey` is in `FSTFieldValue`.
2018-12-10 11:57:08 -08:00
Greg Soltis 2c9f881b67 Save schema version on downgrade, add test to verify (#2153)
* Save schema version on downgrade, add test to verify
2018-12-05 21:48:01 -08:00
Greg Soltis 305f87215e Wire together LRU GC (#1905)
* Wire together LRU GC
* Default enabled, threshold set to `100 MB`.
2018-11-26 14:33:28 -08:00
Sebastian Schmidt 4aaf9081a3 Porting Multi-Tab Structural Changes (#2049) 2018-11-07 11:22:01 -08:00
Sebastian Schmidt 1291b03c50 Remove Mutation tombstones (#2052)
* Remove Mutation tombstones

* Review comments
2018-11-06 14:24:06 -08:00
Sebastian Schmidt 5bc4cc55a6 Remove Held Write Acks (#2029)
* Drop acknowledged mutations in schema migration (#1818)

Drop acknowledged mutations in schema migration as part of removing held write acks.

Port of firebase/firebase-js-sdk#1149

* Change removeMutationBatch to remove a single batch (#1955)

* Update Firestore Generated Protos

*  Adding UnknownDocument and hasCommittedMutations (#1971)

* Removing held write batch handling from LocalStore (#1997)

* View changes for held write acks (#2004)

* Held Write Acks Unit Test Fixes (#2008)

* Held Write Ack Removal Spec Tests (#2018)

* Fix integration test
2018-11-01 09:44:30 -07:00
Greg Soltis 4c86de9929 Schema migration to ensure sentinel rows exist (#1911)
* Add schema migration to ensure each document has a sentinel row
2018-10-05 16:51:54 -07:00
Gil 86dc058729 Add a lint check for const references to absl::string_view (#1915)
* Check for const absl::string_view&, which should just be absl::string_view.

* Fix uses of const absl::string_view&
2018-10-05 13:58:43 -07:00
Greg Soltis 2e7002bcfd Make sure we actually drop the sentinel rows (#1908)
Make sure we actually drop the sentinel rows when we GC documents
2018-10-04 10:54:37 -07:00
Greg Soltis 7c4a12f09e Implement byteSize methods on lru delegates for use in GC threshold (#1893)
Implement byteSize methods on lru delegates for use in GC threshold
2018-10-01 16:35:23 -07:00
Gil a40a43c058 Port user data handling to C++ (#1823)
Extract useful bits of FSTUserDataConverter for use in C++:
  * Migrate FSTParsedSetData to core::ParsedSetData
  * Migrate FSTParsedUpdateData to core::ParsedUpdateData
  * Migrate FSTParseContext to core::ParseContext
  * Split ParseAccumulator out of ParseContext to make ownership easier

Also:
  * Remove dead code (handled in parseSentinelFieldValue)
  * Flatten App/iOS/Supporting Files into the parent
2018-09-18 09:19:11 -07:00
Gil c856059955 Migrate FSTLevelDBMigrations to C++ LevelDbMigrations (#1714)
* Remove a copy in the common case of adding a new key to a transaction

* Add support for nanopb::Writer wrapping a std::string

* Migrate FSTLevelDBMigrations to LevelDbMigrations
2018-08-20 12:44:52 -07:00
Gil 2f6ff231dd Use new filesystem routines in FSTLevelDB (#1707)
* Convert between leveldb::Status and util::Status

* Use new filesystem routines in FSTLevelDB

Also rework -[FSTPersistence start] to return Status

* Remove unused leveldb::Status to NSError conversions

* Add GoogleTest as a dependency of Firestore_IntegrationTests_iOS
2018-08-17 14:42:05 -07:00
Gil 10cc4f2190 Migrate FSTListenSequenceNumber to C++ ListenSequenceNumber (#1703) 2018-08-16 14:02:03 -07:00
Gil a9f8a8fa7d Migrate FSTBatchID to C++ BatchId (#1700) 2018-08-16 10:44:26 -07:00
Gil 0725a5d4bb Migrate FSTTargetID to C++ TargetId (#1697) 2018-08-15 15:56:45 -07:00
Michael Lehenbauer b88f95c44f Reformat using 'clang-format version 7.0.0 (tags/google/stable/2018-04-24)' (#1681)
This is the version of clang-format that travis is currently using.
2018-08-13 14:18:02 -07:00
Gil 902fccf4b9 Initial filesystem implementation (#1633)
* Move Path into a separate filesystem library

* Add a filesystem interface and an NSFileManager-based implementation

* Use Path/Filesystem in FSTPersistenceTestHelpers
2018-08-13 09:04:13 -07:00
Gil 00f4f6d037 Migrate LevelDB Key classes to C++ (#1651)
* Migrate to LevelDbVersionKey

* Migrate to LevelDbMutationKey

* Migrate to LevelDbDocumentMutationKey

* Migrate to LevelDbMutationQueueKey

* Migrate to LevelDbTargetGlobalKey

* Migrate to LevelDbTargetKey

* Migrate to LevelDbQueryTargetKey

* Migrate to LevelDbTargetDocumentKey

* Migrate to LevelDbDocumentTargetKey

* Migrate to LevelDbRemoteDocumentKey

Also:
  - Remove FSTLevelDBKey and tests
  - Remove StringView and tests
2018-08-08 12:34:03 -07:00
Gil 4b90618818 Migrate [LevelDBKey descriptionForKey:] to DescribeKey() (#1638)
* Convert FSTLevelDBKeyTests to DescribeKey()

Note that the description coverage is already in leveldb_key_test.cc

* Rename Describe to DescribeKey

* Fix pointer alignment in leveldb_key.cc

* Add DescribeKey overloads for other string types

Unforunately, both leveldb::Slice and absl::string_view implicitly
convert from std::string which makes providing overloads for both
ambiguous for anything that converts to either.

* Migrate [FSTLevelDBKey descriptionForKey:] to DescribeKey()
2018-08-06 11:40:27 -07:00
Gil 154f96efd2 Convert FSTLevelDBMutationQueue.userID to std::string (#1637) 2018-08-06 08:11:06 -07:00
Sebastian Schmidt f5ede65deb Don't look up TargetId in notifyLocalViewChanges (#1607) 2018-07-31 10:19:57 -07:00
Greg Soltis c0fcc092f4 Fix some comments (#1606)
* Fix a comment

* Remove another comment
2018-07-31 08:44:39 -07:00
zxu 098f93c5ed fix import pattern (#1604)
* fix import pattern

* fix style

* move check into cpplint.py

* replace than with rather than
2018-07-31 10:30:49 -04:00
Greg Soltis 16a5210b3c Merging the LRU branch (#1600)
* Switch to reference delegates for garbage collection (#1410)

* Bump sequence number on resume token refresh

* Style

* Fix comment formatting

* Add FSTReferenceDelegate definition and documentation

* Add methods to return nil for delegates, wire up inMemoryPins

* Add hook for removing a reference

* Start work on reference delegates

* Fix up tests to support adding documents at a sequence number

* Implement removing references

* Remove from target when dropped from local view

* Fix warning

* Add hooks for removal from mutation queue

* Add hooks for limbo document updates

* Style

* Drop commented-out code

* Fixup after merging master

* Start importing delegate implementations

* Memory LRU tests pass

* Make memory lru work, start fixing up lru tests. All with hidden sequence numbers

* Most LRU tests passing w/ memory

* Big LRU test passes w/ memory

* Leveldb LRU tests pass

* All passing except local store tests

* All tests pass

* Fix localstore tests

* Revert removeQueryData change

* Remove collectGarbage calls

* Remove NoOp GC

* More cleanup

* Refactor FSTHelpers

* Cleanup and commenting

* Comments and cleanup

* Comments

* Drop nullable for reference delegates, and old persistence method on spec tests

* Fix missing sequence bump, remove unneeded targetID

* Style

* Drop commented-out code

* Satisfy the linters and copyrighters, etc.

* Drop more commented-out code

* Fix import -> include, NSUInteger -> int

* Start work on test comments

* FSTDocumentKey -> DocumentKey

* Sample of LRU test refactor (#1424)

* Sample of test refactor

* Drop errant 'struct' keyword

* More test refactoring

* Make gc a member variable too

* Tests refactored to use member variables

* Assign delegates procedurally, rather than passing in a block

* Fix up FSTLevelDB a little

* listens -> listenTargets

* Rename GCEnabled

* remove isSentinel

* reword comment

* Add case for non-eager GC to test

* Style

* Cleaning the lint trap

* Clean up LRU tests (#1433)

* Add copious helpers and some docs to lru tests

* Style

* Method name refactor, add comment

* Drop FSTGarbageCollector and FSTEagerGarbageCollector (#1440)

* Drop FSTGarbageCollector and FSTEagerGarbageCollector

* Drop unused containsKey from FSTLevelDBMutationQueue

* Style

* Include sequence number in FSTQueryData hash and isEqual (#1568)

* Include sequence number in FSTQueryData hash and isEqual methods

* Style

* Switch to util::Hash

* Fix preserving query cache metadata (#1569)

* Add failing test

* Move the restart tests to the leveldb test suite

* Style

* Add new persistence test helper

* Fix test failure

* Review feedback

* Style
2018-07-27 13:58:11 -07:00