Commit Graph
52 Commits
Author SHA1 Message Date
Jason Hu 59a42fa4f2 Fix XCode Analysis Issues for XCode 10.1 (#3014) 2019-05-17 09:01:35 -04:00
Gil 77215913c3 Port FSTLocalDocumentsView to C++ (#2489)
* Port FSTLocalDocumentsView to C++.
* Replace callers of FSTLocalDocumentsView
* Remove FSTLocalDocumentsView
2019-03-22 13:46:48 -07:00
Michael Lehenbauer bd0b8e9449 Collection Group Query Internals (#2378)
Includes the new index and all internals for collection group queries, but the public API will not be exposed until backend support is ready.
2019-03-08 13:37:00 -08:00
Sebastian Schmidt 657f53a9b8 Adding FieldValue.increment() (#2075) 2019-03-06 14:51:30 -08:00
Konstantin Varlamov a22b4cd08e Pass FSTMutations using a vector (#2357) 2019-02-07 22:03:17 -05:00
Greg Soltis 388f267d6e Remove FSTMutationQueue (#2319)
* Remove FSTMutationQueue
2019-01-29 16:01:59 -08:00
Konstantin Varlamov 8626e31feb C++ migration: port FSTRemoteEvent (#2320) 2019-01-29 18:55:25 -05:00
Konstantin Varlamov f08a8bc6dd C++ migration: port FSTTargetChange (#2318) 2019-01-29 15:10:18 -05:00
Greg Soltis 7c51215882 Port FSTListenSequence (#2322) 2019-01-28 16:01:30 -08:00
Konstantin Varlamov faad116a5b C++ migration: eliminate FSTBoxedTargetId (#2301) 2019-01-24 17:01:31 -05:00
Konstantin Varlamov a52abec999 Stop persisting last acknowledged batch ID (#2243)
After #2029, acknowledged batches are always immediately removed, so persisting the last acknowledged batch ID is no longer necessary.
2019-01-11 12:43:07 -05:00
Paul Beusterien 7c53489e03 Travis to Xcode 10.1 and clang-format to 8.0.0 (tags/google/stable/2018-08-24) (#2222)
* Travis to Xcode 10.1
* Update to clang-format 8
* Update clang-format homebrew link
* Work around space in filename style.sh issue
2019-01-02 09:27:31 -08:00
Greg Soltis 900ca3b83d Implement QueryCache interface and port QueryCache tests (#2209)
* Implement QueryCache interface and port tests

* Port production usages of QueryCache (#2211)

* Remove FSTQueryCache and implementations

* Switch size() to size_t
2018-12-21 22:08:34 -08:00
Greg Soltis 466880b094 Continue work on ReferenceSet (#2213)
* Migrate FSTDocumentReference to C++

* Change SortedSet template parameter ordering

Makes it easier to specify a comparator without specifying what the
empty member of the underlying map is.

* Migrate MemoryMutationQueue to C++ references by key

* migrate.py

* CMake

* Finish porting ReferenceSet

* Swap reference set implementation

* Port MemoryQueryCache to use ported ReferenceSet

* Port FSTReferenceSetTest

* Port usage for limbo document refs

* Port LRU and LocalStore usages

* Remove FSTReferenceSet and FSTDocumentReference

* Style

* Add newline
2018-12-21 21:03:13 -08:00
Greg Soltis ab05b96810 Port FSTRemoteDocumentCache (#2196)
* Remove FSTRemoteDocumentCache
2018-12-18 13:24:35 -08:00
Konstantin Varlamov 9f6ed13914 Port performance optimizations to speed up reading large collections from Android (#2140)
Straightforward port of firebase/firebase-android-sdk#123.
2018-12-10 14:39:22 -08:00
Konstantin Varlamov 673b08592a C++: replace FSTMaybeDocumentDictionary with a C++ equivalent (#2139)
Also eliminate most usages of `FSTDocumentKey`, remove most methods from the Objective-C class and make it just a wrapper over `DocumentKey`. The only usage that cannot be directly replaced by C++ `DocumentKey` is in `FSTFieldValue`.
2018-12-10 11:57:08 -08:00
Greg Soltis 305f87215e Wire together LRU GC (#1905)
* Wire together LRU GC
* Default enabled, threshold set to `100 MB`.
2018-11-26 14:33:28 -08:00
Sebastian Schmidt 4aaf9081a3 Porting Multi-Tab Structural Changes (#2049) 2018-11-07 11:22:01 -08:00
Sebastian Schmidt 5bc4cc55a6 Remove Held Write Acks (#2029)
* Drop acknowledged mutations in schema migration (#1818)

Drop acknowledged mutations in schema migration as part of removing held write acks.

Port of firebase/firebase-js-sdk#1149

* Change removeMutationBatch to remove a single batch (#1955)

* Update Firestore Generated Protos

*  Adding UnknownDocument and hasCommittedMutations (#1971)

* Removing held write batch handling from LocalStore (#1997)

* View changes for held write acks (#2004)

* Held Write Acks Unit Test Fixes (#2008)

* Held Write Ack Removal Spec Tests (#2018)

* Fix integration test
2018-11-01 09:44:30 -07:00
Sebastian Schmidt ceb991adcd Port Mutation Batch API from Android/Web (#1963) 2018-10-17 14:49:40 -07:00
Gil 10cc4f2190 Migrate FSTListenSequenceNumber to C++ ListenSequenceNumber (#1703) 2018-08-16 14:02:03 -07:00
Gil a9f8a8fa7d Migrate FSTBatchID to C++ BatchId (#1700) 2018-08-16 10:44:26 -07:00
Gil bb1a44861f Implement global resume token (#1696) 2018-08-15 17:24:20 -07:00
Gil 0725a5d4bb Migrate FSTTargetID to C++ TargetId (#1697) 2018-08-15 15:56:45 -07:00
Sebastian Schmidt f5ede65deb Don't look up TargetId in notifyLocalViewChanges (#1607) 2018-07-31 10:19:57 -07:00
Greg Soltis 16a5210b3c Merging the LRU branch (#1600)
* Switch to reference delegates for garbage collection (#1410)

* Bump sequence number on resume token refresh

* Style

* Fix comment formatting

* Add FSTReferenceDelegate definition and documentation

* Add methods to return nil for delegates, wire up inMemoryPins

* Add hook for removing a reference

* Start work on reference delegates

* Fix up tests to support adding documents at a sequence number

* Implement removing references

* Remove from target when dropped from local view

* Fix warning

* Add hooks for removal from mutation queue

* Add hooks for limbo document updates

* Style

* Drop commented-out code

* Fixup after merging master

* Start importing delegate implementations

* Memory LRU tests pass

* Make memory lru work, start fixing up lru tests. All with hidden sequence numbers

* Most LRU tests passing w/ memory

* Big LRU test passes w/ memory

* Leveldb LRU tests pass

* All passing except local store tests

* All tests pass

* Fix localstore tests

* Revert removeQueryData change

* Remove collectGarbage calls

* Remove NoOp GC

* More cleanup

* Refactor FSTHelpers

* Cleanup and commenting

* Comments and cleanup

* Comments

* Drop nullable for reference delegates, and old persistence method on spec tests

* Fix missing sequence bump, remove unneeded targetID

* Style

* Drop commented-out code

* Satisfy the linters and copyrighters, etc.

* Drop more commented-out code

* Fix import -> include, NSUInteger -> int

* Start work on test comments

* FSTDocumentKey -> DocumentKey

* Sample of LRU test refactor (#1424)

* Sample of test refactor

* Drop errant 'struct' keyword

* More test refactoring

* Make gc a member variable too

* Tests refactored to use member variables

* Assign delegates procedurally, rather than passing in a block

* Fix up FSTLevelDB a little

* listens -> listenTargets

* Rename GCEnabled

* remove isSentinel

* reword comment

* Add case for non-eager GC to test

* Style

* Cleaning the lint trap

* Clean up LRU tests (#1433)

* Add copious helpers and some docs to lru tests

* Style

* Method name refactor, add comment

* Drop FSTGarbageCollector and FSTEagerGarbageCollector (#1440)

* Drop FSTGarbageCollector and FSTEagerGarbageCollector

* Drop unused containsKey from FSTLevelDBMutationQueue

* Style

* Include sequence number in FSTQueryData hash and isEqual (#1568)

* Include sequence number in FSTQueryData hash and isEqual methods

* Style

* Switch to util::Hash

* Fix preserving query cache metadata (#1569)

* Add failing test

* Move the restart tests to the leveldb test suite

* Style

* Add new persistence test helper

* Fix test failure

* Review feedback

* Style
2018-07-27 13:58:11 -07:00
Gil fe0952c832 Allow remote updates from watch to heal a cache with synthesized deletes in it (#1557)
* Update spec tests from the js-sdk

* Allow remote updates from watch to heal a cache with synthesized deletes in it

Port of https://github.com/firebase/firebase-js-sdk/pull/1015

Addresses #1548
2018-07-19 07:15:47 -07:00
Sebastian Schmidt afea8d5aac Refactor Remote Event (#1396) 2018-06-12 10:58:35 -07:00
Gil 8037a4bf79 Replace Objective-C assertions with C++ assertions (#1327)
* Migrate FSTFail to HARD_FAIL
* FSTCFail -> HARD_FAIL
* FSTCAssert -> HARD_ASSERT
* FSTAssert -> HARD_ASSERT
* Replace FSTAssert with NSAssert in dead Objective-C code
* Remove FSTAssert.h
2018-05-24 11:00:23 -07:00
Gil af19a1b072 Reimplement C++ logging (#1313)
* Add direct support for formatting Objective-C objects
* Rewrite log.h
* Convert FSTWarn to LOG_WARN
* Convert FSTLog to LOG_DEBUG
* Remove FSTLogger
2018-05-23 07:00:05 -07:00
Greg Soltis dddd75037d Introduce ReferenceDelegates (#1222)
* Bump sequence number on resume token refresh

* Style

* Fix comment formatting

* Add FSTReferenceDelegate definition and documentation

* Add methods to return nil for delegates, wire up inMemoryPins

* Add hook for removing a reference

* Start work on reference delegates

* Fix up tests to support adding documents at a sequence number

* Implement removing references

* Remove from target when dropped from local view

* Fix warning

* Add hooks for removal from mutation queue

* Add hooks for limbo document updates

* Style

* Drop commented-out code

* Fixup after merging master

* Drop sequence number plumbing

* Style

* Drop errant semicolon
2018-05-08 10:49:54 -07:00
Gil 5af6de9e69 Migrate FSTDocumentVersionDictionary to C++ DocumentVersionMap (#1231) 2018-05-05 09:00:53 -07:00
Gil a080e481b5 Port FSTDocumentKeySet to C++ DocumentKeySet (#1229)
* Define a Comparator for DocumentKey

* Automated migration from FSTDocumentKeySet to DocumentKeySet

* Manual fixups for DocumentKeySet

* Delete FSTDocumentKeySet
2018-05-05 08:10:51 -07:00
Greg Soltis 66f2ce4c39 Bump sequence numbers when receiving a new resume token (#1214)
* Bump sequence number on resume token refresh

* Style

* Fix comment formatting
2018-05-03 12:11:39 -07:00
zxu 4e1ffe4649 Port SnapshotVersion for the rest Firestore modulo (#1185)
* remove conversions in `Local` and `Remote`

* replace `FSTSnapshotVersion` in `Model`

* replace `FSTSnapshotVersion` in the rest

* let test check equal with C++ equal

* fix newly added code

* address changes
2018-05-01 13:37:53 -04:00
zxu f9bd6d3c94 Port SnapshotVersion for Local (#1182)
* no-brainer replace `FSTSnapshotVersion` with `SnapshotVersion`

* refresh code

* fix LocalStore

* fix LocalStore (partial revert 78e6c38)

* address changes
2018-04-25 14:05:36 -04:00
Gil 0a66187053 Add lint checking for Objective-C++ sources (#1048)
* lint.sh now lints Objective-C++ too

* cpplint checks system-style includes that should be user-style

This prevents it from recognizing project sources as if they were C
system headers and then complaining that they're in the wrong place.

* cpplint checks #imports and #includes

* cpplint checks that C++ system headers aren't #imported

* cpplint checks for C system headers that could be C++ system headers

* cpplint checks that Objective-C sources include their headers
2018-04-09 11:54:39 -07:00
Greg Soltis c46e7f2928 Drop FSTRemoteDocumentChangeBuffer (#1013) 2018-04-02 16:01:03 -07:00
Greg Soltis 3f36f5467d Drop FSTWriteGroup (#986)
* Drop write group from remote document change buffer

* Unwind some group dependendencies in local store

* Write group dropped from local store

* Drop write group from mutation queue tests

* Drop write group usage from query cache tests

* Drop write groups from remote document cache tests

* Drop write groups from remote document change buffer tests

* Drop write groups and the write group tracker

* Style

* Put the action in transaction

* Merge master, fix test

* Fix some compiler warnings but mostly trigger travis

* Responses to feedback
2018-03-30 10:18:25 -07:00
Greg Soltis 861ad89de9 Remove unused shutdown methods (#993)
* Drop mutation queue shutdown

* Drop query cache shutdown

* Drop remote document cache shutdown

* Drop local store shutdown

* drop newline
2018-03-29 10:00:01 -07:00
Greg Soltis d9b2d312d9 Remove some usages of FSTWriteGroup (#983)
* FSTMutationQueue no longer needs groups

* Remove write group from mutation queue entirely

* Remove write group from query cache

* Mark groups unused in remote document cache

* Drop write group entirely from remote document cache

* style

* Drop commented-out code

* Drop commented-out code
2018-03-29 08:55:10 -07:00
Greg Soltis 5d38a3512d Running a transaction (#969)
* Start work on leveldb transactions

* Style

* Working API. Not plumbed in yet

* Move files into correct place

* Wrangling file locations and associations

* Tests pass

* Add some comments

* style

* Fix copyright

* Rewrite iterator internals to handle deletion-while-iterating. Also add tests for same

* Switch to strings instead of slices

* Style

* More style fixes

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Response to feedback before updating docs

* Style

* Add comment

* Initialize version_

* Satisfy the linter

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Checkpoint before implementing BatchDescription

* Style

* Initial plumbing for leveldb local parts

* Add model::BatchId

* Port leveldb_key.{h,cc}

* Add string StartsWith

* Add leveldb_key_test.cc to the project

* Revert back to using leveldb::Slice for read/describe

These operations universally operate on keys obtained from leveldb so
it's actually unhelpful to force all the callers to make
absl::string_views from them.

* Everything passing

* Drop unused function

* Style

* STart work on reads

* Swap reads in queryCache to use transactions

* Fix up tests of querycache

* Drop commented out code

* Cleanup

* Style

* Fix up for passing tests

* style

* Renaming

* Style

* Start work on ToString for transactions

* Add ToString() method to LevelDbTransaction

* Style

* lint

* Fix includes, drop runTransaction

* current_transaction -> currentTransaction

* LevelDbTransaction::NewIterator now returns a unique_ptr

* Style

* Revert addition of util::StartsWith

* Add log line

* Style

* Add log line

* Style

* Add debug log line for commits, drop unused BatchDescription

* STart work on reads

* Swap reads in queryCache to use transactions

* Start on remote documents

* Transition mutation queue and remote documents to use transactions

* Style

* Make everything pass

* Make everything pass

* Make it compile

* Style

* Style

* Revert name change, use DefaultReadOptions()

* Style

* Example of running a transaction with a lambda

* Drop errant typo

* Drop duplicate method declarations

* replace usage of auto w/ decltype

* Drop an unnecessary _Nullable. Add some nullability warning suppression

* use absl::make_unique, handle void return type

* Style

* Wrap backing persistence and expectation of backing persistence for transaction runner into a method

* More comments, trigger CI
2018-03-26 14:29:51 -07:00
Greg Soltis f77ec68a88 Switch LevelDB MutationQueue and RemoteDocumentCache to use transactions (#968)
* Start work on leveldb transactions

* Style

* Working API. Not plumbed in yet

* Move files into correct place

* Wrangling file locations and associations

* Tests pass

* Add some comments

* style

* Fix copyright

* Rewrite iterator internals to handle deletion-while-iterating. Also add tests for same

* Switch to strings instead of slices

* Style

* More style fixes

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Response to feedback before updating docs

* Style

* Add comment

* Initialize version_

* Satisfy the linter

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Checkpoint before implementing BatchDescription

* Style

* Initial plumbing for leveldb local parts

* Add model::BatchId

* Port leveldb_key.{h,cc}

* Add string StartsWith

* Add leveldb_key_test.cc to the project

* Revert back to using leveldb::Slice for read/describe

These operations universally operate on keys obtained from leveldb so
it's actually unhelpful to force all the callers to make
absl::string_views from them.

* Everything passing

* Drop unused function

* Style

* STart work on reads

* Swap reads in queryCache to use transactions

* Fix up tests of querycache

* Drop commented out code

* Cleanup

* Style

* Fix up for passing tests

* style

* Renaming

* Style

* Start work on ToString for transactions

* Add ToString() method to LevelDbTransaction

* Style

* lint

* Fix includes, drop runTransaction

* current_transaction -> currentTransaction

* LevelDbTransaction::NewIterator now returns a unique_ptr

* Style

* Revert addition of util::StartsWith

* Add log line

* Style

* Add log line

* Style

* Add debug log line for commits, drop unused BatchDescription

* STart work on reads

* Swap reads in queryCache to use transactions

* Start on remote documents

* Transition mutation queue and remote documents to use transactions

* Style

* Make everything pass

* Make everything pass

* Make it compile

* Style

* Style

* Revert name change, use DefaultReadOptions()

* Style

* Handle iterators returning bad statuses
2018-03-26 11:57:34 -07:00
zxu b3050e42d8 port C++ DocumentKey to Remote/* (#965)
* port C++ DocumentKey to Remote's

* port C++ DocumentKey to Remote's

* address changes

* address changes

* address changes
2018-03-24 11:06:32 -04:00
zxu 0ccfd6a3dc port C++ DocumentKey to Local/* (#963)
* port C++ DocumentKey to Local's

* address changes
2018-03-23 17:52:01 -04:00
Greg Soltis 352b790b6e Switch FSTLevelDBQueryCache to use transactions (#942)
* Start work on leveldb transactions

* Style

* Working API. Not plumbed in yet

* Move files into correct place

* Wrangling file locations and associations

* Tests pass

* Add some comments

* style

* Fix copyright

* Rewrite iterator internals to handle deletion-while-iterating. Also add tests for same

* Switch to strings instead of slices

* Style

* More style fixes

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Response to feedback before updating docs

* Style

* Add comment

* Initialize version_

* Satisfy the linter

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Checkpoint before implementing BatchDescription

* Style

* Initial plumbing for leveldb local parts

* Add model::BatchId

* Port leveldb_key.{h,cc}

* Add string StartsWith

* Add leveldb_key_test.cc to the project

* Revert back to using leveldb::Slice for read/describe

These operations universally operate on keys obtained from leveldb so
it's actually unhelpful to force all the callers to make
absl::string_views from them.

* Everything passing

* Drop unused function

* Style

* STart work on reads

* Swap reads in queryCache to use transactions

* Fix up tests of querycache

* Drop commented out code

* Cleanup

* Style

* Fix up for passing tests

* style

* Renaming

* Style

* Start work on ToString for transactions

* Add ToString() method to LevelDbTransaction

* Style

* lint

* Fix includes, drop runTransaction

* current_transaction -> currentTransaction

* LevelDbTransaction::NewIterator now returns a unique_ptr

* Style

* Revert addition of util::StartsWith

* Add log line

* Style

* Add log line

* Style

* Add debug log line for commits, drop unused BatchDescription

* Fix an include, but mostly try to trigger a travis build
2018-03-22 17:31:16 -07:00
Konstantin Varlamov 14ea068f5f Make FSTTimestamp into a public Firestore class (#698)
- FSTTimestamp is now FIRTimestamp, under Firestore/Source/{Public,API}. This is a temporary solution; eventually, FIRTimestamp is supposed to live somewhere under Firebase;
- move most internal Timestamp methods to the public header (the only exception is ISOString).
2018-02-20 17:33:57 -05:00
zxu fd9fd271d0 replacing Auth/FSTUser by C++ auth implementation (#804)
* replacing Auth/FSTUser by C++ auth implementation

* address changes
2018-02-15 17:23:08 -05:00
Greg Soltis 5f5f808258 Keep track of number of queries in the query cache (#776)
* Implement schema versions

* Style fixes

* newlines, copyrights, assumptions

* Fix nullability

* Raw ptr -> shared_ptr

* kVersionTableGlobal -> kVersionGlobalTable

* Drop utils, move into static methods

* Drop extra include

* Add a few more comments

* Move version constant into migrations file

* formatting?

* Fix comment

* Split add and update queryData

* Work on adding targetCount

* More work on count

* Using shared_ptr

* Implement count for query cache

* use quotes

* Add cast

* Styling

* Revert year bump in copyright

* Add adversarial key to migration test

* Add comment

* Fix style
2018-02-13 11:12:17 -08:00