Commit Graph
21 Commits
Author SHA1 Message Date
Gil 00f4f6d037 Migrate LevelDB Key classes to C++ (#1651)
* Migrate to LevelDbVersionKey

* Migrate to LevelDbMutationKey

* Migrate to LevelDbDocumentMutationKey

* Migrate to LevelDbMutationQueueKey

* Migrate to LevelDbTargetGlobalKey

* Migrate to LevelDbTargetKey

* Migrate to LevelDbQueryTargetKey

* Migrate to LevelDbTargetDocumentKey

* Migrate to LevelDbDocumentTargetKey

* Migrate to LevelDbRemoteDocumentKey

Also:
  - Remove FSTLevelDBKey and tests
  - Remove StringView and tests
2018-08-08 12:34:03 -07:00
Greg Soltis 16a5210b3c Merging the LRU branch (#1600)
* Switch to reference delegates for garbage collection (#1410)

* Bump sequence number on resume token refresh

* Style

* Fix comment formatting

* Add FSTReferenceDelegate definition and documentation

* Add methods to return nil for delegates, wire up inMemoryPins

* Add hook for removing a reference

* Start work on reference delegates

* Fix up tests to support adding documents at a sequence number

* Implement removing references

* Remove from target when dropped from local view

* Fix warning

* Add hooks for removal from mutation queue

* Add hooks for limbo document updates

* Style

* Drop commented-out code

* Fixup after merging master

* Start importing delegate implementations

* Memory LRU tests pass

* Make memory lru work, start fixing up lru tests. All with hidden sequence numbers

* Most LRU tests passing w/ memory

* Big LRU test passes w/ memory

* Leveldb LRU tests pass

* All passing except local store tests

* All tests pass

* Fix localstore tests

* Revert removeQueryData change

* Remove collectGarbage calls

* Remove NoOp GC

* More cleanup

* Refactor FSTHelpers

* Cleanup and commenting

* Comments and cleanup

* Comments

* Drop nullable for reference delegates, and old persistence method on spec tests

* Fix missing sequence bump, remove unneeded targetID

* Style

* Drop commented-out code

* Satisfy the linters and copyrighters, etc.

* Drop more commented-out code

* Fix import -> include, NSUInteger -> int

* Start work on test comments

* FSTDocumentKey -> DocumentKey

* Sample of LRU test refactor (#1424)

* Sample of test refactor

* Drop errant 'struct' keyword

* More test refactoring

* Make gc a member variable too

* Tests refactored to use member variables

* Assign delegates procedurally, rather than passing in a block

* Fix up FSTLevelDB a little

* listens -> listenTargets

* Rename GCEnabled

* remove isSentinel

* reword comment

* Add case for non-eager GC to test

* Style

* Cleaning the lint trap

* Clean up LRU tests (#1433)

* Add copious helpers and some docs to lru tests

* Style

* Method name refactor, add comment

* Drop FSTGarbageCollector and FSTEagerGarbageCollector (#1440)

* Drop FSTGarbageCollector and FSTEagerGarbageCollector

* Drop unused containsKey from FSTLevelDBMutationQueue

* Style

* Include sequence number in FSTQueryData hash and isEqual (#1568)

* Include sequence number in FSTQueryData hash and isEqual methods

* Style

* Switch to util::Hash

* Fix preserving query cache metadata (#1569)

* Add failing test

* Move the restart tests to the leveldb test suite

* Style

* Add new persistence test helper

* Fix test failure

* Review feedback

* Style
2018-07-27 13:58:11 -07:00
Gil 756da9c10c Add a schema migration that drops the query cache (#1558)
This is a force fix for potential existence filter mismatches caused by
firebase/firebase-ios-sdk#1548

The essential problem is that when resuming a query, the server is
allowed to forget deletes. If the number of incorrectly synthesized
deletes matches the number of server-forgotten deletes then the
existence filter can give a false positive, preventing the cache from
self healing.

Dropping the query cache clears any client-side resume token which
prevents a false positive existence filter mismatch.

Note that the remote document cache and mutation queues are unaffected
so any cached documents that do exist will still work while offline

https://github.com/firebase/firebase-js-sdk/pull/1019
2018-07-19 11:21:22 -07:00
Gil afd897619f Minor LevelDB cleanups (#1401)
* Add leveldb_transaction* to the CMake build

* Use std::unique_ptr for leveldb::DB pointers
2018-06-13 08:12:09 -07:00
Gil 03d26a76a0 Remove nearly all usages of WrapNSStringNoCopy (#1351) 2018-05-30 10:24:40 -07:00
Gil 8037a4bf79 Replace Objective-C assertions with C++ assertions (#1327)
* Migrate FSTFail to HARD_FAIL
* FSTCFail -> HARD_FAIL
* FSTCAssert -> HARD_ASSERT
* FSTAssert -> HARD_ASSERT
* Replace FSTAssert with NSAssert in dead Objective-C code
* Remove FSTAssert.h
2018-05-24 11:00:23 -07:00
Gil af19a1b072 Reimplement C++ logging (#1313)
* Add direct support for formatting Objective-C objects
* Rewrite log.h
* Convert FSTWarn to LOG_WARN
* Convert FSTLog to LOG_DEBUG
* Remove FSTLogger
2018-05-23 07:00:05 -07:00
Greg Soltis dddd75037d Introduce ReferenceDelegates (#1222)
* Bump sequence number on resume token refresh

* Style

* Fix comment formatting

* Add FSTReferenceDelegate definition and documentation

* Add methods to return nil for delegates, wire up inMemoryPins

* Add hook for removing a reference

* Start work on reference delegates

* Fix up tests to support adding documents at a sequence number

* Implement removing references

* Remove from target when dropped from local view

* Fix warning

* Add hooks for removal from mutation queue

* Add hooks for limbo document updates

* Style

* Drop commented-out code

* Fixup after merging master

* Drop sequence number plumbing

* Style

* Drop errant semicolon
2018-05-08 10:49:54 -07:00
Gil 0a66187053 Add lint checking for Objective-C++ sources (#1048)
* lint.sh now lints Objective-C++ too

* cpplint checks system-style includes that should be user-style

This prevents it from recognizing project sources as if they were C
system headers and then complaining that they're in the wrong place.

* cpplint checks #imports and #includes

* cpplint checks that C++ system headers aren't #imported

* cpplint checks for C system headers that could be C++ system headers

* cpplint checks that Objective-C sources include their headers
2018-04-09 11:54:39 -07:00
Greg Soltis 3f36f5467d Drop FSTWriteGroup (#986)
* Drop write group from remote document change buffer

* Unwind some group dependendencies in local store

* Write group dropped from local store

* Drop write group from mutation queue tests

* Drop write group usage from query cache tests

* Drop write groups from remote document cache tests

* Drop write groups from remote document change buffer tests

* Drop write groups and the write group tracker

* Style

* Put the action in transaction

* Merge master, fix test

* Fix some compiler warnings but mostly trigger travis

* Responses to feedback
2018-03-30 10:18:25 -07:00
Greg Soltis 5d38a3512d Running a transaction (#969)
* Start work on leveldb transactions

* Style

* Working API. Not plumbed in yet

* Move files into correct place

* Wrangling file locations and associations

* Tests pass

* Add some comments

* style

* Fix copyright

* Rewrite iterator internals to handle deletion-while-iterating. Also add tests for same

* Switch to strings instead of slices

* Style

* More style fixes

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Response to feedback before updating docs

* Style

* Add comment

* Initialize version_

* Satisfy the linter

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Checkpoint before implementing BatchDescription

* Style

* Initial plumbing for leveldb local parts

* Add model::BatchId

* Port leveldb_key.{h,cc}

* Add string StartsWith

* Add leveldb_key_test.cc to the project

* Revert back to using leveldb::Slice for read/describe

These operations universally operate on keys obtained from leveldb so
it's actually unhelpful to force all the callers to make
absl::string_views from them.

* Everything passing

* Drop unused function

* Style

* STart work on reads

* Swap reads in queryCache to use transactions

* Fix up tests of querycache

* Drop commented out code

* Cleanup

* Style

* Fix up for passing tests

* style

* Renaming

* Style

* Start work on ToString for transactions

* Add ToString() method to LevelDbTransaction

* Style

* lint

* Fix includes, drop runTransaction

* current_transaction -> currentTransaction

* LevelDbTransaction::NewIterator now returns a unique_ptr

* Style

* Revert addition of util::StartsWith

* Add log line

* Style

* Add log line

* Style

* Add debug log line for commits, drop unused BatchDescription

* STart work on reads

* Swap reads in queryCache to use transactions

* Start on remote documents

* Transition mutation queue and remote documents to use transactions

* Style

* Make everything pass

* Make everything pass

* Make it compile

* Style

* Style

* Revert name change, use DefaultReadOptions()

* Style

* Example of running a transaction with a lambda

* Drop errant typo

* Drop duplicate method declarations

* replace usage of auto w/ decltype

* Drop an unnecessary _Nullable. Add some nullability warning suppression

* use absl::make_unique, handle void return type

* Style

* Wrap backing persistence and expectation of backing persistence for transaction runner into a method

* More comments, trigger CI
2018-03-26 14:29:51 -07:00
Greg Soltis f77ec68a88 Switch LevelDB MutationQueue and RemoteDocumentCache to use transactions (#968)
* Start work on leveldb transactions

* Style

* Working API. Not plumbed in yet

* Move files into correct place

* Wrangling file locations and associations

* Tests pass

* Add some comments

* style

* Fix copyright

* Rewrite iterator internals to handle deletion-while-iterating. Also add tests for same

* Switch to strings instead of slices

* Style

* More style fixes

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Response to feedback before updating docs

* Style

* Add comment

* Initialize version_

* Satisfy the linter

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Checkpoint before implementing BatchDescription

* Style

* Initial plumbing for leveldb local parts

* Add model::BatchId

* Port leveldb_key.{h,cc}

* Add string StartsWith

* Add leveldb_key_test.cc to the project

* Revert back to using leveldb::Slice for read/describe

These operations universally operate on keys obtained from leveldb so
it's actually unhelpful to force all the callers to make
absl::string_views from them.

* Everything passing

* Drop unused function

* Style

* STart work on reads

* Swap reads in queryCache to use transactions

* Fix up tests of querycache

* Drop commented out code

* Cleanup

* Style

* Fix up for passing tests

* style

* Renaming

* Style

* Start work on ToString for transactions

* Add ToString() method to LevelDbTransaction

* Style

* lint

* Fix includes, drop runTransaction

* current_transaction -> currentTransaction

* LevelDbTransaction::NewIterator now returns a unique_ptr

* Style

* Revert addition of util::StartsWith

* Add log line

* Style

* Add log line

* Style

* Add debug log line for commits, drop unused BatchDescription

* STart work on reads

* Swap reads in queryCache to use transactions

* Start on remote documents

* Transition mutation queue and remote documents to use transactions

* Style

* Make everything pass

* Make everything pass

* Make it compile

* Style

* Style

* Revert name change, use DefaultReadOptions()

* Style

* Handle iterators returning bad statuses
2018-03-26 11:57:34 -07:00
Greg Soltis 352b790b6e Switch FSTLevelDBQueryCache to use transactions (#942)
* Start work on leveldb transactions

* Style

* Working API. Not plumbed in yet

* Move files into correct place

* Wrangling file locations and associations

* Tests pass

* Add some comments

* style

* Fix copyright

* Rewrite iterator internals to handle deletion-while-iterating. Also add tests for same

* Switch to strings instead of slices

* Style

* More style fixes

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Response to feedback before updating docs

* Style

* Add comment

* Initialize version_

* Satisfy the linter

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Checkpoint before implementing BatchDescription

* Style

* Initial plumbing for leveldb local parts

* Add model::BatchId

* Port leveldb_key.{h,cc}

* Add string StartsWith

* Add leveldb_key_test.cc to the project

* Revert back to using leveldb::Slice for read/describe

These operations universally operate on keys obtained from leveldb so
it's actually unhelpful to force all the callers to make
absl::string_views from them.

* Everything passing

* Drop unused function

* Style

* STart work on reads

* Swap reads in queryCache to use transactions

* Fix up tests of querycache

* Drop commented out code

* Cleanup

* Style

* Fix up for passing tests

* style

* Renaming

* Style

* Start work on ToString for transactions

* Add ToString() method to LevelDbTransaction

* Style

* lint

* Fix includes, drop runTransaction

* current_transaction -> currentTransaction

* LevelDbTransaction::NewIterator now returns a unique_ptr

* Style

* Revert addition of util::StartsWith

* Add log line

* Style

* Add log line

* Style

* Add debug log line for commits, drop unused BatchDescription

* Fix an include, but mostly try to trigger a travis build
2018-03-22 17:31:16 -07:00
Greg Soltis 4afaafdc77 Change write groups to use transactions (#912)
* Start work on leveldb transactions

* Style

* Working API. Not plumbed in yet

* Move files into correct place

* Wrangling file locations and associations

* Tests pass

* Add some comments

* style

* Fix copyright

* Rewrite iterator internals to handle deletion-while-iterating. Also add tests for same

* Switch to strings instead of slices

* Style

* More style fixes

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Response to feedback before updating docs

* Style

* Add comment

* Initialize version_

* Satisfy the linter

* Start switching writegroup over

* Swap out write group tracking for transaction usage

* Style

* Checkpoint before implementing BatchDescription

* Style

* Everything passing

* Drop unused function

* Style

* Renaming

* Style

* Add log line

* Style

* Add debug log line for commits, drop unused BatchDescription
2018-03-22 15:45:05 -07:00
zxu 9e815620e9 grand PR to port the remaining paths (FieldPath and ResourcePath). (#865)
* naively remove FSTPath import and source/test files.

* port FieldPath, part I

* port FieldPath, part II

* port ResourcePath, part I

* port ResourcePath, part II

* the grand commit to fix build errors

* use testutil:: helper instead of those from FSTHelpers

* fix test and lint

* use c_str in errmsg directly

* fix

* fix

* make code clean

* fix integration test I missed

* fix to avoid naming collision in preprocessor

* address changes

* address changes

* address changes

* fix: fieldMask are actually shared with different context.

* address changes

* address changes
2018-03-14 08:57:31 -04:00
zxu fd9fd271d0 replacing Auth/FSTUser by C++ auth implementation (#804)
* replacing Auth/FSTUser by C++ auth implementation

* address changes
2018-02-15 17:23:08 -05:00
zxu a441190635 Implement Firestore DatabaseInfo and port both Database{Id,Info} C++ to the iOS code (#738)
* implement Firestore DatabaseInfo in C++

* temporary stash changes; blocking on the massive renaming of .m to .mm

* add database_info_test to project

* finish port DatabaseId and fix style, modular fixing DatabaseInfo

* port DatabaseInfo

* remove FSTDatabase{ID,Info} and their tests from project

* fix unit test

* use namespace alias

* use namespace alias, leftover

* address more changes

* refactoring to use raw pointer instead of value for property

* address changes

* remove self->

* fix style

* remove the name suffix Alloc

* fix a bug
2018-02-06 13:51:11 -05:00
Greg Soltis 3cbdbf2652 Schema migrations for LevelDB (#728)
* Implement schema versions

* Style fixes

* newlines, copyrights, assumptions

* Fix nullability

* Raw ptr -> shared_ptr

* kVersionTableGlobal -> kVersionGlobalTable

* Drop utils, move into static methods

* Drop extra include

* Add a few more comments

* Move version constant into migrations file

* formatting?

* Fix comment
2018-01-30 14:36:22 -08:00
Michael Lehenbauer dc0b29e9cf Run style.sh (#636)
Run style.sh (w/ clang-format version 6)
2018-01-10 09:28:44 -08:00
Gil 4fbc96904f Use fully qualified imports in Firestore (#467)
This simplifies the import process back into google3 and allows us to add additional directories to the project without needing to update the project files for each directory we add. This shows up most clearly in the change to Firestore/Example/Firestore.xcodeproj/project.pbxproj: this no longer needs to list essentially every directory in the project as a header search path.

* Clang-format configuration for C++ files
* Add support C++-only sources to the podspec
* Podspec support for fully qualified imports
* xcodeproj changes for fully-qualified imports
* Use fully-qualified imports in Firestore sources
2017-11-16 20:52:01 -08:00
Gil bde743ed25 Release 4.3.0 (#327)
Initial release of Firestore at 0.8.0
Bump FirebaseCommunity to 0.1.3
2017-10-03 08:55:22 -07:00