Commit Graph
241 Commits
Author SHA1 Message Date
Paul Beusterien 5d0a1c8ccb Merge remote-tracking branch 'origin/firebase7-main' into pb-merge-firebase7 2020-09-30 14:23:40 -07:00
Sebastian Schmidt 24a77d04c8 Import Spec tests (#6525)
* Import Spec tests

* Init

* Add recovery specs

* Update FSTSyncEngineTestDriver.mm
2020-09-22 18:48:04 -07:00
Paul Beusterien a39b515f3a Update podspecs to min iOS 10 (#6517) 2020-09-22 15:11:35 -07:00
Gil 73fcb194d8 Add conditional conformance to Hashable to Firestore property wrapper types (#6249)
* Remove the extension that makes NSDate ServerTimestampWrappable.

This turns out not to actually work because NSDate does not conform to
Codable. Users can supply their own extensions that make NSDate conform
to Codable and also supply an extension like this one to make NSDate
conform to ServerTimestampWrappable.

Add a test that verifies ServerTimestamps of Dates actually work. The
equivalent test for NSDate does not compile.

* Fix minimum Swift version for FirebaseFirestoreSwift

FirebaseFirestoreSwift was already using synthesized Equatable and
Hashable conformance which requires Swift 4.1.

* Add conditional conformance to Hashable to Firestore property wrapper types

Mark these types as conforming to Hashable if the underlying type itself
conforms to hashable. This implementation relies on synthesized
Equatable and Hashable conformance that shipped as a part of Swift 4.1.

* Fix lint/style

* Changelog
2020-08-12 10:26:57 -04:00
Gil e462d04ddb Make Executors have deterministic shutdown (#5547)
In particular:

  * All tasks are guaranteed to either run to completion or not run at all;
  * Executors can now be destroyed from the threads they own (if applicable); and
  * Many common concepts between the Executor implementations have been harmonized and shared.

Note:

  * This change does not yet implement `Executor::Dispose` or clean up any of the higher level componentry to take advantage of these changes.
  * The executors implement shutdown by canceling any tasks that haven't started yet. This diverges from the plan (which called for a separate `UserCallbackExecutor` to handle this), but in the end it turns out to be simpler to just cancel everything and make cancel handle tasks-in-progress in a reasonable way. LMK if you have grave objections to this, in which case I can rework this.

This is the foundation for addressing firebase/quickstart-unity#638.
2020-05-18 08:30:02 -07:00
Gil b57f8b2ce6 Add defer, a general facility for ad-hoc RAII. (#5554) 2020-05-13 10:29:58 -07:00
Gil e3f3a78aef Remove the Firestore_SwiftTests_iOS target (#5458)
The tests in this target are already covered by
Firestore_IntegrationTests_iOS and also run on the other platforms
through the other platform variants of this target.

Also remove an explicit override of CLANG_WARN_COMMA in the
Firestore_Example_macOS target. This should have been cleaned up earlier
--no other targets have this setting.
2020-04-26 15:01:42 -07:00
Gil 7a58eba2e5 Remove firebase/firestore from paths in Firestore/core (#5374)
* Manual Xcode project fixups

* Remove `firebase/firestore` from paths in Firestore/core

This renames all Firestore files under `Firestore/core` to remove the
interior `firebase/firestore` directory structure. This is motivated by
build failures under Windows where the filenames of intermediates were
too long.

Concretely:

  * `Firebase/core/src/firebase/firestore` becomes `Firebase/core/src`
  * `Firebase/core/test/firebase/firestore` becomes
    `Firebase/core/test/unit`

Moving the tests into a `unit` subdirectory aligns Firestore with the
directory structure in
[AddNewPod](https://github.com/firebase/firebase-ios-sdk/blob/master/AddNewPod.md),
modified for C++ (see discussion in #3224).
2020-04-20 14:57:34 -07:00
Gil af7898c01e Merge remote_test_util and remote_testing libraries (#5254)
These two libraries duplicate each other in purpose:

  * firebase_firestore_remote_test_util
  * firebase_firestore_remote_testing

Additionally, several of the testing utilities in remote_test_util are
really only applicable to tests in the remote library anyway. These have
been put directly into firebase_firestore_remote_test.
2020-04-03 15:57:07 -07:00
Gil bbc8de404c Add a benchmark for query execution (#4731)
* Use a valid project id

This avoids tripping up over newly added validation in the emulator

* Allow passing arguments to the emulator

e.g. to pass a non-default host

* Add a benchmark for reading documents

Baseline results:

Run on (12 X 2900 MHz CPU s)
CPU Caches:
  L1 Data 32K (x6)
  L1 Instruction 32K (x6)
  L2 Unified 262K (x6)
  L3 Unified 12582K (x1)
Load Average: 2.11, 1.96, 1.92
---------------------------------------------------------------------
Benchmark                           Time             CPU   Iterations
---------------------------------------------------------------------
BM_QueryMatching/0/1           168359 ns        13750 ns        53878
BM_QueryMatching/1/1           179118 ns        16706 ns        40193
BM_QueryMatching/1/10          721218 ns        35729 ns        10000
BM_QueryMatching/10/10         726533 ns        51864 ns        13587
BM_QueryMatching/1/100        5326040 ns        42820 ns         1000
BM_QueryMatching/100/100      5734101 ns       335289 ns         1000
BM_QueryMatching/1/1000      54110241 ns        41310 ns          100
BM_QueryMatching/10/1000     54584725 ns        95060 ns          100
BM_QueryMatching/100/1000    55083753 ns       527750 ns          100
BM_QueryMatching/1000/1000   58299823 ns        29130 ns          100
BM_QueryAll/1                  108892 ns        13355 ns        51197
BM_QueryAll/10                 657727 ns        43867 ns        10000
BM_QueryAll/100               5786091 ns       328244 ns         1000
BM_QueryAll/1000             60092767 ns        19380 ns          100

* Add a dedicated XCTest/Google Benchmark adapter

Remove FSTLevelDBBenchmarkTests. This benchmark appears to explore some
trade-off in document-size vs indexing but doesn't seem useful to
re-run.
2020-01-24 09:09:43 -08:00
Gil ecb26bf438 Migrate LevelDB data out of ~/Documents (#4609)
This moves iOS and macOS user data from `NSDocumentDirectory` to `NSApplicationSupportDirectory`, adding a pre-open migration step that:

  * Moves old data into the new location
  * Cleans up any resulting empty directories in the old Documents folder
  * Stops before removing the Documents folder itself (see tests)

Note that other platforms don't need a migration:

  * tvOS was already storing documents in `NSCachesDirectory`
  * Linux, Windows, and other UNIX haven't been publicly released or aren't changing.

See individual commits for details. Most notably this adds some additional filesystem manipulation routines to our (sadly) growing library.

Fixes #843.
2020-01-16 17:36:23 -08:00
wu-hui 51e17ac8c5 Rename QueryData|Cache to TargetData|Cache (#4670)
* Rename done by CLion

QueryCache -> TargetCache
QueryData -> TargetData

* Fix cmake list indention

* Fixing ordering.
2020-01-15 13:47:26 -05:00
Gil 6cc1a449ea Clean up Objective-C interop and other dead code (#4601)
* Remove unused -[FIRFirestore isLoggingEnabled]

Use util::LogIsLoggable(util::kLogLevelDebug) directly.

* Remove objc_compatibility.h

* Remove/cleanup unused #if __OBJC__ blocks

* Remove dead code

* Remove MakeNSStringNoCopy

* Remove worthless include

* Tidy up FIRFirestore

* Remove dead code and use forward declarations in FSTHelpers

* Minor fixes for LevelDbTransaction:

  * Make methods const where possible
  * Make key() and value() return const std::string&, allowing callers
    to create their own const references

* Remove Comparator<NSString*>
2020-01-08 18:40:29 -08:00
Sebastian Schmidt d261cdeb9b Finish Index-Free queries (still disabled) (#4484) 2020-01-07 13:29:20 -08:00
Gil e7172d5f09 Post-migration cleanups (#4451)
* Fix naming violations.

* Remove xcgmock

* Remove FSTTypes.h

* Remove FSTClasses.h
2019-12-07 18:33:18 -08:00
Gil c2fa88e3b0 Rewrite local store tests in C++ (#4421)
* Move FSTLocalStoreTests to core/test

* Add testutils for constructing TransformMutations

* Rewrite local store tests in C++
2019-12-05 08:37:15 -08:00
Sebastian Schmidt bd9d9d8b3c Index-Free Queries (disabled, some code + tests missing) (#4418) 2019-12-03 12:24:41 -08:00
Gil f0157ae7c5 Port FSTLRUGarbageCollectorTests to C++ (#4283) 2019-11-12 14:21:39 -08:00
Gil d056539200 Migrate FSTLevelDBMigrationsTests to C++ (#4272) 2019-11-11 12:15:46 -08:00
Gil 69d39f660e Port FSTLevelDBTransactionTests to C++ (#4270)
* Move FSTLevelDBTransactionTests to C++

* Update CMake project

* Update Xcode project

* Rewrite FSTLevelDBTransactionTests in C++
2019-11-08 14:48:32 -08:00
Konstantin Varlamov db8cb2321a C++ migration: cleanup leftover bits of Objective-C (#4190)
* remove `OBJC_CLASS` (it's no longer used);
* make most sources `.cc` instead of `.mm` (all exceptions are Apple platform-specific code).
2019-11-07 21:43:53 -05:00
Konstantin Varlamov d0686d2f63 C++ migration: implement pretty-printing of Nanopb-generated messages (#4184)
Note: the output is indistinguishable from what is generated by the Objective-C library, with two exceptions:
* class names are different (due to using prefixes, e.g., `GCFS`);
* order in which map fields are outputted can be different (which I presume is to be expected).

I verified this by running integration tests and using simple diffing code to compare outputs: in the destructor of each `Message<T>` (if it's not moved-from), serialize the `Message` to bytes, create an Objective-C proto out of those bytes, convert both protos to their string form, compare the strings (ignoring minor differences in indentation, etc.). Out of 5.7K comparisons, there were just two cases of differences:
* one was in `SerializerTest.FailOnInvalidInputBytes`; since the input in question is invalid, this doesn't seem to require much attention;
* another one was due to differing floating point precision: C++ produces `1.79769e+308` while Objective-C produces `1.79769313486232e+308`. Added a TODO.

Example output:
```
2019-10-28 21:44:56.941749-0400 Firestore_Example_iOS[78883:2094242] 6.11.0 - [Firebase/Firestore][I-FST000001] WriteStream (7f903ca53d50) write request: <WriteRequest 0x70000991ef08>: {
    writes {
      update {
        name: "projects/varconst-11-a51fd/databases/(default)/documents/test-collection/u6QizHeYjqt8Xe61kBoi"
        fields {
          key: "key"
          value {
            string_value: "value"
          }
        }
      }
    }
    stream_token: "\031\020hB\002\201\364\265\265"
}
```
2019-11-07 19:37:28 -05:00
Gil 540f76d38e Port FSTMutationQueueTests to C++ (#4208) 2019-11-06 19:55:13 -08:00
Gil 1953cc59a4 Integrate with Abseil via CocoaPods (#4097)
Use Abseil 0.20190808
2019-11-05 09:12:34 -08:00
wu-hui 2f53582530 Various minor fixes to make SDK compile on Linux (#4185) 2019-11-04 14:39:44 -05:00
Gil 3ea7197090 Port FSTQueryCacheTests to C++ (#4160)
* Move FSTQueryCacheTests files to Firestore/core

* Rewrite QueryCache tests in C++

* Fix test flake in local store tests

This doesn't affect production because we always set the additional
references pointer there, but it's technically optional and some tests
don't do this.
2019-11-02 09:09:25 -05:00
Konstantin Varlamov 8b23a3266d Remove Objective-C serializers and make local/ and remote/ (almost) fully C++ (#4151)
There are more sources that can be `.cc` now (e.g., in API); these are left for a follow-up PR.

The dependency on Objective-C proto library cannot be removed just yet due to current lack of pretty-printing support for Nanopb.
2019-10-23 20:56:44 -04:00
Gil 6fd815a5ed Port FSTRemoteDocumentCacheTests to C++ (#4136) 2019-10-23 12:28:16 -07:00
Gil 3f1fbe7e9a Migrate FSTPersistenceTestHelpers to C++ (#4122)
* Move FSTPersistenceHelpers to core/test/.../local

* Rewrite persistence_testing in C++

* Migrate from FSTPersistenceTestHelpers to C++ persistence_testing

* Rename .mm to .cc where possible in IndexManager tests

* Remove Objective-C gunk from index_manager_test.h
2019-10-21 20:12:46 -07:00
Konstantin Varlamov 0f2a0bfeef Use C++ serializer for communicating with the backend (#4013)
Replace all uses of `FSTSerializerBeta` in `remote_objc_bridge.{h,mm}` with the C++ `Serializer`. Also, make most of `remote/` sources pure C++ and remove no-longer-used methods in `FSTSerializerBeta`. All integration tests pass, under Address Sanitizer as well.

Unfortunately, pretty-printing is still delegated to Objective-C, so that `remote_objc_bridge.mm` still has to be an Objective-C++ source (and which prevents inlining `remote_objc_bridge` into the streams).

#no-changelog Will add a changelog entry when local serializer is ported, too.
2019-10-18 17:58:14 -04:00
Paul Beusterien d8e3a11bb1 Restore Firebase pod to closed source location (#4112) 2019-10-17 09:45:04 -07:00
Gil f32cc12ce0 Build Firestore with Xcode 11 (#4005)
* Drop dependency on OCMock

* Move Firestore onto Xcode 11 in Travis

* Special case simulators based on Xcode version, not product

This allows Firestore's build with Xcode 8.3.3 to continue to work.

* Make the Firestore Podfile install only one platform at a time

Use the PLATFORM environment variable to limit which targets are
modified. This makes the selected platform work but leaves the other
platforms in a broken state in the Xcode UI.

* Fix code signing errors on macOS builds
2019-10-07 08:46:06 -07:00
Gil 11de159e8e Migrate FSTRemoteEventTests to C++ (#3965)
* move (unchanged) into Firestore/core

* split out TestTargetMetadataProvider

... also rename to not start with "test".

* rewrite in C++
2019-10-04 11:55:20 -07:00
Gil 1a004119fa Project updates for recently added/moved files (#3994)
Occasionally Xcode will save these out--I can't predict when.
2019-10-03 16:02:14 -07:00
Sebastian Schmidt 41c411e8f4 Add QueryData.with() helpers (#3991) 2019-10-03 14:52:12 -07:00
Gil b37ab3b433 Migrate FSTViewSnapshotTests to C++ (#3919)
* view_snapshot_test: move (unchanged) from FSTViewSnapshotTest.mm

* view_snapshot_test: automated changes

* view_snapshot_test: sync_project.rb generated changes

* view_snapshot_test: manual fixes

* view_snapshot_test: shorten DocumentViewChange::Type
2019-09-30 19:55:03 -07:00
Gil 7d06d34e05 Migrate tests written as C++ into the C++ tree (#3917)
* field_transform_test: move into Firestore/core

* field_transform_test: project changes

* transform_operation_test: move into Firestore/core

* transform_operation_test: project changes
2019-09-30 18:07:59 -07:00
Gil 62c4fa30d2 Migrate FSTQueryListenerTests to C++ (#3901) 2019-09-30 13:17:53 -07:00
Gil 9ec602efcb Migrate FSTViewTests to C++ (#3900)
* Add view testing utilities

* Copy FSTViewTests.mm to view_test.cc unchanged

* Migrate FSTViewTests to C++
2019-09-30 11:26:42 -07:00
Gil 3185ff61d5 Add Expectations and Async/Await for testing (#3899)
* Add Expectations and Async/Await for testing

* Migrate async tests to Expectations and Async/Await
2019-09-27 12:25:34 -07:00
Gil 971445ce35 Add cross-platform support for creating Executors (#3897)
* Add Executor::CreateSerial

... a standard way to create an Executor for most cases where we don't
care about the implementation.

* Add ExecutorForTesting/AsyncQueueForTesting

Uses Executor::CreateSerial to do the heavy lifting.

* Use AsyncQueueForTesting

* Avoid direct references to ExecutorLibdispatch in prod code

* Rewrite time_testing in terms of absl::Time

* Migrate async_test_util's now to time_testing

* Fix AddressSanitizer stack-buffer-underflow error

This was reading random data off the stack until it hit a negative
integer's worth of data. The test passed because this was generating the
input and expected values, which were much longer than expected given
the input.
2019-09-26 10:41:36 -07:00
wu-hui 1ce982de85 Port FSTSyncEngine to C++ SyncEngine 2/2 (#3741) 2019-09-05 18:40:58 -04:00
Konstantin Varlamov 5da6b69a7d Fix accessing a destroyed object (#3721)
This was caused by running under Asan with XCode 11. Apparently, previous versions of XCode cannot catch it (which is why Travis is passing, apparently).

Also:
* enable "Detect use of stack after return" option for Asan. AFAIU, it can only be enabled in the scheme. It's already enabled for unit tests, this PR additionally enables it for integration tests;
* make Asan failures fail Travis build. They were originally not considered blocking due to some sporadic failures. I haven't seen those for some time, and worst case scenario, we can revert this change.
2019-08-29 14:14:09 -04:00
Gil 83b4e7cb57 Change many Firestore/core .mm files to .cc (#3591)
Rename .mm files .cc where all Objective-C has been removed
2019-08-19 10:44:57 -07:00
Gil aefd4323a2 Remove Objective-C immutable sets and maps (#3590)
* Remove Firestore/third_party/Immutable

* Update podspec and sync_project

* Project updates
2019-08-19 09:09:20 -07:00
Gil dd7fbf885e Migrate FSTMutationTests to C++ (#3571)
* Tidy up mutation_test.cc ahead of filling it out

* Move FSTUserDataConverter tests out of FSTMutationTests

* Add operator<< for Document

The compiler won't apply the one for MaybeDocument.

* Port FSTMutationTests to C++

* Delete FSTMutationTests
2019-08-19 08:21:43 -07:00
Gil 0c23f38397 Migrate Objective-C document types to C++ (#3531)
* Rewrite Document-related classes as a polymorphic value type

* Pass ObjectValue without rvalue references

* Rename HasLocalMutations

* Port C++ users to new Documents

* Port FSTDocumentTests to C++

* Port FSTDocumentSetTests to C++

* Migrate DocumentMap and DocumentSet to C++ Document

* Migrate test helpers

* Delete FSTDocumentSetTests

* Delete FSTDocumentTests

* Mechanical changes to use C++ Document

* Updates to the project file

* Nontrivial test updates

* Remove FSTTestDeletedDoc

* Remove FSTTestUnknownDoc

* Remove FSTDocument

* Convert usage of FSTDocument to C++ Document

* Fix Serializer usage of Documents
2019-08-13 13:59:00 -07:00
Gil 5c19bc26a5 Migrate FSTQueryTests to C++ / remove FSTQuery (#3485)
* Use C++ Documents in FSTQueryTests

* Fix FieldPath::kDocumentKeyPath definition

* Add missing operator<< to Document and DocumentKey

* Migrate FSTQueryTests.mm to C++ query_test.cc

* Remove FSTQueryTests

* Remove FSTQuery
2019-08-02 13:23:58 -07:00
Gil 2700b6101e Add AppendOnlyList (#3457)
* Add AppendOnlyList

* Avoid iterator invalidation
2019-08-01 10:21:29 -07:00
Gil 2acc073636 Remove vector_of_ptr (#3458) 2019-07-30 07:10:28 -07:00