Commit Graph
415 Commits
Author SHA1 Message Date
Gil d9b7a33f0c Use forward declarations instead of IWYU (#4868)
* Remove rarely used template argumnets from SortedMap/SortedSet

* Remove unused STATUS_CHECK_OK

* Rename QuerySnapshot::Listener to QuerySnapshotListener

This makes the type something that can be forward declared.

* Rename DocumentSnapshot::Listener to DocumentSnapshotListener

This makes the type something that can be forward declared.

* Rename ViewSnapshot::Listener to ViewSnapshotListener

This makes the type something that can be forward declared.

* Put most QueryListener implementation in .cc

* Actually use sync_engine_callback.h

* Add forward declaration headers for api, core, and model

These centralize forward declarations, including declarations for some
hard-to-forward-declare types.

* Move includes of hard_assert.h out of other headers

* Make testutil definitions out of line

* Move definitions out-of-line

* Move DocumentKey definitions out-of-line

* Move definitions out-of-line

* Shorten unnecessarily long namespace usages

* Use default destructors where possible

* Delete unused MutationBatch::kUnknown constant

* Use forward declarations instead of IWYU

* Misc fixups from Linux/GCC 9.2.1

* Fix testutil under VS 2017

Forward declarations of functions local to a function body end up
generating unresolved externals. Reduce most of these back to just
calling FieldValue::FromX methods.
2020-02-15 14:34:38 -08:00
Sebastian Schmidt d80ccb03e8 Update Spec test protos (#4814) 2020-02-04 13:38:13 -08:00
Gil bff0d44be9 Add Firestore pod-lib-lint (#4797) 2020-02-03 09:23:39 -08:00
Gil 968da25ca8 Fix the Linux build under GCC 7.4.0 (#4761)
* Don't bother searching for XCTest on non-Apple platforms

* Force working directory to be in the project root

This makes the spec tests work when cmake is invoked with a relative filename.

* Fix build error on Linux/GCC 7.4.0
2020-01-29 10:23:42 -08:00
Gil ecb26bf438 Migrate LevelDB data out of ~/Documents (#4609)
This moves iOS and macOS user data from `NSDocumentDirectory` to `NSApplicationSupportDirectory`, adding a pre-open migration step that:

  * Moves old data into the new location
  * Cleans up any resulting empty directories in the old Documents folder
  * Stops before removing the Documents folder itself (see tests)

Note that other platforms don't need a migration:

  * tvOS was already storing documents in `NSCachesDirectory`
  * Linux, Windows, and other UNIX haven't been publicly released or aren't changing.

See individual commits for details. Most notably this adds some additional filesystem manipulation routines to our (sadly) growing library.

Fixes #843.
2020-01-16 17:36:23 -08:00
Brian Chen 9313d61a21 Add verify support (#4658) 2020-01-16 09:57:38 -08:00
wu-hui 51e17ac8c5 Rename QueryData|Cache to TargetData|Cache (#4670)
* Rename done by CLion

QueryCache -> TargetCache
QueryData -> TargetData

* Fix cmake list indention

* Fixing ordering.
2020-01-15 13:47:26 -05:00
Gil ff21f1ecb3 Make a Filesystem class that can be subclassed for testing (#4647)
This change does not yet change all the classes that use the filesystem
to take Filesystem as a dependency. Those changes will come in follow-up
PRs.

This change is motivated by feedback in #4609. There I had created a
complicated interface to LevelDbOpener just so that different migration
scenarios could be tested, when in reality, I just needed to be able to
plumb through different values for the result of AppDataDir and related
functions.
2020-01-10 06:58:01 -08:00
Sebastian Schmidt 423a62f874 Enable Index-Free queries (#4618) 2020-01-09 16:40:47 -08:00
Gil 6cc1a449ea Clean up Objective-C interop and other dead code (#4601)
* Remove unused -[FIRFirestore isLoggingEnabled]

Use util::LogIsLoggable(util::kLogLevelDebug) directly.

* Remove objc_compatibility.h

* Remove/cleanup unused #if __OBJC__ blocks

* Remove dead code

* Remove MakeNSStringNoCopy

* Remove worthless include

* Tidy up FIRFirestore

* Remove dead code and use forward declarations in FSTHelpers

* Minor fixes for LevelDbTransaction:

  * Make methods const where possible
  * Make key() and value() return const std::string&, allowing callers
    to create their own const references

* Remove Comparator<NSString*>
2020-01-08 18:40:29 -08:00
wu-hui 14e28121e2 Release LimitToLast (#4616)
* [1/4] Split Query and Target, and make LocalStore/RemoteStore work with Target. (#4413)

* [2/4]LimitToLast: Make SyncEngine work with Target (#4491)

* [3/4] Implement LimitToLast (#4505)
2020-01-08 09:04:28 -08:00
Gil 9a4f4e73c7 Miscellaneous fixes (#4584)
* Fix use after move in LevelDbPersistenceForTesting

* Fix size-related errors when compiling in 32-bit mode

* Fix test-only memory leaks

* Fix test-only memory leaks

* Prevent spec tests from appearing to leak

The spec tests aren't actually leaking, but they accumulate enough
garbage without clearing the autorelease pool that Instruments
complains.

* Avoid temporaries in methods that @throw

ARC does not release temporaries when an Objective-C exception is
thrown. Use C++ equivalents directly to avoid leaks as detected by
Instruments.

* Fix memory leak in acknowledge mutation

Any previous value must be freed before the next one is assigned.

* Manually implement assignment operators in ByteString

The single by-value implementation wasn't preventing the compiler
generated standard versions from being generated which was causing
leaks.

* Prevent leaking GrpcCompletions that complete after Shutdown

After shutdown, the AsyncQueue silently drops Enqueue requests which
prevents these blocks from being deleted.

* Avoid racing with enqueued operations

It's possible for the Executor to immediately execute the given block,
at which point the block might observe the async queue in a
not-shutting-down state.
2020-01-08 07:11:05 -08:00
Sebastian Schmidt d261cdeb9b Finish Index-Free queries (still disabled) (#4484) 2020-01-07 13:29:20 -08:00
Gil df3989b8a2 Fix lint warnings spotted by google3's linter (#4538) 2019-12-31 11:35:51 -06:00
Gil 16447a5e2b Restrict use of Abseil types in public C++ accessible APIs (#4503)
* Change string parameters from absl::string_view to std::string

No abseil types can be used as parameters in this package.

This fixes an issue where Unity builds fail due to missing symbols.

The underlying issue is that Abseil in CocoaPods is an LTS release and
LTS releases contain an inline namespace, making types like
absl::string_view link as absl::lts_2019_08_08::string_view. When
compiling Firestore in google3 it uses the abseil in //third_party/absl,
which is not an LTS release and does not include the inline namespace.
This disagreement causes issues for the public C++ API which is compiled
in google3 and consumes code in the API package.

* Avoid absl::any in interfaces shared with public C++

* Make resource-related methods class members

This gives them access to private methods of friends of Serializer.

* Access control FieldPath/ResourcePath construction

* Access control Document construction

* Access control FieldValue construction

* Access control TransformOperation methods

* Remove DocumentSnapshot constructor that takes metadata pieces individually

* Restrict access to DocumentSnapshot from optional<Document>

* Add comments indicating how Abseil types are restricted

* Make DocumentKey::FromPathString take std::string
2019-12-17 11:32:48 -08:00
Sebastian Schmidt 656b9c0889 Remove unused test helpers (#4473) 2019-12-10 13:07:52 -08:00
Gil e7172d5f09 Post-migration cleanups (#4451)
* Fix naming violations.

* Remove xcgmock

* Remove FSTTypes.h

* Remove FSTClasses.h
2019-12-07 18:33:18 -08:00
Gil 85365cb089 Get the remainder of Objective-C tests running under CMake (#4430)
* Add Objective-C API tests to the CMake build

* Fix signedness warnings under the CMake build

* Get the remainder of Objective-C tests running under CMake

Also:
  * Add a Path::HasExtension check
  * Change spec tests to pick up json without NSBundle
2019-12-06 10:11:05 -08:00
Gil c2fa88e3b0 Rewrite local store tests in C++ (#4421)
* Move FSTLocalStoreTests to core/test

* Add testutils for constructing TransformMutations

* Rewrite local store tests in C++
2019-12-05 08:37:15 -08:00
Sebastian Schmidt bd9d9d8b3c Index-Free Queries (disabled, some code + tests missing) (#4418) 2019-12-03 12:24:41 -08:00
wu-hui 2a8d069b77 Run integration tests from CMake on mac (#4276) 2019-11-19 13:40:43 -05:00
Gil f0157ae7c5 Port FSTLRUGarbageCollectorTests to C++ (#4283) 2019-11-12 14:21:39 -08:00
Brian Chen 0170957544 Allow querying for arrays in IN queries (#4285) 2019-11-12 13:07:58 -08:00
Gil d056539200 Migrate FSTLevelDBMigrationsTests to C++ (#4272) 2019-11-11 12:15:46 -08:00
Gil 69d39f660e Port FSTLevelDBTransactionTests to C++ (#4270)
* Move FSTLevelDBTransactionTests to C++

* Update CMake project

* Update Xcode project

* Rewrite FSTLevelDBTransactionTests in C++
2019-11-08 14:48:32 -08:00
Konstantin Varlamov 62ef244cde C++ migration: remove the dependency on Objective-C Protobuf library (#4189) 2019-11-07 20:39:53 -05:00
Gil 540f76d38e Port FSTMutationQueueTests to C++ (#4208) 2019-11-06 19:55:13 -08:00
Gil 38d9643a00 Disallow empty document fields (#4229)
Port of firebase/firebase-android-sdk#643.

Also make error messages align with Android's.
2019-11-04 17:03:10 -06:00
wu-hui 2f53582530 Various minor fixes to make SDK compile on Linux (#4185) 2019-11-04 14:39:44 -05:00
Gil 3ea7197090 Port FSTQueryCacheTests to C++ (#4160)
* Move FSTQueryCacheTests files to Firestore/core

* Rewrite QueryCache tests in C++

* Fix test flake in local store tests

This doesn't affect production because we always set the additional
references pointer there, but it's technically optional and some tests
don't do this.
2019-11-02 09:09:25 -05:00
Brian Chen 251a70f26a Make IN queries publicly available (#4174) 2019-10-29 14:13:02 -07:00
Brian Chen 441d7eda8b Change id to NSArray<id> for IN queries (#4182) 2019-10-28 22:22:37 -07:00
Konstantin Varlamov 8b23a3266d Remove Objective-C serializers and make local/ and remote/ (almost) fully C++ (#4151)
There are more sources that can be `.cc` now (e.g., in API); these are left for a follow-up PR.

The dependency on Objective-C proto library cannot be removed just yet due to current lack of pretty-printing support for Nanopb.
2019-10-23 20:56:44 -04:00
Gil 6fd815a5ed Port FSTRemoteDocumentCacheTests to C++ (#4136) 2019-10-23 12:28:16 -07:00
Konstantin Varlamov 2c53bb9139 C++ migration: use C++ serializer for communicating with the local store (#4116)
After this PR is merged, the Objective-C serializers can be completely removed from the project. Also, this PR changes the `Message` class quite a bit; it's probably in its final form now.

I did some performance testing locally and came to the conclusion that replacing Objective-C serializers with their C++ versions brings no performance downgrade; actually, there seems to be a very slight improvement. I tested writing large batches and reading large collections:
* both are network-dominated; wall time is about twice longer than CPU time;
* serialization represents a pretty small percentage of the total CPU time;
* for writing, C++ serialization is a little more efficient, taking ~6% of total CPU time, compared to ~8% in the Objective-C version. I've seen similar results for reading.

All integration tests pass, under Address Sanitizer as well.
2019-10-22 21:52:55 -04:00
Gil 3f1fbe7e9a Migrate FSTPersistenceTestHelpers to C++ (#4122)
* Move FSTPersistenceHelpers to core/test/.../local

* Rewrite persistence_testing in C++

* Migrate from FSTPersistenceTestHelpers to C++ persistence_testing

* Rename .mm to .cc where possible in IndexManager tests

* Remove Objective-C gunk from index_manager_test.h
2019-10-21 20:12:46 -07:00
Konstantin Varlamov 0f2a0bfeef Use C++ serializer for communicating with the backend (#4013)
Replace all uses of `FSTSerializerBeta` in `remote_objc_bridge.{h,mm}` with the C++ `Serializer`. Also, make most of `remote/` sources pure C++ and remove no-longer-used methods in `FSTSerializerBeta`. All integration tests pass, under Address Sanitizer as well.

Unfortunately, pretty-printing is still delegated to Objective-C, so that `remote_objc_bridge.mm` still has to be an Objective-C++ source (and which prevents inlining `remote_objc_bridge` into the streams).

#no-changelog Will add a changelog entry when local serializer is ported, too.
2019-10-18 17:58:14 -04:00
Gil b1be1a9c70 Ensure that projectIDs are validated (#4073)
* Avoid priming the backend for emulator tests

* Validate that nil project ids are rejected
2019-10-17 10:03:01 -07:00
Gil 11de159e8e Migrate FSTRemoteEventTests to C++ (#3965)
* move (unchanged) into Firestore/core

* split out TestTargetMetadataProvider

... also rename to not start with "test".

* rewrite in C++
2019-10-04 11:55:20 -07:00
Konstantin Varlamov 893002541b Serializer: implement encoding and decoding of Target (#3932) 2019-10-03 19:05:05 -04:00
Sebastian Schmidt 41c411e8f4 Add QueryData.with() helpers (#3991) 2019-10-03 14:52:12 -07:00
Gil b37ab3b433 Migrate FSTViewSnapshotTests to C++ (#3919)
* view_snapshot_test: move (unchanged) from FSTViewSnapshotTest.mm

* view_snapshot_test: automated changes

* view_snapshot_test: sync_project.rb generated changes

* view_snapshot_test: manual fixes

* view_snapshot_test: shorten DocumentViewChange::Type
2019-09-30 19:55:03 -07:00
Gil 7d06d34e05 Migrate tests written as C++ into the C++ tree (#3917)
* field_transform_test: move into Firestore/core

* field_transform_test: project changes

* transform_operation_test: move into Firestore/core

* transform_operation_test: project changes
2019-09-30 18:07:59 -07:00
Gil 62c4fa30d2 Migrate FSTQueryListenerTests to C++ (#3901) 2019-09-30 13:17:53 -07:00
Gil 9ec602efcb Migrate FSTViewTests to C++ (#3900)
* Add view testing utilities

* Copy FSTViewTests.mm to view_test.cc unchanged

* Migrate FSTViewTests to C++
2019-09-30 11:26:42 -07:00
Gil 971445ce35 Add cross-platform support for creating Executors (#3897)
* Add Executor::CreateSerial

... a standard way to create an Executor for most cases where we don't
care about the implementation.

* Add ExecutorForTesting/AsyncQueueForTesting

Uses Executor::CreateSerial to do the heavy lifting.

* Use AsyncQueueForTesting

* Avoid direct references to ExecutorLibdispatch in prod code

* Rewrite time_testing in terms of absl::Time

* Migrate async_test_util's now to time_testing

* Fix AddressSanitizer stack-buffer-underflow error

This was reading random data off the stack until it hit a negative
integer's worth of data. The test passed because this was generating the
input and expected values, which were much longer than expected given
the input.
2019-09-26 10:41:36 -07:00
Brian Chen b1eb602ef4 Rename spec test functions (#3895) 2019-09-20 16:30:11 -04:00
Brian Chen 9d230de9c0 Adding onSnapshotsInSync (not public) (#3850) 2019-09-20 14:15:57 -04:00
wu-hui b48636bf52 Port local store to C++ 2/2 (#3865)
Complete LocalStore port.
2019-09-18 14:00:32 -04:00
Marek Gilbert 3e939a7ee2 Merge branch 'firestore-master' 2019-09-13 12:41:25 -07:00