Commit Graph
676 Commits
Author SHA1 Message Date
Gil e404054a9f Refactor and clean up the CMake build ahead of CMake/iOS support (#5051)
* Use early exit in objc_framework CMakeLists.txt

* Clean up objc_framework CMakeLists.txt

  * Use `version` consistently
  * Use `firebase_ios_version` for the repo-level version
  * Remove INCLUDE directories that are no longer needed
  * Sort dependencies

* Simplify Example/App/CMakeLists.txt

  * Add `OBJC_FLAGS` directly, avoiding a separate `sources` variable.
  * Add resources directly

* Rename test host app to firebase_firestore_example_app

* Add a firebase_ios prefix to functions and variables

This reduces the chance of collision with other projects.

* Simplify compiler_setup

Now that compiler_setup.cmake no longer has side-effects, fold
compiler_id.cmake and archive_options.cmake into it and move
the include of compiler_setup up to the top of the main build.
2020-03-10 20:50:13 -07:00
Gil d9b7a33f0c Use forward declarations instead of IWYU (#4868)
* Remove rarely used template argumnets from SortedMap/SortedSet

* Remove unused STATUS_CHECK_OK

* Rename QuerySnapshot::Listener to QuerySnapshotListener

This makes the type something that can be forward declared.

* Rename DocumentSnapshot::Listener to DocumentSnapshotListener

This makes the type something that can be forward declared.

* Rename ViewSnapshot::Listener to ViewSnapshotListener

This makes the type something that can be forward declared.

* Put most QueryListener implementation in .cc

* Actually use sync_engine_callback.h

* Add forward declaration headers for api, core, and model

These centralize forward declarations, including declarations for some
hard-to-forward-declare types.

* Move includes of hard_assert.h out of other headers

* Make testutil definitions out of line

* Move definitions out-of-line

* Move DocumentKey definitions out-of-line

* Move definitions out-of-line

* Shorten unnecessarily long namespace usages

* Use default destructors where possible

* Delete unused MutationBatch::kUnknown constant

* Use forward declarations instead of IWYU

* Misc fixups from Linux/GCC 9.2.1

* Fix testutil under VS 2017

Forward declarations of functions local to a function body end up
generating unresolved externals. Reduce most of these back to just
calling FieldValue::FromX methods.
2020-02-15 14:34:38 -08:00
Gil f321ad85a9 Decode documents in a background thread (#4857)
* Add Index-free benchmarks

* Add BackgroundQueue

Port of firebase/firebase-android-sdk#559

* Decode documents in background threads to improve performance

Run on (12 X 2900 MHz CPU s)
CPU Caches:
  L1 Data 32K (x6)
  L1 Instruction 32K (x6)
  L2 Unified 262K (x6)
  L3 Unified 12582K (x1)
Load Average: 2.24, 2.18, 1.96
----------------------------------------------------------------------------------
Benchmark                           Time             CPU   Iterations  Improvement
----------------------------------------------------------------------------------
BM_QueryMatching/0/1              132 us         12.7 us        53524        21.4%
BM_QueryMatching/1/1              150 us         18.1 us        37305        16.2%
BM_QueryMatching/1/10             371 us         27.6 us        10000        48.5%
BM_QueryMatching/10/10            429 us         64.3 us        11349        41.0%
BM_QueryMatching/1/100           2573 us         29.0 us         1000        51.7% (max)
BM_QueryMatching/100/100         3123 us          416 us         1000        45.5%
BM_QueryMatching/1/1000         26785 us         33.5 us          100        50.5%
BM_QueryMatching/10/1000        28389 us         74.5 us          100        48.0%
BM_QueryMatching/100/1000       27883 us          470 us          100        49.4%
BM_QueryMatching/1000/1000      32189 us         34.3 us          100        44.8%
BM_QueryAll/1                     151 us         15.2 us        45385       -49.5% (min)
BM_QueryAll/10                    444 us         60.7 us        12829        32.5%
BM_QueryAll/100                  3175 us          414 us         1000        45.1%
BM_QueryAll/1000                30222 us         18.8 us          100        49.7%
Mean                                                                         35.3%

* Insert into a vector to reduce contention

Run on (12 X 2900 MHz CPU s)
CPU Caches:
  L1 Data 32K (x6)
  L1 Instruction 32K (x6)
  L2 Unified 262K (x6)
  L3 Unified 12582K (x1)
Load Average: 1.43, 2.10, 1.98
----------------------------------------------------------------------------------
Benchmark                           Time             CPU   Iterations  Improvement
----------------------------------------------------------------------------------
BM_QueryMatching/0/1              127 us         12.2 us        56131         3.8%
BM_QueryMatching/1/1              142 us         16.6 us        39200         5.3%
BM_QueryMatching/1/10             369 us         26.6 us        10000         0.5%
BM_QueryMatching/10/10            425 us         59.2 us        12724         0.9%
BM_QueryMatching/1/100           2194 us         28.9 us        10000        14.7%
BM_QueryMatching/100/100         2765 us          405 us         1000        11.5%
BM_QueryMatching/1/1000         21095 us         32.8 us         1000        21.2%
BM_QueryMatching/10/1000        20797 us         66.4 us         1000        26.7% (max)
BM_QueryMatching/100/1000       21915 us          408 us         1000        21.4%
BM_QueryMatching/1000/1000      23700 us         30.9 us         1000        26.4%
BM_QueryAll/1                     136 us         14.3 us        44799         9.9%
BM_QueryAll/10                    443 us         57.3 us        10000         0.2% (min)
BM_QueryAll/100                  2607 us          389 us         1000        17.9%
BM_QueryAll/1000                23512 us         19.4 us         1000        22.2%
Mean                                                                         13.1%
2020-02-15 11:09:25 -08:00
Gil 9d202ae761 Drop support for building Firestore with Xcode 8 (#4842) 2020-02-06 17:37:16 -08:00
Gil 2c3585f1c8 Drop Firestore's Podfile dependency on Firebase (#4841)
This makes it possible to build and test on a release branch before the
Firebase pod is released.
2020-02-06 16:41:53 -08:00
Sebastian Schmidt d80ccb03e8 Update Spec test protos (#4814) 2020-02-04 13:38:13 -08:00
Gil bff0d44be9 Add Firestore pod-lib-lint (#4797) 2020-02-03 09:23:39 -08:00
Gil 968da25ca8 Fix the Linux build under GCC 7.4.0 (#4761)
* Don't bother searching for XCTest on non-Apple platforms

* Force working directory to be in the project root

This makes the spec tests work when cmake is invoked with a relative filename.

* Fix build error on Linux/GCC 7.4.0
2020-01-29 10:23:42 -08:00
Maksym MalyhinandPaul Beusterien a09718a7f2 Merge release 6.16.0 (#4763)
* Update versions for Release 6.16.0

* FirebasePerformance 3.1.10

* Firebase podspec - set ios.deployment_target = '9.0'  for InAppMessaging

* Delete duplicate function (#4744) (#4748)

Co-authored-by: Paul Beusterien <paulbeusterien@google.com>

* Firebase.podspec: Revert Performance version bump (#4758)

Co-authored-by: Paul Beusterien <paulbeusterien@google.com>
2020-01-28 18:04:14 -05:00
Gil bbc8de404c Add a benchmark for query execution (#4731)
* Use a valid project id

This avoids tripping up over newly added validation in the emulator

* Allow passing arguments to the emulator

e.g. to pass a non-default host

* Add a benchmark for reading documents

Baseline results:

Run on (12 X 2900 MHz CPU s)
CPU Caches:
  L1 Data 32K (x6)
  L1 Instruction 32K (x6)
  L2 Unified 262K (x6)
  L3 Unified 12582K (x1)
Load Average: 2.11, 1.96, 1.92
---------------------------------------------------------------------
Benchmark                           Time             CPU   Iterations
---------------------------------------------------------------------
BM_QueryMatching/0/1           168359 ns        13750 ns        53878
BM_QueryMatching/1/1           179118 ns        16706 ns        40193
BM_QueryMatching/1/10          721218 ns        35729 ns        10000
BM_QueryMatching/10/10         726533 ns        51864 ns        13587
BM_QueryMatching/1/100        5326040 ns        42820 ns         1000
BM_QueryMatching/100/100      5734101 ns       335289 ns         1000
BM_QueryMatching/1/1000      54110241 ns        41310 ns          100
BM_QueryMatching/10/1000     54584725 ns        95060 ns          100
BM_QueryMatching/100/1000    55083753 ns       527750 ns          100
BM_QueryMatching/1000/1000   58299823 ns        29130 ns          100
BM_QueryAll/1                  108892 ns        13355 ns        51197
BM_QueryAll/10                 657727 ns        43867 ns        10000
BM_QueryAll/100               5786091 ns       328244 ns         1000
BM_QueryAll/1000             60092767 ns        19380 ns          100

* Add a dedicated XCTest/Google Benchmark adapter

Remove FSTLevelDBBenchmarkTests. This benchmark appears to explore some
trade-off in document-size vs indexing but doesn't seem useful to
re-run.
2020-01-24 09:09:43 -08:00
Gil ecb26bf438 Migrate LevelDB data out of ~/Documents (#4609)
This moves iOS and macOS user data from `NSDocumentDirectory` to `NSApplicationSupportDirectory`, adding a pre-open migration step that:

  * Moves old data into the new location
  * Cleans up any resulting empty directories in the old Documents folder
  * Stops before removing the Documents folder itself (see tests)

Note that other platforms don't need a migration:

  * tvOS was already storing documents in `NSCachesDirectory`
  * Linux, Windows, and other UNIX haven't been publicly released or aren't changing.

See individual commits for details. Most notably this adds some additional filesystem manipulation routines to our (sadly) growing library.

Fixes #843.
2020-01-16 17:36:23 -08:00
Brian Chen 9313d61a21 Add verify support (#4658) 2020-01-16 09:57:38 -08:00
wu-hui 51e17ac8c5 Rename QueryData|Cache to TargetData|Cache (#4670)
* Rename done by CLion

QueryCache -> TargetCache
QueryData -> TargetData

* Fix cmake list indention

* Fixing ordering.
2020-01-15 13:47:26 -05:00
2328d1f365 Merging the 6.15.0 release into master (#4682)
* Update versions for Release 6.15.0

* Update podspec dependencies for M62 (#4640)

* Cherry-picks for M62. (#4650)

* Include GoogleUtilities in the release manifest.

* Crashlytics fix header import ordering issue (#4649)

Co-authored-by: Sam Edson <samedson@google.com>

Co-authored-by: Ryan Wilson <wilsonryan@google.com>
2020-01-14 16:21:34 -08:00
Gil ff21f1ecb3 Make a Filesystem class that can be subclassed for testing (#4647)
This change does not yet change all the classes that use the filesystem
to take Filesystem as a dependency. Those changes will come in follow-up
PRs.

This change is motivated by feedback in #4609. There I had created a
complicated interface to LevelDbOpener just so that different migration
scenarios could be tested, when in reality, I just needed to be able to
plumb through different values for the result of AppDataDir and related
functions.
2020-01-10 06:58:01 -08:00
Sebastian Schmidt 423a62f874 Enable Index-Free queries (#4618) 2020-01-09 16:40:47 -08:00
Gil 6cc1a449ea Clean up Objective-C interop and other dead code (#4601)
* Remove unused -[FIRFirestore isLoggingEnabled]

Use util::LogIsLoggable(util::kLogLevelDebug) directly.

* Remove objc_compatibility.h

* Remove/cleanup unused #if __OBJC__ blocks

* Remove dead code

* Remove MakeNSStringNoCopy

* Remove worthless include

* Tidy up FIRFirestore

* Remove dead code and use forward declarations in FSTHelpers

* Minor fixes for LevelDbTransaction:

  * Make methods const where possible
  * Make key() and value() return const std::string&, allowing callers
    to create their own const references

* Remove Comparator<NSString*>
2020-01-08 18:40:29 -08:00
wu-hui 14e28121e2 Release LimitToLast (#4616)
* [1/4] Split Query and Target, and make LocalStore/RemoteStore work with Target. (#4413)

* [2/4]LimitToLast: Make SyncEngine work with Target (#4491)

* [3/4] Implement LimitToLast (#4505)
2020-01-08 09:04:28 -08:00
Gil 9a4f4e73c7 Miscellaneous fixes (#4584)
* Fix use after move in LevelDbPersistenceForTesting

* Fix size-related errors when compiling in 32-bit mode

* Fix test-only memory leaks

* Fix test-only memory leaks

* Prevent spec tests from appearing to leak

The spec tests aren't actually leaking, but they accumulate enough
garbage without clearing the autorelease pool that Instruments
complains.

* Avoid temporaries in methods that @throw

ARC does not release temporaries when an Objective-C exception is
thrown. Use C++ equivalents directly to avoid leaks as detected by
Instruments.

* Fix memory leak in acknowledge mutation

Any previous value must be freed before the next one is assigned.

* Manually implement assignment operators in ByteString

The single by-value implementation wasn't preventing the compiler
generated standard versions from being generated which was causing
leaks.

* Prevent leaking GrpcCompletions that complete after Shutdown

After shutdown, the AsyncQueue silently drops Enqueue requests which
prevents these blocks from being deleted.

* Avoid racing with enqueued operations

It's possible for the Executor to immediately execute the given block,
at which point the block might observe the async queue in a
not-shutting-down state.
2020-01-08 07:11:05 -08:00
Sebastian Schmidt d261cdeb9b Finish Index-Free queries (still disabled) (#4484) 2020-01-07 13:29:20 -08:00
Gil df3989b8a2 Fix lint warnings spotted by google3's linter (#4538) 2019-12-31 11:35:51 -06:00
Paul Beusterien 76c766a866 Release 6.14.0 (#4513) 2019-12-17 11:36:36 -08:00
Gil 16447a5e2b Restrict use of Abseil types in public C++ accessible APIs (#4503)
* Change string parameters from absl::string_view to std::string

No abseil types can be used as parameters in this package.

This fixes an issue where Unity builds fail due to missing symbols.

The underlying issue is that Abseil in CocoaPods is an LTS release and
LTS releases contain an inline namespace, making types like
absl::string_view link as absl::lts_2019_08_08::string_view. When
compiling Firestore in google3 it uses the abseil in //third_party/absl,
which is not an LTS release and does not include the inline namespace.
This disagreement causes issues for the public C++ API which is compiled
in google3 and consumes code in the API package.

* Avoid absl::any in interfaces shared with public C++

* Make resource-related methods class members

This gives them access to private methods of friends of Serializer.

* Access control FieldPath/ResourcePath construction

* Access control Document construction

* Access control FieldValue construction

* Access control TransformOperation methods

* Remove DocumentSnapshot constructor that takes metadata pieces individually

* Restrict access to DocumentSnapshot from optional<Document>

* Add comments indicating how Abseil types are restricted

* Make DocumentKey::FromPathString take std::string
2019-12-17 11:32:48 -08:00
Sebastian Schmidt 656b9c0889 Remove unused test helpers (#4473) 2019-12-10 13:07:52 -08:00
Gil e7172d5f09 Post-migration cleanups (#4451)
* Fix naming violations.

* Remove xcgmock

* Remove FSTTypes.h

* Remove FSTClasses.h
2019-12-07 18:33:18 -08:00
Gil 85365cb089 Get the remainder of Objective-C tests running under CMake (#4430)
* Add Objective-C API tests to the CMake build

* Fix signedness warnings under the CMake build

* Get the remainder of Objective-C tests running under CMake

Also:
  * Add a Path::HasExtension check
  * Change spec tests to pick up json without NSBundle
2019-12-06 10:11:05 -08:00
Gil c2fa88e3b0 Rewrite local store tests in C++ (#4421)
* Move FSTLocalStoreTests to core/test

* Add testutils for constructing TransformMutations

* Rewrite local store tests in C++
2019-12-05 08:37:15 -08:00
Sebastian Schmidt bd9d9d8b3c Index-Free Queries (disabled, some code + tests missing) (#4418) 2019-12-03 12:24:41 -08:00
wu-hui 2a8d069b77 Run integration tests from CMake on mac (#4276) 2019-11-19 13:40:43 -05:00
Ryan Wilson 3261d120a9 Update versions for Release 6.13.0 (#4318) 2019-11-14 16:50:23 -08:00
Vinay Guthal be17ade458 add heartbeat feature for ios sdks (#4098)
* add heartbeat feature

* fix client logtype

* make class functions

* add subspec changes

* remove test file

* add heartbeat  tests

* use FileDataStorage instead of Storage

* add style

* use file data storage instead of storage

* fix test location

* add tests

* address comments

* update storage heartbeat with the new api

* update format

* update formatting and use correct enums

* rename

* update storage

* fix heartbeat

* update heartbeat tests

* fix styling

* Revert change

* heartbeat storage

* add comments

* update storage

* address comments

* add comments

* address comments

* fix style

* add style

* add style

* move heartbeat date storage

* move to environment

* update tests

* add tests

* small change

* add google utilities

* delete the header file

* update tests

* remove headers

* fix comment

* gul heartbeat date storage

* Update versions for heartbeat PR (#4311)

* update podfile

* Fix Firestore build under CMake

GoogleUtilities must now build SecureCoding
2019-11-14 15:22:20 -05:00
Gil f0157ae7c5 Port FSTLRUGarbageCollectorTests to C++ (#4283) 2019-11-12 14:21:39 -08:00
Brian Chen 0170957544 Allow querying for arrays in IN queries (#4285) 2019-11-12 13:07:58 -08:00
Gil d056539200 Migrate FSTLevelDBMigrationsTests to C++ (#4272) 2019-11-11 12:15:46 -08:00
Gil 69d39f660e Port FSTLevelDBTransactionTests to C++ (#4270)
* Move FSTLevelDBTransactionTests to C++

* Update CMake project

* Update Xcode project

* Rewrite FSTLevelDBTransactionTests in C++
2019-11-08 14:48:32 -08:00
Konstantin Varlamov db8cb2321a C++ migration: cleanup leftover bits of Objective-C (#4190)
* remove `OBJC_CLASS` (it's no longer used);
* make most sources `.cc` instead of `.mm` (all exceptions are Apple platform-specific code).
2019-11-07 21:43:53 -05:00
Konstantin Varlamov 62ef244cde C++ migration: remove the dependency on Objective-C Protobuf library (#4189) 2019-11-07 20:39:53 -05:00
Konstantin Varlamov d0686d2f63 C++ migration: implement pretty-printing of Nanopb-generated messages (#4184)
Note: the output is indistinguishable from what is generated by the Objective-C library, with two exceptions:
* class names are different (due to using prefixes, e.g., `GCFS`);
* order in which map fields are outputted can be different (which I presume is to be expected).

I verified this by running integration tests and using simple diffing code to compare outputs: in the destructor of each `Message<T>` (if it's not moved-from), serialize the `Message` to bytes, create an Objective-C proto out of those bytes, convert both protos to their string form, compare the strings (ignoring minor differences in indentation, etc.). Out of 5.7K comparisons, there were just two cases of differences:
* one was in `SerializerTest.FailOnInvalidInputBytes`; since the input in question is invalid, this doesn't seem to require much attention;
* another one was due to differing floating point precision: C++ produces `1.79769e+308` while Objective-C produces `1.79769313486232e+308`. Added a TODO.

Example output:
```
2019-10-28 21:44:56.941749-0400 Firestore_Example_iOS[78883:2094242] 6.11.0 - [Firebase/Firestore][I-FST000001] WriteStream (7f903ca53d50) write request: <WriteRequest 0x70000991ef08>: {
    writes {
      update {
        name: "projects/varconst-11-a51fd/databases/(default)/documents/test-collection/u6QizHeYjqt8Xe61kBoi"
        fields {
          key: "key"
          value {
            string_value: "value"
          }
        }
      }
    }
    stream_token: "\031\020hB\002\201\364\265\265"
}
```
2019-11-07 19:37:28 -05:00
Gil 12fb9f61d2 Upgrade gRPC to 1.24.3; upgrade protobuf to 3.9.2 (#4252)
* Upgrade gRPC to 1.24.3

* Upgrade C++ Protobuf to 3.9.2

Disable -Wunreachable-code in protobuf-generated code

* Regenerate C++ protos with protobuf 3.9.2

* Generate objc protos with Protobuf 3.9.2
2019-11-07 15:23:14 -08:00
Gil 6317c62710 Upgrade GoogleTest to 1.10.0 (#4250) 2019-11-07 12:01:15 -08:00
Gil 540f76d38e Port FSTMutationQueueTests to C++ (#4208) 2019-11-06 19:55:13 -08:00
Gil f89f624aed Require CMake 3.5.1; upgrade Google Benchmark to 1.5.0 (#4249)
* Require CMake 3.5.1

This is the version required by the latest versions of gRPC and Google
Benchmark. See discussion https://github.com/google/benchmark/pull/613.

Remove version check around find_package(XCTest)

Also, add a dependency on objc protos that was missing but somehow
implicitly being added before.

* Upgrade Google Benchmark to 1.5.0
2019-11-06 19:43:01 -08:00
Maksym Malyhin d18d515739 Merging the 6.12.0 release into master (#4223)
* Update versions for Release 6.12.0

* Update versions for Release 6.12.0 - Firestore

* Update versions for Release 6.12.0

* Update CHANGELOG for Firestore v1.7.0 (#4195)

* FirebaseCore version bump to 6.3.3 (#4213)

* Release manifst updated. (#4219)

* Revert "Add OIDC nonce support (#4183)" (#4220) (#4222)

This reverts commit ffe4a3a5b4.

* Revert Auth podspec version bump. (#4232)

* Release manifest - Auth removed (#4233)

* 6.12.0: Firebase.podspec FirebaseAuth pod version reverted, cherry-pick #4231 (#4234)

* Firebase.podspec - revert Auth

* check_whitespaces.sh: ignore notices. (#4231)

* Remove unnecessary NOTICES file (#4242)
2019-11-06 14:56:42 -05:00
Gil 1953cc59a4 Integrate with Abseil via CocoaPods (#4097)
Use Abseil 0.20190808
2019-11-05 09:12:34 -08:00
Gil 38d9643a00 Disallow empty document fields (#4229)
Port of firebase/firebase-android-sdk#643.

Also make error messages align with Android's.
2019-11-04 17:03:10 -06:00
wu-hui 2f53582530 Various minor fixes to make SDK compile on Linux (#4185) 2019-11-04 14:39:44 -05:00
Gil 3ea7197090 Port FSTQueryCacheTests to C++ (#4160)
* Move FSTQueryCacheTests files to Firestore/core

* Rewrite QueryCache tests in C++

* Fix test flake in local store tests

This doesn't affect production because we always set the additional
references pointer there, but it's technically optional and some tests
don't do this.
2019-11-02 09:09:25 -05:00
Konstantin Varlamov 04177614d0 Fix fuzz build (#4192) 2019-10-30 14:30:52 -04:00
Brian Chen 251a70f26a Make IN queries publicly available (#4174) 2019-10-29 14:13:02 -07:00
Brian Chen 441d7eda8b Change id to NSArray<id> for IN queries (#4182) 2019-10-28 22:22:37 -07:00