If an upload coordinator gets created during `+load` calls, it can end up running its immediately, while +load calls are still running. It's generally a bad idea to be splitting off threads while +load calls are still running.
The upload coordinator checks every 30 seconds, and doesn't really need to run right away. Since it's easy to transitively create an upload coordinator from other calls that happen during +load, we'll just push off the initial run of the timer slightly. That makes it less likely to interfere (or deadlock) with other load calls.
See #7171 for an example of a problem solved by this change.
* Move extra public headers to Internal and fix imports
* Cleanup
* Use relative paths in internal headers
* Fix imports in internal headers
* ./scripts/style.sh
* Fix GDTTestApp
* Fix test app
* Use real uploader for watchOS test app
* comma
* Revert "Revert GDTCORStorage changes for now (#5778)"
This reverts commit 93eb6d8c48.
* Store events and implement unit tests (#5790)
Resolves many TODOs in the flat file storage system
Bug fixes found by unit tests are also included.
* Get rid of event iterators and add separate folders for batching (#5829)
* Store events and implement unit tests (#5790)
Resolves many TODOs in the flat file storage system
Bug fixes found by unit tests are also included.
* Change to explicit nil comparison
* Address test comments
- Makes all APIs async
- Add thread-safety test
- Comments on where files are stored
- Broke up the library data get/set API into two blocks
* Implement events and batch expirations (#5842)
The upload coordinator will signal an expiration check whenever the app foregrounds.
* Remove prioritizers, upload packages, data futures, and backwards compatibility (#5874)
* Remove prioritizers, upload packages, data futures, and backwards compatibility
* Update a comment and console logging
* Reset the storage for every unit test, move event generation out of core init
* Return a nil request if data is nil.
* Fix prioritizer reference in watchOS test app
* Remove unnecessary check for termination
* Give more leeway for eventID generation
Change the transport API to a GDTCORTarget
Don't continue a long test after a failure
* Return nil if an eventID couldn't be created
Fix some swift in the watchOS test app
* Fix all rawValue GDTCORTarget usage
* Update CHANGELOGs
* Remove initializer annotations (#5878)
* Remove initializer designation from GDTCOREvent
Change the target property to the target type
* Add a couple more types
* Make implicit retain explicit
Nothing I do seems to get this warning to trigger locally...
* Address loss of precision issue
* Re-enable a disabled test
Don't delete while enumerating
* Additional test tweaks and hasEvents is now on the queue and async
* Adjust GDTCCT to the new API
* Always attempt to create a path, get rid of implicit self refs
* Change all waiting for be DISPATCH_TIME_FOREVER for debugging purposes
I suspect the travis machines are just overwhelmed, since the test errors don't make any sense.
* Try using a different separator
* Enable debug logging
* Enable verbose pod lib lint output
* Handle cases where the mappingID contains path components
* Revert "Enable debug logging"
This reverts commit b3e4a6ecc5.
* Revert "Enable verbose pod lib lint output"
This reverts commit a5b54460f9.
* Revert "Change all waiting for be DISPATCH_TIME_FOREVER for debugging purposes"
This reverts commit 55e4fff5dc.
* Attempt to create a folder before writing
* Temporarily enable verbose pod lib lint, again
* Temporarily add debug logging back in
* Don't double store events
Also print out errors if there are any
* Remove debug logging
* Put debug logs back in for Crashlytics test
* Update mock APIs to match and use a real target
* remove debug logging
* Apply nullability specifiers to Crashlytics mock class
Update the device model identified by GDTCCT library to be more detailed and in parity with Clearcut.
Current GDT implementation returns "iPhone". With this change, the device model identified would be iPhoneA,B (Eg. iPhone 8,1).
Co-authored-by: Paul Beusterien <paulbeusterien@google.com>
* Revert "Refactor event saving and searching (#5773)"
This reverts commit a61786cbf1.
* Revert "Expand the storage protocol and provide an event iterator (#5742)"
This reverts commit 641d8531a3.
* Revert "Implement helpers for storing and finding events (#5738)"
This reverts commit 9a0aed81b0.
* Refactor event saving and searching
No more symlinks to try and speed searching. Just get a directory's contents and check the filename, which will contain metadata about the event.
* Change one string formatter
* Fix the type of some params, remove unused code
* Minor update
* Continue running after failing to process a file
* Expand the storage protocol and provide an event iterator
* Remove unused variable
* Remove unused variable
* Make implicit self retain explicit
* Just check for NSFileTypeRegular instead
And fix a comment
* Missed a damned implicit self retain
* Implement some tests for the new API
* Fix yet another implicit self retain
* Implement helpers for storing and finding events
* Fix missing imports
* Add clarifying comments and remove unneeded nullability
Also remove a test and fix the behavior of searchPathsWithEventSelector
* Added missing docs and properly implemented +reset
* Provide a library data mechanism on storage instances
This could allow most singletons to no longer need NSCoding, they just store the individual fields they need with an appropriately unique key
* Additional tests
And no more implicit removal
* Implement read error capture
* Make test file data keys more unique
Also, there _should_ be an error reading from a non-existent file.
* Fixes the count file not being found
Calling the new GDTCORRootDirectory() function ensures that the directory needed is created before an attempt to write is made.
Fixes#5549
* Update CHANGELOG and versions
* Add watchOS support for GoogleDataTransport with specifying notification center and WKExtensionDelegate for watchOS application, handling background tasks and network reachability flags on watchOS.
* Add independent watchOS test app for GoogleDataTransport and GoogleDataTransportCCTSupport.
* Add companion watchOS test app and notification service extension for GoogleDataTransportCCTSupport.
* Don't leak upload packages or deadlock on lifecycle events
GDTCORUploadPackages would call targetToStorage in -initWithTarget. When this method was running during app terminate, this would cause a deadlock because calling targetToStorage is re-entrant while GDTCORRegistrar is sending lifecycle events to all the instances it's tracking.
NSTimer's invalidate method is also called whenever a package is completed, retried, or expired, without regard to the presence of a package handler.
A unit test is added to ensure that GDTCORUploadPackage's are not leaked by a retain
TSAN caught no issues
* Don't mark GDTCORUploadPackage -init as unavailable
* Change removeItemAtURL to removeItemAtPath
It seems to work better, for reasons that aren't clear. It may have something to do with forming URLs being more strict.
* Add logging levels, fix failed write events
* Update comment
* Make mph-gdtsqlite run travis (#3861)
* Implement a SQLite helper (#4075)
* Add support for in-memory and other types of stores (#4158)
* Add support for in-memory and other types of stores
* Temporarily add -x to know what's causing check.sh to fail
* Remove -x
* DO NOT MERGE TO MASTER: Remove commit math from check.sh temporarily
* Initial implementation of GDTCORDatabase (#4180)
* Initial implementation of GDTCORDatabase
* Style
* Check result of running a statement
* Simplify init
Also support >1 :memory: db
* Write test docs and fix one tsan issue (#4211)
* wip moving storage to sqlite
* Revert "wip moving storage to sqlite"
This reverts commit ee1895f1bd.
* Address tsan issue and document tests
* Address implicit self retains
Don't know why these aren't surfaced locally, I have the warning on.
* Implement bulk sql execution (#4528)
* Change custom prioritization dictionary to bytes (#4534)
* Change custom prioritization dictionary to bytes
* Fix a missed change for customBytes
* Refactor GDT's storage to be an interface (#5022)
* Refactor GDT's storage to be an interface
- Events are now removed by eventID
- Events now generate an ID when they are alloc/inited
- Copying events also copies their eventID
- Event hashing takes into account the eventID
- A static method in GDTCOREvent creates event IDs as the concat of <session counter><event counter during session>
- Session counters are incremented and stored in user defaults at app start
- Storage instances register themselves with GDTCORRegistrar like prioritizers and uploaders
- Storage fakes are refactored to be registered with GDTCORRegistrar
- Added some missing docs and fixed some typos
* Remove an unintended change from assert macro
* Use a variable
* Additional docs and test fixes
* Move saving the counter value to a file
* Update docs
* Rename GDTCORStorage to GDTCORFlatFileStorage (#5079)
* Rename GDTCORStorage to GDTCORFlatFileStorage
* Fix the NSCoding keys
* Fixes to get building and unit tests passing
* Address lack of thread safety affecting lifecycle tests
* Style
* Transition customPrioritizationDict to customBytes
Adds a category on GDTCOREvent to make checking network connection info easier.
* Remove all sqlite and database stufff or the purposes of merging back to maste (#5330)
* Update .travis.yml
* Fix a couple of race conditions and module import of a private header
The header and API in question should be considered hidden. An alternative would be expose the API in the private header .
A race condition existed in the upload package itself--it should hand copies of itself and not self to prevent asynchronous mutation whilst a different queue is also executing an asynchronous mutation. One alternative would be to create mutable an immutable variants of the package. This could manifest as an EXC_BAD_ACCESS in GDTCCTPrioritizer.m
* Assert that the IMP is present, because there shouldn't be a silent failure
* Address GDT test issues
1. Fixes the event generator such that derived event fileURL will now be non-nil
2. Removes reachability check for integration tests since being locally connected via LAN but not connected to the internet would cause the test disabling check to pass. Instead, we attempt an actual NSURLRequest to a WAN target and set the value based on the success of that request.
* Create GDT's directory on first usage
* Libraries should save state if a completion block is requested
Currently, the onComplete block of sendXEvent:onComplete: is invoked when the event bytes are written to disk. However, if the app crashes before a lifecycle event occurs (trigger GDTCORStorage to serialize to disk), that event if effectively lost because the metadata has all been lost. This change triggers the saving of state of GDTCORStorage and prioritizers (if they implement it) if an onComplete block has been given.
* Remove flaky check
* Update GDTCORStorage.m
* Refactor event fileURLs to be derived
Apparently, since iOS 8, it's possible for the app's bundle to change directories as often as every launch. This is a mechanism that's hopefully slightly more robust against that, as data is supposed to carry over, the UUID component of the app's absolute directory might just change.
* Convert a couple of tests, actually use a non-absolute URL in GDTCOREvent
* Make sure to use the event fileURL property as the source of truth.
Add some additional logs and improve the integration test
* Fix KVC not working for the removed property
Paul Beusterien