Fine grained partial failure in the BulkMutateJobService
July 22nd, 2011 | Published in Google Adwords API, Uncategorized
Since its introduction the BulkMutateJobService has provided support for partial failures, allowing some operations to succeed even though others have failed. Over the next few weeks we'll be rolling out improvements to this functionality, allowing for more granular failures when possible.
Bulk mutate jobs process operations in batches, and a failed operation will cause the entire batch to fail, but not the rest of the job. The results for a failed batch start with a FailureResult that holds all the errors generated by the operations in that batch. It is followed by a series of BatchFailureResults which act as placeholders for the rest of the operations in the batch. More information about how these failures are reported can be found in a previous blog post we did on the topic.
In v201008 we enabled partial failure support for the synchronous AdGroupCriterionService. Unlike the behavior in the BulkMutateJobService, this version of partial failure allows individual operations to fail without affecting successful ones. This method of partial failure is preferred as it cuts down on the number of retries needed.
While we can't update the BulkMutateJobService to process all operations this way, we are able to enable it for operations that use an underlying service that does support granular partial failures. Starting August 1st, operations of type AdGroupCriterionOperation and AdGroupAdOperation processed by the BulkMutateJobService may start failing granularly instead of in batches.
Let's see how this will change the results returned.
Before the change the invalid operations will cause other valid operations to fail. The FailureResult is the first result in the batch, even if the operation at the position was valid. The field path of the errors indicates the index of the invalid operations relative to the start of the batch.
After the change invalid operations will not affect other valid operations in the same batch. Each invalid operation will have a corresponding FailureResult. There may still be cases where BatchFailureResults are returned for these operations, such as if there is a general error with the account that cannot be attributed to any specific operation.
So how does this affect code you've already written to process BulkMutateJob results? The good news is that if you've written your code to allow for a variable batch size (no hard coded batch size) then no changes are needed! Invalid operations are essentially processed as batches of one, and the same logic that you use to process the results today should work perfectly with these new granular failures. If your application is currently expecting a batch of a specific size then you should update it to detect the start and end of batches dynamically.
If you have any questions about how these new granular failures work, or feedback on how we can further improve the error handling in the API, add a post to the AdWords API forum.
Update 2011-08-25
This change has been fully deployed and all BulkMutateJobs will be processed with granular partial failures enabled.
-
Eric Koleda, AdWords API Team
Bulk mutate jobs process operations in batches, and a failed operation will cause the entire batch to fail, but not the rest of the job. The results for a failed batch start with a FailureResult that holds all the errors generated by the operations in that batch. It is followed by a series of BatchFailureResults which act as placeholders for the rest of the operations in the batch. More information about how these failures are reported can be found in a previous blog post we did on the topic.
In v201008 we enabled partial failure support for the synchronous AdGroupCriterionService. Unlike the behavior in the BulkMutateJobService, this version of partial failure allows individual operations to fail without affecting successful ones. This method of partial failure is preferred as it cuts down on the number of retries needed.
While we can't update the BulkMutateJobService to process all operations this way, we are able to enable it for operations that use an underlying service that does support granular partial failures. Starting August 1st, operations of type AdGroupCriterionOperation and AdGroupAdOperation processed by the BulkMutateJobService may start failing granularly instead of in batches.
Let's see how this will change the results returned.
Existing partial failure behavior
| Operation | Result |
|---|---|
| AdGroupCriterionOperation |
FailureResult
|
| AdGroupCriterionOperation (Invalid) |
BatchFailureResult
|
| AdGroupCriterionOperation (Invalid) |
BatchFailureResult
|
| AdGroupCriterionOperation |
BatchFailureResult
|
Before the change the invalid operations will cause other valid operations to fail. The FailureResult is the first result in the batch, even if the operation at the position was valid. The field path of the errors indicates the index of the invalid operations relative to the start of the batch.
New granular partial failure behavior
| Operation | Result |
|---|---|
| AdGroupCriterionOperation | ReturnValueResult |
| AdGroupCriterionOperation (Invalid) |
FailureResult
|
| AdGroupCriterionOperation (Invalid) |
FailureResult
|
| AdGroupCriterionOperation | ReturnValueResult |
After the change invalid operations will not affect other valid operations in the same batch. Each invalid operation will have a corresponding FailureResult. There may still be cases where BatchFailureResults are returned for these operations, such as if there is a general error with the account that cannot be attributed to any specific operation.
So how does this affect code you've already written to process BulkMutateJob results? The good news is that if you've written your code to allow for a variable batch size (no hard coded batch size) then no changes are needed! Invalid operations are essentially processed as batches of one, and the same logic that you use to process the results today should work perfectly with these new granular failures. If your application is currently expecting a batch of a specific size then you should update it to detect the start and end of batches dynamically.
If you have any questions about how these new granular failures work, or feedback on how we can further improve the error handling in the API, add a post to the AdWords API forum.
Update 2011-08-25
This change has been fully deployed and all BulkMutateJobs will be processed with granular partial failures enabled.
-
Eric Koleda, AdWords API Team