fix: surface CRT request-body publisher errors - #7307
Open
zoewangg wants to merge 1 commit into
Open
Conversation
The CRT request-body adapters let a publisher onError escape the native sendRequestBody upcall, so CRT printed it to stderr and failed the future with a generic wrapper. Catch it at the adapter and fail the operation with the original error, delivered to onError exactly once. Closes #6715
joviegas
reviewed
Aug 21, 2026
| @@ -0,0 +1,6 @@ | |||
| { | |||
| "type": "bugfix", | |||
| "category": "AWS CRT Async HTTP Client", | |||
Contributor
There was a problem hiding this comment.
question (non-blocking): The sync path CrtRequestInputStreamAdapter#sendRequestBody also throws a RuntimeException out of the native CRT upcall. Does it hit the same stderr-and-discard issue as #6715 ?
joviegas
approved these changes
Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
When uploading through a CRT-based client with
AsyncRequestBody.fromPublisher(publisher)and the publisher emitsonError, the exception was printed to the process stderr on a CRT event-loop thread, e.g.:The upload still failed, so the symptom was unexpected stderr noise plus a misleading exception on the returned future. The Netty client does not do this. Reported in #6715 (CRT-based S3 client via
S3AsyncClient.crtCreate()+S3TransferManager.upload()).Root cause.
ByteBufferStoringSubscriber.onErrorstores the error, andtransferTo(...)re-throws it as aRuntimeException. ThattransferTocall runs inside the CRT request-body adapter'ssendRequestBody, which is a native JNI upcall (the adapter implements CRT'sHttpRequestBodyStream) with no try/catch. When the upcall returns with a pending Java exception, CRT's native glue callsExceptionDescribe(which prints the exception + stack trace to the process's native stderr) and thenExceptionClear, and raisesAWS_ERROR_HTTP_CALLBACK_FAILURE. So the exception is printed and discarded natively, and the future completes with a genericSdkClientException("Failed to send the request: ...CALLBACK_FAILURE")whose cause is null — the original error survives only on stderr.This affects both CRT request-body adapters:
S3CrtRequestBodyStreamAdapter(S3 CRT client, the one #6715 reports) and the structurally identicalCrtRequestBodyAdapter(aws-crt-client).Modifications
Two coordinated parts, no public API changes (all
@SdkInternalApi):1. Contain the error at the JNI boundary (both adapters).
sendRequestBodynow wraps thetransferTo(...)call intry/catch (RuntimeException). On a publisher error it fails the operation with the original error instead of letting it escape the upcall, and returnsfalse(nevertrue— returning "body complete" would upload a zero-byte object and report success).ByteBufferStoringSubscriber.transferTo's throwing contract is unchanged; the fix is contained at the adapter.2. Deliver the original error to the caller, exactly once (per-client wiring). The caller-facing future is driven by
SdkAsyncHttpResponseHandler.onError, not by the HTTP client's execute future, so the two clients differ in how the error is surfaced:S3 CRT (
S3CrtRequestBodyStreamAdapter+S3CrtAsyncHttpClient+S3CrtResponseHandlerAdapter): the adapter completes the operation's execute future exceptionally with the original error.S3CrtAsyncHttpClientwires that future (the sameresultFuturethe response handler holds) into the adapter.S3CrtResponseHandlerAdapter'sresultFuture.whenCompletenow deliversresponseHandler.onError(originalError)first and then cancels + closes the meta request (reusing the existing cleanup wiring). A one-shotAtomicBooleanguard around the singleonErrornotification ensures the later, cancel-derivedAWS_ERROR_S3_CANCELEDonFinisheddoes not re-notify — first error (the original) wins.aws-crt-client (
CrtRequestBodyAdapter+CrtRequestAdapter+CrtAsyncRequestExecutor+CrtResponseAdapter): the adapter routes the body error throughCrtResponseAdapter.failRequest(...), which deliversonError(originalError)and completes the request future. Completing the future triggerscloseConnection(), which cancels the stream so CRT delivers a cancel-derivedonResponseComplete; a one-shot guard aroundCrtResponseAdapter'sonErrorchokepoint suppresses that second notification. (This also fixes a latent double-onErroron the existingonResponseBody-write-failure path.)Net: on both clients, a request-body publisher error now produces no native stderr line and completes the caller's future exceptionally carrying the original error, delivered to
onErrorexactly once — matching the Netty client.Testing
Adapter/handler-level unit tests plus two end-to-end WireMock functional tests that exercise the real native CRT path.
S3CrtRequestBodyStreamAdapterTest:sendRequestBodyno longer throws on a publisher error; the wired execute future completes exceptionally with the original error (RuntimeExceptionas-is;IOExceptionwrapped asUncheckedIOException); repeated calls stay quiet and preserve the original error. (Updated the two tests that previously assertedsendRequestBodythrows.)S3CrtResponseHandlerAdapterTest: completing the execute future with a body error deliversonErrorexactly once with the original error, cancels + closes the meta request, and a subsequentonFinished(S3_CANCELED)does not re-notify; same exactly-once behavior for a pipeline-forwarded timeout (ApiCallAttemptTimeoutException).CrtRequestBodyAdapterTest(new) andCrtResponseHandlerTest: the body-error sink is invoked once with the original error;failRequest(...)followed by a cancel-derivedonResponseCompletenotifiesonErrorexactly once with the original error.AwsCrtAsyncHttpClientWireMockTest(new e2e): a realAwsCrtAsyncHttpClientPUT to WireMock with an erroring request-body publisher completes the returned future exceptionally with the originalIllegalArgumentException.S3CrtClientWiremockTest(new e2e): a realS3AsyncClient.crtBuilder()(endpoint-overridden to WireMock)putObjectwith an erroring body completes the future exceptionally with the original error.Note: the stderr line is written by native
ExceptionDescribe, which bypasses bothSystem.setErr(...)andThread.setDefaultUncaughtExceptionHandler, so it is not assertable at the Java layer; the tests verify the future carries the original error, which is the meaningful, executed proof that the exception no longer escapes the upcall.Build scope:
services/s3andhttp-clients/aws-crt-clientbuilt withmvn install(module-level), all affected tests green. A full-repomvn installwas not run.Types of changes
Checklist
mvn installsucceedsscripts/new-changescript and following the instructions. Commit the new file created by the script in.changes/next-releasewith your changes.License
Closes #6715