gRPC cpp client with dialog flow gives back Reaching stream deadline, please restart your stream., ABORTED, code=10

GRPC core version: 1.48.0 Dialogflow version: v2 language: c++

I am using bidirectional streaming using the completion queue asynchronous method to get the audio sent using StreamingDetectIntentRequest message from the client end to Dialogflow agent, and based on the recommendations given in https://grpc.io/blog/deadlines/ I have my code set with C++ options as follows:

ClientContext context;
time_point line = std::chrono::system_clock::now() +
    std::chrono::seconds(5);
context.set_deadline(deadline);

If I set the deadline to a value that is less than 6 seconds, and I start sending the audio, I get back the audio write failures within a second with grpc::Status "Reaching stream deadline, please restart your stream., ABORTED, code=10"

But if I set the deadline to >= 6 seconds, I get the expected behavior which is to see the status "Deadline Exceeded, DEADLINE_EXCEEDED, code=4" after around the set deadline value of greater than 6 seconds.

Question: Why the stream is reaching the deadline when the context deadline is set to a lower value of 5 seconds? I have not set any deadline for the stream other than this. Note: I am aware that we can send up to 1 minute's worth of audio on a single RPC call using StreamingDetectIntentRequest