Intro
When I wrote my last blog post about HttpClient (You're (probably still) using HttpClient wrong and it is destabilizing your software) I promised to do a post about error handling since I intentionally left that out.
This post will be somewhat of a journey, we will start with some happy path code where error handling is non existent and (hopefully) end up in a much better place.
One note before we start, the code in this post will use a class called Result. Think about it as something like a monad, Vladimir Khorikov has a great post on this topic.
I've used a modified version of this Result class for years with great success in a lot of different projects. It has a few benefits in my opinion:
- You don't need to rely on exceptions.
- You are forced to think about (and handle) failures.
- It makes the code a lot easier to read.
Simple example:
var result = await _something.GetData();
if (result.Success)
{
return result.Data;
}
else if(result.Retryable)
{
// Handle retry
}
else
{
// Handle fail
_logger.LogError(result.Message, result.Errors);
}
A note about Polly
Polly is a great tool that will help you dealing with timeouts, exceptions, retries and so on when using HttpClient. I've choosen NOT to use Polly in this post, simply because I believe that it's important to understand what happens behind the scenes of such a library before using it. I will do a follow up on this post where I use Polly, don't worry. :)
The problem
We want to fetch some information about a GitHub repository from some (imaginary) API. The API is really unstable and we need to handle all of the different errors.
This is the code we have to work with, let's call it...
...The Happy Path.
public class GithubHttpClient
{
private readonly HttpClient _httpClient;
public GithubHttpClient(HttpClient httpClient)
{
_httpClient = httpClient ?? throw new ArgumentNullException(nameof(httpClient));
}
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
using (var response = await _httpClient.SendAsync(request))
{
var responseJson = await response.Content.ReadAsStringAsync();
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
return Result<GitHubRepositoryDto>.Ok(repository);
}
}
}
Nice, we've some code. It will send a GET request to the API, read the JSON from the response, deserialize it and return the result, GREAT.
Let's write a test and verify that everything works as it should.
Test
[Fact]
public async Task GivenSuccessfulResponseFromGitHub_WhenGetRepository_ThenReturnsOkResult()
{
var responseBody = new GitHubRepositoryDto
{
Id = 1,
Name = "jos.httpclient",
Stars = 999999,
Url = "https://github.com/joseftw/jos.httpclient"
};
var apiResponse = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(JsonConvert.SerializeObject(responseBody))
};
var httpClient = new HttpClient(new MockedHttpMessageHandler(apiResponse))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient);
var result = await sut.GetRepository(1);
result.Success.ShouldBeTrue();
}
Outcome
Test SUCCEEDED.
So, what are we doing here?
We are creating a HttpResponseMessage with status code 200 OK and a body that contains a successful response from the API. We then create a new HttpClient with the overload that takes a HttpMessageHandler. The MockedHttpMessageHandler looks like this:
MockedHttpMessageHandler
public class MockedHttpMessageHandler : HttpMessageHandler
{
private readonly Func<Task<HttpResponseMessage>> _responseFactory;
private readonly HttpResponseMessage _httpResponseMessage;
public MockedHttpMessageHandler(HttpResponseMessage httpResponseMessage)
{
_httpResponseMessage = httpResponseMessage;
}
public MockedHttpMessageHandler(Func<Task<HttpResponseMessage>> responseFactory)
{
_responseFactory = responseFactory ?? throw new ArgumentNullException(nameof(responseFactory));
}
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
cancellationToken.ThrowIfCancellationRequested();
if (_responseFactory != null)
{
return await _responseFactory.Invoke();
}
return _httpResponseMessage;
}
}
This allows us to control what the HttpClient methods returns.
Let's add another test that verifies that the the deserialization of the data works as well.
[Fact]
public async Task GivenSuccessfulResponseFromGitHub_WhenGetRepository_ThenReturnsSaidRepository()
{
var responseBody = new GitHubRepositoryDto
{
Id = 1,
Name = "jos.httpclient",
Stars = 999999,
Url = "https://github.com/joseftw/jos.httpclient"
};
var apiResponse = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(JsonConvert.SerializeObject(responseBody))
};
var httpClient = new HttpClient(new MockedHttpMessageHandler(apiResponse))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient);
var result = await sut.GetRepository(1);
result.Data.Id.ShouldBe(1);
result.Data.Name.ShouldBe("jos.httpclient");
result.Data.Url.ShouldBe("https://github.com/joseftw/jos.httpclient");
result.Data.Stars.ShouldBe(999999);
}
Let's focus on errors.
Null response content
What happens if the response content is null?
Test
[Fact]
public async Task GivenNullResponseContent_WhenGetRepository_ThenReturnsFailResult()
{
var response = new HttpResponseMessage(HttpStatusCode.OK);
var httpClient = new HttpClient(new MockedHttpMessageHandler(response))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient);
var result = await sut.GetRepository(1);
result.Success.ShouldBeFalse();
result.Message.ShouldBe("Response from SendAsync was null");
}
Outcome
Test FAILED.
The test failed because of a NullReferenceException.
We are calling ReadAsStringAsync on the response content which is null, not good.
Fix
GitHubClient
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
using (var response = await _httpClient.SendAsync(request))
{
if (response.Content == null)
{
return Result<GitHubRepositoryDto>.Fail("Response content was null");
}
var responseJson = await response.Content.ReadAsStringAsync();
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
return Result<GitHubRepositoryDto>.Ok(repository);
}
}
We've added a check, if response.Content is null, we return a Fail result.
Empty response body
What happens if the response content is empty?
Test
[Fact]
public async Task GivenEmptyResponseContent_WhenGetRepository_ThenReturnsFailResult()
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(string.Empty)
};
var httpClient = new HttpClient(new MockedHttpMessageHandler(response))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient);
var result = await sut.GetRepository(1);
result.Success.ShouldBeFalse();
result.Message.ShouldBe("Failed to deserialize response");
}
Outcome
Test FAILED.
The test fails because...success is TRUE. (??)
Here's why:
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
We are passing an empty string to the DeserializeObject method, it will return null. We will come back to how to handle deserialization errors later, for now, let's just fix our code.
Fix
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
using (var response = await _httpClient.SendAsync(request))
{
if (response.Content == null)
{
return Result<GitHubRepositoryDto>.Fail("Response content was null");
}
var responseJson = await response.Content.ReadAsStringAsync();
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
if (repository == null)
{
return Result<GitHubRepositoryDto>.Fail("Failed to deserialize response");
}
return Result<GitHubRepositoryDto>.Ok(repository);
}
}
We've now added a null check, if repository is null, we return a Fail result.
Exceptions
What happens if we encounter some exceptions when calling the API?
Test
[Fact]
public async Task GivenExceptionWhenCallingTheApi_WhenGetRepository_ThenReturnsRetryResult()
{
var httpClient = new HttpClient(new MockedHttpMessageHandler(() => throw new HttpRequestException("Boom from test")))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient);
var result = await sut.GetRepository(1);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeTrue();
result.Message.ShouldBe("Exception when calling the API");
}
Outcome
Test FAILED.
Turns out we are not handling exceptions at all, what a shocker.
Fix
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
try
{
using (var response = await _httpClient.SendAsync(request))
{
if (response.Content == null)
{
return Result<GitHubRepositoryDto>.Fail("Response content was null");
}
var responseJson = await response.Content.ReadAsStringAsync();
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
if (repository == null)
{
return Result<GitHubRepositoryDto>.Fail("Failed to deserialize response");
}
return Result<GitHubRepositoryDto>.Ok(repository);
}
}
catch (HttpRequestException exception)
{
_logger.LogError(exception, "HttpRequestException when calling the API");
return Result<GitHubRepositoryDto>.Retry("HttpRequestException when calling the API");
}
catch (Exception exception)
{
_logger.LogError(exception, "Unhandled exception when calling the API");
// Here it's up to you if you want to throw or return Retry/Fail, im choosing to FAIL.
return Result<GitHubRepositoryDto>.Fail("Unhandled exception when calling the API");
}
}
Here I've choosen to wrap everything in a try/catch. If an HttpRequestException happens, we choose to return a Retry Result.
If it's not an HttpRequestException, we choose to return a Fail Result. Here it's really up to you as a developer to make a choice, either you choose to return some kind of result (fail/retry) or you just log and throw it, it's up to you.
I've also added a test for exceptions != HttpRequestException:
[Fact]
public async Task GivenExceptionWhenCallingTheApi_WhenGetRepository_ThenReturnsRetryResult()
{
var httpClient = new HttpClient(new MockedHttpMessageHandler(() => throw new Exception("Boom from test")))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient, _fakeLogger);
var result = await sut.GetRepository(1);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeFalse();
result.Message.ShouldBe("Unhandled exception when calling the API");
}
Great, we are now handling all kinds of exceptions, surely we must be done now?.
Nope, let's move on to...
...Status codes
If a project doesn't exists, the API will return a 404 with a different JSON body.
The body will look like this:
{
"error": "The repository was not found"
}
Do we handle that already?
Test
[Fact]
public async Task GivenNotFoundResponseFromTheApi_WhenGetRepository_ThenReturnsFailResult()
{
var response = new HttpResponseMessage(HttpStatusCode.NotFound)
{
Content = new StringContent("{\"error\": \"The repository was not found\"}")
};
var httpClient = new HttpClient(new MockedHttpMessageHandler(response))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient, _fakeLogger);
var result = await sut.GetRepository(1);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeFalse();
result.Message.ShouldBe("The repository was not found");
}
Outcome
Test FAILED because Success is...true?!
Turns out, our extremely naive deserialization strikes again. The deserialized repository is not null now, it's infact an instance of GitHubRepositoryDto where all properties have their default value.
That's bad.
We are still going to cover deserialization later, so let's just fix the code for now.
Fix
We only want to deserialize to a GitHubRepositoryDto if the response status code is 200 OK. If not, we will return a fail result.
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
try
{
using (var response = await _httpClient.SendAsync(request))
{
if (response.StatusCode == HttpStatusCode.OK)
{
if (response.Content == null)
{
return Result<GitHubRepositoryDto>.Fail("Response content was null");
}
var responseJson = await response.Content.ReadAsStringAsync();
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
if (repository == null)
{
return Result<GitHubRepositoryDto>.Fail("Failed to deserialize response");
}
return Result<GitHubRepositoryDto>.Ok(repository);
}
return Result<GitHubRepositoryDto>.Fail(response.StatusCode == HttpStatusCode.NotFound
? "The repository was not found"
: "Did not receive 200 OK status code");
}
}
catch (HttpRequestException exception)
{
_logger.LogError(exception, "HttpRequestException when calling the API");
return Result<GitHubRepositoryDto>.Retry("HttpRequestException when calling the API");
}
catch (Exception exception)
{
_logger.LogError(exception, "Unhandled exception when calling the API");
// Here it's up to you if you want to throw or return Retry/Fail, im choosing to FAIL.
return Result<GitHubRepositoryDto>.Fail("Unhandled exception when calling the API");
}
}
I've added a check, if status code == 200 OK, then we deserialize. If not, we return a fail result. This is really naive and will be improved later in this post.
Retryable status codes
Right now, if we receive a status code that is not 200 OK, we return a Result.Fail result. But what if we receive a 500? 503? 408 and so on? If we recieve a status code that is worth retrying later on, I want to handle that by returning a Result.Retry.
Test
[Theory]
[InlineData(HttpStatusCode.InternalServerError)]
[InlineData(HttpStatusCode.GatewayTimeout)]
[InlineData(HttpStatusCode.RequestTimeout)]
[InlineData(HttpStatusCode.BadGateway)]
[InlineData(HttpStatusCode.TooManyRequests)]
public async Task GivenRetryableStatusCode_WhenGetRepository_ThenReturnsRetryResult(HttpStatusCode statusCode)
{
var response = new HttpResponseMessage(statusCode);
var httpClient = new HttpClient(new MockedHttpMessageHandler(response))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient, _fakeLogger);
var result = await sut.GetRepository(1);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeTrue();
result.Message.ShouldBe($"Received retryable status code '{statusCode}'");
}
Outcome
Test FAILED because Retryable == false.
Fix
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
try
{
using (var response = await _httpClient.SendAsync(request))
{
if (response.StatusCode == HttpStatusCode.OK)
{
if (response.Content == null)
{
return Result<GitHubRepositoryDto>.Fail("Response content was null");
}
var responseJson = await response.Content.ReadAsStringAsync();
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
if (repository == null)
{
return Result<GitHubRepositoryDto>.Fail("Failed to deserialize response");
}
return Result<GitHubRepositoryDto>.Ok(repository);
}
if (RetryableStatusCodes.Contains(response.StatusCode))
{
return Result<GitHubRepositoryDto>.Retry($"Received retryable status code '{response.StatusCode}'");
}
return Result<GitHubRepositoryDto>.Fail(response.StatusCode == HttpStatusCode.NotFound
? "The repository was not found"
: "Did not receive 200 OK status code");
}
}
catch (HttpRequestException exception)
{
_logger.LogError(exception, "HttpRequestException when calling the API");
return Result<GitHubRepositoryDto>.Retry("HttpRequestException when calling the API");
}
catch (Exception exception)
{
_logger.LogError(exception, "Unhandled exception when calling the API");
// Here it's up to you if you want to throw or return Retry/Fail, im choosing to FAIL.
return Result<GitHubRepositoryDto>.Fail("Unhandled exception when calling the API");
}
}
Here I've introduced a HashSet<HttpStatusCode> and added a few status codes. If the set contains the response status code, I return Result.Retry.
Timeouts
What about timeouts? What happens if we can't get a response in reasonable time?
The default timeout for HttpClient is 100 seconds. In our case, that is NOT reasonable (quite frankly, I don't think it's ever reasonable...).
A more reasonable timeout would be something like 2-3 seconds in our case.
When a timeout occours, HttpClient throws a TimeoutException, so we will mimic the behaviour by throwing the exception from our MockedHttpMessageHandler.
Test
[Fact]
public async Task GivenTimeoutFromHttpClient_WhenGetRepository_ThenReturnsRetryResult()
{
var httpClient = new HttpClient(new MockedHttpMessageHandler(() => throw new TimeoutException("Timeout from test")))
{
BaseAddress = new Uri("http://localhost"),
Timeout = TimeSpan.FromMilliseconds(100) // This doesn't do anything in the test, just to show how to set the timeout.
};
var sut = new GithubHttpClient(httpClient, _fakeLogger);
var result = await sut.GetRepository(1);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeTrue();
result.Message.ShouldBe("TimeoutException during call to API");
}
Outcome
Test FAILED because Retryable == false.
Fix
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
try
{
using (var response = await _httpClient.SendAsync(request))
{
if (response.StatusCode == HttpStatusCode.OK)
{
if (response.Content == null)
{
return Result<GitHubRepositoryDto>.Fail("Response content was null");
}
var responseJson = await response.Content.ReadAsStringAsync();
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
if (repository == null)
{
return Result<GitHubRepositoryDto>.Fail("Failed to deserialize response");
}
return Result<GitHubRepositoryDto>.Ok(repository);
}
if (RetryableStatusCodes.Contains(response.StatusCode))
{
return Result<GitHubRepositoryDto>.Retry($"Received retryable status code '{response.StatusCode}'");
}
return Result<GitHubRepositoryDto>.Fail(response.StatusCode == HttpStatusCode.NotFound
? "The repository was not found"
: "Did not receive 200 OK status code");
}
}
catch (HttpRequestException exception)
{
_logger.LogError(exception, "HttpRequestException when calling the API");
return Result<GitHubRepositoryDto>.Retry("HttpRequestException when calling the API");
}
catch (TimeoutException exception)
{
_logger.LogError(exception, "TimeoutException during call to API");
return Result<GitHubRepositoryDto>.Retry("TimeoutException during call to API");
}
catch (Exception exception)
{
_logger.LogError(exception, "Unhandled exception when calling the API");
// Here it's up to you if you want to throw or return Retry/Fail, im choosing to FAIL.
return Result<GitHubRepositoryDto>.Fail("Unhandled exception when calling the API");
}
}
Now we are catching TimeoutException and return a Retry result.
CancellationToken "timeouts"
Since we are working with async code we SHOULD be using CancellationTokens so let's add that.
Our GetRepositorymethod will now look like this:
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id, CancellationToken cancellationToken)
We are also passing the CancellationToken to the HttpClient SendAsync method.
Test
[Fact]
public async Task GivenCanceledCancellationToken_WhenGetRepository_ThenReturnsRetryResult()
{
var response = new HttpResponseMessage(HttpStatusCode.OK);
var httpClient = new HttpClient(new MockedHttpMessageHandler(response))
{
BaseAddress = new Uri("http://localhost")
};
var cancelledCancellationToken = new CancellationToken(true);
var sut = new GithubHttpClient(httpClient, _fakeLogger);
var result = await sut.GetRepository(1, cancelledCancellationToken);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeTrue();
result.Message.ShouldBe("Task was canceled during call to API");
}
[Fact]
public async Task GivenTaskCanceledException_WhenGetRepository_ThenReturnsRetryResult()
{
var httpClient = new HttpClient(new MockedHttpMessageHandler(() => throw new TaskCanceledException("canceled from test")))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GithubHttpClient(httpClient, _fakeLogger);
var result = await sut.GetRepository(1, CancellationToken.None);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeTrue();
result.Message.ShouldBe("Task was canceled during call to API");
}
Outcome
Tests FAILED.
The tests failed because Retryable == false.
Note that we added 2 tests here, one that passes in a canceled CancellationToken and one that throws a TaskCanceledException. The reason for doing this is that our MockedHttpMessageHandler (and the standard one used by HttpClient afaik) uses this code:
cancellationToken.ThrowIfCancellationRequested()
That code throws a OperationCanceledException. But there is also a posibility that some code throws a TaskCanceledException instead, so we want to handle both cases.
Fix
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id, CancellationToken cancellationToken)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
try
{
using (var response = await _httpClient.SendAsync(request, cancellationToken))
{
if (response.StatusCode == HttpStatusCode.OK)
{
if (response.Content == null)
{
return Result<GitHubRepositoryDto>.Fail("Response content was null");
}
var responseJson = await response.Content.ReadAsStringAsync();
var repository = JsonConvert.DeserializeObject<GitHubRepositoryDto>(responseJson);
if (repository == null)
{
return Result<GitHubRepositoryDto>.Fail("Failed to deserialize response");
}
return Result<GitHubRepositoryDto>.Ok(repository);
}
if (RetryableStatusCodes.Contains(response.StatusCode))
{
return Result<GitHubRepositoryDto>.Retry($"Received retryable status code '{response.StatusCode}'");
}
return Result<GitHubRepositoryDto>.Fail(response.StatusCode == HttpStatusCode.NotFound
? "The repository was not found"
: "Did not receive 200 OK status code");
}
}
catch (HttpRequestException exception)
{
_logger.LogError(exception, "HttpRequestException when calling the API");
return Result<GitHubRepositoryDto>.Retry("HttpRequestException when calling the API");
}
catch (TimeoutException exception)
{
_logger.LogError(exception, "TimeoutException during call to API");
return Result<GitHubRepositoryDto>.Retry("TimeoutException during call to API");
}
catch (OperationCanceledException exception)
{
_logger.LogError(exception, "Task was canceled during call to API");
return Result<GitHubRepositoryDto>.Retry("Task was canceled during call to API");
}
catch (Exception exception)
{
_logger.LogError(exception, "Unhandled exception when calling the API");
// Here it's up to you if you want to throw or return Retry/Fail, im choosing to FAIL.
return Result<GitHubRepositoryDto>.Fail("Unhandled exception when calling the API");
}
}
Now we are catching OperationCanceledException and return a Retry result. TaskCanceledException inherits from OperationCanceledException.
Deserialization
Some might have noticed that we are deserializing the http response to a string. That is not best practices and goes against my previous post You're (probably still) using HttpClient wrong and it is destabilizing your software. I choosed to do that because I wanted to keep the examples simple. But now the time has come to tackle deserialization in a better way. I will create a deserializer (based on Newtonsoft) and inject it instead of using the static api. Here's what it will look like:
IJsonSerializer
public interface IJsonSerializer
{
Task<Result<T>> TryDeserializeAsync<T>(Stream streamContent, CancellationToken cancellationToken);
Task<T> DeserializeAsync<T>(Stream stream, CancellationToken cancellationToken);
}
MyJsonSerializer
public class MyJsonSerializer : IJsonSerializer
{
private readonly JsonSerializer _jsonSerializer;
public MyJsonSerializer()
{
_jsonSerializer = new JsonSerializer { ContractResolver = JsonSerializerSettings.ContractResolver };
}
public static readonly JsonSerializerSettings JsonSerializerSettings = new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
};
public async Task<Result<T>> TryDeserializeAsync<T>(Stream streamContent, CancellationToken cancellationToken) where T : class
{
try
{
var result = await DeserializeAsync<T>(streamContent, cancellationToken);
return Result<T>.Ok(result);
}
catch (Exception e)
{
return Result<T>.Fail(e.Message);
}
}
public async Task<T> DeserializeAsync<T>(Stream stream, CancellationToken cancellationToken)
{
using (var streamReader = new StreamReader(stream))
using (var jsonReader = new JsonTextReader(streamReader))
{
var loaded = await JToken.LoadAsync(jsonReader, cancellationToken);
return loaded.ToObject<T>(_jsonSerializer);
}
}
}
Test
[Fact]
public async Task GivenDeserializationError_WhenGetRepository_ThenReturnsFailResult()
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("invalid json")
};
var httpClient = new HttpClient(new MockedHttpMessageHandler(response))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GitHubHttpClient(httpClient, faultyDeserializer, _fakeLogger);
var result = await sut.GetRepository(1, CancellationToken.None);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeFalse();
result.Message.ShouldBe("Failed to deserialize response");
}
Outcome
Test FAILED.
This test obviously failed because we are not using the MyJsonSerializer yet.
Fix
public async Task<Result<GitHubRepositoryDto>> GetRepository(int id, CancellationToken cancellationToken)
{
var request = new HttpRequestMessage(HttpMethod.Get, $"api/v1/repositories/{id}");
try
{
using (var response = await _httpClient.SendAsync(request, cancellationToken))
{
if (response.StatusCode == HttpStatusCode.OK)
{
if (response.Content == null)
{
return Result<GitHubRepositoryDto>.Fail("Response content was null");
}
var responseStream = await response.Content.ReadAsStreamAsync();
var deserializationResult = await _jsonDeserializer.TryDeserializeAsync<GitHubRepositoryDto>(responseStream, cancellationToken);
return deserializationResult.Success
? deserializationResult
: Result<GitHubRepositoryDto>.Fail($"Failed to deserialize stream to {nameof(GitHubRepositoryDto)}");
}
if (RetryableStatusCodes.Contains(response.StatusCode))
{
return Result<GitHubRepositoryDto>.Retry($"Received retryable status code '{response.StatusCode}'");
}
return Result<GitHubRepositoryDto>.Fail(response.StatusCode == HttpStatusCode.NotFound
? "The repository was not found"
: "Did not receive 200 OK status code");
}
}
catch (HttpRequestException exception)
{
_logger.LogError(exception, "HttpRequestException when calling the API");
return Result<GitHubRepositoryDto>.Retry("HttpRequestException when calling the API");
}
catch (TimeoutException exception)
{
_logger.LogError(exception, "TimeoutException during call to API");
return Result<GitHubRepositoryDto>.Retry("TimeoutException during call to API");
}
catch (OperationCanceledException exception)
{
_logger.LogError(exception, "Task was canceled during call to API");
return Result<GitHubRepositoryDto>.Retry("Task was canceled during call to API");
}
catch (Exception exception)
{
_logger.LogError(exception, "Unhandled exception when calling the API");
// Here it's up to you if you want to throw or return Retry/Fail, im choosing to FAIL.
return Result<GitHubRepositoryDto>.Fail("Unhandled exception when calling the API");
}
}
We've now replaced the static deserialization call with my new deserializer. Note that we're able to directly return the deserialization result since the TryDeserializeAsync has the same signature as the GetRepository method.
Great, we're now asynchronously deserializing the response...
...but how do we handle the following responses?
{}
or just an empty response like in our example earlier?
[Theory]
[InlineData("")]
[InlineData("{}")]
public async Task GivenInvalidJsonResponse_WhenGetRepository_ThenReturnsFailResult(string responseJson)
{
var response = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(responseJson)
};
var httpClient = new HttpClient(new MockedHttpMessageHandler(response))
{
BaseAddress = new Uri("http://localhost")
};
var sut = new GitHubHttpClient(httpClient, new MyJsonDeserializer(), _fakeLogger);
var result = await sut.GetRepository(1, CancellationToken.None);
result.Success.ShouldBeFalse();
result.Retryable.ShouldBeFalse();
result.Message.ShouldBe("Failed to deserialize response");
}
Outcome
The test that passes an empty string succeeds, but the test passing {} fails.
The deserialization still does not think that the empty object is a problem and will happily
create a new GitHubRepositoryDtoinstance.
Fix
public class GitHubRepositoryDto
{
public GitHubRepositoryDto(int? id, string name, string url, int? stars)
{
Id = id ?? throw new ArgumentNullException(nameof(id));
Name = name ?? throw new ArgumentNullException(nameof(name));
Url = url ?? throw new ArgumentNullException(nameof(url));
Stars = stars ?? throw new ArgumentNullException(nameof(stars));
}
public int Id { get; }
public string Name { get; }
public string Url { get; }
public int Stars { get; }
}
By removing the setters and adding a constructor to the GitHubRepositoryDto we can add some simple "validation" to the dto. This is also why I'm still using Newtonsoft instead of System.Text.Json, as far as I know, System.Text.Json does not currently support this.
If we now run the test again, both passes.
Final thoughts
Some of you might think "DAMN that's a lot of code for just handling some simple http calls". Well, as you've seen, there's a lot of different things that can (and will) go wrong when dealing with http.
Also if you look at the code in GitHubHttpClient, it's really easy to make it generic and move it to some kind of BaseHttpClient and reuse it (just replace GitHubRepositoryDto with T). Usually I create a BaseHttpClient with different virtual methods so I can choose to override specific behaviour for different integrations, since no integration ever looks the same... :)
I'm NOT using HttpCompletionOption.ResponseHeadersRead in this post simply because I came across the following bug report. It boils down to that the ReadAsStringAsync, ReadAsStreamAsync and ReadAsByteArrayAsync does not provide CancellationToken support and can deadlock in combination with ResponseHeadersRead. It will be fixed in .NET 5.
Thank you for reading!
