CosmosDb Abstraction

CosmosDb Abstraction

CosmosDb components are a pain to test. There's almost a 1/2 dozen objects to fake and configure to be able to get back the object you want to then do work on for the test.

This can be abstracted from the test and hide the complexity in some class or something... if there's one place I don't want to hide complexity, it's my tests.

Abstract 3rd Party Code

What we oughta do then... abstract all the cosmos bits so it never talks to our bits. Shocker, I know.

I ended up starting with a pattern pretty close to what we do at work; but it wasn't great. It was a lot of duplicative boiler plate all over.

I thought I was gonna have to hack in the old way I did things... but NOPE. One of the ingestion components hasn't been touched, so it still has an old style. Here's a simplified version

public sealed class SelectAllCardsQuery : ICosmosQuery<ICollection<ArtistCards>>
{
    private readonly ICosmosContainerAdapter _cosmosContainer;
    private readonly IQueryDefinition _queryDefinition;

    public async Task<ICollection<ArtistCards>> ExecuteAsync()
    {
        Dictionary<string, ArtistCards> artistCollection = new();

        QueryDefinition query = _queryDefinition.Query();
        using FeedIterator<CardArtistDataCosmosItem> iterator = await _cosmosContainer.GetItemQueryIterator<CardArtistDataCosmosItem>(query).NoSync();
        while (iterator.HasMoreResults)
        {
            FeedResponse<CardArtistDataCosmosItem> cosmosEntrySetSummaries = await iterator.ReadNextAsync().NoSync();
            foreach (CardArtistDataCosmosItem item in cosmosEntrySetSummaries)
            {
                artistCollection.Add(ProcessIntoItemToAdd(item));
            }
        }

        return artistCollection.Values;
    }
}

Walking through the method here we have the QueryDefinition which is a Cosmos class. Easy enough to create one for tests; but still a CosmosDb class in a class with our business logic.

The Cosmos Container is abstracted out a bit; but that abstraction returns a FeedIterator<T> which is a cosmos class. Which then produces some FeedResponse<T> objects. In this, already somewhat abstracted for, there are 3 CosmosDb classes that have to be configured. Configuring them is a PAIN IN THE ASS.

While this code isn't complicated; there is some behavior in the ProcessIntoItemToAdd that should be tested. It's a private method and I don't ascribe to making it public for testing. Kinda pretty against it.

I'm also against tough to configure tests. So this... this needs some abstraction.

public abstract class CosmosContainerQuery<T> : ICosmosContainerQuery<T>
{
    private readonly ICosmosContainerAdapter _cosmosContainer;

    public async Task<ICollection<T>> ExecuteAsync(QueryDefinition query)
    {
        List<T> list = new();
        FeedIterator<T> iterator = await _cosmosContainer.GetItemQueryIterator<T>(query).NoSync();
        while (iterator.HasMoreResults)
        {
            FeedResponse<T> items = await iterator.ReadNextAsync().NoSync();
            list.AddRange(items);
        }
        return list;
    }
}

Now I have an abstract class which is inherited by a knowledge class to provide the correct ICosmosContainerAdapter

public sealed class DiscoverArtistCardsCosmosContainerQuery<T> : CosmosContainerQuery<T>
{
    public DiscoverArtistCardsCosmosContainerQuery() : base(new DiscoveryArtistCardsContainer()) { }
}

AND THAT is used as composition in the new object allowing us to ignore all the cosmos classes

public sealed class SelectAllCardsQuery : ICosmosQuery<ICollection<ArtistCards>>
{
    public async Task<ICollection<ArtistCards>> ExecuteAsync()
    {
        Dictionary<string, ArtistCards> artistCollection = new();
        IEnumerable<ArtistCardIdsModel> artistCardIdsModels = await _cosmosContainerQuery.ExecuteAsync(_queryDefinition).NoSync();
        foreach (CardArtistDataCosmosItem item in cosmosEntrySetSummaries)
        {
            artistCollection.Add(item.Id, ProcessIntoItemToAdd(item));
        }
        return artistCollection.values
    }
}

We can fake the _cosmosContainerQuery object and the test provides whatever it needs.

It's great in my stuff and having this abstraction on hand, in a known works state speeds us up a lot when we need to apply the pattern at work.

There's an IAsyncEnumerable version of the cosmos abstraction so all the records aren't pulled in at once.

public abstract class CosmosContainerQueryAsync<T> : ICosmosContainerQueryAsync<T>
{
    private readonly ICosmosContainerAdapter _cosmosContainer;

    protected CosmosContainerQueryAsync(ICosmosContainerAdapter cosmosContainer) => _cosmosContainer = cosmosContainer;

    public async IAsyncEnumerable<T> ExecuteAsync(QueryDefinition query)
    {
        FeedIterator<T> iterator = await _cosmosContainer.GetItemQueryIterator<T>(query).NoSync();
        while (iterator.HasMoreResults)
        {
            FeedResponse<T> items = await iterator.ReadNextAsync().NoSync();
            foreach (T item in items)
            {
                yield return item;
            }
        }
    }
}

I hope this can help someone get better testability when using cosmos.

Show Comments