Integrating AWS SDK v2 with Akka streams

It's been a while since I last used AWS S3 a lot. In the mean time AWS came up with a new SDK and I started working with Scala and Akka full time. The SDK is still in beta but it's hard to imagine it'll take them more than a few more months to finish. Akka is still convenient but choke-full of APIs at different levels of complexity. Among other things I am researching how some of them could play together nicely. While digging, invariably a nugget or two can be found.

I wanted to plug S3 file download functionality from the SDK into a code path based on Akka streams. The former is all about CompletableFutures, the latter is about Source/Sink abstractions wired together with mostly built-in stages. Both are relatively large and self-sufficient frameworks with their own thread pools and coding idioms. I found one easy way to make them work together.

The trick is to notice that the AsyncResponseHandler returns a reactive stream Publisher. Yes, a third famous API in a file of less than a hundred lines. Akka can create a Source form a Publisher. It was also an unusual opportunity to use a Scala Promise (a quick reminder - in Scala a Promise provides write access to the state represented by a Future for read access). The idea is

• as is custom in Scala, a time consuming operation is wrapped into a Future
• a Promise is created before a download operation is started
• when the download operation is started, the Promise is used to create a Future that is returned to the client
• when the download operation finishes, the Promise is completed successfully
• there is also the error case where the Promise is used to convey the error to the client

A unit test illustrates how a created Akka Source can be used to actually transfer some data using built-in Akka file Sink. I use a small S3 file made publicly available by some kind strangers.  

My first impression from this exercise is that it is reasonable but feels a little unnatural. Yes, the code itself is quite concise and the performance penalty must be negligible. But the j.u.c ecosystem is mostly alien to idiomatic Scala.