Localstack: Using S3 with .Net Core

  • AWS
Photo by Frank Mckenna

In this post with saw how to locally test AWS with Docker running from a Docker Compose file but there are scenarios where we need to start a LocalStack container using DotNet, for example, we are preparing an environment for run integration tests of a Web API that use S3 Services. However, in this post, we will be using S3 with .Net Core Console App to do that.

In this post, you will learn how to:

  • Run LocalStack into a Docker Container using DotNet.
  • Access to panel UI of LocalStack.
  • Create a bucket.
  • Upload and list files into a bucket.

Prerequisites

This walkthrough assumes that will run LocalStack in a Windows machine and meet these requirements:

Startup

Creating a DotNet Solution

  1. First of all, we create a .Net Core Console App.
  2. Also, add the following Nuget packages:
    • Install-Package Docker.DotNet -Version 3.125.2
    • Install-Package AWSSDK.S3 -Version 3.3.110.57
  3. Next, create the LocalStackContainer class, this class will have all configuration for download the LocalStack image, services, and port mapping between the host and the container, these are required for access to S3.
using Docker.DotNet;
using Docker.DotNet.Models;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.InteropServices;
using System.Threading.Tasks;

namespace ConsoleApp
{
    public class LocalStackContainer : IAsyncDisposable
    {
        private readonly DockerClient _dockerClient;
        private const string _imageName = "localstack/localstack:0.11.0";
        private string _containerId;
        private const string _containerName = "localstack-container";

        public LocalStackContainer() => _dockerClient = new DockerClientConfiguration(new Uri(GetDockerUri())).CreateClient();

        public async Task RunContainer()
        {
            try
            {
                await CleanUpContainer();

                await GetImage();

                await StartContainer();
            }
            catch (Exception ex)
            {
                await CleanUpContainer();
                throw new Exception(ex.Message);
            }
        }

        private async Task GetImage()
        {
            await _dockerClient.Images
                .CreateImageAsync(new ImagesCreateParameters
                {
                    FromImage = _imageName
                },
                new AuthConfig(),
                new Progress<JSONMessage>());
        }

        private async Task StartContainer()
        {
            var response = await _dockerClient.Containers.CreateContainerAsync(new CreateContainerParameters
            {
                Name = _containerName,
                Image = _imageName,
                ExposedPorts = new Dictionary<string, EmptyStruct>
                {
                    { "4572", default},
                    { "8081", default},
                },

                HostConfig = new HostConfig
                {
                    PortBindings = new Dictionary<string, IList<PortBinding>>
                    {
                        {"4572", new List<PortBinding> {new PortBinding { HostPort = "4572" } }},
                        {"8081", new List<PortBinding> {new PortBinding { HostPort = "8081" } }}
                    }
                },
                Env = new List<string> { "SERVICES=s3:4572", "PORT_WEB_UI=8081" }
            });

            _containerId = response.ID;
            Console.WriteLine("Starting up the localstack container");
            await _dockerClient.Containers.StartContainerAsync(_containerId, null);
            Console.WriteLine("LocalStack is running");
        }

        private async Task CleanUpContainer()
        {
            var containertToDelete = (await _dockerClient.Containers
                                        .ListContainersAsync(new ContainersListParameters { All = true }))
                                        .Where(x => x.Names.Any(n => n.Contains(_containerName)));

            if (containertToDelete.Any())
            {
                foreach (var item in containertToDelete)
                {
                    if (item.State.Equals("running"))
                        await _dockerClient.Containers.KillContainerAsync(item.ID, new ContainerKillParameters());
                    else
                        await _dockerClient.Containers.StopContainerAsync(item.ID, new ContainerStopParameters());

                    await _dockerClient.Containers.RemoveContainerAsync(item.ID, new ContainerRemoveParameters { Force = true });
                }
            }
        }

        private string GetDockerUri()
        {
            var isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);

            if (isWindows)
            {
                return "npipe://./pipe/docker_engine";
            }

            var isLinux = RuntimeInformation.IsOSPlatform(OSPlatform.Linux);

            if (isLinux)
            {
                return "unix:/var/run/docker.sock";
            }

            throw new Exception("Unable to determine what OS this is running on");
        }

        public async ValueTask DisposeAsync()
        {
            if (_containerId != null)
            {
                Console.WriteLine("Shutdown localstack container");
                await CleanUpContainer();
            }
        }
    }
}
  1. Then, at the Program.cs class add the code for start and excute some operations against the local container and the S3 services.
using Amazon.Runtime;
using Amazon.S3;
using System;
using System.IO;
using System.Threading.Tasks;

namespace ConsoleApp
{
    internal class Program
    {
        private static async Task Main(string[] args)
        {
            await RunS3Example();
        }

        private static async Task RunS3Example()
        {
            await using (var stacklocal = new LocalStackContainer())
            {
                await stacklocal.RunContainer();

                var config = new AmazonS3Config()
                {
                    ServiceURL = "http://localhost:4572/",
                    ForcePathStyle = true,
                    Timeout = TimeSpan.FromSeconds(5)
                };
                var credentials = new BasicAWSCredentials("tem", "temp");

                IAmazonS3 s3Client = new AmazonS3Client(credentials, config);

                var bucketName = "bucket-test";

                Console.WriteLine($"Creating bucket {bucketName}");
                await s3Client.PutBucketAsync(bucketName);

                var list = await s3Client.ListBucketsAsync();

                Console.WriteLine("Reading buckets...");
                list.Buckets.ForEach((bucket) => { Console.WriteLine(bucket.BucketName); });

                const string path = @"dummy-file.jpg";

                var file = File.Create(path);

                Console.WriteLine("Uploading a file..");
                await s3Client.UploadObjectFromStreamAsync(bucketName, path, file, null);

                Console.WriteLine($"List files in bucket {bucketName}");
                var filesResponse = await s3Client.ListObjectsAsync(bucketName);
                filesResponse.S3Objects.ForEach((s3Object) => Console.WriteLine(s3Object.Key));
            }

            Console.ReadLine();
        }
    }
}

Conclusions

To sum up, using S3 with .Net Core and LocalStack for prepare for a development environment o run tests is easy. Although is possible to run LocalStack using a simple and readable docker-compose.yml file as we see in this post.

Additional resources

View or download sample code for this post.


Leave a Reply

Your email address will not be published. Required fields are marked *