Building a Microservices Ecosystem with .NET 8.0: Orders, Products, and Customers

In this comprehensive guide, we'll expand on our previous microservices example to create a more complex ecosystem. We'll build three interconnected microservices using .NET 8.0: Order Service, Product Service, and Customer Service. We'll then demonstrate how these services interact to create a robust e-commerce backend.

Table of Contents

1. Introduction
2. Architecture Overview
3. Setting Up the Microservices
4. Implementing the Microservices
   4.1 Product Service
   4.2 Customer Service
   4.3 Order Service
5. Inter-Service Communication
6. API Gateway
7. Running the Ecosystem
   
8. Microservices Best Practice
9. Conclusion

1. Introduction

Microservices architecture allows us to build complex systems by breaking them down into smaller, manageable services. In this article, we'll create three microservices that work together to manage an e-commerce platform:

• Product Service: Manages product information and inventory
• Customer Service: Handles customer data and authentication
• Order Service: Processes and manages orders

We'll use .NET 8.0 to build these services and demonstrate how they can communicate with each other to fulfill business operations.

2. Architecture Overview

Here's a high-level overview of our microservices ecosystem:

[API Gateway]
    |
    |--- [Product Service]
    |
    |--- [Customer Service]
    |
    |--- [Order Service]

• The API Gateway will route requests to the appropriate service.
• Each service will have its own database.
• Services will communicate with each other using HTTP/REST.

3. Setting Up the Microservices

Let's start by creating three separate projects for our microservices:

bash
dotnet new webapi -n ProductService
dotnet new webapi -n CustomerService
dotnet new webapi -n OrderService

For each project, add the following NuGet packages:

bash
dotnet add package Microsoft.EntityFrameworkCore.SqlServer
dotnet add package AutoMapper.Extensions.Microsoft.DependencyInjection
dotnet add package MediatR.Extensions.Microsoft.DependencyInjection
dotnet add package Microsoft.Extensions.Http

4. Implementing the Microservices

4.1 Product Service

Let's implement the Product Service:

csharp
// ProductService/Models/Product.cs
public class Product
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public decimal Price { get; set; }
    public int StockQuantity { get; set; }
}

// ProductService/Data/ProductDbContext.cs
public class ProductDbContext : DbContext
{
    public ProductDbContext(DbContextOptions<ProductDbContext> options) : base(options) { }
    public DbSet<Product> Products { get; set; }
}

// ProductService/Controllers/ProductsController.cs
[ApiController]
[Route("api/[controller]")]
public class ProductsController : ControllerBase
{
    private readonly ProductDbContext _context;

    public ProductsController(ProductDbContext context)
    {
        _context = context;
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<Product>> GetProduct(Guid id)
    {
        var product = await _context.Products.FindAsync(id);
        if (product == null) return NotFound();
        return product;
    }

    [HttpPost]
    public async Task<ActionResult<Product>> CreateProduct(Product product)
    {
        _context.Products.Add(product);
        await _context.SaveChangesAsync();
        return CreatedAtAction(nameof(GetProduct), new { id = product.Id }, product);
    }

    [HttpPut("{id}")]
    public async Task<IActionResult> UpdateStock(Guid id, int quantity)
    {
        var product = await _context.Products.FindAsync(id);
        if (product == null) return NotFound();
        product.StockQuantity = quantity;
        await _context.SaveChangesAsync();
        return NoContent();
    }
}

4.2 Customer Service

Now, let's implement the Customer Service:

csharp
// CustomerService/Models/Customer.cs
public class Customer
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public string Email { get; set; }
}

// CustomerService/Data/CustomerDbContext.cs
public class CustomerDbContext : DbContext
{
    public CustomerDbContext(DbContextOptions<CustomerDbContext> options) : base(options) { }
    public DbSet<Customer> Customers { get; set; }
}

// CustomerService/Controllers/CustomersController.cs
[ApiController]
[Route("api/[controller]")]
public class CustomersController : ControllerBase
{
    private readonly CustomerDbContext _context;

    public CustomersController(CustomerDbContext context)
    {
        _context = context;
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<Customer>> GetCustomer(Guid id)
    {
        var customer = await _context.Customers.FindAsync(id);
        if (customer == null) return NotFound();
        return customer;
    }

    [HttpPost]
    public async Task<ActionResult<Customer>> CreateCustomer(Customer customer)
    {
        _context.Customers.Add(customer);
        await _context.SaveChangesAsync();
        return CreatedAtAction(nameof(GetCustomer), new { id = customer.Id }, customer);
    }
}

4.3 Order Service

Finally, let's implement the Order Service, which will interact with both Product and Customer services:

csharp
// OrderService/Models/Order.cs
public class Order
{
    public Guid Id { get; set; }
    public Guid CustomerId { get; set; }
    public List<OrderItem> Items { get; set; }
    public DateTime OrderDate { get; set; }
    public decimal TotalAmount { get; set; }
}

public class OrderItem
{
    public Guid ProductId { get; set; }
    public int Quantity { get; set; }
    public decimal UnitPrice { get; set; }
}

// OrderService/Data/OrderDbContext.cs
public class OrderDbContext : DbContext
{
    public OrderDbContext(DbContextOptions<OrderDbContext> options) : base(options) { }
    public DbSet<Order> Orders { get; set; }
}

// OrderService/Services/ProductService.cs
public class ProductService
{
    private readonly HttpClient _httpClient;

    public ProductService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<bool> UpdateStock(Guid productId, int quantity)
    {
        var response = await _httpClient.PutAsJsonAsync($"api/products/{productId}", quantity);
        return response.IsSuccessStatusCode;
    }
}

// OrderService/Services/CustomerService.cs
public class CustomerService
{
    private readonly HttpClient _httpClient;

    public CustomerService(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<bool> CustomerExists(Guid customerId)
    {
        var response = await _httpClient.GetAsync($"api/customers/{customerId}");
        return response.IsSuccessStatusCode;
    }
}

// OrderService/Controllers/OrdersController.cs
[ApiController]
[Route("api/[controller]")]
public class OrdersController : ControllerBase
{
    private readonly OrderDbContext _context;
    private readonly ProductService _productService;
    private readonly CustomerService _customerService;

    public OrdersController(OrderDbContext context, ProductService productService, CustomerService customerService)
    {
        _context = context;
        _productService = productService;
        _customerService = customerService;
    }

    [HttpPost]
    public async Task<ActionResult<Order>> CreateOrder(Order order)
    {
        // Check if customer exists
        if (!await _customerService.CustomerExists(order.CustomerId))
            return BadRequest("Invalid customer");

        // Update product stock
        foreach (var item in order.Items)
        {
            if (!await _productService.UpdateStock(item.ProductId, -item.Quantity))
                return BadRequest($"Failed to update stock for product {item.ProductId}");
        }

        order.OrderDate = DateTime.UtcNow;
        _context.Orders.Add(order);
        await _context.SaveChangesAsync();

        return CreatedAtAction(nameof(GetOrder), new { id = order.Id }, order);
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<Order>> GetOrder(Guid id)
    {
        var order = await _context.Orders.FindAsync(id);
        if (order == null) return NotFound();
        return order;
    }
}

5. Inter-Service Communication

As you can see in the Order Service, we're using HttpClient to communicate with the Product and Customer services. This is a simple form of inter-service communication. In a production environment, you might want to consider more robust solutions like service discovery, message queues, or event-driven architectures.

6. API Gateway

To simplify client interactions with our microservices, we can implement an API Gateway. Here's a simple example using YARP (Yet Another Reverse Proxy):

bash
dotnet new web -n ApiGateway
cd ApiGateway
dotnet add package Microsoft.ReverseProxy

Then, update the Program.cs file:

csharp
// ApiGateway/Program.cs
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

var app = builder.Build();

app.MapReverseProxy();

app.Run();

And add the following to your appsettings.json:

json
{
  "ReverseProxy": {
    "Routes": {
      "products": {
        "ClusterId": "products",
        "Match": {
          "Path": "/products/{**catch-all}"
        },
        "Transforms": [
          { "PathPattern": "api/products/{**catch-all}" }
        ]
      },
      "customers": {
        "ClusterId": "customers",
        "Match": {
          "Path": "/customers/{**catch-all}"
        },
        "Transforms": [
          { "PathPattern": "api/customers/{**catch-all}" }
        ]
      },
      "orders": {
        "ClusterId": "orders",
        "Match": {
          "Path": "/orders/{**catch-all}"
        },
        "Transforms": [
          { "PathPattern": "api/orders/{**catch-all}" }
        ]
      }
    },
    "Clusters": {
      "products": {
        "Destinations": {
          "destination1": {
            "Address": "https://localhost:5001"
          }
        }
      },
      "customers": {
        "Destinations": {
          "destination1": {
            "Address": "https://localhost:5002"
          }
        }
      },
      "orders": {
        "Destinations": {
          "destination1": {
            "Address": "https://localhost:5003"
          }
        }
      }
    }
  }
}

7. Running the Ecosystem

To run our microservices ecosystem:

1. Start each microservice (Product, Customer, Order) on different ports.
2. Start the API Gateway.
3. Use the API Gateway URL to interact with the services.

For example, to create an order:

http
POST https://localhost:5000/orders
Content-Type: application/json

{
  "customerId": "00000000-0000-0000-0000-000000000001",
  "items": [
    {
      "productId": "00000000-0000-0000-0000-000000000001",
      "quantity": 2,
      "unitPrice": 10.99
    }
  ],
  "totalAmount": 21.98
}

This request will:

1. Check if the customer exists via the Customer Service
2. Update the product stock via the Product Service
3. Create the order in the Order Service
   

8. Microservices Best Practices

When developing a microservices architecture, it's crucial to follow best practices to ensure your system is robust, scalable, and maintainable. Here are some key best practices to consider:

8.1 Design Principles

1. Single Responsibility Principle: Each microservice should have a single, well-defined responsibility. In our example, we have separate services for products, customers, and orders.
2. Database per Service: Each microservice should have its own database. This ensures loose coupling and allows each service to choose the most appropriate database technology.
3. API First Design: Design your service APIs before implementing the services. This helps in clearly defining the service boundaries and interactions.
4. Stateless Services: Design your services to be stateless. This makes them easier to scale horizontally.

8.2 Development Practices

5. Use of Domain-Driven Design (DDD): Apply DDD principles to model your microservices around business domains.
6. Continuous Integration and Continuous Deployment (CI/CD): Implement robust CI/CD pipelines for each microservice to automate testing and deployment.
7. Containerization: Use containerization technologies like Docker to ensure consistency across different environments and facilitate easy deployment.
8. Automated Testing: Implement comprehensive unit tests, integration tests, and contract tests for each microservice.

8.3 Operational Practices

9. Centralized Logging: Implement a centralized logging system to aggregate logs from all microservices for easier debugging and monitoring.
10. Distributed Tracing: Use distributed tracing to track requests as they flow through your microservices ecosystem.
11. Health Checks: Implement health check endpoints in each service to facilitate monitoring and auto-healing.
12. Circuit Breaker Pattern: Implement circuit breakers to prevent cascading failures when a service is down.

8.4 Communication Practices

13. API Gateway: Use an API gateway to handle cross-cutting concerns like authentication, SSL termination, and routing.
14. Service Discovery: Implement service discovery to allow services to find and communicate with each other dynamically.
15. Event-Driven Architecture: Consider using an event-driven architecture for loose coupling and better scalability.
16. Asynchronous Communication: Use asynchronous communication where possible to improve responsiveness and scalability.

8.5 Data Management Practices

17. Data Consistency: Use patterns like Saga for managing data consistency across services in distributed transactions.
18. CQRS Pattern: Consider using the Command Query Responsibility Segregation (CQRS) pattern for complex domains with different read and write operations.
19. API Versioning: Version your APIs to allow for backward compatibility as services evolve.
20. Data Backup and Recovery: Implement robust data backup and recovery processes for each service's database.

Implementation Example: Health Checks

Let's implement health checks in our services as an example of applying these best practices. We'll add health checks to the Order Service:

csharp
// OrderService/Program.cs
var builder = WebApplication.CreateBuilder(args);

// ... other configurations ...

builder.Services.AddHealthChecks()
    .AddDbContextCheck<OrderDbContext>()
    .AddUrlGroup(new Uri("https://localhost:5001/health"), name: "product-service")
    .AddUrlGroup(new Uri("https://localhost:5002/health"), name: "customer-service");

var app = builder.Build();

// ... other middleware ...

app.MapHealthChecks("/health");

app.Run();

This adds a health check endpoint that checks:

• The Order Service's database connection
• The availability of the Product Service
• The availability of the Customer Service

You would then add similar health check endpoints to the Product and Customer services.

By implementing these best practices, you can create a more robust, scalable, and maintainable microservices architecture. Remember, not all practices may be necessary or applicable to every project. Always consider your specific requirements and constraints when deciding which practices to adopt.

9. Conclusion

In this article, we've created a microservices ecosystem using .NET 8.0, demonstrating how different services can work together to create a complex e-commerce backend. We've covered:

• Creating individual microservices for Products, Customers, and Orders
• Implementing inter-service communication
• Setting up an API Gateway to simplify client interactions

This architecture allows for independent scaling and deployment of services, making it easier to manage and evolve complex systems over time.

Remember, this is a simplified example. In a production environment, you'd need to consider additional factors such as:

• Authentication and authorization
• Resilience patterns (e.g., Circuit Breaker, Retry)
• Monitoring and logging
• Data consistency across services
• Testing strategies for microservices

As you continue to develop your microservices architecture, keep these considerations in mind to build a robust, scalable, and maintainable system. 

In addition to the basic implementation, we've now covered key best practices for developing microservices. By following these practices, you can create a more robust, scalable, and maintainable microservices architecture. Remember to continuously evaluate and refine your approach as your system grows and evolves.

Implementing the Cache-Aside Pattern in .NET 8.0 APIs with Entity Framework Core using Redis Cache

Introduction

In this blog post, we'll explore how to implement the Cache-Aside pattern in a .NET 8.0 API using Entity Framework Core. We'll focus on a real-world scenario: an online transaction processing system for an e-commerce platform. By the end of this post, you'll understand how to effectively use caching to improve your API's performance and reduce database load.

What is the Cache-Aside Pattern?

The Cache-Aside pattern is a caching strategy where the application is responsible for maintaining the cache. When data is requested, the application first checks the cache. If the data is not found (a cache miss), it retrieves the data from the database, stores it in the cache, and then returns it to the caller.

Our Scenario: E-commerce Order Processing

We'll build an API for an e-commerce platform that handles order processing. Our focus will be on the following operations:

1. Retrieving product details
2. Placing an order
3. Retrieving order status

Setting Up the Project

First, let's set up our .NET 8.0 API project with Entity Framework Core.

bash
dotnet new webapi -n ECommerceApi
cd ECommerceApi
dotnet add package Microsoft.EntityFrameworkCore.SqlServer
dotnet add package Microsoft.Extensions.Caching.StackExchangeRedis

Implementing the Data Models

Let's create our data models for products and orders.

csharp
public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    public decimal Price { get; set; }
    public int StockQuantity { get; set; }
}

public class Order
{
    public int Id { get; set; }
    public string CustomerEmail { get; set; }
    public DateTime OrderDate { get; set; }
    public decimal TotalAmount { get; set; }
    public string Status { get; set; }
    public List<OrderItem> Items { get; set; }
}

public class OrderItem
{
    public int Id { get; set; }
    public int ProductId { get; set; }
    public int Quantity { get; set; }
    public decimal UnitPrice { get; set; }
}

Setting Up Entity Framework Core

Now, let's set up our DbContext and configure Entity Framework Core.

csharp
public class ECommerceContext : DbContext
{
    public ECommerceContext(DbContextOptions<ECommerceContext> options) : base(options) { }

    public DbSet<Product> Products { get; set; }
    public DbSet<Order> Orders { get; set; }
    public DbSet<OrderItem> OrderItems { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        // Configure your entity relationships and constraints here
    }
}

Add the following to your Program.cs:

csharp
builder.Services.AddDbContext<ECommerceContext>(options =>
    options.UseSqlServer(builder.Configuration.GetConnectionString("DefaultConnection")));

Implementing the Cache-Aside Pattern

We'll use Redis as our distributed cache. Add the following to your Program.cs:

csharp
builder.Services.AddStackExchangeRedisCache(options =>
{
    options.Configuration = builder.Configuration.GetConnectionString("RedisConnection");
    options.InstanceName = "ECommerceCache_";
});

Now, let's create a caching service that implements the Cache-Aside pattern:

csharp
public interface ICacheService
{
    Task<T> GetOrSetAsync<T>(string key, Func<Task<T>> getItemCallback, TimeSpan expirationTime);
    Task RemoveAsync(string key);
}

public class RedisCacheService : ICacheService
{
    private readonly IDistributedCache _cache;
    private readonly ILogger<RedisCacheService> _logger;

    public RedisCacheService(IDistributedCache cache, ILogger<RedisCacheService> logger)
    {
        _cache = cache;
        _logger = logger;
    }

    public async Task<T> GetOrSetAsync<T>(string key, Func<Task<T>> getItemCallback, TimeSpan expirationTime)
    {
        var cachedResult = await _cache.GetStringAsync(key);

        if (cachedResult != null)
        {
            _logger.LogInformation("Cache hit for key: {Key}", key);
            return JsonSerializer.Deserialize<T>(cachedResult);
        }

        _logger.LogInformation("Cache miss for key: {Key}", key);
        var result = await getItemCallback();

        await _cache.SetStringAsync(key, JsonSerializer.Serialize(result), new DistributedCacheEntryOptions
        {
            AbsoluteExpirationRelativeToNow = expirationTime
        });

        return result;
    }

    public async Task RemoveAsync(string key)
    {
        await _cache.RemoveAsync(key);
        _logger.LogInformation("Removed cache for key: {Key}", key);
    }
}

Register the cache service in Program.cs:

csharp
builder.Services.AddSingleton<ICacheService, RedisCacheService>();

Implementing the API Endpoints

Now, let's implement our API endpoints using the Cache-Aside pattern.

Product Controller

csharp
[ApiController]
[Route("api/[controller]")]
public class ProductsController : ControllerBase
{
    private readonly ECommerceContext _context;
    private readonly ICacheService _cacheService;
    private readonly ILogger<ProductsController> _logger;

    public ProductsController(ECommerceContext context, ICacheService cacheService, ILogger<ProductsController> logger)
    {
        _context = context;
        _cacheService = cacheService;
        _logger = logger;
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<Product>> GetProduct(int id)
    {
        var cacheKey = $"product_{id}";
        var product = await _cacheService.GetOrSetAsync(cacheKey,
            async () =>
            {
                _logger.LogInformation("Fetching product {Id} from database", id);
                return await _context.Products.FindAsync(id);
            },
            TimeSpan.FromMinutes(10));

        if (product == null)
        {
            return NotFound();
        }

        return product;
    }

    [HttpPost]
    public async Task<ActionResult<Product>> CreateProduct(Product product)
    {
        _context.Products.Add(product);
        await _context.SaveChangesAsync();

        // Invalidate cache for this product
        await _cacheService.RemoveAsync($"product_{product.Id}");

        return CreatedAtAction(nameof(GetProduct), new { id = product.Id }, product);
    }

    [HttpPut("{id}")]
    public async Task<IActionResult> UpdateProduct(int id, Product product)
    {
        if (id != product.Id)
        {
            return BadRequest();
        }

        _context.Entry(product).State = EntityState.Modified;

        try
        {
            await _context.SaveChangesAsync();

            // Invalidate cache for this product
            await _cacheService.RemoveAsync($"product_{id}");
        }
        catch (DbUpdateConcurrencyException)
        {
            if (!await ProductExists(id))
            {
                return NotFound();
            }
            else
            {
                throw;
            }
        }

        return NoContent();
    }

    private async Task<bool> ProductExists(int id)
    {
        return await _context.Products.AnyAsync(e => e.Id == id);
    }
}

Order Controller

csharp
[ApiController]
[Route("api/[controller]")]
public class OrdersController : ControllerBase
{
    private readonly ECommerceContext _context;
    private readonly ICacheService _cacheService;
    private readonly ILogger<OrdersController> _logger;

    public OrdersController(ECommerceContext context, ICacheService cacheService, ILogger<OrdersController> logger)
    {
        _context = context;
        _cacheService = cacheService;
        _logger = logger;
    }

    [HttpPost]
    public async Task<ActionResult<Order>> PlaceOrder(Order order)
    {
        using var transaction = await _context.Database.BeginTransactionAsync();

        try
        {
            foreach (var item in order.Items)
            {
                var product = await _context.Products.FindAsync(item.ProductId);
                if (product == null || product.StockQuantity < item.Quantity)
                {
                    throw new InvalidOperationException($"Insufficient stock for product {item.ProductId}");
                }

                product.StockQuantity -= item.Quantity;
                _context.Entry(product).State = EntityState.Modified;

                // Invalidate cache for this product
                await _cacheService.RemoveAsync($"product_{item.ProductId}");
            }

            order.OrderDate = DateTime.UtcNow;
            order.Status = "Pending";

            _context.Orders.Add(order);
            await _context.SaveChangesAsync();

            await transaction.CommitAsync();

            return CreatedAtAction(nameof(GetOrder), new { id = order.Id }, order);
        }
        catch (Exception ex)
        {
            await transaction.RollbackAsync();
            _logger.LogError(ex, "Error placing order");
            return StatusCode(500, "An error occurred while placing the order.");
        }
    }

    [HttpGet("{id}")]
    public async Task<ActionResult<Order>> GetOrder(int id)
    {
        var cacheKey = $"order_{id}";
        var order = await _cacheService.GetOrSetAsync(cacheKey,
            async () =>
            {
                _logger.LogInformation("Fetching order {Id} from database", id);
                return await _context.Orders
                    .Include(o => o.Items)
                    .FirstOrDefaultAsync(o => o.Id == id);
            },
            TimeSpan.FromMinutes(5));

        if (order == null)
        {
            return NotFound();
        }

        return order;
    }

    [HttpPut("{id}/status")]
    public async Task<IActionResult> UpdateOrderStatus(int id, string status)
    {
        var order = await _context.Orders.FindAsync(id);
        if (order == null)
        {
            return NotFound();
        }

        order.Status = status;
        await _context.SaveChangesAsync();

        // Invalidate cache for this order
        await _cacheService.RemoveAsync($"order_{id}");

        return NoContent();
    }
}

Performance Considerations

1. Cache Expiration: We've set different expiration times for products (10 minutes) and orders (5 minutes). Adjust these based on your specific requirements and data volatility.
2. Cache Invalidation: We invalidate the cache when products are updated or when order statuses change. This ensures that the cached data remains consistent with the database.
3. Batch Operations: For high-volume scenarios, consider implementing batch cache operations to reduce network overhead.
4. Monitoring: Implement proper logging and monitoring to track cache hit/miss ratios and identify potential bottlenecks.

Conclusion

We've implemented the Cache-Aside pattern in our .NET 8.0 API using Entity Framework Core and Redis. This approach significantly reduces database load for read-heavy operations while ensuring data consistency for write operations.

Key takeaways:

1. The Cache-Aside pattern improves performance for frequently accessed, relatively static data.
2. Proper cache invalidation is crucial to maintain data consistency.
3. Use distributed caching (like Redis) for scalability in multi-instance deployments.
4. Adjust cache expiration times based on your data's volatility and consistency requirements.

By following these practices, you can build high-performance, scalable APIs that can handle the demands of modern e-commerce platforms.