Building a Modular Metallurgical Simulator with Streamlit and Layered Architecture

The Challenge of Integrating Complex Logic and Dynamic UIs

In the metallurgy-smart-calcs project, our goal was to develop an interactive metallurgical simulator capable of handling complex calculations for copper, lime, and salt processes, while also incorporating real-time market data for economic valuation. The core challenge lay in creating a robust application that could deliver an intuitive user experience via Streamlit, maintain complex business logic, and integrate external APIs, all without devolving into a monolithic, hard-to-maintain codebase.

The Problem: Coupling UI, Logic, and External Data

When developing interactive applications with intricate domain-specific calculations and external data dependencies, a common pitfall is tightly coupling the presentation layer (UI) with the business logic and data fetching mechanisms. This can lead to several issues:

1. Reduced Testability: Business rules become difficult to test in isolation without spinning up the UI or mocking extensive dependencies.
2. Poor Maintainability: Changes in the UI or an external API can ripple through the entire application, requiring modifications in unrelated parts of the codebase.
3. Scalability Issues: As the application grows, managing the complexity of intertwined components becomes overwhelming.

Our aim was to avoid these problems by designing a clear, modular architecture.

The Solution: A Layered Architectural Approach

To tackle these challenges, we implemented a layered architecture, separating concerns into distinct modules:

1. Interactive User Interface with Streamlit

Streamlit provided a rapid way to build an interactive front-end. This layer is responsible solely for user interaction: displaying input fields, process selections, output visualizations (like a sensitivity chart using plotly.express), and utility features such as unit conversion (e.g., % <-> ppm).

2. Encapsulating Domain Logic

The core business rules and calculations are encapsulated within a dedicated src/metallurgy/domain/ layer. This is where the actual 'smart calculations' reside. We structured this layer with:

• calculations.py: Contains specific hydro- and pyro-metallurgical formulas and conversion functions.
• models.py: Defines data classes and enums for representing metallurgical processes, materials, and calculation parameters, ensuring type safety and clarity.
• validators.py: Handles input checks and business rule validations, ensuring data integrity before calculations proceed.

This separation ensures that the complex metallurgical formulas are testable and reusable independently of the UI.

# src/metallurgy/domain/calculations.py

def calculate_copper_recovery(ore_grade: float, leaching_efficiency: float) -> float:
    """Calculates copper recovery percentage."""
    if not (0 <= ore_grade <= 100 and 0 <= leaching_efficiency <= 100):
        raise ValueError("Grades and efficiencies must be between 0 and 100.")
    return (ore_grade / 100) * (leaching_efficiency / 100) * 100

# src/metallurgy/domain/models.py
from dataclasses import dataclass
from enum import Enum

class ProcessType(Enum):
    HYDROMETALLURGY = "Hydrometallurgy"
    PYROMETALLURGY = "Pyrometallurgy"

@dataclass
class CopperProcessInputs:
    process: ProcessType
    ore_tonnes: float
    grade_percentage: float

3. Integrating External Market Data

To provide real-world economic context, the src/metallurgy/services/market_data.py module was created. This service is responsible for fetching live copper prices from Yahoo Finance using the yfinance library and applying a LB_PER_METRIC_TON conversion to estimate projected USD value. This isolates the external API interaction from the core domain logic.

# src/metallurgy/services/market_data.py
import yfinance as yf

LB_PER_METRIC_TON = 2204.62

def get_current_copper_price_usd_per_lb() -> float:
    """Fetches the current copper price in USD per pound."""
    try:
        # 'HG=F' is the ticker for Copper Futures
        copper_ticker = yf.Ticker("HG=F") 
        hist = copper_ticker.history(period="1d")
        current_price_usd_per_metric_ton = hist['Close'].iloc[-1]
        return current_price_usd_per_metric_ton / LB_PER_METRIC_TON
    except Exception as e:
        print(f"Error fetching copper price: {e}")
        return 3.50 # Fallback value

4. Robust Testing with Pytest

Crucially, automated tests were implemented using pytest for the domain logic and services. The conftest.py setup ensures that all modules can be imported and tested correctly. This guarantees the reliability of calculations and data fetching, allowing developers to refactor with confidence.

Results and Benefits

This layered approach yielded significant benefits:

• Clear Separation of Concerns: UI, business logic, and external data services are distinct, making the codebase easier to understand and navigate.
• Enhanced Testability: The core domain logic and market_data service are unit-testable, ensuring calculations and data fetching are accurate.
• Improved Maintainability: Changes in one layer (e.g., UI design, a new calculation formula, or a market data API change) have minimal impact on other layers.
• Flexibility: The domain logic can potentially be reused in other applications or APIs without modification.

Getting Started with Layered Design

If you're building an application that combines an interactive front-end with complex back-end logic and external integrations, consider these steps:

1. Define Your Domain: Clearly identify and encapsulate your core business rules and calculations in a dedicated layer.
2. Separate UI Concerns: Use a framework like Streamlit to focus solely on presentation and user interaction.
3. Abstract External Services: Create specific service layers for interacting with databases, APIs, or other external systems.
4. Prioritize Testing: Write unit tests for your domain logic and service layers to ensure robustness and correctness.

Key Insight

Just as a complex machine is built from specialized, interconnected components, a robust software application benefits immensely from a layered architecture. By giving each part a specific job, we create a system that is not only powerful in its current form but also adaptable and resilient to future changes. If your application's logic feels like a tangled knot, it's a sign to start untangling into distinct, testable layers. The initial investment in architectural clarity pays dividends in long-term maintainability and scalability, allowing innovation to flourish without constant fear of breaking existing functionality.

Generated with Gitvlg.com