Workflow-Driven Modeling for the Compute Continuum: An Optimization Approach to Automated System and Workload Scheduling

The convergence of IoT, edge, cloud, and HPC technologies creates a heterogeneous compute continuum requiring sophisticated workload management. Current tools like SLURM, Kubernetes, and Snakemake lack automated optimization for cross-platform resource allocation, forcing users to manually map workloads across diverse infrastructures. We present a comprehensive framework integrating heterogeneous system and workload modeling integration with Snakemake followed by different tools and techniques like Mixed Integer Linear Programming (MILP) for multi-objective optimization to automate task mapping and scheduling across the compute continuum. Our approach extends Snakemake scheduler with formal mathematical models that optimize resource utilization and makespan. Experimental evaluation demonstrates that MILP-based solution achieves optimal scheduling for small-scale workflows (5x5 tasks) in 0.02 seconds, while heuristic methods provide 99.9% faster solutions for large-scale scenarios (5000×5000 tasks) with only 5-10% deviation from optimal makespan. For parallel workflows, the optimization achieves up to 16.7% makespan reduction compared to sequential scheduling approaches.