Add DuckDB query layer and benchmark visualizations for Quartz Solar Forecast

[yuvraajnarula]

Pull Request

Description

This PR introduces DuckDB as a query layer for Quartz Solar Forecast, enabling fast SQL-based aggregation and filtering on Parquet/CSV forecast data without loading entire files into memory. It also provides a benchmark suite comparing DuckDB vs Pandas performance, with integrated Seaborn plots for execution time and memory usage.

Currently, forecast outputs and historical weather data are stored in Parquet and CSV files. Users need to load entire datasets into memory or write custom aggregation scripts, which is inefficient for large datasets.

• DuckDB integration provides:
• Direct SQL queries on Parquet files without loading all data into memory.
• Fast aggregation, filtering, and joining across multiple Parquet files.
• Easy integration with Python via Pandas DataFrames.
• Efficient ad hoc analysis and model evaluation.

Fixes #323

How Has This Been Tested?

• Install pip install duckdb
• Run examples/duck_db_benchmark.py.
• Confirm that the benchmark CSV is generated:benchmarks_option_b/all_benchmarks.csv.

Check that Seaborn plots for execution time and memory usage display Pandas vs DuckDB side by side

• Yes

If your changes affect data processing, have you plotted any changes? i.e. have you done a quick sanity check?

• Yes

Checklist:

• My code follows OCF's coding style guidelines
• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• I have checked my code and corrected any misspellings

[yuvraajnarula]

Speed and Memory comparision

[yuvraajnarula]

@peterdudfield, any thoughts on this?