multi-sheet-reading-and-analysis

name: multi-sheet-reading-and-analysis description: "用于读取多工作表Excel文件，动态评估数据量以启用Parquet大文件优化，并执行正则清洗、分类汇总、线性拟合及生成带格式的图表与结果文件。"

Step1 统计多工作表总行数，并根据数据量级（如≥1万行）动态启用Parquet格式转换以优化大文件读取性能。

import pandas as pd
import os
from openpyxl import load_workbook

file_path = "your_excel_file.xlsx"
xls = pd.ExcelFile(file_path)
sheet_names = xls.sheet_names

# 统计所有sheet的数据行数
total_rows = 0
for sheet in sheet_names:
    wb = load_workbook(file_path, read_only=True, data_only=True)
    ws = wb[sheet]
    max_row = ws.max_row
    data_rows = max_row - 1 if max_row > 0 else 0
    total_rows += data_rows
    wb.close()

print(f"总数据行数: {total_rows}")

# 大文件优化：转换为Parquet格式读取
if total_rows >= 10000:
    df = pd.read_excel(file_path, sheet_name=sheet_names[0])
    parquet_path = '/tmp/temp_data.parquet'
    df.to_parquet(parquet_path, engine='pyarrow')
    df = pd.read_parquet(parquet_path)
else:
    df = pd.read_excel(file_path, sheet_name=sheet_names[0])