condition-filtering-and-large-file-optimization

name: condition-filtering-and-large-file-optimization description: "根据数据规模动态选择处理策略。"

condition_filtering

Note: This sub-skill covers one step of the Excel analysis workflow. For the full pipeline (file reading, row counting, large-file optimization, export), see the parent workflow SKILL.md.

Step1 执行多维度数据清洗与条件筛选，包含列名自动识别、RGB 颜色过滤、前缀匹配及正则提取。

# 1. 自动识别同义列名并筛选非空值
target_cols = ['域名', '缩写', 'code', 'domain']
for col in target_cols:
    if col in df.columns:
        df = df[df[col].notna()]
        break

# 2. 基于数值通道的精确筛选（如 RGB 颜色过滤）
# 技巧：多条件组合筛选时使用 & 符号
if all(c in df.columns for c in ['Red', 'Green', 'Blue']):
    df = df[(df['Red'] == 0) & (df['Green'] == 0) & (df['Blue'] == 0)]

# 3. 基于字符串前缀筛选并进行数值转换计算
if '编号' in df.columns:
    # 筛选特定前缀的项目
    df = df[df['编号'].astype(str).str.startswith('TXL3')]
    # 技巧：使用 errors='coerce' 处理无法转换的脏数据
    df['val_a'] = pd.to_numeric(df['技工'], errors='coerce')
    df['val_b'] = pd.to_numeric(df['普工'], errors='coerce')
    df['total_val'] = df['val_a'] + df['val_b']
    avg_val = df['total_val'].mean()

# 4. 基于特定分类值的筛选与统计
if '钢筋级别' in df.columns:
    sub_df = df[df['钢筋级别'] == 'Ⅱ'].copy()
    sub_df['target_val'] = pd.to_numeric(sub_df['屈服荷载'], errors='coerce')
    avg_target = sub_df['target_val'].mean()

# 5. 正则表达式匹配提取特定字段
if '命令' in df.columns:
    pattern = r'--pct-'
    matched_df = df[df['命令'].astype(str).str.contains(pattern, na=False)]
    # 提取关键列保留追溯性
    extracted_data = matched_df[['NO', '命令', '说明']].copy()