invalid-data-cleaning

by OpenSenseNova5.8k

用于大规模Excel数据的预处理，通过统计总行数判断是否转换为Parquet格式以提升读写效率，并使用正则表达式清洗指定文本列（如仅保留中文字符），最后导出清洗后的文件并提供下载链接。

Install

The skill file stays in the publisher's repository.

Paste the copied command in a terminal. It installs this SKILL.md into that agent.

npx skills add OpenSenseNova/SenseNova-Skills --skill invalid-data-cleaning -a cursor -y

Download skill skills/sn-da-excel-workflow/capability/excel-data-cleaning/invalid-data-cleaning/SKILL.md on GitHub

Related skills

View all

Questions

How do I install invalid-data-cleaning?: Download skills/sn-da-excel-workflow/capability/excel-data-cleaning/invalid-data-cleaning/SKILL.md from the publisher, or copy the path and open the repository. PlugMCP lists the skill and does not host the file.
Where does invalid-data-cleaning live?: The publisher keeps it at https://github.com/OpenSenseNova/SenseNova-Skills.
Which agents can use invalid-data-cleaning?: Claude, Cursor, and Codex can read a SKILL.md you add from the publisher's repository.

name: invalid-data-cleaning description: "用于大规模Excel数据的预处理，通过统计总行数判断是否转换为Parquet格式以提升读写效率，并使用正则表达式清洗指定文本列（如仅保留中文字符），最后导出清洗后的文件并提供下载链接。"

Invalid_Data_Cleaning

This sub-skill covers one capability of the Excel workflow. For reading/counting/Parquet optimization, see the parent workflow SKILL.md.

Skill Steps

Step1 根据总行数判断是否数据量过大，若满足条件，则将 Excel 文件转换为 Parquet 格式提升读写效率，再读取数据进行后续分析。

import pandas as pd

file_path = "input_data.xlsx"
parquet_path = "temp_data.parquet"

# 读取 Excel 文件并转换为 Parquet 格式
xls = pd.ExcelFile(file_path)
dfs = []
for sheet in xls.sheet_names:
    df_sheet = pd.read_excel(xls, sheet_name=sheet)
    dfs.append(df_sheet)

# 合并所有 sheet 数据并写入 Parquet 文件
if dfs:
    df_all = pd.concat(dfs, ignore_index=True)
    df_all.to_parquet(parquet_path, engine='pyarrow', index=False)

# 读取 Parquet 文件用于后续处理
df = pd.read_parquet(parquet_path)