CSV Row Chunking

CSV row chunking is a method of splitting CSV files into smaller chunks based on the number of rows, rather than character count. This approach is particularly useful for structured data where you want to process CSV files in manageable row-based chunks while preserving the integrity of individual records.

Parameter	Type	Default	Description
rows_per_chunk	int	100	The number of rows to include in each chunk.
skip_header	bool	False	Whether to skip the header row when chunking.
clean_rows	bool	True	Whether to clean and normalize row data.
include_header_in_chunks	bool	False	Whether to include the header row in each chunk.
max_chunk_size	int	5000	Maximum character size for each chunk (fallback limit).