Markdown chunking is a method of splitting markdown based on structure like headers, paragraphs and sections.
This is useful when you want to process large markdown documents in smaller, manageable pieces.
Parameter
Type
Default
Description
chunk_size
int
5000
The maximum size of each chunk.
overlap
int
0
The number of characters to overlap between chunks.
