A paragraph, a block of code and a table don’t need to be split the same way.
This is a dev post classified by AGK Intelligence as AI dev tools (a tool drop), kept by the Dev Radar because it carries real work, not commentary.
A paragraph, a block of code and a table don’t need to be split the same way. Chonkie gives you different ways to split documents for RAG. You can divide text by token count, sentences or meaning, with separate methods for code and Markdown tables. You can also combine the steps in one pipeline: split the document, add overlap between chunks, create embeddings and send them to a vector database. For example, start with larger Markdown sections, then divide those into smaller chunks based on meaning. It works as a Python library or a self-hosted API, and you can install only the components
Posted by Sandhya (1.4k followers) 1 h ago · 0 likes · 29 views · view the original post on X. Kept by the Dev Radar as AI dev tools.
More dev work like this
- Ista dropping a coder Version of DS4.1 with half the experts! — @DJLougen
- Give your AI agent a document library it can actually search. — @DanKornas
- The fastest startups to $400M ARR: — @guilleflorvs
- Meet deepj-tinystories: a text-generation model from the deepj-prism family. It's small,… — @HuggingModels
- Slowly my entire gaming PC's desktop is ported to web (lots of old games) — @levelsio
- Kimi K2.7 Code is live on Chutes. — @chutes_ai
- I may or may have not just decompiled Battlefield 1942's raw C engine with Claude Code… — @levelsio
- Pydantic AI agents get durable with DBOS, and all you need is a database 💾 — @marcelotryle
Every post is read and classified by AGK Intelligence: what it is, which market it belongs to, and whether the link is a real tool. 31.2k posts from 5k X accounts over the last 21 days, 9k tools, 12 markets. New posts are collected several times an hour and the full ranking is recomputed every hour — last update 2026-11-02 15:27 UTC. Full method.