Part 2 for 3060 users looking to run @PrismML Ternary-Bonsai-2-27B-PQ2_O
This is a dev post classified by Jev as Other (a tutorial), kept by the Dev Radar because it carries real work, not commentary.
Part 2 for 3060 users looking to run @PrismML Ternary-Bonsai-2-27B-PQ2_O Here’s roughly how much context I can fit with different KV cache types using -np 1 and the default -b / -ub values: F16: ~45K Q8_0: ~88K Q4_0: ~145K I’m leaving -c unset, so llama.cpp automatically picks the maximum context it can fit while leaving roughly ~1 GB of VRAM free. Other flags and background VRAM usage can change these numbers, but this should give 12 GB RTX 3060 users a good baseline. I'm running this on PrismML's llama.cpp fork, you can find instruction in their hf model card. For most use-cases, it's
Posted by AJ (7.6k followers) 1 days ago · 70 likes · 5.5k views · view the original post on X. Kept by the Dev Radar as Other.
More dev work like this
- You can run locally Uncensored Ternary Bonsai 2-27B Same 5.9GB file. — @0x0SojalSec
- dear rtx 3060 owners, and every 12gb card behind it. bonsai 2 27b dense went from 26 to… — @sudoingX
- Nice: banked codex reset for all of us. — @kimmonismus
- Convolutional Neural Networks. #BigData #Analytics #DataScience #AI #MachineLearning… — @gp_pulipaka
- Reinforcement Learning and Optimal Control! #BigData #Analytics #DataScience #AI… — @gp_pulipaka
- The #Mathematical #Programming - #FPGA. #BigData #Analytics #DataScience #AI… — @gp_pulipaka
- A Constraint Based Approach, ML. #BigData #Analytics #DataScience #AI #MachineLearning… — @gp_pulipaka
- FBX対応も問題無さそう — @koguGameDev
Every post is read and classified by Jev (TypeSafe): what it is, which market it belongs to, and whether the link is a real tool. 14.4k posts from 4.7k X accounts over the last 21 days, 1.6k tools, 12 markets. Collected every 5 minutes, fully re-ranked every hour — last update 2026-09-19 21:10 UTC. Full method.