Well, I was finally successful in getting Qwen3.6 35B-A3B APEX IQuality to run using both ik_llama.cpp and llama-cpp-turboquant. Now I just need to tweak and refine the flags a bit more. A totally offline model is not feasible for me, as it needs to be able to do web searches and synthesis, and to connect and transact within multiple blockchain ecosystems, Hive included. 😁🙏💚✨🤙
!ALIVE
!BBH
!PIZZA
!PIXY
!UNICOIN
For those who prioritize security, such LLMs should be used on a separate computer (which may be expensive) or in a virtual machine (which will reduce its efficiency) !INDEED. 🤔🤯🤓
!WEIRD
!HOPE
!PIZZA
There are multiple ways to prioritize security without crippling your setup. 😁🙏💚✨🤙
!ALIVE
!BBH
!PIXY
!PIZZA
Yes, and there would be pros and cons of each !INDEED, which an experienced user would barely have any problem choosing from. 🤔🧘♂️😎🤓
!HOPE
!WEIRD
!PIZZA
I'm far more concerned with the idea of important private data being accessed, passing through, and being recorded on the servers where cloud-based frontier models are hosted. That's a big reason, aside from being completely unwilling to pay to use them, that I choose to work with only locally-hosted models. 😁🙏💚✨🤙
!HOPE
!INDEED
!PIZZA
!WEIRD
I was thinking about locally-hosted AI LLMs, despite not being on cloud servers, send ("unintentionally" or otherwise) personal data on the machine to somewhere else on the Internet !INDEED. 🤔🤯 It is already a given on cloud-based AI LLMs that users should care to avoid putting any personal or sensitive information there. 🤯🤓
!WEIRD
!HOPE
!WINEX
For locally-hosted models, there's really no way for them to interact with the outside world unless it's via an agent harness, like OpenClaw, Hermes, Pi, Mercury, or any of the other agentic frameworks, since they give LLMs the tools and capabilities to do so. Because that's the case, strict controls can be placed on how they do so. 😁🙏💚✨🤙
!HOPE
!INDEED
!WINE
!WEIRD