https://lmcorner.net/tags/privatellm/Recent content in PrivateLLM on The .NET + LM SandboxHugo -- 0.150.1en-usMon, 23 Mar 2026 23:54:03 +0100https://lmcorner.net/posts/local-llm/Mon, 23 Mar 2026 23:54:03 +0100https://lmcorner.net/posts/local-llm/<h3 id="hello-there-">Hello there! 🖖</h3> <h3 id="recap">Recap</h3> <ul> <li><strong>Tool:</strong> llama.cpp</li> <li><strong>OS</strong>: Windows</li> </ul> <pre tabindex="0"><code>1. [Hugging Face Models](https://huggingface.co/models) 2. App: llama.cpp 3. Model: SmolVLM-500M-Instruct-GGUF 4. Win + R -&gt; winget install llama.cpp 5. CMD -&gt; llama-cli -hf ggml-org/SmolVLM-500M-Instruct-GGUF:Q8_0 /// Cleanup 1. winget list llama.cpp 2. winget uninstall --id ggml.llamacpp </code></pre><h2 id="step-by-step-implementation">Step-by-Step Implementation</h2> <h3 id="quick-way-to-run-llamacpp-on-windows">Quick Way to Run llama.cpp on Windows</h3> <ol> <li>Navigate to <a href="https://huggingface.co/models">Hugging Face Models</a></li> </ol> <ul> <li><strong>App:</strong> llama.cpp</li> <li><strong>Model:</strong> SmolVLM-500M-Instruct-GGUF <img alt="Hugging Face Model Select" loading="lazy" src="https://lmcorner.net/posts/local-llm/hf-1.png"></li> </ul> <ol start="2"> <li>Click &ldquo;Use this model&rdquo; -&gt; &ldquo;llama.cpp&rdquo;</li> </ol> <ul> <li>A modal window will appear with instructions on how to install llama.cpp and a command to run the selected model. <img alt="Hugging Face Model App Select" loading="lazy" src="https://lmcorner.net/posts/local-llm/hf-2.png"></li> </ul> <ol start="3"> <li>Use WinGet to install and run:</li> </ol> <pre tabindex="0"><code># Press Win + R and type powershell (or use Terminal/CMD) # 1. Install llama.cpp via Windows Package Manager winget install llama.cpp # 2. Download and run the model directly from Hugging Face in the console llama-cli -hf ggml-org/SmolVLM-500M-Instruct-GGUF:Q8_0 </code></pre><p><img alt="alt text" loading="lazy" src="https://lmcorner.net/posts/local-llm/hf-3.png"></p>