Get the latest tech news
DeepSeek-R1-671B-Q4_K_M with 1 or 2 Arc A770 on Xeon
Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discr...
This guide demonstrates how to use llama.cpp portable zip to directly run llama.cpp on Intel GPU with ipex-llm(without the need of manual installations). Open "Command Prompt" (cmd), and enter the extracted folder through cd /d PATH\TO\EXTRACTED\FOLDER To use GPU acceleration, several environment variables are required or recommended before running llama.cpp. Open a "Terminal", and enter the extracted folder through cd /PATH/TO/EXTRACTED/FOLDER To use GPU acceleration, several environment variables are required or recommended before running llama.cpp.
Or read this on Hacker News