Get the latest tech news
Self hosting a Copilot replacement: my personal experience
Being able to run a Large Language Model locally also means to be able to use existing models (fine tuned for coding) to implement a self hosted solution to replace GitHub Copilot. In this post I will talk about my personal experience.
After I recently experimented with local LLMs using Ollama, I wanted to figure out if I could use some of these models to replace GitHub Copilot. Models like codellama:7b-code-q4_K_M or codellama:13b-code-q4_K_M were giving me better results but despite having 32 GB RAM available and a quite fast CPU, they were taking 3-4 seconds to complete what I was typing, making themselves useless (at least for my use case). While the idea of having a personal and private instance of a code assistant is interesting (and can also be the only available option in certain environments), the reality is that achieving the same level of performance as GitHub Copilot is quite challenging.
Or read this on Hacker News
