Get the latest tech news
Lm.rs: Minimal CPU LLM inference in Rust with no dependency
Minimal LLM inference in Rust. Contribute to samuel-vitorino/lm.rs development by creating an account on GitHub.
Isn't it incredible that in a few years, we could have AGI running in a few lines of poorly written Rust code? Download the.safetensors, config.json and tokenizer.model files from the original model's page on huggingface (So we don't have to clone the pytorch repo). Compile the rust code with cargo (make sure to pass the target-cpu flag):
Or read this on Hacker News