Recently, I’ve been exploring security Large Language Models (LLMs) and how to run Large Language Models (LLMs) directly on personal devices. I prioritized tools with end-to-end encryption, offline functionality, Vim support, and ease of installation.

Below are my top recommendations, which I will update regularly.

Disclaimer: Local LLMs may require more hardware resources.

AI Chat Bot

  • LM Studio
    • Graphical User Interface (GUI) based chat bot
    • Run locally
    • Free and Open Source
    • Supports MacOS, Linux, and Windows
  • Ollama
    • Command Line Interface (CLI) based chat bot
    • Run locally
    • Free and Open Source
    • Supports MacOS, Linux, and Windows

Coding Assistant