diff --git a/docs/ai-chat.md b/docs/ai-chat.md new file mode 100755 index 0000000000..dae42c56f9 --- /dev/null +++ b/docs/ai-chat.md @@ -0,0 +1,280 @@ +--- +meta_title: "Recommended AI Chat: Private ChatGPT Alternatives - Privacy Guides" +title: "AI Chat" +icon: material/assistant +description: Unlike OpenAI's ChatGPT and its Big Tech competitors, these AI tools do not train their models using your conversations. +cover: ai-chatbots.webp +--- +Protects against the following threat(s): + +- [:material-account-cash: Surveillance Capitalism](basics/common-threats.md#surveillance-as-a-business-model){ .pg-brown } +- [:material-close-outline: Censorship](basics/common-threats.md#avoiding-censorship){ .pg-blue-gray } + +Since the release of ChatGPT in 2022, interactions with Large Language Models (LLMs) have become increasingly common. LLMs can help us write better, understand unfamiliar subjects, or answer a wide range of questions. Based on a vast amount of data scraped from the web, they can statistically predict the next word. + +However, to improve the quality of LLMs, developers of AI software often use [Reinforcement Learning from Human Feedback](https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback) (RLHF). This entails the possibility of AI companies reading your private AI chats as well as storing them, which introduces a risk of data breaches. Furthermore, there is a real possibility that an LLM will leak your private chat information in future conversations with other users. To solve these problems, you can use trusted and privacy-focused providers or run AI models locally so your data never leaves your device. + +
+Ethical and Privacy Concerns about LLMs +AI models have been trained on massive amounts of public *and* private data. If you are concerned about these practices, you can either refuse to use AI or use [truly open-source models](https://proton.me/blog/how-to-build-privacy-first-ai), which publicly release their training datasets and therefore weren't trained on private data. One such model is [Olmoe](https://allenai.org/blog/olmoe) made by [Allenai](https://allenai.org/open-data). + +[Ethical concerns](https://www.thelancet.com/journals/landig/article/PIIS2588-7500(24)00061-X/fulltext) about AI range from their impact on climate to their potential for discrimination. +
+ + +## Cloud Providers + +The AI chat cloud providers listed here do not train their models using your chats and do not retain your chats for more than a month, based on each service's privacy policy. However, there is **no guarantee** that these privacy policies are honored. Read our [full list of criteria](#criteria) for more information. + +When using cloud-based AI chat tools, be mindful of the personal information you share. Even if a service doesn't store your conversations, there's still a risk of sensitive data being exposed or misused. To protect your privacy and security, **do not share sensitive information** related to health, finance, or other highly personal matters. + +A quick **overview** of the two providers we recommend: + +| Feature | DuckDuckGo AI | Brave Leo | +|---------|---------------|-----------| +| Tor Access | :material-check:{ .pg-green } Official onion service | :material-alert-outline:{ .pg-orange } Android-only (Orbot) | +| Rate Limits | :material-check:{ .pg-green } High | :material-alert-outline:{ .pg-orange } Low-Medium[^1] | +| Self-hosted Models | :material-close:{ .pg-red } | :material-check:{ .pg-green } | +| Web Search Integration | :material-close:{ .pg-red } | :material-check:{ .pg-green } | +| Multi-language Support | :material-check:{ .pg-green } | :material-alert-outline:{ .pg-orange } Limited | +| Account Required | :material-close:{ .pg-red } | :material-close:{ .pg-red } | +| Mobile Support | :material-check:{ .pg-green } | :material-check:{ .pg-green } only on Brave | + +[^1]: Rate limits vary by model, with Llama having the lowest restrictions + +### DuckDuckGo AI Chat + +
+ +![DuckDuckGo logo](assets/img/ai-chat/duckai.svg){align=right} + +**DuckDuckGo AI Chat** is a web frontend for AI models. It is made by the popular [search engine provider](search-engines.md) of the same name. +It is available directly on [DuckDuckGo](https://duckduckgo.com), [duck.ai](https://duck.ai), or [DuckDuckGo onion site](https://duckduckgogg41xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/chat). + +DuckDuckGo give you access to open-weights models from Meta and Mistral, as well as proprietary models from Anthropic and OpenAI. We strongly recommend you use open-weights models, because for those, no chat history is stored by Together.ai, the AI cloud platform DuckDuckGo uses to provide those models. +Furthermore, to protect your IP adress and prevent fingerprinting, DuckDuckGo proxies your chats through their servers. + +[:octicons-home-16: Homepage](https://duck.ai){ .md-button .md-button--primary } +[:simple-torbrowser:](https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/chat){ .card-link title="Onion Service" } +[:octicons-eye-16:](https://duckduckgo.com/aichat/privacy-terms){ .card-link title="Privacy Policy" } +[:octicons-info-16:](https://help.duckduckgo.com){ .card-link title="Documentation" } + +
+ +DuckDuckGo has agreements with their third-party providers that guarantee that they will not use your data for training their AI models. Proprietary model providers can keep a chat history for up to 30 days. For open-weights model, Duck uses the [together.ai](https://together.ai) AI cloud platform, and has disabled history for those chats. + +
+

Proprietary Model Providers Retain Your Chats

+We advise against using proprietary models from Anthropic or OpenAI because those providers keep a chat history for up to 30 days. +
+
+

DuckDuckGo Doesn't Self-Host Open Models

+You will have to trust the together.ai cloud platform to honor their commitments to not store chats. +
+ +### Brave Leo + +
+ +![Brave Logo](assets/img/ai-chat/leo.svg){align=right} + +**Brave Leo** is an AI assistant available inside the [Brave browser](https://brave.com), a browser we [recommend](tools/#private-web-browsers). + +Brave Leo supports a variety of models, including open-weights models from Meta and Mistral, and proprietary models from Anthropic. We **strongly recommend** that you use **open-weights models**, because **Brave self-hosts them** and for those open-weights models, they **discards all chat data** after you close your session. + +Additionally, the ["Bring Your Own Model"](https://brave.com/blog/byom-nightly/) (BYOM) feature allows you to use one of your local AI models directly in Brave. + +[:octicons-home-16: Homepage](https://brave.com/leo){ .md-button .md-button--primary } +[:octicons-eye-16:](https://brave.com/privacy/browser/#brave-leo){ .card-link title="Privacy Policy" } +[:octicons-info-16:](https://github.com/brave/brave-browser/wiki/Brave-Leo){ .card-link title="Documentation" } + +
+The default model is Mixtral, which has a low rate limit of 5 messages per hour. However, you can switch to the Llama model, which has "no" rate limits. + +Leo can enhance its knowledge through web searches, similar to Microsoft Copilot. However, Brave's AI solution still faces challenges with multi-language support and contextual understanding. + +
+

Page Content is Sent by Default

+By default, Brave Leo includes the webpage you are currently on as context for the AI model. While this can often be convenient, it also represents a privacy risk for pages with private information, such as your mailbox or social media. However, this feature cannot be globally disabled. Therefore, you'll need to **manually toggle off "Shape answers based on the page's contents"** for pages with PII. +
+
+

Proprietary Model Providers Retain Your Chats

+We advise against using Anthropic's Claude proprietary models because Anthropic keeps chat history for up to 30 days. +
+ +## Local AI Chat + +**Running AI models locally** offers a more private and secure alternative to cloud-based solutions, as **your data never leaves your device** and is therefore never shared with third-party providers. This provides peace of mind and **allows you to share sensitive information**. + +For the best experience, a dedicated GPU with sufficient VRAM or a modern system with fast LPDDR5X memory is recommended. Fortunately, it is possible to run smaller models locally even without a high-end computer or dedicated GPU. A computer with at least 8GB of RAM will be sufficient to run smaller models at lower speeds. Below is a table with more precise information : +
+Hardware Requirements for Local Models +Here are typical requirements for different model sizes: + +- 7B parameter models: 8GB RAM minimum, 16GB recommended +- 13B parameter models: 16GB RAM minimum, 32GB recommended +- 70B parameter models: Dedicated GPU with 24GB+ VRAM recommended +- Quantized models (4-bit): Can run with roughly half these requirements +
+ + +**To run AI locally, you need both an AI client and an AI model**. + +### Download AI models + +There are many permissively licensed **models available to download**. **[Hugging Face](https://huggingface.co/models?library=gguf)** is a platform that lets you browse, research, and download models in common formats like GGUF. Companies that provide good open-weights models include big names like Mistral, Meta, Microsoft, and Google. But there are also many community models and 'fine-tunes' available. For consumer-grade hardware, it is generally recommended to use [quantized models](https://huggingface.co/docs/optimum/en/concept_guides/quantization) for the best balance between model quality and performance. +To help you choose a model that fit your needs, you can look at leaderboards and benchmarks. The most widely-used leaderboard is [LM Arena](https://lmarena.ai/), a "Community-driven Evaluation for Best AI chatbots". There is also the [OpenLLM Leaderboard](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard), which focus on the performance of open-weights models on common benchmarks like MMLU-PRO. However, there are also specialed benchmarks, that for example measure [emotional intelligence](https://eqbench.com/), ["uncensored general intelligence"](https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard), and many [others](https://www.nebuly.com/blog/llm-leaderboards). + +
+Model Security and Verification +When downloading AI models, especially from Hugging Face, it's important to verify their authenticity. Look for: + +- Model cards with clear documentation +- Verified organization badge +- Community reviews and usage statistics +- **When available**, verify the file checksum (a type of anti-tampering fingerprint). On Hugging Face, you can find the hash by: + + 1. Clicking on a model file + 2. Looking for "Copy SHA256" button below the file + 3. Comparing this hash with the one you get after downloading (using tools like `sha256sum` on Linux/macOS or `certutil -hashfile file SHA256` on Windows) + +Those steps help ensure you're not downloading potentially malicious models. +
+ + +### AI chat clients + +| Feature | [Kobold.cpp](#koboldcpp) | [Ollama](#ollama) | [Llamafile](#llamafile) | +|---------|------------|---------|-----------| +| GPU Support | :material-check:{ .pg-green } | :material-check:{ .pg-green } | :material-check:{ .pg-green } | +| Image Generation | :material-check:{ .pg-green } | :material-close:{ .pg-red } | :material-close:{ .pg-red } | +| Speech Recognition | :material-check:{ .pg-green } | :material-close:{ .pg-red } | :material-close:{ .pg-red } | +| Auto-download Models | :material-close:{ .pg-red } | :material-check:{ .pg-green } | :material-alert-outline:{ .pg-orange } Few models available | +| Custom Parameters | :material-check:{ .pg-green } | :material-close:{ .pg-red } | :material-alert-outline:{ .pg-orange } | +| Multi-platform | :material-check:{ .pg-green } | :material-check:{ .pg-green } | :material-alert-outline:{ .pg-orange } Size limitations on Windows | + +#### Kobold.cpp + +
+ +![Kobold.cpp Logo](assets/img/ai-chat/kobold.png){align=right} + +Kobold.cpp is an AI client that runs locally on your Windows, Mac, or Linux computer. + +In addition to supporting a large range of text models, Kobold.cpp also supports image generators such as [Stable Diffusion](https://stability.ai/stable-image), and automatic speech recognition tools, such as [Whisper](https://github.com/ggerganov/whisper.cpp). + +[:octicons-home-16: Homepage](https://github.com/LostRuins/koboldcpp){ .md-button .md-button--primary } +[:octicons-info-16:](https://github.com/LostRuins/koboldcpp/wiki){ .card-link title="Documentation" } +[:octicons-code-16:](https://github.com/LostRuins/koboldcpp){ .card-link title="Source Code" } +[:octicons-lock-16:](https://github.com/LostRuins/koboldcpp/blob/2f3597c29abea8b6da28f21e714b6b24a5aca79b/SECURITY.md){ .card-link title="Security Policy"} + +
+Downloads + +- [:fontawesome-brands-windows: Windows](https://github.com/LostRuins/koboldcpp/releases) +- [:simple-apple: macOS](https://github.com/LostRuins/koboldcpp/releases) +- [:simple-linux: Linux](https://github.com/LostRuins/koboldcpp/releases) + +
+ +
+ +Kobold shines best when you are looking for heavy customization and tweaking, such as for roleplaying purposes. It allows you to modify parameters such as the AI model temperature and the AI chat's system prompt. It also supports creating a network tunnel to access AI models from other devices, such as your phone. + +
+

Compatibility Issues

+Kobold.cpp might not run on computers without AVX/AVX2 support. +
+ +#### Ollama + +
+ +![Ollama Logo](assets/img/ai-chat/ollama.svg){align=right} + +Ollama is an easy-to-use command-line AI assistant that is available on macOS, Linux, and Windows. + +In addition to supporting a wide range of text models, Ollama also supports [LLaVA](https://github.com/haotian-liu/LLaVA) models and has *experimental* support for Meta's [Llama vision capabilities](https://huggingface.co/blog/llama32#what-is-llama-32-vision). + +[:octicons-home-16: Homepage](https://github.com/ollama/ollama){ .md-button .md-button--primary } +[:octicons-info-16:](https://github.com/ollama/ollama){ .card-link title="Documentation" } +[:octicons-code-16:](https://github.com/ollama/ollama){ .card-link title="Source Code"} + +
+Downloads +- [:fontawesome-brands-windows: Windows](https://ollama.com/download/linux) +- [:simple-apple: macOS](https://ollama.com/download/mac) +- [:simple-linux: Linux](https://ollama.com/download/linux) +
+ +
+ +Ollama shines best when you're looking for an AI client that has great compatibility and ease of use. It runs on all desktop platforms and doesn't involve any manual setup, while still using inference and other techniques to make outputs faster. + +It also simplifies the process of setting up a local AI chat, as it downloads the AI model you want to use automatically. For example, running `ollama run llama3.2` will automatically download and run the Llama 3.2 model. Furthermore, ollama maintains their own [model library](https://ollama.com/library/) where they host the files of various AI models. This ensures models are vetted for both performance and security, eliminating the need to manually verify model authenticity. + +#### Llamafile + +
+ +![Llamafile Logo](assets/img/ai-chat/llamafile.svg){align=right} + +Llamafile is a single-file executable that allows users to run large language models locally on their own computers, without any setup involved. It is available on Linux, macOS, and Windows. + +The Mozilla-run project also supports LLaVA. However, it does not support speech recognition or image generation. It's also very lightweight and compatible. + +[:octicons-home-16: Homepage](https://github.com/Mozilla-Ocho/llamafile/){ .md-button .md-button--primary } +[:octicons-info-16:](https://github.com/Mozilla-Ocho/llamafile/?tab=readme-ov-file#llamafile){ .card-link title="Documentation" } +[:octicons-code-16:](https://github.com/ollama/ollama){ .card-link title="Source Code" } +[:octicons-lock-16:](https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#security){ .card-link title="Security Policy" } +
+Downloads +- [:fontawesome-solid-desktop: Desktop](https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#quickstart) +
+ +
+ +
+

Few Models Available

+Mozilla has only made llamafiles available for some Llama and Mistral models, while there are few third-party llamafiles available. Another issue is that Windows limits .exe files to 4GB, and most models are larger than that. To fix both of those issues, you can [load external weights](https://github.com/Mozilla-Ocho/llamafile?tab=readme-ov-file#using-llamafile-with-external-weights). +
+ +## Criteria + +Please note we are not affiliated with any of the projects we recommend. In addition to [our standard criteria](about/criteria.md), we have developed a clear set of requirements to allow us to provide objective recommendations. We suggest you familiarize yourself with this list before choosing to use a project and conduct your own research to ensure it's the right choice for you. + +### Minimum Requirements + +#### Cloud Providers + +- The provider or third-parties they use must not use your chats for training. +- The provider or third-parties they use must not retain your chats for more than 30 days. +- Must be accessible privately (no account required, accepts requests from VPN users). +- Must provide models they host themselves or with a third-party that acts on their behalf. +- Must provide at least one model with high rate limits, to allow an user to use it for medium to heavy workloads. + +#### Local AI clients + +- Must be open-source. +- Must not send personal data, including chat data. +- Must be available on Linux. +- Must not require a GPU. +- Must have support for GPU-powered fast inference. +- Must not require an internet connection. + +### Best-Case + +Our best-case criteria represent what we *would* like to see from the perfect project in this category. Our recommendations may not include any or all of this functionality, but those which do may rank higher than others on this page. + +#### Cloud Providers + +- Should not retain your chats. +- Should be accessible anonymously trough Tor. +- Should only offer self-hosted open-weights models. +- Should not be rate-limited. + +#### Local AI clients +- Should be multi-platform. +- Should be easy to download and set up, such as having a one-click install process. +- Should have a built-in model downloader option. +- Should be customizable (the user can modify the LLM paramaters, such as its system prompt or its temperature). diff --git a/docs/tools.md b/docs/tools.md index ba6f577a46..673c8c37f5 100644 --- a/docs/tools.md +++ b/docs/tools.md @@ -470,6 +470,31 @@ For encrypting your OS drive, we typically recommend using the encryption tool y [Learn more :material-arrow-right-drop-circle:](frontends.md) +### AI Chat + +#### Cloud Providers + +
+ +- ![Duck AI logo](assets/img/ai-chat/duckai.svg){ .twemoji loading=lazy }[Duck AI](ai-chat.md#duckduckgo-ai-chat) +- ![Leo AI logo](assets/img/ai-chat/leo.svg){ .twemoji loading=lazy }[Brave Leo](ai-chat.md#brave-leo) + +
+ +[Learn more :material-arrow-right-drop-circle:](ai-chat.md#cloud-providers) + +#### Local AI + +
+ +- ![Kobold logo](assets/img/ai-chat/kobold.png){ .twemoji loading=lazy }[Kobold.cpp](ai-chat.md#koboldcpp) +- ![Llamafile logo](assets/img/ai-chat/llamafile.svg){ .twemoji loading=lazy }[Llamafile](ai-chat.md#llamafile) +- ![Ollama logo](assets/img/ai-chat/ollama.svg){ .twemoji loading=lazy }[Ollama](ai-chat.md#ollama) + +
+ +[Learn more :material-arrow-right-drop-circle:](ai-chat.md#local-ai-chat) + ### Language Tools
diff --git a/includes/abbreviations.en.txt b/includes/abbreviations.en.txt index 03c478c7a7..9824fe583c 100644 --- a/includes/abbreviations.en.txt +++ b/includes/abbreviations.en.txt @@ -49,6 +49,8 @@ *[ISPs]: Internet Service Providers *[JNI]: Java Native Interface *[KYC]: Know Your Customer +*[LLaVA]: Large Language and Vision Assistant (multimodal AI model) +*[LLMs]: Largue Language Models (AI models such as ChatGPT) *[LUKS]: Linux Unified Key Setup (Full-Disk Encryption) *[MAC]: Media Access Control *[MDAG]: Microsoft Defender Application Guard @@ -62,6 +64,7 @@ *[OCSP]: Online Certificate Status Protocol *[OEM]: Original Equipment Manufacturer *[OEMs]: Original Equipment Manufacturers +*[open-weights]: An open weights-model is an AI model that anyone can download and use, but for which the underlying training data and/or algorithms are proprietary. *[OS]: Operating System *[OTP]: One-Time Password *[OTPs]: One-Time Passwords @@ -72,7 +75,7 @@ *[PGP]: Pretty Good Privacy (see OpenPGP) *[PII]: Personally Identifiable Information *[QNAME]: Qualified Name -*[QUIC]: A network protocol based on UDP, but aiming to combine the speed of UDP with the reliability of TCP. +*[rate limits]: Rate limits are restrictions that a service imposes on the number of times a user can access their services within a specified period of time. *[rolling release]: Updates which are released frequently rather than set intervals *[RSS]: Really Simple Syndication *[SELinux]: Security-Enhanced Linux @@ -86,6 +89,8 @@ *[SaaS]: Software as a Service (cloud software) *[SoC]: System on Chip *[SSO]: Single sign-on +*[system prompt]: The system prompt of an AI chat is the general instructions given by human to guide how it should operate. +*[temperature]: AI temperature is a parameter used in AI models to control the level of randomness and creativity in the generated text. *[TCP]: Transmission Control Protocol *[TEE]: Trusted Execution Environment *[TLS]: Transport Layer Security diff --git a/mkdocs.yml b/mkdocs.yml index bcab6a69d7..ba8d4d1af3 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -403,6 +403,7 @@ nav: - "search-engines.md" - "vpn.md" - !ENV [NAV_SOFTWARE, "Software"]: + - "ai-chat.md" - "calendar.md" - "cryptocurrency.md" - "data-redaction.md" diff --git a/theme/assets/img/ai-chat/duckai.svg b/theme/assets/img/ai-chat/duckai.svg new file mode 100644 index 0000000000..f747a6e345 --- /dev/null +++ b/theme/assets/img/ai-chat/duckai.svg @@ -0,0 +1,2 @@ + + diff --git a/theme/assets/img/ai-chat/kobold.png b/theme/assets/img/ai-chat/kobold.png new file mode 100644 index 0000000000..b20a1c4220 Binary files /dev/null and b/theme/assets/img/ai-chat/kobold.png differ diff --git a/theme/assets/img/ai-chat/leo.svg b/theme/assets/img/ai-chat/leo.svg new file mode 100644 index 0000000000..8e1ef5e770 --- /dev/null +++ b/theme/assets/img/ai-chat/leo.svg @@ -0,0 +1,2 @@ + + diff --git a/theme/assets/img/ai-chat/llamafile.svg b/theme/assets/img/ai-chat/llamafile.svg new file mode 100644 index 0000000000..cbbe782733 --- /dev/null +++ b/theme/assets/img/ai-chat/llamafile.svg @@ -0,0 +1,2 @@ + + diff --git a/theme/assets/img/ai-chat/ollama.svg b/theme/assets/img/ai-chat/ollama.svg new file mode 100644 index 0000000000..4dfed8200e --- /dev/null +++ b/theme/assets/img/ai-chat/ollama.svg @@ -0,0 +1,2 @@ + + diff --git a/theme/assets/img/cover/ai-chatbots.webp b/theme/assets/img/cover/ai-chatbots.webp new file mode 100644 index 0000000000..cdba241ca9 Binary files /dev/null and b/theme/assets/img/cover/ai-chatbots.webp differ