Running Elephas offline with LM Studio (Silicon Chip)
How We Got Elephas Offline: Let's Dive In!
There are two apps that you can use,
- LM Studio (Supports only M1, M2, M3 macs)
- Jan.ai (Supports both Silicon and Intel Macs)
In this post, we will see about LMStudio.
Why LM Studio
LM Studio offers a straightforward solution: download AI models to your local system for enhanced data security, bypassing risky unknown endpoints.
But only for (M1, M2, M3) Mac users.
Installing LM Studio (M1, M2, and M3)
Visit the page LM studio and download to set up on your Mac
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F3c7d3913-5a3b-4225-bcfd-85ad957f1942%2FUntitled.png?table=block&id=eec1f38a-01b2-45bf-a3c5-1c166c590926&cache=v2)
Configuring AI models
After installing LM Studio into your system, Do LM Studio → search→ Search the model is, “Llama-3-8B-Instruct-32k-v0.1-GGUF” and download it.
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F0d2b5d25-1472-421a-8071-7571760003e2%2FUntitled.png?table=block&id=67706897-fefc-406b-bdf1-95188d732a64&cache=v2)
There are many AI models available in the LM Studio, we suggest to use, Llama3 quantised model from here,
Running Local Server
Select the downloaded model,
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F6bbf8518-5d31-4ff2-8cc2-3d8355984da2%2FUntitled.png?table=block&id=85d828c3-62fc-42c6-b6e3-ea809cb80850&cache=v2)
Click the “Start Server” button
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2Fb4b08c74-dc38-4529-9ec7-fc2fcd774000%2FUntitled.png?table=block&id=e70ebeda-ddae-41c6-9b4a-98d261583938&cache=v2)
How use in connect with Elephas
Now go to Elephas → Preference → AI Providers → Custom AI, and Enter your local host url.
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F7b6f614c-2640-42b7-a3a9-70f2682ea380%2FUntitled.png?table=block&id=dcf7fccb-6259-4fc9-9636-db5b33bf6771&cache=v2)
And Tap on refresh models
How use in Elephas
Now go to Elephas → Preference → General, and you can pick a feature and select any LM Studio models available,
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2Fcec8919c-9e20-4a66-98df-5dba6c322cb0%2FUntitled.png?table=block&id=aa29deae-02ba-4e37-94cf-f348d046ea99&cache=v2)
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F1aa67d7a-2a5b-4c6a-b8df-823753014f00%2FUntitled.png?table=block&id=07e9e638-b142-4967-8ca3-9d1eb57eb848&cache=v2)
Super Chat
In Super Chat as well, you can select the LM studio models
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2Fe12b12a6-e9f4-4c62-86c0-3521dc17ef9e%2FUntitled.png?table=block&id=a8ebda57-34d6-46b8-a87e-0360925232b1&cache=v2)
Super brain
Super brain indexing still requires internet connection as it depends on backend AI engine, we will add the offline capability soon.
You can now (from 10.x version) index Super Brain files using offline indexing models. Check out LM Studio’s Text Embeddings | LM Studio
Make sure to choose the embedding model before starting the server,
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F63db89a5-9387-401f-9360-c4c113fa3bd7%2FCleanShot_2024-09-03_at_15.20.182x.png?table=block&id=41cfb3a2-cc61-41de-bf05-e9878da0f2e8&cache=v2)
When creating a brain, choose the new local model,
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2Fe3747f27-53cf-4013-a34f-a783619a9332%2FCleanShot_2024-09-03_at_15.22.252x.png?table=block&id=2d968ab5-a4db-42d2-ad0b-ae9b9689e775&cache=v2)
The performance may vary depending on your machine hardware.
Need help? We're Here for You!
Contact us in support@elephas.app.