Using the fastest inference system Groq with Elephas
Groq AI provides fastest inference models, i.e responses are generated at a faster speed. They have some of the popular AI models hosted, including,
- Llama 3 70B
- Llama 3 7B
- Mixtral 8x 8B
- Gemma 7B
Here is the pricing info,
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2Fa9c3136e-3e03-4626-ae69-ef95c01dca52%2FCleanShot_2024-05-14_at_20.56.312x.png?table=block&id=5f036cca-fd84-4060-ab64-2d9cff1d9d44&cache=v2)
$.59 for Llama 3 70B, one of the powerful open source models. Here are the details from Meta’s launch page https://ai.meta.com/blog/meta-llama-3/
Llama 3 provided better response for English content than GPT-3.5 Turbo model in some tests.
Creating an API key with Groq
Visit https://console.groq.com/login and create your new account.
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F3c25deae-9e90-4cd1-b963-ce76c9bb6612%2F88774d85-3cbd-4c49-b795-b3868e168ad6.png?table=block&id=cf598af9-e282-4749-bc59-97ba155d399f&cache=v2)
then visit https://console.groq.com/keys to set up a new key.
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F88775825-6b34-4a86-b3b4-f7cda5c70b7e%2FCleanShot_2024-05-14_at_21.08.112x.png?table=block&id=98965504-7e80-48bf-8463-9fe6d79ed836&cache=v2)
Then you can fill that API key under Elephas → Preferences → AI Providers, select “Groq”, enter the API key you just created.
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F96aee523-2e8e-484f-bdc6-f35edae7efab%2FCleanShot_2024-08-16_at_09.55.312x.png?table=block&id=b7a710b5-f0f4-426b-984e-ab4b570d993f&cache=v2)
Then you can use the models provided by Groq AI in Chat Settings and Cost & Accuracy parts underPreferences → General
![Notion image](https://www.notion.so/image/https%3A%2F%2Fprod-files-secure.s3.us-west-2.amazonaws.com%2Fbd710b66-ff12-4d1d-b5ae-52dc27aa0eda%2F6d0c5c25-803b-481b-aa84-04ad03c0240b%2FCleanShot_2024-05-14_at_21.13.222x.png?table=block&id=22f53703-45c0-4e81-8515-d6ed0aeabc41&cache=v2)
Did this answer your question?
😞
😐
🤩