Generative Models with AI Horde (Behind the Book Bingo Generator, Part I)
Read about Book Bingo Generator
At the time of writing this blog post, accessing large language models usually requires access to a cloud service (such as OpenAI’s GPT4 API), or a computer with a beefy GPU and lots of video RAM. There are some tools that make it easier to run large language models locally, with or without GPU, and even from your phone! However, these tools often still require a fair amount of technical know-how to set up and use. Free alternatives like ChatGPT or Bing Chat might provide a simple way to interact with an LLM, but they do not allow much creativity in how they might be used outside of the default chat interface.
Happily, an effort called AI Horde is underway to create a crowd-sourced, distributed AI cluster for text and image generation. Thanks to people who do have access to GPU power and donate it to the horde, anyone can prompt a variety of open source LLMs. You can chat with models hosted on the AI Horde through KoboldAI Lite or generate images with ArtBot
AI Horde uses a unique rewards system called Kudos to help manage its resources. Users who sign up get Kudos for adding resources or helping improve the community in other ways. Kudos can also be spent to create texts or images with AI.
Kudos is earned through:
- Donating compute to the Horde as a worker node
- Rating AI generated images for LAION to support the development of free datasets for training models.
- Donations of Kudos from other users who want to reward positive contributions to the community.
While Kudos determine how fast you access AI models, everyone can use the service, even without Kudos or an API key, albeit with lower priority.
In fact, the Book Bingo Generator uses anonymous access to generate bingo cards publicly. However, you could potentially run book bingo generator locally and add your own API key for faster access! I’ll cover this in a future blog post.
It’s important to note that many of the open source models available on AI Horde are not regulated in the way that ChatGPT is and may not incorporate content policies or safety measures that are built into ChatGPT. While some may consider that a benefit, it can also mean that the model could generate potentially offensive content. This is always something to keep in mind when deciding if AI Horde is a good fit for a particular use case, especially in a public or educational setting.
For example, a goal when designing Bingo Card Generator was that it could make generative AI accessible in a fun and unique way to lots of people, including kids. However, during my initial testing, a couple reading challenge categories were generated that were not exactly kid appropriate. I think I mitigated this by modifying my prompt to request family friendly challenges suitable for middle grade readers, but I can’t guarantee that they will always be appropriate.
All in all, AI Horde provides a way to experiment and play with a wide variety of innovative open source models outside the standard OpenAI API. This can be a journey of discovery, with some models providing great responses to the right prompts, and others struggling to generate good content. Ultimately though, it offers a great community that people can easily contribute to and access generative AI technology without an expensive GPU, subscription or API fee. I believe that services like AI Horde are essential to ensure that all kinds of people find ways to participate in the development of the AI technologies that are poised to disrupt and transform our world.
In the next blog post in this series, I’ll go over a few technical details about how the Book Bingo Generator works, and how to run your own local instance of Book Bingo Generator with an AI Horde API key to speed up performance.
Header image by Gerd Altman.