New Livestream: Running Uncensored and Open-Source LLMs on Your Local Machine

Anna Rovinskaia

Updated on May 24, 2024.

On May 23, 2024, we hosted our IntelliJ IDEA Livestream episode with Kenneth Kousen, where we discussed working with open-source LLMs on your local hardware and using the newest Java features to access these models.

Session abstract

General AI models, like ChatGPT, Claude AI, or Gemini, have a broader scope, and their answers to questions can be correspondingly imaginative. But many vendors don’t want to be held responsible for awkward answers, so they add “guard rails” to limit the responses. Those limitations often restrict the models so much as to make them unable to answer reasonable questions.

In this talk, we’ll discuss the Ollama system, which allows you to download and run open-source models on your local hardware. That means you can try out so-called “uncensored” models, with limited guard rails. What’s more, because everything is running locally, no private or proprietary information is shared over the Internet. Ollama also exposes the models through a tiny web server, so you can access the service programmatically.

We’ll look at how to do all of that, and how to use the newest Java features, like sealed interfaces, records, and pattern matching, to access AI models on your own hardware.

Asking questions

Kenneth will try to answer all of your questions during the session. If we run out of time, we’ll publish the answers to any remaining questions in a follow-up blog post.

Your speaker and host

Speaker

Kenneth Kousen

Ken Kousen is a Java Champion, JavaOne Rock Star, and Grails Rock Star. He is the author of the Pragmatic Library books Mockito Made Clear and Help Your Boss Help You, the O’Reilly books Kotlin Cookbook, Modern Java Recipes, and Gradle Recipes for Android, and the Manning book Making Java Groovy. He’s a regular speaker on the No Fluff, Just Stuff conference tour, and has given presentations at conferences around the world. Beyond this, he is also the creator of the Tales from the jar side newsletter and companion YouTube channel.
His academic background includes BS degrees in Mechanical Engineering and Mathematics from MIT, an MA and PhD in Aerospace Engineering from Princeton, and an MS in Computer Science from RPI. He is currently serving as the president of the Connecticut-based Kousen IT, Inc.

Host

Mala Gupta

A Java Champion and JUG leader, Mala has authored multiple books with Manning, Packt, and O’Reilly Publications. She has more than two decades of experience in the software industry and is a regular speaker at industry conferences around the world. She is an active supporter of Java certification as a path to career advancement.

Happy developing!

Subscribe to IntelliJ IDEA Blog updates

Subscribe form

By submitting this form, I agree to the JetBrains Privacy Policy Notification icon

By submitting this form, I agree that JetBrains s.r.o. ("JetBrains") may use my name, email address, and location data to send me newsletters, including commercial communications, and to process my personal data for this purpose. I agree that JetBrains may process said data using third-party services for this purpose in accordance with the JetBrains Privacy Policy. I understand that I can revoke this consent at any time in my profile. In addition, an unsubscribe link is included in each email.

Submit

Thanks, we've got you!