Ruan Pretorius
github profile: https://github.com/ruankie
gitlab profile: https://github.com/ruankie
Hi, I'm Ruan, a Data Scientist from melio.ai
Some of my favourite things in life are coffee, Python and AI.
Lately, I've been diving into LangChain and LLM agents. I've also been investigating how to better monitor and evaluate LLM applications.
I'm excited to share what I've learned and connect with fellow Python enthusiasts!
Accepted Talks:
Monitoring and Evaluating LLM Apps with Langfuse
Overview:
In the rapidly evolving landscape of AI, the ability to quickly iterate on prototypes and ensure robust performance of apps built around language models is crucial for developers and data scientists. This talk focuses on leveraging free and open-source tools to build, monitor, and evaluate LLM applications, providing a cost-effective approach to rapid development and deployment within the Python ecosystem.
Topics Covered:
- Setting up local LLMs for fast prototyping (7 min):
- Using Ollama for easy setup and deployment on local machines.
- Running LLMs locally to build and test prototypes without incurring costs.
- Monitoring LLM apps with Langfuse (15 min):
- Introduction to Langfuse, an open-source LLM engineering platform.
- Configuring Langfuse for tracing and evaluation.
- Implementing Langfuse's Python decorator and SDK for detailed monitoring.
- LLM-assisted evaluation (7 min):
- Using Langfuse to set up evaluation datasets and scoring responses of your LLM app.
Target Audience and Takeaways:
This talk is aimed at developers and data scientists who are interested in monitoring and optimising their LLM applications. By the end of the session, attendees will have a better understanding of how to set up and monitor their LLM apps using Langfuse, as well as how to leverage local LLMs for rapid prototyping and evaluation. They will walk away with actionable insights and practical knowledge to enhance their workflows, making their AI solutions more efficient and reliable.
