Integrate Langtrace with LiteLLM

Introduction

We are excited to announce that Langtrace now supports LiteLLM natively. This means:

• If you are using LiteLLM for inference, you can setup Langtrace to automatically capture metrics such as token usage, cost, latency and request metadata.

• Additionally you can install Langtrace to your LiteLLM proxy and set it up to send metrics to any OpenTelemetry compatible observability tool in addition to Langtrace.

Setup

1. Sign up to Langtrace, create a project and get a Langtrace API key

2. Install Langtrace SDK

pip install -U langtrace-python-sdk

3. Setup the .env var

export LANGTRACE_API_KEY=YOUR_LANGTRACE_API_KEY

4. Initialize Langtrace in your code

import os
from langtrace_python_sdk import langtrace # Must precede any llm module imports
langtrace.init(api_key = os.environ['LANGTRACE_API_KEY'])

# Your code here

5. See the traces in Langtrace

For more information check out our docs here - https://docs.langtrace.ai/supported-integrations/llm-frameworks/litellm

Useful Resources

• Getting started with Langtrace https://docs.langtrace.ai/introduction

• Langtrace Website https://langtrace.ai/

• Langtrace Discord https://discord.langtrace.ai/

• Langtrace Github https://github.com/Scale3-Labs/langtrace

• Langtrace Twitter(X) https://x.com/langtrace_ai

• Langtrace Linkedin https://www.linkedin.com/company/langtrace/about/