[ops] setup LLM tracing to grafana (using mlflow autologger) #269

StatGPT Backend version

0.5.0

What is the problem this feature will solve?

• currently we log LLM-calls for observability. this approach has its limitations (unclear order of calls; params missing, for example tool calls are not logged, only tool responses)
• using LLM tracing will improve dev and debug experience

What is the proposed feature or solution?

• mlflow has autologging feature, enabled with 2 lines of code

  import mlflow
  mlflow.langchain.autolog()

• it creates open telemetry traces for all LLM calls
• traces are logged to mlflow server by default
• but we can redirect traces to grafana server (or even create a dual export to both mlflow and grafana) using following guide: https://mlflow.org/docs/latest/genai/tracing/opentelemetry/export

What alternatives have you considered?

• keeping logs as they are
• switch to langchain native loggers (though they produce lots of uninformative logs)
• use mlflow autlogger, but log traces directly into mlflow, not to grafana