Measuring Financial Analyst Engagement using LLMs

As a freelance machine learning engineer, I worked with a finance professor on a research project aimed at quantifying analyst engagement during earnings calls using large language models. The client came in with an open-ended research question, and I played a key role in translating it into a clear technical plan and deliverables. My contributions included:

  • preprocessing over 1,000 earnings call transcripts

  • generating vector embeddings using Hugging Face Transformers and the OpenAI API

  • computing semantic similarity between analyst questions and management answers to create engagement scores

  • Using LLMs to determine whether questions asked had previously been answered or not

The project provided the client with a quantitative framework for evaluating analyst performance based on natural language interaction. My tech stack included:

  • Python and it’s Data Science libraries (Pandas and Numpy)

  • PyTorch

  • Hugging Face Transformers

  • OpenAI API

  • Pandas, NumPy