Summary

Iveco Group partners with Nebuly to understand its LLM users' and shape business decisions across tens of LLM use cases.

Use case

Employees Copilot

Sector

Automotive

llM in use

Azure OpenAI & AWS Bedrock

EMPLOYEES

35K+

nebuly deployment

Self-hosted on Iveco Group Public Cloud

Oct 03, 2024
Iveco Group’s
AI-Powered Transformation
Iveco Group, a global automotive leader, employs more than 36,000 people around the world and has 20 industrial sites and 31 R&D centres. The Group quickly recognized the potential of AI after the launch of ChatGPT and established an IT Team of Excellence to speed up AI adoption in IT projects and solutions. LLMs are now at the heart of the Iveco Group's strategy to enhance efficiency and competitiveness.

Nebuly’s LLM User Analytics was implemented to capture user behavior within each LLM use case, enabling continuous AI improvements and transparent reporting across the organization. Iveco Group is rapidly scaling its AI efforts, experimenting and deploying tens of LLM-based applications - powered by Azure OpenAI and AWS Bedrock. Their flagship GenAI initiative, ChatIVG, improves access to information, automates tasks, and boosts efficiency for thousands of employees every day.
The perception of new AI technologies among Iveco Group employees has been extremely positive, marking a significant step forward in the company’s digital transformation.”

Fabio RicciatoHead of IT Customer Journey & Digital Architecture, Artificial Intelligence and DEX

How to ensure business success with GenAI?

As Iveco Group's IT Artificial Intelligence Team rolled out dozens of chatbots, they identified three critical requirements to ensure user satisfaction and business results:
01. Understand the most common user questions.Do not rely solely on time-consuming and intrusive user interviews.
02. Capture more feedback to refine the chatbot experience. Existing user surveys showed signs of bias and lacked the volume needed to drive LLM improvements.
03. Provide business stakeholders with insights into user behavior and satisfaction.This is need for better investment allocation and cost to value optimization.
Iveco Group considered building these capabilities in-house, but the complexity and variety of use cases would have required significant time and resources. When Iveco Group connected with Nebuly, it became clear that their partnership could be mutually beneficial in optimizing LLM user experience and driving ROI from GenAI initiatives.
How Iveco Group uses Nebuly?
The Nebuly platform automatically analyzes user behavior with LLM products at scale. This allows Iveco Group to rely on Nebuly as a single source of truth for the success of their AI initiatives.
Nebuly gives us confidence that we have a strong foundation of GenAI user behavior data that we can trust, which enables us to move much faster with our AI agenda.”

Fabio RicciatoHead of IT Customer Journey & Digital Architecture, Artificial Intelligence and DEX

01. Single pane of glass for GenAI user behavior data
Nebuly’s User Intelligence features categorize interactions into topics and user intents, allowing Iveco Group to gain instant visibility into tens of thousands of conversations.

I used to have to rely entirely on user interviews to understand what users were trying to accomplish with our copilots. While hearing from end-users is very valuable and we will continue to do so, the process is not scalable due to time constraints. With Nebuly, we can validate findings from user surveys, but most importantly we can extend our intelligence to 100% of our chatbot users with just one click.
Now, I have the ability to analyze any AI project at any time in as much detail as I need. In fact, with Nebuly I can zoom out to understand what topics are trending across different projects, but I can also drill down to a specific user intent while understanding the user sentiment. Nebuly’s platform is incredibly flexible.”

Andrea RussottoIT Artificial Intelligence / Generative AI and Prompt Engineering

02. Powering continuous improvement in LLM user experience
By adding the Nebuly analytics platform alongside their traditional user surveys, Iveco Group increased the amount of user feedback by 100x and improved feedback quality.

User surveys are great in some ways, but research also shows that they introduce an element of bias. Nebuly takes bias out of the equation with their unobtrusive and fully automated feedback collection process.

Their approach of capturing the dissatisfaction of chatbot users based on their response to model generations makes so much sense, because it replicates what we humans do every time we talk. With Nebuly, we have data we can trust to prioritize LLM improvements based on real user feedback.”

Pasquale MorraIT Artificial Intelligence / IT AI Platform and Architecture

03. Sharing AI insights in an enterprise setting
With numerous use cases and business stakeholders, Iveco found Nebuly’s reporting feature particularly useful. It allows them to create reports for each use cases and share insights across units.

As strange as it sounds, the business stakeholders driving AI use cases had zero visibility into how these initiatives were going. Every time they had a question, it ended up on our plate in the AI team - and we didn’t have much visibility ourselves. With Nebuly, we were able to easily share relevant insights with all stakeholders, who now have a clear picture of why, and how, their AI use cases are being used in production.

Our stakeholders love it, and it also significantly reduces the reporting burden on the AI team, allowing us to focus on rolling out new initiatives. It’s a win-win.”

Andrea RussottoIT Artificial Intelligence / Generative AI and Prompt Engineering

As Iveco Group anticipates a 40% increase in GenAI projects by mid-2025, Nebuly’s LLM analytics platform will be key in enabling Iveco Group’s teams to continuously adapt AI solutions to real user needs—whether for employees, customers, or other stakeholders.

We’ve been extremely happy with the close collaboration with Nebuly. As the number and complexity of GenAI use cases increase, the need for a solution like Nebuly is clear.”

Fabio RicciatoHead of IT Customer Journey & Digital Architecture, Artificial Intelligence and DEX

Nebuly fosters a culture of continuous learning and curiosity, which aligns with our DNA. This keeps our teams motivated, up-to-date on essential AI skills, and focused on innovation.”

Andrea RussottoIT Artificial Intelligence / Generative AI and Prompt Engineering

A Future of Continuous Innovation.

Ready to get started?
Start exploring now.
Request your trial now and learn more about all the features of the #1 User Experience platform for LLMs.