Case Study

Transforming Vehicle Telemetry through Big Data Platform

Boosting Operational Efficiency and Cost Reduction in Automotive Industry
1-Aug-09-2024-11-05-49-2053-AM
%

Reduction in Total Cost of Ownership (TCO)

-TB / Day

Migration of Data Ingestion and Processing

+ 

Table Segregation of Data

Background

The automotive industry increasingly leverages big data and advanced data analytics to boost vehicle performance, safety, and autonomous driving capabilities. Modern vehicles generate extensive telemetry data, which helps in predictive maintenance, design optimisation,and improved customer experiences. AI and machine learning in telematics enable real-time decision-making and intelligent vehicle systems.

A leading US-based Tier 1 automotive major aimed to leverage  these trends to manage and analyse vast amounts of vehicle telemetry data generated daily. They envisioned to develop a scalable, distributed big data platform to process, ingest, and store large-scale data in both offline and real-time environments.

Challenge

The client faced significant challenges in developing a robust big data platform capable of processing and analysing 50 TB of daily data from various sensors such as LiDAR, Radar, CAN signals, IMU, and video. Key issues included managing real-time data processing and efficient querying for analytics and decision-making, converting CAN data from .mat files to AVRO and CSV while maintaining data integrity, designing schemas for over 90 tables with around 50 columns each for seamless data integration, and supporting high-concurrency dashboard queries with advanced optimisation to prevent performance bottlenecks.

Solution

​To address these challenges, Tata Elxsi developed a big data platform, scalable Hadoop solution on the Hortonworks data platform to process, store, and analyse large-scale data supporting the autonomous vehicle validation program. ​Key features included ingesting vehicle CAN, sensor, and GPS data in AVRO, SQL, and Hadoop formats. The team designed a robust database schema, storing data in a Hive warehouse, and implemented partitioning and bucketing in Hive tables for faster retrieval. Data was stored in .orc format to optimise space utiliz\sation, while Apache Arrow was used for in-memory processing, and Spark for distributed computing. Presto handled concurrent dashboard queries efficiently, with data visualisation using Apache Zeppelin.​

3-Aug-09-2024-10-30-57-0805-AM
4-Aug-09-2024-10-31-05-7354-AM
2-Aug-09-2024-10-30-47-9818-AM

Impact

​Tata Elxsi’s innovative solution has made a significant impact on the client by automating the entire software development lifecycle through a robust big data platform. This automation led to the creation of comprehensive project documentation, optimisation of resource management, and the detailing of precise hardware and software specifications. The development process was made more efficient with the automated creation of drivers, modules, and system integration, streamlining the workflow. The testing phase was also enhanced with the automated generation of test plans and cases, ensuring a thorough quality assurance process. The key highlights of this solution include a 40% reduction in Total Cost of Ownership (TCO), reduced job execution time, and faster response time. These advancements underscore Tata Elxsi’s commitment to leveraging technology to drive efficiency and precision in software development.​

Services rendered

Tata Elxsi

  • Big Data Platform set-up
  • Data Migration and Integration
  • Schema Design and Data Warehousing
  • Partitioning and Bucketing Implementation
  • In-memory and Distributed Data Processing
  • Distributed Query Engine Optimisation
  • Data Visualisation​

Contact us to know more →