Future-Proofing Data Processing

Enabling parallel processing and AI integration with Google Cloud.

The Customer

Spectricity is a fabless company that develops and manufactures spectral sensing solutions using CMOS technologies. Their solutions are designed for high-volume production and use in consumer and mobile devices. The company was incorporated in 2018, and is backed by venture capital. Its work builds on an exclusively-licensed patent portfolio from imec, a world-leading R&D and innovation hub in nano-electronics and digital technologies.

The Problem

Spectricity’s existing solutions were built with traditional data science tools on local devices, which limited their ability to fully realize the potential of their spectral sensing solutions. The large files they worked with could not be efficiently processed, and the main challenge was enabling parallel preprocessing of data in a way that integrated seamlessly with the rest of the pipeline.

The Goal

To design a future-proof framework that would allow Spectricity to build and deploy scalable, efficient ML solutions capable of handling large datasets, while improving the flexibility and efficiency of their data preprocessing processes.

Google Technologies used

AI Platform

Cloud Dataflow

Cloud Storage

The Solution

Tensr worked closely with Spectricity to capture their requirements through workshops and design a robust, scalable framework for ML solutions. The solution involved migrating data preprocessing to Apache Beam running on Google’s Dataflow service, enabling efficient processing of large HDF5 files. The files were stored securely in Google’s GCS data lake, while data for analysis and dashboarding was stored in a serverless DWH (BigQuery). This solution ensured parallel processing and improved data handling for AI models.

Customer gain

Spectricity’s pipelines evolved from small-scale, local ML processes to robust, distributed systems. These new pipelines are now scalable, easy to retrain, monitor, and manage, making them suitable for industrial-scale applications and allowing Spectricity to fully leverage the potential of their technology.