Kinetica, one of HEXstream’s partners, is an advanced GPU-based database which enables advanced filtering, aggregation, and visualization capabilities. It is a columnar, memory-first database that is optimized for both CPU and GPU processing.  It is useful for streaming analytics, artificial intelligence, machine learning, geospatial analytics, and many other functions.  For the last six months, my team and I have worked with a large retail client to help them decrease the response times of their customer-facing applications.  Building these applications using cutting-edge technologies like Spark, Kafka, and Java Spring Boot has increased customer satisfaction by making them much faster and more responsive.

We are supporting critical customer-facing applications that handle more than 100 million read requests and ingest more than 100 million records every day.  During these first six months, our team has built a couple of applications for the client which save time spent processing and aggregating using the Kinetica database’s powerful GPU-accelerated capabilities.  One of the applications supports the real-time ingestion of records from Kafka into Kinetica and performs complex queries, including group by and aggregations, and provides the response to the client in microseconds to milliseconds. We have also worked on an extension project to update half a million records every hour and provide the response to the downstream applications in a second.

Some highlights from our current engagement:

• One of our newly built applications programmatically loads data into the Kinetica database, distributed across all the ranks and capability to read records directly from respective ranks (shards).

• There had been an ingestion application that ran once a day, taking more than three hours to complete the job. We helped the client to reduce the time taken by the job by 90%, which in turn saved 160 minutes of resource utilization and time. Now, it takes only 20 minutes to complete data ingestion instead of three hours, and our client is very happy with the work we have done.

As an application designer and developer, I love the fact that Kinetica’s product development team is always available to me and my team for collaboration and innovation. We have direct access to their entire development team, including their CTO who, despite his very busy schedule, always makes himself available to listen to our ideas and clients’ feature requests.

When we face a challenge, we all group together and find a solution as a team. Case in point, we recently had a use case where the client was having issues understanding database response times. We quickly decided to create an application to ingest real-time database log data after parsing them and storing them in the Kinetica database.  We then created charts and dashboards to monitor database logs in a real time. We completed this application, deployed it across the entire cluster, and started monitoring them, all in less than a week. This helped us to capture database response times which were always in the range of microseconds to milliseconds and built trust between client and customer applications.

Kinetica is a great partner to HEXstream! We love using Kinetica’s robust and nimble platform to serve our clients’ real-time analytics needs. Together we are helping our clients handle voluminous and dynamic data and build innovative solutions that were not possible before. We at HEXstream look forward to continue working with Kinetica to help our clients for many years to come.