Overview
Learn how OpenTug partnered with CloudEQS to implement Snowflake and AWS creating a modern, automated data platform with real-time analytics and enhanced security for their enterprise clients.
About OpenTug
OpenTug is a marine logistics platform dedicated to optimizing freight efficiency on inland and coastal waterways by streamlining booking, quoting and tracking. Their products offer a comprehensive suite for cargo and barge management, including rapid quote generation, seamless booking, real-time tracking, and dynamic reporting.
“The transportation metrics and real-time visibility that are standard in other logistics modes, such as trucking or rail, have long been absent in barge transport. OpenTug is changing that, and we are promoting barge as a truly competitive, sustainable shipping solution.” – Mike Baldwin, Co-founder and Chief Operating Officer at OpenTug
When a large customer issued a clear directive for Snowflake as the foundation for their data analytics platform, Mike knew it was time to usher in a new era of enterprise growth. Selecting a partner to to implement a cloud-native data platform that integrated with Amazon Web Services (AWS) and Snowflake within 6-weeks was critical.
Problem: Scaling Analytical Workload and Securing Data Access To Meet Client Requirements
For companies managing a fleet or coordinating shipments, a lack of visibility into their own supply chain means lost opportunities to optimize operations. Whether it’s empty cargo legs, underutilized assets, or unreliable cost estimates, the result is the same – inefficiency, increased emissions, and unnecessary expense.
For OpenTug, their existing Postgres architecture was leading to the following challenges:
- Scaling Analytical Workloads – Postgres handled transactions well but faltered under the analytical workloads and large-scale data demands of enterprise clients.
- Granular Access Controls for Secure Data Share – lacked fine grain access controls required to securely share data subsets across client organizations.
- Manual data extraction and transformation – created bottlenecks, delayed insights, and increased errors.
Solution: CloudEQS + Snowflake + AWS
As Snowflake and AWS experts, the CloudEQS team implemented a robust data pipeline architecture and implemented Snowflake as their next-generation data platform.
Details of the solution include:
- Automated data integration pipeline – The backbone of the solution is an intelligent data integration pipeline built on AWS Lambda. This serverless architecture automatically extracts data every 15-minutes to capture incremental changes. Rather than performing full data loads that could impact system performance, the solution intelligently identifies and processes only new or modified records.
- Role-Based Access Control (RBAC) and Row Level Security– Leveraging their RBAC best practices, the CloudEQS team created granular permissions to align with OpenTug’s organizational structure and service roles were established to enable secure system-to-system communications. Row level security policies ensure users only access data appropriate to their authorization level.
- Snowflake External Listing for Data Share – to facilitate secure data sharing with external partners and customers, CloudEQS leveraged Snowflake’s External Listing capabilities, allowing controlled access to specific datasets without compromising security or requiring complex data duplication.
- External Integration and Data Access – secure integration between AWS and Snowflake through external staging was established to maximize the value of the data platform.
- Medallion Architecture – Leveraging our best practices guide, the CloudEQS team implemented a multi-layered approach in Snowflake to ensure data quality, security, and performance.
- Secure Views – For business users, a MASTER_KPI_DETAILS view was created, a comprehensive dataset that aggregates key performance indicators from multiple sources. Specifically designed to be customer-facing through Sigma, this view has built-in flexibility allowing for customizations based on specific client requirements without affecting the underlying data structure.
Other vendors were either low-cost with limited expertise or firms that wouldn’t prioritize the work. CloudEQS combined technical knowledge with genuine enthusiasm and accessible management. – Mike Baldwin, CTO & Cofounder
Outcomes
The complex Snowflake implementation was delivered within the required six-week timeline, and the OpenTug team are already seeing key advantages:
- Real-time data availability: 15-minute refresh cycles for access to near real-time information.
- Scalable architecture & Future- Proof Platform: Modular, Cloud-native design design automatically scales with business growth.
- Enhanced security: Multi-layered security controls protect sensitive data while enabling appropriate access.
- Operational efficiency: Automated processes reduce manual intervention and potential errors.
- Cost optimization: Incremental loading and serverless computing minimize infrastructure costs.
- Secure Data Share & External Sigma Dashboards: Leveraging Snowflake external data and secure views, OpenTug’s clients have direct access to the raw data within their Snowflake instance and can self-serve key KPIs through their external Sigma Computing Dashboards.
Summary
Working with CloudEQS, OpenTug is ready for sustained growth in a fast-paced market where data platform capabilities have become table stakes for enterprise engagements. The Snowflake implementation transformed their data landscape from a fragmented, manually intensive process into a modern, automated platform that serves as the foundation for data-driven decision making.
Needing to implement Snowflake and Sigma together? Speak to an expert today.
Comments are closed