In recent years, “The Cloud” has become a rising approach for effectively storing, retrieving, and managing data. But, the thought of migrating data from traditional on-prem means to the cloud may seem too daunting or not feasible for some organizations given the decisions and changes that need to take place. What organizations sometimes don’t realize is the benefits far outweigh the reasons to remain as is. By moving to the cloud, enterprises can achieve cost efficiency, have greater agility and scalability, maintain security, and even gain a competitive edge.
If you are an organization or know of an organization interested in making the switch through the use of Talend, follow this Q&A between VP of Product Delivery Josh Dodge and Data Management Architect Zeeshan Mohammed. In it, they’ll share the differences between on-prem and cloud, the level of effort organizations need to make the switch, and how to alleviate risk.
JD - Q1: What is “Talend On-Prem”?
ZM - A1: On-prem software is installed on, and runs from, systems within the company’s building, corporate complex, or privately controlled data center. When it comes to Talend, depending on the product(s) you’ve licensed, this may mean you need to install and maintain multiple databases, artifact repositories, execution servers, and web service portals.
The following graphic shows a typical Talend on-prem architecture:
Security: security is implemented by the user so you can configure it end-to-end to meet your security policies
Control: the user has full control of the on-prem infrastructure. If a query is taking too long to execute, you fully control the fix.
Regulation: if the business is within a highly regulated vertical (healthcare, finance, etc.), regulations may not allow the data to be stored in a public data center. Staying on-prem may simply be mandatory.
Trust: while difficult to measure, we cannot under-emphasize the importance of trust. We’ve seen many examples where the IT professionals responsible for the security and agility of data have more trust in a system they built on their own from the ground up than they will in a system provide by a cloud provider.
Cost: in today’s world of cheap storage and bottom-dollar compute time, on-prem models are becoming less and less cost-effective. It will likely take thousands of dollars to stand up a solution on-prem that could be acquired from a cloud provider for just a few dollars a month through a subscription.
Human Resources: significant human resource overhead is required to stand up, and more importantly maintain an on-prem solution. DBAs, front-end developers, Hadoop experts, network administrators, 24/7/365 - the human resource demands can be enormous.
Q2: What is “Talend Cloud”?
A2: When it comes to the Talend Cloud (TC), the management and maintenance of the system’s infrastructure is abstracted from the user. This abstraction is done to alleviate the user from the human resource and financial costs of managing the solution on-prem. Talend offers several types of licensing models for their cloud offering including a SaaS edition (cloud-to-cloud integration) and Hybrid edition (cloud-to-cloud and ground-to-cloud integration), and your data is never stored in the Talend Cloud - only metadata (Jobscript) is stored remotely in order for job orchestration to operate successfully.
- Cost: no hardware costs means lower costs overall. And you only have to pay for what you use (storage and compute) when working with most of today’s cloud providers, including Talend.
Maintenance: the Talend Cloud is maintained by Talend, not by you. This eliminates downtime, weekend maintenance shifts, and frazzled employees. Patches are made available so your system is always up-to-date.
Scalability: If you need more cloud or remote engines, or licensed developers accessing the system because your business is growing, it’s as simple as ordering more from Talend and with a few clicks you’ve scaled out. In the on-prem world, this scaling could take months to accomplish.
Configurability: you do not have all the configuration options of the on-prem solution so highly tuned jobs that performed well on-prem may not perform as well in the cloud without modification.
Security: while it is SSAE16 SOC 2 Type II certified and frequently audited, the abstraction inherent in the Talend Cloud may still be a security concern to highly regulated industries.
Q3: What does a Talend Cloud architecture for data integration look like?
A3: The graphics below show a Cloud-to-Cloud and Hybrid Talend Cloud architecture. In both designs, the cloud sources (i.e., SalesForce or an RDS like Amazon Relational Database Service) connect to either the Talend Cloud Engine, or the Talend Remote Engine, for job processing. Which type of engine you’ll use will depend on whether your data must be processed locally or in a private cloud behind your firewall in which case you’ll use a Remote Engine. In both cases, the Talend Studio runs in your own private Talend Cloud.
Q4: Is the Talend Cloud a secure option for storing sensitive/priority data?
A4: Talend has publicly available documentation showing that the Talend Cloud is highly secure. “Talend Integration Cloud uses a SSAE 16 SOC 2 Type I certified data center, a SSAE 16 SOC 2 Type II certified application, and maintains security and data protection in the cloud according to Amazon Web Services cloud compliance.“
Q5: What is the level of effort to move from Talend On-Prem to Talend Cloud?
A5: The answer to this question really depends on the complexity of what already exists in your Talend on-prem solution. That said, Talend does provide an application for moving your jobs, projects, and users from on-prem to the cloud so it’s a fairly small level of effort to make this change.
Q6: What risks are there in a move like this?
A6: While cloud technologies have evolved tremendously in the last decade, eliminating many of the risks we once faced, it’s difficult to gauge your risks without going through a discovery process with a reputable, trusted advisor. Connect with TESCHGlobal so we can explore your options with you.
Q7: For organizations focused on decreasing hardware costs and increasing developer efficiency this year, are there financial and functional benefits by moving to the Talend Cloud?
A7: Definitely! Talend manages the Talend Cloud, your team doesn’t; this alone eliminates a large swath of your hardware costs. And when it comes to developer efficiency and quality of life, your developers won’t have to focus on all those pesky issues with the infrastructure... instead they can focus on development… building your next great thing or fixing that persistent problem. The intrinsic value found in this move cannot be under-emphasized!
Q8: What would make a good proof of concept (POC) for evaluating whether adopting the Talend Cloud is a good change for organizations?
A8: We recommend that you rebuild a legacy process with the new Talend Cloud, then analyze the results that stem from the legacy system compared to the new one. If the results (throughput, scalability, etc.) are positive, then create a roadmap that shows how you can improve your processes over time, systematically and quantifiably, through the Talend Cloud. TESCHGlobal has years of experience standing up these POCs, assisting companies in this journey. We’re here to help.
Q9: Has TG helped customers adopt the Talend Cloud? If so, what are some key takeaways that can be shared?
A9: Yes! DMD Marketing, a pioneer in healthcare digital communications and connectivity, needed a scalable integration platform to support its substantial growth and data management demands. Their data needed to be staged, cleansed, processed and published quickly so DMD clients have access to credible information on-demand. TG partnered with DMD to understand first and foremost their business requirements. Then an implementation roadmap was developed with crucial deliverables including, but not limited to, the configuration of the entire TC platform with a migration strategy to convert legacy stored procedure processes with Talend’s modern workflows and integration jobs. Through the migration, it has allowed them to achieve data agility and decreased development turnaround times. In addition, they have realized decreased R&D costs due to a subscription-based model with AWS; decreased dependency on infrastructure management; and extracted the management of hardware and software allowing the focus to be on development efforts.
Q10: If organizations are interested in this change and want to get started, what should be their next step?
A10: Contact TG, we love complex problems. We’ll provide a discovery process that helps you understand your requirements and help build out a POC, if appropriate. If the POC meets or exceeds your requirements, we’ll help you lift-and-shift from on-prem to the cloud, and we’ll help you measure the value you’re deriving from the change so it continues to make sense.
By migrating to the cloud, you’ll be more efficient, save dollars, and take another step towards achieving a 360-degree view of your data. At TG, we guide customers through their data journey to help them realize their potential through modernization of their current data management processes.