Why implement CI/CD, and what approach do I take for my Talend environment?
Updated: Feb 10
Authored by: Jason Buys
Automating the software development lifecycle (SDLC) is a step many organizations take in their journey to digital transformation. Automation translates into the agility to bring enhancements and innovations to market faster and more frequently, and to pivot when customer preferences change. And CI/CD is the term used to facilitate automation in the world of software. However, while a lot of companies aspire to do it, not everyone agrees on what precisely CI/CD entails. We’ll explain some of the buzzwords you’re hearing around CI/CD, the requirements for implementing it and will share a few insights about CI/CD with your Talend solution.
DEFINING CI/CD, DEVOPS AND AGILE METHODOLOGY
Though developers will talk about CI/CD in terms of Constant Irritation/Continual Disappointment with regard to getting it working, conventionally it represents Continuous Integration/Continuous Deployment. The goal of CI/CD is to reduce the number of people who interact with SDLC technologies so code can be changed more rapidly and with fewer errors. DevOps and Agile methodology are related concepts.
Continuous Integration (CI) is the practice of merging code changes regularly and applying automated tests. Rather than testing manually to see that code changes carry out their purpose correctly, in CI, pre-defined checks assure the organization that the code changes are working as planned.
Continuous Deployment (CD) is the frequent, automatic promotion of code to higher environments after CI and successful tests. CD is based on the idea that if code changes pass the test, they should start being used immediately. CD can also refer to Continuous Delivery, a term that means code changes are readied for on-demand deployment.
DevOps is the change in business culture and IT needed to create more responsiveness. It emphasizes cross-training, collaboration and communication, and eliminates traditional business silos. While CI/CD provides the tools and process to automate the SDLC, DevOps is the change in people that can lead to getting the most value from CI/CD.
Agile methodology focuses on process changes that accelerate delivery and reduce re-work. It’s about both people and process - removing process barriers and enabling collaboration using daily standups, and shorter cycles of development. Agile methodology is the common alternative to the “waterfall” style that represents a linear process for code development and a more siloed work style.
REQUIREMENTS FOR IMPLEMENTING CI/CD
CI/CD removes an artificial layer of human intervention in the SDLC that, in the past, required multiple teams. Developers would write the code, Quality Assurance would test it manually, and Systems/Ops would move the code to production. Implementing CI/CD takes a shift in people, process and technology:
People – One of the biggest shifts with CI/CD is the way developers work. With CI/CD, code requirements are generated by developers working closing with business analysts too. Together, they’re thinking more fully about the requirements for code change in total. Developers also write the testing requirements that are the basis of testing automation.
Process – Automated testing and automated code promotion all become part of the SDLC process. Writing the testing happens much earlier in the SDLC and is much more collaborative. Automated testing eliminates the need for QA to do manual tests.
Technology – Organizations need testing software that is specific to the language they’re using and the application they’re developing. (The testing software that’s right for website code is very different than code for data analytics, for example.) They also need a generic framework like Jenkins that enables plug-in connections to different platforms so developers can build their unique CD pipeline. Finally, they need source control with a platform like GIT for version storage.
WHY SHIFT TO CI/CD?
While there can be challenges in changing people, process and technology, the capabilities made possible by CI/CD are vital for digital transformation. The major advantages are frequency and reliability. Organizations gain the freedom to add new features and make changes faster without waiting for large releases. There’s also peace of mind. With CI/CD, it’s easier to identify and correct defects earlier, and with properly written testing, getting the green light means fear of code failure is replaced with trust. As long as the tests have integrity, code can be changed and deployed quickly with confidence.
THE SECRET TO TALEND CI/CD
Organizations that use Talend can absolutely make the shift to CI/CD. Because Talend is a code-generating tool, Talend CI/CD is different than traditional CI/CD. Recognizing that, and playing by Talend’s rules, is the key.
Talend adds a “generate” step which can be time-consuming and impacts CI/CD in two ways. With Talend’s parent-child job design, developers have to be very specific in order to run child jobs alone. Inserting a brief, manual step and using a naming convention are two of the ways to handle it.
Second, with Talend there’s the inability to use pure GIT pull requests because the code stored in GIT is not the runnable code which is tested. Developers need to recognize that they can only modify through the lens of the Talend platform.
GETTING STARTED WITH CI/CD
Once you’ve made the decision to shift to CI/CD, you’ll need to put plans in place for people and process change. From the technology aspect, some of the first steps include selecting the most applicable testing suite and defining the right CD pipeline for your organization, which drives the development of a GIT workflow strategy including version maintenance and a release schedule.
Deciding what to automate first is often a significant challenge. If you’d like to explore your organization’s most strategic approach to CI/CD with your Talend solution, TG can help. Reach out to our specialized team and we’ll start the conversation today.