We at TESCHglobal are extremely excited to help Talend, and their customers, ride the wave of modernizing data management in the cloud! This is an exciting evolution for Talend and we are urging all our customers to consider the benefits.
by Robert Griswold and Jason Buys
This article will describe how a DevOps approach to data management can be implemented with Talend. The term DevOps is used to describe the union of agile development methodology with operations. The term continuous integration (CI) refers to a development practice that allows developers to integrate code into a shared repository via an on-demand or scheduled basis. The functional steps of CI are building source code, testing pre-defined unit tests, and eventually deploying the code to an artifact repository. This automated process not only allows for more regular software releases, but also enables the detection of errors introduced by new code, and prohibits that new code from making it to releases.
Below are the environment requirements and Jenkins functional walkthrough of steps that are needed to create this CI process:
To set up the CI environment the following are needed:
- Jenkins server configured with the JDK, Maven and GIT Plugins
- A second Talend CommandLine (not already dedicated to TAC)
- The Talend CI Builder plugin installed in Nexus or local Maven repository
- Access for GIT/SVN (containing Talend jobs) and Nexus (libraries and built artifacts)
While it may appear daunting, the process outlined below is something that has been implemented with customers very quickly; provided the needed environment is available, and test jobs are created the process is quite simple.
The following steps are a walkthrough of the steps of the CI process as it is implemented with Talend jobs.
Trigger for Jenkins CI Project
Step 1 Talend Studio - Code Commit
- The process is started when a studio user commits code to the GIT/SVN repository.
- The “code” generated by Talend studio consists of items and properties XML files.
- The act of committing the code can be one way to trigger the Jenkins workflow
Jenkins Step to Generate and Compile Sources
Step 2 Check Out Sources
- Jenkins checks out the XML files from the repository
- Jenkins also checks out any custom Java or Routines from the repository
Step 3 Generate Sources
- The Talend CommandLine service generates the Java code from the XML files
Step 4 Compile Sources
- The generated java source code is now compiled as directed by a Maven POM file
Jenkins Step to Run Unit Tests
Step 5 Run Tests
- This step will run any unit tests created in the Talend studio by the developer
- These unit tests can be created by right clicking on a component in the studio and providing expected input and output criteria for the selected group of components
Jenkins Job to Package and Publish
Step 6 Package and Publish
- This step will create a zip file with scripts, contexts, JVM parameters, and java libraries
- This zip archive will then be published to a Nexus server artifact repository
- This resulting artifact’s version need not be built again as it can be retrieved from the Nexus repository by whichever environment it is to be deployed in.
Deployment to Job Servers
The Talend lifecycle of CI as it is officially documented requires that an TAC administrator retrieve the jobs from Nexus via the Job conductor, and manually deploy them to be run. This is the only area in which the automated fluidity of the CI/CD process requires human interaction. There is, however, a way to use a Talend native function to also automate the deployment.
Step 7 Meta Servlet Deployment
- The Talend metaservlet can be used to deploy and schedule jobs from Nexus to a Talend job server. The Talend metaservlet is a REST interface used to accomplish many of the same tasks which can be done in the Talend Administration Console (TAC).
- Rather than writing a script, an actual Talend job can be used to parameterize and build the JSON needed to execute the MetaServlet REST calls.
- Example of REST Parameters sent to the Talend metaservlet
Step 7 Manual Deployment in TAC
- Use the Talend Administration Console’s Job conductor to deploy and schedule the jobs from Nexus to the Talend job servers
Why tackle Continuous Integration? It can seem complex and the benefits might not be immediately apparent. After wrestling with the evolving Talend documentation and truly learning what each step in the process does, it becomes a very repeatable process to help others with. The risks are simply the time spent setting up the Jenkins server and corresponding technologies. Once CI is setup and test driven development is in place it will probably be difficult to live without. It eliminates a lot of human touch and puts a great deal of rigor in agile DevOps processes.
As a team of software engineers and developers we dive in head-first into new technologies, and for the better part of 2 years we have been architecting and building IoT solutions! Through our projects, research, and partnerships we’ve been paddling hard to catch this wave. Here is an overview approach to IoT via a reference architecture:
leveraging best practices throughout the data management life cycle. We will take into account: the sources, processing, storage capabilities, constraints and end-user needs.
Whatever your data mission statement is “One Click”, “360 View”, “Right Data at the Right place and Time” etc…. there is no clear right or wrong answer. Let TESCHGlobal help you get the most value and insight from your data, resources and capabilities. We can help with full life cycle using industry-top data architects and scientists.
HealthLX, Inc. Submits Entry To The HL7 C-CDA Rendering Tool Challenge
Nimble healthcare data integration solutions firm shows off skills by creating a C-CDA viewer prototype.
Grafton, WI - June 2016: In an effort to exhibit their healthcare data management proficiencies, the talented team at HealthLX accepted and submitted their entry in the Health Level Seven (HL7) and Office of the National Coordinator for Health Information Technology (ONC) rendering tool challenge on May 31, 2016.
The challenge was created to inspire the development of HL7 tools, specifically the development of a Consolidated Clinical Document Architecture (C-CDA) rendering tool that makes the data exchange between healthcare providers and patients humanly legible.
According to HL7, the industry is calling for enhanced human readability and relevance as clinicians are frustrated with the usability of current C-CDA documents. Today, an overabundance of data is displayed by EHR systems and providers struggle to page and sort through all of the data to find the essential and relevant clinical information they need to make decisions. Development of an easy-to-use viewer with a modern user-interface provides clinicians the ability to save time and insure they have relevant patient information to make correct decisions.
The team at HealthLX aspired to make the tool easy-to-use and flexible to give a broad range of clinical users the ability to view relevant patient information quickly and clearly based on their specialty; all in a slick and compatible user-interface.
Will Tesch, founder of of HealthLX, praises his team’s ability to quickly step-up to the challenge and deliver a flexible, well-designed solution that elegantly delivers the desired usability and user experience demanded by this challenge.
Will added, “C-CDA is a foundational document for patients care history that once digitally available and shareable with doctors or care managers can play a significant role in ensuring decisions are made quickly and accurately by these providers. This benefits everyone; consumers, providers, and payers, and contributes to the “Triple Aim” of healthcare. We look forward to continually displaying our ability to rapidly design and develop healthcare data interoperability to the healthcare community in every way we can.”
To take a look at HealthLX’s C-CDA rendering tool submission click HERE .
About Health Level Seven
Founded in 1987, Health Level Seven International (HL7) is a not-for-profit, ANSI-accredited standards developing organization dedicated to providing a comprehensive framework and related standards for the exchange, integration, sharing, and retrieval of electronic health information that supports clinical practice and the management, delivery and evaluation of health services. Learn more at www.hl7.org
About HealthLX, Inc.
HealthLX, offers a full integration solution set designed for Healthcare software solution firms to bridge the gap of interoperability and innovation, necessary to enable them to quickly implement their solutions in diverse environments. HealthLX’s solution-set includes open source integration software that creates and manages connections and data-flows, integration consulting and connectivity services, and solution management. For Healthcare software solution firms that want to make integration a differentiator, visit www.healthlx.com
TESCHGLOBAL FORMS STRATEGIC PARTNERSHIP WITH FOUNDRY HEALTH
Exceeding the Demands of Healthcare Innovation
Milwaukee, WI, May 29, 2015 (PRWeb) -- TESCHGlobal LLC, a global technology professional services firm, has recently established a strategic partnership agreement with Foundry Health LLC, a digital health company that seeks to empower health professionals by building software tools that put the patient back at the center of each encounter. This strategic partnership will assist both TESCHGlobal and Foundry Health in the growth and development of healthcare innovation in the U.S., while also supporting the global growth initiatives of both firms.
"As Foundry Health continues to expand its ClinSpark™ software offering to an ever expanding area of the world, we are delighted to be partnering with TESCHGlobal. The TESCHGlobal team will allow us to scale up to meet the growing demands for the ClinSpark™ system," stated Brock Heinz, CEO at Foundry Health. “We are excited about our new partnership with Foundry Health,” says Will Tesch, President and CEO of TESCHGlobal. "Foundry Health has an innovative team and is led by a true engineering genius. Our support of Foundry Health and their implementation and support needs to scale is aligned closely with our past success with other Healthcare organizations like Foundry Health. This partnership will enable ClinSpark™ to meet its growing presence globally while also focusing on their product development.”
About Foundry Health
Foundry Health is a digital health company focused on building software tools that deliver creative strategic solutions. As the creators of ClinSpark™, the world’s first CDISC ODM certified phase 1 eSource system, Foundry Health is driven to empower health professionals and build successful custom solutions that exceed customer needs. For more information, visit www.foundryhealth.com.