Deployment Resources

DZone's Featured Deployment Resources

How To Use KubeDB and Postgres Sidecar for Database Integrations in Kubernetes

By Viacheslav Aksenov

In contemporary web development, a recurring challenge revolves around harmonizing the convenience and simplicity of using a database with a web application. My name is Viacheslav Aksenov, and in this article, I aim to explore several of the most popular approaches for integrating databases and web applications within the Kubernetes ecosystem. These examples are examined within the context of a testing environment, where constraints are more relaxed. However, these practices can serve as a foundation applicable to production environments as well. One Service, One Database. Why? Running a database alongside a microservice aligns with the principles outlined in the Twelve-Factor App methodology. One key factor is "Backing Services" (Factor III), which suggests treating databases, message queues, and other services as attached resources to be attached or detached seamlessly. By co-locating the database with the microservice, we adhere to the principle of having a single codebase that includes the application and its dependencies, making it easier to manage, scale, and deploy. Additionally, it promotes encapsulation and modularity, allowing the microservice to be self-contained and portable across different environments, following the principles of the Twelve-Factor App. This approach enhances the maintainability and scalability of the entire application architecture. For this task, you can leverage various tools, and one example is using KubeDB. What Is KubeDB? KubeDB is an open-source project that provides a database management framework for Kubernetes, an open-source container orchestration platform. KubeDB simplifies the deployment, management, and scaling of various database systems within Kubernetes clusters. We used the following benefits from using this tool: Database operators: Postgres operator to simplify the process of deploying and managing database instances on Kubernetes. Monitoring and alerts: KubeDB integrates with monitoring and alerting tools like Prometheus and Grafana, enabling you to keep an eye on the health and performance of your database instances. Security: KubeDB helps you set up secure access to your databases using authentication mechanisms and secrets management. And it is very easy to set up the deployment. deployment.yaml: YAML apiVersion: kubedb.com/v1alpha2 kind: PostgreSQL metadata: name: your-postgresql spec: version: "11" storageType: Durable storage: storageClassName: <YOUR_STORAGE_CLASS> accessModes: - ReadWriteOnce resources: requests: storage: 1Gi terminationPolicy: WipeOut databaseSecret: secretName: your-postgresql-secret databaseURLFromSecret: true replicas: 1 users: - name: <YOUR_DB_USER> passwordSecret: secretName: your-postgresql-secret passwordKey: password databaseName: <YOUR_DB_NAME> Then, you can use the credentials and properties of this database to connect your service's pod to it with deployment.yaml: YAML apiVersion: apps/v1 kind: Deployment metadata: name: your-microservice spec: replicas: 1 selector: matchLabels: app: your-microservice template: metadata: labels: app: your-microservice spec: containers: - name: your-microservice-container image: your-microservice-image:tag ports: - containerPort: 80 env: - name: DATABASE_URL value: "postgres://<YOUR_DB_USER>:<YOUR_DB_PASSWORD>@<YOUR_DB_HOST>:<YOUR_DB_PORT>/<YOUR_DB_NAME>" --- apiVersion: v1 kind: Service metadata: name: your-microservice-service spec: selector: app: your-microservice ports: - protocol: TCP port: 80 targetPort: 80 And if, for some reason, you are not ready to use KubeDB or don't require the full functional of their product, you can use the Postgresql container as a sidecar for your test environment. Postgres Container as a Sidecar In the context of Kubernetes and databases like PostgreSQL, a sidecar is a separate container that runs alongside the main application container within a pod. The sidecar pattern is commonly used to enhance or extend the functionality of the main application container without directly impacting its core logic. Let's see the example of a configuration for a small Spring Boot Kotlin service that handles cat names. deployment.yaml: YAML apiVersion: apps/v1 kind: Deployment metadata: name: cat-svc labels: app: cat-svc spec: replicas: 1 selector: matchLabels: app: cat-svc template: metadata: labels: app: cat-svc type: http spec: containers: - name: cat-svc image: cat-svc:0.0.1 ports: - name: http containerPort: 8080 protocol: TCP readinessProbe: httpGet: path: /actuator/health port: 8080 initialDelaySeconds: 30 timeoutSeconds: 10 periodSeconds: 10 livenessProbe: httpGet: path: /actuator/health port: 8080 initialDelaySeconds: 60 timeoutSeconds: 10 periodSeconds: 30 env: - name: PLACES_DATABASE value: localhost:5432/cats - name: POSTGRES_USER value: pwd - name: POSTGRES_PASSWORD value: postgres - name: cat-postgres image: postgres:11.1 ports: - name: http containerPort: 5432 protocol: TCP env: - name: POSTGRES_USER value: pwd - name: POSTGRES_PASSWORD value: postgres - name: POSTGRES_DB value: cats Dockerfile FROM gradle:8.3.0-jdk17 COPY . . EXPOSE 8080 CMD ["gradle", "bootRun"] And for local run, it is possible to use docker-compose with the following configuration. docker-compose.yaml: YAML version: '3.8' services: cat-postgres: image: postgres:12.13 restart: always ports: - "5432:5432" environment: POSTGRES_PASSWORD: postgres POSTGRES_USER: postgres POSTGRES_DB: cats # volumes: # - ./init.sql:/docker-entrypoint-initdb.d/create_tables.sql - if you want to run any script before an app # - ./db-data/:/var/lib/postgresql/data/ service: image: cat-svc:0.0.1 restart: always ports: - '8080:8080' environment: SPRING_PROFILES_ACTIVE: prod PLACES_DATABASE: cat-postgres:5432/cats POSTGRES_PASSWORD: postgres POSTGRES_USER: postgres Migrations The big thing that has to be decided before using this approach is the migration question. The best option in this approach is to delegate the migration process to any tool that can work within your app infrastructure. For example, for Java World, you could use Flyway or Liquibase. Flyway is a popular open-source database migration tool. It allows you to version control your database schema and apply changes in a structured manner. Flyway supports multiple databases, including PostgreSQL, MySQL, and Oracle. Liquibase is an open-source database migration tool that supports tracking, managing, and applying database changes. It provides a way to define database changes using XML, YAML, or SQL, and it supports various databases. Pros of Using a PostgreSQL Sidecar in Kubernetes Separation of concerns: Sidecars allow you to separate specific functionalities (e.g., database migrations, backups) from the main application logic. Сompliance with microservice architecture. Simplified deployment: Sidecars can be deployed and managed alongside the main application using the same deployment configurations, simplifying the overall deployment process. You don't need to support separated database for testing the environment. And it leads to decreasing the complexity of tests (you don't need to think about collisions while you are running many CI with tests for the same table) Cons of Using a PostgreSQL Sidecar in Kubernetes Resource overhead: Running additional containers consumes resources (CPU, memory) on the node, which may impact the overall performance and resource utilization of the Kubernetes cluster. It's best to use as few resources as possible. Startup order: The main application may become dependent on the sidecar for certain functionalities, potentially leading to issues if there are discrepancies or version mismatches between the main application and the sidecar. Arranging containers in a specific order without additional configuration can be somewhat challenging. However, this shouldn't pose a problem in test environments due to the quick startup of the PostgreSQL container. In most scenarios, the PostgreSQL container will initiate before any of your business applications. Even if the application attempts to run before PostgreSQL is ready, it will encounter a failure and be automatically restarted by the default Kubernetes mechanism until the database becomes available. Learning curve: Adopting the sidecar pattern may require a learning curve for development and operations teams, particularly if they are new to the concept of containerized sidecar architectures. Once the setup is complete, new team members should encounter no issues with this approach. Conclusion In conclusion, the choice between using KubDB and the PostgreSQL sidecar approach for integrating web applications and databases in a test environment ultimately depends on your specific requirements and preferences. KubDB offers a comprehensive solution with Kubernetes-native features, streamlining the management of databases alongside web services. On the other hand, the PostgreSQL sidecar approach provides flexibility and fine-grained control over how databases and web applications interact. Whether you opt for the simplicity and seamless integration provided by KubDB or the customization potential inherent in the sidecar pattern, both approaches lay a solid foundation for test environments. The key lies in understanding the unique demands of your project and selecting the method that aligns best with your development workflow, scalability needs, and overall application architecture. Whichever path you choose, the insights gained from exploring these approaches in a test setting can pave the way for a robust and efficient integration strategy in your production environment. More

Deployment of Spring MVC App on a Local Tomcat Server

By Sohail Shaikh

This guide delves into the meticulous steps of deploying a Spring MVC application on a local Tomcat server. This hands-on tutorial is designed to equip you with the skills essential for seamless deployment within your development environment. Follow along to enhance your proficiency in deploying robust and reliable Spring MVC apps, ensuring a smooth transition from development to production. Introduction In the preliminary stages, it's crucial to recognize the pivotal role of deploying a Spring MVC application on a local Tomcat server. This initial step holds immense significance as it grants developers the opportunity to rigorously test their applications within an environment closely mirroring the production setup. The emphasis on local deployment sets the stage for a seamless transition, ensuring that the application, when deemed ready for release, aligns effortlessly with the intricacies of the production environment. This strategic approach enhances reliability and mitigates potential challenges in the later stages of the development life cycle. Prerequisites To get started, ensure you have the necessary tools and software installed: Spring MVC Project: A well-structured Spring MVC project. Tomcat Server: Download and install Apache Tomcat, the popular servlet container. Integrated Development Environment (IDE): Use your preferred IDE (Eclipse, IntelliJ, etc.) for efficient development. Configuring the Spring MVC App Initiating the deployment process entails meticulous configuration of your Spring MVC app development project. Navigate to your project within the Integrated Development Environment (IDE) and focus on pivotal files such as `web.xml` and `dispatcher-servlet.xml` These files house crucial configurations that dictate the behavior of your Spring MVC application. Pay meticulous attention to details like servlet mappings and context configurations within these files. This configuration step is foundational, as it establishes the groundwork for the application's interaction with the servlet container, paving the way for a well-orchestrated deployment on the local Tomcat server. 1. Create the Spring Configuration Class In a typical Spring MVC application, you create a Java configuration class to define the application's beans and configuration settings. Let's call this class 'AppConfig'. Java import org.springframework.context.annotation.ComponentScan; import org.springframework.context.annotation.Configuration; import org.springframework.web.servlet.config.annotation.EnableWebMvc; @Configuration @EnableWebMvc @ComponentScan(basePackages = "com.example.controller") // Replace with your actual controller package public class AppConfig { // Additional configurations or bean definitions can go here } Explanation '@Configuration': Marks the class as a configuration class. '@EnableWebMvc': Enables Spring MVC features. '@ComponentScan': Scans for Spring components (like controllers) in the specified package. 2. Create the DispatcherServlet Configuration Create a class that extends 'AbstractAnnotationConfigDispatcherServletInitializer' to configure the DispatcherServlet. Java import org.springframework.web.servlet.support.AbstractAnnotationConfigDispatcherServletInitializer; public class MyWebAppInitializer extends AbstractAnnotationConfigDispatcherServletInitializer { @Override protected Class<?>[] getRootConfigClasses() { return null; // No root configuration for this example } @Override protected Class<?>[] getServletConfigClasses() { return new Class[]{AppConfig.class}; // Specify your configuration class } @Override protected String[] getServletMappings() { return new String[]{"/"}; } } Explanation 'getServletConfigClasses()' : Specifies the configuration class (in this case, AppConfig) for the DispatcherServlet. 'getServletMappings()' : Maps the DispatcherServlet to the root URL ("/"). Now, you've configured the basic setup for a Spring MVC application. This includes setting up component scanning, enabling MVC features, and configuring the DispatcherServlet. Adjust the package names and additional configurations based on your application's structure and requirements. Setting up Tomcat Server Locally Transitioning to the next phase involves the establishment of a local Tomcat server. Start by downloading the latest version of Apache Tomcat from the official website and meticulously follow the installation instructions. Once the installation process is complete, the next pivotal step is configuring Tomcat within your Integrated Development Environment (IDE). If you're using Eclipse, for example, seamlessly navigate to the server tab, initiate the addition of a new server, and opt for Tomcat from the available options. This localized setup ensures a synchronized and conducive environment for the impending deployment of your Spring MVC application. Building the Spring MVC App As you progress, it's imperative to verify that your Spring MVC project is poised for a seamless build. Leverage automation tools such as Maven or Gradle to expedite this process efficiently. Integrate the requisite dependencies into your project configuration file, such as the `pom.xml` for Maven users. Execute the build command to orchestrate the compilation and assembly of your project. This step ensures that your Spring MVC application is equipped with all the necessary components and dependencies, laying a solid foundation for subsequent phases of deployment on the local Tomcat server. 1. Project Structure Ensure that your project follows a standard Maven directory structure: CSS project-root │ ├── src │ ├── main │ │ ├── java │ │ │ └── com │ │ │ └── example │ │ │ ├── controller │ │ │ │ └── MyController.java │ │ │ └── AppConfig.java │ │ └── resources │ └── webapp │ └── WEB-INF │ └── views │ ├── pom.xml └── web.xml /* Write CSS Here */ 2. MyController.java: Sample Controller Create a simple controller that handles requests. This is a basic example; you can expand it based on your application requirements. Java package com.example.controller; import org.springframework.stereotype.Controller; import org.springframework.ui.Model; import org.springframework.web.bind.annotation.RequestMapping; @Controller public class MyController { @RequestMapping("/hello") public String hello(Model model) { model.addAttribute("message", "Hello, Spring MVC!"); return "hello"; // This corresponds to the view name } } 3. View ('hello.jsp') Create a simple JSP file under 'src/main/webapp/WEB-INF/views/hello.jsp' Java Server Pages <%@ page contentType="text/html;charset=UTF-8" language="java" %> <html> <head> <title>Hello Page</title> </head> <body> <h2>${message}</h2> </body> </html> 4. 'AppConfig.java': Configuration Ensure that AppConfig.java scans the package where your controllers are located Java package com.example; import org.springframework.context.annotation.ComponentScan; import org.springframework.context.annotation.Configuration; import org.springframework.web.servlet.config.annotation.EnableWebMvc; @Configuration @EnableWebMvc @ComponentScan(basePackages = "com.example.controller") public class AppConfig { // Additional configurations or bean definitions can go here } 5. 'web.xml': Web Application Configuration Configure the DispatcherServlet in web.xml XML <web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_4_0.xsd" version="4.0"> <servlet> <servlet-name>dispatcher</servlet-name> <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> <init-param> <param-name>contextConfigLocation</param-name> <param-value>/WEB-INF/dispatcher-servlet.xml</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>dispatcher</servlet-name> <url-pattern>/</url-pattern> </servlet-mapping> </web-app> 6. 'dispatcher-servlet'.xml Create a dispatcher-servlet.xml file under src/main/webapp/WEB-INF/ to define additional Spring MVC configurations: XML <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:mvc="http://www.springframework.org/schema/mvc" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc.xsd">  <context:component-scan base-package="com.example.controller"/>  <mvc:annotation-driven/>  <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <property name="prefix" value="/WEB-INF/views/"/> <property name="suffix" value=".jsp"/> </bean> </beans> 7. Run the Application Run your application (this depends on your IDE). Access the hello endpoint at http://localhost:8080/your-app-context/hello. You should see the "Hello, Spring MVC!" message. Remember to replace "your-app-context" with the actual context path of your deployed application. War File Creation Transitioning to the packaging phase, it's time to create a deployable Web Application Archive (WAR) file for your Spring MVC application. This file serves as the standardized encapsulation of your Java web application. Utilize prevalent build tools like Maven to automate this process, simplifying the generation of the WAR file. Typically, you'll find this compiled archive neatly organized within the target directory. The WAR file encapsulates your Spring MVC app, ready to be seamlessly deployed onto the local Tomcat server, marking a pivotal step towards actualizing the functionality of your application in a real-world web environment. Deploying on Tomcat Embarking on the deployment phase, the excitement builds as you launch your application onto the local Tomcat server. This involves a straightforward process: copy the previously generated WAR file into the designated `webapps` directory within your Tomcat installation. This directory serves as the portal for deploying web applications. Subsequently, initiate or restart the Tomcat server and watch as it autonomously detects and deploys your Spring MVC application. This automated deployment mechanism streamlines the process, ensuring that your application is swiftly up and running on the local Tomcat server, ready for comprehensive testing and further development iterations. Testing the Deployed App Upon successful deployment, it's time to conduct a comprehensive test of your Spring MVC application. Open your web browser and enter the address `http://localhost:8080/your-app-context`, replacing `your-app-context` with the precise context path assigned to your deployed application. This step allows you to visually inspect and interact with your application in a real-time web environment. If all configurations align seamlessly, you should witness your Spring MVC app dynamically come to life, marking a pivotal moment in the deployment process and affirming the correct integration of your application with the local Tomcat server. Tips for Efficient Development To enhance your development workflow, consider the following tips: Hot swapping: Leverage hot-swapping features in your IDE to avoid restarting the server after every code change. Logging: Implement comprehensive logging to troubleshoot any issues during deployment. Monitoring: Utilize tools like JConsole or VisualVM to monitor your application's performance metrics. Conclusion In reaching this conclusion, congratulations are in order! The successful deployment of your Spring MVC app on a local Tomcat server marks a significant milestone. This guide has imparted a foundational understanding of the deployment process, a vital asset for a seamless transition to production environments. As you persist in honing your development skills, bear in mind that adept deployment practices are instrumental in delivering applications of utmost robustness and reliability. Your achievement in this deployment endeavor underscores your capability to orchestrate a streamlined and effective deployment pipeline for future projects. Well done! More

Canary Releases With Apache APISIX

By Nicolas Fränkel

CORE

Automate JBoss Web Server Deployment With the Red Hat Certified Content Collection for JWS

By Romain Pelisse

Testing Swing Application

By Szymon Tarnowski

CORE

Production-Like Testing Environments in Software Development

In software development, where innovation and efficiency intertwine, the unexpected often takes center stage. Picture this: a bustling team, tirelessly working on a cutting-edge software release, with expectations soaring high. The buzz of excitement turns into a tense hush as news arrives—an important bug has been unearthed, and it's not just any bug: it's a showstopper. The gravity of the situation intensifies as the source of this revelation is our most crucial customer, the foundation of our success, and one of the main reasons why we’ve made it so far as a software development company. The Situation The Critical Revelation The email arrives—an urgent message from the major client whose operations hinge on the seamless functioning of our software. A showstopper bug has reared its head, casting a shadow over their operations and demanding immediate attention. Panic ensues as the team grapples with the realization that a flaw threatens not just the software's integrity but also the trust of our most pivotal partner. The Urgent Call A hastily convened virtual meeting brings together developers, testers, and project managers in a virtual war room. The urgency in the customer's voice resonates through the call, emphasizing the magnitude of the situation. It's clear—this is not a bug that can wait for the next release; it demands an immediate remedy. The Grim Reality As the team delves into the details of the reported bug, a grim reality unfolds. Despite the urgency, reproducing the issue becomes an elusive challenge. Attempts to replicate the bug in the existing testing environments yield no success. Frustration mounts as time passes and the pressure to deliver a solution intensifies. The Reflection In the midst of the chaos, a reflective moment emerges. What if our testing environments mirrored the intricacies of the production landscape more closely? Could this gap in reproduction have been bridged if our testing environments faithfully emulated the conditions under which the bug had manifested for our customer? The Hypothetical Solution The contemplation leads to a hypothetical scenario—a world where production-like testing environments are seamlessly integrated into the development workflow. In this parallel reality, the bug, though elusive in the controlled testing environments, would have been unmasked, dissected, and addressed with urgency. The Lesson Learned As the team races against time to find a workaround for the showstopper, the lesson becomes perfectly clear. The importance of production-like testing environments transcends theoretical discussions; it becomes a mandate for resilience and responsiveness. In a landscape where the unexpected is the only constant, the ability to replicate real-world conditions in testing environments emerges as a basis for averting crises and fortifying the reliability of our software releases. Why Production-Like Environments Matter Production-like testing environments play a pivotal role in identifying potential issues before software reaches the production stage, contributing to a more robust and efficient development process. Designed to closely mimic the conditions of the actual production environment, this includes replicating the hardware, software, network configurations, and other parameters that characterize the production setting. By creating an environment that mirrors production, development and testing teams can uncover issues that may not be apparent in isolated or artificial testing setups. Here is a snapshot of what makes such environments important: Improved Software Quality By mirroring the production environment, testing teams can uncover and resolve environment-specific issues that might have gone undetected in other testing phases. This leads to enhanced software quality and a reduced risk of production downtime or performance bottlenecks. Enhanced User Experience Production-like testing environments may allow for thorough user acceptance testing, ensuring that the software meets user expectations and functions seamlessly in real-world scenarios. This translates into a positive user experience and increased customer satisfaction. Early Issue Detection By testing in an environment that closely resembles production, teams can catch potential problems early in the development lifecycle. This reduces the likelihood of disruptive deployments where critical issues occur when the software is deployed to production. This helps organizations maintain a smooth and reliable software release process. Accurate Performance Testing Performance testing is more meaningful when conducted in an environment that replicates the conditions of actual use. This includes factors such as the number of concurrent users, data volume, and network conditions. Implementing Production-Like Testing Environments We can use IaC, containerization and orchestration, effective data management, monitoring and logging to implement production-like testing environments. In what follows we will explore how we can use all the above in more detail. Infrastructure As Code (IaC) IaC is an approach to managing and provisioning computing infrastructure through machine-readable script files. In the context of testing environments, IaC plays a crucial role in automating the setup and configuration of infrastructure, ensuring consistency and repeatability. It involves expressing infrastructure configurations, such as servers, networks, and databases, in code files. These files describe the desired state of the infrastructure. Advantages of IaC in Testing Environments Consistency: IaC ensures consistent deployment and configuration of testing environments, reducing the risk of environment-related issues caused by manual errors or discrepancies. Scalability: Automated provisioning allows for the quick and scalable creation of multiple testing environments, accommodating the needs of diverse testing scenarios. Reproducibility: IaC makes it possible to reproduce identical environments at different stages of the development lifecycle, from development to testing to production. Collaboration and traceability: IaC scripts, stored in version control, facilitate collaboration among team members. Changes can be tracked, reviewed, and rolled back if necessary. IaC fits seamlessly into continuous integration/continuous deployment (CI/CD) workflows. As code changes are made, IaC scripts can be automatically triggered to provision or update testing environments. This integration ensures that testing environments are always aligned with the latest codebase. Embracing concepts like immutable infrastructure, where environments are treated as disposable and replaced rather than modified may enhance consistency and reliability. Containerization and Orchestration Containerization and orchestration have emerged as pivotal strategies for software development and testing. These practices revolutionize the way applications are deployed, managed, and tested. Containerization allows developers to encapsulate applications and their dependencies into a standardized unit (the container). This unit ensures that the application runs consistently across different computing environments. Orchestration involves the coordination and management of multiple containers to ensure they work together seamlessly. Kubernetes is a powerful open-source orchestration platform that automates the deployment, scaling, and management of containerized applications. Benefits for Testing Environments Consistency: Containers ensure consistency between development, testing, and production environments, reducing the "it works on my machine" problem. Isolation: Each container runs in isolation, preventing conflicts between dependencies and ensuring that testing environments are not affected by changes in other parts of the system. Quick deployment: Containers can be spun up and torn down rapidly, facilitating quick and efficient testing cycles. Scalability: Orchestration platforms facilitate the easy scaling of testing environments to accommodate varying workloads. Efficiency and resource utilization: Containers are lightweight and share the host OS kernel, making them more efficient in terms of resource utilization compared to traditional virtual machines. Improved collaboration: Containers and orchestration enhance collaboration between development and testing teams by providing a standardized and reproducible environment. Proper security practices should be implemented to secure containers, especially when dealing with sensitive data or in production environments. Data Management Data management is a critical aspect of implementing production-like testing environments, and it involves carefully handling and manipulating data to replicate real-world scenarios. Whether it's replicating production data or generating synthetic data, this strategy is essential for testing how applications interact with and handle different data volumes while ensuring the integrity of the data. Replicating Production Data This involves using a copy of the actual data from the production environment in the testing environment. A major benefit of this approach is realism. Production data provides a realistic representation of the data the application will process, offering insights into how the system behaves under authentic conditions. Such data often reflects the complexity and diversity of actual usage, helping to uncover issues related to data relationships, structure, and distribution. A major challenge is privacy and security. Handling sensitive or personally identifiable information requires careful consideration to comply with privacy and security regulations. Synthetic Data Generation Generating synthetic data involves creating artificial datasets that closely resemble real-world scenarios. This approach is particularly useful when replicating production data is impractical or poses privacy concerns. One benefit is control. Synthetic data provides control over the characteristics of the dataset, allowing for the creation of specific scenarios, edge cases, and data variations. A challenge of synthetic data is realism. Creating synthetic data that accurately represents the complexity and distribution of real-world data can be challenging. For example, ensuring that synthetic data preserves relationships between different data elements may be crucial for meaningful testing. Data Masking and Anonymization When using real production data, especially in testing environments where privacy is a concern, data masking and anonymization techniques can be applied to protect sensitive information. By obscuring or replacing sensitive information with masked or anonymized equivalents, organizations can navigate legal requirements and privacy standards seamlessly. One of the challenges associated with data masking and anonymization is the need for consistent application to maintain the integrity and relationships within the dataset. In scenarios where databases have intricate relationships, ensuring that masked or anonymized data retains these connections is crucial for meaningful testing. Striking the right balance between obscuring sensitive details and preserving the authenticity of data for realistic testing is another challenge. Monitoring and Logging The objective here is not only to observe metrics for the system under test but also to maintain a dynamic equilibrium that mirrors the real-world conditions of production. Ongoing monitoring and validation of data in the testing environment may ensure that it remains consistent, relevant, and representative of real-world conditions. These practices involve the systematic collection, analysis, and visualization of data related to an application's performance, behavior, and issues. It helps in capturing performance metrics, identifying bottlenecks, and gaining insights into the application's behavior. Monitoring Proximity To keep the testing environment in close proximity to the production environment we may: Align performance metrics: Regularly compare key performance indicators (KPIs) between testing and production, adjusting testing conditions to replicate the expected production behavior. This may allow for a consistent benchmark. Simulate realistic user loads: Simulate user loads in testing that closely mimic the patterns and volumes observed in the production environment. Utilize load testing tools to replicate varying levels of user activity, allowing for the assessment of application performance under conditions reflective of production. Consistently utilize resources: Maintain consistency in resource utilization patterns between testing and production environments. Monitor CPU usage, memory consumption, and other resource metrics during testing to ensure they align with the metrics observed in the production setting. Logging Proximity To align testing environments with production realities we may strive for: Event logging harmony: Check that event logging in testing environments harmonizes with the diversity and significance of events recorded in production. Develop a comprehensive event-logging strategy that mirrors the types of events deemed crucial in production scenarios. This involves capturing events related to user interactions, system processes, and critical transactions. Error logging fidelity: Align error logging in testing with the diversity and fidelity required for effective troubleshooting in production scenarios. Log errors encountered during testing rigorously, capturing not only error messages but also contextual information such as stack traces and data points. This mirrors the thorough error logging essential for root cause analysis in production. Audit logging consistency: Check that audit logging in testing environments is consistent with the recording of actions and transactions in production. Capture and record user actions, system modifications, and other relevant activities during testing. This ensures that the audit trail in testing aligns with the stringent requirements of compliance and accountability observed in production. Wrapping Up Starting with a personal experience from my early years in software development, this article showcases why production-like testing environments can be a critical component for delivering high-quality, reliable software. By closely simulating real-world conditions, teams can detect and address issues early in the development process, leading to more robust and resilient applications. Embracing strategies like IaC, containerization, effective data management, monitoring, and logging, can enhance the effectiveness of production-like testing environments, ultimately contributing to a smoother and more reliable software release cycle.

By Stelios Manioudakis

Getting Started With Jenkins

Jenkins has been a staple in software automation for over a decade due largely to its feature-rich tooling and adaptability. While many impressive alternatives have entered the space, Jenkins remains one of the vanguards. Despite its success, Jenkins can have a significant learning curve, and jumping into the vast world of Jenkins plugins and features can quickly become overwhelming. In this article, we will break down that complexity by first understanding the fundamentals and concepts that underpin Jenkins. With that foundation, we will learn how to create a simple pipeline in Jenkins to build and test an application. Lastly, we will look at how to advance this simple example into a more complex project and explore some alternatives to Jenkins. What Is Jenkins? Fundamentals and Concepts Jenkins is a software automation service that helps us script tasks like builds. Of particular interest is Jenkins's ability to create a pipeline, or a discrete set of ordered tasks. Before we create a pipeline in Jenkins, we must first understand what a pipeline is and why it is useful. This understanding starts with a journey through the history of software development. Big Bang Integration Before automation, we were forced to manually build and test our applications locally. Once our local tests passed, we would commit our changes to a remote repository to integrate them with the changes made by other developers. At a predetermined point — usually, as a release approached — the Quality Assurance (QA) team would take the code in our remote repository and test it. While our local tests may have passed before we committed them, and the local tests of other developers worked before they committed them, there was no guarantee that our combined changes would work. We would instead have to wait until QA tested everything together. This moment of truth was usually called the Big Bang. In the (likely) event that the tests failed, we would then have to hunt through all of the commits to see which one (or ones) was the culprit. Continuous Integration The process of running our tests and verifying our code after each commit is called Continuous Integration (CI). As the name implies, CI differs from Big Bang integration by continuously integrating code and verifying that it works. The Big Bang integration approach may work for small projects or prototypes, but it is a massive hindrance for medium- or large-scale projects. Ideally, we want to know if our tests pass when we merge our code with the remote repository. This requires two main changes to our process: Automating tests Executing automated tests after each check-in While automated tests could be created for a Big Bang project, they are not required. They are, however, required for our new process to work. It would be prohibitive to manually test the tests for any project with multiple commits per day. Instead, we need a test suite that can be run automatically, wherever and whenever necessary. The second requirement is that our automated tests are run each time a commit is made to the remote repository. This requires that some service (external or co-located with our repository) check out the repository after each commit, run our tests, and report if the tests passed or failed. This process could be run periodically, but ideally, it should be run every time a commit is made so that we can trace exactly which commit caused our test suite to fail. With CI, instead of waiting until some point in the future to see if our code works, we know at any given time whether our code works; what's more, we also know exactly when and where a failure originates when it stops working. CI is a massive leap forward in software automation. There are very few projects today that do not use some level of CI to ensure that each commit does not "break the build." While this is a great improvement, it is only a half-step relative to the process that our code traverses from commit to delivery. Continuous Delivery When we looked at our manual build process, we rightly saw an opportunity to automate the build and test stages of our process; but this is only a small part of the overall process. For most software, we do not just build and unit test; we also run higher-level tests (such as integration and system tests), deliver our final product to our customers, and a wide array of steps in between. If we are following the mindset of CI, it begs the question: Why not automate the entire business process, from build to delivery, and run each step in the process sequentially until our product is automatically delivered to the customer? This revolutionary approach is called Continuous Delivery (CD). Like CI, CD continuously integrates our code as we make commits, but unlike CI, CD does not stop after unit tests are complete. Instead, CD challenges us to automate every step in our business process until the final product is automatically delivered to the customer. This sequence of automated steps is called a pipeline. A pipeline consists of stages, which are groups of steps executed in parallel. For one stage to start, all the steps of the previous stage must complete successfully. An example of a common CI/CD pipeline is illustrated below: While the particular stages and steps of a CI/CD pipeline may vary, they all share a common definition: they are simple abstractions of the business process that a software product must complete before it is delivered to the customer. Even without CI/CD, every software delivery includes a delivery process; we execute the process manually. CI/CD does not introduce anything new to the process: it simply automates each stage so that the pipeline can be executed automatically. Learn more about CI/CD Software Design Patterns. CI/CD is a very involved topic, and it can be overwhelming at first glance, but it can be summed up with a few main concepts: A pipeline is an abstraction of the business process we use to deliver a product. A pipeline is composed of an ordered set of stages. A stage is composed of a set of steps that are run in parallel. A stage cannot start executing until all of the steps in a previous stage have completed. A trigger is the first event in a pipeline that initiates the first stage in a pipeline (i.e., a commit to a repository). A pipeline is executed after every commit to a repository. The deliverable from a pipeline is not delivered to a customer unless all of the stages pass. This last point is where CI/CD shines: We know that any artifact delivered to a customer is the last working artifact that successfully passes through the pipeline. Likewise, we know that any time a commit results in a passing artifact, it is automatically delivered to the customer (the customer does not have to wait for us to deliver it or wait for multiple commits to receive the latest delivery). For more information on pipelines and CI/CD in general, see the following articles: "How To Build an Effective CI/CD Pipeline" "Continuous Test Automation Using CI/CD: How CI/CD Has Revolutionized Automated Testing" CI/CD and Jenkins At this point, we have a foundational understanding of what CI/CD is and why it is important. In our discussion of CI/CD, we left out one important point: what actually executes the pipeline? Whatever this remaining piece is, it must be capable of doing the following: Scan a remote repository for commits Clone the latest code from a repository Define a pipeline and its constituent stages using scripts and other automated mechanism Run the automated steps in the pipeline Report the status of a pipeline execution (e.g., pass or fail) Deliver the final artifacts to some internal or external location This is where Jenkins comes in. Jenkins is an automation server that can be used to perform all of the steps above. While Jenkins is a very powerful automation service that can do more than just CI/CD (Jenkins can conceivably automate just about any process), it has the tools necessary to create a functional CI/CD pipeline and execute it after a commit to our repository. Jenkins has a long history — and capturing all its ins and outs would consume volumes — but at its core, Jenkins is a powerful CI/CD tool used by many of the largest software companies. Its rich set of features and plugins, along with its time-tested reliability, has cemented it as a staple in the software automation community. For more general information on Jenkins, see the official Jenkins documentation. Jenkins Builds: Setting up a Pipeline Our main goal in this tutorial is to set up a simple pipeline using Jenkins. While most Jenkins pipelines (or any pipeline in general) will include numerous, possibly complex stages, we will start by creating a minimally viable pipeline with a single stage. We will then split our single-stage pipeline into a two-stage pipeline. From there, we will examine how to use this simple pipeline as a starting point for a production-ready pipeline. Setting up Jenkins To set up Jenkins, we will need to complete three steps: Install Docker Build and run the Jenkins Docker container Configure Jenkins Installing Docker Before we install Docker, we need to create a DockerHub account. DockerHub is the Docker equivalent of GitHub and acts as a registry of preconfigured container images, such as Ubuntu, MongoDB, and Jenkins. We will use these preconfigured containers as a starting point for installing Jenkins, as well as a starting point for the projects that we build in Jenkins. To create a DockerHub account: Navigate to the DockerHub Sign-Up page. Enter your email and desired username and password, or link to a Google or GitHub account. Submit your account information. Verify your account using the email sent to the email address entered above. Login to your new DockerHub account. For our first Jenkins project, the default, Personal account will suffice since it allows us to download 200 containers every 6 hours (at the time of writing). If we were creating a Jenkins pipeline for a business product or a team project, we should look for an upgraded account, such as Pro or Business. For more information, see the Docker Pricing page. Review related documentation on how to health check your Docker Containers. Once we have created a DockerHub account, we can install the Docker Desktop application. Docker Desktop is a visual application that allows us to buiu Docker images and start them as containers. Docker Desktop is supported on Windows, Mac, and Linux. For more information on how to install Docker Desktop on each of these platforms, see the following Docker pages: Install Docker Desktop on Windows Install Docker Desktop on Mac Install Docker Desktop on Linux Once Docker Desktop is installed, we need to log in to our DockerHub account to link it to our Docker Desktop installation: Open Docker Desktop. Click the Login link. Log in to DockerHub in the opened browser tab. Return to Docker Desktop after logging in. Accept the license agreement. Once our account is linked, we are ready to pull the Docker Jenkins image and start the container. Running the Jenkins Docker Container With Docker now set up, we can create a new Jenkins image that includes all the necessary packages and run the image as a container. For this section, we will use the Windows setup process as an example. The setup process for macOS and Linux is similar but slightly different. For more information on setting up the Jenkins container on macOS or Linux, see Installing Jenkins with Docker on macOS and Linux. First, we need to create a bridge network for Jenkins using the following command: Shell docker network create jenkins Next, we need to run a docker:dinb image: Shell docker run --name jenkins-docker --rm --detach ^ --privileged --network jenkins --network-alias docker ^ --env DOCKER_TLS_CERTDIR=/certs ^ --volume jenkins-docker-certs:/certs/client ^ --volume jenkins-data:/var/jenkins_home ^ --publish 2376:2376 ^ docker:dind The docker:dind (dind stands for Docker-in-Docker) is an image provided by Docker that allows us to run Docker inside a Docker container. We will need Docker to be installed inside our container to run our pipeline since Jenkins will start a new Docker container within the Jenkins container to execute the steps of our pipeline. Next, we must create a Docker image based on the Jenkins image. This custom image includes all the features, such as the Docker CLI, that Jenkins needs to execute our pipeline. To create this image, we can save the following Dockerfile in the current directory: Dockerfile FROM jenkins/jenkins:2.414.3-jdk17 USER root RUN apt-get update && apt-get install -y lsb-release RUN curl -fsSLo /usr/share/keyrings/docker-archive-keyring.asc \ https://download.docker.com/linux/debian/gpg RUN echo "deb [arch=$(dpkg --print-architecture) \ signed-by=/usr/share/keyrings/docker-archive-keyring.asc] \ https://download.docker.com/linux/debian \ $(lsb_release -cs) stable" > /etc/apt/sources.list.d/docker.list RUN apt-get update && apt-get install -y docker-ce-cli USER jenkins RUN jenkins-plugin-cli --plugins "blueocean docker-workflow" Once the Dockerfile has been saved, we can create a new image from it: Shell docker build -t myjenkins-blueocean:2.414.3-1 . This command names the new image myjenkins-blueocean (with a version of 2.414.3-1) and assumes the Dockerfile is in the current directory. Note that we can use any valid Docker image name that we wish. At the time of writing, a valid image name abides by the following criteria: The [name] must be valid ASCII and can contain lowercase and uppercase letters, digits, underscores, periods, and hyphens. It cannot start with a period or hyphen and must be no longer than 128 characters. Lastly, we can start our container using the following command: Shell docker run --name jenkins-blueocean --restart=on-failure --detach ^ --network jenkins --env DOCKER_HOST=tcp://docker:2376 ^ --env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 ^ --volume jenkins-data:/var/jenkins_home ^ --volume jenkins-docker-certs:/certs/client:ro ^ --publish 8080:8080 --publish 50000:50000 myjenkins-blueocean:2.414.3-1 We can confirm that our Jenkins container (named myjenkins-blueocean) is running by completing the following steps: Open Docker Desktop. Click the Containers tab on the left panel. Ensure that the myjenkins-blueocean container is running. The running container will resemble the following in the Docker Desktop GUI: At this point, our Jenkins container is ready. Again, the process for creating the Jenkins container for Mac and Linux is similar to that of Windows. For more information, see the following pages: Installing Jenkins with Docker on Windows Installing Jenkins with Docker on macOS and Linux (linked previously in this article) Configuring Jenkins Once the Jenkins container is running, we can access the Jenkins User Interface (UI) through our browser at http://localhost:8080. The Jenkins welcome screen will give us a prompt requesting the Administrator password. We can find this password and complete the Jenkins installation using the following steps: Open Docker Desktop. Click the Containers tab on the left panel. Click our running Jenkins container (myjenkins-blueocean). Click the Logs tab (this tab should open by default) Find the lines in the log that resemble the following: Plain Text 2023-10-31 11:25:45 ************************************************************* 2023-10-31 11:25:45 ************************************************************* 2023-10-31 11:25:45 ************************************************************* 2023-10-31 11:25:45 2023-10-31 11:25:45 Jenkins initial setup is required. An admin user has been created and a password generated. 2023-10-31 11:25:45 Please use the following password to proceed to installation: 2023-10-31 11:25:45 2023-10-31 11:25:45 080be1abb4e04be59a0428a85c02c6e9 2023-10-31 11:25:45 2023-10-31 11:25:45 This may also be found at: /var/jenkins_home/secrets/initialAdminPassword 2023-10-31 11:25:45 2023-10-31 11:25:45 ************************************************************* 2023-10-31 11:25:45 ************************************************************* 2023-10-31 11:25:45 ************************************************************* In this example, the administrative password is 080be1abb4e04be59a0428a85c02c6e9. Input this password into the Jenkins welcome page (located at http://localhost:8080). Click the Continue button. Click the Install Suggested Plugins button on the Customize Jenkins page. Wait for the Jenkins setup to complete. Enter a desired username, password, email, and full name. Click the Save and Continue button. Enter http://localhost:8080/ (the default value) on the Instance Configuration page. Click the Save and Finish button. Click the Start using Jenkins button. At this point, Jenkins is running and configured, and we are now ready to create our Jenkins pipeline. Note that we performed a very basic installation, and the installation we perform for a business project or a larger team will vary. For example, we may need additional plugins to allow our team to log in, or we may have additional security concerns — such as not running Jenkins on HTTP or on localhost:8080. For more information on how to set up a Jenkins container, see the Jenkins Docker Installation page. Creating a Pipeline Our next step is to create a pipeline to execute our build. Pipelines can be complex, depending on the business process being automated, but it's a good idea to start small and grow. In keeping with this philosophy, we will start with a simple pipeline: A single stage with a single step that runs mvn clean package to create an artifact. From there, we will divide the pipeline into two stages: a build stage and a test stage. To accomplish this, we will: Install the Docker Pipeline plugin if it is not already installed. Add a Jenkinsfile to our project. Create a pipeline in Jenkins that uses our project. Configure our pipeline to build automatically when our project repository changes. Separate our single-stage pipeline into a build and test stage. Installing the Pipeline Plugin Sometimes, the Docker Pipeline plugin (which is needed to create our pipeline) may not be installed by default. To check if the plugin is installed, we must complete the following steps: Click Manage Jenkins in the left panel. Click the Plugins button under System Configuration. Click Installed plugins in the left panel. Search for docker pipeline in the search installed plugins search field. Ensure that the Docker Pipeline plugin is enabled. If the Docker Pipeline plugin is installed, we can skip the installation process. If the plugin is not installed, we can install it using the following steps: Click Manage Jenkins in the left panel. Click the Plugins button under System Configuration. Click Available plugins in the left panel. Search for docker pipeline in the search available plugins search field. Check the checkbox for the Docker Pipeline plugin. Click the Install button on the top right. Wait for the download and installation to complete. With the plugin installed, we can now start working on our pipeline. Related Guide: How to Replace CURL in Scripts with the Jenkins HTTP Request Plugin. Adding a Simple Jenkinsfile Before we create our pipeline, we first need to add a Jenkinsfile to our project. A Jenkinsfile is a configuration file that resides at the top level of our repository and configures the pipeline that Jenkins will run when our project is checked out. A Jenkinsfile is similar to a Dockerfile but deals with pipeline configurations rather than Docker image configurations. Note that this section will use the jenkins-example-project as a reference project. This repository is publicly available, so we can use and build it from any Jenkins deployment, even if Jenkins is deployed on our machine. We will start with a simple Jenkinsfile (located in the root directory of the project we will build) that creates a pipeline with a single step (Build): Plain Text pipeline { agent { docker { image 'maven:3.9.5-eclipse-temurin-17-alpine' args '-v /root/.m2:/root/.m2' } } stages { stage('Build') { steps { sh 'mvn clean package' } } } } The agent section of a Jenkins file configures where the pipeline will execute. In this case, our pipeline will execute on a Docker container run from the maven:3.9.5-eclipse-temurin-17-alpine Docker image. The -v /root/.m2:/root/.m2 argument creates a two-way mapping between the /root/.m2 directory within the Docker container and the /root/.m2 directory within our Docker host. According to the Jenkins documentation: This args parameter creates a reciprocal mapping between the /root/.m2 directories in the short-lived Maven Docker container and that of your Docker host’s filesystem....You do this mainly to ensure that the artifacts for building your Java application, which Maven downloads while your Pipeline is being executed, are retained in the Maven repository after the Maven container is gone. This prevents Maven from downloading the same artifacts during successive Pipeline runs. Lastly, we create our stages under the stages section and define our single stage: Build. This stage has a single step that runs the shell command mvn clean package. More information on the full suite of Jenkinsfile syntax can be found on the Pipeline Syntax page and more information on the mvn clean package command can be found in the Maven in 5 Minutes tutorial. Creating the Pipeline With our Jenkinsfile in place, we can now create a pipeline that will use this Jenkinsfile to execute a build. To set up the pipeline, we must complete the following steps: Navigate to the Jenkins Dashboard page (http://localhost:8080/). Click + New Item in the left panel. Enter a name for the pipeline (such as Example-Pipeline). Click Multibranch Pipeline. Click the OK button. Click Add Source under the Branch Sources section. Select GitHub. Enter the URL of the repository to build in the Repository HTTPS URL field (for example, https://github.com/albanoj2/jenkins-example-project) Click the Validate button. Ensure that the message Credentials ok. Connected to [project-url] is displayed. Click the Save button. Saving the pipeline configuration will kick off the first execution of the pipeline. To view our latest execution, we need to navigate to the Example-Pipeline dashboard by clicking on the Example-Pipeline link in the pipeline table on the main Jenkins page (http://localhost:8080): Jenkins structures its dashboard in a hierarchy that resembles the following: Main Dashboard → Pipeline → Branch In this case, Example-Pipeline is the page that displays information about our newly created pipeline. Within this page is a branch table with a row for each branch we track from our repository. In our case, we are only tracking the master branch, but if we tracked other branches, we would see more than one row for each of our branches: Each tracked branch is run according to the Jenkinsfile for that branch. Conceivably, the pipeline for one branch may differ from the pipeline for another (since their Jenkinsfiles may differ), so we should not assume that the execution of each pipeline under the Example-Pipeline pipeline will be the same. We can also track Pull Requests (PR) in our repository similarly to branches: For each PR we track, a new row will be added to the PR table (accessed by clicking the Pull Requests link next to the Branches link above the table), which allows us to see the pipeline executions for each PR. For more information on tracking branches and PRs, see the Jenkins Pipeline Branches and Pull Requests page. If we click on the master branch, the master branch page shows us that both stages of our pipeline (and, therefore, the entire pipeline) were completed successfully. While we only definite a single stage (Build) in our pipeline, Jenkins will implicitly add a stage (Checkout SCM, or Software Configuration Management), which checks out our repository. Once the repository is checked out, Jenkins runs our pipeline stages against the local clone. Running the Pipeline Automatically By default, our pipeline will only be executed manually. To automatically execute the pipeline, we have two options: Scan the repository periodically Create a webhook To scan the repository periodically, we can change the pipeline configuration: Navigate to the Jenkin Dashboard. Click the Example-Pipeline pipeline. Click the Configuration tab in the left panel. Check the Periodically if not otherwise run checkbox. Set the Interval to the desired amount. Click the Save button. This will poll the repository periodically and execute the pipeline if a change is detected (if an execution was not otherwise started manually). Creating a webhook is more synchronized and does not require polling, but it does require a bit more configuration. For more information about how to set up a webhook for Jenkins and a GitHub repository, see how to add a GitHub webbook in your Jenkins pipeline. Running Tests Separately While a single-stage pipeline is a good starting point, it is unrealistic in production environments. To demonstrate a multi-stage pipeline, we will split our existing build stage into two stages: a build and a test stage. In our existing build stage, we built and tested our project using the mvn clean package command. With our two-stage pipeline, we will skip our tests in the build stage using the -DskipTests=true Maven flag and add a second stage that runs only our tests using the mvn test command. Implementing this in our project results in the following Jenkinsfile: Plain Text pipeline { agent { docker { image 'maven:3.9.5-eclipse-temurin-17-alpine' args '-v /root/.m2:/root/.m2' } } stages { stage('Build') { steps { sh 'mvn clean package -DskipTests=true' } } stage('Test') { steps { sh 'mvn test' } post { always { junit 'target/surefire-reports/*.xml' } } } } } All but one of the changes to our Jenkinsfile is known: in addition to running mvn test, we also create a postprocessing step using the post section. In this section, we define a postprocessing step that is always run and tells the Jenkins JUnit Plugin where to look for our JUnit test artifacts. The JUnit Plugin is installed by default and gives us a visualization of the number of passed and failed tests over time. Related Tutorial: Publish Maven Artifacts to Nexus OSS Using Pipelines or Maven Jobs When our next build runs (which should be picked up automatically due to our SCM trigger change), we see that a new stage, Test, has been added to our Example-Pipeline master page. Note that since our previous build did not include this stage, it is grayed out. Looking at the top right of the same page, we see a graph of our tests over time. In our example project, there are nine tests, so our graph stays uniform at nine tests. If we only have one pipeline execution incorporating the JUnit Plugin, then we may not see this graph filled like above. As we run more executions, we will see the graph start to fill over time. Jenkins Features: Extending the Jenkins Pipeline The example pipeline we have created is a good starting point, but it is not realistic for medium- and large-scale applications, and it only scratches the surface of what Jenkins pipelines are capable of. As we start to build more sophisticated pipelines, we will begin to require more sophisticated features, such as: Deploying artifacts to artifact repositories Deploying Docker images to a container image repository Connecting to external services using credentials and authentication Using more advanced UIs, such as Blue Ocean Building and testing applications on different environments and operating systems Building and testing applications in parallel across multiple worker nodes Standing up complex test and deployment environments in Kubernetes Restricting access to designated administrators The list of possibilities is nearly endless, but it suffices to say that realistic projects will need to build off the simple project we have created here and add more rich features and more complex tooling to accomplish their goals. The following resources are great places to start: Jenkins User Guide Jenkins: Build a Java app with Maven Jenkins: Blue Ocean Jenkins: Scaling Pipelines Jenkins: Security Jenkins "Building a Continuous Delivery Pipeline Using Jenkins" Alternatives to Jenkins Jenkins is a very powerful tool that has many advantages over its competitors, including: General automation tooling (not just CI/CD) A vast library of plugins The ability to manage multiple projects in a single location A wide user base and community knowledge Venerability and time-tested adoption Despite that, it is important to know its alternatives and understand where they outshine Jenkins. The following is a list of Jenkins's most popular alternatives (not comprehensive) and some advantages they may have over Jenkins when building a pipeline: GitHub Actions: GitHub Actions is the GitHub implementation of CI/CD. The biggest advantage of Actions over Jenkins is that Actions is directly incorporated with GitHub. This means a pipeline built in Actions (known as a workflow) can be accessed in the same GitHub repository where our code, issues, and Wikis are located. This means we do not have to manage a separate Jenkins server and can access all of the data that supports our code in one location. While Jenkins has a wider range of plugins and integrations that can be used, GitHub Actions should be seriously considered if we are building a pipeline for code already stored in GitHub. GitLab CI/CD: Similar to GitHub Actions, GitLab CI/CD is the native pipeline builder for GitLab repositories. The advantages that GitLab CI/CD has over Jenkins are analogous to GitHub Actions's advantages: All of the tools surrounding our pipelines are located in the same application where our code is stored. GitLab CI/CD should be seriously considered when using GitLab as a remote repository. Learn how to auto deploy Spring Boot apps with GitLab CI/CD. Other alternatives to Jenkins are also common and may be useful to explore when setting up a pipeline: Circle CI Travis CI GoCD TeamCity While Jenkins has many advantages, it is important to explore alternative options to see which CI/CD solution is the best for the task at hand. Read DZone's coverage of Jenkins VS Bamboo, and Jenkins VS Gitlab. Conclusion Jenkins has been a vanguard in software automation and CI/CD since its inception in 2011. Despite this success, jumping straight into Jenkins can quickly become overwhelming. In this article, we looked at the fundamentals of CI/CD and how we can apply those concepts to create a working pipeline in Jenkins. Although this is a good starting point, it only scratches the surface of what Jenkins is capable of. As we create more and more complex projects and want to deliver more sophisticated products, we can take the knowledge we learned and use it as a building block to deliver software efficiently and effectively using Jenkins. Tutorial: Docker, Kubernetes, and Azure DevOps More Information For more information on CI/CD and Jenkins, see the following resources: The Jenkins User Handbook Continuous Delivery by Jez Humble and David Farley ContinuousDelivery.com The Jenkins Website

By Justin Albano

CORE

Distributed Cloud Architecture for Resilient Systems

This is an article from DZone's 2023 Observability and Application Performance Trend Report.For more: Read the Report Employing cloud services can incur a great deal of risk if not planned and designed correctly. In fact, this is really no different than the challenges that are inherit within a single on-premises data center implementation. Power outages and network issues are common examples of challenges that can put your service — and your business — at risk. For AWS cloud service, we have seen large-scale regional outages that are documented on the AWS Post-Event Summaries page. To gain a broader look at other cloud providers and services, the danluu/post-mortems repository provides a more holistic view of the cloud in general. It's time for service owners relying (or planning) on a single region to think hard about the best way to design resilient cloud services. While I will utilize AWS for this article, it is solely because of my level of expertise with the platform and not because one cloud platform should be considered better than another. A Single-Region Approach Is Doomed to Fail A cloud-based service implementation can be designed to leverage multiple availability zones. Think of availability zones as distinct locations within a specific region, but they are isolated from other availability zones in that region. Consider the following cloud-based service running on AWS inside the Kubernetes platform: Figure 1: Cloud-based service utilizing Kubernetes with multiple availability zones In Figure 1, inbound requests are handled by Route 53, arrive at a load balancer, and are directed to a Kubernetes cluster. The controller routes requests to the service that has three instances running, each in a different availability zone. For persistence, an Aurora Serverless database has been adopted. While this design protects from the loss of one or two availability zones, the service is considered at risk when a region-wide outage occurs, similar to the AWS outage in the US-EAST-1 region on December 7th, 2021. A common mitigation strategy is to implement stand-by patterns that can become active when unexpected outages occur. However, these stand-by approaches can lead to bigger issues if they are not consistently participating by handling a portion of all requests. Transitioning to More Than Two With single-region services at risk, it's important to understand how to best proceed. For that, we can draw upon the simple example of a trucking business. If you have a single driver who operates a single truck, your business is down when the truck or driver is unable to fulfill their duties. The immediate thought here is to add a second truck and driver. However, the better answer is to increase the fleet by two, which allows for an unexpected issue to complicate the original situation. This is known as the "n + 2" rule, which becomes important when there are expectations set between you and your customers. For the trucking business, it might be a guaranteed delivery time. For your cloud-based service, it will likely be measured in service-level objectives (SLOs) and service-level agreements (SLAs). It is common to set SLOs as four nines, meaning your service is operating as expected 99.99% of the time. This translates to the following error budgets, or down time, for the service: Month = 4 minutes and 21 seconds Week = 1 minute and 0.48 seconds Day = 8.6 seconds If your SLAs include financial penalties, the importance of implementing the n + 2 rule becomes critical to making sure your services are available in the wake of an unexpected regional outage. Remember, that December 7, 2021 outage at AWS lasted more than eight hours. The cloud-based service from Figure 1 can be expanded to employ a multi-region design: Figure 2: Multi-region cloud-based service utilizing Kubernetes and multiple availability zones With a multi-region design, requests are handled by Route 53 but are directed to the best region to handle the request. The ambiguous term "best" is used intentionally, as the criteria could be based upon geographical proximity, least latency, or both. From there, the in-region Kubernetes cluster handles the request — still with three different availability zones. Figure 2 also introduces the observability layer, which provides the ability to monitor cloud-based components and establish SLOs at the country and regional levels. This will be discussed in more detail shortly. Getting Out of the Toil Game Google Site Reliability Engineering's Eric Harvieux defined toil as noted below: "Toil is the kind of work that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows." When designing services that run in multiple regions, the amount of toil that exists with a single region becomes dramatically larger. Consider the example of creating a manager-approved change request every time code is deployed into the production instance. In the single-region example, the change request might be a bit annoying, but it is something a software engineer is willing to tolerate. Now, with two additional regions, this will translate to three times the amount of change requests, all with at least one human-based approval being required. An obtainable and desirable end-state should still include change requests, but these requests should become part of the continuous delivery (CD) lifecycle and be created automatically. Additionally, the observability layer introduced in Figure 2 should be leveraged by the CD tooling in order to monitor deployments — rolling back in the event of any unforeseen circumstances. With this approach, the need for human-based approvals is diminished, and unnecessary toil is removed from both the software engineer requesting the deployment and the approving manager. Harnessing the Power of Observability Observability platforms measure a system's state by leverage metrics, logs, and traces. This means that a given service can be measured by the outputs it provides. Leading observability platforms go a step further and allow for the creation of synthetic API tests that can be used to exercise resources for a given service. Tests can include assertions that introduce expectations — like a particular GET request will respond with an expected response code and payload within a given time period. Otherwise, the test will be marked as failed. SLOs can be attached to each synthetic test, and each test can be executed in multiple geographical locations, all monitored from the observability platform. Taking this approach allows service owners the ability to understand service performance from multiple entry points. With the multi-region model, tests can be created and performance thereby monitored at the regional and global levels separately, thus producing a high degree of certainty on the level of performance being produced in each region. In every case, the power of observability can justify the need for manual human-based change approvals as noted above. Bringing It All Together From the 10,000-foot level, the multiregion service implementation from Figure 2 can be placed onto a United States map. In Figure 3, the database connectivity is mapped to demonstrate the inner-region communication, while the observability and cloud metrics data are gathered from AWS and the observability platform globally. Figure 3: Multi-region service adoption placed near the respective AWS regions Service owners have peace of mind that their service is fully functional in three regions by implementing the n + 2 rule. In this scenario, the implementation is prepared to survive two complete region outages. As an example, the eight-hour AWS outage referenced above would not have an impact on the service's SLOs/ SLAs during the time when one of the three regions is unavailable. Charting a Plan Toward Multi-Region Implementing a multi-region footprint for your service without increasing toil is possible, but it does require planning. Some high-level action items are noted below: Understand your persistence layer – Understanding your persistence layer early on is key. If multiple-write regions are not a possibility, alternative approaches will be required. Adopt Infrastructure as Code – The ability to define your cloud infrastructure via code is critical to eliminate toil and increase the ability to adopt additional regions, or even zones. Use containerization – The underlying service is best when containerized. Build the container you wish to deploy during the continuous integration stage and scan for vulnerabilities within every layer of the container for added safety. Reduce time to deploy – Get into the habit of releasing often, as it only makes your team stronger. Establish SLOs and synthetics – Take the time to set SLOs for your service and write synthetic tests to constantly measure your service — across every environment. Automate deployments – Leverage observability during the CD stage to deploy when a merge-to-main event occurs. If a dev deploys and no alerts are emitted, move on to the next environment and continue all the way to production. Conclusion It's important to understand the limitations of the platform where your services are running. Leveraging a single region offered by your cloud provider is only successful when there are zero region-wide outages. Based upon prior history, this is no longer good enough and is certain to happen again. No cloud provider is ever going to be 100% immune from a region-wide outage. A better approach is to utilize the n + 2 rule and increase the number of regions your service is running in by two additional regions. In taking this approach, the service will still be able to respond to customer requests in the event of not only one regional outage but also any form of outage in a second region where the service is running. By adopting the n + 2 approach, there is a far better chance at meeting SLAs set with your customers. Getting to this point will certainly present challenges but should also provide the opportunity to cut down (or even eliminate) toil within your organization. In the end, your customers will benefit from increased service resiliency, and your team will benefit from significant productivity gains. Have a really great day! Resources AWS Post-Event Summaries, AWS Summary of the AWS Service Event in the Northern Virginia (US-EAST-1) Region, AWS danluu/post-mortems, GitHub "Identifying and Tracking Toil Using SRE Principles" by Eric Harvieux, 2020 "Failure Recovery: When the Cure Is Worse Than the Disease" by Guo et al., 2013 This is an article from DZone's 2023 Observability and Application Performance Trend Report.For more: Read the Report

By John Vester

CORE

Use Amazon Bedrock and LangChain To Build an Application To Chat With Web Pages

Conversational interaction with large language model (LLM) based solutions (for example, a chatbot) is quite common. Although production grade LLMs are trained using a huge corpus of data, their knowledge base is inherently limited to the information present in the training data, and they may not possess real-time or the most up-to-date knowledge. Here is an example: This is perfectly acceptable. But, the real problem is "hallucination" wherein LLMs may generate inaccurate information, while sounding confident. Rather than having an open-ended conversation, it's good to narrow down the scope by providing additional context/information that's required to solve the problem or answer our questions. For example, instead of asking about Amazon Bedrock, one can provide a link to the documentation page (for instance General guidelines for Amazon Bedrock LLM users) and ask specific queries. The same can be done by Q&A over data in text or PDF documents. This can be achieved in many ways. The easiest one is to pass on the information to the LLM directly. But there are other popular techniques such as RAG (Retrieval Augmented Generation) that involve accessing data from external systems, Typically, this is done by combining Semantic search (with Vector databases). In this blog, we will explore the simple way (and leave the RAG technique for a future post). A framework like LangChain can simplify this for us since it provides abstractions to use the appropriate prompt (which can be customized), load data from sources (documents, web links) and inject it (as context) with the question/prompt. One of the previous blogs provided an introduction to using langchaingo with Amazon Bedrock, which is a fully managed service that makes base models from Amazon and third-party model providers (such as Anthropic, Cohere, and more) accessible through an API. It also walked you through how to extend langchaingo to work with the Anthropic Claude (v2) model in Amazon Bedrock. We will continue to build on that foundation and reuse the implementation. By the end, you will have a web application deployed to AWS App Runner that provides a fast, simple, and cost-effective way to deploy from source code or a container image directly to a scalable and secure web application. You can then use it to ask questions based on the content of a link/URL of your choice! The code is available on GitHub Application Overview The application is written in Go, but the concepts apply to any other language you might choose. As mentioned before, it uses langchaingo as the framework to interact with the Anthropic Claude (v2) model on Amazon Bedrock. The web app uses the Go embed package to serve the static file for the frontend part (HTML, JavaScript and CSS) from directly within the binary. To make sure that the contents of the link are included as context for the LLM, the applications uses the LoadStuffQA chain along with a prompt. You can refer to the application code here The chain is initialized in the init function: Go func init() { var err error region := os.Getenv("AWS_REGION") if region == "" { region = defaultRegion } llm, err = claude.New(region) chain = chains.LoadStuffQA(llm) } Note that the line llm, err = claude.New(region) comes from the langchaingo-amazon-bedrock-llm project that provides Amazon Bedrock extension for LangChain Go. The user can provide a link that serves as the source of information that they can ask questions. The content from the link will be represented as a Document. LangChain makes it easy to fetch data from various sources. In this case we fetch HTML content from a web URL (we will use AWS documentation as an example), but the same can be done for a text file or a PDF doc. The HTTP handler loadData invokes getDocsFromLink function that loads HTML data from the web link. The bulk of the work is done by this line - docs, err := documentloaders.NewHTML(resp.Body).Load(context.Background()) that combines NewHTML and Load functions to get the job done. Go func loadData(w http.ResponseWriter, r *http.Request) { //... details omitted } //... func getDocsFromLink(link string) ([]schema.Document, error) { resp, err := http.Get(link) defer resp.Body.Close() docs, err := documentloaders.NewHTML(resp.Body).Load(context.Background()) return docs, nil } Once contents of the user provided link are loaded, they can start asking questions. This is handled by the chat HTTP handler shown below (error handling and other parts omitted for brevity): Go func chat(w http.ResponseWriter, req *http.Request) { body, err := io.ReadAll(req.Body) chatMessage := string(body) answer, err := chains.Call(context.Background(), chain, map[string]any{ "input_documents": docs, "question": chatMessage, }, chains.WithMaxTokens(2048)) w.Write([]byte(answer["text"].(string))) } The user message (question) is fed as an input to the chains.Call function along with the content from the web URL/link. The LLM response is returned from the handler which is then rendered by the frontend to the user. Deploy the Application and Start Chatting... As a prerequisite, make sure you have Go, AWS CDK and Docker installed. You can easily deploy the entire solution with CDK. You can refer to the CDK code on GitHub Clone this GitHub repository, change to the right directory and start the deployment with cdk deploy. Shell git clone https://github.com/build-on-aws/amazon-bedrock-apprunner-chatterdox-webapp/ cd amazon-bedrock-apprunner-chatterdox-webapp/cdk export DOCKER_DEFAULT_PLATFORM=linux/amd64 cdk deploy This will start creating the AWS resources required for the application. You can keep track of the progress in the terminal or navigate to AWS console: CloudFormation > Stacks > ChatterdoxStack Once all the resources are created, you can try out the application. You should have: App Runner service - this is the web application App Runner Instance (IAM) role - this allows access to Amazon Bedrock Once complete, you should get a confirmation along with the values for the App Runner service endpoint. To access the application, enter the App Runner service endpoint URL in your web browser to navigate to the website. Start by entering a link to a page. For example, the NoSQL design for DynamoDB documentation page - AWS Documentation Once a valid link is provided (click Submit), the chat area will be enabled. You can now start asking questions about the page contents (enter a message in the text box and click Send). You can continue the chat conversation by asking additional questions. Conclusion Although this was a relatively simple application, it's evident how LangChain abstracted the complexity by providing an easy way to combine the following: Fetching data from the source (web link), Adding it as a context along with the prompt, and, Invoking the LLM Once the core functionality was ready, we were able to expose it as a user-facing web app on App Runner. This approach works quite well for single-page content or small documents. If you try to pass in a lot of content with the prompt, you will likely run into token limit constraints. To mitigate this, you can use other chains such as MapReduceDocuments, MapRerankDocuments etc. Other approach includes using RAG, which might be covered in a different blog. Until then, Happy Building!

By Abhishek Gupta

CORE

How to Deploy GitLab on AWS EC2 With Walrus

Walrus, the open-source application management platform, equips your team with templates designed to optimize best practices. In this article, we'll walk you through the process of creating an AWS GitLab template and deploying a GitLab server on an AWS EC2 instance. Prerequisites A GitHub or GitLab Repository for storing the template. Walrus installed. Create a Repository on GitHub Create a new repository on GitHub of your own. Here we use the repository demo. Clone the repository to your local machine. Go git clone git@gitlab.com:seal-eyod/gitlab-on-aws.git Create Template Files Go to the cloned repository directory. Go cd gitlab-on-aws Create files in the directory as follows: Go - gitlab-on-aws - main.tf - outputs.tf - variables.tf - README.md The main.tf file defines the resources to be created. Here we define the resource for the template to create an AWS EC2 instance and run a Gitlab server on it. Go data "aws_ami" "ubuntu" { most_recent = true filter { name = "name" values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"] } filter { name = "virtualization-type" values = ["hvm"] } owners = ["099720109477"] # Canonical } data "aws_security_group" "selected" { name = var.security_group_name } data "aws_subnets" "selected" { filter { name = "vpc-id" values = [data.aws_security_group.selected.vpc_id] } } resource "aws_instance" "gitlab" { ami = data.aws_ami.ubuntu.id instance_type = var.instance_type subnet_id = data.aws_subnets.selected.ids.0 vpc_security_group_ids = [data.aws_security_group.selected.id] key_name = var.key_name user_data = <<-EOF #!/bin/bash set -ex; public_ip=$(curl http://169.254.169.254/latest/meta-data/public-ipv4) curl -fsSL https://get.docker.com | bash && sudo usermod -aG docker ubuntu docker run -d --privileged --restart=always -p 80:80 -p 443:443 \ -e GITLAB_ROOT_PASSWORD="${var.gitlab_root_password}" \ "${var.gitlab_image}" EOF tags = { "Name" = "${var.gitlab_metadata_application_instance_name}-gitlab" } root_block_device { volume_size = var.disk_size } } resource "null_resource" "gitlab_health_check" { depends_on = [ aws_instance.gitlab, ] triggers = { always_run = timestamp() } provisioner "local-exec" { command = "for i in `seq 1 100`; do curl -k -s $ENDPOINT >/dev/null && exit 0 || true; sleep 5; done; echo TIMEOUT && exit 1" interpreter = ["/bin/sh", "-c"] environment = { ENDPOINT = "http://${aws_instance.gitlab.public_ip}" } } } The variables.tf file defines the variables used in the template. Walrus will use the variables to generate the form for users to fill in. Walrus uses the @label and @group annotations to define the labels and groups of the variables. The optional @options annotation is used to define the dropdown options of the variable, if the @options annotation is not defined, the variable will be displayed as a text box in the form. More details about the annotations. In this example, we define two groups: Basic and AWS. It will be displayed as two tabs in the form when creating a service using this template. Go # @group "Basic" variable "gitlab_image" { type = string description = "gitlab image" default = "gitlab/gitlab-ce" } # @group "Basic" variable "gitlab_root_password" { type = string description = "gitlab root password" default = "seal123456" sensitive = true } # @group "AWS" # @options ["t3.medium", "c5.xlarge"] variable "instance_type" { type = string description = "Instance type" default = "t3.medium" } # @group "AWS" variable "disk_size" { type = number description = "Root disk size in GiB" default = 50 } # @group "AWS" variable "key_name" { type = string description = "AWS key name" default = "xueying" } # @group "AWS" variable "security_group_name" { type = string description = "Security group Name" default = "all-open" } # @hidden variable "gitlab_metadata_application_instance_name" { type = string description = "gitlab metadata application instance name." default = "bar" } The outputs.tf file defines the outputs of the template which will be displayed to the user after the service is created. The outputs of the template of a service could also be referenced by other services. In this example, we define an output gitlab_url which is the URL of the Gitlab instance. Go output "gitlab_url" { description = "The URL of the GitLab instance" value = "http://${aws_instance.gitlab.public_ip}" } The README.md file is the description of the template. It will be displayed to the user when creating a service using this template. Here we can use the tool terraform-docs to generate the description of the template. You need to follow the document of the project and install the tool on your laptop, and run the following command to generate README.md file for the template. Go terraform-docs markdown . > README.md The generated README.md file is as follows: Go # Gitlab on AWS This is a terraform module that will create a Gitlab instance on AWS. ## Providers | Name | Version | |------|---------| | aws | n/a | ## Inputs | Name | Description | Type | Default | Required | |-------------------------------------------|--------------------------------------------|:--------:|:--------------------:|:--------:| | gitlab_image | Gitlab image | `string` | `"gitlab/gitlab-ce"` | no | | gitlab_root_password | Gitlab root password | `string` | `"seal123456"` | no | | instance\_type | Instance type | `string` | `"t3.medium"` | no | | disk\_size | Root disk size in GiB | `number` | `50` | no | | security\_group\_name | Security group Name | `string` | `"all-open"` | no | | gitlab_metadata_application_instance_name | gitlab metadata application instance name. | `string` | `"bar"` | no | ## Outputs | Name | Description | |------------|-------------| | gitlab_url | Gitlab URL | Commit and Tag Version Go git add . git commit -m "add template files" git push -u origin main Create a tag for the template version. Go git tag v0.0.1 git push --tags Create a Template on Walrus Open Walrus in your browser and log in. Navigate to the Template tab within the Operations Hub and craft a new template by selecting the one we've recently created.For reference, let's name this template gitlab-on-aws. After Walrus syncs the template, you can see the template in Operations Hub. Once the import task is finished, you'll find the template proudly showcased in the list. Take note that the template boasts two versions: v0.0.1 and v0.0.2. Deploy GitLab Server on AWS 1. Add AWS Cloud Provider in the Connectors tab within the Operations Hub. 2. Add Connector to the Environment. 3. Generate a service utilizing the gitlab-on-aws template. The form groups and labels are dynamically generated based on the annotations specified in the template variables we've previously defined. Notably, there are two groups, and the input variables are outlined in the variables.tf file of the template.To ensure network traffic management, an EC2 instance requires a security group. We can create a security group named all-open to permit all network traffic. For enhanced security, you can customize the group's rules as needed. 4. After the security group is created in your AWS target region, you can save and apply the service. Once the deployment process is complete, the Gitlab instance is successfully provisioned on AWS. The GitLab URL will be shown in the outputs. Once you have the URL, you can access the GitLab server. Conclusion You've just witnessed the entire process of template creation in Walrus, a tool that can significantly streamline the deployment process. Furthermore, Walrus is compatible with a plethora of mature templates from the Terraform community.

By Alex Chen

What Is CI/CD? Beginner’s Guide To Continuous Integration and Deployments

CI/CD Explained CI/CD stands for continuous integration and continuous deployment and they are the backbone of modern-day DevOps deployment practices. CI/CD is the process that allows software to be continuously built, tested, automated, and delivered in a continuous cadence. In a rapidly developing world with increasing requirements, the development and integration process need to be at the same speed to ensure business delivery. What Is Continuous Integration? CI, or continuous integration, works on automated tests and builds. Changes made by developers are stored in a source branch of a shared repository. Any changes committed to this branch go through builds and testing before merging. This ensures consistent quality checks of the code that gets merged. As multiple developers work on different complex features, the changes are made to a common repository with changes merged in increments. Code changes go through pre-designed automated builds. Code is tested for any bugs making sure it does not break the current workflow. Once all the checks, unit tests, and integration tests are cleared, the code can be merged into the source branch. The additional checks ensure code quality and versioning makes it easier to track any changes in case of issues. Continuous integration has paved the path for rapid development and incremental merging making it easier to fulfill business requirements faster. What Is Continuous Delivery? CD, or continuous deployment, works on making the deployment process easier and bridges the gap between developers, operations teams, and business requirements. This process automatically deploys a ready, tested code to the production environment. But, through the process of automating the effort taken for deployment, frequent deployments can be handled by the operations team. This enables more business requirements to be delivered at a faster rate. CD can also stand for continuous delivery, which includes the testing of code for bugs before it is deployed to the pre-production environment. Once tests are complete and bugs are fixed, they can then be deployed to production. This process allows for a production-ready version of the code to always be present with newly tested changes added in continuous increments. As code gets merged in short increments, it is easy to test and scan for bugs before getting merged in the pre-production and production environments. Code is already scanned in the automated pipelines before getting handed to the testing teams. This cycle of repeated scanning and testing helps reduce issues and also helps in faster debugging. Continuous integration allows for continuous delivery, which is followed by continuous deployment. Figure 1: CI/CD What Is the Difference Between Continuous Integration (CI) and Continuous Deployment (CD)? The biggest difference between CI and CD is that CI focuses on prepping and branching code for the production environment, and CD focuses on automation and ensuring that this production-ready code is released. Continuous integration includes merging the developed features into a shared repository. It is then built and unit-tested to make sure it is ready for production. This stage also includes UI testing if needed. Once a deployment-ready code version is ready we can move to the next phase, i.e., continuous deployment. The operations team then picks the code version for automated tests to ensure a bug-free code. Once the functionality is tested, the code is merged into production using automated deployment pipelines. Hence, both CI and CD work in sync to deliver at a rapid frequency with reduced manual efforts. Fundamentals of Continuous Integration Continuous integration is also an important practice when it comes to Agile software development. Code changes are merged into a shared repository and undergo automated tests and checks. This helps in identifying possible issues and bugs at an earlier stage. As multiple developers may work on the same code repository, this step ensures there are proper checks in place that test the code, validate the code, and get a peer review before the changes get merged. Read DZone's guide to DevOps code reviews. Continuous integration works best if developers merge the code in small increments. This helps keep track of all the features and possible bug fixes that get merged into the shared code repository. Fundamentals of Continuous Deployment Continuous deployment enables frequent production deployments by automating the deployment process. As a result of CI, a production-ready version of code is always present in the pre-production environment. This allows developers and testers alike to run automated integration and regression tests, UI tests, and more in the staging environment. Once the tests are successfully run and the expected criteria are met, the code can be easily pushed to a live environment by either the Development or Operations teams. Advantages and Disadvantages of CI/CD Implementation CI/CD implementation can have both pros and cons. Having a faster deployment cycle can also lead to other problems down the line. Below are a few benefits and drawbacks of CI/CD implementation. Advantages of CI/CD Disadvantages of CI/CD Automated tests and builds: Automated tests and builds take the strain off of the developers and testers and bring consistency to the code. This is an important step in the CI/CD world. Rapid deployments where they are not needed: There might be businesses that do not appreciate rapid change. A faster rollout period may not be suitable for the business model. Deep testing before deployment can also ensure fewer bugs and problems down the line. Better code quality: Every commit goes through certain predefined checks before getting merged into the main branch. This ensures consistent code quality and minimal bugs or plausible issues to be detected at an earlier stage. Monitoring: Faster rollout leads to less deep testing. Continuous monitoring is important in such cases to quickly identify any issues as they come. Hence monitoring is a crucial part of a CI/CD process. Faster rollout: Automated deployment leads to faster rollout. More features can be released to the end user in smaller chunks. Business requirements are delivered faster keeping up with increasing demands and changes. Issues and fixes: No thorough testing may lead to escaped corner cases also known as bugs. Some cases may be left unnoticed for longer periods. Better transparency: As multiple developers work on a common repository, it is easier to track the changes and maintain transparency. Various version management tools help track history and versions with additional checks before merging to ensure no overlaps or conflicts in the changes. Dependency management: A change made in one microservice can cause a cascading chain of issues. Orchestration is required in such cases to ensure less breakage due to any change added in one part of the service. Faster rollbacks and fixes: As the history and versioning are tracked, it is easier to roll back any change(s) that are causing issues in the application. Any fixes made can also be deployed to production faster. Managing resources: With continuous changes being made development and operations teams need to also keep up with the continuous requirements and maintenance of pipelines. Popular CI/CD Tools Below are a few common CI/CD tools that make life easier for the development teams: AWS AWS, or Amazon Web Services, is a popular DevOps and CI/CD tool. Similarly to Azure, it provides the infrastructure needed for a CI/CD implementation. DZone's previously covered building CI/CD Pipelines with AWS. Azure DevOps Azure DevOps services by Microsoft provide a suite of services to run a CI/CD implementation. From continuous builds to deployments, Azure DevOps handles everything in one platform. Bitbucket Bitbucket is a cloud version system developed by Atlassian. Bitbucket Pipelines is a CI tool that is easily integrated with Bitbucket. GitLab In addition to providing all features of GitHub, GitLab also provides a complete CI/CD setup. From wiki, branch management, versioning, and builds, to deployment, GitLab provides an array of services. Jenkins Jenkins is built using Java and is a commonly used CI/CD tool. It is an open-source continuous integration tool. It is easy to plug in and helps manage builds, automated checks, and deployments. It is very handy for real-time testing and reporting. Learn how to setup a CI/CD pipeline from scratch. Alternative Comparisons: Jenkins VS GitLab and Jenkins VS Bamboo. Conclusion As said by Stan Lee, "With great power comes great responsibility." CI/CD provides a powerful array of tools to enable rapid development and deployment of features to keep up with business requirements. CI/CD is a constant process enabling continuous change. Once it is adapted accurately, teams can easily deal with new requirements, and fix and rollout any bugs or issues as they come. CI/CD is also often used in DevOps practices. Review these best practices further by reading this DevOps Tutorial. With new tools available in the markets adoption or migration to CI/CD has become easier than before. However one needs to assess if CI/CD is the right approach depending on their business use case and available resources. Please share your experience with CI/CD and your favorite CI/CD tool in the comments below.

By Bhagyashree Nigade

CORE

Feature Flags for CI/CD

What Are Feature Flags? Feature flags are a software development technique that help to turn certain functionality on and off during runtime without the deployment of code. For both feature flags and modern development in general, it is always focused on the race to deploy software faster to the customers. However, it is not only that the software has to reach the customer faster, but it also has to be done with lesser risk. Feature flags are a potent tool (set of patterns or techniques) that can be used to reinforce the CI/CD pipeline by increasing the velocity and decreasing the risk of the software deployed to the production environment. Feature flags are also known as feature bits, feature flippers, feature gates, conditional features, feature switches, or feature toggles (even though the last one may have a subtle distinction which we will see a bit later). Related: CI/CD Software Development Patterns. Feature flags help to control and experiment over the feature lifecycle. They are a DevOps best practice that are often observed in distributed version control systems. Even incomplete features can be pushed to production because feature flags help to separate deployment from release. Earlier, the lowest level of control was at the deployment level. Now, feature flags move the lowest level of control to each individual item or artifact (feature, update, or bug fixes) that’s in production which makes it even more granular than the production deployment. Feature Flags Deployment Feature flags can be implemented as: Properties in JSON files or config maps A feature flag service Once we have a good use case (e.g., show or hide a button to access the feature, etc.) to use the feature flags, we will have to see where to implement the flag (frontend, backend, or a mix of both). With a feature flag service, we must install the SDK and create and code the flags within the feature flag platform and then we wrap the new paths of the code or new features within the flags. This enables the feature flags, and the new feature can be toggled on or off through a configuration file or a visual interface as part of the feature flagging platform. We also set up the flag rules so that we may manage various scenarios. You may use different SDKs depending on the language of each service used. This also helps product managers to run some experiments on the new features. After the feature flags are live, we must manage them, which is also known as feature flag management. After the feature flag has served its purpose or no longer serving its purpose, we need to remove them to avoid the technical debt of having the feature flags being left in the codebase. This can also be automated within the service platform. DZone's previously covered how to trigger pipelines with jobs in Azure DevOps. Feature Toggles vs. Feature Flags From an objective perspective, there may be no specific difference between a feature toggle and a feature flag, and for all practical purposes, you may consider them as similar terms. However, feature toggles may carry a subtle connotation of a heavier binary "on/off" for the whole application, whereas feature flags could be much lighter and can manage ramp-up testing more easily. For example, a toggle could be an on/off switch (show ads on the site, don't show ads) and it could be augmented by a flag like (Region1 gets ads from provider A, Region2 gets ads from provider B). Toggling may turn off all the ads, but a feature flag might be able to switch from provider B to provider D. Types of Feature Flags There are different types of feature flags based on various scenarios and in this section, we will look at some of the important types of flags. The fundamental benefits of the feature flags are their ability to ship alternative code pathways within a deployable environment and the ability to choose specific pathways at runtime. Different user scenarios indicate that this benefit can be applied in multiple modes in different contexts. Two important facets that can be applied to categorize the types of feature flags are longevity (how long the flag will be alive), and dynamism (what is the frequency of the switching decision), even though we may also have other factors for consideration. Release Flags For teams practicing continuous delivery, release flags enable faster shipping velocity for the customer and trunk-based development. These flags allow incomplete and untested code pathways which can be shipped to production as latent code. The flag also facilitates the continuous delivery principle of separating the feature release from the deployment of code. These flags are very useful for product managers to manage the delivery of the product to the customers as per the requirements. Operational Flags Operational flags are used for managing the operational aspects of the system’s behavior. If we have a feature that is being rolled out and it has unclear performance issues, we should be able to quickly disable/degrade that feature in production, when required. These are generally short-lived flags but we also have some long-lived flags, a.k.a. kill switches, which can help in degrading non-vital system functionality in production when there are heavy loads. These long-lived flags may also be seen as a manually managed circuit breaker that can be triggered if we cross the set thresholds. The flags are very useful to quickly respond during production issues and they also need to be re-configured quickly so that they are ready for the next set of issues that may occur. Experimental Flags Experimental flags are generally used in A/B or multivariate testing. Users are placed in cohorts and at runtime, the toggle router will send different users across different code pathways based on which cohort they belong to. By tracking the aggregate behavior of the different cohorts, the effect of different code pathways can be observed, and this can help to make data-driven optimizations to the application functionalities like search variables that have the most impact on the user. These flags need to operate with the same configuration for a specific time period (as decided by traffic patterns and other factors so that the results of the experiment are not invalidated) in order to generate statistically significant results. However, since this may not be possible in a production environment where each request may be from a different user, these flags are highly dynamic and need to be managed appropriately. Customer/Permission Flags Customer/permissions flags restrict or change the type of features or product experience that a user gets from a product. One example of this is a premium feature that only some users get based on a subscription. Martin Fowler adds that the technique of turning on new features for a set of internal or beta users as a champagne brunch – an early instance of tasting your own medicine or drinking your own champagne. These flags are quite long-lived flags (many years) compared to other flags. Additionally, as the permissions are specific to a user, switching decisions is generally on a per-request basis, and hence, these flags are very dynamic. Feature Flags and CI/CD Feature flags are one of the important tools that helps the CI/CD pipeline to work better and deliver code faster to the customer. Continuous integration means integrating code changes from the development teams/members every few hours. With continuous delivery, the software is ready for deployment. With continuous deployment, we deploy the software as soon as it is ready, using an automated process. CI and CD are therefore observed to have great benefits because when they work in tandem, they shorten the software development lifecycle (SDLC). However, software has bugs, and delivering code continuously and quickly can rapidly turn from an asset to a liability, and this is where feature flags give us a way to enable or disable new features without a build or a deployment. In effect, they are acting as a safety net just like tests, which also act as a safety net to let us know if the code is broken. We can ship new features and turn them on or off, as required. Thus, feature flags are part of the release and rollout processes. Many engineering teams are now discussing how to implement continuous testing into the DevOps CI/CD pipeline. Implementation Techniques of Feature Flags Below are a few important implementation patterns and practices that may help to reduce messy toggle point issues. Avoiding Conditionals Generally, toggle or switch points are implemented using 'if' statements for short-lived toggles. However, for long-lived toggles or for multiple toggle points, we may use some sort of a strategy pattern to implement alternative code pathways that are a more maintainable approach. Decision Points and Decision Logic Should be Decoupled An issue with feature flags is that we may couple the toggle point (where the toggling decision is made) with the toggle router (the logic behind the decision). This can create rigidity due to the toggle points being linked/hard-wired to the feature directly and we may not be able to modify the sub-feature functionalities easily. By decoupling the decision logic from the decision point, we may be able to manage toggle scope changes more effectively. Inversion of Decision If the application is linked to the feature flagging service or platform, we again have to deal with rigidity as the application is harder to work with and think in isolation, and it also becomes difficult to test it. These issues can be resolved by applying the software design principle – inversion of control by decoupling the application from the feature flagging service. Related: Create a Release Pipeline with Azure DevOps. How Feature Flags Can Improve Release Management Some of the benefits of using feature flags for release management are: Turn on/off without deployment Test directly in production Segment the users based on different attributes Segments are users or groups of users that have some attributes tied to them like location or email ID. Be sure to group segments as collections so that feature flags are tied to specific apps (which are the web pages). Here are some benefits of feature flag service platforms for release management: Can be centrally managed On/off without modifying your properties in your apps/web pages Audit and usage data Conclusion Feature flags in conjunction with CI/CD and release management help in improving many aspects of software delivery. To name a few, these include shipping velocity and reduced time-to-market with less fear of bugs being released in production. They also introduce complexity and challenges in the code that need to be monitored and managed appropriately. In order to use feature flags effectively, it should be an organization-wide initiative and it should not be limited to a few developers only. To further your reading, learn more about running a JMeter test with Jenkins pipelines.

By Badri Srinivasan

CORE

Implementing Real-Time Datadog Monitoring in Deployments

In today's tech landscape, where application systems are numerous and complex, real-time monitoring during deployments has transitioned from being a luxury to an absolute necessity. Ensuring that all the components of an application are functioning as expected during and immediately after deployment while also keeping an eye on essential application metrics is paramount to the health and functionality of any software application. This is where Datadog steps in — a leading monitoring and analytics platform that brings visibility into every part of the infrastructure, from front-end apps to the underlying hardware. In tandem with this is Ansible, a robust tool for automation, particularly in deployment and configuration management. In this article, we will discover how Datadog real-time monitoring can be integrated into Ansible-based deployments and how this integration can be leveraged during deployments. This concept and methodology can be applied to similar sets of monitoring and deployment tools as well. Why Integrate Real-Time Monitoring in Deployments? In the ever-evolving realm of DevOps, the line between development and operations is continuously blurring. This integration drives a growing need for continuous oversight throughout the entire lifecycle of an application, not just post-deployment. Here's why integrating Datadog with your deployment processes and within your deployment scripts is both timely and essential: Immediate Feedback: One of the primary benefits of real-time monitoring during deployments is the instant feedback loop it creates. When an issue arises after deploying to a host or hosts during a rolling deployment, the real-time monitoring data can be immediately used to make a decision to pause or initiate a deployment rollback. This quick turnaround can mean the difference between a minor hiccup and a major catastrophe, especially for applications where even a 1-minute downtime can result in a substantial number of errors and lost revenue. Resource and Performance Oversight: As new features or changes are deployed, there's always the risk of inadvertently impacting performance, resource utilization, and the associated costs. With such real-time monitoring, teams can get an immediate read on how these changes affect system performance and resource utilization, thereby determining any immediate remediations necessary to ensure that users continue to have an optimal experience. Proactive Issue Resolution: Rather than reacting to problems after they've affected end-users, integrating Datadog directly into the deployment process allows teams to proactively address and prevent potential issues from snowballing into a major outage. This proactive approach can increase uptime, more stable releases, and higher overall user satisfaction. The Process of Implementing Real-Time Monitoring Into Deployment As soon as the deployment tool is triggered and the underneath scripts start to execute, we pre-determine an ideal place to perform monitoring checks based on our application needs and send one or more Datadog API requests querying either metrics, monitor data or any other information that helps us determine the health of deployments and the application in general. Then, we add logic in our scripts so that the API response from Datadog can be parsed and an appropriate decision can be made whether to roll forward to the next group or not. For example, if we determine that there are too many errors and the monitors are firing, we parse that information accordingly and decide to abort the deployment from going forward to the next group, thereby reducing the blast radius of a potential production incident. The below flowchart is a representation of how the process typically works. However, the stages need to be tweaked based on your application needs. Deployment flow with integrated monitoring. Utilizing Datadog and Its API Interface for Real-Time Queries Beyond the foundational monitoring capabilities, Datadog offers another pivotal advantage that empowers DevOps teams: its robust API interface. This isn't just a feature; it's a transformative tool. With the ability to query metrics, traces, and logs programmatically, teams can dynamically integrate Datadog deeper into their operations. This allows for tailored monitoring configurations, automated alert setups, and on-the-fly extraction of pertinent data. This real-time querying isn't just about fetching data; it's about informing deployment decisions, refining application performance, and creating a more synergetic tech ecosystem. By leveraging Datadog's API, monitoring becomes not just a passive observation but an active driver of optimized deployment workflows. Datadog monitors are tools that keep an eye on your tech setup, checking things like performance and errors. They give quick updates, so if something goes wrong, you get alerted right away. This helps teams fix problems faster and keep everything running smoothly. In this implementation, we're going to query the monitor's data to check for any alerts that are firing. Alternatively, we can also query metrics and other similar data that help determine the health of the application. The following is a sample example to fetch the details of a particular monitor (obtained from Datadog's API reference sheet). Sample Curl request to a Datadog API endpoint. Using Ansible as an Example in Deployment Automation As we delve deeper into sophisticated monitoring with tools like Datadog, it's essential to understand the deployment mechanisms that underpin these applications. We're going to use Ansible in our case as an example. This open-source automation tool stands out for its simplicity and power. Ansible uses declarative language to define system configurations, making it both human-readable and straightforward to integrate with various platforms and tools. In the context of deployments, Ansible ensures consistent and repeatable application rollouts, mitigating many of the risks associated with manual processes. When coupled with real-time monitoring solutions like Datadog, Ansible not only deploys applications but also guarantees they perform optimally post-deployment. This synergy between deployment automation and real-time monitoring underscores a robust, responsive, and resilient deployment ecosystem. The code snippets below show how we can implement Datadog querying in Ansible. Querying monitors with a tag called 'deployment_priority: blocker' as an example: Monitor querying implemented in Ansible. Next, parsing the status of all such monitors returned from Datadog and making a decision whether to abort or continue to the next host or group of deployments. Iterative monitor parsing and decision-making. We now have the capability to parse Datadog monitoring information and make informed decisions in our deployment process. This concludes the implementation portion. Summary The intersection of deployment automation and real-time monitoring is where modern DevOps truly shines. In this exploration, we've used Ansible as a prime example of the power of deployment tools, emphasizing its capacity to deliver consistent and reliable rollouts. When combined with the granular, real-time insights of a platform like Datadog, we unlock operational efficiency and reliability. As the tech landscape continues to evolve, tools like Ansible and Datadog stand as a testament to the potential of integrated, intelligent DevOps practices. Whether you're a seasoned DevOps professional or just beginning your journey, there's immense value in understanding and employing such synergies for a future-ready and resilient tech ecosystem.

By Prithvish Kovelamudi

Serverless Image Generation Application Using Generative AI on AWS

Whether it's crafting personalized content or tailoring images to user preferences, the ability to generate visual assets based on a description is quite powerful. But text-to-image conversion typically involves deploying an end-to-end machine learning solution, which is quite resource-intensive. What if this capability was an API call away, thereby making the process simpler and more accessible for developers? This tutorial will walk you through how to use AWS CDK to deploy a Serverless image generation application implemented using AWS Lambda and Amazon Bedrock, which is a fully managed service that makes base models from Amazon and third-party model providers (such as Anthropic, Cohere, and more) accessible through an API. Developers can leverage leading foundation models through a single API while maintaining the flexibility to adopt new models in the future. The solution is deployed as a static website hosted on Amazon S3 accessible via an Amazon CloudFront domain. Users can enter the image description which will be passed on to a Lambda function (via Amazon API Gateway) which in turn will invoke the Stable Diffusion model on Amazon Bedrock to generate the image. The entire solution is built using Go - this includes the Lambda function (using the aws-lambda-go library) as well as the complete solution deployment using AWS CDK. The code is available on GitHub. Prerequisites Before starting this tutorial, you will need the following: An AWS Account (if you don't yet have one, you can create one and set up your environment here) Go (v1.19 or higher) AWS CDK AWS CLI Git Docker Clone this GitHub repository and change it to the right directory: git clone https://github.com/build-on-aws/amazon-bedrock-lambda-image-generation-golang cd amazon-bedrock-lambda-image-generation-golang Deploy the Solution Using AWS CDK To start the deployment, simply invoke cdk deploy. cd cdk export DOCKER_DEFAULT_PLATFORM=linux/amd64 cdk deploy You will see a list of resources that will be created and will need to provide your confirmation to proceed (output shortened for brevity). Bundling asset BedrockLambdaImgeGenWebsiteStack/bedrock-imagegen-s3/Code/Stage... ✨ Synthesis time: 7.84s //.... omitted This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening). Please confirm you intend to make the following modifications: //.... omitted Do you wish to deploy these changes (y/n)? y This will start creating the AWS resources required for the application. If you want to see the AWS CloudFormation template which will be used behind the scenes, run cdk synth and check the cdk.out folder. You can keep track of the progress in the terminal or navigate to the AWS console: CloudFormation > Stacks > BedrockLambdaImgeGenWebsiteStack. Once all the resources are created, you can try out the application. You should have: The image generation Lambda function and API Gateway An S3 bucket to host the website's HTML page CloudFront distribution And a few other components (like IAM roles, permissions, S3 Bucket policy, etc.) The deployment can take a bit of time since creating the CloudFront distribution is a time-consuming process. Once complete, you should get a confirmation along with the values for the S3 bucket name, API Gateway URL, and the CloudFront domain name. Update the HTML Page and Copy It to the S3 Bucket Open the index.html file in the GitHub repo, and locate the following text: ENTER_API_GATEWAY_URL. Replace this with the API Gateway URL that you received as the CDK deployment output above. To copy the file to S3, I used the AWS CLI: aws s3 cp index.html s3://<name of the S3 bucket from CDK output> Verify that the file was uploaded: aws s3 ls s3://<name of the S3 bucket from CDK output> Now you are ready to access the website! Verify the Solution Enter the CloudFront domain name in your web browser to navigate to the website. You should see the website with a pre-populated description that can be used as a prompt. Click Generate Image to start the process. After a few seconds, you should see the generated image. Modify the Model Parameters The Stability Diffusion model allows us to refine the generation parameters as per our requirements. The Stability.ai Diffusion models support the following controls: Prompt strength (cfg_scale) controls the image's fidelity to the prompt, with lower values increasing randomness. Generation step (steps) determines the accuracy of the result, with more steps producing more precise images. Seed (seed) sets the initial noise level, allowing for reproducible results when using the same seed and settings. Click Show Configuration to edit these. Max values for cfg_steps and steps are 30 and 150, respectively. Don’t Forget To Clean Up Once you're done, to delete all the services, simply use: cdk destroy #output prompt (choose 'y' to continue) Are you sure you want to delete: BedrockLambdaImgeGenWebsiteStack (y/n)? You were able to set up and try the complete solution. Before we wrap up, let's quickly walk through some of the important parts of the code to get a better understanding of what's going the behind the scenes. Code Walkthrough Since we will only focus on the important bits, a lot of the code (print statements, error handling, etc.) has been omitted for brevity. CDK You can refer to the CDK code here. We start by creating the API Gateway and the S3 bucket. apigw := awscdkapigatewayv2alpha.NewHttpApi(stack, jsii.String("image-gen-http-api"), nil) bucket := awss3.NewBucket(stack, jsii.String("website-s3-bucket"), &awss3.BucketProps{ BlockPublicAccess: awss3.BlockPublicAccess_BLOCK_ALL(), RemovalPolicy: awscdk.RemovalPolicy_DESTROY, AutoDeleteObjects: jsii.Bool(true), }) Then we create the CloudFront Origin Access Identity and grant S3 bucket read permissions to the CloudFront Origin Access Identity principal. Then we create the CloudFront Distribution: Specify the S3 bucket as the origin. Specify the Origin Access Identity that we created before. oai := awscloudfront.NewOriginAccessIdentity(stack, jsii.String("OAI"), nil) bucket.GrantRead(oai.GrantPrincipal(), "*") distribution := awscloudfront.NewDistribution(stack, jsii.String("MyDistribution"), &awscloudfront.DistributionProps{ DefaultBehavior: &awscloudfront.BehaviorOptions{ Origin: awscloudfrontorigins.NewS3Origin(bucket, &awscloudfrontorigins.S3OriginProps{ OriginAccessIdentity: oai, }), }, DefaultRootObject: jsii.String("index.html"), //name of the file in S3 }) Then, we create the image generation Lambda function along with IAM permissions (to the function execution IAM role) to allow it to invoke Bedrock operations. function := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("bedrock-imagegen-s3"), &awscdklambdagoalpha.GoFunctionProps{ Runtime: awslambda.Runtime_GO_1_X(), Entry: jsii.String(functionDir), Timeout: awscdk.Duration_Seconds(jsii.Number(30)), }) function.AddToRolePolicy(awsiam.NewPolicyStatement(&awsiam.PolicyStatementProps{ Actions: jsii.Strings("bedrock:*"), Effect: awsiam.Effect_ALLOW, Resources: jsii.Strings("*"), })) Finally, we configure Lambda function integration with API Gateway, add the HTTP routes, and specify the API Gateway endpoint, S3 bucket name, and CloudFront domain name as CloudFormation outputs. functionIntg := awscdkapigatewayv2integrationsalpha.NewHttpLambdaIntegration(jsii.String("function-integration"), function, nil) apigw.AddRoutes(&awscdkapigatewayv2alpha.AddRoutesOptions{ Path: jsii.String("/"), Methods: &[]awscdkapigatewayv2alpha.HttpMethod{awscdkapigatewayv2alpha.HttpMethod_POST}, Integration: functionIntg}) awscdk.NewCfnOutput(stack, jsii.String("apigw URL"), &awscdk.CfnOutputProps{Value: apigw.Url(), Description: jsii.String("API Gateway endpoint")}) awscdk.NewCfnOutput(stack, jsii.String("cloud front domain name"), &awscdk.CfnOutputProps{Value: distribution.DomainName(), Description: jsii.String("cloud front domain name")}) awscdk.NewCfnOutput(stack, jsii.String("s3 bucket name"), &awscdk.CfnOutputProps{Value: bucket.BucketName(), Description: jsii.String("s3 bucket name")}) Lambda Function You can refer to the Lambda Function code here. In the function handler, we extract the prompt from the HTTP request body and the configuration from the query parameters. Then it's used to call the model using bedrockruntime.InvokeModel function. Note the JSON payload sent to Amazon Bedrock is represented by an instance of the Request struct. The output body returned from the Amazon Bedrock Stability Diffusion model is a JSON payload that is converted into a Response struct that contains the generated image as a base64 string. This is returned as an events.APIGatewayV2HTTPResponse object along with CORS headers. func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) { prompt := req.Body cfgScaleF, _ := strconv.ParseFloat(req.QueryStringParameters["cfg_scale"], 64) seed, _ := strconv.Atoi(req.QueryStringParameters["seed"]) steps, _ := strconv.Atoi(req.QueryStringParameters["steps"]) payload := Request{ TextPrompts: []TextPrompt{{Text: prompt}, CfgScale: cfgScaleF, Steps: steps, } if seed > 0 { payload.Seed = seed } payloadBytes, err := json.Marshal(payload) output, err := brc.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{ Body: payloadBytes, ModelId: aws.String(stableDiffusionXLModelID), ContentType: aws.String("application/json"), }) var resp Response err = json.Unmarshal(output.Body, &resp) image := resp.Artifacts[0].Base64 return events.APIGatewayV2HTTPResponse{ StatusCode: http.StatusOK, Body: image, IsBase64Encoded: false, Headers: map[string]string{ "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Methods": "POST,OPTIONS", }, }, nil } //request/response model type Request struct { TextPrompts []TextPrompt `json:"text_prompts"` CfgScale float64 `json:"cfg_scale"` Steps int `json:"steps"` Seed int `json:"seed"` } type TextPrompt struct { Text string `json:"text"` } type Response struct { Result string `json:"result"` Artifacts []Artifact `json:"artifacts"` } type Artifact struct { Base64 string `json:"base64"` FinishReason string `json:"finishReason"` } Conclusion In this tutorial, you used AWS CDK to deploy a serverless image generation solution that was implemented using Amazon Bedrock and AWS Lambda and was accessed using a static website on S3 via a CloudFront domain. If you are interested in an introductory guide to using the AWS Go SDK and Amazon Bedrock Foundation Models (FMs), check out this blog post. Happy building!

By Abhishek Gupta

CORE

Dockerizing ML Models: A Deployment Guide

In the rapidly evolving domain of machine learning (ML), the ability to seamlessly package and deploy models is as crucial as the development of the models themselves. Containerization has emerged as the game-changing solution to this, offering a streamlined path from the local development environment to production. Docker, a leading platform in containerization, provides the tools necessary to encapsulate ML applications into portable and scalable containers. This article delves into the step-by-step process of containerizing a simple ML application with Docker, making it accessible to ML practitioners and enthusiasts alike. Whether you're looking to share your ML models with the world or seeking a more efficient deployment strategy, this tutorial is designed to equip you with the fundamental skills to transform your ML workflows using Docker. Docker and Containerization Docker is a powerful platform that has revolutionized the development and distribution of applications by utilizing containerization, a lightweight alternative to full-machine virtualization. Containerization involves encapsulating an application and its environment — dependencies, libraries, and configuration files — into a container, which is a portable and consistent unit of software. This approach ensures that the application runs uniformly and consistently across any infrastructure, from a developer's laptop to a high-compute cloud-based server. Unlike traditional virtual machines that replicate an entire operating system, Docker containers share the host system's kernel, making them much more efficient, fast to start, and less resource-intensive. Docker's simple and straightforward syntax hides the complexity often involved in deployment processes, streamlining the workflow and enabling a DevOps approach to the lifecycle management of the software development process. Tutorial Below is a step-by-step tutorial that will guide you through the process of containerizing a simple ML application using Docker. Setting Up Your Development Environment Before you start, make sure you have Docker installed on your machine. If not, you can download it from the Docker website. Creating a Simple Machine Learning Application For this tutorial, let's create a simple Python application that uses the Scikit-learn library to train a model on the Iris dataset. Create a Project Directory Open your terminal or command prompt and run the following: Shell mkdir ml-docker-app cd ml-docker-app Set up a Python Virtual Environment (Optional, but Recommended) Shell python3 -m venv venv On Windows use venv\Scripts\activate Create a requirements.txt File List the Python packages that your application requires. For our simple ML application: Shell scikit-learn==1.0.2 pandas==1.3.5 Create the Machine Learning Application Script Save the following code into a file named app.py in the ml-docker-app directory: Python from sklearn import datasets from sklearn.model_selection import train_test_split from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import accuracy_score import joblib # Load dataset iris = datasets.load_iris() X = iris.data y = iris.target # Split dataset into training set and test set X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3) # Create a Gaussian Classifier clf = RandomForestClassifier() # Train the model using the training sets clf.fit(X_train, y_train) # Predict the response for test dataset y_pred = clf.predict(X_test) # Model Accuracy, how often is the classifier correct? print(f"Accuracy: {accuracy_score(y_test, y_pred)}") # Save the trained model joblib.dump(clf, 'iris_model.pkl') Install the Dependencies Run the following command to install the dependencies listed in requirements.txt: Shell pip install -r requirements.txt Run Your Application Run your application to make sure it works: Shell python3 app.py You should see the accuracy of the model printed to the console and a file named iris_model.pkl created, which contains the trained model. This script provides an end-to-end flow of a very basic machine learning task: loading data, preprocessing it, training a model, evaluating the model, and then saving the trained model for future use. Containerize the Application With Docker Create a ‘Dockerfile’ In the root of your ml-docker-app directory, create a file named Dockerfile with the following content: Python # Use an official Python runtime as a parent image FROM python:3.9-slim # Set the working directory in the container WORKDIR /usr/src/app # Copy the current directory contents into the container at /usr/src/app COPY . . # Install any needed packages specified in requirements.txt RUN pip install --no-cache-dir -r requirements.txt # Run app.py when the container launches Build the Docker Image Run the following command in your terminal to build the Docker image: Shell docker build -t ml-docker-app . Run the Docker Container Once the image is built, run your application in a Docker container: Shell docker run ml-docker-app If everything is set up correctly, Docker will run your Python script inside a container, and you should see the accuracy of the model outputted to your terminal, just like when you ran the script natively. Tag and Push the Container to DockerHub Log in to Docker Hub from the Command Line Once you have a Docker Hub account, you need to log in through the command line on your local machine. Open your terminal and run: Shell docker login You will be prompted to enter your Docker ID and password. Once logged in successfully, you can push images to your Docker Hub repository. Tag Your Docker Image Before you can push an image to Docker Hub, it must be tagged with your Docker Hub username. If you don’t tag it correctly, Docker will not know where to push the image. Assuming your Docker ID is a username, and you want to name your Docker image ml-docker-app, run: Shell docker tag ml-docker-app username/ml-docker-app This will tag the local ml-docker-app image as username/ml-docker-app, which prepares it to be pushed to your Docker Hub repository. Push the Image to Docker Hub To push the image to Docker Hub, use the docker push command followed by the name of the image you want to push: Shell docker push username/ml-docker-app Docker will upload the image to your Docker Hub repository. Check the Pushed Container Image on Docker Hub You can go to your Docker Hub repository and see the recently pushed image. That's it! You have successfully containerized a simple machine learning application, pushed it to Docker Hub, and made it available to be pulled and run from anywhere.

By Pavan Belagatti

CORE

Deployment

DZone's Featured Deployment Resources

Top Deployment Experts

The Latest Deployment Topics