PipeCD

Today we are excited to announce the open-source availability of PipeCD: a continuous delivery system for declarative Kubernetes, Serverless, and Infrastructure applications.PipeCD aims to provide a unified CD solution for multiple application kinds on multi-cloud that empowers engineers to deploy faster with more confidence.It is also available as a GitOps tool that enables doing deployment operations by pull request on Git.BackgroundAs one of our Developer Productivity team’s missions, we aim to empower engineers to deploy their services faster, more frequently with reliability.Martin Fowler, in his book Continuous Delivery, points out that “The biggest risk to any software effort is that you end up building something that isn’t useful. The earlier and more frequently you get working software in front of real users, the quicker you get feedback to find out how valuable it really is.”Recently, with the popularity of cloud services and the container technology, engineers have even more options in choosing the infrastructure model, the cloud services, which are most suitable for their team’s requirements.At CyberAgent, we have a large number of services from many teams where each team can have a different infrastructure model and a different cloud service. Some big projects also deploy their services on multi-cloud.This diversification leads to a problem that we are facing, lacking a robust CD system for all teams.So we decided to create a new CD system that provides a unified interface for many application kinds to improve the developer experience.Key FeaturesWhile designing PipeCD, we focused on the following 4 key features with the aim of creating a CD system that provides a good experience for both developers and operators.VisibilityVisibility is one of the most requested factors we received when surveying engineers in our company.Visibility is the ability to see what’s going on in the cluster, the ability to see how each component of the application had been deployed, the ability to know quickly why the deployment was failed.Visibility for a team leader is the ability to know the delivery performance of the team and what metrics should be improved.With PipeCD, we always strive to maximize the visibility for engineers, operators as well as team leaders. Currently, it includes:Deployment pipeline UI shows clarify what is happeningSeparate logs viewer for each individual deploymentRealtime visualization of application component and stateDeployment notifications to slack, webhook endpointsInsights show the delivery performanceIn addition, the entire state of the service is managed through Git, so you can view the whole state of the cluster and all audit logs provided by Git.AutomationAutomation reduces or removes repetitive overhead of frequent releases. So maximizing automation helps to minimize human error during the deployment process, as well as reduce the amount of work engineers need to do.PipeCD has the following automated functionalities:Automated deployment analysis based on metrics, logs, emitted requestsAutomatically roll back to the previous state as soon as analysis or a pipeline stage failsAutomatically detect configuration drift to notify and render the changesAutomatically watch and detect the new container images to deploySecureThe CD system often carries a lot of credentials needed to access the cluster and the necessary services of the teams.Ensuring the safety of the teams is always on our top priority.So while designing the PipeCD, we decided not to store those credentials in a central place. Instead of that, all user’s credentials always stay inside their clusters.Support single sign-on and role-based access controlCredentials are not exposed outside the cluster and not saved in the control-planePiped makes only outbound requests and can run inside a restricted networkMulti-provider & Multi-TenancyMulti-provider means supporting multiple cloud services, multiple container registries, multiple monitoring services for doing deployment analysis.You can use PipeCD to deploy your Kubernetes applications, CloudRun, AWS Lambda application and Terraform application.It also supports doing progressive delivery with canary and blue-green strategy.Easy to operate multi-cluster, multi-tenancy by separating control-plane and pipedSupport multiple analysis providers including Prometheus, Datadog, Stackdriver, and moreSupport multiple application kinds on multi-cloud including Kubernetes, Terraform, Cloud Run, AWS LambdaWhile designing PipeCD, we simplified its architecture by minimizing the number of components, so you do not have to install many things to enable all features.In addition, PipeCD also supports storing data in several fully-managed services to minimize the operating cost.Currently, we have completed the basic features and many of the features are in the alpha status. We are working hard to increase the stability and planning to release a stable version in the next months.CommunityPipeCD team hopes to receive a warm welcome and feedback from the open-source community.We value every contribution and invite you to join us on GitHub, Slack and Twitter.Visit our website and documentation at https://pipecd.devCheck out the code at https://github.com/pipe-cd/pipe or explore the examplesJoin us on Slack @cloud-native/pipecd to chat with other developersFollow us on Twitter @pipecd_dev to get the latest newsPipeCD team is hiring engineers/interns to work on PipeCD. Please contact us if you are interested.ThanksFinally, we would like to thank the existing open-source CD projects like Spinnaker, FluxCD, ArgoCD… PipeCD has been built on many great ideas from those great projects.PipeCD team would also like to thank CyberAgent’s engineers and collaborators from other companies, who have sent us so much valuable feedback throughout the development process to this day.

Blog: Announcing PipeCD