Back To Schedule
Thursday, June 20 • 1:30pm - 1:55pm
PRO TALK: Lessons Learned in Building CI/CD for Serverless Architectures

Log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.
https://slides.com/anthonyc-1/lessons-learned-in-building-ci-cd-for-serverless-architectures/live#/ Why Serverless architecture (SA)? What are its pros (and some cons)? Common Pros: - Lower CapEx/OpEx Common Cons: - Increase complexity Key-enabler of other paradigms - CI/CD - Event/Data-driven - Agile (which affects business units; technology enabling business) Why we need CI/CD (for SA)? Fast iteration/Increase productivity/Time to Market/Lower cost -> Business win Why Serverless architecture (SA) as a paradigm is ideally suited for integration with CI/CD? How What are some of the general guideline and architecture principle for SA to make suitable for CI/CD? General principals of microservice (MS) all apply to SA, but it’s even more suitable as with SA you no longer manage your infrastructure the same way. What are some of these principals for MS that applies to SA? * Your SA infrastructure (as code) can now more closely be treated as code. You can and should treat your SA part of the infrastructure as cattle. ** THIS ENABLES CI/CD. ** Small, single unit of responsibility: This is a core tenant of UNIX and computing, do one thing and do it well. This leads to smaller blast radius; easier to unit test and refractor. When you build SA, you follow this same principle. ** THIS ENABLES CI/CD. ** Short-lived and stateless: As characteristics of Serverless Functions; this forces the developers to think about clearly delineating computing functions and its data. ** THIS ENABLES CI/CD and also design data driven architecture ** What are some its pitfalls? (See Challenges) What kind of tools may be helpful? (See part of How) Your tooling should be flexible but manageable. I recommend using SA framework (we use Serverless) to help build your SA infrastructure. For CI/CD, I would look at your organization security requirement and see if your CI/CD Challenges Greater technological complexity; can be difficult to troubleshoot. This can be alleviated by building your SA architecture into units of functions; for example we use step functions to manage lambda functions so that we have a clear way of how the lambda functions interact. Cultural: Your infrastructure team need to understand core tenants of DevOps, understand SA and CI/CD. Ideally your team should already have experiences of some of IaC, SA, CI/CD. There are some required skillsets to achieve these goals. Security: Restricting Production from developer; requires every deploy from CI/CD Newer tools that are less mature. Your need to treat your SA infrastructure as code. How we overcame those challenges (Show) How our Data Engineering team (intersection.com) leverages Serverless Framework to implement our data pipelines with CI/CD. I will give a overview and demo on how we use Serverless Framework to create Data Pipeline with AWS Step Function, all of which are part of a CI/CD pipeline. Recommended: Start small; start SA first before CI/CD and/or start CI/CD without SA.

avatar for Anthony Cheng

Anthony Cheng

Data Engineer, Intersection
Data/DevOps Engineer with more than a decade of infrastructure experiences, my career has spanned from working on "Iron Age" data center with network and system infrastructure to working in programmatic "Infrastructure as Code" with AWS. In my career, I have worked with start-up... Read More →

Thursday June 20, 2019 1:30pm - 1:55pm EDT
Workshop Stage 1