Heroku Embraces CI/CD by Automating Staging on GitHub
In a very short period of time, GitHub has evolved from a sharing facility for files used in open source projects, to the project support system for the world’s open source applications. By GitHub Inc.’s estimates, its service expanded in 2015 from 17 million to 26 million separate repositories, and its membership grew by 25 percent in the period to 10 million accounts.
Version and revision control has become the jewel in GitHub’s crown. Salesforce’s Heroku is one of the world’s first great PaaS platforms, and as such, is first and foremost a command line-driven system. For the past few years, Heroku has supported GitHub as an alternate repository, but mainly for the benefit of developers who know how to substitute GitHub for Heroku in Heroku’s syntax.
Usually, when the concept of continuous integration and continuous deployment gets confused with a specific product, it’s CloudBees’ Jenkins. Yet by instituting version control even at a very basic level, GitHub has been leveraged by some platform makers as a framework for continuous integration/continuous development (CI/CD). Meanwhile, building a CI/CD pipeline with Heroku has involved leveraging an altogether different platform, such as CircleCI.
CI/CD is the game-changer here. First of all, it introduces safeguards into the process of committing changes to production, instituting filters and rules that prevent alterations in untested and unapproved circumstances. It prevents running applications from breaking.
But secondly, it brings administrators and DevOps professionals into the process, by utilizing tools and styles of work that are more in keeping with what these groups expect, rather than raw Linux command lines.
‘A Prescriptive Solution’
Last September, Heroku introduced Flow, a simplified, pipeline-oriented construct that implements CI/CD using a purely visual system, no more difficult than provisioning services on Amazon AWS. At that time, as part of Flow, Heroku launched Pipelines into beta. Pipelines replaces the verbiage one normally deals with when interacting with both GitHub and Heroku, with a graphical system depicting all active versions of an application project in their respective stages of evolution.
Last week, Heroku released Pipelines into general availability. To mark the occasion, Heroku CEO Adam Gross and product manager Ike DeLorenzo gave The New Stack a demonstration of Flow and Pipelines at work. It was more the type of demonstration that an executive, such as a CIO, could appreciate: one with less syntax and more double-click.
“What we’re trying to do is provide a kind of prescriptive solution for our customers and developers, around how they can have a kind of Agile, continuous delivery system,” Gross told us.
“In the world of infrastructure and PaaS in general, a lot has been left to the imagination of the reader, and it’s very challenging for organizations to stitch together all the right tools and processes to enable them to have these more modern development environments that correspond with productivity,” Gross said.
What Gross implied was that many current Heroku customers (more to the point, the people in charge of the budget) didn’t see how Heroku and continuous delivery connected — even if they actually already did, by way of those syntax-heavy command lines.
The way Heroku Flow and Pipelines work are by dividing the lifecycle of each application in phases — by default, the common three phases representing “development,” staging (or testing), and production. For the sake of greater clarity, Pipelines reserves the first phase for “review apps” that have been pushed to GitHub as candidates.
Once the GitHub repository is connected to the Heroku pipeline, as Ike DeLorenzo showed us, there are any number of connection points that may be devised between the two. One of these points may take place during the GitHub pull request when a package is ready for evaluation. Pipelines enables DevOps to manifest these pull requests as review apps.
“Typically when developers are in GitHub and they make a code change,” explained DeLorenzo, “they create a pull request. This pull request is sent around to the other people on the team, everybody evaluates it, and then someone approves the pull request. This is the core workflow of peer approval on GitHub.
“Previous to review apps, customers were looking at code and speculating at how it might run,” he noted. “Now you can see how it actually runs in the review app.”
Each review app (pull request) is represented by a card (perhaps taking a cue from Trello). Evaluating the code within the review app is a matter of launching it directly from the card; Heroku stages the app automatically within the browser. DeLorenzo said that GitHub Inc. calls this feature, “bringing pull requests to life.”
When a developer makes changes to code and commits those changes, DeLorenzo showed how Pipelines automates the process of generating the pull request. So it’s not a matter of the developer doing something in GitHub and Heroku picking up on the event; it’s a two-way workflow using the GitHub API, worked out in collaboration with GitHub directly.
“This is the first time that PaaS as an industry has provided this kind of high fidelity,” argues CEO Gross. “Under the hood, we’re not just watching a pull request or seeing how code is changed; we’re actually spinning up a complete application environment end-to-end with this new code.”
Pushing Pull
DeLorenzo argued that by compounding the number of pull requests into review apps, instead of clogging the GitHub pipeline, more developers are brought into the open source review process. “You include designers, product managers, and stakeholders who do not code but can click and look at the thing running.”
But by staging review apps and staging “staging” apps (and also giving users the option to create more stages), doesn’t Pipelines’ new model blur the distinctions between the clear delegations of responsibility in conventional CI/CD?
“You’re right in that you’ve got these containers and these apps that are running code in different stages of the workflow,” responded DeLorenzo. “The difference is the workflow. You can have an individual programmer on your team write some crazy feature that you’re not sure you ever want to be in the master branch, and that’s created as a pull request. That pull request might be approved, and then merged into staging; or it might be denied, and it’s just deleted and destroyed.
“So the pull request is really testing on the individual work on a feature,” he continued. “The staging server is the integration testing, the integration of all those features in different pull requests.”
From staging, Pipelines can be set up to trigger an automatic battery of tests, similar to those that the Accenture Platform triggers within Jenkins. This way, code can be evaluated by an automated service to meet a basic set of reliability and security criteria before and/or after it is evaluated by people. Once it’s clear that staging code has passed the full battery of tests, in Pipelines, it may be “promoted” to production.
“This is perhaps the most exciting part of the whole thing,” said DeLorenzo. For public cloud deployments, promotion lets DevOps select availability zones for optimized or generalized deployments as DevOps may prefer.
“Once you’ve already built the code, and it’s been tested in an app that’s pretty much identical to the production environments, you have a very low probability that they’re going to fail in production,” DeLorenzo said.
Users of GitHub will be able to read audit trails generated by Heroku, detailing the dates and times in which pull requests were spun up as review apps, or when code has been “pushed to master” and built in staging. Heroku appears to understand that GitHub users are not going to stop using GitHub simply because some PaaS wrapped it up in a GUI. So it establishes links between the two.
It’s a workflow model that lends itself well, at least for now, to the traditional concept of the application — not so much the microservices concept, where functions interact with one another without the boundaries of class. Adam Gross is aware of that and gave us the impression that this is a gamble he’s willing to take, to address the needs of Heroku’s current customer base.
“Part of our goal in getting this product out is to see where people take it,” Gross told us, “so we can learn and adapt.”
Title image of the Alaska Pipeline by Ryan McFarland, licensed under Creative Commons 2.0.
