Supercharge GitHub Pages with Jekyll and Travis CI

GitHub Pages are a great way to create and manage a website for your GitHub user, organization, or a project for FREE. In addition to supporting regular HTML content, it also supports Jekyll, a popular static site generator. If you haven’t already, you should familiarize yourself with both GitHub Pages and Jekyll. There aren’t many ways to get a static website up and running faster, but GitHub Pages does come with its own set of limitations. While GitHub Pages is powered by Jekyll, all sites are generated using the --safe option which disallows any custom plugins, according to the Jekyll docs. When I decided to include GitHub repository details from a repository that was hosted outside of my organization, I quickly discovered how limiting the --safe option is. Not to worry, by using Travis CI we will build the site pages on our own and then publish the static site to GitHub Pages.

“Vintage black and white car engine” by RKTKN on Unsplash

By default, GitHub Pages makes repository metadata available to the Jekyll sites it builds. This data is available in the site.github object and includes information such as public repositories (if you are creating a user or organization site). For the organization site I updated, there was one repository hosted inside of another organization. The --safe build made it so there was no way for me to fetch any repository metadata from outside of the organization. Jekyll sites are generated from JSON data and the GitHub API makes repository metadata available in JSON format. I knew there had to be a way to append a 3rd party repository’s metadata into the metadata already provided by GitHub. Enter the Jekyll-Get plugin. This plugin pulls in JSON data from external sources and makes it available to the site generator. I now had a way to get the JSON metadata. All I needed to do was create my own build process and publish that static site to the GitHub repository.

In order to get the most of this tutorial, I suggest you already have a GitHub Pages site created. This tutorial will not cover the basics of getting started with GitHub pages. If you need to create a new site, start here. While this tutorial is focused on solving the particular problem mentioned above, the steps covered will work with any plugin or custom Jekyll build process.

What we will cover

• Using Jekyll build instead of the GitHub Pages --safe build
• Installing and using the Jekyll-Get plugin to fetch JSON data
• Organizing the repository for the new build process
• Setting up Travis CI to build and deploy the site

Using Jekyll build instead of the GitHub Pages — safe build

The first thing that we need to do is replace the dependency on GitHub Pages with Jekyll in the Gemfile. Open the Gemfile, find

gem "github-pages", group::jekyll_plugins

and replace it with:

gem "jekyll"
gem "json"
gem "hash-joiner"

Then from the command line run:

bundle install; bundle exec jekyll serve;

At this point, your site will likely be empty because there isn’t any GitHub metadata being fetched. Let’s add that now.

Installing and using the Jekyll-Get plugin to fetch JSON data

According to the installation instruction for the Jekyll-Get plugin: “Add this file to _plugins in the root of your Jekyll site.” From the CLI in the root of your Jekyll site:

mkdir _plugins
curl -o ./_plugins/jekyll_get.rb https://raw.githubusercontent.com/18F/jekyll-get/master/jekyll_get.rb

Add the following line to the _config.yml file in the site root folder:

plugins_dir: ./_plugins

Now you can specify which GitHub JSON data to fetch. Add this section below the line added above and replace <ORG_NAME> with your own organization:

jekyll_get:
  - data: github
    json: 'https://api.github.com/orgs/<ORG_NAME>/repos'
    cache: false

If your GitHub page is for a user instead of a repository, use this instead and replace <USER_NAME>with your own username:

jekyll_get:
  - data: github
    json: 'https://api.github.com/users/<USER_NAME>/repos'
    cache: false

Since the default GitHub pages build provided public_repositories as a set of available data under site.github, you will need to replace instances of site.github.public_repositories with site.data.github in your index.html file. If you need data other than repositories, look through the GitHub API and find for the proper request URL for the data you need.

Organizing the repository for the new build process

If you are creating a site for a User or Organization, GitHub Pages will only publish content from the master branch. Since we aren’t using the github-pages gem any longer, we’ll need to set up the repository so that the generated site content gets published to master. To solve this problem, I created a release branch and pull-request all of my changes to this branch. I also set the release branch to be the default branch.

Below we will set up Travis CI to build when there are updates to the release branch and publish the static site to master. I ended up with a repository with 3 protected branches:

master <- generated static site content
release <- Jekyll code to be generated into site
develop <- Branch that contains changes until merged into release

Setting up Travis CI to build and deploy the site

We’re now at a point where the Jekyll config builds without the --safe option, the content is fetched through JSON APIs, and the repository is set up ready for updates to be pushed to the master branch, but GitHub Pages doesn’t know how to build the site any longer. Not to worry, we’ll set up Travis CI to build and publish the generated site to the repository. If you haven’t already, head over to Travis CI and create an account. Once your Travis CI account has been able to sync repositories from GitHub, look for the repository and click the button to active this repository.

You can then click on Settings and customize your project however you like. I personally like active “Build pushed branches” and de-activate “Build pushed pull-requests”. We’ll need to add a GitHub Token for Travis CI to be able to publish the generated site to master. Go to https://github.com/settings/tokens and create a personal access token. Give the token a name, check the box to give the token access to “Repo”, and then click “Generate Token.”

Once have your generated token, make sure to copy it immediately as it will only be available this one time. Take this copied token back to the Settings page in Travis CI, create an Environment Variable named GITHUB_TOKEN, paste the token in the value field, make sure that “Display value in build log” is not checked, then click Add.

In order for Travis CI to know how to build your project, it needs a .travis.yml file. Create a blank one in the root of your project:

touch .travis.yml

Then paste the following contents into the .travis.yml file:

language: ruby
cache: bundler
branches:
  only:
  - release
script:
  - JEKYLL_ENV=production bundle exec jekyll build --destination site
deploy:
  provider: pages
  local-dir: ./site
  target-branch: master
  email: deploy@travis-ci.org
  name: Deployment Bot
  skip-cleanup: true
  github-token: $GITHUB_TOKEN
  keep-history: true
  on:
    branch: release

Commit this file, push it to your release branch in GitHub. This push will trigger a build in Travis CI. Here is what the .travis.yml is doing.

language: ruby
cache: bundler

This sets up the environment for Travis CI and tells it we use Ruby and Bundler for the build.

branches:
  only:
  - release

We only want to generate site content from the release branch.

script:
  - JEKYLL_ENV=production bundle exec jekyll build --destination site

Here is where we remove the --safe flag from the Jekyll build. This sets the environment to production and generates the site content into the site folder.

deploy:
  provider: pages
  local-dir: ./site
  target-branch: master
  email: deploy@travis-ci.org
  name: Deployment Bot
  skip-cleanup: true
  github-token: $GITHUB_TOKEN
  keep-history: true
  on:
    branch: release

Here we use the GitHub Pages Deployment Provider for Travis CI. Since we build the site to the site folder in the previous step, we need to set the value to match. The other important detail is skip-cleanup must be true or the assets generated in the previous build phase will be deleted and won’t be deployed to GitHub.

Extra Credit

If you are pulling in dynamic content from your GitHub Repos, like star count or number of forks, likely you will want that to be updated regularly. Go back to the Travis CI settings and set up a daily build for the release branch in the Cron Jobs section.

Wrapping it up

This tutorial was intended to show you how I solved a certain problem. If you want to use GitHub Pages to host static sites, likely you will outgrow the github-pages gem rather quickly and need to start using Jekyll Plugins. GitHub Pages is a great service and with Jekyll Plugins and Travis CI there is no limit to the content you can create.