Thanks!

Running html-proofer in the Travis build would also be possible, but that would also mean that an error in the site would cause the whole project to assume a "build failing" status.

It's something we've looked into. Ideally, I'd ask git to list the files changed in this pull request, and only run html-proofer if any files in the site directory were modified. That's the ideal solution. We'd want to put installation & running it into one script.

Getting a diff of files since the last commit isn't difficult (something like git diff --name-only origin $(git log --pretty=format:"%h" -2 | tail -1) would do).

I'm not too sure on how Travis works exactly; how does it separate the pull requests from commits (i.e. will the line above work in all cases, or will we have to do different things for each situation?)

I'd ask git to list the files changed in this pull request, and only run html-proofer if any files in the site directory were modified.

@parkr And only install Proofer if any files were modified. The installation time was too long for you in #2311

@alfredxing That works for one commit, but doesn't get the whole PR, AFAICT. For reference, here's what Travis does:

$ git clone --depth=50 git://github.com/jekyll/jekyll.git jekyll/jekyll
$ cd jekyll/jekyll
$ git fetch origin +refs/pull/2601/merge: 
From git://github.com/jekyll/jekyll
 * branch            refs/pull/2601/merge -> FETCH_HEAD
$ git checkout -qf FETCH_HEAD

@penibelst Yeah, I really don't want to make testing slow down. Installing Nokogiri takes a long time, even when using the system libraries.

@parkr I set up a test repo, and the command seems to work for both commits and PR's (regardless of how many commits there are in a PR).
In the case of multi-commit PR's, I believe it works because it compares the tree of the last commit made in the PR with origin. This should result in a list of all changes made in the PR.
The script doesn't "work" for pull request merges, but I guess in the case there's really no need to run the tests again if they already ran in the PR.

Travis results from the test repo:

• Single commit to master
• PR with 2 commits and 2 files changed
• Merging the aforementioned PR

The script doesn't "work" for pull request merges, but I guess in the case there's really no need to run the tests again if they already ran in the PR.

👍 Awesome! Want to submit a pr? Please call it script/proof. It should:

1. Check if site files were modified. Quickly exit 0 if none were. If any were, continue.
2. Check to make sure html-proofer is installed. Install it (with NOKOGIRI_USE_SYSTEM_LIBRARIES=true set) if it's not installed. You can do this via gem list or the preferred command -v htmlproof.
3. Run jekyll build -s site -d _site
4. Run htmlproof ./_site

I think it would be fine if this was run inside script/cibuild (but the proofing script should be separate).

PR submitted! 🍴