This updates the test coverage GitHub Actions workflow to generate a human-readable HTML report in addition to the current machine readable one submitted to Codecov.
Props jorbin.
Fixes#60476.
git-svn-id: https://develop.svn.wordpress.org/trunk@57570 602fd350-edb4-49c9-b593-d223f7449a82
Since being introduced, the GitHub Actions workflow responsible for generating and submitting a test coverage report has used PHP 7.4. At the time, there were some issues with running the test suite on PHP 8.0+ which could have resulted in inaccurate reporting.
The test suite and WordPress in general are much more stable on 8.x now. The test coverage report should now be generated using the Docker container tagged `latest` (currently 8.2).
This will result in a very small decrease in the percentage of lines covered by test (-0.05%), but a slight increase in the percentage of functions and methods covered (+0.08%).
Props johnbillion.
See #59647.
git-svn-id: https://develop.svn.wordpress.org/trunk@57566 602fd350-edb4-49c9-b593-d223f7449a82
Problematic changes to the test reporter were merged upstream. This specific SHA also did not fix the issue.
Props javiercasares.
See #59647.
git-svn-id: https://develop.svn.wordpress.org/trunk@57552 602fd350-edb4-49c9-b593-d223f7449a82
An upstream change in the Hosting Test Reporter is causing failures in the PHPUnit workflow. This temporarily pins an older hash to the step that checks the reporter out.
Props youknowriad.
See #59647.
git-svn-id: https://develop.svn.wordpress.org/trunk@57551 602fd350-edb4-49c9-b593-d223f7449a82
Version 4 of the action now requires a token to be provided in order to upload coverage results.
Follow up to [57534].
Props swissspidy.
See #59658.
git-svn-id: https://develop.svn.wordpress.org/trunk@57536 602fd350-edb4-49c9-b593-d223f7449a82
This updates the `codecov/codecov-action` from version `3.1.5` to `4.0.1`.
Version 4 switches to using the Codecov CLI to upload test report date, and changes the version of Node.js used for the action to 20.x. This fixes the notices currently shown for the test coverage workflow.
Props: mukesh27.
See #59658.
git-svn-id: https://develop.svn.wordpress.org/trunk@57534 602fd350-edb4-49c9-b593-d223f7449a82
This makes a few improvements made to the Props Bot workflow:
- The bot will no longer run on draft PRs.
- The bot will no longer run on closed PRs.
- The bot will no longer run when a comment is deleted (this should almost never happen).
Props mamaduka, gziolo.
See #60417.
git-svn-id: https://develop.svn.wordpress.org/trunk@57523 602fd350-edb4-49c9-b593-d223f7449a82
Props Bot is a new GitHub Action that will compile a list of contributors for a given pull request. The bot will leave a comment with a list of contributors formatted for use in both Trac SVN and GitHub.
Props dharm1025, desrosj, jorbin, jeffpaul, dd32, pento, gziolo, swissspidy, talldanwp, noisysocks, youknowriad, peterwilsoncc, joemcgill, chrisdavidmiles, wpscholar, annezazu, chanthaboune, desrosjbot.
See #60417.
git-svn-id: https://develop.svn.wordpress.org/trunk@57517 602fd350-edb4-49c9-b593-d223f7449a82
This updates the `slackapi/slack-github-action` from `1.24.0` to `1.25.0`. This fixes more GitHub Action deprecated notices.
Follow up to [57362].
See #59805.
git-svn-id: https://develop.svn.wordpress.org/trunk@57376 602fd350-edb4-49c9-b593-d223f7449a82
This updates the following third-party GitHub Actions to their latest versions:
- `actions/setup-node` from `3.8.1` to `4.0.1`
- `actions/upload-artifact` from `3.1.2` to `4.3.0`
- `shivammathur/setup-php` from `2.28.0` to `2.29.0`
- `actions/cache` from `3.3.2` to `4.0.0`
- `codecov/codecov-action` from `3.1.4` to `3.1.5`
Most notably, these updates silence newly encountered notices as a result of GitHub beginning to transition away from Node.js 16 to Node.js 20 (see https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/).
Props swissspidy.
See #59805.
git-svn-id: https://develop.svn.wordpress.org/trunk@57362 602fd350-edb4-49c9-b593-d223f7449a82
The Test Build Processes workflow started failing recently on MacOS runners due to “JavaScript heap out of memory” errors (see https://github.com/WordPress/wordpress-develop/actions/runs/7421385568/job/20209241826#step:8:82).
This increases the maximum memory size of the old memory section in Node from the default of 4GB to 8GB (specified in megabytes) to avoid unnecessary failures while ways to optimize the Gutenberg build process are explored.
Props dmsnell, joemcgill, hellofromTonya, isabel_brison.
See #59805.
git-svn-id: https://develop.svn.wordpress.org/trunk@57250 602fd350-edb4-49c9-b593-d223f7449a82
SVN support has officially been sunset by GitHub. While SVN was not has not been utilized in GitHub Action workflows, the version of SVN being used has been output for debugging purposes.
This removes those debug lines to prevent encountering failures as new versions of test runners are pushed out without `svn` installed.
See https://github.blog/changelog/2024-01-08-subversion-has-been-sunset/.
See #59805.
git-svn-id: https://develop.svn.wordpress.org/trunk@57249 602fd350-edb4-49c9-b593-d223f7449a82
When the JSON data files that supply the PHP/MySQL version support to the installation test workflow are changed, the workflow should be run to confirm the changes are correct. This updates the `path` filtering rules to ensure this happens.
Follow up to [57218], [57219].
See #58977.
git-svn-id: https://develop.svn.wordpress.org/trunk@57220 602fd350-edb4-49c9-b593-d223f7449a82
In [56661], a new GitHub Actions workflow was introduced that focused on running some minimal installation tests for a version of WordPress for every PHP and MySQL combination.
This workflow has tested well, but lacks flexibility and possesses one flaw: tests are only ever performed with currently supported versions, even if the version being tested had a different support policy.
This updates the workflow to be more flexible, allowing all versions of WordPress currently receiving security fixes (back through 4.1) to be tested under the correct support policy.
Additionally, the workflow can now run against the `nightly` build of WordPress. This replaces `latest` as the new default. This allows the tests to be run at any point during a release cycle without the need for an officially tagged version.
Props jorbin, joemcgill, costdev.
See #58977.
git-svn-id: https://develop.svn.wordpress.org/trunk@57218 602fd350-edb4-49c9-b593-d223f7449a82
This bumps the minimum required version of Node.js/npm from 16.19.1 and 8.19.3 to 20.10.0 and 10.2.3.
Since 20.10.0 is the latest 20.x version of Node.js, the `check-latest` option has been enabled for `actions/setup-node` in GitHub Actions workflows. This performs an additional external call to the Node.js API confirming the latest version is installed on the runner for use. In testing, it seems that 20.10.0 was not consistently deployed to all runner machines in use. This should be removed in the near future when the version of Node.js is reliably above the new minimum requirement.
The Gutenberg repository has also been updated to use the same values for `engines`.
Props jorbin, joemcgill, swissspidy, benharri, dhrupo, flootr, gziolo, noahtallen.
See #59663.
git-svn-id: https://develop.svn.wordpress.org/trunk@57212 602fd350-edb4-49c9-b593-d223f7449a82
Uses the `workflow_run` trigger to only leave pull request comments after the build jobs finish.
Props zieladam, desrosj.
See #59416.
git-svn-id: https://develop.svn.wordpress.org/trunk@57210 602fd350-edb4-49c9-b593-d223f7449a82
This adds a bit more context to the E2E workflow artifact names in order to avoid duplicates being uploaded.
With the update to v4 of `actions/upload-artifact` in [57197], artifacts are now uploaded on a per job basis. Multiple jobs cannot upload the same artifact.
Props johnbillion.
See #59805.
git-svn-id: https://develop.svn.wordpress.org/trunk@57203 602fd350-edb4-49c9-b593-d223f7449a82
This updates the following GitHub Actions to their latest versions:
- `actions/checkout`
- `actions/setup-node`
- `actions/upload-artifact`
- `actions/cache`
- `actions/github-script`
- `shivammathur/setup-php`
See #59805.
git-svn-id: https://develop.svn.wordpress.org/trunk@57197 602fd350-edb4-49c9-b593-d223f7449a82
This updates the Dependabot configuration file to make use of `groups`, configuring all third-party GitHub Action updates available into a single pull request to help reduce noise.
Props bradparbs.
See #59805.
git-svn-id: https://develop.svn.wordpress.org/trunk@57180 602fd350-edb4-49c9-b593-d223f7449a82
Fixes a 403 error in the pull-request-comments.yml job by switching from a custom workflow dispatch call to the pull_request_target trigger.
Follow up to [57124], [57174].
Props ockham.
See #59416.
git-svn-id: https://develop.svn.wordpress.org/trunk@57178 602fd350-edb4-49c9-b593-d223f7449a82
Fixes a problem in WordPress artifact upload pipeline. The callable-test-core-build-process.yml used to zip the WordPress dist directory after npm run grunt clean runs, producing an empty zip file. This commit moves the zip before the cleanup task.
Follow up to [57124].
Props ockham, dmsnell.
See #59416.
git-svn-id: https://develop.svn.wordpress.org/trunk@57174 602fd350-edb4-49c9-b593-d223f7449a82
This updates the test strategy matrix in the PHPUnit test GitHub Actions workflow to include the most recent MariaDB release, 11.2.
While 11.0 and 11.1 are still maintained, all three are short-term stable releases, which see support dropped after 1 year.
Until a larger discussion around how the project supports database software releases with shorter support windows takes place (short-term in MariaDB or “innovation” releases in MySQL), testing only against the latest short-term release should be sufficient.
Props johnbillion, l1nuxjedi.
See #59806.
git-svn-id: https://develop.svn.wordpress.org/trunk@57152 602fd350-edb4-49c9-b593-d223f7449a82
The MySQL project has introduced a new versioning strategy with two types of releases: innovation and long-term support. While long-term support is a familiar concept, it’s not clear how innovation releases should be supported in various WordPress versions for a number of reasons.
Mainly, innovation releases are very short-lived. Each one is only supported until the next innovation version is released with a goal of one per quarter. Even though these versions will have a short lifespan they are described as producton-grade and generally available.
More discussion is needed to determine exactly how each WordPress release should support these new innovation releases. But while the parameters for proper support are determined, the PHPUnit tests can be run against these innovation releases to detect any potential problems or incompatibilities.
This change adds the `8.1` and `8.2` innovation releases to the testing matrix for the PHPUnit test worfklow.
Props johnbillion, jorbin.
See #59779.
git-svn-id: https://develop.svn.wordpress.org/trunk@57132 602fd350-edb4-49c9-b593-d223f7449a82
When this job fails, it should not fail the entire workflow. The next update to the pull request will attempt to dispatch the commenting workflow.
Follow up to [57124].
Props johnbillion.
See #59416.
git-svn-id: https://develop.svn.wordpress.org/trunk@57125 602fd350-edb4-49c9-b593-d223f7449a82
After [56958], pull requests to `wordpress-develop` can be tested within a WordPress Playground instance. However this capability is not obvious.
This makes several updates to GitHub action workflows to avoid unnecessary extra runs/jobs while also commenting on pull requests with proper context for testing in Playground once a build is available for the PR.
Changes:
- The `build.yml` workflow has been deleted. The WordPress build process is already being tested within the `test-build-processes.yml` workflow. Now that workflow will produce and upload the built artifact for the given PR.
- A `pull-request-comments.yml` workflow has been added to handle the commenting. To avoid having multiple workflows that comment on PRs, the logic within the `welcome-new-contributors.yml` workflow has been merged into this one.
- The `pull_request` event trigger lacks the needed permission to comment on or update pull requests. Instead of using `pull_request_target` to navigate around this, the `test-build-processes.yml` workflow will now use `workflow_dispatch` to initiate a run of the commenting workflow. This adds the ability to only comment on the PR when a build has been successfully created and uploaded as an artifact.
Props swissspidy, zieladam, jeffpaul.
See #59416.
git-svn-id: https://develop.svn.wordpress.org/trunk@57124 602fd350-edb4-49c9-b593-d223f7449a82
This corrects the order the arguments are passed to the `contains()` function in the Performance Testing workflow.
Because the arguments were passed incorrectly, the expression was not evaluating correctly.
Follow up to [56972].
Props hellofromTonya.
See #59805.
git-svn-id: https://develop.svn.wordpress.org/trunk@57085 602fd350-edb4-49c9-b593-d223f7449a82
Adds new tests for localized sites as well as the dashboard.
Also amends Server-Timing output to measure memory usage in all scenarios.
Props swissspidy, joemcgill, flixos90, mukesh27, mamaduka.
See #59656.
Fixes#59815.
git-svn-id: https://develop.svn.wordpress.org/trunk@57083 602fd350-edb4-49c9-b593-d223f7449a82
This increases the number of times to retry restarting a workflow from 10 to 15, and the `timeout-minutes` value to `30`.
For workflows with complex strategy matrix, the exponential backoff of 10 retries is still not enough to account for the GitHub Actions UI taking a long time to catch up.
Follow up to [56829] and [56830].
See #58867.
git-svn-id: https://develop.svn.wordpress.org/trunk@57052 602fd350-edb4-49c9-b593-d223f7449a82
A very common contributor setup is having a copy of the `gutenberg` development repository within a checkout of the `wordpress-develop` repository. On occasion, there are some strange incompatibilities that come up when using this setup. A few examples can be seen in #58671 and #59634.
This changeset helps ensure that these edge cases are not introduced by testing the Gutenberg plugin’s build process within WordPress configured to run from both the `src` and `build` directories.
This also renames the “Test npm” workflow to a more general “Test Build Processes”, which more accurately describes what is actually being tested within it and allows these new test jobs to be grouped in.
And finally, the logic within the workflow has been split out into two callable workflows. This helps avoid code duplication within the workflow, and allows for better grouping on the workflow run screen.
Props swissspidy, aferica, SergeyBiryukov, antonvlasenko, desrosj.
Fixes#59632. See #58671, #59634.
git-svn-id: https://develop.svn.wordpress.org/trunk@56976 602fd350-edb4-49c9-b593-d223f7449a82
This adds the `PUPPETEER_SKIP_DOWNLOAD` environment variable to the Build WordPress workflow to skip downloading Puppeteer browser binaries unnecessarily.
Follow up to [56958].
See #59416, #59517, #58863.
git-svn-id: https://develop.svn.wordpress.org/trunk@56973 602fd350-edb4-49c9-b593-d223f7449a82
When a branch is created, there is no previous commit to reference in the `github.event.before` context, which causes the performance workflow to fail because there is no previous commit to perform a comparison with.
This adds a condition to check that `github.event.before` is not set to `0000000000000000000000000000000000000000`, which is the default value when there are no previous commits.
Props swissspidy.
See #588867.
git-svn-id: https://develop.svn.wordpress.org/trunk@56972 602fd350-edb4-49c9-b593-d223f7449a82
Storing build files enables reusing them in WordPress Playground and ultimately implementing a Pull Request.
Props desrosj, bernhard-reiter
See #59416.
git-svn-id: https://develop.svn.wordpress.org/trunk@56958 602fd350-edb4-49c9-b593-d223f7449a82
This adds a global environment variable to the Test old branches workflow to make the supported version more clear and easier to update in the future. This will also make it easier to reference in more places as this workflow grows.
See #58867.
git-svn-id: https://develop.svn.wordpress.org/trunk@56955 602fd350-edb4-49c9-b593-d223f7449a82
The `PUPPETEER_SKIP_DOWNLOAD` environment variable is used to prevent Puppeteer from automatically downloading browser binaries.
It was removed in [56926] due to the migration to Playwright. However, because of the QUnit tests, Puppeteer is actually still a dependency.
Until those tests change, we have to keep this environment variable to prevent unnecessary downloads on CI.
Props SergeyBiryukov.
Fixes#59517.
git-svn-id: https://develop.svn.wordpress.org/trunk@56954 602fd350-edb4-49c9-b593-d223f7449a82
This is a follow-up to [56926], where this logic has been reintroduced after it was originally removed in [56198].
Props desrosj.
See #59517.
git-svn-id: https://develop.svn.wordpress.org/trunk@56942 602fd350-edb4-49c9-b593-d223f7449a82
As per the migration plan shared last year, this migrates all browser-based tests in WordPress core to use Playwright.
This includes end-to-end, performance, and visual regression tests.
Props swissspidy, mamaduka, kevin940726, bartkalisz, desrosj, adamsilverstein.
Fixes#59517.
git-svn-id: https://develop.svn.wordpress.org/trunk@56926 602fd350-edb4-49c9-b593-d223f7449a82
Following [56829], the previous `timeout-minutes` value of `5` is insufficient when approaching 10 retries.
See #58867.
git-svn-id: https://develop.svn.wordpress.org/trunk@56830 602fd350-edb4-49c9-b593-d223f7449a82
This increases the number of times to retry restarting a workflow from 2 to 10. Retries use exponential backoff to space out retries.
In most cases, only 2 retries was sufficient. However, there are occasionally scenarios where the original workflow is still running and cannot be restarted, resulting in an error. This typically happens during periods of ongoing service degradation, or workflows with a significant number of jobs (GitHub Actions is sometimes very slow to mark these as finished).
See #58867.
git-svn-id: https://develop.svn.wordpress.org/trunk@56829 602fd350-edb4-49c9-b593-d223f7449a82
[56780] changed Slack failure notifications to not send a failure notification during the first run of a workflow. Because workflows automatically restart once, a failure during the first run can be ignored.
This adjusts the workflow to also not send a “fixed” notification when the second run of a workflow succeeds after a failure. Because the original failure is no longer reported, these notifications are unnecessary.
See #58867.
git-svn-id: https://develop.svn.wordpress.org/trunk@56827 602fd350-edb4-49c9-b593-d223f7449a82
This was causing the version of WordPress being tested to be listed as `latest` even when that is not the case.
Props davidbaumwald.
See #58977.
git-svn-id: https://develop.svn.wordpress.org/trunk@56823 602fd350-edb4-49c9-b593-d223f7449a82
Because the contents of binary files are not included in patches by default, occasionally empty versions of these files are committed on accident. This is the most commonly happens within default themes because they usually contain image and font files.
This adds a job to the theme related GitHub Action workflow that checks for the presence of zero-byte files so that these instances are caught more easily.
Props costdev, hellofromTonya, luminuu, onemaggie.
Fixes#59527.
git-svn-id: https://develop.svn.wordpress.org/trunk@56792 602fd350-edb4-49c9-b593-d223f7449a82
After [53947], all workflows will automatically be restarted once in an attempt to rule out reasons for failures, such as timeouts or network hiccups.
Second attempt at [56404], which was previously reverted in [56407].
See #58867.
git-svn-id: https://develop.svn.wordpress.org/trunk@56780 602fd350-edb4-49c9-b593-d223f7449a82
