Skip to content

Conversation

@mgascam
Copy link
Contributor

@mgascam mgascam commented Oct 1, 2025

Fixes #WOOPMNT-5344

This PR is part of a larger effort to migrate WooPayments E2E tests to the QIT (Quality Insights Toolkit) stack. The work is split into stacked PRs for easier review.

Changes proposed in this Pull Request

  • Migrates Playwright-based E2E shopper specs to run with QIT. Expands the QIT E2E test suite to cover core shopper flows and payment methods
  • Implements new test fixtures, helpers, and configuration for QIT E2E
  • Updates the E2E runner and environment setup scripts
  • Removes inline skip flags (SKIP_WC_BLOCKS_TESTS) from 4 WooCommerce Blocks specs in favor of tag-based filtering

Skip Flags Removal

What Changed:

// ❌ Before: Environment-based skip logic
const shouldRunWCBlocksTests = process.env.SKIP_WC_BLOCKS_TESTS !== '1';

describeif( shouldRunWCBlocksTests )(
    'WooCommerce Blocks > Test',
    { tag: '@critical' },
    () => {
        // test code
    }
);

// ✅ After: Native Playwright tag-based filtering
test.describe(
    'WooCommerce Blocks > Test',
    { tag: [ '@critical', '@blocks' ] },
    () => {
        // test code
    }
);

Benefits:

  • Removed ~16 lines of boilerplate across 4 files
  • Better control: Use --tag=@blocks or --skip-tag=@blocks at runtime
  • No need to set environment variables like SKIP_WC_BLOCKS_TESTS=1

Note: I started the shopper migration in this branch, but had to discard the approach.

Testing instructions

Run ALL tests:

Use npm run test:qit-e2e to run all the specs. Note: The shopper-myaccount-saved-cards.spec.ts usually fails when not running in isolation, this needs more investigation so I created this issue to look into it separately, since the current goal is to migrate the tests.

Run a single spec (multiple formats work):

Full path from project root
npm run test:qit-e2e -- tests/qit/e2e/specs/woopayments/shopper/shopper-checkout-purchase.spec.ts

Just the filename (it will search for it)
npm run test:qit-e2e -- shopper-checkout-purchase.spec.ts

Partial path
npm run test:qit-e2e -- e2e/specs/woopayments/shopper/shopper-checkout-purchase.spec.ts

Run tests by tag:

Run all shopper tests
npm run test:qit-e2e -- --tag=shopper

Run all critical tests
npm run test:qit-e2e -- --tag=critical

Run all WC blocks tests
npm run test:qit-e2e -- --tag=blocks

Combine tag with specific directory:

Run critical shopper tests only
npm run test:qit-e2e -- tests/qit/e2e/specs/woopayments/shopper --tag=critical


  • Run npm run changelog to add a changelog file, choose patch to leave it empty if the change is not significant. You can add multiple changelog files in one PR by running this command a few times.
  • Covered with tests (or have a good reason not to test in description ☝️)
  • Tested on mobile (or does not apply)

Post merge

@mgascam mgascam changed the title [QIT][E2E] Migrate shopper e2e specs [E2E] [QIT] Migrate shopper e2e specs Oct 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Oct 1, 2025

Test the build

Option 1. Jetpack Beta

  • Install and activate Jetpack Beta.
  • Use this build by searching for PR number 11069 or branch name dev/qit-e2e-shopper-specs-migration in your-test.site/wp-admin/admin.php?page=jetpack-beta&plugin=woocommerce-payments

Option 2. Jurassic Ninja - available for logged-in A12s

🚀 Launch a JN site with this branch 🚀

ℹ️ Install this Tampermonkey script to get more options.


Build info:

  • Latest commit: e16aa7e
  • Build time: 2025-10-10 14:42:02 UTC

Note: the build is updated when a new commit is pushed to this PR.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 1, 2025

Size Change: 0 B

Total Size: 873 kB

ℹ️ View Unchanged
Filename Size
release/woocommerce-payments/assets/css/admin.css 1.45 kB
release/woocommerce-payments/assets/css/admin.rtl.css 1.45 kB
release/woocommerce-payments/assets/css/success.css 1.06 kB
release/woocommerce-payments/assets/css/success.rtl.css 1.06 kB
release/woocommerce-payments/dist/blocks-checkout-rtl.css 2.77 kB
release/woocommerce-payments/dist/blocks-checkout.css 2.77 kB
release/woocommerce-payments/dist/blocks-checkout.js 54.3 kB
release/woocommerce-payments/dist/cart-block-rtl.css 113 B
release/woocommerce-payments/dist/cart-block.css 112 B
release/woocommerce-payments/dist/cart-block.js 16.7 kB
release/woocommerce-payments/dist/cart.js 5.27 kB
release/woocommerce-payments/dist/checkout-rtl.css 1.08 kB
release/woocommerce-payments/dist/checkout.css 1.08 kB
release/woocommerce-payments/dist/checkout.js 34.4 kB
release/woocommerce-payments/dist/express-checkout-rtl.css 367 B
release/woocommerce-payments/dist/express-checkout.css 367 B
release/woocommerce-payments/dist/express-checkout.js 16.8 kB
release/woocommerce-payments/dist/frontend-tracks.js 833 B
release/woocommerce-payments/dist/index-rtl.css 21.2 kB
release/woocommerce-payments/dist/index.css 21.2 kB
release/woocommerce-payments/dist/index.js 153 kB
release/woocommerce-payments/dist/multi-currency-analytics.js 1.08 kB
release/woocommerce-payments/dist/multi-currency-rtl.css 3.82 kB
release/woocommerce-payments/dist/multi-currency-switcher-block.js 18.2 kB
release/woocommerce-payments/dist/multi-currency.css 3.83 kB
release/woocommerce-payments/dist/multi-currency.js 24.6 kB
release/woocommerce-payments/dist/order-rtl.css 740 B
release/woocommerce-payments/dist/order.css 740 B
release/woocommerce-payments/dist/order.js 21.2 kB
release/woocommerce-payments/dist/plugins-page-rtl.css 484 B
release/woocommerce-payments/dist/plugins-page.css 484 B
release/woocommerce-payments/dist/plugins-page.js 2.64 kB
release/woocommerce-payments/dist/product-details-rtl.css 433 B
release/woocommerce-payments/dist/product-details.css 436 B
release/woocommerce-payments/dist/product-details.js 12.3 kB
release/woocommerce-payments/dist/settings-rtl.css 11.8 kB
release/woocommerce-payments/dist/settings.css 11.7 kB
release/woocommerce-payments/dist/settings.js 140 kB
release/woocommerce-payments/dist/subscription-edit-page.js 703 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal-rtl.css 527 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.css 527 B
release/woocommerce-payments/dist/subscription-product-onboarding-modal.js 1.98 kB
release/woocommerce-payments/dist/subscription-product-onboarding-toast.js 730 B
release/woocommerce-payments/dist/subscriptions-empty-state-rtl.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.css 120 B
release/woocommerce-payments/dist/subscriptions-empty-state.js 1.9 kB
release/woocommerce-payments/dist/success.js 6.03 kB
release/woocommerce-payments/dist/tos-rtl.css 235 B
release/woocommerce-payments/dist/tos.css 235 B
release/woocommerce-payments/dist/tos.js 3 kB
release/woocommerce-payments/dist/woopay-direct-checkout.js 5.68 kB
release/woocommerce-payments/dist/woopay-express-button.js 22.8 kB
release/woocommerce-payments/dist/woopay-rtl.css 4.29 kB
release/woocommerce-payments/dist/woopay.css 4.27 kB
release/woocommerce-payments/dist/woopay.js 70.8 kB
release/woocommerce-payments/includes/subscriptions/assets/css/plugin-page.css 625 B
release/woocommerce-payments/includes/subscriptions/assets/js/plugin-page.js 814 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/i18n-loader.js 2.46 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/build/jetpack-script-data.js 957 B
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/i18n-loader.js 1.02 kB
release/woocommerce-payments/vendor/automattic/jetpack-assets/src/js/script-data.js 69 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/babel.config.js 163 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.css 2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.js 14.3 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/identity-crisis.rtl.css 2.47 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.css 10.1 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.js 29.7 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-connection.rtl.css 10.1 kB
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.css 198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.js 280 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-admin-create-user.rtl.css 198 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.css 625 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.js 333 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-login.rtl.css 626 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-sso-users.js 417 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/jetpack-users-connection.js 161 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-ajax.js 521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/dist/tracks-callables.js 585 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.css 215 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-admin-create-user.js 521 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.css 721 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-login.js 412 B
release/woocommerce-payments/vendor/automattic/jetpack-connection/src/sso/jetpack-sso-users.js 625 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/about.css 1.04 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-empty-state.css 294 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin-order-statuses.css 408 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/admin.css 3.59 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/checkout.css 301 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/modal.css 746 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/view-subscription.css 574 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/css/wcs-upgrade.css 414 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin-pointers.js 543 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/admin.js 9.4 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.js 6.78 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/jstz.min.js 3.84 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-coupon.js 545 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/meta-boxes-subscription.js 2.52 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.js 22.2 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/moment.min.js 11.7 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/payment-method-restrictions.js 1.29 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/admin/wcs-meta-boxes-order.js 507 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/payment-methods.js 358 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/single-product.js 428 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/view-subscription.js 1.38 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/frontend/wcs-cart.js 782 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/modal.js 1.09 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/assets/js/wcs-upgrade.js 1.26 kB
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.css 391 B
release/woocommerce-payments/vendor/woocommerce/subscriptions-core/build/index.js 3.04 kB

compressed-size-action

@mgascam mgascam marked this pull request as ready for review October 1, 2025 07:58
@mgascam mgascam requested a review from a team October 1, 2025 07:58
- Introduced WP-CLI commands for QIT setup and status checks to assist in development environments.
- Created a new class `WP_CLI_QIT_Dev_Command` for handling Jetpack connection setup and account data refresh.
- Added E2E test runner script and configuration for QIT, including basic tests for WooCommerce Payments functionality.
- Updated package.json to include a new script for running QIT E2E tests.
- Enhanced default environment configuration for QIT testing with necessary credentials.
- Implemented setup script for WooCommerce Payments E2E tests to configure products, customers, and Jetpack connections.
- Added ESLint configuration for E2E tests to accommodate Playwright testing framework.
- Created initial E2E test cases to verify basic functionality and plugin activation.
…nce setup functions, and add connection validation tests
- Add .github/workflows/qit-e2e.yml with comprehensive CI setup
- Support multiple WooCommerce and PHP version combinations
- Enable manual workflow dispatch with custom parameters
- Update e2e-runner.sh to handle CI environment options
- Add CI.md documentation explaining workflow usage and setup

The workflow runs on PR, push to main branches, scheduled, and manual dispatch.
Includes proper authentication and environment variable handling for CI context.
Security improvements:
- Add GitHub Actions log masking for all sensitive tokens
- Use environment variables instead of command-line arguments in WP-CLI
- Update WP_CLI_QIT_Dev_Command to prioritize env vars over CLI args
- Remove tokens from command-line to prevent exposure in process lists
- Add comprehensive security documentation and best practices

This prevents token leakage through:
- Process lists (ps aux)
- Command history
- GitHub Actions logs
- Error output and debugging info

All tests continue to pass with enhanced security measures.
@mgascam mgascam force-pushed the dev/qit-e2e-shopper-specs-migration branch from e16aa7e to 57b077c Compare November 11, 2025 19:06
@mgascam mgascam changed the base branch from dev/qit-e2e-workflows to develop November 11, 2025 19:08
@mgascam
Copy link
Contributor Author

mgascam commented Nov 13, 2025

Closing this PR in favor of smaller chunks to facilitate reviews, starting with #11133

@mgascam mgascam closed this Nov 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants