How to Run Playwright Tests in Bitbucket – Step-by-Step Guide

1. Introduction

What is Playwright?

1. Playwright is an open-source browser automation framework developed by Microsoft.
2. It supports Chromium, Firefox, and WebKit for end-to-end testing, enabling cross-browser testing in a single API.
3. Features include headless/headed execution, mobile emulation, network interception, and auto-waiting for elements.

Why Run Playwright Tests in Bitbucket?

1. Bitbucket Pipelines provides native CI/CD integration, allowing automated test execution on every code commit.
2. Combining Playwright with Bitbucket ensures consistent testing environments, faster feedback loops, and seamless collaboration across teams.
3. Bitbucket’s YAML-based configuration simplifies defining workflows for running Playwright tests.

2. Prerequisites

Node.js and npm Installation

1. Playwright requires Node.js (v14+) and npm (Node Package Manager).
2. Guide students to install Node.js from nodejs.org and verify with:

Playwright Setup

1. Install Playwright in a project using:

1. This command sets up a Playwright project, installs browsers, and creates sample tests.

Bitbucket Account and Repository

1. Students need a Bitbucket account and a Git repository to host their code.
2. Ensure they can push code to Bitbucket and enable Pipelines in repository settings.

3. Setting Up a Playwright Project

Initialize a Playwright Project

1. Walk through the Playwright initialization process:
2. Choose TypeScript/JavaScript.
3. Select browsers to test (default: all).
4. Configure folder structure (e.g., tests/, playwright.config.ts).

Writing a Sample Playwright Test

1. Create a simple test (e.g., navigate to a page and assert a title):

1. Explain test structure: test blocks, page fixtures, and assertions.

4. Integrating Playwright with Bitbucket Pipelines

Introduction to Bitbucket Pipelines

1. Bitbucket Pipelines uses a YAML file (bitbucket-pipelines.yml) to define CI/CD workflows.
2. Pipelines run in isolated Docker containers, triggered by commits or pull requests.

Configuring bitbucket-pipelines.yml

1. Example configuration for Playwright:

1. Caching: Speeds up future runs by reusing node_modules.
2. Artifacts: Save test reports for later review.

Installing Dependencies in the Pipeline

1. Use npx playwright install --with-deps to install browsers and OS dependencies in CI.

5. Running Playwright Tests in CI/CD

Configuring Headless vs. Headed Mode

1. Playwright runs in headless mode by default (no UI). To run headed:

1. In CI, headless is recommended for speed and resource efficiency.

Parallel Test Execution

1. Split tests across workers using the --workers flag:

1. Configure parallelism in playwright.config.ts:

Handling Browser Binaries in CI

1. Bitbucket’s Docker environment may lack browser dependencies. Use:

1. to install system-level dependencies (e.g., libnss3 for Chromium).

6. Artifacts and Reporting

Generating Playwright Test Reports

1. Playwright auto-generates an HTML report in playwright-report/ after tests run.
2. Customize reports with --reporter (e.g., line, dot, html).

Storing Reports as Pipeline Artifacts

1. Define artifacts in bitbucket-pipelines.yml to save reports:

1. Students can download reports from the Pipelines UI post-execution.

7. Troubleshooting Common Issues

Debugging Pipeline Failures

1. Check Pipeline logs for errors (e.g., missing dependencies, test failures).
2. Reproduce issues locally with npx playwright test.

Handling Timeouts and Flaky Tests

1. Increase timeouts in playwright.config.ts:

1. Use test.slow() to mark slow tests or retries with --retries.

Managing Browser Dependencies

1. Ensure Docker image includes required libraries (e.g., use ubuntu:latest as the base image).

8. Best Practices

Caching Dependencies

1. Cache node_modules and Playwright browsers:

Using Environment Variables

1. Store secrets (e.g., credentials) in Bitbucket Repository Variables and access them in tests via process.env.

Optimizing Test Execution

1. Split tests into shards for parallel runs:

1. Trigger Playwright Tests Using Bitbucket API

Why Use the API?

1. Run Playwright tests on-demand (e.g., via scripts or external tools).
2. Integrate testing into custom workflows (e.g., nightly runs, post-deployment checks).

Steps to Implement:

1. Bitbucket REST API Basics:
2. Authentication: Use App Passwords or OAuth 2.0.
3. Endpoint: POST /2.0/repositories/{workspace}/{repo_slug}/pipelines/
4. Trigger Pipelines via cURL:

1. Replace {workspace}, {repo}, and branch/pattern as needed.
2. Webhooks for Event-Driven Testing:
3. Configure Bitbucket webhooks to trigger tests on events like pull requests or pushes.

2. Generating and Accessing Playwright Reports

Playwright’s Built-in Reports:

1. HTML Report: Interactive dashboard with logs, screenshots, and traces.

1. JUnit/JSON Reports: For integration with tools like Jenkins or Azure DevOps.

Custom Reporters:

1. Use third-party reporters (e.g., Allure) or build custom ones.
2. Example in playwright.config.ts:

Automating Report Sharing:

1. Upload reports to cloud storage (e.g., AWS S3).
2. Use Bitbucket Notifications to email reports.

3. Debugging Failed Tests

Playwright Trace Viewer:

1. Capture traces for failed tests:

1. View traces locally:

CI Debugging Tips:

1. Run tests in headed mode temporarily in CI (add --headed).
2. Use timeout overrides for slow CI environments:

Analyzing CI Logs:

1. Check Bitbucket Pipeline logs for errors like missing dependencies or browser crashes.

4. Maintaining and Updating Tests

Version Compatibility:

1. Keep Playwright updated:

1. Use npx playwright install to update browsers.

Refactoring Tests:

1. Use the Page Object Model (POM) to organize code.
2. Example:

Handling Flaky Tests:

1. Enable retries in playwright.config.ts:

1. Use test.describe.configure({ retries: 3 }) for specific test suites.

5. Advanced Topics

Testing in Multiple Environments:

1. Use environment variables to switch configurations:

Security Best Practices:

1. Store secrets (e.g., passwords) in Bitbucket Repository Variables (not in code).
2. Restrict Pipeline permissions to least privilege.

Integrating with Test Management Tools:

1. Export results to tools like TestRail or Xray using custom scripts.

Additional Considerations

1. Performance Testing: Use Playwright to measure page load times or Lighthouse metrics.
2. Visual Regression Testing: Integrate tools like Percy or Happo.
3. Cross-Browser Testing: Configure multiple browsers in playwright.config.ts.

Conclusion

1. Recap steps: setup Playwright, configure Bitbucket Pipelines, run tests in CI, and analyze reports.
2. Emphasize benefits: automated regression testing, cross-browser compatibility, and faster releases.

Additional Resources

1. Playwright Docs: playwright.dev
2. Bitbucket Pipelines: Bitbucket CI/CD Guide

Key Takeaways for Students

1. Playwright simplifies cross-browser testing with a unified API.
2. Bitbucket Pipelines automates test execution, ensuring code quality before deployment.
3. Use artifacts, caching, and parallelization to optimize CI/CD workflows.