Dynamic Test Data Seeding & Cross-Platform Order History Automation for an E-commerce Platform
An e-commerce platform launched a new Order History feature across web and iOS but had no real user orders in production-like environments - making it difficult to validate different order states, edge cases, and UI behavior with large datasets. We designed and delivered a test automation strategy built on API-level validation, dynamic per-run data seeding, and cross-platform UI testing with Cypress and XCUITest - giving the team repeatable, environment-independent runs and full coverage of order scenarios across both platforms.
Background & Challenge
- No Production-Like Order Data: The new Order History feature had no real user orders available in production-like environments, blocking realistic validation.
- Multiple Order States to Validate: Orders could exist in completed, cancelled, refunded, or pending states, each requiring distinct verification.
- Edge Cases Without Coverage: Empty histories and API failures had no reliable way to be triggered or asserted against.
- UI Behavior With Large Datasets: Pagination and performance under realistic load could not be validated without controllable data volumes.
- Limited Backend Dependencies & Insufficient UI Tests: The team did not control production data, backend dependencies were limited, and UI tests alone could not deliver the required coverage.
How We Helped
Strategic Assessment & Approach Evaluation
- Mapped the testing problem as a test data control problem rather than a pure testing problem.
- Evaluated four data strategies - mocking API responses, a dedicated dummy endpoint, seeding test accounts with real orders, and a hybrid approach - against their control, realism, and maintenance tradeoffs.
- Defined a hybrid baseline: mocks for edge cases, seeded data for core flows, and minimal real backend validation tests.
API-Level Automation Foundation
- Built an API automation layer that creates orders programmatically, fetches order history, and asserts pagination and status transitions.
- Established this layer as the foundation, ensuring backend correctness independently of the UI and isolating UI defects from backend issues.
Dynamic Test Data Seeding
- Replaced static accounts and dummy endpoints with per-run data generation - each run creates a unique test user, seeds orders via API, tags the data with a run identifier (e.g., qa-run-123), and cleans up after execution.
- Delivered a data lifecycle that is fully controlled, repeatable, and environment-independent, supporting full end-to-end coverage.
Cross-Platform UI Automation
- Built UI test suites in Cypress for web and XCUITest for iOS, running main flows against the real backend with seeded data.
- Reserved mocks for edge cases only - empty states and API failures such as 500 errors and timeouts - while keeping pagination and status rendering tested against real backend behavior.
- Avoided over-reliance on mocks to keep tests stable without drifting from real backend behavior.
Scenario Matrix & Cross-Platform Validation
- Automated a scenario matrix covering users with zero orders, one order, many orders (pagination), and mixed statuses, executed in CI on every build.
- Used the same seeded user across web and iOS and compared order count, statuses, and pagination between platforms to detect inconsistencies early.
Outcomes & Benefits
- Fully Controlled, Repeatable Test Data: Dynamic seeding replaced static accounts and dummy endpoints with a per-run data lifecycle that is fully controlled and repeatable across runs.
- Environment-Independent Execution: Tests run consistently across environments without manual data setup or environment-specific fixtures.
- Full End-to-End Scenario Coverage: The scenario matrix covers users with zero, one, and many orders, plus mixed statuses, on every build in CI.
- Backend Correctness Validated Independently of UI: API-level automation confirms order logic, pagination, and status transitions before UI tests run, isolating layer-specific defects.
- Stable Tests Without Mock Drift: Using real backend with seeded data for core flows and mocks only for edge cases keeps suites stable while avoiding drift from real backend behavior.
- Early Cross-Platform Inconsistency Detection: Sharing the same seeded user across web and iOS surfaces parity issues in order count, statuses, and pagination during CI rather than after release.
Tech Stack
- Cypress for web UI automation
- XCUITest for iOS UI automation
- Stripe sandbox
- PostgreSQL
- Postman
- GitHub and GitHub Actions
- Bitrise
- Swift
- Typescript
- Python
