Purpose
This policy establishes the requirements for software testing at Carriyo, ensuring that all software deployed to production has been adequately tested for functionality, security, performance, and reliability. Comprehensive testing reduces the risk of defects, vulnerabilities, and service disruptions reaching the production environment.
Scope
This policy applies to all software developed, modified, or integrated by Carriyo, including application code, APIs, infrastructure-as-code, database migrations, and third-party integrations. It applies to all engineering personnel involved in software development and testing.
Policy Statements
Testing Principles
- Testing is an integral part of Carriyo's software development lifecycle, not an afterthought.
- The goal of testing is to identify defects and vulnerabilities as early as possible, when they are least costly to fix.
- Automated testing is preferred and mandated for regression coverage. Manual testing supplements automation for exploratory and usability scenarios.
- Test environments must be representative of production but must never use real customer data unless anonymised or synthetic.
Unit Testing
- All application code must have unit tests covering core business logic and critical code paths.
- Minimum unit test coverage thresholds are defined per service and enforced in the CI/CD pipeline.
- Unit tests must be fast, isolated, and repeatable. Tests that depend on external services must use mocks or stubs.
- Developers are responsible for writing and maintaining unit tests for the code they produce.
Integration Testing
- Integration tests verify that components and services interact correctly across boundaries (API contracts, database interactions, message queues, third-party integrations).
- Integration tests are executed automatically as part of the CI/CD pipeline.
- Critical integration points (payment processing, carrier API integrations, webhook delivery) have dedicated integration test suites.
End-to-End Testing
- End-to-end (E2E) tests validate complete user workflows and business processes through the full application stack.
- E2E tests are maintained for critical customer-facing workflows, including shipment creation, tracking, notifications, and returns.
- E2E tests are executed in a staging environment that mirrors the production architecture.
Security Testing (A.8.29)
- Security testing is integrated into the CI/CD pipeline through automated tools:
- SAST (Static Application Security Testing): Analyses source code for vulnerabilities.
- SCA (Software Composition Analysis): Identifies known vulnerabilities in dependencies.
- DAST (Dynamic Application Security Testing): Tests running applications for vulnerabilities.
- Container scanning: Checks container images for known CVEs.
- Critical and high severity security findings must be remediated before production deployment.
- Annual penetration testing is conducted by an independent third party. Findings are tracked and remediated per the Risk Assessment Policy (CAR-ISMS-POL-015).
- Security regression tests are maintained for previously identified vulnerabilities to prevent reintroduction.
Performance Testing
- Performance and load testing is required for changes that may affect system capacity, response times, or resource consumption.
- Baseline performance metrics are established for critical services and monitored continuously.
- Load tests simulate expected peak traffic to verify that the system meets performance SLAs.
- Performance degradation beyond defined thresholds blocks production deployment.
Test Data Management
- Production customer data must not be used in test environments unless it has been fully anonymised or replaced with synthetic data.
- Test data generation tools and fixtures are maintained to provide realistic but non-sensitive test data.
- Test environments are periodically refreshed with updated anonymised data sets.
- Access to test environments follows the principle of least privilege.
Defect Management
- Defects discovered during testing are logged, classified by severity, and tracked to resolution.
- Critical and high severity defects must be resolved before production deployment.
- Defect trends are analysed to identify systemic issues and drive process improvements.
Third-Party and Dependency Testing (A.8.33)
- Third-party components and dependencies are tested for compatibility and security before adoption.
- Dependency updates are tested through the standard CI/CD pipeline before deployment.
- Major dependency upgrades undergo additional testing, including regression testing of affected functionality.
Roles and Responsibilities
| Role | Responsibility |
|---|---|
| Engineering Teams | Write and maintain tests; achieve coverage targets; remediate defects; manage test data |
| QA Engineers | Design test strategies; maintain E2E test suites; conduct exploratory testing; manage defect triage |
| DevOps | Maintain CI/CD pipeline testing stages; manage test environments; ensure security scans are operational |
| Security Team | Define security testing requirements; review security scan results; coordinate penetration testing |
| Engineering Leads | Ensure testing standards are met; approve releases; allocate time for testing activities |
| CISO | Approve policy; oversee security testing programme; review penetration test findings |
Compliance and Enforcement
Compliance is monitored through CI/CD pipeline metrics (test pass rates, coverage levels, security scan results), defect tracking reports, and periodic audits of testing practices. Releases that bypass mandatory testing stages are treated as policy violations. Persistent gaps in test coverage or quality are escalated to engineering leadership and, if necessary, to the CISO.