Property-based Testing
A software testing method in which invariants that code should satisfy are declared, and the testing framework automatically generates large numbers of random inputs to verify that these invariants always hold. It complements traditional unit testing with manually specified inputs and outputs, and is often used to catch edge-case defects.
Timeline (last 90 days)
For the deduplication function in a data pipeline, its invariants could be 'the number of output records does not exceed the number of input records' or 'all input statuses are preserved'.
The 'transaction count preservation' check added by the author is a kind of property test, and within 30 seconds it caught the bug that had taken 3 days of manual investigation to find.
Common means of avoiding synthetic data blind spots include: sampling anonymized production data and feeding it back, explicitly injecting same-name/same-title samples into the test set, doing property-based testing on the deduplication logic, and tracking business-level metrics such as 'number of unique titles returned per query' in CI.
Combined with tools such as property-based testing and formal specifications, the black-box verification pattern can reduce the human review burden.
All Facts (4)
For the deduplication function in a data pipeline, its invariants could be 'the number of output records does not exceed the number of input records' or 'all input statuses are preserved'.
50%UnverifiedCommon means of avoiding synthetic data blind spots include: sampling anonymized production data and feeding it back, explicitly injecting same-name/same-title samples into the test set, doing property-based testing on the deduplication logic, and tracking business-level metrics such as 'number of unique titles returned per query' in CI.
50%UnverifiedCombined with tools such as property-based testing and formal specifications, the black-box verification pattern can reduce the human review burden.
50%UnverifiedThe 'transaction count preservation' check added by the author is a kind of property test, and within 30 seconds it caught the bug that had taken 3 days of manual investigation to find.
50%