Level-up Your Testing Toolkit!

Janina

Micha

I am…

👨‍💻 a Software Developer/
    Software Crafting enthusiast

👧🏼 a father to a daughter

🥦 a vegetarian


I like…

🏄‍♂️ my flow to be continuous,

🙅‍♂️ my stress minimized ⇒ QA test automation

🏃‍♂️ running

🇫🇷 learning French

🍺 drinking Craft Beer

I am…

👩‍💻 a Software Developer/Crafter

💡 an Ambassador for learning

     a co-organizer for a security conference 


I like…

💡Learning & Improving ⇒ Software Teaming

🔄 Automatization

🎨 Drawing

👩‍🌾 Gardening

🎮 Video Games

🐶 Dogs 💜

Parameteriz.Testing

722-45173-78

€ fewer tests

€ test duplication

€ more values tested

free shipping (OSS)

Join the QAC
community

555722.de/78

Call now! Only one test left!! Secure the parameter!

JUnit

pytest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3))
        .isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5))
        .isEqualTo("Buzz");
  }

  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15))
        .isEqualTo("FizzBuzz");
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  // …
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_2() {
    assertThat(fizzbuzz(2))
        .isEqualTo("2");
  }

  @Test
  void fizzbuzz_4() {
    assertThat(fizzbuzz(4))
        .isEqualTo("4");
  }

  @Test
  void fizzbuzz_7() {
    assertThat(fizzbuzz(7))
        .isEqualTo("7");
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1))
        .isEqualTo("1");
  }

  @Test
  void fizzbuzz_2() {
    assertThat(fizzbuzz(2))
        .isEqualTo("2");
  }

  @Test
  void fizzbuzz_4() {
    assertThat(fizzbuzz(4))
        .isEqualTo("4");
  }

  @Test
  void fizzbuzz_7() {
    assertThat(fizzbuzz(7))
        .isEqualTo("7");
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }
  
  // …
}
class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {5, 10, 20, 25, 35, 40, 50, 55})
  void fizzbuzz_divisible_by_5(int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {15, 30, 45, 60, 75, 90, 105, 120})
  void fizzbuzz_divisible_by_3_and_5(int divisibleBy3And5) {
    assertThat(fizzbuzz(divisibleBy3And5))
        .isEqualTo("FizzBuzz");
  }
}

Value

  • Unit testing
  • Coding required
  • Often included in your test framework
  • Many libs available
  • Property-Based Testing
  • Unit Testing

Build

€ more values tested

722-45173-78

€ test duplication

€ fewer tests

Parameterized Testing

  • Less test code duplication
  • Better expression of intent
  • More variants tested

1.000.000s

Application

Buyers also liked…

Energy Label

Property-Based Testing

722-45173-728

€ refactor unit tests

€ find surprises late

€ one at a time

free shipping (OSS)

Join the QAC
community

555722.de/728

I propose to properly test properties

QuickCheck

jqwik, QuickTheories

JSVerify

Hypothesis

Rantly

FsCheck

class FizzBuzzTest {

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {1, 2, 4, 7, 8, 11, 13, 14})
  void fizzbuzz_not_divisible_by_3_or_5(int notBy3Or5) {
    assertThat(fizzbuzz(notBy3Or5))
        .isEqualTo(String.valueOf(notBy3Or5));
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {5, 10, 20, 25, 35, 40, 50, 55})
  void fizzbuzz_divisible_by_5(int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {15, 30, 45, 60, 75, 90, 105, 120})
  void fizzbuzz_divisible_by_3_and_5(int divisibleBy3And5) {
    assertThat(fizzbuzz(divisibleBy3And5))
        .isEqualTo("FizzBuzz");
  }
}

class FizzBuzzTest {

  // …

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  // …
}

class FizzBuzzTest {

  // …
  
  @Property
  void fizzbuzz_divisible_by_3_property(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  @ParameterizedTest(name = "fizzbuzz({0})")
  @ValueSource(ints = {3, 6, 9, 12, 18, 21, 24, 27})
  void fizzbuzz_divisible_by_3(int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }

  // …
}

class FizzBuzzTest {

  // …
  
  @Property
  void fizzbuzz_divisible_by_3(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  // …
}

timestamp = 2024-07-16T09:50:25.850958, PropertyBasedTest:fizzbuzz divisible by 3 = 
                              |-----------------------jqwik-----------------------
tries = 1000                  | # of calls to property
checks = 1000                 | # of not rejected calls
generation = RANDOMIZED       | parameters are randomly generated
after-failure = SAMPLE_FIRST  | try previously failed sample, then previous seed
when-fixed-seed = ALLOW       | fixing the random seed is allowed
edge-cases#mode = MIXIN       | edge cases are mixed in
edge-cases#total = 1          | # of all combined edge cases
edge-cases#tried = 1          | # of edge cases tried in current run
seed = 7192800264771943696    | random seed to reproduce generated values
class FizzBuzzTest {

  // …

  @Property
  void fizzbuzz_divisible_by_3(@ForAll("divisibleBy3") int divisibleBy3) {
    assertThat(fizzbuzz(divisibleBy3))
        .isEqualTo("Fizz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy3() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 == 0 && i % 5 != 0);
  }

  @Property
  void fizzbuzz_divisible_by_5(@ForAll("divisibleBy5") int divisibleBy5) {
    assertThat(fizzbuzz(divisibleBy5))
        .isEqualTo("Buzz");
  }
  @Provide
  Arbitrary<Integer> divisibleBy5() {
    return Arbitraries
        .integers()
        .filter(i -> i % 3 != 0 && i % 5 == 0);
  }
  
  // …
}

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

Value

  • Unit testing
  • Somewhat advanced coding skills required
  • Many libs available
  • Parameterized Testing
  • Unit Testing

Build

  • Less test code duplication
  • Better expression of intent
  • More variants tested
  • …

Application

Buyers also liked…

Energy Label

1.000.000s

Approval Testing

722-45173-28

€ one verification

€ long assertions

€ see all differences

free shipping (OSS)

Join the QAC
community

555722.de/28

I approve this tooling!

Approval Tests for Node.js

… ApprovalTests

record Address(
    String id,
    String firstName,
    String lastName,
    String streetName,
    String houseNumber,
    String city,
    String country,
    String phone,
    String latitude,
    String longitude,
    String email,
    String postalCode
) {}

record Order(
    String id,
    int version,
    List<Item> items,
    List<Coupon> coupons,
    LocalDateTime orderTimeStamp,
    LocalDate deliveryDate,
    List<Price> shippingCost,
    Customer customer,
    Address shippingAddress,
    Address billingAddress
) {}

record Item(
    String id,
    String name,
    int amount,
    Price price
) {}

record Coupon(
    String id,
    String description,
    int reducedRateInPercentage
) {}

record Price(
    int value,
    String monetaryUnit,
    String currency
) {}

record Customer(
    String id,
    String firstName,
    String lastName
) {}

class ShopOrderTest {

  @Test
  void assertionTest() throws JsonProcessingException {
  	// given
    String orderId = "someOrderId";
    ShopOrder order = aDefaultOrder(orderId);
        
    // when
    anOrderWasProcessed(order);
	
    // then
    OrderResult orderResult = jsonMapper.readValue(callRestEndpoint(orderId), OrderResult.class);

    assertThat(orderResult.getId()).isEqualTo("someOrderId");
    assertThat(orderResult.getVersion()).isEqualTo(1);

    ItemResult item = orderResult.getItems().getFirst();
    assertThat(item.getId()).isEqualTo("someItemId");
    assertThat(item.getName()).isEqualTo("ATD 3 Conf. Days");
    assertThat(item.getAmount()).isEqualTo(2);

    PriceResult itemPrice = item.getPrice();
    assertThat(itemPrice.getValue()).isEqualTo(225000);
    assertThat(itemPrice.getMonetaryUnit()).isEqualTo("cent");
    assertThat(itemPrice.getCurrency()).isEqualTo("EUR");

    CouponResult coupon = orderResult.getCoupons().getFirst();
    assertThat(coupon.getId()).isEqualTo("someCouponId");
    assertThat(coupon.getDescription()).isEqualTo("Speaker Coupon");
    assertThat(coupon.getReducedRateInPercentage()).isEqualTo(100);

    assertThat(orderResult.getOrderTimeStamp()).isEqualTo(LocalDateTime.of(2024, 7, 19, 11, 45));
    assertThat(orderResult.getDeliveryDate()).isEqualTo(LocalDate.of(2024, 11, 22));

    PriceResult shippingCost = orderResult.getShippingCost().getFirst();
    assertThat(shippingCost.getValue()).isEqualTo(500);
    assertThat(shippingCost.getMonetaryUnit()).isEqualTo("cent");
    assertThat(shippingCost.getCurrency()).isEqualTo("EUR");

    CustomerResult customer = orderResult.getCustomer();
    assertThat(customer.getId()).isEqualTo("someCustomerId");
    assertThat(customer.getFirstName()).isEqualTo("REWE");
    assertThat(customer.getLastName()).isEqualTo("Digital");

    AddressResult shippingAddress = orderResult.getShippingAddress();
    assertThat(shippingAddress.getId()).isEqualTo("someShippingAddressId");
    assertThat(shippingAddress.getFirstName()).isEqualTo("Janina");
    assertThat(shippingAddress.getLastName()).isEqualTo("Nemec");
    assertThat(shippingAddress.getStreetName()).isEqualTo("Schanzenstr.");
    assertThat(shippingAddress.getHouseNumber()).isEqualTo("6-20");
    assertThat(shippingAddress.getPostalCode()).isEqualTo("51063");
    assertThat(shippingAddress.getCity()).isEqualTo("Köln");
    assertThat(shippingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(shippingAddress.getPhone()).isEqualTo("0221 9758420");
    assertThat(shippingAddress.getLatitude()).isEqualTo("50.96490882194811");
    assertThat(shippingAddress.getLongitude()).isEqualTo("7.014472855463499");
    assertThat(shippingAddress.getEmail()).isEqualTo("kontakt@rewe-digital.com");

    AddressResult billingAddress = orderResult.getBillingAddress();
    assertThat(billingAddress.getId()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.getFirstName()).isEqualTo("Micha");
    assertThat(billingAddress.getLastName()).isEqualTo("Kutz");
    assertThat(billingAddress.getStreetName()).isEqualTo("Domstr.");
    assertThat(billingAddress.getHouseNumber()).isEqualTo("20");
    assertThat(billingAddress.getPostalCode()).isEqualTo("50668");
    assertThat(billingAddress.getCity()).isEqualTo("Köln");
    assertThat(billingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(billingAddress.getPhone()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.getLatitude()).isEqualTo("50.94603935915518");
    assertThat(billingAddress.getLongitude()).isEqualTo("6.959302840118697");
    assertThat(billingAddress.getEmail()).isEqualTo("info@rewe-group.com");
  }
}

record Address(
    String id,
    String firstName,
    String lastName,
    String streetName,
    String houseNumber,
    String city,
    String country,
    String phone,
    String latitude,
    String longitude,
    String email,
    String postalCode
) {}

record Order(
    String id,
    int version,
    List<Item> items,
    List<Coupon> coupons,
    LocalDateTime orderTimeStamp,
    LocalDate deliveryDate,
    List<Price> shippingCost,
    Customer customer,
    Address shippingAddress,
    Address billingAddress
) {}

record Item(
    String id,
    String name,
    int amount,
    Price price
) {}

record Coupon(
    String id,
    String description,
    int reducedRateInPercentage
) {}

record Price(
    int value,
    String monetaryUnit,
    String currency
) {}

record Customer(
    String id,
    String firstName,
    String lastName
) {}

record Address(
    String id,
    String firstName,
    String lastName,
    String streetName,
    String houseNumber,
    String city,
    String country,
    String phone,
    String latitude,
    String longitude,
    String email,
    String postalCode
) {}

class AddressAssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder shopOrder = anyOrder(orderId)
        .billingAddress(anAddress().id("someBillingAddressId")
            .firstName("Janina").lastName("Nemec")
            .streetName("Domstr.").houseNumber("20")
            .postalCode("50668").city("Köln").country("Deutschland")
            .phone("+49 221 1490").email("info@rewe-group.com").build()
        ).build();

    anOrderWasProcessed(shopOrder);

    AddressResult billingAddress = jsonMapper.readValue(callRestEndpointForBillingAddress(orderId));
    assertThat(billingAddress.getId()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.getFirstName()).isEqualTo("Janina");
    assertThat(billingAddress.getLastName()).isEqualTo("Nemec");
    assertThat(billingAddress.getStreetName()).isEqualTo("Domstr.");
    assertThat(billingAddress.getHouseNumber()).isEqualTo("20");
    assertThat(billingAddress.getPostalCode()).isEqualTo("50668");
    assertThat(billingAddress.getCity()).isEqualTo("Köln");
    assertThat(billingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(billingAddress.getPhone()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.getLatitude()).isEqualTo("50.94603935915518");
    assertThat(billingAddress.getLongitude()).isEqualTo("6.959302840118697");
    assertThat(billingAddress.getStatus()).isEqualTo(CustomerStatus.NEW_CUSTOMER);
    assertThat(billingAddress.getEmail()).isEqualTo("info@rewe-group.com");
  }
}

class AddressAssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder shopOrder = anyOrder(orderId)
        .billingAddress(anAddress().id("someBillingAddressId")
            .firstName("Janina").lastName("Nemec")
            .streetName("Domstr.").houseNumber("20")
            .postalCode("50668").city("Köln").country("Deutschland")
            .phone("+49 221 1490").email("info@rewe-group.com").build()
        ).build();

    anOrderWasProcessed(shopOrder);

    AddressResult billingAddress = jsonMapper.readValue(callRestEndpointForBillingAddress(orderId));
    assertThat(billingAddress.getId()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.getFirstName()).isEqualTo("Janina");
    assertThat(billingAddress.getLastName()).isEqualTo("Nemec");
    assertThat(billingAddress.getStreetName()).isEqualTo("Domstr.");
    assertThat(billingAddress.getHouseNumber()).isEqualTo("20");
    assertThat(billingAddress.getPostalCode()).isEqualTo("50668");
    assertThat(billingAddress.getCity()).isEqualTo("Köln");
    assertThat(billingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(billingAddress.getPhone()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.getStatus()).isEqualTo(CustomerStatus.NEW_CUSTOMER);
    assertThat(billingAddress.getEmail()).isEqualTo("info@rewe-group.com");
  }
}


{
  "id": "someBillingAddressId",
  "firstName": "Janina",
  "lastName": "Nemec",
  "streetName": "Domstr.",
  "houseNumber": "20",
  "city": "Köln",
  "country": "Deutschland",
  "phone": "+49 221 1490",
  "latitude": "50.94603935915518",
  "longitude": "6.959302840118697",
  "email": "info@rewe-group.com",
  "postalCode": "50668",
  "status": "new_customer"
}


{
  "id": "someBillingAddressId",
  "firstName": "Janina",
  "lastName": "Nemec",
  "streetName": "Domstr.",
  "houseNumber": "20",
  "city": "Köln",
  "country": "Deutschland",
  "phone": "+49 221 1490",
  "email": "info@rewe-group.com",
  "postalCode": "50668",
  "status": "new_customer"
}

Expected

Actual

class AddressAssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder shopOrder = anyOrder(orderId)
        .billingAddress(anAddress().id("someBillingAddressId")
            .firstName("Janina").lastName("Nemec")
            .streetName("Domstr.").houseNumber("20")
            .postalCode("50668").city("Köln").country("Deutschland")
            .phone("+49 221 1490").email("info@rewe-group.com").build()
        ).build();

    anOrderWasProcessed(shopOrder);

    AddressResult billingAddress = jsonMapper.readValue(callRestEndpointForBillingAddress(orderId));
    assertThat(billingAddress.getId()).isEqualTo("someBillingAddressId");
    assertThat(billingAddress.getFirstName()).isEqualTo("Janina");
    assertThat(billingAddress.getLastName()).isEqualTo("Nemec");
    assertThat(billingAddress.getStreetName()).isEqualTo("Domstr.");
    assertThat(billingAddress.getHouseNumber()).isEqualTo("20");
    assertThat(billingAddress.getPostalCode()).isEqualTo("50668");
    assertThat(billingAddress.getCity()).isEqualTo("Köln");
    assertThat(billingAddress.getCountry()).isEqualTo("Deutschland");
    assertThat(billingAddress.getPhone()).isEqualTo("+49 221 1490");
    assertThat(billingAddress.getStatus()).isEqualTo(CustomerStatus.NEW_CUSTOMER);
    assertThat(billingAddress.getEmail()).isEqualTo("info@rewe-group.com");
  }
}


class AddressAssertionTest {

  @Test
  void assertionTest() throws JsonProcessingException {
    String orderId = "someOrderId";
    ShopOrder shopOrder = anyOrder(orderId)
        .billingAddress(anAddress().id("someBillingAddressId")
            .firstName("Janina").lastName("Nemec")
            .streetName("Domstr.").houseNumber("20")
            .postalCode("50668").city("Köln").country("Deutschland")
            .phone("+49 221 1490").email("info@rewe-group.com").build()
        ).build();

    anOrderWasProcessed(shopOrder);

    JsonApprovals.verifyJson(callRestEndpointForBillingAddress(orderId));
  }
}

Actual

Expected

722-45173-28

€ see all differences

€ one verification

Approval Testing

€ long assertions

Value

  • Unit testing, integration testing, (GUI) testing
  • A little coding skills required
  • Many libs available
  • Exploratory Testing

Build

  • More concise result verification
  • More complete result verification

Application

Buyers also liked…

Energy Label

1.000.000s

Mutation Testing

722-45173-68

€ create mutants

€ untested code

€ find missing cases

free shipping (OSS)

Join the QAC
community

555722.de/68

Teenage Mutatant Hero Testing

PIT

Mutant

Stryker Mutator

Mutatest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
$ ./gradlew pitest

class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 4/5 (80%)
>> Generated 10 mutations Killed 8 (80%)
>> Mutations with no coverage 1. Test strength 89%
>> Ran 11 tests (1.1 tests per mutation)
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isNotNull();
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 5/5 (100%)
>> Generated 10 mutations Killed 8 (80%)
>> Mutations with no coverage 0. Test strength 80%
>> Ran 13 tests (1.3 tests per mutation)
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isNotNull();
  }
}
class FizzBuzzTest {

  @Test
  void fizzbuzz_1() {
    assertThat(fizzbuzz(1)).isEqualTo("1");
  }

  @Test
  void fizzbuzz_3() {
    assertThat(fizzbuzz(3)).isEqualTo("Fizz");
  }

  @Test
  void fizzbuzz_5() {
    assertThat(fizzbuzz(5)).isEqualTo("Buzz");
  }
 
  @Test
  void fizzbuzz_15() {
    assertThat(fizzbuzz(15)).isEqualTo("FizzBuzz");
  }
}
$ ./gradlew pitest
…
================================================================================
- Statistics
================================================================================
>> Line Coverage (for mutated classes only): 5/5 (100%)
>> Generated 10 mutations Killed 10 (100%)
>> Mutations with no coverage 0. Test strength 100%
>> Ran 11 tests (1.1 tests per mutation)

Value

  • Unit testing
  • Tooling can be tricky
  • Static code analysis

Build

  • Detect holes in your test coverage
  • Much better insight compared to code coverage

Application

Buyers also liked…

Energy Label

10× less

€ find missing cases

722-45173-68

€ untested code

€ create mutants

Mutation Testing

Ensemble Testing

722-45173-38

€ early feedback

€ works on my machine

€ learn & improve

free shipping (OSS)

Join the QAC
community

555722.de/38

Let's test together!!

"All the brilliant people working on the same thing, at the same time, in the same space, and on the same computer"

- Woody Zuill

Ensemble Testing

Ensemble Testing

Why do ensemble testing?

  • More ideas what to test

  • Better insight into user perspective for developers

  • Developers can help improve test setup

  • The product owner can give direct feedback

  • Can reduce pressure

  • Everyone learns from each other

Plan

Understand

Value

  • Great (free) remote tooling available
  • Office setup also possible (hardware separately available)
  • Please adhere to the manual to avoid known side effects
  • Can be applied to manual testing and automation
  • Team programming
  • Exploratory testing

Build

  • Get everyone on the team involved in testing
  • Learn from each other
  • More immediate feedback & problem solving

Application

Buyers also liked…

Energy Label

More insights

€ learn & improve

722-45173-38

€ works on my machine  

€ early feedback

Ensemble Testing

Feature Toggles

722-45173-387

€ canary testing

€ everyone suffers

€ ab-testing

free shipping (OSS)

Join the QAC
community

555722.de/38

Throw in on production like it's 1999 :)

How are feature toggles related to testing?

Testing in production

Canary Testing

A/B-Testing

Timed release

Easy Rollback

Plan

Understand

Value

  • Great (free) remote tooling available
  • ⚠️ Important to coordinate toggles across the product
  • 🧹 Cleaning required: remove toggles after release!
  • Continuous delivery & deployment (CD)
  • Monitoring & alerting
  • Service/Site Reliability Engineering (SRE)

Build

  • Deployment ≠ Release ⇒ more continuous work
  • Seamless and coordinated rollbacks possible

Application

Buyers also liked…

Energy Label

More insights

722-45173-387

€ a/b testing

€ delayed releases

€ canary releases

Feature Toggles

€ more values tested

722-45173-78

€ test duplication

€ fewer tests

Parameterized Testing

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

€ see all differences

€ one verification

722-45173-28

Approval Testing

€ long assertions

€ one at a time

722-45173-728

€ find surprises late

€ refactor unit tests

Property-Based Testing

€ find missing cases

722-45173-68

€ untested code

€ create mutants

Mutation Testing

€ learn & improve

722-45173-38

€ works on my machine  

€ early feedback

Ensemble Testing

722-45173-387

€ a/b testing

€ delayed releases

€ canary releases

Feature Toggles

Rate the session 🙏

Get the slides 👆