added REST model test, tests failing

2026-07-10 21:41:28 +02:00
parent c32442166c
commit d169adf3af
16 changed files with 1967 additions and 430 deletions
@@ -0,0 +1,370 @@
+# Model Tests for Flowable Apps
+
+This page describes how to write automated tests for Flowable app models (BPMN processes, CMMN cases) in the `customer-work` module. Model tests run as plain JUnit tests against an in-memory H2 database — no Docker, no running platform needed.
+
+Test data can be provided in three ways:
+
+1. Directly in Java (simple, single-scenario tests)
+2. As JSON parameter files (one scenario per file)
+3. As Excel workbooks (many scenarios as rows; can be edited with Microsoft Excel or LibreOffice Calc)
+
+---
+
+## 1. Project structure
+
+All model-test code and data lives in the `customer-work` module:
+
+```
+customer-work/
+├── src/main/java/com/customer/work/
+│   └── service/                            # custom runtime code used by the models
+│       └── JsonUtils.java                  #   JSON conversion helpers for service tasks
+│
+├── src/test/java/com/customer/work/
+│   ├── config/
+│   │   └── TestConfiguration.java          # test Spring configuration
+│   └── model/                              # the test framework
+│       ├── FlowableModelTest.java          #   @FlowableModelTest meta-annotation
+│       ├── FlowableModelTestUtils.java     #   main utility bean (start, complete, assert, load)
+│       ├── FlowableExcelParser.java        #   reads .xlsx workbooks from the classpath
+│       ├── FlowableExcelMapper.java        #   maps workbook rows to test parameters
+│       ├── FlowableJsonParser.java         #   converts plain test data to model variable types
+│       ├── EmailDto.java                   #   captured email (subject, receivers, content)
+│       ├── TestMailServer.java             #   GreenMail SMTP server for email assertions
+│       ├── TestMailServerExtension.java    #   starts/stops the mail server per test
+│       └── test/                           # the actual test classes
+│           └── ModelTest.java              #   tests for the TST_APP models
+│
+└── src/test/resources/
+    ├── application.properties              # H2 datasource, mail server port, app location
+    ├── test-auto-deploy-apps/
+    │   └── TST_APP.zip                     # the app under test (exported from Design)
+    └── model/test/                         # test data, one directory per model
+        ├── C001/
+        │   └── T001.json                   # task completion payload
+        ├── P001/
+        │   └── initiator.json              # JSON parameter file
+        ├── P002/
+        │   ├── boolean1.json … date.json   # JSON parameter files
+        │   └── p002Test.xlsx               # Excel workbook with test cases
+        └── P005/
+            └── T001.json
+```
+
+Conventions:
+
+- Test data goes to `src/test/resources/model/test/<MODEL>/`, where `<MODEL>` is the short name of the model under test (e.g. `P002` for process `TST_P002`).
+- JSON files are named after the scenario they cover (`boolean1.json`, `initiator.json`); task completion payloads are named after the task (`T001.json`).
+- One Excel workbook per model (`p002Test.xlsx`) holds all data-driven cases for that model.
+- Test classes live in `com.customer.work.model.test`; the framework classes in `com.customer.work.model` normally do not need to be touched when adding new tests.
+
+## 2. How it works
+
+Every test deploys the app under test and starts the model through a generated **wrapper process**:
+
+```
+[Wrapper process <KEY>_T]  →  Call Activity  →  [Process under test <KEY>]
+```
+
+The wrapper is generated at runtime by `FlowableModelTestUtils.createRootTestProcessInstance(testKey, variables)`. This makes the test realistic for models that read or write variables on their **root** process (e.g. via `varutil` with the `root.` prefix), and it allows testing the model's **in/out parameter mappings**:
+
+- Variables passed at the top level are set on the **wrapper (root) process**.
+- The special variable `__IN` (a map) creates **in-mappings** on the call activity: each entry `"name": value` maps `${__IN.name}` to the variable `name` of the called process.
+- The special variable `__OUT` (a map) creates **out-mappings**: each entry `"source": "target"` copies variable `source` of the called process to variable `target` of the root process when the call activity completes.
+
+The method returns the IDs of both instances:
+
+| Key | Content |
+|---|---|
+| `ROOT_PROCESS_ID` | ID of the generated wrapper (root) process instance |
+| `TEST_PROCESS_ID` | ID of the process under test (the call activity child) |
+
+Assertions are usually made against the **historic payload of the root process** (`getHistProcessPayload`), so they see both the root variables and everything mapped out of the model under test.
+
+### Test infrastructure
+
+| Piece | Purpose |
+|---|---|
+| `@FlowableModelTest` | Meta-annotation: boots the Spring context with all Flowable engine test extensions, makes each test `@Transactional` (automatic rollback) and starts the test mail server |
+| `FlowableModelTestUtils` | The main utility bean: start instances, complete tasks, execute timers, load test data, assert emails / audit records |
+| `FlowableExcelParser` / `FlowableExcelMapper` | Read Excel workbooks from the classpath and turn each data row into test parameters |
+| `TestMailServer` (GreenMail) | Captures emails sent by the model; configured via `flowable.mail.server.host/port` in `src/test/resources/application.properties` |
+
+### App deployment
+
+The app under test must be available as a zip in `src/test/resources/test-auto-deploy-apps/` (e.g. `TST_APP.zip`). It is auto-deployed at context start via:
+
+```properties
+flowable.app.resource-location=classpath*:/test-auto-deploy-apps/
+```
+
+To refresh the zip from a running Flowable Design, temporarily enable the `exportApp` test (remove `@Disabled`) and run it once:
+
+```java
+flowableModelTest.exportApp("http://localhost:8106", "myWorkspace", "TST_APP", "admin", "test");
+```
+
+---
+
+## 3. Writing a model test class
+
+Annotate the class with `@FlowableModelTest` and inject `FlowableModelTestUtils`:
+
+```java
+@FlowableModelTest
+public class ModelTest {
+
+    @Autowired
+    protected FlowableModelTestUtils flowableModelTest;
+
+    @Test
+    public void p005HappyPathTest() {
+        // Start TST_P005 wrapped in a root process, no start variables
+        ObjectNode processIds = flowableModelTest.createRootTestProcessInstance("TST_P005", flowableModelTest.emptyMap());
+
+        // Work on the open user task
+        flowableModelTest.claimOpenTask("TST_P005_T001", "admin");
+        flowableModelTest.completeOpenTask("TST_P005_T001",
+                flowableModelTest.loadObjectNodeFromResources("model/test/P005/T001.json"), null);
+
+        // Assert against the historic root process payload
+        Map<String, Object> rootPayload = flowableModelTest.getHistProcessPayload(
+                processIds.get(FlowableModelTestUtils.ROOT_PROCESS_ID).asText());
+        Assertions.assertThat(rootPayload.get("dataEntry")).isEqualTo("my root text");
+    }
+}
+```
+
+### Frequently used utility methods
+
+| Method | Purpose |
+|---|---|
+| `createRootTestProcessInstance(key, vars)` | Deploy wrapper + start the model under test, returns the two instance IDs |
+| `startCaseInstance(key, vars)` | Start a CMMN case directly |
+| `setTestAuthenticatedUser(userId, tenantId, groups...)` | Set the authenticated user for subsequent engine calls |
+| `claimOpenTask(taskKey, userId)` / `completeOpenTask(taskKey, vars, outcome)` | Claim / complete the single open task with the given task definition key. `vars` may be flat (`"a.b": 1`) — existing task variables are merged automatically |
+| `executeTimer(processId)` | Fire the active timer job of the given process instance |
+| `getHistProcessPayload(id)` / `getRuntimeCasePayload(id)` / `getHistoryCasePayload(id)` | Read variables for assertions |
+| `getMailList()` | All emails captured by the test mail server |
+| `getAuditTrail()` | All audit records |
+| `loadObjectNodeFromResources(path)` / `loadTestVarsFromResources(path)` | Load JSON test data from `src/test/resources` (see below) |
+| `getJsonArgumentsFromExcel(path)` / `getObjArgumentsFromExcel(path)` | Excel rows as JUnit parameterized-test arguments (see below) |
+| `testJsonExcelRow(path, row)` / `testObjExcelRow(path, row)` | Run one Excel row: start, fire timers, assert payload / audit / email counts |
+
+### Running the tests
+
+```bash
+cd C-2025.2
+./mvnw test -pl customer-work -Dtest=ModelTest    # one class
+./mvnw test -pl customer-work                     # whole module
+```
+
+---
+
+## 4. JSON parameter files
+
+JSON files define the start parameters for one test scenario. They live under `src/test/resources/model/test/<MODEL>/` and are loaded with `loadTestVarsFromResources(path)`.
+
+### Structure
+
+All parameters must be declared **explicitly** in one of three top-level blocks:
+
+| Block | Meaning |
+|---|---|
+| `__ROOT` | Variables set on the root (wrapper) process |
+| `__IN` | In-mappings of the call activity: `"name": value` passes the value as variable `name` into the model under test |
+| `__OUT` | Out-mappings: `"source": "target"` copies variable `source` of the model to variable `target` of the root process on completion |
+
+Every block is optional, but **any other top-level field fails the test** with:
+
+```
+IllegalArgumentException: Implicit parameter 'param' in model/test/P002/boolean1.json:
+parameters must be declared explicitly inside __ROOT, __IN or __OUT
+```
+
+Keys inside `__ROOT` must not start with `__`.
+
+### Example: `model/test/P002/boolean1.json`
+
+```json
+{
+  "__ROOT": {
+    "param": true
+  },
+  "__IN": {
+    "param": false
+  },
+  "__OUT": {
+    "result": "out"
+  }
+}
+```
+
+This sets `param = true` on the root process, passes `param = false` into `TST_P002`, and copies the model's `result` variable to the root variable `out` when the model completes.
+
+### Example: using JSON files in a parameterized test
+
+```java
+private Stream<Arguments> p002TestData() {
+    return Stream.of(
+            Arguments.of("model/test/P002/boolean1.json", true, false),
+            Arguments.of("model/test/P002/int.json", 123, 456)
+    );
+}
+
+@ParameterizedTest
+@MethodSource("p002TestData")
+public void p002Test(String path, Object result, Object out) {
+    ObjectNode processIds = flowableModelTest.createRootTestProcessInstance("TST_P002",
+            flowableModelTest.loadTestVarsFromResources(path));
+    Map<String, Object> rootPayload = flowableModelTest.getHistProcessPayload(
+            processIds.get(FlowableModelTestUtils.ROOT_PROCESS_ID).asText());
+
+    Assertions.assertThat(rootPayload.get("result")).isEqualTo(result);
+    Assertions.assertThat(rootPayload.get("out")).isEqualTo(out);
+}
+```
+
+---
+
+## 5. Excel test workbooks
+
+For data-driven testing of many scenarios, define one workbook per model, e.g. `src/test/resources/model/test/P002/p002Test.xlsx`. Both **Microsoft Excel** and **LibreOffice Calc** can be used to create and edit the file (`.xlsx` format).
+
+Each **sheet** is a group of test cases, each **data row** is one test case that starts the model, optionally fires timers, and asserts the results.
+
+### Sheet layout
+
+| Row | Content |
+|---|---|
+| 1 | **Path** of each column's value (see path notation below) |
+| 2 | **Type** of each column: `String`, `Boolean`, `Integer`, `Long`, `Double`, `Date` |
+| 3+ | One test case per row |
+
+Parsing rules:
+
+- The column count is taken from row 1: columns are read from the first cell up to the **first empty header cell**.
+- Rows are read until the **first completely empty row** — everything below is ignored (useful for comments or temporarily disabled cases).
+- **Empty cells are skipped** — the variable / check is simply not used for that row.
+- Formulas are evaluated (e.g. `=TRUE()`); the displayed value is what counts.
+
+### Path notation (row 1)
+
+The path decides where the value goes. Nested objects use `.`, array elements use `[index]`:
+
+| Path prefix | Meaning |
+|---|---|
+| `root.<name>` | Variable on the root process (like `__ROOT` in JSON) |
+| `in.<name>` | In-mapping into the model (like `__IN`) |
+| `out.<source>` | Out-mapping; the cell value is the **target** variable name (like `__OUT`) |
+| `id` | **Required.** Process definition key of the model under test, e.g. `TST_P002` |
+| `timer` | Number of timer jobs to fire after start (Integer) |
+| `test.<name>` | Expected value of variable `<name>` in the root process payload after the run |
+| `audit` | Expected total number of audit records (Integer) |
+| `email` | Expected total number of sent emails (Integer) |
+
+Nested examples:
+
+| Header | Effect |
+|---|---|
+| `root.param` | Root variable `param` |
+| `root.param[0]`, `root.param[1]` | Root variable `param` as array with two elements |
+| `root.items[1].name` | Root variable `items`, array whose second element is an object with field `name` |
+| `test.result[0]` | Asserts the first element of array variable `result` |
+
+Array elements that are skipped (e.g. only `[1]` is filled) are padded with empty objects, because Flowable cannot store `null` list elements.
+
+### Special cell values
+
+| Value | In input columns (`root.`, `in.`, `out.`) | In `test.` columns |
+|---|---|---|
+| *(empty cell)* | Variable not set | No check |
+| `__NULL` | Variable set to JSON `null` | No check (a null expectation matches anything) |
+| `__N_A` | – | Asserts the variable/field is **absent or null** |
+| `__EMPTY` | Empty string `""` (type `String` only) | Expects `""` |
+| `__BLANK` | Single space `" "` (type `String` only) | Expects `" "` |
+
+### Types (row 2)
+
+| Type | Cell content |
+|---|---|
+| `String` | Any text |
+| `Boolean` | `TRUE` / `FALSE` (or a formula like `=TRUE()`) |
+| `Integer`, `Long`, `Double` | Numeric value |
+| `Date` | A date cell displayed as `yyyy-MM-dd` or `yyyy-MM-dd HH:mm:ss`. The displayed (local) date is interpreted as **UTC** |
+
+### Example sheet
+
+| | A | B | C | D | E | F | G | H |
+|---|---|---|---|---|---|---|---|---|
+| **1** | `root.nr` | `root.param` | `in.param` | `out.result` | `id` | `timer` | `test.result` | `test.outResult` |
+| **2** | `String` | `String` | `String` | `String` | `String` | `Integer` | `String` | `String` |
+| **3** | 11 | | | | TST_P002 | | | |
+| **4** | 12 | hello root | hello | outResult | TST_P002 | 1 | hello root | hello |
+| **5** | 13 | | hello | outResult | TST_P002 | 0 | `__N_A` | `__NULL` |
+
+- Row 3: starts `TST_P002` with no parameters and no checks (smoke test).
+- Row 4: sets root and in parameters, maps `result` out to `outResult`, fires one timer, then asserts `result == "hello root"` and `outResult == "hello"`.
+- Row 5: no root param; asserts that `result` was **not** set (`__N_A`) and skips the `outResult` check.
+
+Tip: a column like `root.nr` is a handy row number — it appears in the test log line of every row, which makes failures easy to locate.
+
+### How assertions work
+
+After the run, the expected values from the `test.` columns are compared as a **subset** against the historic root process payload: every expected field must exist with the same (string-compared) value; extra variables in the payload are ignored. On failure the test prints both sides:
+
+```
+test : {"result":"hello root","outResult":"hello"}
+root : {"nr":"12","param":"hello root", ...}
+```
+
+If `audit` / `email` columns are present, the total counts are asserted as well.
+
+### Wiring the workbook into JUnit
+
+Every data row of every sheet becomes one JUnit test invocation:
+
+```java
+private Stream<Arguments> p002ExcelTestData() {
+    return flowableModelTest.getJsonArgumentsFromExcel("model/test/P002/p002Test.xlsx");
+}
+
+@ParameterizedTest
+@MethodSource("p002ExcelTestData")
+public void p002ExcelTest(String path, JsonNode argument) {
+    flowableModelTest.testJsonExcelRow(path, argument);
+}
+```
+
+The `Obj` variant works with plain Java maps and additionally returns a result map that can drive detailed email / audit assertions (they only run for rows where the `email` / `audit` count is > 0):
+
+```java
+private Stream<Arguments> p002ExcelTestData2() {
+    return flowableModelTest.getObjArgumentsFromExcel("model/test/P002/p002Test.xlsx");
+}
+
+@ParameterizedTest
+@MethodSource("p002ExcelTestData2")
+public void p002ExcelTest2(String path, Map<String, Object> argument) {
+    Map<String, Object> result = flowableModelTest.testObjExcelRow(path, argument);
+    flowableModelTest.checkAndAssertEmail(result, 1,
+            "Email", "Test", "test@flowable.com");
+    flowableModelTest.checkAndAssertAuditRecord(result, 1,
+            "Audit record", "TST_P002 Audit trail entry", "system", null);
+}
+```
+
+---
+
+## 6. Troubleshooting
+
+| Symptom | Cause / fix |
+|---|---|
+| `Implicit parameter 'x' in <file>` | A top-level field outside `__ROOT` / `__IN` / `__OUT` in a JSON parameter file — move it into the right block |
+| `Resource not found: <path>` | Path is relative to `src/test/resources` and is loaded from the classpath — check spelling and location |
+| `Column 'id' missing in row of <file>` | The Excel sheet has no `id` column, or the cell is empty for that row |
+| `No open task with key X found` | The model did not reach the expected user task — or more than one task with that key is open |
+| `No active timer` | The `timer` count in the row is higher than the number of timer jobs the model actually creates |
+| `Missing ']' in path` / `Invalid array index in path` | Malformed header path in row 1, e.g. `root.param[0` or `root.param[a]` |
+| `Variable type not supported: X` | Typo in row 2 — allowed: `String`, `Boolean`, `Integer`, `Long`, `Double`, `Date` |
+| Subset assertion fails with `test:` / `root:` output | Compare both printed JSON documents; remember values are compared as strings and `__N_A` requires absence |