Custom Jest Test Runner Implementation
This challenge asks you to build a simplified, custom test runner that integrates with Jest. While Jest provides a robust testing framework, understanding how test runners work under the hood is valuable for debugging, extending Jest's functionality, or even building your own testing solutions. You'll be creating a runner that discovers and executes test files, reporting results back to Jest in a format it understands.
Problem Description
You need to implement a custom test runner that Jest can use. This runner should:
- Discover Test Files: Given a directory, identify all files matching a specified glob pattern (e.g.,
**/*.[jt]s?(x)). These files are considered potential test files. - Execute Test Files: For each discovered test file, execute the file using Node.js. The execution should be isolated (e.g., using
vmorevalcarefully, or a separate process). - Parse Test Results: The test files are expected to output test results in a specific format (described below). Your runner must parse this output and extract information about passed, failed, pending, and skipped tests.
- Report Results to Jest: Format the parsed results into a Jest-compatible format and return them. This format is a JavaScript object with properties like
numPassed,numFailed,numPending,numSkipped, and an array oftestResults(each containing information about an individual test).
Test File Output Format:
Each test file should output a JSON string to stdout representing the test results. The JSON should have the following structure:
{
"passed": 5,
"failed": 2,
"pending": 1,
"skipped": 0,
"testResults": [
{
"title": "Test 1",
"status": "passed"
},
{
"title": "Test 2",
"status": "failed",
"err": "Error message"
},
// ... more test results
]
}
status can be "passed", "failed", "pending", or "skipped". err is only present for failed tests and contains the error message.
Examples
Example 1:
Input: Directory containing two test files:
- test1.ts: `console.log(JSON.stringify({passed: 1, failed: 0, pending: 0, skipped: 0, testResults: [{title: "Test 1", status: "passed"}]});`
- test2.ts: `console.log(JSON.stringify({passed: 0, failed: 1, pending: 0, skipped: 0, testResults: [{title: "Test 2", status: "failed", err: "Something went wrong"}]});`
Glob pattern: `**/*.ts`
Output:
```json
{
"numPassed": 1,
"numFailed": 1,
"numPending": 0,
"numSkipped": 0,
"testResults": [
{
"title": "Test 1",
"status": "passed"
},
{
"title": "Test 2",
"status": "failed",
"err": "Something went wrong"
}
]
}
Explanation: The runner discovers both test files, executes them, parses the JSON output, and aggregates the results into the Jest-compatible format.
Example 2:
Input: Directory containing one test file:
- test1.ts: `console.log(JSON.stringify({passed: 2, failed: 0, pending: 1, skipped: 0, testResults: [{title: "Test A", status: "passed"}, {title: "Test B", status: "passed"}, {title: "Test C", status: "pending"}]});`
Glob pattern: `**/*.ts`
Output:
```json
{
"numPassed": 2,
"numFailed": 0,
"numPending": 1,
"numSkipped": 0,
"testResults": [
{
"title": "Test A",
"status": "passed"
},
{
"title": "Test B",
"status": "passed"
},
{
"title": "Test C",
"status": "pending"
}
]
}
Explanation: The runner discovers and executes the single test file, parses the JSON output, and aggregates the results.
Constraints
- Glob Pattern: The glob pattern will be provided as a string.
- File Execution: You must execute the test files using Node.js. Consider using
child_process.spawnorchild_process.execfor this. - Error Handling: Handle errors gracefully during file discovery and execution. If a file fails to execute, report it as a failed test with an appropriate error message.
- Performance: While not a primary concern, avoid excessively slow file discovery or execution.
- Typescript: The solution must be written in Typescript.
Notes
- This is a simplified test runner. It does not need to support all of Jest's features (e.g., mocking, spies, timers).
- Focus on the core functionality of discovering, executing, and reporting test results.
- Consider using a library like
globto simplify file discovery. - Be mindful of security implications when executing arbitrary code (test files). Sandboxing techniques might be necessary in a production environment, but are not required for this challenge.
- The Jest integration part is not required. Focus on the core runner logic. You can assume Jest will call your runner with the directory and glob pattern. Your runner should return the Jest-compatible results object.