@empiricalrun/playwright-utils@0.63.1
Playwright utils for test code repos of our customers
AI Security Review
scanned 20d ago · by lpm-firewall-aiThe cache replay creates a real, conditional code-execution capability: code influenced by page content is compiled and run in Node during later overlay handling. No install hook or direct credential exfiltration was established, so the appropriate verdict is suspicious rather than malicious.
Static reason
High-risk behavior combination matched malicious policy.; previous stored version diff introduced dangerous source
Trigger
A Playwright click or hover is intercepted by an overlay, followed by another matching overlay dismissal during a test run.
Impact
Agent-generated code runs in the package's Node process and may access its runtime privileges and environment. The inspected path does not itself show credential transfer.
Mechanism
The handler sends overlay text to the agent, saves its returned code in a project-local cache, then compiles and executes matching cached code with AsyncFunction.
AI rationale
The cache replay creates a real, conditional code-execution capability: code influenced by page content is compiled and run in Node during later overlay handling. No install hook or direct credential exfiltration was established, so the appropriate verdict is suspicious rather than malicious.
Evidence
dist/test/scripts/pw-locator-patch/actions/click.jsdist/test/scripts/pw-locator-patch/dismiss-overlays/index.jsdist/test/scripts/pw-locator-patch/dismiss-overlays/prompt.jsdist/test/scripts/pw-locator-patch/dismiss-overlays/cache.jsdist/api-worker-client.jspackage.json
Network endpoints1
api.empirical.run
Decision evidence
public snapshotAI called this Suspicious at 86.0% confidence as Dangerous Capability with medium false-positive risk.
Evidence for warning
- The manifest exports runtime modules and defines no install lifecycle hook.
- When a Playwright click encounters an overlay, the handler invokes dismissal and saves the agent result.
- The overlay text is included in the agent task, and the returned code is stored as a cache candidate.
- The cache is written under the project directory and replayed when overlay markup and text match.
- Replay compiles cached code with Node's AsyncFunction constructor and executes it.
- The agent uses the Empirical LLM proxy, whose default host is api.empirical.run.
Evidence against
- Execution is conditional on a failed click or hover caused by an overlay and a later matching cache entry.
- The inspected sources show no install-time execution or direct transfer of credentials by this code path.
Behavioral surface
ChildProcessCryptoDynamicRequireEnvironmentVarsFilesystemNetworkShell
Source & flagged code
8 flaggeddist/reporter/local-test.jsView file
6exports.mergeForLocalTest = mergeForLocalTest;
L7: const child_process_1 = require("child_process");
L8: const path_1 = __importDefault(require("path"));
High
Child Process
Package source references child process execution.
dist/reporter/local-test.jsView on unpkg · L613try {
L14: (0, child_process_1.execFileSync)("npx", ["playwright", "merge-reports", outputDir, "--reporter", "html,json"], {
L15: cwd: currentWorkingDir,
High
Runtime Package Install
Package source invokes a package manager install command at runtime.
dist/reporter/local-test.jsView on unpkg · L13dist/kv.jsView file
3exports.KVClient = void 0;
L4: const api_worker_client_1 = require("./api-worker-client");
L5: class KVClient {
Medium
Findings
3 Critical5 High4 Medium5 Low
CriticalCredential Exfiltrationdist/webhook.js
CriticalTrigger Reachable Dangerous Capabilitydist/webhook.js
CriticalPrevious Version Dangerous Deltadist/test/scripts/pw-locator-patch/dismiss-overlays/cache.js
HighChild Processdist/reporter/local-test.js
HighShell
HighTrigger Reachable Credential Exfiltrationdist/webhook.js
HighTrigger Reachable External Post Callbackdist/webhook.js
HighRuntime Package Installdist/reporter/local-test.js
MediumDynamic Requiredist/kv.js
MediumNetwork
MediumEnvironment Vars
MediumStructural Risk Force Deep Review
Low
Affected versions and remediation
This report applies to @empiricalrun/playwright-utils@0.63.1.
See all versions of @empiricalrun/playwright-utils for other recorded verdicts.
- Review the evidence and your use of @empiricalrun/playwright-utils@0.63.1 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.