@empiricalrun/playwright-utils@0.58.1
Playwright utils for test code repos of our customers
AI Security Review
scanned 3mo ago · by lpm-firewall-aiThe scanner's webhook exfiltration claim is not supported: credentials are used as API authentication for explicit provider operations. However, automatic agent invocation plus cached dynamic execution is unresolved dangerous capability.
Static reason
High-risk behavior combination matched malicious policy.
Trigger
A consumer uses baseTestFixture and an intercepted Playwright click triggers overlay dismissal.
Impact
Untrusted page content may influence an agent-driven browser action and persisted executable cache.
Mechanism
Automatic computer-use agent with cached dynamic JavaScript execution.
AI rationale
The scanner's webhook exfiltration claim is not supported: credentials are used as API authentication for explicit provider operations. However, automatic agent invocation plus cached dynamic execution is unresolved dangerous capability.
Evidence
package.jsondist/test/index.jsdist/test/scripts/pw-locator-patch/dismiss-overlays/index.jsdist/test/scripts/pw-locator-patch/dismiss-overlays/cache.jsdist/webhook.jsdist/api-worker-client.js.empiricalrun/overlay-cache.json
Network endpoints4
api.empirical.runtelemetry-worker.empirical-run.workers.devwebhook.siteinbox.empirical.run
Decision evidence
public snapshotAI called this Suspicious at 88.0% confidence as Dangerous Capability with medium false-positive risk.
Evidence for warning
- dist/test/index.js enables agentCapabilities by default in its Playwright fixture.
- dist/test/scripts/pw-locator-patch/dismiss-overlays/index.js invokes a computer-use agent after an intercepted click.
- dist/test/scripts/pw-locator-patch/dismiss-overlays/cache.js persists generated code then executes it with AsyncFunction.
Evidence against
- package.json has no preinstall, install, postinstall, or bin hook.
- Webhook code uses configured provider/API credentials for explicit webhook test helpers; it does not send environment contents to webhook.site.
- Networked reporting/uploads require explicit project, run, backend, and credential environment configuration.
- No source evidence of credential harvesting, stealth persistence, or foreign AI-agent-control writes.
Behavioral surface
ChildProcessCryptoEnvironmentVarsFilesystemNetwork
HighEntropyStringsObfuscated
Source & flagged code
6 flaggeddist/reporter/local-test.jsView file
6exports.mergeForLocalTest = mergeForLocalTest;
L7: const child_process_1 = require("child_process");
L8: const path_1 = __importDefault(require("path"));
High
Child Process
Package source references child process execution.
dist/reporter/local-test.jsView on unpkg · L613try {
L14: (0, child_process_1.execFileSync)("npx", ["playwright", "merge-reports", outputDir, "--reporter", "html,json"], {
L15: cwd: currentWorkingDir,
High
Runtime Package Install
Package source invokes a package manager install command at runtime.
dist/reporter/local-test.jsView on unpkg · L13dist/webhook.jsView file
6// ── webhook.site provider ──
L7: const WEBHOOK_SITE_BASE_URL = "https://webhook.site";
L8: const DEFAULT_TOKEN_EXPIRY = 604800;
...
L20: const expiry = options.expiry === undefined ? DEFAULT_TOKEN_EXPIRY : options.expiry;
L21: if (process.env.EMPIRICALRUN_API_KEY) {
L22: const apiClient = new api_worker_client_1.ApiWorkerClient();
...
L25: path: "/token",
L26: body: { expiry },
L27: });
...
L43: if (!response.ok) {
L44: const body = await response.text().catch(() => "");
L45: throw new Error(`webhook.site error [${response.status} ${response.statusText}]: ${body}`);
Findings
2 Critical4 High3 Medium6 Low
CriticalCredential Exfiltrationdist/webhook.js
CriticalTrigger Reachable Dangerous Capabilitydist/webhook.js
HighChild Processdist/reporter/local-test.js
HighTrigger Reachable Credential Exfiltrationdist/webhook.js
HighTrigger Reachable External Post Callbackdist/webhook.js
HighRuntime Package Installdist/reporter/local-test.js
MediumNetwork
MediumEnvironment Vars
MediumStructural Risk Force Deep Review
LowScripts Present
LowFilesystem
LowObfuscated
LowHigh Entropy Strings
LowUrl Strings
Affected versions and remediation
This report applies to @empiricalrun/playwright-utils@0.58.1.
See all versions of @empiricalrun/playwright-utils for other recorded verdicts.
- Review the evidence and your use of @empiricalrun/playwright-utils@0.58.1 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.