@empiricalrun/playwright-utils@0.66.1
Playwright utils for test code repos of our customers
AI Security Review
scanned 1d ago · by lpm-firewall-aiThe package sends content-bearing overlay and test telemetry to its own service when the related feature runs, with no telemetry disclosure found in the README. The behavior is suspicious rather than malicious because it uses the package's service and does not exfiltrate credentials to a foreign host.
Static reason
High-risk behavior combination matched malicious policy.
Trigger
When the overlay-dismissal feature runs and EMPIRICALRUN_API_KEY is set.
Impact
Overlay text or test context containing sensitive information may be disclosed to Empirical without a telemetry disclosure found in the README.
Mechanism
The feature sends an overlay.dismissed event, including page text and test details, to the telemetry endpoint using the Empirical API client.
AI rationale
The package sends content-bearing overlay and test telemetry to its own service when the related feature runs, with no telemetry disclosure found in the README. The behavior is suspicious rather than malicious because it uses the package's service and does not exfiltrate credentials to a foreign host.
Evidence
dist/telemetry.jsdist/test/scripts/pw-locator-patch/dismiss-overlays/index.jsdist/api-worker-client.js
Network endpoints1
api.empirical.run/api/v1/telemetry/track
Decision evidence
public snapshotAI called this Suspicious at 87.0% confidence as Unknown with medium false-positive risk.
Evidence for warning
- Telemetry is sent to /api/v1/telemetry/track when EMPIRICALRUN_API_KEY is set.
- The overlay.dismissed event includes overlay text and description, the failing locator, project and repository names, and test context.
- The overlay-dismissal flow collects that text and calls the telemetry function during feature execution.
- The API client defaults to https://api.empirical.run and permits a user-configured DASHBOARD_DOMAIN.
Evidence against
- The telemetry requires the package's Empirical API key and goes to its own service by default.
- The inspected manifest scripts are build, test, lint and development tasks; no install lifecycle hook was found.
- The flagged npx command runs Playwright's merge-reports command only from the reporter's local-testing path.
Behavioral surface
ChildProcessCryptoDynamicRequireEnvironmentVarsFilesystemNetworkShell
HighEntropyStrings
Source & flagged code
7 flaggeddist/reporter/local-test.jsView file
6exports.mergeForLocalTest = mergeForLocalTest;
L7: const child_process_1 = require("child_process");
L8: const path_1 = __importDefault(require("path"));
High
Child Process
Package source references child process execution.
dist/reporter/local-test.jsView on unpkg · L613try {
L14: (0, child_process_1.execFileSync)("npx", ["playwright", "merge-reports", outputDir, "--reporter", "html,json"], {
L15: cwd: currentWorkingDir,
High
Runtime Package Install
Package source invokes a package manager install command at runtime.
dist/reporter/local-test.jsView on unpkg · L13dist/kv.jsView file
3exports.KVClient = exports.KvPreconditionFailedError = void 0;
L4: const api_worker_client_1 = require("./api-worker-client");
L5: class KvPreconditionFailedError extends Error {
Findings
2 Critical5 High4 Medium5 Low
CriticalCredential Exfiltrationdist/webhook.js
CriticalTrigger Reachable Dangerous Capabilitydist/webhook.js
HighChild Processdist/reporter/local-test.js
HighShell
HighTrigger Reachable Credential Exfiltrationdist/webhook.js
HighTrigger Reachable External Post Callbackdist/webhook.js
HighRuntime Package Installdist/reporter/local-test.js
MediumDynamic Requiredist/kv.js
MediumNetwork
MediumEnvironment Vars
MediumStructural Risk Force Deep Review
LowScripts Present
LowFilesystem
LowHigh Entropy Strings
Affected versions and remediation
This report applies to @empiricalrun/playwright-utils@0.66.1.
See all versions of @empiricalrun/playwright-utils for other recorded verdicts.
- Review the evidence and your use of @empiricalrun/playwright-utils@0.66.1 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.