evals@2.10.0
Arize AX onboarding — instrument your app with tracing via your coding agent
AI Security Review
scanned 2mo ago · by lpm-firewall-aiThe package has a real user-invoked global agent-extension and telemetry capability, including silent global skill installation. Its install lifecycle is inert and the broader tracing setup is explicitly gated, so the source supports a warning rather than a block.
Static reason
One or more suspicious static signals were detected.
Trigger
Run npx evals, select a detected coding agent, and later explicitly approve agent tracing.
Impact
Remote skills can persist in the selected agent's global scope; approved tracing can send prompts and tool data to Arize.
Mechanism
Global remote skill installation and opt-in agent-session telemetry.
AI rationale
The package has a real user-invoked global agent-extension and telemetry capability, including silent global skill installation. Its install lifecycle is inert and the broader tracing setup is explicitly gated, so the source supports a warning rather than a block.
Evidence
cli.jsonboarding-prompt.mdanalytics.jsvendor/coding_harness_tracing-0.1.0-py3-none-any.whl~/.arize/anon_client_id~/.arize/onboarding~/.arize/harness~/.codex/config.toml
Network endpoints2
app.arize.com/api/analytics/npx-evalsgithub.com/Arize-ai/coding-harness-tracing.git
Decision evidence
public snapshotAI called this Suspicious at 91.0% confidence as Dangerous Capability with low false-positive risk.
Evidence for warning
- After a user selects an agent, the CLI silently runs an unpinned remote global skills installation with confirmation suppressed.
- The bundled agent-tracing flow is machine-wide and can collect prompts, tool commands, paths, URLs, and tool output after an explicit approval.
- The bundled analytics code persists a stable anonymous identifier and posts allowlisted funnel events to Arize.
Evidence against
- The only npm postinstall action prints a message; it does not modify files or launch code.
- The tracing prompt prominently requires a separate explicit yes before installing the harness or enabling capture.
- No self-dependency, obfuscated loader, credential harvesting, or automatic install-time agent-control mutation was found.
Behavioral surface
ChildProcessCryptoEnvironmentVarsFilesystemNetworkShell
HighEntropyStringsUrlStrings
Source & flagged code
7 flaggedpackage.jsonView file
•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
High
Install Time Lifecycle Scripts
Package defines install-time lifecycle scripts.
package.jsonView on unpkg•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
Medium
Ambiguous Install Lifecycle Script
Install-time lifecycle script is not statically allowlisted and needs review.
package.jsonView on unpkgbin.jsView file
15//
L16: // MIN_MAJOR tracks ink's own `engines.node`, and package.json's `engines` field
L17: // must say the same thing; test/cli.test.js asserts they agree.
...
L23: if (!Number.isInteger(major) || major < MIN_MAJOR) {
L24: const shell = process.platform === 'win32'
L25: ? 'irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex'
L26: : 'bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)';
High
Sandbox Evasion Gated Capability
Findings
3 High6 Medium5 Low
HighInstall Time Lifecycle Scriptspackage.json
HighSandbox Evasion Gated Capabilitybin.js
HighShips High Entropy Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumAmbiguous Install Lifecycle Scriptpackage.json
MediumNetwork
MediumEnvironment Vars
MediumShips Build Helperstart.ps1
MediumShips Compressed Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumStructural Risk Force Deep Review
LowScripts Present
LowFilesystem
LowHigh Entropy Strings
LowUrl Strings
LowNested Archive Needs Inspectionvendor/python_dotenv-1.2.1-py3-none-any.whl
Affected versions and remediation
This report applies to evals@2.10.0.
See all versions of evals for other recorded verdicts.
- Review the evidence and your use of evals@2.10.0 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.