evals@2.7.0
Arize AX onboarding — instrument your app with tracing via your coding agent
AI Security Review
scanned 3mo ago · by lpm-firewall-aiThis is a consent-gated, first-party agent-extension setup with a concrete telemetry collection surface, not covert install-time malware. Warn so users receive visibility before enabling it.
Static reason
One or more suspicious static signals were detected.
Trigger
User runs `npx evals`, selects an agent, and separately approves agent tracing.
Impact
Approved prompt, tool-command, and tool-output content may be sent as tracing telemetry.
Mechanism
Explicitly consented agent telemetry-hook installation.
AI rationale
This is a consent-gated, first-party agent-extension setup with a concrete telemetry collection surface, not covert install-time malware. Warn so users receive visibility before enabling it.
Evidence
package.jsoncli.jsonboarding-prompt.mdvendor/coding_harness_tracing-0.1.0-py3-none-any.whlvendor/harness-install.sh~/.arize/onboarding/onboarding-prompt.md~/.arize/onboarding/harness.env~/.arize/harness~/.codex/config.toml~/.codex/arize-env.sh~/.codex/sessions
Network endpoints1
otlp.arize.com:443
Decision evidence
public snapshotAI called this Suspicious at 93.0% confidence as Dangerous Capability with low false-positive risk.
Evidence for warning
- Vendored tracing wheel installs agent settings/hooks and captures session data.
- Agent-tracing prompt explicitly targets every session and tool outputs.
- Harness installation is invoked only through the user-run onboarding flow.
Evidence against
- postinstall only prints an installation message.
- CLI stages a bundled prompt, then launches a user-selected local agent.
- Prompt requires a separate explicit yes before machine-wide tracing.
- No install-time payload execution, secret harvesting, or exfiltration was found.
Behavioral surface
ChildProcessEnvironmentVarsFilesystemShell
HighEntropyStringsUrlStrings
Source & flagged code
7 flaggedpackage.jsonView file
•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
High
Install Time Lifecycle Scripts
Package defines install-time lifecycle scripts.
package.jsonView on unpkg•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
Medium
Ambiguous Install Lifecycle Script
Install-time lifecycle script is not statically allowlisted and needs review.
package.jsonView on unpkgbin.jsView file
15//
L16: // MIN_MAJOR tracks ink's own `engines.node`, and package.json's `engines` field
L17: // must say the same thing; test/cli.test.js asserts they agree.
...
L23: if (!Number.isInteger(major) || major < MIN_MAJOR) {
L24: const shell = process.platform === 'win32'
L25: ? 'irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex'
L26: : 'bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)';
High
Sandbox Evasion Gated Capability
Findings
3 High5 Medium5 Low
HighInstall Time Lifecycle Scriptspackage.json
HighSandbox Evasion Gated Capabilitybin.js
HighShips High Entropy Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumAmbiguous Install Lifecycle Scriptpackage.json
MediumEnvironment Vars
MediumShips Build Helperstart.ps1
MediumShips Compressed Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumStructural Risk Force Deep Review
LowScripts Present
LowFilesystem
LowHigh Entropy Strings
LowUrl Strings
LowNested Archive Needs Inspectionvendor/python_dotenv-1.2.1-py3-none-any.whl
Affected versions and remediation
This report applies to evals@2.7.0.
See all versions of evals for other recorded verdicts.
- Review the evidence and your use of evals@2.7.0 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.