evals@2.5.0
Arize AX onboarding — instrument your app with tracing via your coding agent
AI Security Review
scanned 3mo ago · by lpm-firewall-aiNo malicious install-time behavior is present, but the package ships and guides installation of a machine-wide agent telemetry harness. Its consent gate makes this a warned extension-lifecycle risk rather than a blockable malicious chain.
Static reason
One or more suspicious static signals were detected.
Trigger
User runs the CLI, selects an installed coding agent, then approves the tracing setup.
Impact
Accepted capture categories can include prompts, tool commands, paths, URLs, and tool output from future agent sessions.
Mechanism
Seeded agent prompt installs a machine-wide tracing harness and agent settings hook.
AI rationale
No malicious install-time behavior is present, but the package ships and guides installation of a machine-wide agent telemetry harness. Its consent gate makes this a warned extension-lifecycle risk rather than a blockable malicious chain.
Evidence
package.jsoncli.jsonboarding-prompt.mdvendor/coding_harness_tracing-0.1.0-py3-none-any.whl~/.arize/onboarding.env~/.arize/harness~/.codex/config.toml~/.codex/arize-env.sh
Network endpoints2
raw.githubusercontent.com/Arize-ai/coding-harness-tracing/main/install.shotlp.arize.com:443
Decision evidence
public snapshotAI called this Suspicious at 93.0% confidence as Dangerous Capability with low false-positive risk.
Evidence for warning
- User-launched CLI seeds an AI agent with an onboarding prompt.
- Prompt offers machine-wide agent tracing that captures prompts, tool commands, file paths, URLs, and tool output.
- After explicit approval, the prompt directs installation of a harness that writes under ~/.arize and agent settings.
- Bundled harness wheel contains Codex tracing code that reads session rollouts and sends OTLP traces to otlp.arize.com:443.
Evidence against
- postinstall only prints an installation message.
- CLI requires an interactive terminal and a user menu selection before launching an agent.
- Prompt requires a separate explicit yes before machine-wide tracing and exposes capture categories.
Behavioral surface
ChildProcessEnvironmentVarsFilesystemShell
UrlStrings
Source & flagged code
7 flaggedpackage.jsonView file
•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
High
Install Time Lifecycle Scripts
Package defines install-time lifecycle scripts.
package.jsonView on unpkg•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
Medium
Ambiguous Install Lifecycle Script
Install-time lifecycle script is not statically allowlisted and needs review.
package.jsonView on unpkgstart.ps1View file
•path = start.ps1
kind = build_helper
sizeBytes = 7010
magicHex = [redacted]
Medium
Findings
2 High6 Medium4 Low
HighInstall Time Lifecycle Scriptspackage.json
HighShips High Entropy Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumAmbiguous Install Lifecycle Scriptpackage.json
MediumEnvironment Vars
MediumShips Build Helperstart.ps1
MediumShips Compressed Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumStructural Risk Force Deep Review
MediumAi Review Evidencecli.js
LowScripts Present
LowFilesystem
LowUrl Strings
LowNested Archive Needs Inspectionvendor/python_dotenv-1.2.1-py3-none-any.whl
Affected versions and remediation
This report applies to evals@2.5.0.
See all versions of evals for other recorded verdicts.
- Review the evidence and your use of evals@2.5.0 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.