evals@2.9.0
Arize AX onboarding — instrument your app with tracing via your coding agent
AI Security Review
scanned 2mo ago · by lpm-firewall-aiThe package has a consent-gated, user-invoked global agent-extension and tracing capability. It is not malicious because the install hook is inert and the source requires explicit interaction and approval before broader agent changes.
Static reason
One or more suspicious static signals were detected.
Trigger
The user runs npx evals, selects an installed coding agent, and later approves agent tracing.
Impact
A selected agent gains Arize skills; approved tracing can alter its settings and collect the categories the user accepts.
Mechanism
Global agent skill installation and consent-gated tracing setup.
AI rationale
The package has a consent-gated, user-invoked global agent-extension and tracing capability. It is not malicious because the install hook is inert and the source requires explicit interaction and approval before broader agent changes.
Evidence
package.jsoncli.jsonboarding-prompt.mdvendor/harness-install.sh~/.arize/onboarding~/.arize/harness<agent>'s settings file
Network endpoints3
raw.githubusercontent.comgithub.comcdn.jsdelivr.net
Decision evidence
public snapshotAI called this Suspicious at 91.0% confidence as Dangerous Capability with low false-positive risk.
Evidence for warning
- After an agent is selected, the CLI installs all Arize skills globally without a second scope prompt.
- The bundled prompt offers machine-wide agent tracing that changes the agent settings file and can capture prompts and tool output.
- The prompt can fetch and run a remote harness installer after explicit user approval.
Evidence against
- The postinstall hook only prints a message.
- Global skill installation runs only after the user starts the CLI and selects an agent.
- The prompt requires a separate explicit yes before agent-wide tracing and says not to bypass permission checks.
- No credential theft, covert exfiltration, destructive action, or install-time agent mutation was found.
Behavioral surface
ChildProcessEnvironmentVarsFilesystemShell
HighEntropyStringsUrlStrings
Source & flagged code
7 flaggedpackage.jsonView file
•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
High
Install Time Lifecycle Scripts
Package defines install-time lifecycle scripts.
package.jsonView on unpkg•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
Medium
Ambiguous Install Lifecycle Script
Install-time lifecycle script is not statically allowlisted and needs review.
package.jsonView on unpkgbin.jsView file
15//
L16: // MIN_MAJOR tracks ink's own `engines.node`, and package.json's `engines` field
L17: // must say the same thing; test/cli.test.js asserts they agree.
...
L23: if (!Number.isInteger(major) || major < MIN_MAJOR) {
L24: const shell = process.platform === 'win32'
L25: ? 'irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex'
L26: : 'bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)';
High
Sandbox Evasion Gated Capability
Findings
3 High5 Medium5 Low
HighInstall Time Lifecycle Scriptspackage.json
HighSandbox Evasion Gated Capabilitybin.js
HighShips High Entropy Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumAmbiguous Install Lifecycle Scriptpackage.json
MediumEnvironment Vars
MediumShips Build Helperstart.ps1
MediumShips Compressed Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumStructural Risk Force Deep Review
LowScripts Present
LowFilesystem
LowHigh Entropy Strings
LowUrl Strings
LowNested Archive Needs Inspectionvendor/python_dotenv-1.2.1-py3-none-any.whl
Affected versions and remediation
This report applies to evals@2.9.0.
See all versions of evals for other recorded verdicts.
- Review the evidence and your use of evals@2.9.0 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.