evals@2.11.0
Arize AX onboarding — instrument your app with tracing via your coding agent
AI Security Review
scanned 27d ago · by lpm-firewall-aiThe postinstall hook is informational, but launching a selected coding agent triggers automatic global installation of Arize skills with prompts suppressed. This is a first-party agent extension setup, so the appropriate verdict is a warning rather than a publish block.
Static reason
One or more suspicious static signals were detected.
Trigger
The user selects an agent to launch from the interactive CLI.
Impact
Arize skills are added to the selected agent's global skill configuration; the automatic setup warrants a warning under the agent integration policy.
Mechanism
The launcher runs npx skills add for the selected agent with global installation and prompt-suppression flags.
AI rationale
The postinstall hook is informational, but launching a selected coding agent triggers automatic global installation of Arize skills with prompts suppressed. This is a first-party agent extension setup, so the appropriate verdict is a warning rather than a publish block.
Evidence
cli.js
Decision evidence
public snapshotAI called this Suspicious at 87.0% confidence as Dangerous Capability with low false-positive risk.
Evidence for warning
- package.json defines a postinstall hook that only prints a message.
- After the user selects an agent to launch, cli.js runs npx skills add with --global and --yes to install Arize skills for that agent.
- bin.js checks the Node version and dynamically imports cli.js; it does not perform agent configuration during package installation.
Evidence against
- The Arize skills install is part of the interactive launch flow, not the npm postinstall hook.
- The inspected source shows no credential forwarding or unrelated network recipient.
Behavioral surface
ChildProcessCryptoEnvironmentVarsFilesystemNetworkShell
HighEntropyStringsUrlStrings
Source & flagged code
7 flaggedpackage.jsonView file
•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
High
Install Time Lifecycle Scripts
Package defines install-time lifecycle scripts.
package.jsonView on unpkg•scripts.postinstall = node -e "console.log('Arize evals installed — run: npx evals')"
Medium
Ambiguous Install Lifecycle Script
Install-time lifecycle script is not statically allowlisted and needs review.
package.jsonView on unpkgbin.jsView file
15//
L16: // MIN_MAJOR tracks ink's own `engines.node`, and package.json's `engines` field
L17: // must say the same thing; test/cli.test.js asserts they agree.
...
L23: if (!Number.isInteger(major) || major < MIN_MAJOR) {
L24: const shell = process.platform === 'win32'
L25: ? 'irm https://cdn.jsdelivr.net/npm/evals/start.ps1 | iex'
L26: : 'bash <(curl -fsSL https://cdn.jsdelivr.net/npm/evals/start.sh)';
High
Sandbox Evasion Gated Capability
Findings
3 High6 Medium5 Low
HighInstall Time Lifecycle Scriptspackage.json
HighSandbox Evasion Gated Capabilitybin.js
HighShips High Entropy Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumAmbiguous Install Lifecycle Scriptpackage.json
MediumNetwork
MediumEnvironment Vars
MediumShips Build Helperstart.ps1
MediumShips Compressed Blobvendor/python_dotenv-1.2.1-py3-none-any.whl
MediumStructural Risk Force Deep Review
LowScripts Present
LowFilesystem
LowHigh Entropy Strings
LowUrl Strings
LowNested Archive Needs Inspectionvendor/python_dotenv-1.2.1-py3-none-any.whl
Affected versions and remediation
This report applies to evals@2.11.0.
See all versions of evals for other recorded verdicts.
- Review the evidence and your use of evals@2.11.0 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.