@vercel/devlow-bench@16.4.4
Benchmarking tool for the developer workflow
AI Security Review
scanned 2d ago · by lpm-firewall-aiThe package only loads reporting interfaces when the user selects an upload option, but those interfaces include the local username and Git metadata in the uploaded benchmark data. Since the CLI describes uploading results without identifying those extra fields, classify this as suspicious telemetry rather than malicious collection.
Static reason
No blocking static signals were detected.
Trigger
A user runs the CLI with --datadog or --snowflake and benchmark measurements are reported.
Impact
Uploaded metrics can reveal the local account name and repository commit or branch to the selected reporting destination.
Mechanism
The reporting interfaces attach the local username and Git metadata to benchmark data, then send it to Datadog or a configured Snowflake batch URI.
AI rationale
The package only loads reporting interfaces when the user selects an upload option, but those interfaces include the local username and Git metadata in the uploaded benchmark data. Since the CLI describes uploading results without identifying those extra fields, classify this as suspicious telemetry rather than malicious collection.
Evidence
dist/cli.jsdist/interfaces/constants.jsdist/interfaces/datadog.jsdist/interfaces/snowflake.js
Network endpoints2
Datadog API (client default endpoint; DATADOG_HOST may override)SNOWFLAKE_BATCH_URI or the --snowflake batchUri argument
Decision evidence
public snapshotAI called this Suspicious at 86.0% confidence as Unknown with medium false-positive risk.
Evidence for warning
- The CLI describes Datadog and Snowflake uploads as explicit options, and only loads those interfaces when the corresponding option is set.
- The upload interfaces attach the local username and Git commit and branch metadata to benchmark data.
- Snowflake sends benchmark records by POST to the configured batch URI.
Evidence against
- The package has no install lifecycle scripts; its manifest points to a CLI and library entry point.
- The Datadog and Snowflake interfaces are dynamically imported only when their respective CLI options are enabled.
- The upload destinations are the Datadog client’s endpoint or a user-configured Snowflake URI, rather than a hard-coded unrelated collector.
Behavioral surface
ChildProcessCryptoDynamicRequireEnvironmentVarsFilesystemNetworkShell
HighEntropyStringsUrlStrings
Source & flagged code
6 flaggeddist/describe.jsView file
15try {
L16: iface = await import(`./interfaces/${ifaceName}.js`);
L17: }
Medium
Dynamic Require
Package source references dynamic require/import behavior.
dist/describe.jsView on unpkg · L15dist/cli.jsView file
29ts to Datadog (requires DATADOG_API_KEY).')
.option('--snowflake [batchUri]', 'Upload the results to Snowflake (requires SNOWFLAKE_TOPIC_NAME and SNOWFLAKE_SCHEMA_ID).')
.action(runRun);
program
.command('compare')
.description('Compare two snapshot CSVs side-by-side, with p50/p90/p99 plus a Mann–Whitney U p-value per metric.')
.argument('<baseline>', 'Baseline snapshot CSV path.')
.argument('<current>', 'Current snapshot CSV path.')
Medium
Ai Review Evidence
The CLI describes Datadog and Snowflake uploads as explicit options, and only loads those interfaces when the corresponding option is set.
dist/cli.jsView on unpkg · L29126mport('./interfaces/json.js')).default(opts.json, { n }),
opts.datadog &&
(await import('./interfaces/datadog.js')).default(typeof opts.datadog === 'string' ? { host: opts.datadog } : undefined),
opts.snowflake &&
(await import('./interfaces/snowflake.js')).default(typeof opts.snowflake === 'string'
? { gatewayUri: opts.snowflake }
: undefined),
opts.console !== false &&
(await import('./interfaces/consol
Findings
8 Medium4 Low
MediumDynamic Requiredist/describe.js
MediumNetwork
MediumEnvironment Vars
MediumAi Review Evidencedist/cli.js
MediumAi Review Evidencedist/cli.js
MediumAi Review Evidencedist/interfaces/constants.js
MediumAi Review Evidencedist/interfaces/datadog.js
MediumAi Review Evidencedist/interfaces/snowflake.js
LowScripts Present
LowFilesystem
LowHigh Entropy Strings
LowUrl Strings
Affected versions and remediation
This report applies to @vercel/devlow-bench@16.4.4.
See all versions of @vercel/devlow-bench for other recorded verdicts.
- Review the evidence and your use of @vercel/devlow-bench@16.4.4 before allowing it. Restrict the permissions described in this report.
- Choose an independently verified alternative or release. This report does not establish that other versions are safe.
Evidence last updated: .
Think this verdict is wrong? Dispute it on GitHub or read how reports are produced.