registry / oh-my-knowledge / 0.48.0

oh-my-knowledge@0.48.0

Evaluation framework for LLM knowledge inputs — prompts, RAG corpora, skills, agent workflows. Fix the model, vary the artifact. Built-in statistical rigor: bootstrap CI, Krippendorff α, length-debias, saturation curves.

Static Scan Results

scanned 17d ago · by rust-scanner

Static analysis completed at 93.0% confidence. No malicious behavior was detected; 11 low-signal pattern(s) were surfaced and cleared.

Static reason

No blocking static signals were detected.; previous stored version diff introduced dangerous source

Decision evidence

public snapshot

Behavioral surface

Source

ChildProcessCryptoDynamicRequireEnvironmentVarsFilesystemNetworkShell

Supply chain

HighEntropyStringsTelemetryUrlStrings

ManifestNo manifest risk signals triggered.

scanned 226 file(s), 3.16 MB of source, external domains: 127.0.0.1, api.anthropic.com, api.openai.com, github.com, registry.npmjs.org, wiki.example.com, www.w3.org

Source & flagged code

2 flagged

dist/grading/assertions.jsView file

Lines 355-395javascript

355 process.stderr.write(`[omk] semantic_similarity judge returned non-JSON: ${result.output.slice(0, 100)}\n`);

356 }

357 }

358 catch (parseErr) {

359 process.stderr.write(`[omk] semantic_similarity judge parse error: ${getErrorMessage(parseErr)}\n`);

360 }

361 }

362 }

363 else if (assertion.type === 'faithfulness' ||

364 assertion.type === 'answer_relevancy' ||

365 assertion.type === 'context_recall') {

366 const ragResult = await runRagJudge(assertion, output, sample, executor, judgeModel);

367 asyncCostUSD += ragResult.costUSD;

368 if (ragResult.costReportedByExecutor === false)

369 anyCostUnreported = true;

370 passed = ragResult.passed;

371 message = ragResult.message;

372 }

373 else if (assertion.type === 'custom') {

374 try {

375 const fnPath = resolve(samplesDir, assertion.fn);

376 const mod = await import(fnPath);

377 const fn = mod.default || mod.check;

Medium

Dynamic Require

Package source references dynamic require/import behavior.

dist/grading/assertions.jsView on unpkg · L375

378 if (!fn)

379 throw new Error('custom assertion module must export default or check');

380 const result = await Promise.race([

381 fn(output, { sample, assertion }),

382 new Promise((_, reject) => setTimeout(() => reject(new Error(`custom assertion timed out (${CUSTOM_ASSERTION_TIMEOUT_MS / 1000}s)`)), CUSTOM_ASSERTION_TIMEOUT_MS)),

383 ]);

384 passed = Boolean(result.pass);

385 message = result.message || '';

386 }

387 catch (err) {

388 passed = false;

389 message = `custom assertion error: ${getErrorMessage(err)}`;

390 }

391 }

392 details.push({

393 type: assertion.type,

394 value: assertion.reference || assertion.fn || '',

395 weight,

dist/renderer/observation-inbox-renderer.jsView file

•matchType = previous_version_dangerous_delta matchedPackage = oh-my-knowledge@0.46.0 matchedIdentity = npm:b2gtbXkta25vd2xlZGdl:0.46.0 similarity = 0.942 summary = stored previous version shares package body but lacks this dangerous source file

High

Previous Version Dangerous Delta

This package version adds a dangerous source file absent from the previous stored version; route for source-aware review.

dist/renderer/observation-inbox-renderer.jsView on unpkg

Source & flagged code

2 flagged · loading source

dist/grading/assertions.jsView file

Static Scan Results

Decision evidence

Source & flagged code

Source & flagged code

Findings

Source & flagged code