Build a Vendor Security Review and Evidence Agent
Build an agent that organizes vendor security documents, maps evidence to a supplied review checklist, and identifies missing coverage. Produce a reviewer case with follow-up questions and a human-owned decision record.
What you are building
The core problem, expected build, and operating context for this challenge.
A procurement security reviewer assessing a prospective software vendor.
Build an agent that organizes vendor security documents, maps evidence to a supplied review checklist, and identifies missing coverage. Produce a reviewer case with follow-up questions and a human-owned decision record.
Vendor profile and data flows
Security evidence records
Review checklist
Build an inventory of vendor services and subprocessors.
Map evidence to checklist controls and service scope.
Identify missing, expired, or contradictory evidence.
Prepare follow-up questions and a review case for a human decision.
A vendor review workspace with a coverage matrix, source documents, evidence gaps, and reviewer notes.
Show current access-control evidence for Vendor, request current backup evidence, and identify missing StorageCo coverage.
A working local application with the specified review interface.
Source code, dependency lockfile, and setup commands.
A recorded end-to-end demo using the supplied starter data.
Example outputs, action logs, and automated tests.
These fictional records are a small development fixture, not a production dataset. Implement the mock services described in the brief.
```json { "vendor": "Example Vendor", "processors": [ "Vendor", "StorageCo" ], "checklist": [ "access_control", "backup_testing" ], "evidence": [ { "id": "E1", "processor": "Vendor", "control": "access_control", "status": "current" }, { "id": "E2", "processor": "Vendor", "control": "backup_testing", "status": "expired" } ] } ```
Use fictional security records. Do not perform intrusive testing or certify a vendor. Choose an available model that supports your implementation. Record its identifier and version.
Review the complete application, not only the starter-case answers.
The matrix distinguishes current, expired, and missing evidence.
Follow-up questions identify the affected processor and control.
No vendor approval is issued automatically. Add one normal case, one incomplete-input case, and one failure-recovery case.
Participation guide
Use fictional security records. Do not perform intrusive testing or certify a vendor.
How work is evaluated
Review workflow completion, interface usability, evidence quality, and safe failure handling. No agent execution has been verified.
A working local application with the specified review interface. Source code, dependency lockfile, and setup commands. A recorded end-to-end demo using the supplied starter data. Example outputs, action logs, and automated tests.
Challenge datasets
Public datasets, plus the private datasets you have access to.
What you should walk away with
Build an inventory of vendor services and subprocessors.
Map evidence to checklist controls and service scope.
Identify missing, expired, or contradictory evidence.
Prepare follow-up questions and a review case for a human decision.
Reference links and supporting material
Supplied fictional starter records: Vendor profile and data flows, Security evidence records, Review checklist. The overview includes the complete JSON bundle. Implement local mock services and extend the sample coverage.
Challenge setup
Configuration summary for this challenge.
Challenge input
Input details are not configured.
LlamaIndex
Data framework for LLM applications
Evaluated output
No output checks are configured.
- No evaluation checks are configured.
- 1 public reference case
- 1 configured tool
Summary shows the first task, up to three tools, and up to three evidence items. Open the technical recipe to view all configured items.
View technical recipe
Configured tools
- LlamaIndex ยท Optional
Evaluation contract
- No evaluation checks are configured.
Reference cases
- 1 public reference case
Workflow tasks
No workflow tasks are configured.
Recipe freeze marker
No recipe freeze time is recorded.