KB(Knowledge Base) Importer

KB-Importer is a java application which is used to import vulnerabilities in Steady Backend from Project KB.

Project KB

Project KB supports the creation, management and aggregation of a distributed, collaborative knowledge base of vulnerabilities that affect open-source software.

This guide talks about gathering the data in Steady from Project KB.

Setup

  1. Use the following link to setup and configure Project KB - installing-the-kaybee-tool.
  2. Update the variables in kaybeeconf.yaml configuration file. The variables are
    • USER_TOKEN - Token specified in your Steady .env file.
    • BACKEND_URL - URL of the rest-backend service (Mandatory).
    • CIA_URL - URL of the deployed rest-lib-util service (This is used if you are uploading the affected artifacts).
    • KB_IMPORTER_PATH - Jar path of your kb-importer module.

Usage

  1. Run kaybee merge to get the vulnerablity information from Project-KB.
  2. Run kaybee export -t steady which outputs a shell script steady.sh which is is then run to update all vulnerability data in Steady.
  3. To get specific vulnerability information run kaybee export -t steady -f .kaybee/merged/CVE-2019-0191/statement.yaml which outputs a shell script steady.sh which is is then run to update CVE-2019-0191 vulnerability data in Steady.

Technical Information

This information is to know how KB-Importer works in detail. The directory structure and the files are automatically generated by the shell script steady.sh as shown above.

To add a vulnerability manually using KB-Importer the folloing folder structure and command has to be used:

Command Line arguments for KB-Importer:

  • directory (-d) <arg> : A directory containing commits information for a vulnerability. More details regarding the directory structure is shown below.
  • overwrite (-o) : Overwrite the analysis of a vulnerability if it already exists in the backend.
  • upload (-u) : Upload construct changes. Without the -u option, the resulting JSON will be saved to file for you to review (e.g., to check that the list of construct changes is not empty).
  • verbose (-v) : Verbose mode to get more detailed logs.

Other useful/required configurations:

  • vulas.shared.backend.serviceUrl : URL of the deployed rest-backend service (Mandatory).
  • vulas.shared.cia.serviceUrl : URL of the deployed rest-lib-util service (This is used if you are uploading the affected artifacts as shown in the below example).

Directory structure should be as follows:

vulnerability_dir          #vulnerability directory specified with option -d. Note: The directory name need not be the same as the vulnerability id
├── commit1_dir            #commit directory. Note: The directory name need not be the same as the commit id
│   ├── before             #directory containing the files/folders before the vulnerability fix
│   ├── after              #directory containing the files/folders after the vulnerability fix
│   └── metadata.json      #commit metadata file for each commit. More info can be found below
├── commit2_dir            #mutiple commit directories if there are mutiple commits for a vulnerability fix
│   ├── before
│   ├── after
│   └── metadata.json
└── metadata.json          #vulnerability metadata file

Vulnerability metadata.json example:

{ 
  "vulnerability_id": "CVE-2016-1000338", 
  "aliases": ["VULN-1000338","VULN-0002"],
  "affected_artifacts": [
    { 
      "id": "pkg:maven/org.bouncycastle/bcprov-jdk15on@1.47",
      "reason": "Manual review",
      "affected": true
    },
    { 
      "id": "pkg:maven/org.bouncycastle/bcprov-jdk15@1.43",
      "reason": "Manual review",
      "affected": false
    }
  ],
  "notes": [
    {
      "links": [
        "https://www.bouncycastle.org/releasenotes.html",
        "https://ubuntu.com/security/notices/USN-3727-1"
      ],
      "text": "In Bouncy Castle JCE Provider version 1.55 and earlier the DSA does not fully validate ASN.1 encoding of signature on verification. It is possible to inject extra elements in the sequence making up the signature and still have it validate, which in some cases may allow the introduction of 'invisible' data into a signed structure."
    },
    {
      "links": [
        "https://lists.debian.org/debian-lts-announce/2018/07/msg00009.html"
      ],
      "text": "It was discovered that Bouncy Castle incorrectly handled certain crypto algorithms. A remote attacker could possibly use these issues to obtain sensitive information, including private keys."
    }
  ]
}

Information about the fields of vulnerability metadata.json file:

  • vulnerability_id: Vulnerability ID.
  • aliases: Aliases used for the vulnerability id.
  • affected_artifacts: Affected Libraries for the vulnerability.
  • notes: This is an array of links and text. Here, the links array consists of links which provide comprehensive vulnerability information and the text is the vulnerability description. (optional, it must be provided for vulnerabilities not available from the NVD).

Commit metadata.json example file:

{
  "repository" : "https://github.com/bcgit/bc-java",
  "branch" : "DEFAULT_BRANCH",
  "timestamp" : "1476490865",
  "commit_id" : "843c2e60f67d71faf81d236f448ebbe56c62c647"
}

Information about the fields of commit metadata.json file:

  • repository: URL of the VCS repository hosting the library project.
  • branch: Branch name.
  • timestamp: Timestamp when the commit was made.
  • commit_id: VCS commit id.

Example run for kb-importer:

java -Dvulas.shared.backend.header.X-Vulas-Client-Token=<token> -Dvulas.shared.cia.serviceUrl=<cia-url> -Dvulas.shared.backend.serviceUrl=<backend-url> -jar kb-importer-3.1.14-SNAPSHOT-jar-with-dependencies.jar -d ./CVEdata/CVE-2016-1000338 -v -u -o

Limitation

Project KB only supports Java specific vulnerabilities. Python vulnerabilities will be supported once For python, init files are to be checked out is fixed.