Updating Vulnerability Data

The process of importing vulnerability data from Project KB to the backend of Eclipse Steady is realized using two tools in combination:

  • kaybee (from project KB)
  • kb-importer (part of Eclipse Steady).

This document describes the overall process but we refer to the documentation of project KB for further details on kaybee.

Process overview

kaybee is used to download vulnerability data from project KB (and possibly from other user-defined sources) and to produce a script via its export command (the script is called steady.sh in the following).

By executing the steady.sh script obtained via the kaybee export command, the vulnerability data are imported to the target backend system (specified a the top of the script -- the user is supposed to edit it before running it).

Because kb-importer is invoked from the steady.sh script, in practice you should never have the need to invoke it directly; from the user perspective, the process effectively boils down to:

  1. obtaining vulnerability data from project KB using kaybee
  2. running the steady.sh script generated by kaybee to send such data to the Eclipse Steady backend (the script, under the hood, uses kb-importer).

Setup

  1. Use the following link to setup and configure project KB - installing-the-kaybee-tool.
  2. Update the variables in kaybeeconf.yaml configuration file. The variables are
    • USER_TOKEN - Token specified in your Steady .env file.
    • BACKEND_URL - URL of the rest-backend service (Mandatory).
    • CIA_URL - URL of the deployed rest-lib-util service (This is used if you are uploading the affected artifacts).
    • KB_IMPORTER_PATH - Jar path of your kb-importer module.

Usage

  1. Run kaybee merge to get the vulnerablity information from project KB.
  2. Run kaybee export -t steady to produce the shell script steady.sh which is is then run to update all vulnerability data in Steady.
  3. To get specific vulnerability information run kaybee export -t steady -f .kaybee/merged/CVE-2019-0191/statement.yaml which outputs a shell script steady.sh which is is then run to update CVE-2019-0191 vulnerability data in Steady.

KB-importer: Technical Documentation

This information is to know how KB-Importer works in detail.

In order for the tool to work, it assumes a particular directory structure and content to be available, as documented below.

NOTE: the structure and the files are automatically generated by the shell script obtained by running the kaybee export command as shown above, none of the following is supposed to be created manually!

To add a vulnerability manually using KB-Importer the following folder structure and command has to be used:

Command Line arguments for KB-Importer:

  • directory (-d) <arg> : A directory containing commits information for a vulnerability. More details regarding the directory structure is shown below.
  • overwrite (-o) : Overwrite the analysis of a vulnerability if it already exists in the backend.
  • upload (-u) : Upload construct changes. Without the -u option, the resulting JSON will be saved to file for you to review (e.g., to check that the list of construct changes is not empty).
  • verbose (-v) : Verbose mode to get more detailed logs.

Other useful/required configurations:

  • vulas.shared.backend.serviceUrl : URL of the deployed rest-backend service (Mandatory).
  • vulas.shared.cia.serviceUrl : URL of the deployed rest-lib-util service (This is used if you are uploading the affected artifacts as shown in the below example).

Directory structure should be as follows: 

vulnerability_dir          #vulnerability directory specified with option -d. Note: The directory name need not be the same as the vulnerability id
├── commit1_dir            #commit directory. Note: The directory name need not be the same as the commit id
│   ├── before             #directory containing the files/folders before the vulnerability fix
│   ├── after              #directory containing the files/folders after the vulnerability fix
│   └── metadata.json      #commit metadata file for each commit. More info can be found below
├── commit2_dir            #mutiple commit directories if there are mutiple commits for a vulnerability fix
│   ├── before
│   ├── after
│   └── metadata.json
└── metadata.json          #vulnerability metadata file

Vulnerability metadata.json example: 

{ 
  "vulnerability_id": "CVE-2016-1000338", 
  "aliases": ["VULN-1000338","VULN-0002"],
  "affected_artifacts": [
    { 
      "id": "pkg:maven/org.bouncycastle/bcprov-jdk15on@1.47",
      "reason": "Manual review",
      "affected": true
    },
    { 
      "id": "pkg:maven/org.bouncycastle/bcprov-jdk15@1.43",
      "reason": "Manual review",
      "affected": false
    }
  ],
  "notes": [
    {
      "links": [
        "https://www.bouncycastle.org/releasenotes.html",
        "https://ubuntu.com/security/notices/USN-3727-1"
      ],
      "text": "In Bouncy Castle JCE Provider version 1.55 and earlier the DSA does not fully validate ASN.1 encoding of signature on verification. It is possible to inject extra elements in the sequence making up the signature and still have it validate, which in some cases may allow the introduction of 'invisible' data into a signed structure."
    },
    {
      "links": [
        "https://lists.debian.org/debian-lts-announce/2018/07/msg00009.html"
      ],
      "text": "It was discovered that Bouncy Castle incorrectly handled certain crypto algorithms. A remote attacker could possibly use these issues to obtain sensitive information, including private keys."
    }
  ]
}

Information about the fields of vulnerability metadata.json file:

  • vulnerability_id: Vulnerability ID.
  • aliases: Aliases used for the vulnerability id.
  • affected_artifacts: Affected Libraries for the vulnerability.
  • notes: This is an array of links and text. Here, the links array consists of links which provide comprehensive vulnerability information and the text is the vulnerability description. (optional, it must be provided for vulnerabilities not available from the NVD).

Commit metadata.json example file: 

{
  "repository" : "https://github.com/bcgit/bc-java",
  "branch" : "DEFAULT_BRANCH",
  "timestamp" : "1476490865",
  "commit_id" : "843c2e60f67d71faf81d236f448ebbe56c62c647"
}

Information about the fields of commit metadata.json file:

  • repository: URL of the VCS repository hosting the library project.
  • branch: Branch name.
  • timestamp: Timestamp when the commit was made.
  • commit_id: VCS commit id.

Example run for kb-importer: 

java -Dvulas.shared.backend.header.X-Vulas-Client-Token=<token> -Dvulas.shared.cia.serviceUrl=<cia-url> -Dvulas.shared.backend.serviceUrl=<backend-url> -jar kb-importer-3.1.14-SNAPSHOT-jar-with-dependencies.jar -d ./CVEdata/CVE-2016-1000338 -v -u -o

Limitation

Project KB only supports Java specific vulnerabilities. Python vulnerabilities will be supported once For python, init files are to be checked out is fixed.