Skip to content

Latest commit

 

History

History
612 lines (467 loc) · 20.9 KB

README.en.md

File metadata and controls

612 lines (467 loc) · 20.9 KB

cve-ease project

Project Introduction

cve-ease
is a platform focused on CVE information. It collects various CVE details published by the community and notifies users through multiple channels such as email, Wecom, DingTalk, etc. Users can view detailed CVE information on the cve-ease platform, including vulnerability descriptions, impacted systems, and suggested fixes, and choose the appropriate remediation based on their system environment.

The goal of the cve-ease platform is to help users quickly understand and address vulnerabilities in their systems, improving system security and stability.

cve-ease is an independent innovation project by Tianyi Cloud, which has been open-sourced in the openEuler community. We look forward to welcoming community members to join the project development and collaboratively build a secure, stable, and reliable domestic operating system ecosystem.

Open Source License:

  • This repository strictly follows the Mulan Permissive License, Version 2
  • This repository has been submitted as a high-quality open-source project after thorough review and preparation under the Tianyi Cloud Technology Co Ltd.'s open-source process. All related documentation and materials have been completed and are available.
  • A dedicated person from the company is assigned to maintain this repository, with LTS (Long-Term Support) for continuous updates and development.

Software Architecture

cve-ease is a platform dedicated to CVE information. Its architecture is composed of four main modules: CVE Crawler, CVE Analyzer, CVE Notifier, and CVE Frontend. Below we introduce each module’s function and design.

  • CVE Crawler

This module is responsible for fetching CVE information from various data sources provided by the openEuler community and storing it in relational databases like MySQL. The key information comes from the cve-manager project. Currently, cve-manager supports fetching CVE data from the following sources: NVD, CNNVD, CNVD, RedHat, Ubuntu, Debian, etc. cve-ease uses Python to implement multiple crawler scripts, each corresponding to a specific data source, which can run periodically or manually. The crawler script formats and stores the fetched CVE information for further analysis and processing.

  • CVE Analyzer

This module is responsible for parsing, categorizing, and scoring CVE information. cve-ease uses a Python script to periodically read raw CVE data from the relational database and perform the following operations: parsing basic CVE attributes (e.g., ID, title, description), categorizing CVE impacts (e.g., operating systems, software packages), scoring the severity (e.g., CVSS score), and matching CVE repair suggestions (e.g., patch links). The processed structured CVE information is stored in SQL format in the database for future queries and display.

  • CVE Notifier

This module sends CVE notifications to users via email, WeChat, DingTalk, etc., based on user subscription settings. cve-ease uses a Python script to regularly fetch structured CVE data from MySQL and filter based on user interests (e.g., operating systems, software packages). It then generates suitable notification content for different channels (e.g., text, images) and uses APIs to send the notifications (e.g., SMTP for email, HTTP for WeChat or DingTalk messages). The notifier script logs the send results and updates the subscription status in MySQL.

  • CVE Frontend

This module provides a user-friendly CLI command for users to view, search, and subscribe to CVE information.

The architecture of cve-ease is designed to be efficient, flexible, and scalable, providing users with timely and accurate security vulnerability information.

Development Plan

  1. Repodata support for multi-vendor OSV (Operating System Software Provider)
  2. motd login broadcast feature
  3. DNF plugin extension for repair features
  4. Automatic repair of specific packages
  5. Add awareness ability for specific packages
  6. ...

We greatly welcome your valuable suggestions regarding the cve-ease development direction. If you have any ideas or suggestions, please feel free to share them with us, and we would appreciate it very much~

Installation Guide

Currently, cve-ease is in rapid development, and the supported installation methods include direct installation, container installation, and RPM package installation.

Direct Installation

git clone https://gitee.com/openeuler/cve-ease cve-ease.git
cd cve-ease.git/cve-ease
make install

Container Installation

git clone https://gitee.com/openeuler/cve-ease cve-ease.git
cd cve-ease.git/cve-ease
make run-in-docker

RPM package installation

git clone https://gitee.com/openeuler/cve-ease cve-ease.git
cd cve-ease.git/cve-ease
make gensrpm
cd ..
rpm -ivh *.src.rpm
cd ~/rpmbuild
rpmbuild -ba SPECS/cve-ease.spec
cd RPMS/noarch
rpm -ivh *.rpm

Usage Instructions

Help Information

  • If the cve-ease command is executed without any parameters, it displays help information.
  • There are several subcommands under cve-ease, classified into basic, info, and notifier categories.
  • The help subcommand is used to display help information for different command categories.
# cve-ease

Available commands:

basic commands:
      config                    Print cve-ease config
      daemon                    Run as daemon without interactive
      motd                      Motd info manager
      service                   Service manager

info commands:
      cve                       OpenEuler CVE info
      cvrf                      OpenEuler CVRF info
      db                        Database manager
      help                      List available commands
      logger                    Logger config
      repodata                  Repodata info
      rpm                       Rpm info
      sa                        OpenEuler security notice info

notifier commands:
      dingding                  Notifier of dingding
      feishu                    Notifier of feishu
      mail163                   Notifier of mail163
      mailqq                    Notifier of mailqq
      wecom                     Notifier of wecom

Try "cve-ease --help" for help about global gconfig
Try "cve-ease help" to get all available commands
Try "cve-ease <command> --help" for help about the gconfig of a particular command
Try "cve-ease help <category>" to get commands under a particular category
Available commands are: basic, info, notifier

# cve-ease help info
Available commands:

info commands:
      cve                       OpenEuler CVE info
      cvrf                      OpenEuler CVRF info
      db                        Database manager
      help                      List available commands
      logger                    Logger config
      repodata                  Repodata info
      rpm                       Rpm info
      sa                        OpenEuler security notice info

Try "cve-ease --help" for help about global gconfig
Try "cve-ease help" to get all available commands
Try "cve-ease <command> --help" for help about the gconfig of a particular command
Try "cve-ease help <category>" to get commands under a particular category
Available commands are: basic, info, notifier

Configuration File

The configuration file is located at /etc/cve-ease/cve-ease.cfg

[main]
pid_file_path = /var/log/cve-ease/cve-ease.pid
lock_file_path = /var/log/cve-ease/cve-ease.lock

# log configuration

# debug/ error(default) / warn
log_level = debug
log_file_path = /var/log/cve-ease/cve-ease.log
log_maxbytes = 10240
log_backup_num = 30

# sql configuration
db_type = sqlite
db_file_path = /usr/share/cve-ease/cve-ease.db
db_user =
db_password =
db_host =
db_port =
product = openEuler-20.03-LTS-SP1
expiration_days = 14

# notifier
notifier_record_num = 9

# filter
focus_on = kernel,systemd,openssh,openssl

[wecom]
enabled = 1
# https://developer.work.weixin.qq.com/document/path/91770?version=4.0.19.6020&platform=win
# https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=fe9eae1f-xxxx-4ae3-xxxx-ecf9f77abba6

update_key = 2142ef2a-d99d-417d-8c31-b550b0fcb4e3
status_key = 2142ef2a-d99d-417d-8c31-b550b0fcb4e3


[dingding]
enabled = 1
# just for test
update_key = 81907155a6cc88004e1ed6bcdd86c68d5b21565ed59d549ca031abc93d90d9cb
status_key = 81907155a6cc88004e1ed6bcdd86c68d5b21565ed59d549ca031abc93d90d9cb


[feishu]
enabled = 1
# just for test
update_key = 5575739b-f59d-48db-b737-63672b2c32ab
status_key = 5575739b-f59d-48db-b737-63672b2c32ab


[mail163]
enabled = 0
mail_sender = xxxxxxx@163.com
mail_recver = xxxxxxx@163.com
mail_smtp_token = xxxxxx


[mailqq]
enabled = 0
mail_sender = xxxxxxx@qq.com
mail_recver = xxxxxxx@qq.com
mail_smtp_token = xxxxxxxx

cve-ease Service

The CVE-ease service, consisting of the cve-ease.service and cve-ease.timer files, is triggered periodically by the systemd timer mechanism.

# /usr/lib/systemd/system/cve-ease.timer
#  CTyunOS cve-ease: MulanPSL2
#
#  This file is part of cve-ease.
#

[Unit]
Description=CTyunOS cve-ease Project
Documentation=https://gitee.com/openeuler/cve-ease

[Timer]
OnBootSec=1m
OnUnitActiveSec=10m
RandomizedDelaySec=10

[Install]
WantedBy=timers.target
# systemctl enable --now cve-ease.timer
Created symlink /etc/systemd/system/timers.target.wants/cve-ease.timer → /usr/lib/systemd/system/cve-ease.timer.
# systemctl status cve-ease.timer
● cve-ease.timer - CTyunOS cve-ease Project
   Loaded: loaded (/usr/lib/systemd/system/cve-ease.timer; enabled; vendor preset: disabled)
   Active: active (waiting) since Sat 2023-03-18 17:55:53 CST; 5s ago
  Trigger: Sat 2023-03-18 18:05:55 CST; 9min left
     Docs: https://gitee.com/openeuler/cve-ease

Mar 18 17:55:53 56d941221b41 systemd[1]: Started CTyunOS cve-ease Project.
# systemctl status cve-ease.service
● cve-ease.service - CTyunOS cve-ease project
   Loaded: loaded (/usr/lib/systemd/system/cve-ease.service; disabled; vendor preset: disabled)
   Active: inactive (dead) since Sat 2023-03-18 17:55:56 CST; 5s ago
     Docs: https://gitee.com/openeuler/cve-ease
  Process: 196 ExecStart=/usr/bin/cve-ease daemon (code=exited, status=0/SUCCESS)
 Main PID: 196 (code=exited, status=0/SUCCESS)

Mar 18 17:55:53 56d941221b41 systemd[1]: Starting CTyunOS cve-ease project...
Mar 18 17:55:56 56d941221b41 systemd[1]: cve-ease.service: Succeeded.
Mar 18 17:55:56 56d941221b41 systemd[1]: Started CTyunOS cve-ease project.

Basic Commands

config-related Subcommands

Usage: cve-ease config <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help     show this help message and exit
  -r, --rawdata  print raw config file content
cve-ease config     # Display configuration file path and valid configurations
cve-ease config -r  # Display configuration file path and raw data

Daemon Service

  • The daemon command is the systemd service entry and is generally not executed directly.
  • This service is periodically triggered by the corresponding cve-ease systemd timer service.
# /usr/lib/systemd/system/cve-ease.service
#  CTyunOS cve-ease: MulanPSL2
#
#  This file is part of cve-ease.
#

[Unit]
Description=CTyunOS cve-ease project
Documentation=https://gitee.com/openeuler/cve-ease

[Service]
Type=oneshot
ExecStart=/usr/bin/cve-ease daemon

[Install]
WantedBy=multi-user.target

motd Update Notification-related Subcommands

  • TODO (To be implemented)

Service-related Subcommands

Control commands related to the cve-ease service.

Usage: cve-ease service <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help     show this help message and exit
  -k, --kill     kill cve-ease service
  -r, --restart  restart cve-ease service
  -s, --status   get cve-ease service status
  -v, --verbose  show verbose output
cve-ease service -k   # Pause cve-ease service
cve-ease service -r   # Restart cve-ease service
cve-ease service -s   # Check cve-ease service status

Info Command Categories

cve Subcommand

Crawl CVE announcements from the openEuler community at: openEuler Official CVE Announcements

Usage: cve-ease cve <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help       show this help message and exit
  -r, --rawdata    get cve cache and print raw data without write db
  -m, --makecache  get cve cache
  -l, --list       list all cve info
  -t, --total      get cve info statistics
  -v, --verbose    show verbose output
cve-ease cve -m # Crawl CVE information and store it in the database
cve-ease cve -l # Fetch and format CVE information from the database
cve-ease cve -t # Fetch and display CVE statistics from the database
cve-ease cve -r # Crawl CVE information and display raw data (without storing it in the database)

sa Subcommand

Crawl security announcements from the openEuler community at: openEuler Official SA Announcements

Usage: cve-ease sa <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help       show this help message and exit
  -r, --rawdata    get sa cache and print raw data without write db
  -m, --makecache  get sa cache
  -l, --list       list all sa info
  -t, --total      get sa info statistics
  -v, --verbose    show verbose output
cve-ease sa -m # Crawl SA information and store it in the database
cve-ease sa -l # Fetch and format SA information from the database
cve-ease sa -t # Fetch and display SA statistics from the database
cve-ease sa -r # Crawl SA information and display raw data (without storing it in the database)

cvrf Subcommand

Security announcement related commands

cve-ease cvrf -m # Crawl CVRF information and store it in the database
cve-ease cvrf -l # Fetch and format CVRF information from the database
cve-ease cvrf -t # Fetch and display CVRF statistics from the database

rpm Subcommand

Usage: cve-ease rpm <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help     show this help message and exit
  -l, --list     list all rpm info
  -v, --verbose  show verbose output
cve-ease rpm -l # Use the rpm interface to list information about RPM packages currently installed in the system

repodata Subcommand

Usage: cve-ease repodata <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help            show this help message and exit
  -m, --makecache       cache repodata to database
  -p PRODUCT, --product=PRODUCT
                        specific product (work with --check)
  --osv=OSV             specific osv rpm release
  -t, --total           get total rpm statistics
  -l, --list            list all rpm
  -c, --check           check repo cve
  -v, --verbose         show verbose output
cve-ease repodata -p ctyunos2 -m  # Select ctyunos2 as the OSV version, cache its source data, and write it to the database
cve-ease repodata --osv ctyunos2 -p openEuler-22.03-LTS -c # Compare ctyunos2 sources with openEuler sources
cve-ease repodata -l # List package information contained in the database
cve-ease repodata -t # Retrieve statistics on source packages from the database

logger Subcommand

Usage: cve-ease logger <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help     show this help message and exit
  -l, --list     list all logger info
  -t, --total    get logger statistics
  -v, --verbose  show verbose output

db Subcommand

Usage: cve-ease db <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help     show this help message and exit
  -p, --purge    purge db and recreate it (Danger Operation)
  -s, --stats    get database statistics
  -v, --verbose  show verbose output

Notifier Commands for Message Notifications

WeCom Group Robot

Usage: cve-ease wecom <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help            show this help message and exit
  -t, --test            run test
  -v, --verbose         show verbose output
  -c CONTENT, --content=CONTENT
                        show verbose output
cve-ease wecom -t     # Send a test message to the WeCom group
cve-ease wecom -t -c 'helloworld'   # Send a custom test message to the WeCom group

DingTalk Group Robot

Usage: cve-ease dingding <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help            show this help message and exit
  -t, --test            run test
  -v, --verbose         show verbose output
  -c CONTENT, --content=CONTENT
                        show verbose output
cve-ease dingding -t     # Send a test message to the DingTalk group
cve-ease dingding -t -c 'helloworld'   # Send a custom test message to the DingTalk group

Feishu Group Robot

Usage: cve-ease feishu <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help            show this help message and exit
  -t, --test            run test
  -v, --verbose         show verbose output
  -c CONTENT, --content=CONTENT
                        show verbose output
cve-ease feishu -t     # Send a test message to the Feishu group
cve-ease feishu -t -c 'helloworld'   # Send a custom test message to the Feishu group

163 Mailbox

Usage: cve-ease mail163 <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help            show this help message and exit
  -t, --test            run test
  -v, --verbose         show verbose output
  -c CONTENT, --content=CONTENT
                        show verbose output
cve-ease mail163 -t     # Send a test message to a 163 mailbox
cve-ease mail163 -t -c 'helloworld'   # Send a custom test message to a 163 mailbox

QQ Mailbox

Usage: cve-ease mailqq <options>
(Specify the --help global option for a list of other help options)

Options:
  -h, --help            show this help message and exit
  -t, --test            run test
  -v, --verbose         show verbose output
  -c CONTENT, --content=CONTENT
                        show verbose output
cve-ease mailqq -t     # Send a test message to a QQ mailbox
cve-ease mailqq -t -c 'helloworld'   # Send a custom test message to a QQ mailbox

How to Contribute

  1. Fork this repository.
  2. During the current rapid iteration phase, only the master branch is used. Make changes directly on the master branch and push them.
  3. Create a pull request (PR), clearly describing the specific functionality and purpose of the PR, along with relevant test cases.
  4. Notify the repository maintainer to review the PR.

Core Developers and Contact Information

Community Group

Wechat group