Termux ID: Enumeration -->

    Social Items

Home / Enumeration

InSpy is a python based LinkedIn enumeration tool. Inspy has two functionalities: TechSpy and EmpSpy.
  • TechSpy - Crawls LinkedIn job listings for technlogoies used by the provided company. InSpy attempts to identify technologies by matching job descriptions to keywords from a new line delimited file.
  • EmpSpy - Crawls LinkedIn for employees working at the provided company. InSpy searches for employees by title and/or departments from a new line delimited file. InSpy may also create emails for the identified employees if the user specifies an email format.

Installation
Run 
pip install -r requirements.txt
within the cloned InSpy directory.

Help
InSpy - A LinkedIn enumeration tool by Jonathan Broche (@jonathanbroche)

positional arguments:
  company               Company name to use for tasks.

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit

Technology Search:
  --techspy [file]      Crawl LinkedIn job listings for technologies used by
                        the company. Technologies imported from a new line
                        delimited file. [Default: tech-list-small.txt]
  --limit int           Limit the number of job listings to crawl. [Default:
                        50]

Employee Harvesting:
  --empspy [file]       Discover employees by title and/or department. Titles
                        and departments are imported from a new line delimited
                        file. [Default: title-list-small.txt]
  --emailformat string  Create email addresses for discovered employees using
                        a known format. [Accepted Formats: first.last@xyz.com,
                        last.first@xyz.com, first_last@xyz.com, last_first@xyz.com, 
                        firstl@xyz.com, lfirst@xyz.com,
                        flast@xyz.com, lastf@xyz.com, first@xyz.com,
                        last@xyz.com]

Output Options:
  --html file           Print results in HTML file.
  --csv file            Print results in CSV format.
  --json file           Print results in JSON.


InSpy - A Linkedin Enumeration Tool


Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu, and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster, and ReverseDNS.
subbrute was integrated with Sublist3r to increase the possibility of finding more subdomains using bruteforce with an improved wordlist. The credit goes to TheRook who is the author of subbrute.

Installation
git clone https://github.com/aboul3la/Sublist3r.git

Recommended Python Version:
Sublist3r currently supports Python 2 and Python 3.
  • The recommended version for Python 2 is 2.7.x
  • The recommened version for Python 3 is 3.4.x

Dependencies:
Sublist3r depends on the requests, dnspython, and argparse python modules.
These dependencies can be installed using the requirements file:
  • Installation on Windows:
c:\python27\python.exe -m pip install -r requirements.txt
  • Installation on Linux
sudo pip install -r requirements.txt
Alternatively, each module can be installed independently as shown below.

Requests Module (http://docs.python-requests.org/en/latest/)
  • Install for Windows:
c:\python27\python.exe -m pip install requests
  • Install for Ubuntu/Debian:
sudo apt-get install python-requests
  • Install for Centos/Redhat:
sudo yum install python-requests
  • Install using pip on Linux:
sudo pip install requests

dnspython Module (http://www.dnspython.org/)
  • Install for Windows:
c:\python27\python.exe -m pip install dnspython
  • Install for Ubuntu/Debian:
sudo apt-get install python-dnspython
  • Install using pip:
sudo pip install dnspython

argparse Module
  • Install for Ubuntu/Debian:
sudo apt-get install python-argparse
  • Install for Centos/Redhat:
sudo yum install python-argparse
  • Install using pip:
sudo pip install argparse
for coloring in windows install the following libraries
c:\python27\python.exe -m pip install win_unicode_console colorama

Usage
Short Form Long Form Description
-d --domain Domain name to enumerate subdomains of
-b --bruteforce Enable the subbrute bruteforce module
-p --ports Scan the found subdomains against specific tcp ports
-v --verbose Enable the verbose mode and display results in realtime
-t --threads Number of threads to use for subbrute bruteforce
-e --engines Specify a comma-separated list of search engines
-o --output Save the results to text file
-h --help show the help message and exit

Examples
  • To list all the basic options and switches use -h switch:
python sublist3r.py -h
  • To enumerate subdomains of specific domain:
python sublist3r.py -d example.com
  • To enumerate subdomains of specific domain and show only subdomains which have open ports 80 and 443 :
python sublist3r.py -d example.com -p 80,443
  • To enumerate subdomains of specific domain and show the results in realtime:
python sublist3r.py -v -d example.com
  • To enumerate subdomains and enable the bruteforce module:
python sublist3r.py -b -d example.com
  • To enumerate subdomains and use specific engines such Google, Yahoo and Virustotal engines
python sublist3r.py -e google,yahoo,virustotal -d example.com

Using Sublist3r as a module in your python scripts
Example
import sublist3r 
subdomains = sublist3r.main(domain, no_threads, savefile, ports, silent, verbose, enable_bruteforce, engines)
The main function will return a set of unique subdomains found by Sublist3r
Function Usage:
  • domain: The domain you want to enumerate subdomains of.
  • savefile: save the output into text file.
  • ports: specify a comma-sperated list of the tcp ports to scan.
  • silent: set sublist3r to work in silent mode during the execution (helpful when you don't need a lot of noise).
  • verbose: display the found subdomains in real time.
  • enable_bruteforce: enable the bruteforce module.
  • engines: (Optional) to choose specific engines.
Example to enumerate subdomains of Yahoo.com:
import sublist3r 
subdomains = sublist3r.main('yahoo.com', 40, 'yahoo_subdomains.txt', ports= None, silent=False, verbose= False, enable_bruteforce= False, engines=None)

Credits


Sublist3r v1.0 - Fast subdomains enumeration tool for penetration testers


Zeus is an advanced reconnaissance utility designed to make web application reconnaissance simple. Zeus comes complete with a powerful built-in URL parsing engine, multiple search engine compatibility, the ability to extract URLs from both ban and webcache URLs, the ability to run multiple vulnerability assessments on the target, and is able to bypass search engine captchas.

Features
  • A powerful built in URL parsing engine
  • Multiple search engine compatibility (DuckDuckGo, AOL, Bing, and Google default is Google)
  • Ability to extract the URL from Google's ban URL thus bypassing IP blocks
  • Ability to extract from Google's webcache URL
  • Proxy compatibility (http, https, socks4, socks5)
  • Tor proxy compatibility and Tor browser emulation
  • Parse robots.txt/sitemap.xml and save them to a file
  • Multiple vulnerability assessments (XSS, SQLi, clickjacking, port scanning, admin panel finding, whois lookups, and more)
  • Tamper scripts to obfuscate XSS payloads
  • Can run with a custom default user-agent, one of over 4000 random user-agents, or a personal user-agent
  • Automatic issue creation when an unexpected error arises
  • Ability to crawl a webpage and pull all the links
  • Can run a singular dork, multiple dorks in a given file, or a random dork from a list of over 5000 carefully researched dorks
  • Dork blacklisting when no sites are found with the search query, will save the query to a blacklist file
  • Identify WAF/IPS/IDS protection of over 20 different firewalls
  • Header protection enumeration to check what kind of protection is provided via HTTP headers
  • Saving cookies, headers, and other vital information to log files
  • and much more...

Screenshots

Running without a mandatory options, or running the --help flag will output Zeus's help menu:


 A basic dork scan with the -d flag, from the given dork will launch an automated browser and pull the Google page results:


  Calling the -s flag will prompt for you to start the sqlmap API server python sqlmapapi.py -s from sqlmap, it will then connect to the API and perform a sqlmap scan on the found URL's.


You can see more screenshots here

Demo


Requirements
There are some requirements for this to be run successfully.

Basic requirements
  • libxml2-dev, libxslt1-dev, python-dev are required for the installation process
  • Firefox web browser is required as of now, you will need Firefox version <=57 >=51 (between 51 and 57). Full functionality for other browsers will eventually be added.
  • If you want to run sqlmap through the URL's you will need sqlmap somewhere on your system.
  • If you want to run a port scan using nmap on the URL's IP addresses. You will need nmap on your system.
  • Geckodriver is required to run the firefox web browser and will be installed the first time you run. It will be added to your /usr/bin so that it can be run in your ENV PATH.
  • You must be sudo for the first time running this so that you can add the driver to your PATH, you also may need to run as sudo depending on your permissions. NOTE: Depending on permissions you may need to be sudo for any run involving the geckodriver
  • xvfb is required by pyvirtualdisplay, it will be installed if not installed on your first run

Python package requirements
  • selenium-webdriver package is required to automate the web browser and bypass API calls.
  • requests package is required to connect to the URL, and the sqlmap API
  • python-nmap package is required to run nmap on the URL's IP addresses
  • whichcraft package is required to check if nmap and sqlmap are on your system if you want to use them
  • pyvirtualdisplay package is required to hide the browser display while finding the search URL
  • lxml is required to parse XML data for the sitemap and save it as such
  • psutil is required to search for running sqlmap API sessions
  • beautifulsoup is required to pull all the HREF descriptor tags and parse the HTML into an easily workable syntax

Installation
You can download the latest tar.gz, the latest zip, or you can find the current stable release here. Alternatively you can install the latest development version by following the instructions that best match your operating system:
NOTE: (optional but highly advised) add sqlmap and nmap to your environment PATH by moving them to /usr/bin or by adding them to the PATH via terminal

Ubuntu/Debian
sudo apt-get install libxml2-dev libxslt1-dev python-dev &&  git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py

centOS
sudo apt-get install gcc python-devel libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py

Others
sudo apt-get install libxml2-dev libxslt1-dev python-dev && git clone https://github.com/ekultek/zeus-scanner.git && cd zeus-scanner && sudo pip2 install -r requirements.txt && sudo python zeus.py
This will install all the package requirements along with the geckodriver


Zeus-Scanner - Advanced Reconnaissance Utility


kernelpop is a framework for performing automated kernel exploit enumeration on Linux, Mac, and Windows hosts.

Requirements
python3

Currently supported CVE's:
*CVE-2017-1000367
*CVE-2017-1000112
*CVE-2017-7308
*CVE-2017-6074
*CVE-2017-5123
*CVE-2016-5195
*CVE-2016-2384
*CVE-2016-0728
*CVE-2015-7547
*CVE-2015-1328
*CVE-2014-4699
*CVE-2014-4014
*CVE-2014-3153
*CVE-2014-0196
*CVE-2009-1185

Run modes

default mode
The default mode runs with the command python3 kernelpop.py. This processes information about the host kernel and compares it to the known kernel exploits available to the program. It then outputs a list of potentially useful vulnerabilities and attached exploits.


brute-enumeration mode
The brute-enumeration mode performs the same checks, but then goes beyond and checks the computer for exploit prerequisites to see if the operating system is set up in the required vulnerable state for successful exploitation.



input mode
The input mode allows you to perform enumeration with just the output of a uname -a command, which makes it useful as a host-side enumeration tool.



One feature currently only partially implemented is brute-exploit mode. This is set to prepare, compile, and run exploits in order to confirm an exploitable kernel. However, as of now it would only run on the box the program is executed on. Given that this is a fairly large project to bring on to someone else's computer, and makes considerable noise when compiling and checking exploit attempts, it would be poor OPSEC to use in an actual engagement. At some point in the future, I would like to integrate it into my other project pysploit for enumeration and the reckless, noisy, exploity brute-exploit mode.

Exploit sources
https://github.com/SecWiki/linux-kernel-exploits
https://www.exploit-db.com/local/
https://github.com/SecWiki/windows-kernel-exploits


Kernelpop - Kernel Privilege Escalation Enumeration And Exploitation Framework


nullinux is an internal penetration testing tool for Linux that can be used to enumerate OS information, domain information, shares, directories, and users through SMB null sessions. Unlike many of the enumeration tools out there already, nullinux can enumerate multiple targets at once and when finished, creates a users.txt file of all users found on the host(s). This file is formatted for direct implementation and further exploitation.This program assumes Python 2.7, and the Samba package are installed on the machine.

Features

  1. Enumerating Multiple Hosts at Once
     nullinux provides several options for enumerating users and shares from multiple hosts. Although this can be completed with other tools using a bash loop, nullinux simplifies this process.

     Bash Loop: x=0;until [ $x -eq “3” ]; do enum4linux 192.168.1.$x; ((x++)); done
    OR
    python nullinux.py -all 192.168.1.1-5
    python nullinux.py 192.168.1.1,192.168.1.7

  2. Creates user.txt file of all users enumerated
    By evoking the command line argument “-users”, nullinux will attempt to enumerate users through all available options and display the results on screen. nullinux will compile all users collected during enumeration into a single .txt file, free of duplicates. This nullinux_user.txt file can then be used for password spraying or other internal attacks.

    Nullinux can also be set to use non-invasive and lengthy techniques to enumerate users through the "-quick" option. This will perform a quick enumeration of users leaving out brute force options such as known usernames, rid cycling, and enumerating the LSA.

  3. Dynamic Share Enumeration
    The command line argument “-shares” will attempt to enumerate all available shares on the host and automatically attempt to enumerate the directory of each share found. Simplifying what used to be a very manual process. Both “-shares” and “-users” can be implemented together using the “-all” command line argument.

    python nullinux.py -shares 192.168.1.1
    python nullinux.py -all 10.0.0.1

Usage
                 nullinux | v4.0
         SMB Null Session Enumeration Tool

Scanning:
    -shares             Dynamically Enumerate all possible
                        shares. (formally: --enumshares)

    -users              Enumerate users through a variety of
                        techniques. (formally: --enumusers)

    -quick              Quickly enumerate users, leaving out brute
                        force options. (used with: -users, or -all)

    -all                Enumerate both users and shares
                        (formally: --all)

Host:
    -U                  Set username (optional)
    -P                  Set password (optional)

More Options:
    -v                  Verbose Output
    -h                  Help menu

Example Usage:
    python nullinux.py -users -quick DC1.Domain.net
    python nullinux.py -all 192.168.0.0-5
    python nullinux.py -shares -U 'Domain\User' -P 'Password1' 10.0.0.1,10.0.0.5
    python nullinux.py 10.0.0.0/24


nullinux - SMB null Session Identification and Enumeration Tool


BaRMIe is a tool for enumerating and attacking Java RMI (Remote Method Invocation) services.
RMI services often expose dangerous functionality without adequate security controls, however RMI services tend to pass under the radar during security assessments due to the lack of effective testing tools. In 2008 Adam Boulton spoke at AppSec USA (YouTube) and released some RMI attack tools which disappeared soon after, however even with those tools a successful zero-knowledge attack relies on a significant brute force attack (~64-bits/9 quintillion possibilities) being performed over the network.
The goal of BaRMIe is to enable security professionals to identify, attack, and secure insecure RMI services. Using partial RMI interfaces from existing software, BaRMIe can interact directly with those services without first brute forcing 64-bits over the network.

Disclaimer
BaRMIe was written to aid security professionals in identifying insecure RMI services on systems which the user has prior permission to attack. Unauthorised access to computer systems is illegal and BaRMIe must be used in accordance with all relevant laws. Failure to do so could lead to you being prosecuted. The developers of BaRMIe assume no liability and are not responsible for any misuse or damage caused by this program.

Usage
Use of BaRMIe is straightforward. Run BaRMIe with no parameters for usage information.
$ java -jar BaRMIe.jar
  ▄▄▄▄    ▄▄▄       ██▀███   ███▄ ▄███▓ ██▓▓█████
 ▓█████▄ ▒████▄    ▓██ ▒ ██▒▓██▒▀█▀ ██▒▓██▒▓█   ▀
 ▒██▒ ▄██▒██  ▀█▄  ▓██ ░▄█ ▒▓██    ▓██░▒██▒▒███
 ▒██░█▀  ░██▄▄▄▄██ ▒██▀▀█▄  ▒██    ▒██ ░██░▒▓█  ▄
 ░▓█  ▀█▓ ▓█   ▓██▒░██▓ ▒██▒▒██▒   ░██▒░██░░▒████▒
 ░▒▓███▀▒ ▒▒   ▓▒█░░ ▒▓ ░▒▓░░ ▒░   ░  ░░▓  ░░ ▒░ ░
 ▒░▒   ░   ▒   ▒▒ ░  ░▒ ░ ▒░░  ░      ░ ▒ ░ ░ ░  ░
  ░    ░   ░   ▒     ░░   ░ ░      ░    ▒ ░   ░
  ░            ░  ░   ░            ░    ░     ░  ░
       ░                                     v1.0
             Java RMI enumeration tool.
               Written by Nicky Bloor (@NickstaDB)

Warning: BaRMIe was written to aid security professionals in identifying the
         insecure use of RMI services on systems which the user has prior
         permission to attack. BaRMIe must be used in accordance with all
         relevant laws. Failure to do so could lead to your prosecution.
         The developers assume no liability and are not responsible for any
         misuse or damage caused by this program.

Usage:
  BaRMIe -enum [options] [host] [port]
    Enumerate RMI services on the given endpoint(s).
    Note: if -enum is not specified, this is the default mode.
  BaRMIe -attack [options] [host] [port]
    Enumerate and attack the given target(s).
Options:
  --threads  The number of threads to use for enumeration (default 10).
  --timeout  The timeout for blocking socket operations (default 5,000ms).
  --targets  A file containing targets to scan.
             The file should contain a single host or space-separated
             host and port pair per line.
             Alternatively, all nmap output formats are supported, BaRMIe will
             parse nmap output for port 1099, 'rmiregistry', or 'Java RMI'
             services to target.
             Note: [host] [port] not supported when --targets is used.
Reliability:
    A +/- system is used to indicate attack reliability as follows:
      [+  ]: Indicates an application-specific attack
      [-  ]: Indicates a JRE attack
      [ + ]: Attack insecure methods (such as 'writeFile' without auth)
      [ - ]: Attack Java deserialization (i.e. Object parameters)
      [  +]: Does not require non-default dependencies
      [  -]: Non-default dependencies are required
Enumeration mode (-enum) extracts details of objects that are exposed through an RMI registry service and lists any known attacks that affect the endpoint.
Attack mode (-attack) first enumerates the given targets, then provides a menu system for launching known attacks against RMI services.
A single target can be specified on the command line. Alternatively BaRMIe can extract targets from a simple text file or nmap output.

No Vulnerable Targets Identified?
Great! This is your opportunity to help improve BaRMIe! BaRMIe relies on some knowledge of the classes exposed over RMI so contributions will go a long way in improving BaRMIe and the security of RMI services.
If you have access to JAR files or source code for the target application then producing an attack is as simple as compiling code against the relevant JAR files. Retrieve the relevant remote object using the LocateRegistry and Registry classes and call the desired methods. Alternatively look for remote methods that accept arbitrary objects or otherwise non-primitive parameters as these can be used to deliver deserialization payloads. More documentation on attacking RMI and producing attacks for BaRMIe will be made available in the near future.
Alternatively, get in touch, and provide as much detail as possible including BaRMIe -enum output and ideally the relevant JAR files.

Attack Types
BaRMIe is capable of performing three types of attacks against RMI services. A brief description of each follows. Further technical details will be published in the near future at https://nickbloor.co.uk/. In addition to this, I presented the results of my research at 44CON 2017 and the slides can be found here: BaRMIe - Poking Java's Back Door.

1. Attacking Insecure Methods
The first and most straightforward method of attacking insecure RMI services is to simply call insecure remote methods. Often dangerous functionality is exposed over RMI which can be triggered by simply retrieving the remote object reference and calling the dangerous method. The following code is an example of this:
//Get a reference to the remote RMI registry service
Registry reg = LocateRegistry.getRegistry(targetHost, targetPort);

//Get a reference to the target RMI object
Foo bar = (Foo)reg.lookup(objectName);

//Call the remote executeCommand() method
bar.executeCommand(cmd);

2. Deserialization via Object-type Paraeters
Some RMI services do not expose dangerous functionality, or they implement security controls such as authentication and session management. If the RMI service exposes a method that accepts an arbitrary Object as a parameter then the method can be used as an entry point for deserialization attacks. Some examples of such methods can be seen below:
public void setOption(String name, Object value);
public void addAll(List values);

3. Deserialization via Illegal Method Invocation
Due to the use of serialization, and insecure handling of method parameters on the server, it is possible to use any method with non-primitive parameter types as an entry point for deserialization attacks. BaRMIe achieves this by using TCP proxies to modify method parameters at the network level, essentially triggering illegal method invocations. Some examples of vulnerable methods can be seen below:
public void setName(String name);
public Long add(Integer i1, Integer i2);
public void sum(int[] values);
The parameters to each of these methods can be replaced with a deserialization payload as the method invocation passes through a proxy. This attack is possible because Java does not attempt to verify that remote method parameters received over the network are compatible with the actual parameter types before deserializing them.


BaRMIe - Java RMI Enumeration And Attack Tool


Intelligence and Reconnaissance Package/Bundle installer.
IntRec-Pack is a Bash script designed to download, install and deploy several quality OSINT, Recon and Threat Intelligence tools. Due to the fact it manages the installation of the various dependencies related to these programs as well it aims to be a comprehensive assistant in setting up your intelligence gathering environment. Below is an overview of the tools and utilities it will help you set up.

+-----------------------+-------------------------------------------+
| Tool                  | Utility type and feature summary          |
+-----------------------+-------------------------------------------+
|1. QuickScan         | Port Scanner/WHOIS/Domain Resolver        | 
|2. DNSRecon         | Advanced DNS Enumeration & Domain Utility |
|3. Sublist3r           | OSINT Based Subdomain Enumeration         |
|4. TekDefense-Automator| OSINT Based IP, URL and Hash Analyzer     |
|5. TheHarvester        | eMail, vHost, Domain and PII Enumeration  |
|6. IOC-Parser          | Threat Intel, parses IOC data from reports|
|7. PyParser-CVE        | Multi Source Exploit Parser/CVE Lookup    |
|8. Mimir               | HoneyDB CLI/Threat Intelligence Utility   |
|9. Harbinger           | Cymon.io, Virus Total, Threat Feed Parser |
|10.Spiderfoot          | Advanced OSINT/Reconnaissance Framework   |
+-----------------------+-------------------------------------------+
Furthermore I have included functionality within the Bash script that allows the user to easily pull up two web based resources. Namely OSINT-Framework and HoneyDB. The former serves as a curated list of open source intelligence tools, websites and related materials for use as a reference guide. While the latter is an OSINT aggregative threat intelligence pool that collects and organizes data provided by HoneyPy honeypots. My Command Line Interface for which is included in the selection of tools available for download with IntRec-Pack as well.

Usage
Clone the tool from the repo and make it executable like so.
git clone https://github.com/NullArray/IntRec-Pack.git
cd IntRec-Pack
chmod +x intrec.sh
After which it can be started from the command line with sudo ./intrec.sh. Upon doing so you will be presented with a menu the options for which are as follows:
1) Help                  4) Specify Install Location
2) List and Install      5) Online Resources
3) Install All           6) Quit
The help option displays further usage information and general details about the tool. List and Install will list all the tools available for download/installation and lets you select the ones you would like. Upon doing so the tool plus it's dependencies will be installed in the current working directory. Unless the Specify Install Location option has been used to provide a path to a custom location. Install All will download and install all the tools available with this script and Online Resources will open the web applications previously mentioned.

Update
The script has been updated to version 1.0.1.

Changelog
Each installation operation now has its own function in order to make the script modular. This will also allow for the easy addition of operations that would install other/more tools in the future.
Additional checks have been added to the script in order to look for the presence of utilities such as wget, git and pip. This is important because some distros such as Debian and Devuan do not come with some of these utilities installed by default. Should the script find any of these utilities are missing it will attempt to automatically resolve the issue. Making the script effective and compatible with most Debian based distros.
From now on IntRec-Pack will check to see if it has been started with super user privilege. Since there are a lot of sudo commands in the script this will prevent the user from running into trouble halfway through the execution.


IntRec-Pack - Intelligence and Reconnaissance Package/Bundle installer



cve-search is a tool to import CVE (Common Vulnerabilities and Exposures) and CPE (Common Platform Enumeration) into a MongoDB to facilitate search and processing of CVEs.
The main objective of the software is to avoid doing direct and public lookup into the public CVE databases. This is usually faster to do local lookups and limits your sensitive queries via the Internet.
cve-search includes a back-end to store vulnerabilities and related information, an intuitive web interface for search and managing vulnerabilities, a series of tools to query the system and a web API interface.
cve-search is used by many organizations including the public CVE services of CIRCL.

Requirements
The requirements can be installed with pip:
sudo pip3 install -r requirements.txt

Installation of MongoDB
First, you'll need to have a Python 3 installation (3.3 or higher). Then you need to install MongoDB (2.2) from source (this should also work with any standard packages from your favorite distribution). Don't forget to install the headers for development while installing MongoDB. You can go to http://docs.mongodb.org/manual/installation/ for to get the packages for your distribution, or http://www.mongodb.org/downloads for the source code.

Populating the database
For the initial run, you need to populate the CVE database by running:
./sbin/db_mgmt.py -p
./sbin/db_mgmt_cpe_dictionary.py
./sbin/db_updater.py -c
It will fetch all the existing XML files from the Common Vulnerabilities and Exposures database and the Common Platform Enumeration. The initial Common Platform Enumeration (CPE) import might take some time depending of your configuration.
If you want to add the cross-references from NIST, Red Hat and other vendors:
./sbin/db_mgmt_ref.py
A more detailed documentation can be found in the Documentations folder of the project.

Databases and collections
The MongoDB database is called cvedb and there are 11 collections:
  • cves (Common Vulnerabilities and Exposure items) - source NVD NIST
  • cpe (Common Platform Enumeration items) - source NVD NIST
  • cwe (Common Weakness Enumeration items) - source NVD NIST
  • capec (Common Attack Pattern Enumeration and Classification) - source NVD NIST
  • ranking (ranking rules per group) - local cve-search
  • d2sec (Exploitation reference from D2 Elliot Web Exploitation Framework) - source d2sec.com
  • MITRE Reference Key/Maps - source MITRE reference Key/Maps
  • ms - (Microsoft Bulletin (Security Vulnerabilities and Bulletin)) - source Microsoft
  • exploitdb (Offensive Security - Exploit Database) - source offensive security
  • info (metadata of each collection like last-modified) - local cve-search
  • via4 VIA4CVE cross-references.
The Redis database has 3 databases:
  • 10: The cpe (Common Platform Enumeration) cache - source MongoDB cvedb collection cpe
  • 11: The notification database - source cve-search
  • 12: The CVE reference database is a cross-reference database to CVE ids against various vendors ID - source NVD NIST/MITRE
The reference database has 3 additional sources:

Updating the database
An updater script helps to start the db_mgmt_*
./sbin/db_updater.py -v
You can run it in a crontab, logging is done in syslog by default.

Repopulating the database
To easily drop and re-populate all the databases
./sbin/db_updater.py -v -f
This will drop all the existing external sources and reimport everything. This operation can take some time and it's usually only required when new attributes parsing are added in cve-search.

Usage
You can search the database using search.py
./bin/search.py -p cisco:ios:12.4
./bin/search.py -p cisco:ios:12.4 -o json
./bin/search.py -f nagios -n
./bin/search.py -p microsoft:windows_7 -o html
If you want to search all the WebEx vulnerabilities and only printing the official references from the supplier.
./bin/search.py -p webex: -o csv  -v "cisco"
You can also dump the JSON for a specific CVE ID.
./bin/search.py -c CVE-2010-3333
Or you can use the XMPP bot
./bin/search_xmpp.py -j mybot@jabber.org -p strongpassword
Or dump the last 2 CVE entries in RSS or Atom format
./bin/dump_last.py -f atom -l 2
Or you can use the webinterface.
./web/index.py

Usage of the ranking database
There is a ranking database allowing to rank software vulnerabilities based on their common platform enumeration name. The ranking can be done per organization or department within your organization or any meaningful name for you.
As an example, you can add a partial CPE name like "sap:netweaver" which is very critical for your accounting department.
./sbin/db_ranking.py  -c "sap:netweaver" -g "accounting" -r 3
and then you can lookup the ranking (-r option) for a specific CVE-ID:
./bin/search.py -c CVE-2012-4341  -r  -n

Advanced usage
As cve-search is based on a set of tools, it can be used and combined with standard Unix tools. If you ever wonder what are the top vendors using the term "unknown" for their vulnerabilities:
python3 bin/search_fulltext.py -q unknown -f | jq -c '. | .vulnerable_configuration[0]' | cut -f5 -d: | sort  | uniq -c  | sort -nr | head -10

1500 oracle
381 sun
372 hp
232 google
208 ibm
126 mozilla
103 microsoft
100 adobe
 78 apple
 68 linux
You can compare CVSS (Common Vulnerability Scoring System ) values of some products based on their CPE name. Like comparing oracle:java versus sun:jre and using R to make some statistics about their CVSS values:
python3 bin/search.py -p oracle:java -o json  | jq -r '.cvss' | Rscript -e 'summary(as.numeric(read.table(file("stdin"))[,1]))'
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
1.800   5.350   9.300   7.832  10.000  10.000


python3 bin/search.py -p sun:jre -o json  | jq -r '.cvss' | Rscript -e 'summary(as.numeric(read.table(file("stdin"))[,1]))'
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
0.000   5.000   7.500   7.333  10.000  10.000

Fulltext indexing
If you want to index all the CVEs from your current MongoDB collection:
./sbin/db_fulltext.py
and you query the fulltext index (to get a list of matching CVE-ID):
./bin/search_fulltext.py -q NFS -q Linux
or to query the fulltext index and output the JSON object for each CVE-ID:
./bin/search_fulltext.py -q NFS -q Linux -f

Fulltext visualization
The fulltext indexer visualization is using the fulltext indexes to build a list of the most common keywords used in CVE. NLTK is required to generate the keywords with the most common English stopwords and lemmatize the output. NTLK for Python 3 exists but you need to use the alpha version of NLTK.
./bin/search_fulltext.py  -g -s >cve.json


You can see a visualization on the demo site.

Web interface
The web interface is a minimal interface to see the last CVE entries and query a specific CVE. You'll need flask in order to run the website and Flask-PyMongo. To start the web interface:
cd ./web
./index.py
Then you can connect on http://127.0.0.1:5000/ to browser the last CVE.

Web API interface
The web interface includes a minimal JSON API to get CVE by ID, by vendor or product. A public version of the API is also accessible on cve.circl.lu.
List the know vendors in JSON
curl http://127.0.0.1:5000/api/browse/
Dump the product of a specific vendor in JSON
curl  http://127.0.0.1:5000/api/browse/zyxel 
{
  "product": [
    "n300_netusb_nbg-419n",
    "n300_netusb_nbg-419n_firmware",
    "p-660h-61",
    "p-660h-63",
    "p-660h-67",
    "p-660h-d1",
    "p-660h-d3",
    "p-660h-t1",
    "p-660h-t3",
    "p-660hw",
    "p-660hw_d1",
    "p-660hw_d3",
    "p-660hw_t3"
  ],
  "vendor": "zyxel"
}
Find the associated vulnerabilities to a vendor and a product
curl  http://127.0.0.1:5000/api/search/zyxel/p-660hw
[{"cwe": "CWE-352", "references": ["http://www.exploit-db.com/exploits/33518", "http://secunia.com/advisories/58513", "http://packetstormsecurity.com/files/126812/Zyxel-P-660HW-T1-Cross-Site-Request-Forgery.html", "http://osvdb.org/show/osvdb/107449"], "vulnerable_configuration": ["cpe:/h:zyxel:p-660hw:_t1:v3"], "Published": "2014-06-16T14:55:09.713-04:00", "id": "CVE-2014-4162", "Modified": "2014-07-17T01:07:29.683-04:00", "cvss": 6.8, "summary": "Multiple cross-site request forgery (CSRF) vulnerabilities in the Zyxel P-660HW-T1 (v3) wireless router allow remote attackers to hijack the authentication of administrators for requests that change the (1) wifi password or (2) SSID via a request to Forms/WLAN_General_1."}, {"cwe": "CWE-20", "references": ["http://www.kb.cert.org/vuls/id/893726"], "vulnerable_configuration": ["cpe:/h:zyxel:p-660h-63:-", "cpe:/h:zyxel:p-660h-t1:-", "cpe:/h:zyxel:p-660h-d3:-", "cpe:/h:zyxel:p-660h-t3:v2", "cpe:/h:zyxel:p-660h-t1:v2", "cpe:/h:zyxel:p-660h-d1:-", "cpe:/h:zyxel:p-660h-67:-", "cpe:/h:zyxel:p-660h-61:-", "cpe:/h:zyxel:p-660hw_t3:v2", "cpe:/h:zyxel:p-660hw_t3:-", "cpe:/h:zyxel:p-660hw_d3:-", "cpe:/h:zyxel:p-660hw_d1:v2", "cpe:/h:zyxel:p-660hw_d1:-", "cpe:/h:zyxel:p-660hw:_t1:v2", "cpe:/h:zyxel:p-660hw:_t1:-"], "Published": "2014-04-01T23:58:16.967-04:00", "id": "CVE-2013-3588", "Modified": "2014-04-02T11:29:53.243-04:00", "cvss": 7.8, "summary": "The web management interface on Zyxel P660 devices allows remote attackers to cause a denial of service (reboot) via a flood of TCP SYN packets."}, {"cwe": "CWE-79", "references": ["http://osvdb.org/ref/99/rompager407.pdf", "http://osvdb.org/99694", "http://antoniovazquezblanco.github.io/docs/advisories/Advisory_RomPagerXSS.pdf"], "vulnerable_configuration": ["cpe:/h:d-link:dsl-2640r:-", "cpe:/h:d-link:dsl-2641r:-", "cpe:/h:huawei:mt882:-", "cpe:/h:sitecom:wl-174:-", "cpe:/h:tp-link:td-8816:-", "cpe:/a:allegrosoft:rompager:4.07", "cpe:/h:zyxel:p-660hw_d1:-"], "Published": "2014-01-16T14:55:04.607-05:00", "id": "CVE-2013-6786", "Modified": "2014-01-17T11:01:47.353-05:00", "cvss": 4.3, "summary": "Cross-site scripting (XSS) vulnerability in Allegro RomPager before 4.51, as used on the ZyXEL P660HW-D1, Huawei MT882, Sitecom WL-174, TP-LINK TD-8816, and D-Link DSL-2640R and DSL-2641R, when the \"forbidden author header\" protection mechanism is bypassed, allows remote attackers to inject arbitrary web script or HTML by requesting a nonexistent URI in conjunction with a crafted HTTP Referer header that is not properly handled in a 404 page.  NOTE: there is no CVE for a \"URL redirection\" issue that some sources list separately."}, {"cwe": "CWE-79", "references": ["http://xforce.iss.net/xforce/xfdb/41109", "http://www.securityfocus.com/archive/1/archive/1/489009/100/0/threaded", "http://www.gnucitizen.org/projects/router-hacking-challenge/"], "vulnerable_configuration": ["cpe:/h:zyxel:p-660hw_t3:v2", "cpe:/h:zyxel:p-660hw:_t1:v2", "cpe:/h:zyxel:p-660hw_d1:v2", "cpe:/h:zyxel:p-660hw_t3:-", "cpe:/h:zyxel:p-660hw:_t1:-", "cpe:/h:zyxel:p-660hw_d3:-", "cpe:/h:zyxel:p-660hw_d1:-"], "Published": "2008-03-10T13:44:00.000-04:00", "id": "CVE-2008-1257", "Modified": "2012-05-31T00:00:00.000-04:00", "cvss": 4.3, "summary": "Cross-site scripting (XSS) vulnerability in Forms/DiagGeneral_2 on the ZyXEL P-660HW series router allows remote attackers to inject arbitrary web script or HTML via the PingIPAddr parameter."}, {"id": "CVE-2008-1256", "references": ["http://xforce.iss.net/xforce/xfdb/41108", "http://www.securityfocus.com/archive/1/archive/1/489009/100/0/threaded", "http://www.gnucitizen.org/projects/router-hacking-challenge/"], "vulnerable_configuration": ["cpe:/h:zyxel:p-660hw"], "Published": "2008-03-10T13:44:00.000-04:00", "Modified": "2011-03-07T22:06:25.080-05:00", "cvss": 10.0, "summary": "The ZyXEL P-660HW series router has \"admin\" as its default password, which allows remote attackers to gain administrative access."}, {"cwe": "CWE-264", "references": ["http://www.securityfocus.com/archive/1/archive/1/489009/100/0/threaded", "http://www.gnucitizen.org/projects/router-hacking-challenge/", "http://xforce.iss.net/xforce/xfdb/41114"], "vulnerable_configuration": ["cpe:/h:zyxel:p-660hw"], "Published": "2008-03-10T13:44:00.000-04:00", "id": "CVE-2008-1255", "Modified": "2008-09-05T17:37:15.440-04:00", "cvss": 10.0, "summary": "The ZyXEL P-660HW series router maintains authentication state by IP address, which allows remote attackers to bypass authentication by establishing a session from a source IP address of a previously authenticated user."}, {"cwe": "CWE-352", "references": ["http://www.securityfocus.com/archive/1/archive/1/489009/100/0/threaded", "http://www.gnucitizen.org/projects/router-hacking-challenge/", "http://xforce.iss.net/xforce/xfdb/41111"], "vulnerable_configuration": ["cpe:/h:zyxel:p-660hw"], "Published": "2008-03-10T13:44:00.000-04:00", "id": "CVE-2008-1254", "Modified": "2008-09-05T17:37:15.287-04:00", "cvss": 6.8, "summary": "Multiple cross-site request forgery (CSRF) vulnerabilities on the ZyXEL P-660HW series router allow remote attackers to (1) change DNS servers and (2) add keywords to the \"bannedlist\" via unspecified vectors."}]

Software using cve-search

cve-search - A Tool To Perform Local Searches For Known Vulnerabilities

Subscribe to: Posts (Atom)