Code Documentation

Frisbee Interface

class frisbee.Frisbee(project: str = 'elastic_gates', log_level: int = 20, save: bool = False)

Class to interact with the core code.

_config_bootstrap() → None

Handle the basic setup of the tool prior to user control.

Bootstrap will load all the available modules for searching and set them up for use by this main class.

_progressive_save(job) → None

Save output to a dictionary as results stream in.

Depending on the options used, Frisbee can run for quite a long time. Each individual job is written after its completed and includes the findings along with the job details.

_reset() → None

Reset some of the state in the class for multi-searches.

get_results() → List[T]

Return results from the search.

search(jobs: List[Dict[str, str]], executor=None) → None

Perform searches based on job orders.

Module Interface

class frisbee.modules.base.Base(log_level=10)

Base module class to assist in writing new modules.

_extract() → None

Extract email addresses from results.

Text content from all crawled pages are ran through a simple email extractor. Data is cleaned prior to running pattern expressions.

_fetch(urls: List[str]) → None

Perform bulk collection of data and return the content.

Gathering responses is handled by the base class and uses futures to speed up the processing. Response data is saved inside a local variable to be used later in extraction.

_format() → None

Format search queries to perform in bulk.

Build up the URLs to call for the search engine. These will be ran through a bulk processor and returned to a detailer.

_process(responses: List[str]) → None

Process search engine results for detailed analysis.

Search engine result pages (SERPs) come back with each request and will need to be extracted in order to crawl the actual hits.

_request_bulk(urls: List[str]) → List[T]

Batch the requests going out.

search() → None

Execute search function and hand to processor.

set_log_level(level: str) → None

Override the default log level of the class.