├── .codecov.yml ├── NEWS.md ├── tests ├── test-all.R └── testthat │ └── test-packettotal.R ├── .travis.yml ├── .gitignore ├── R ├── aaa.R ├── usage.R ├── random.R ├── packettotal-package.R ├── search.R ├── info.R ├── detail.R ├── api-key.R ├── download.R ├── deep-search.R └── graph.R ├── .Rbuildignore ├── NAMESPACE ├── packettotal.Rproj ├── man ├── pt_usage.Rd ├── pt_random.Rd ├── pt_search.Rd ├── packettotal_api_key.Rd ├── pt_info.Rd ├── pt_detail.Rd ├── packettotal.Rd ├── pt_deep_search.Rd ├── pt_download.Rd └── pt_similar.Rd ├── DESCRIPTION ├── CONDUCT.md ├── README.Rmd └── README.md /.codecov.yml: -------------------------------------------------------------------------------- 1 | comment: false 2 | -------------------------------------------------------------------------------- /NEWS.md: -------------------------------------------------------------------------------- 1 | 0.1.0 2 | * Initial release 3 | -------------------------------------------------------------------------------- /tests/test-all.R: -------------------------------------------------------------------------------- 1 | library(testthat) 2 | test_check("packettotal") 3 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: R 2 | sudo: false 3 | cache: packages 4 | 5 | after_success: 6 | - Rscript -e 'covr::codecov()' 7 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .Rproj.user 3 | .Rhistory 4 | .RData 5 | .Rproj 6 | README_cache 7 | src/*.o 8 | src/*.so 9 | src/*.dll 10 | -------------------------------------------------------------------------------- /R/aaa.R: -------------------------------------------------------------------------------- 1 | httr::user_agent( 2 | sprintf( 3 | "packettotal package v%s: (<%s>)", 4 | utils::packageVersion("packettotal"), 5 | utils::packageDescription("packettotal")$URL 6 | ) 7 | ) -> .PACKETTOTAL_UA 8 | -------------------------------------------------------------------------------- /.Rbuildignore: -------------------------------------------------------------------------------- 1 | ^.vscode$ 2 | ^.*\.Rproj$ 3 | ^\.Rproj\.user$ 4 | ^\.travis\.yml$ 5 | ^CONDUCT\.md$ 6 | ^README\.*Rmd$ 7 | ^README\.*html$ 8 | ^NOTES\.*Rmd$ 9 | ^NOTES\.*html$ 10 | ^\.codecov\.yml$ 11 | ^README_files$ 12 | ^doc$ 13 | ^docs$ 14 | ^tmp$ 15 | ^notes$ 16 | ^\.gitlab-ci\.yml$ 17 | -------------------------------------------------------------------------------- /tests/testthat/test-packettotal.R: -------------------------------------------------------------------------------- 1 | context("minimal package functionality") 2 | 3 | skip_on_cran() 4 | skip_on_travis() 5 | 6 | expect_true("user" %in% names(pt_usage())) 7 | 8 | x <- pt_info("d210f4dbea97949f694e849507951881") 9 | expect_true("md5" %in% names(x$pcap_metadata)) 10 | 11 | -------------------------------------------------------------------------------- /NAMESPACE: -------------------------------------------------------------------------------- 1 | # Generated by roxygen2: do not edit by hand 2 | 3 | export(packettotal_api_key) 4 | export(pt_deep_search) 5 | export(pt_detail) 6 | export(pt_download) 7 | export(pt_get_search_results) 8 | export(pt_info) 9 | export(pt_random) 10 | export(pt_search) 11 | export(pt_similar) 12 | export(pt_usage) 13 | import(httr) 14 | importFrom(jsonlite,fromJSON) 15 | -------------------------------------------------------------------------------- /packettotal.Rproj: -------------------------------------------------------------------------------- 1 | Version: 1.0 2 | 3 | RestoreWorkspace: Default 4 | SaveWorkspace: Default 5 | AlwaysSaveHistory: Default 6 | 7 | EnableCodeIndexing: Yes 8 | UseSpacesForTab: Yes 9 | NumSpacesForTab: 2 10 | Encoding: UTF-8 11 | 12 | RnwWeave: Sweave 13 | LaTeX: pdfLaTeX 14 | 15 | StripTrailingWhitespace: Yes 16 | 17 | BuildType: Package 18 | PackageUseDevtools: Yes 19 | PackageInstallArgs: --no-multiarch --with-keep.source 20 | PackageBuildArgs: --resave-data 21 | PackageRoxygenize: rd,collate,namespace 22 | -------------------------------------------------------------------------------- /man/pt_usage.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/usage.R 3 | \name{pt_usage} 4 | \alias{pt_usage} 5 | \title{Retrive usage and subscription plan information.} 6 | \usage{ 7 | pt_usage(api_key = packettotal_api_key()) 8 | } 9 | \arguments{ 10 | \item{api_key}{your \code{\link[=packettotal_api_key]{packettotal_api_key()}}.} 11 | } 12 | \description{ 13 | Handy helper to determine how many requests you have remaining. 14 | } 15 | \examples{ 16 | str(try(pt_usage(), silent=TRUE), 2) 17 | } 18 | -------------------------------------------------------------------------------- /man/pt_random.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/random.R 3 | \name{pt_random} 4 | \alias{pt_random} 5 | \title{Get high-level information about a random PCAP file.} 6 | \usage{ 7 | pt_random(api_key = packettotal_api_key()) 8 | } 9 | \arguments{ 10 | \item{api_key}{your \code{\link[=packettotal_api_key]{packettotal_api_key()}}.} 11 | } 12 | \description{ 13 | Randomly selected PCAPs come from a set of pre-selected, interesting PCAP files. 14 | } 15 | \examples{ 16 | str(try(pt_random(), silent=TRUE), 1) 17 | } 18 | \references{ 19 | \url{https://packettotal.com/api-docs/#/pcaps/get_pcaps} 20 | } 21 | -------------------------------------------------------------------------------- /R/usage.R: -------------------------------------------------------------------------------- 1 | #' Retrive usage and subscription plan information. 2 | #' 3 | #' Handy helper to determine how many requests you have remaining. 4 | #' 5 | #' @param api_key your [packettotal_api_key()]. 6 | #' @export 7 | #' @examples 8 | #' str(try(pt_usage(), silent=TRUE), 2) 9 | pt_usage <- function(api_key = packettotal_api_key()) { 10 | 11 | httr::GET( 12 | url = "https://api.packettotal.com/v1/usage", 13 | httr::add_headers( 14 | `x-api-key` = api_key 15 | ), 16 | .PACKETTOTAL_UA 17 | ) -> res 18 | 19 | httr::stop_for_status(res) 20 | 21 | out <- httr::content(res, as = "text", encoding = "UTF-8") 22 | 23 | out <- jsonlite::fromJSON(out) 24 | 25 | out 26 | 27 | } -------------------------------------------------------------------------------- /man/pt_search.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/search.R 3 | \name{pt_search} 4 | \alias{pt_search} 5 | \title{Search with term or with a valid Lucene query.} 6 | \usage{ 7 | pt_search(query, api_key = packettotal_api_key()) 8 | } 9 | \arguments{ 10 | \item{query}{search term (e.g. an IP address, domain, or file hash) or valid Lucene query} 11 | 12 | \item{api_key}{your \code{\link[=packettotal_api_key]{packettotal_api_key()}}.} 13 | } 14 | \description{ 15 | Receive a set of matches for given query. 16 | } 17 | \examples{ 18 | str(try(pt_search("evil.com"), silent=TRUE), 1) 19 | } 20 | \references{ 21 | 7 | #' @export 8 | #' @examples 9 | #' str(try(pt_random(), silent=TRUE), 1) 10 | pt_random <- function(api_key = packettotal_api_key()) { 11 | 12 | httr::GET( 13 | url = "https://api.packettotal.com/v1/pcaps", 14 | httr::add_headers( 15 | `x-api-key` = api_key 16 | ), 17 | .PACKETTOTAL_UA 18 | ) -> res 19 | 20 | httr::stop_for_status(res) 21 | 22 | out <- httr::content(res, as = "text", encoding = "UTF-8") 23 | 24 | out <- jsonlite::fromJSON(out) 25 | 26 | out 27 | 28 | } -------------------------------------------------------------------------------- /man/packettotal_api_key.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/api-key.R 3 | \name{packettotal_api_key} 4 | \alias{packettotal_api_key} 5 | \title{Get or set PACKETTOTAL_API_KEY value} 6 | \usage{ 7 | packettotal_api_key(force = FALSE) 8 | } 9 | \arguments{ 10 | \item{force}{Force setting a new PacketTotal key for the current environment?} 11 | } 12 | \value{ 13 | atomic character vector containing the PacketTotal api key 14 | } 15 | \description{ 16 | The API wrapper functions in this package all rely on a PacketTotal API 17 | key residing in the environment variable \code{PACKETTOTAL_API_KEY}. 18 | The easiest way to accomplish this is to set it 19 | in the \code{.Renviron} file in your home directory. 20 | } 21 | \references{ 22 | \url{https://packettotal.com/api-docs/} 23 | } 24 | -------------------------------------------------------------------------------- /R/packettotal-package.R: -------------------------------------------------------------------------------- 1 | #' Lookup and Analyze Packet Capture ('PCAP') Files 2 | #' 3 | #' 'PacketTotal' () is an engine for analyzing, 4 | #' categorizing, and sharing packet capture ('PCAP') files. The tool was built 5 | #' with the information security community in mind and has applications in malware 6 | #' analysis and network forensics. Methods are provided to query search for and 7 | #' analyze packet capture files. 8 | #' 9 | #' - URL: 10 | #' - BugReports: 11 | #' 12 | #' @md 13 | #' @name packettotal 14 | #' @docType package 15 | #' @author Bob Rudis (bob@@rud.is) 16 | #' @references - 17 | #' - 18 | #' @keywords internal 19 | #' @import httr 20 | #' @importFrom jsonlite fromJSON 21 | NULL 22 | -------------------------------------------------------------------------------- /man/pt_info.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/info.R 3 | \name{pt_info} 4 | \alias{pt_info} 5 | \title{Get high-level information about a specific PCAP file.} 6 | \usage{ 7 | pt_info(pcap_id, api_key = packettotal_api_key()) 8 | } 9 | \arguments{ 10 | \item{pcap_id}{An md5 hash corresponding to the PCAP file submission on PacketTotal.com. 11 | This hash can be derived by hashing the PCAP file in question.} 12 | 13 | \item{api_key}{your \code{\link[=packettotal_api_key]{packettotal_api_key()}}.} 14 | } 15 | \description{ 16 | Results will contain high-level information, such as what logs were extracted, the date it was analyzed, and additional references. 17 | } 18 | \examples{ 19 | str(try(pt_info("d210f4dbea97949f694e849507951881"), silent=TRUE), 2) 20 | } 21 | \references{ 22 | \url{https://packettotal.com/api-docs/#/pcaps/get_pcaps} 23 | } 24 | -------------------------------------------------------------------------------- /R/search.R: -------------------------------------------------------------------------------- 1 | #' Search with term or with a valid Lucene query. 2 | #' 3 | #' Receive a set of matches for given query. 4 | #' 5 | #' @param query search term (e.g. an IP address, domain, or file hash) or valid Lucene query 6 | #' @param api_key your [packettotal_api_key()]. 7 | #' @export 8 | #' @references res 23 | 24 | httr::stop_for_status(res) 25 | 26 | out <- httr::content(res, as = "text", encoding = "UTF-8") 27 | 28 | out <- jsonlite::fromJSON(out) 29 | 30 | out 31 | 32 | } 33 | 34 | -------------------------------------------------------------------------------- /man/pt_detail.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/detail.R 3 | \name{pt_detail} 4 | \alias{pt_detail} 5 | \title{Get a detailed report of PCAP traffic, carved files, signatures, and top-talkers.} 6 | \usage{ 7 | pt_detail(pcap_id, api_key = packettotal_api_key()) 8 | } 9 | \arguments{ 10 | \item{pcap_id}{An md5 hash corresponding to the PCAP file submission on PacketTotal.com. 11 | This hash can be derived by hashing the PCAP file in question.} 12 | 13 | \item{api_key}{your \code{\link[=packettotal_api_key]{packettotal_api_key()}}.} 14 | } 15 | \description{ 16 | Analysis results contain high-level protocol statistics, signatures, and intelligence that PacketTotal discovered during analysis and enrichment. 17 | } 18 | \examples{ 19 | str(try(pt_detail("d210f4dbea97949f694e849507951881"), silent=TRUE), 2) 20 | } 21 | \references{ 22 | \url{https://packettotal.com/api-docs/#/pcaps/get_pcaps} 23 | } 24 | -------------------------------------------------------------------------------- /man/packettotal.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/packettotal-package.R 3 | \docType{package} 4 | \name{packettotal} 5 | \alias{packettotal} 6 | \alias{packettotal-package} 7 | \title{Lookup and Analyze Packet Capture ('PCAP') Files} 8 | \description{ 9 | 'PacketTotal' (\url{https://packettotal.com/}) is an engine for analyzing, 10 | categorizing, and sharing packet capture ('PCAP') files. The tool was built 11 | with the information security community in mind and has applications in malware 12 | analysis and network forensics. Methods are provided to query search for and 13 | analyze packet capture files. 14 | } 15 | \details{ 16 | \itemize{ 17 | \item URL: \url{https://gitlab.com/hrbrmstr/packettotal} 18 | \item BugReports: \url{https://gitlab.com/hrbrmstr/packettotal/issues} 19 | } 20 | } 21 | \references{ 22 | \itemize{ 23 | \item \url{https://packettotal.com/} 24 | \item \url{https://packettotal.com/api-docs/#/} 25 | } 26 | } 27 | \author{ 28 | Bob Rudis (bob@rud.is) 29 | } 30 | \keyword{internal} 31 | -------------------------------------------------------------------------------- /DESCRIPTION: -------------------------------------------------------------------------------- 1 | Package: packettotal 2 | Type: Package 3 | Title: Lookup and Analyze Packet Capture ('PCAP') Files 4 | Version: 0.1.0 5 | Date: 2019-03-16 6 | Authors@R: c( 7 | person("Bob", "Rudis", email = "bob@rud.is", role = c("aut", "cre"), 8 | comment = c(ORCID = "0000-0001-5670-2640")) 9 | ) 10 | Maintainer: Bob Rudis 11 | Description: 'PacketTotal' () is an engine for analyzing, 12 | categorizing, and sharing packet capture ('PCAP') files. The tool was built 13 | with the information security community in mind and has applications in malware 14 | analysis and network forensics. Methods are provided to query search for and 15 | analyze packet capture files. 16 | URL: https://gitlab.com/hrbrmstr/packettotal 17 | BugReports: https://gitlab.com/hrbrmstr/packettotal/issues 18 | Encoding: UTF-8 19 | License: AGPL 20 | Suggests: 21 | testthat, 22 | covr 23 | Depends: 24 | R (>= 3.2.0) 25 | Imports: 26 | httr, 27 | jsonlite 28 | Roxygen: list(markdown = TRUE) 29 | RoxygenNote: 6.1.1 30 | -------------------------------------------------------------------------------- /R/info.R: -------------------------------------------------------------------------------- 1 | #' Get high-level information about a specific PCAP file. 2 | #' 3 | #' Results will contain high-level information, such as what logs were extracted, the date it was analyzed, and additional references. 4 | #' 5 | #' @param pcap_id An md5 hash corresponding to the PCAP file submission on PacketTotal.com. 6 | #' This hash can be derived by hashing the PCAP file in question. 7 | #' @param api_key your [packettotal_api_key()]. 8 | #' @references 9 | #' @export 10 | #' @examples 11 | #' str(try(pt_info("d210f4dbea97949f694e849507951881"), silent=TRUE), 2) 12 | pt_info <- function(pcap_id, api_key = packettotal_api_key()) { 13 | 14 | httr::GET( 15 | url = sprintf("https://api.packettotal.com/v1/pcaps/%s", pcap_id), 16 | httr::add_headers( 17 | `x-api-key` = api_key 18 | ), 19 | .PACKETTOTAL_UA 20 | ) -> res 21 | 22 | httr::stop_for_status(res) 23 | 24 | out <- httr::content(res, as = "text", encoding = "UTF-8") 25 | 26 | out <- jsonlite::fromJSON(out) 27 | 28 | out 29 | 30 | } -------------------------------------------------------------------------------- /man/pt_deep_search.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/deep-search.R 3 | \name{pt_deep_search} 4 | \alias{pt_deep_search} 5 | \alias{pt_get_search_results} 6 | \title{Create a new deep search task. Search for a term or with a Lucene query.} 7 | \usage{ 8 | pt_deep_search(query, api_key = packettotal_api_key()) 9 | 10 | pt_get_search_results(search_result, api_key = packettotal_api_key()) 11 | } 12 | \arguments{ 13 | \item{query}{search term (e.g. an IP address, domain, or file hash) or valid Lucene query} 14 | 15 | \item{api_key}{your \code{\link[=packettotal_api_key]{packettotal_api_key()}}.} 16 | 17 | \item{search_result}{output from \code{\link[=pt_deep_search]{pt_deep_search()}} or a plain search results id} 18 | } 19 | \description{ 20 | Unlike the more lighweight \code{\link[=pt_search]{pt_search()}} results from this endpoint 21 | will be available at the returned URL. 22 | } 23 | \examples{ 24 | str(try(pt_deep_search("botnet OR malware"), silent=TRUE), 1) 25 | } 26 | \references{ 27 | 9 | #' @export 10 | #' @examples 11 | #' str(try(pt_detail("d210f4dbea97949f694e849507951881"), silent=TRUE), 2) 12 | pt_detail <- function(pcap_id, api_key = packettotal_api_key()) { 13 | 14 | httr::GET( 15 | url = sprintf("https://api.packettotal.com/v1/pcaps/%s/analysis", pcap_id), 16 | httr::add_headers( 17 | `x-api-key` = api_key 18 | ), 19 | .PACKETTOTAL_UA 20 | ) -> res 21 | 22 | httr::stop_for_status(res) 23 | 24 | out <- httr::content(res, as = "text", encoding = "UTF-8") 25 | 26 | out <- jsonlite::fromJSON(out) 27 | 28 | out 29 | 30 | } -------------------------------------------------------------------------------- /R/api-key.R: -------------------------------------------------------------------------------- 1 | #' Get or set PACKETTOTAL_API_KEY value 2 | #' 3 | #' The API wrapper functions in this package all rely on a PacketTotal API 4 | #' key residing in the environment variable `PACKETTOTAL_API_KEY`. 5 | #' The easiest way to accomplish this is to set it 6 | #' in the `.Renviron` file in your home directory. 7 | #' 8 | #' @md 9 | #' @param force Force setting a new PacketTotal key for the current environment? 10 | #' @return atomic character vector containing the PacketTotal api key 11 | #' @references 12 | #' @export 13 | packettotal_api_key <- function(force = FALSE) { 14 | 15 | env <- Sys.getenv('PACKETTOTAL_API_KEY') 16 | if (!identical(env, "") && !force) return(env) 17 | 18 | if (!interactive()) { 19 | stop("Please set env var PACKETTOTAL_API_KEY to your PacketTotal key", 20 | call. = FALSE) 21 | } 22 | 23 | message("Couldn't find env var PACKETTOTAL_API_KEY See ?packettotal_api_key for more details.") 24 | message("Please enter your API key:") 25 | pat <- readline(": ") 26 | 27 | if (identical(pat, "")) { 28 | stop("PacketTotal key entry failed", call. = FALSE) 29 | } 30 | 31 | message("Updating PACKETTOTAL_API_KEY env var") 32 | Sys.setenv(PACKETTOTAL_API_KEY = pat) 33 | 34 | pat 35 | 36 | } 37 | -------------------------------------------------------------------------------- /CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Code of Conduct 2 | 3 | As contributors and maintainers of this project, we pledge to respect all people who 4 | contribute through reporting issues, posting feature requests, updating documentation, 5 | submitting pull requests or patches, and other activities. 6 | 7 | We are committed to making participation in this project a harassment-free experience for 8 | everyone, regardless of level of experience, gender, gender identity and expression, 9 | sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion. 10 | 11 | Examples of unacceptable behavior by participants include the use of sexual language or 12 | imagery, derogatory comments or personal attacks, trolling, public or private harassment, 13 | insults, or other unprofessional conduct. 14 | 15 | Project maintainers have the right and responsibility to remove, edit, or reject comments, 16 | commits, code, wiki edits, issues, and other contributions that are not aligned to this 17 | Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed 18 | from the project team. 19 | 20 | Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by 21 | opening an issue or contacting one or more of the project maintainers. 22 | 23 | This Code of Conduct is adapted from the Contributor Covenant 24 | (http:contributor-covenant.org), version 1.0.0, available at 25 | http://contributor-covenant.org/version/1/0/0/ 26 | -------------------------------------------------------------------------------- /man/pt_download.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/download.R 3 | \name{pt_download} 4 | \alias{pt_download} 5 | \title{Download a PCAP analysis archive. The result is a zip archive containing the PCAP itself, CSVs representing various analysis results, and all carved files.'} 6 | \usage{ 7 | pt_download(pcap_id, dl_dir = getwd(), archive_name = NULL, 8 | api_key = packettotal_api_key()) 9 | } 10 | \arguments{ 11 | \item{pcap_id}{An md5 hash corresponding to the PCAP file submission on PacketTotal.com. 12 | This hash can be derived by hashing the PCAP file in question.} 13 | 14 | \item{dl_dir}{directory where to store the download} 15 | 16 | \item{archive_name}{name of the ZIP file. If left \code{NULL} then a ZIP file 17 | will be created with the name \code{YYYY-mm-dd-pcap_id.zip}.} 18 | 19 | \item{api_key}{your \code{\link[=packettotal_api_key]{packettotal_api_key()}}.} 20 | } 21 | \value{ 22 | if successful and the analysis package is ready then the full path 23 | to the ZIP file is returned (invisibly). If the analysis package 24 | is not ready the return value is "\code{_PROCESSING_}". 25 | } 26 | \description{ 27 | Download a PCAP analysis archive. The result is a zip archive containing the PCAP itself, CSVs representing various analysis results, and all carved files.' 28 | } 29 | \examples{ 30 | str(try(pt_download("536cf06ca83704844d789f56caf22ee6"), silent=TRUE), 2) 31 | } 32 | \references{ 33 | \url{https://packettotal.com/api-docs/#/pcaps/get_pcaps__pcap_id__download} 34 | } 35 | -------------------------------------------------------------------------------- /man/pt_similar.Rd: -------------------------------------------------------------------------------- 1 | % Generated by roxygen2: do not edit by hand 2 | % Please edit documentation in R/graph.R 3 | \name{pt_similar} 4 | \alias{pt_similar} 5 | \title{Get a similarity graph relative to the current PCAP file.} 6 | \usage{ 7 | pt_similar(pcap_id, weighting_mode = c("behavior", "content"), 8 | intensity = c("minimal", "low", "medium", "high"), 9 | prioritize_uncommon_fields = FALSE, api_key = packettotal_api_key()) 10 | } 11 | \arguments{ 12 | \item{pcap_id}{An md5 hash corresponding to the PCAP file submission on PacketTotal.com. 13 | This hash can be derived by hashing the PCAP file in question.} 14 | 15 | \item{weighting_mode}{One of "\code{behavior}" (default) or "\code{content}". Weight search results either based on their similarity to the behaviors exhibited or contents contained within the current PCAP file.} 16 | 17 | \item{intensity}{One of "\code{minimal}" (default), "\code{low}", "\code{medium}", or "\code{high}". The scope of the search, basically translates to the maximum number of aggregations to exhaust. Using a high level intensity, may result in occassional timeouts.} 18 | 19 | \item{prioritize_uncommon_fields}{By default, the most common values are used to seed the initial similarity search. Enabling this parameter, seeds the initial search with the least common values instead.} 20 | 21 | \item{api_key}{your \code{\link[=packettotal_api_key]{packettotal_api_key()}}.} 22 | } 23 | \description{ 24 | Results contain PCAPs that exhibit similar behaviors or contain similar content. Results are organized with the most similar PCAPs on top, and the terms that were found shared within both. 25 | } 26 | \examples{ 27 | str(try(pt_similar("536cf06ca83704844d789f56caf22ee6"), silent=TRUE), 3) 28 | } 29 | \references{ 30 | \url{https://packettotal.com/api-docs/#/pcaps/get_pcaps__pcap_id__similar} 31 | } 32 | -------------------------------------------------------------------------------- /R/download.R: -------------------------------------------------------------------------------- 1 | #' Download a PCAP analysis archive. The result is a zip archive containing the PCAP itself, CSVs representing various analysis results, and all carved files.' 2 | #' 3 | #' @param pcap_id An md5 hash corresponding to the PCAP file submission on PacketTotal.com. 4 | #' This hash can be derived by hashing the PCAP file in question. 5 | #' @param dl_dir directory where to store the download 6 | #' @param archive_name name of the ZIP file. If left `NULL` then a ZIP file 7 | #' will be created with the name `YYYY-mm-dd-pcap_id.zip`. 8 | #' @param api_key your [packettotal_api_key()]. 9 | #' @return if successful and the analysis package is ready then the full path 10 | #' to the ZIP file is returned (invisibly). If the analysis package 11 | #' is not ready the return value is "`_PROCESSING_`". 12 | #' @references 13 | #' @export 14 | #' @examples 15 | #' str(try(pt_download("536cf06ca83704844d789f56caf22ee6"), silent=TRUE), 2) 16 | pt_download <- function(pcap_id, dl_dir = getwd(), archive_name = NULL, 17 | api_key = packettotal_api_key()) { 18 | 19 | dl_dir <- path.expand(dl_dir) 20 | stopifnot(dir.exists(dl_dir)) 21 | 22 | httr::GET( 23 | url = sprintf("https://api.packettotal.com/v1/pcaps/%s/download", pcap_id), 24 | httr::add_headers( 25 | `x-api-key` = api_key 26 | ), 27 | .PACKETTOTAL_UA 28 | ) -> res 29 | 30 | httr::stop_for_status(res) 31 | 32 | status_code <- httr::status_code(res) 33 | 34 | if (status_code == "200") { 35 | out <- httr::content(res, as = "raw", encoding = "UTF-8") 36 | if (is.null(archive_name)) { 37 | loc <- file.path(dl_dir, sprintf("%s-%s.zip", as.character(Sys.Date()), pcap_id)) 38 | } else { 39 | loc <- file.path(dl_dir, archive_name) 40 | } 41 | writeBin( 42 | object = out, 43 | con = loc, 44 | useBytes = TRUE 45 | ) 46 | message("Download is at ", loc) 47 | return(invisible(loc)) 48 | } else { 49 | message( 50 | "PCAP exists but the analysis package is not ready. ", 51 | "Try calling the function again in a few minutes." 52 | ) 53 | } 54 | 55 | } -------------------------------------------------------------------------------- /R/deep-search.R: -------------------------------------------------------------------------------- 1 | #' Create a new deep search task. Search for a term or with a Lucene query. 2 | #' 3 | #' Unlike the more lighweight [pt_search()] results from this endpoint 4 | #' will be available at the returned URL. 5 | #' 6 | #' @param query search term (e.g. an IP address, domain, or file hash) or valid Lucene query 7 | #' @param api_key your [packettotal_api_key()]. 8 | #' @export 9 | #' @references res 25 | 26 | httr::stop_for_status(res) 27 | 28 | out <- httr::content(res, as = "text", encoding = "UTF-8") 29 | 30 | out <- jsonlite::fromJSON(out) 31 | 32 | class(out) <- "pt_search_result" 33 | 34 | out 35 | 36 | } 37 | 38 | #' @rdname pt_deep_search 39 | #' @param search_result output from [pt_deep_search()] or a plain search results id 40 | #' @export 41 | pt_get_search_results <- function(search_result, api_key = packettotal_api_key()) { 42 | 43 | res_url <- NULL 44 | if (inherits(search_result, "pt_search_result")) { 45 | res_url <- sprintf("https://api.packettotal.com%s", search_result$results_uri) 46 | } else if (is.character(search_result)) { 47 | search_result <- search_result[1] 48 | if (grepl("v1/", search_result)) { 49 | res_url <- sprintf("https://api.packettotal.com%s", search_result) 50 | } else { 51 | res_url <- sprintf("https://api.packettotal.com/v1/search/deep/results/%s", search_result) 52 | } 53 | } 54 | 55 | if (is.null(res_url)) stop("Unrecognized search result.", call.=FALSE) 56 | 57 | httr::GET( 58 | url = res_url, 59 | httr::add_headers( 60 | `x-api-key` = api_key 61 | ), 62 | .PACKETTOTAL_UA 63 | ) -> res 64 | 65 | httr::stop_for_status(res) 66 | 67 | out <- httr::content(res, as = "text", encoding = "UTF-8") 68 | 69 | out <- jsonlite::fromJSON(out) 70 | 71 | out 72 | 73 | } -------------------------------------------------------------------------------- /R/graph.R: -------------------------------------------------------------------------------- 1 | #' Get a similarity graph relative to the current PCAP file. 2 | #' 3 | #' Results contain PCAPs that exhibit similar behaviors or contain similar content. Results are organized with the most similar PCAPs on top, and the terms that were found shared within both. 4 | #' 5 | #' @param pcap_id An md5 hash corresponding to the PCAP file submission on PacketTotal.com. 6 | #' This hash can be derived by hashing the PCAP file in question. 7 | #' @param weighting_mode One of "`behavior`" (default) or "`content`". Weight search results either based on their similarity to the behaviors exhibited or contents contained within the current PCAP file. 8 | #' @param intensity One of "`minimal`" (default), "`low`", "`medium`", or "`high`". The scope of the search, basically translates to the maximum number of aggregations to exhaust. Using a high level intensity, may result in occassional timeouts. 9 | #' @param prioritize_uncommon_fields By default, the most common values are used to seed the initial similarity search. Enabling this parameter, seeds the initial search with the least common values instead. 10 | #' @param api_key your [packettotal_api_key()]. 11 | #' @references 12 | #' @export 13 | #' @examples 14 | #' str(try(pt_similar("536cf06ca83704844d789f56caf22ee6"), silent=TRUE), 3) 15 | pt_similar <- function(pcap_id, 16 | weighting_mode = c("behavior", "content"), 17 | intensity = c("minimal", "low", "medium", "high"), 18 | prioritize_uncommon_fields = FALSE, 19 | api_key = packettotal_api_key()) { 20 | 21 | weighting_mode <- match.arg(tolower(weighting_mode), c("behavior", "content")) 22 | intensity <- match.arg(tolower(intensity), c("minimal", "low", "medium", "high")) 23 | prioritize_uncommon_fields <- tolower(as.character(FALSE)) 24 | 25 | httr::GET( 26 | url = sprintf("https://api.packettotal.com/v1/pcaps/%s/similar", pcap_id), 27 | query = list( 28 | weighting_mode = weighting_mode, 29 | intensity = intensity, 30 | prioritize_uncommon_fields = prioritize_uncommon_fields 31 | ), 32 | httr::add_headers( 33 | `x-api-key` = api_key 34 | ), 35 | .PACKETTOTAL_UA 36 | ) -> res 37 | 38 | httr::stop_for_status(res) 39 | 40 | out <- httr::content(res, as = "text", encoding = "UTF-8") 41 | 42 | out <- jsonlite::fromJSON(out) 43 | 44 | out 45 | 46 | } -------------------------------------------------------------------------------- /README.Rmd: -------------------------------------------------------------------------------- 1 | --- 2 | output: rmarkdown::github_document 3 | editor_options: 4 | chunk_output_type: console 5 | --- 6 | ```{r pkg-knitr-opts, include=FALSE} 7 | knitr::opts_chunk$set(collapse=TRUE, fig.retina=2, message=FALSE, warning=FALSE) 8 | options(width=120) 9 | ``` 10 | 11 | [![Travis-CI Build Status](https://travis-ci.org/hrbrmstr/packettotal.svg?branch=master)](https://travis-ci.org/hrbrmstr/packettotal) 12 | [![Coverage Status](https://codecov.io/gh/hrbrmstr/packettotal/branch/master/graph/badge.svg)](https://codecov.io/gh/hrbrmstr/packettotal) 13 | [![CRAN_Status_Badge](https://www.r-pkg.org/badges/version/packettotal)](https://cran.r-project.org/package=packettotal) 14 | 15 | # packettotal 16 | 17 | Lookup and Analyze Packet Capture ('PCAP') Files 18 | 19 | ## Description 20 | 21 | 'PacketTotal' () is an engine for analyzing, 22 | categorizing, and sharing packet capture ('PCAP') files. The tool was built 23 | with the information security community in mind and has applications in malware 24 | analysis and network forensics. Methods are provided to query search for and 25 | analyze packet capture files. 26 | 27 | ## What's Inside The Tin 28 | 29 | The following functions are implemented: 30 | 31 | - `packettotal_api_key`: Get or set PACKETTOTAL_API_KEY value 32 | - `pt_deep_search`/`pt_get_search_results`: Create a new deep search task. Search for a term or with a Lucene query. 33 | - `pt_detail`: Get a detailed report of PCAP traffic, carved files, signatures, and top-talkers. 34 | - `pt_download`: Download a PCAP analysis archive. The result is a zip archive containing the PCAP itself, CSVs representing various analysis results, and all carved files. 35 | - `pt_info`: Get high-level information about a specific PCAP file. 36 | - `pt_random`: Get high-level information about a random PCAP file. 37 | - `pt_search`: Search with term or with a valid Lucene query. 38 | - `pt_similar`: Get a similarity graph relative to the current PCAP file. 39 | - `pt_usage`: Retrive usage and subscription plan information. 40 | 41 | ## Installation 42 | 43 | ```{r install-ex, eval=FALSE} 44 | install.packages("packettotal", repos = "https://cinc.rud.is/") 45 | ``` 46 | 47 | ## Usage 48 | 49 | ```{r lib-ex} 50 | library(packettotal) 51 | 52 | # current version 53 | packageVersion("packettotal") 54 | 55 | ``` 56 | 57 | ```{r random} 58 | str(pt_random(), 2) 59 | ``` 60 | 61 | ```{r search} 62 | str(pt_search("evil.com"), 2) 63 | ``` 64 | 65 | ```{r deep-search} 66 | (res <- pt_deep_search("botnet OR malware")) 67 | 68 | str(pt_get_search_results(res), 2) 69 | ``` 70 | 71 | ```{r info} 72 | str(pt_info("d210f4dbea97949f694e849507951881"), 2) 73 | ``` 74 | 75 | ```{r detail} 76 | str(pt_detail("d210f4dbea97949f694e849507951881"), 2) 77 | ``` 78 | 79 | ```{r similar} 80 | str(pt_similar("536cf06ca83704844d789f56caf22ee6"), 2) 81 | ``` 82 | 83 | ## packettotal Metrics 84 | 85 | ```{r cloc, echo=FALSE} 86 | cloc::cloc_pkg_md() 87 | ``` 88 | 89 | ## Code of Conduct 90 | 91 | Please note that this project is released with a [Contributor Code of Conduct](CONDUCT.md). 92 | By participating in this project you agree to abide by its terms. 93 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | [![Travis-CI Build 3 | Status](https://travis-ci.org/hrbrmstr/packettotal.svg?branch=master)](https://travis-ci.org/hrbrmstr/packettotal) 4 | [![Coverage 5 | Status](https://codecov.io/gh/hrbrmstr/packettotal/branch/master/graph/badge.svg)](https://codecov.io/gh/hrbrmstr/packettotal) 6 | [![CRAN\_Status\_Badge](https://www.r-pkg.org/badges/version/packettotal)](https://cran.r-project.org/package=packettotal) 7 | 8 | # packettotal 9 | 10 | Lookup and Analyze Packet Capture (‘PCAP’) Files 11 | 12 | ## Description 13 | 14 | ‘PacketTotal’ () is an engine for analyzing, 15 | categorizing, and sharing packet capture (‘PCAP’) files. The tool was 16 | built with the information security community in mind and has 17 | applications in malware analysis and network forensics. Methods are 18 | provided to query search for and analyze packet capture files. 19 | 20 | ## What’s Inside The Tin 21 | 22 | The following functions are implemented: 23 | 24 | - `packettotal_api_key`: Get or set PACKETTOTAL\_API\_KEY value 25 | - `pt_deep_search`/`pt_get_search_results`: Create a new deep search 26 | task. Search for a term or with a Lucene query. 27 | - `pt_detail`: Get a detailed report of PCAP traffic, carved files, 28 | signatures, and top-talkers. 29 | - `pt_download`: Download a PCAP analysis archive. The result is a zip 30 | archive containing the PCAP itself, CSVs representing various 31 | analysis results, and all carved files. 32 | - `pt_info`: Get high-level information about a specific PCAP file. 33 | - `pt_random`: Get high-level information about a random PCAP file. 34 | - `pt_search`: Search with term or with a valid Lucene query. 35 | - `pt_similar`: Get a similarity graph relative to the current PCAP 36 | file. 37 | - `pt_usage`: Retrive usage and subscription plan information. 38 | 39 | ## Installation 40 | 41 | ``` r 42 | install.packages("packettotal", repos = "https://cinc.rud.is/") 43 | ``` 44 | 45 | ## Usage 46 | 47 | ``` r 48 | library(packettotal) 49 | 50 | # current version 51 | packageVersion("packettotal") 52 | ## [1] '0.1.0' 53 | ``` 54 | 55 | ``` r 56 | str(pt_random(), 2) 57 | ## List of 1 58 | ## $ pcap_metadata:List of 11 59 | ## ..$ md5 : chr "62286d51c23ee508a4687500becbaf52" 60 | ## ..$ name : chr "20130820_c_win6_00023_pc.pcap" 61 | ## ..$ byte_size : int 1538604 62 | ## ..$ logs : chr [1:8] "conn" "dns" "weird" "files" ... 63 | ## ..$ analyzed_date : chr "2018-10-19 00:55:42" 64 | ## ..$ download_link : chr "/pcaps/62286d51c23ee508a4687500becbaf52/download" 65 | ## ..$ analysis_link : chr "/pcaps/62286d51c23ee508a4687500becbaf52/analysis" 66 | ## ..$ similar_pcaps_link: chr "/pcaps/62286d51c23ee508a4687500becbaf52/similar" 67 | ## ..$ pcap_glyph_link : chr "https://s3.amazonaws.com/packettotalpub/files/62286d51c23ee508a4687500becbaf52/pcap-mosaic.png" 68 | ## ..$ packettotal_link : chr "https://packettotal.com/app/analysis?id=62286d51c23ee508a4687500becbaf52" 69 | ## ..$ message : chr "This PCAP was selected randomly, since no id was specified." 70 | ``` 71 | 72 | ``` r 73 | str(pt_search("evil.com"), 2) 74 | ## List of 2 75 | ## $ result_count: int 5 76 | ## $ results :'data.frame': 5 obs. of 3 variables: 77 | ## ..$ id : chr [1:5] "b2a094b1882f52ab8befd3d8ad9d7f9a" "0826bfbd4a68519945b9af594a5a87d7" "385b9a5b3da0d56260f2be329e110795" "8e13e95bc12ad8415c4d8e8d313affac" ... 78 | ## ..$ found_in :List of 5 79 | ## ..$ match_score: num [1:5] 49.5 49.3 44.2 31.8 31.6 80 | ``` 81 | 82 | ``` r 83 | (res <- pt_deep_search("botnet OR malware")) 84 | ## $search_id 85 | ## [1] "089f9e75d8142e185e84e8668da4b9b8" 86 | ## 87 | ## $message 88 | ## [1] "Deep search exists." 89 | ## 90 | ## $results_uri 91 | ## [1] "/v1/search/deep/results/089f9e75d8142e185e84e8668da4b9b8" 92 | ## 93 | ## attr(,"class") 94 | ## [1] "pt_search_result" 95 | 96 | str(pt_get_search_results(res), 2) 97 | ## List of 2 98 | ## $ results :'data.frame': 1819 obs. of 3 variables: 99 | ## ..$ id : chr [1:1819] "bd00b1dca3e5586dccffdc23579b0d39" "ba796317651f2064b1ca193e6e2cf947" "52419b8eba8af8fe502f8be324b67cb8" "da0023e2c4ca40ac480a4fdb930e7745" ... 100 | ## ..$ found_in :List of 1819 101 | ## ..$ match_score: num [1:1819] 1584 1079 749 704 653 ... 102 | ## $ result_count: int 1819 103 | ``` 104 | 105 | ``` r 106 | str(pt_info("d210f4dbea97949f694e849507951881"), 2) 107 | ## List of 1 108 | ## $ pcap_metadata:List of 10 109 | ## ..$ md5 : chr "d210f4dbea97949f694e849507951881" 110 | ## ..$ name : chr "20180815Emotetinfectipca.pcap" 111 | ## ..$ byte_size : int 1583713 112 | ## ..$ logs : chr [1:10] "conn" "x509" "dns" "ssl" ... 113 | ## ..$ analyzed_date : chr "2019-01-01 06:40:18" 114 | ## ..$ download_link : chr "/pcaps/d210f4dbea97949f694e849507951881/download" 115 | ## ..$ analysis_link : chr "/pcaps/d210f4dbea97949f694e849507951881/analysis" 116 | ## ..$ similar_pcaps_link: chr "/pcaps/d210f4dbea97949f694e849507951881/similar" 117 | ## ..$ pcap_glyph_link : chr "https://s3.amazonaws.com/packettotalpub/files/d210f4dbea97949f694e849507951881/pcap-mosaic.png" 118 | ## ..$ packettotal_link : chr "https://packettotal.com/app/analysis?id=d210f4dbea97949f694e849507951881" 119 | ``` 120 | 121 | ``` r 122 | str(pt_detail("d210f4dbea97949f694e849507951881"), 2) 123 | ## List of 1 124 | ## $ analysis_summary:List of 9 125 | ## ..$ top_talkers :List of 2 126 | ## ..$ connection_statistics:List of 9 127 | ## ..$ dns_statistics :List of 2 128 | ## ..$ file_statistics :List of 3 129 | ## ..$ signatures : chr [1:4] "ET POLICY Office Document Download Containing AutoOpen Macro" "ET POLICY PE EXE or DLL Windows file download HTTP" "SURICATA TLS invalid record version" "SURICATA TLS invalid record/traffic" 130 | ## ..$ external_references :'data.frame': 7 obs. of 2 variables: 131 | ## ..$ malicious_traffic : logi FALSE 132 | ## ..$ accuracy : chr "perfect" 133 | ## ..$ http_statistics :List of 3 134 | ``` 135 | 136 | ``` r 137 | str(pt_similar("536cf06ca83704844d789f56caf22ee6"), 2) 138 | ## List of 4 139 | ## $ similar :List of 2 140 | ## ..$ result_count: int 78 141 | ## ..$ results :'data.frame': 78 obs. of 4 variables: 142 | ## $ intensity : chr "minimal" 143 | ## $ prioritize_uncommon_fields: logi TRUE 144 | ## $ weighting_mode : chr "behavior" 145 | ``` 146 | 147 | ## packettotal Metrics 148 | 149 | | Lang | \# Files | (%) | LoC | (%) | Blank lines | (%) | \# Lines | (%) | 150 | | :--- | -------: | ---: | --: | ---: | ----------: | ---: | -------: | --: | 151 | | R | 13 | 0.93 | 200 | 0.94 | 69 | 0.73 | 125 | 0.7 | 152 | | Rmd | 1 | 0.07 | 13 | 0.06 | 25 | 0.27 | 54 | 0.3 | 153 | 154 | ## Code of Conduct 155 | 156 | Please note that this project is released with a [Contributor Code of 157 | Conduct](CONDUCT.md). By participating in this project you agree to 158 | abide by its terms. 159 | --------------------------------------------------------------------------------