├── .gitignore ├── LICENSE ├── README.md ├── globe.png ├── go.mod ├── go.sum ├── main.go ├── public ├── index.html └── storj-logo.png └── server.go /.gitignore: -------------------------------------------------------------------------------- 1 | uplink-sniff 2 | GeoLite2-City.mmdb 3 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2024 Dan Willoughby 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Network Globe - Real-Time TCP Packet Sniffer for Network Location Visualization 2 | 3 | See where your TCP packets are coming from/going to in Real-time! 4 | 5 | This repository contains a Real-time TCP packet sniffer for network visualization. See the various locations where TCP packets are sent or received. The application parses IP addresses from packets, looks up their locations (lat and lng) using GeoLite2, and visualizes the packet data on a globe using Globe GL. 6 | 7 | ![Network Globe Visualization](./globe.png) 8 | 9 | See a [demo](https://demo.storj.dev) 10 | 11 | ## Features 12 | 13 | - Parses IP addresses from TCP packets using [pcap](https://pkg.go.dev/github.com/google/gopacket/pcap) 14 | - Performs location lookup using [GeoLite2](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en) 15 | - Visualizes locations on a globe with [Globe GL](https://globe.gl/) 16 | - Demo file uploads with [Storj](https://storj.io?ref=network-globe) 17 | 18 | ## Installation 19 | 20 | ### Prerequisites 21 | 22 | - libpcap (for packet capturing) 23 | - GeoLite2-City.mmdb database from [MaxMind](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en) 24 | - Obtain the Free [GeoLite2-City.mmdb database](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en) 25 | - Extract and place the database file (`GeoLite2-City.mmdb`) in the project directory or specify path with `--geolite2-path` flag 26 | 27 | ### Install 28 | 29 | #### Ubuntu 30 | 31 | ```bash 32 | sudo apt install libpcap-dev zip 33 | wget https://github.com/amozoss/network-globe/releases/latest/download/network-globe_linux_amd64.zip 34 | unzip network-globe_linux_amd64.zip 35 | ``` 36 | 37 | #### macOS 38 | 39 | ```bash 40 | wget https://github.com/amozoss/network-globe/releases/latest/download/network-globe_darwin_arm.zip 41 | unzip network-globe_darwin_arm.zip 42 | ``` 43 | 44 | ## Usage 45 | 46 | 1. Give read permissions for pcap to read the network packets: 47 | 48 | ```bash 49 | # for mac 50 | sudo chmod +r /dev/bpf* 51 | ``` 52 | 53 | 1. Configure your network interface: 54 | 55 | ```bash 56 | # List available network devices 57 | ./network-globe --list-devices 58 | # Default device is en0 59 | ./network-globe --device en0 60 | ``` 61 | 62 | 1. Set the GeoLite2 database path: 63 | 64 | ```bash 65 | # Default path is GeoLite2-City.mmdb 66 | ./network-globe --geolite2-path GeoLite2-City.mmdb 67 | ``` 68 | 69 | 1. Start network-globe: 70 | 71 | ```bash 72 | # may need to run with sudo on ubuntu 73 | ./network-globe 74 | ``` 75 | 76 | 1. Open the browser and navigate to 77 | 78 | 1. (Optional) Set the source location (Latitude and Longitude) for the Globe visualization: 79 | 80 | The origin is configured to be in the United States. You can set the source location using the `--lat` and `--lng` flags. 81 | 82 | ```bash 83 | ./network-globe --lat 39.781932 --lng -104.970578 84 | ``` 85 | 86 | 1. (Optional) Demo uploads to Storj: 87 | 88 | - Sign up for a free trial account at [Storj](https://storj.io?ref=network-globe). 89 | - Obtain an [Access Grant](https://docs.storj.io/dcs/access#create-access-grant) 90 | - Set the `--access` and `--bucket` flags with the Access Grant and Bucket name. 91 | 92 | - Use the provided script or your preferred method to upload a file to Storj. 93 | 94 | ## Build 95 | 96 | Clone the repository: 97 | 98 | ```bash 99 | git clone https://github.com/amozoss/network-globe.git 100 | cd network-globe 101 | ``` 102 | 103 | ### linux 104 | 105 | ```bash 106 | sudo apt install libpcap-dev gcc 107 | CGO_ENABLED=1 go build 108 | ``` 109 | 110 | ### macOS 111 | 112 | ```bash 113 | go build 114 | ``` 115 | 116 | ## Project Structure 117 | 118 | - `public/index.html`: HTML file for the Globe GL visualization and websocket connection. 119 | - `main.go`: packet sniffer and IP to lat and lng lookup written in Go using pcap library and MaxMind. 120 | - `server.go`: Handles the packets, formats the message for the frontend, and broadcasts them to the client. 121 | 122 | ## Contributing 123 | 124 | Contributions are welcome! Please submit a pull request or open an issue for any improvements or bug fixes. 125 | 126 | ## License 127 | 128 | This project is licensed under the MIT License. 129 | 130 | ## Acknowledgements 131 | 132 | - [Pcap tutorial](https://www.devdungeon.com/content/packet-capture-injection-and-analysis-gopacket) 133 | - [MaxMind GeoLite2](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data?lang=en) 134 | - [Globe GL](https://github.com/vasturiano/globe.gl) 135 | - [Storj](https://storj.io?ref=network-globe) 136 | -------------------------------------------------------------------------------- /globe.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/amozoss/network-globe/f3dc4b8c3f4bdb03f301c1ea0279ad7b96102c89/globe.png -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module network-globe 2 | 3 | go 1.18 4 | 5 | require ( 6 | github.com/google/gopacket v1.1.19 7 | github.com/gorilla/mux v1.8.0 8 | github.com/gorilla/websocket v1.5.0 9 | github.com/oschwald/maxminddb-golang v1.10.0 10 | storj.io/uplink v1.10.0 11 | ) 12 | 13 | require ( 14 | github.com/calebcase/tmpfile v1.0.3 // indirect 15 | github.com/gogo/protobuf v1.3.2 // indirect 16 | github.com/jtolio/eventkit v0.0.0-20221004135224-074cf276595b // indirect 17 | github.com/klauspost/cpuid/v2 v2.0.12 // indirect 18 | github.com/spacemonkeygo/monkit/v3 v3.0.19 // indirect 19 | github.com/vivint/infectious v0.0.0-20200605153912-25a574ae18a3 // indirect 20 | github.com/zeebo/blake3 v0.2.3 // indirect 21 | github.com/zeebo/errs v1.3.0 // indirect 22 | golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e // indirect 23 | golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde // indirect 24 | golang.org/x/sys v0.0.0-20220804214406-8e32c043e418 // indirect 25 | storj.io/common v0.0.0-20221123115229-fed3e6651b63 // indirect 26 | storj.io/drpc v0.0.32 // indirect 27 | ) 28 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/calebcase/tmpfile v1.0.3 h1:BZrOWZ79gJqQ3XbAQlihYZf/YCV0H4KPIdM5K5oMpJo= 2 | github.com/calebcase/tmpfile v1.0.3/go.mod h1:UAUc01aHeC+pudPagY/lWvt2qS9ZO5Zzof6/tIUzqeI= 3 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 4 | github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q= 5 | github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q= 6 | github.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8= 7 | github.com/google/gopacket v1.1.19/go.mod h1:iJ8V8n6KS+z2U1A8pUwu8bW5SyEMkXJB8Yo/Vo+TKTo= 8 | github.com/google/pprof v0.0.0-20211108044417-e9b028704de0 h1:rsq1yB2xiFLDYYaYdlGBsSkwVzsCo500wMhxvW5A/bk= 9 | github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI= 10 | github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So= 11 | github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc= 12 | github.com/gorilla/websocket v1.5.0/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= 13 | github.com/jtolio/eventkit v0.0.0-20221004135224-074cf276595b h1:tO4MX3k5bvV0Sjv5jYrxStMTJxf1m/TW24XRyHji4aU= 14 | github.com/jtolio/eventkit v0.0.0-20221004135224-074cf276595b/go.mod h1:q7yMR8BavTz/gBNtIT/uF487LMgcuEpNGKISLAjNQes= 15 | github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= 16 | github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= 17 | github.com/klauspost/cpuid/v2 v2.0.12 h1:p9dKCg8i4gmOxtv35DvrYoWqYzQrvEVdjQ762Y0OqZE= 18 | github.com/klauspost/cpuid/v2 v2.0.12/go.mod h1:g2LTdtYhdyuGPqyWyv7qRAmj1WBqxuObKfj5c0PQa7c= 19 | github.com/oschwald/maxminddb-golang v1.10.0 h1:Xp1u0ZhqkSuopaKmk1WwHtjF0H9Hd9181uj2MQ5Vndg= 20 | github.com/oschwald/maxminddb-golang v1.10.0/go.mod h1:Y2ELenReaLAZ0b400URyGwvYxHV1dLIxBuyOsyYjHK0= 21 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 22 | github.com/spacemonkeygo/monkit/v3 v3.0.19 h1:wqBb9bpD7jXkVi4XwIp8jn1fektaVBQ+cp9SHRXgAdo= 23 | github.com/spacemonkeygo/monkit/v3 v3.0.19/go.mod h1:kj1ViJhlyADa7DiA4xVnTuPA46lFKbM7mxQTrXCuJP4= 24 | github.com/stretchr/testify v1.7.3 h1:dAm0YRdRQlWojc3CrCRgPBzG5f941d0zvAKu7qY4e+I= 25 | github.com/vivint/infectious v0.0.0-20200605153912-25a574ae18a3 h1:zMsHhfK9+Wdl1F7sIKLyx3wrOFofpb3rWFbA4HgcK5k= 26 | github.com/vivint/infectious v0.0.0-20200605153912-25a574ae18a3/go.mod h1:R0Gbuw7ElaGSLOZUSwBm/GgVwMd30jWxBDdAyMOeTuc= 27 | github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= 28 | github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= 29 | github.com/zeebo/assert v1.1.0/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0= 30 | github.com/zeebo/assert v1.3.1 h1:vukIABvugfNMZMQO1ABsyQDJDTVQbn+LWSMy1ol1h6A= 31 | github.com/zeebo/assert v1.3.1/go.mod h1:Pq9JiuJQpG8JLJdtkwrJESF0Foym2/D9XMU5ciN/wJ0= 32 | github.com/zeebo/blake3 v0.2.3 h1:TFoLXsjeXqRNFxSbk35Dk4YtszE/MQQGK10BH4ptoTg= 33 | github.com/zeebo/blake3 v0.2.3/go.mod h1:mjJjZpnsyIVtVgTOSpJ9vmRE4wgDeyt2HU3qXvvKCaQ= 34 | github.com/zeebo/errs v1.3.0 h1:hmiaKqgYZzcVgRL1Vkc1Mn2914BbzB0IBxs+ebeutGs= 35 | github.com/zeebo/errs v1.3.0/go.mod h1:sgbWHsvVuTPHcqJJGQ1WhI5KbWlHYz+2+2C/LSEtCw4= 36 | github.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo= 37 | github.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4= 38 | golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= 39 | golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI= 40 | golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto= 41 | golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e h1:T8NU3HyQ8ClP4SEE+KbFlg6n0NhuTsN4MyznaarGsZM= 42 | golang.org/x/crypto v0.0.0-20220525230936-793ad666bf5e/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4= 43 | golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY= 44 | golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= 45 | golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= 46 | golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= 47 | golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= 48 | golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 49 | golang.org/x/net v0.0.0-20190923162816-aa69164e4478/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 50 | golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= 51 | golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= 52 | golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e h1:TsQ7F31D3bUCLeqPT0u+yjp1guoArKaNKmCr22PYgTQ= 53 | golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 54 | golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 55 | golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 56 | golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde h1:ejfdSekXMDxDLbRrJMwUk6KnSLZ2McaUCVcIKM+N6jc= 57 | golang.org/x/sync v0.0.0-20220819030929-7fc1605a5dde/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= 58 | golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= 59 | golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 60 | golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 61 | golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= 62 | golang.org/x/sys v0.0.0-20210514084401-e8d321eab015/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= 63 | golang.org/x/sys v0.0.0-20220804214406-8e32c043e418 h1:9vYwv7OjYaky/tlAeD7C4oC9EsPTlaFl1H2jS++V+ME= 64 | golang.org/x/sys v0.0.0-20220804214406-8e32c043e418/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= 65 | golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= 66 | golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= 67 | golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= 68 | golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo= 69 | golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28= 70 | golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= 71 | golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= 72 | golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 73 | golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 74 | golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 75 | golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= 76 | gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= 77 | storj.io/common v0.0.0-20221123115229-fed3e6651b63 h1:OuleF/3FvZe3Nnu6NdwVr+FvCXjfD4iNNdgfI2kcs3k= 78 | storj.io/common v0.0.0-20221123115229-fed3e6651b63/go.mod h1:+gF7jbVvpjVIVHhK+EJFhfPbudX395lnPq/dKkj/Qys= 79 | storj.io/drpc v0.0.32 h1:5p5ZwsK/VOgapaCu+oxaPVwO6UwIs+iwdMiD50+R4PI= 80 | storj.io/drpc v0.0.32/go.mod h1:6rcOyR/QQkSTX/9L5ZGtlZaE2PtXTTZl8d+ulSeeYEg= 81 | storj.io/uplink v1.10.0 h1:3hS0hszupHSxEoC4DsMpljaRy0uNoijEPVF6siIE28Q= 82 | storj.io/uplink v1.10.0/go.mod h1:gJIQumB8T3tBHPRive51AVpbc+v2xe+P/goFNMSRLG4= 83 | -------------------------------------------------------------------------------- /main.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "context" 5 | "encoding/json" 6 | "errors" 7 | "flag" 8 | "fmt" 9 | "log" 10 | "net" 11 | "net/http" 12 | "strings" 13 | "sync" 14 | "time" 15 | 16 | "github.com/google/gopacket" 17 | "github.com/google/gopacket/layers" 18 | "github.com/google/gopacket/pcap" 19 | "github.com/oschwald/maxminddb-golang" 20 | "storj.io/uplink" 21 | ) 22 | 23 | var ( 24 | snapshotLen int32 = 1024 25 | promiscuous bool = false 26 | err error 27 | timeout time.Duration = 1 * time.Second 28 | handle *pcap.Handle 29 | ) 30 | var dataSentToIP = make(map[string]int) 31 | var mutex = &sync.Mutex{} 32 | 33 | func main() { 34 | frontendDir := flag.String("frontend-dir", "", "Alternative static files directory") 35 | hostname := flag.String("host", "0.0.0.0", "name of the host") 36 | port := flag.Int("port", 8000, "port") 37 | 38 | accessGrant := flag.String("access-grant", "", "access grant for storj") 39 | bucket := flag.String("bucket", "network-globe", "bucket to upload to") 40 | 41 | mmdbPath := flag.String("geolite2-path", "./GeoLite2-City.mmdb", "path to GeoLite2-City.mmdb") 42 | 43 | listDevices := flag.Bool("list-devices", false, "list devices") 44 | // TODO Should be with a format flag and all messages should be in json 45 | isJson := flag.Bool("json", false, "output device list in json") 46 | 47 | batchSize := flag.Int("batch-size", 1, "number of detected connections to batch send to frontend") 48 | srcLat := flag.Float64("lat", 39.781932, "src lat - where lines start") 49 | srcLng := flag.Float64("lng", -104.970578, "src lng - where lines start") 50 | device := flag.String("device", "en0", "list devices") 51 | // TODO better way to do debugging 52 | debug := flag.Bool("debug", true, "debug messages") 53 | flag.Parse() 54 | 55 | if *listDevices { 56 | printDevices(*isJson) 57 | return 58 | } 59 | 60 | ctx := context.Background() 61 | var project *uplink.Project 62 | if *accessGrant != "" { 63 | access, err := uplink.ParseAccess(*accessGrant) 64 | if err != nil { 65 | log.Fatalln(err) 66 | } 67 | project, err = uplink.OpenProject(ctx, access) 68 | if err != nil { 69 | log.Fatalln(err) 70 | } 71 | defer project.Close() 72 | project.EnsureBucket(ctx, *bucket) 73 | } 74 | 75 | server := NewServer(*frontendDir, project, *bucket, *batchSize, *debug) 76 | 77 | go func() { 78 | log.Printf("Listening on %s:%d\n", *hostname, *port) 79 | log.Fatal(http.ListenAndServe(fmt.Sprintf("%s:%d", *hostname, *port), server)) 80 | }() 81 | go func() { 82 | server.StartBroadcasts() 83 | }() 84 | 85 | db, err := maxminddb.Open(*mmdbPath) 86 | if err != nil { 87 | if strings.Contains(err.Error(), "no such file or directory") { 88 | log.Fatal(err.Error() + "\n\nDownload the free GeoLite2-City.mmdb file from:\n\n https://dev.maxmind.com/geoip/geolite2-free-geolocation-data\n\nThen run with `--geolite2-path `\n\n") 89 | return 90 | } 91 | log.Fatal(err) 92 | } 93 | defer db.Close() 94 | 95 | // Open device 96 | handle, err = pcap.OpenLive(*device, snapshotLen, promiscuous, timeout) 97 | if err != nil { 98 | // https://github.com/hortinstein/node-dash-button/issues/15 99 | if strings.Contains(err.Error(), "Permission denied") { 100 | log.Fatal(err.Error() + "\n\nTo fix run in terminal and restart:\n\n sudo chmod +r /dev/bpf*\n\n") 101 | return 102 | } 103 | if strings.Contains(err.Error(), "don't have permission to capture") { 104 | log.Fatal(err.Error() + "\n\nMight need to run as sudo") 105 | return 106 | } 107 | if strings.Contains(err.Error(), "No such device exists") { 108 | log.Fatal(err.Error() + "\n\nDetermine your network device `:\n\n ./network-globe --list-devices\n\nThen run with `--device `\n\n") 109 | return 110 | } 111 | log.Fatal(err) 112 | } 113 | defer handle.Close() 114 | filter := "tcp" 115 | // filter := "tcp[13] & 2!=0" 116 | //filter := "tcp[tcpflags] & (tcp-syn) != 0" 117 | err = handle.SetBPFFilter(filter) 118 | if err != nil { 119 | log.Fatal(err) 120 | } 121 | 122 | fmt.Println("Only capturing TCP packets.") 123 | packetSource := gopacket.NewPacketSource(handle, handle.LinkType()) 124 | myIp := getIPv4FromInterface(*device) 125 | for packet := range packetSource.Packets() { 126 | ip, _, rec, err := ipToCoord(db, packet, myIp) 127 | if err != nil { 128 | //log.Println(err) 129 | continue 130 | } 131 | if ip != nil && rec != nil { 132 | if rec.PacketSize > 900 && *debug { 133 | msg := fmt.Sprintf("%15s -> %-15s %d %s %20s %20s %f,%f\n", ip.SrcIP, ip.DstIP, rec.PacketSize, rec.DataDirection, myIp, rec.Country.Names.En, rec.Location.Latitude, rec.Location.Longitude) 134 | log.Println(msg) 135 | } 136 | 137 | src := LatLng{*srcLat, *srcLng} 138 | dst := LatLng{rec.Location.Latitude, rec.Location.Longitude} 139 | if !ip.SrcIP.Equal(myIp) { 140 | tmp := dst 141 | dst = src 142 | src = tmp 143 | } 144 | server.Queue(&Message{ 145 | Src: src, 146 | Dst: dst, 147 | Direction: rec.DataDirection, 148 | Name: rec.Country.Names.En, 149 | }) 150 | } 151 | } 152 | } 153 | 154 | // map[continent:map[code:NA geoname_id:6255149 names:map[de:Nordamerika en:North America es:Norteamérica fr:Amérique du Nord ja:北アメリカ pt-BR:América do Norte ru:Северная Америка zh-CN:北美洲]] 155 | // 156 | // country:map[geoname_id:6252001 iso_code:US names:map[de:USA en:United States es:Estados Unidos fr:États-Unis ja:アメリカ合衆国 pt-BR:Estados Unidos ru:США zh-CN:美国]] 157 | // location:map[accuracy_radius:1000 latitude:37.751 longitude:-97.822 time_zone:America/Chicago] 158 | // registered_country:map[geoname_id:6252001 iso_code:US names:map[de:USA en:United States es:Estados Unidos fr:États-Unis ja:アメリカ合衆国 pt-BR:Estados Unidos ru:США zh-CN:美国]]] 159 | type record struct { 160 | PacketSize int 161 | DataDirection string 162 | Country struct { 163 | ISOCode string `maxminddb:"iso_code"` 164 | Names struct { 165 | En string `maxminddb:"en"` 166 | } `maxminddb:"names"` 167 | } `maxminddb:"country"` 168 | Location struct { 169 | ISOCode int `maxminddb:"accuracy_radius"` 170 | Latitude float64 `maxminddb:"latitude"` 171 | Longitude float64 `maxminddb:"longitude"` 172 | } `maxminddb:"location"` 173 | } 174 | 175 | func getIPv4FromInterface(device string) net.IP { 176 | iface, err := net.InterfaceByName(device) 177 | if err != nil { 178 | log.Fatal(err) 179 | } 180 | 181 | addrs, err := iface.Addrs() 182 | if err != nil { 183 | log.Fatal(err) 184 | } 185 | 186 | // Get the first IPv4 address of the interface 187 | for _, addr := range addrs { 188 | switch v := addr.(type) { 189 | case *net.IPNet: 190 | // Check if this is an IPv4 address 191 | if v.IP.To4() != nil { 192 | return v.IP 193 | } 194 | case *net.IPAddr: 195 | // Check if this is an IPv4 address 196 | if v.IP.To4() != nil { 197 | return v.IP 198 | } 199 | } 200 | } 201 | 202 | return nil 203 | } 204 | 205 | func ipToCoord(db *maxminddb.Reader, packet gopacket.Packet, myIP net.IP) (ip *layers.IPv4, tcp *layers.TCP, rec *record, err error) { 206 | ipLayer := packet.Layer(layers.LayerTypeIPv4) 207 | if ipLayer != nil { 208 | ip, _ := ipLayer.(*layers.IPv4) 209 | tcpLayer := packet.Layer(layers.LayerTypeTCP) 210 | if tcpLayer != nil { 211 | tcp, _ := tcpLayer.(*layers.TCP) 212 | 213 | // Skip the TCP handshake packets 214 | if len(tcp.Payload) > 0 { 215 | ipLookup := ip.SrcIP 216 | rec = &record{} 217 | if ip.SrcIP.Equal(myIP) { 218 | ipLookup = ip.DstIP 219 | rec.DataDirection = "Upload" 220 | } else { 221 | rec.DataDirection = "Download" 222 | } 223 | 224 | err = db.Lookup(ipLookup, rec) 225 | if err != nil { 226 | log.Fatal(err) 227 | } 228 | 229 | // Determine if the packet is upload or download 230 | rec.PacketSize = len(tcp.Payload) 231 | 232 | // Update the data sent to the destination IP 233 | mutex.Lock() 234 | if rec.DataDirection == "Upload" { 235 | dataSentToIP[ip.DstIP.String()] += len(tcp.Payload) 236 | } else { 237 | dataSentToIP[ip.SrcIP.String()] += len(tcp.Payload) 238 | } 239 | mutex.Unlock() 240 | 241 | return ip, tcp, rec, nil 242 | } 243 | } 244 | } 245 | return nil, nil, nil, errors.New("not found") 246 | } 247 | 248 | func printDevices(isJson bool) { 249 | devices, err := pcap.FindAllDevs() 250 | if err != nil { 251 | log.Fatal(err) 252 | } 253 | 254 | if isJson { 255 | jsonData, err := json.MarshalIndent(devices, "", " ") 256 | if err != nil { 257 | log.Fatal(err) 258 | } 259 | 260 | fmt.Println(string(jsonData)) 261 | } else { 262 | // Print device information 263 | fmt.Println("Devices found:") 264 | for _, device := range devices { 265 | fmt.Println("\nName: ", device.Name) 266 | fmt.Println("Description: ", device.Description) 267 | fmt.Println("Devices addresses: ", device.Description) 268 | for _, address := range device.Addresses { 269 | fmt.Println("- IP address: ", address.IP) 270 | fmt.Println("- Subnet mask: ", address.Netmask) 271 | } 272 | } 273 | } 274 | } 275 | -------------------------------------------------------------------------------- /public/index.html: -------------------------------------------------------------------------------- 1 | 2 | 7 | 8 | 12 | 13 | 17 | 21 | 22 | 23 | 24 | 28 | 29 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 |
43 | 313 | 314 | -------------------------------------------------------------------------------- /public/storj-logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/amozoss/network-globe/f3dc4b8c3f4bdb03f301c1ea0279ad7b96102c89/public/storj-logo.png -------------------------------------------------------------------------------- /server.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "bytes" 5 | "context" 6 | "crypto/rand" 7 | "embed" 8 | "fmt" 9 | "io" 10 | "io/fs" 11 | "log" 12 | "net/http" 13 | "sync" 14 | "time" 15 | 16 | "github.com/gorilla/mux" 17 | "github.com/gorilla/websocket" 18 | "storj.io/uplink" 19 | ) 20 | 21 | var ( 22 | colors []string = []string{ 23 | "#00E366", 24 | "#FF458B", 25 | "#FFC600", 26 | "#FFFFFF", 27 | "#0149FF", 28 | "#9B4FFF", 29 | "#00BFEA", 30 | "#FF7E2E", 31 | "#EBEEF1", 32 | } 33 | uploadInterval time.Duration = 6 * time.Second 34 | //go:embed public/* 35 | files embed.FS 36 | ) 37 | 38 | type embedFileSystem struct { 39 | http.FileSystem 40 | } 41 | 42 | func (e embedFileSystem) Open(name string) (http.File, error) { 43 | fmt.Printf("Opening file: %s\n", name) // Add logging for debugging 44 | return e.FileSystem.Open(name) 45 | } 46 | 47 | type Server struct { 48 | mux *mux.Router 49 | 50 | socketMu sync.Mutex 51 | sockets []*websocket.Conn 52 | // end socketMu 53 | 54 | serverMu sync.Mutex 55 | shouldUpload bool 56 | messages []*Message 57 | srcDestPacketCount map[string]int64 58 | queuedForFrontend map[string]bool 59 | colorIndex int 60 | // end serverMu 61 | 62 | uploadTicker *time.Ticker 63 | project *uplink.Project 64 | bucket string 65 | batchSize int 66 | debug bool 67 | } 68 | 69 | func NewServer(frontendDir string, project *uplink.Project, bucket string, batchSize int, debug bool) *Server { 70 | server := &Server{ 71 | mux: mux.NewRouter(), 72 | colorIndex: 0, 73 | uploadTicker: time.NewTicker(uploadInterval), 74 | project: project, 75 | bucket: bucket, 76 | debug: debug, 77 | srcDestPacketCount: make(map[string]int64), 78 | queuedForFrontend: make(map[string]bool), 79 | } 80 | server.mux.HandleFunc("/ws", server.socketHandler) 81 | var dir http.FileSystem 82 | if frontendDir == "" { 83 | fsys, err := fs.Sub(files, "public") 84 | if err != nil { 85 | panic(err) 86 | } 87 | dir = embedFileSystem{http.FS(fsys)} 88 | } else { 89 | dir = http.Dir(frontendDir) 90 | } 91 | 92 | server.mux.PathPrefix("/").Handler(http.FileServer(dir)) 93 | go func() { 94 | }() 95 | return server 96 | } 97 | 98 | func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) { s.mux.ServeHTTP(w, r) } 99 | 100 | type LatLng struct { 101 | Lat float64 `json:"lat"` 102 | Lng float64 `json:"lng"` 103 | } 104 | 105 | type Message struct { 106 | Src LatLng `json:"src"` 107 | Dst LatLng `json:"dst"` 108 | Direction string `json:"direction"` 109 | Count int `json:"count"` 110 | Name string `json:"name"` 111 | Color string `json:"color"` 112 | } 113 | 114 | type BatchMessage struct { 115 | Messages []*Message `json:"messages"` 116 | } 117 | 118 | func (s *Server) StartBroadcasts() { 119 | for { 120 | select { 121 | case <-s.uploadTicker.C: 122 | if s.project != nil { 123 | s.uplinkUpload() 124 | } 125 | s.Broadcast() 126 | } 127 | } 128 | } 129 | 130 | func (s *Server) setShouldUpload(upload bool) { 131 | s.serverMu.Lock() 132 | defer s.serverMu.Unlock() 133 | s.shouldUpload = upload 134 | } 135 | 136 | func (s *Server) uplinkUpload() { 137 | ctx := context.TODO() 138 | if s.debug { 139 | log.Println("upload started") 140 | } 141 | 142 | upload, err := s.project.UploadObject(ctx, s.bucket, "test.txt", &uplink.UploadOptions{ 143 | Expires: time.Now().Add(1 * time.Hour), 144 | }) 145 | if err != nil { 146 | log.Println("UploadObject error:", err) 147 | } 148 | 149 | // random bytes to test upload 150 | buffer := make([]byte, 8024) 151 | _, err = rand.Read(buffer) 152 | if err != nil { 153 | log.Fatalf("Failed to generate random bytes: %v", err) 154 | } 155 | 156 | // Create a bytes.Reader from the buffer 157 | randomBytes := bytes.NewReader(buffer) 158 | 159 | _, err = io.Copy(upload, randomBytes) 160 | if err != nil { 161 | log.Println("UploadObject io.Copy error:", err) 162 | } 163 | 164 | err = upload.Commit() 165 | if err != nil { 166 | log.Println("upload.Commit error:", err) 167 | } 168 | if s.debug { 169 | log.Println("upload finished") 170 | } 171 | s.setShouldUpload(false) 172 | s.Broadcast() 173 | } 174 | 175 | func (s *Server) socketHandler(w http.ResponseWriter, r *http.Request) { 176 | // Upgrade our raw HTTP connection to a websocket based one 177 | var upgrader = websocket.Upgrader{ 178 | CheckOrigin: func(r *http.Request) bool { 179 | // Check the origin here and return true if it's valid 180 | // For example, allow all origins: 181 | return true 182 | }, 183 | } 184 | 185 | conn, err := upgrader.Upgrade(w, r, nil) 186 | if err != nil { 187 | log.Print("Error during connection upgradation:", err) 188 | return 189 | } 190 | defer conn.Close() 191 | s.socketMu.Lock() 192 | // TODO handle keep-alives and clean-up of the connections 193 | // a more robust solution https://github.com/gorilla/websocket/blob/master/examples/chat/client.go 194 | s.sockets = append(s.sockets, conn) 195 | s.socketMu.Unlock() 196 | // The event loop 197 | for { 198 | _, message, err := conn.ReadMessage() 199 | if err != nil { 200 | log.Println("Error during message reading:", err) 201 | break 202 | } 203 | log.Printf("Received: %s", message) 204 | msg := string(message) 205 | if msg == "start" { 206 | s.setShouldUpload(true) 207 | } 208 | if msg == "routes_done" { 209 | s.setShouldUpload(true) 210 | } 211 | } 212 | } 213 | 214 | func (s *Server) Queue(msg *Message) { 215 | 216 | s.serverMu.Lock() 217 | defer s.serverMu.Unlock() 218 | srcKey := fmt.Sprintf("%f,%f:%f,%f", msg.Src.Lat, msg.Src.Lng, msg.Dst.Lat, msg.Dst.Lng) 219 | s.srcDestPacketCount[srcKey] += 1 220 | 221 | msg.Color = colors[s.colorIndex%(len(colors)-1)] 222 | msg.Count = int(s.srcDestPacketCount[srcKey]) 223 | 224 | if !s.queuedForFrontend[srcKey] { 225 | s.messages = append(s.messages, msg) 226 | } 227 | s.queuedForFrontend[srcKey] = true 228 | } 229 | 230 | // broadcasts to all sockets 231 | func (s *Server) Broadcast() { 232 | 233 | var messages []*Message 234 | s.serverMu.Lock() 235 | messages = s.messages 236 | s.messages = nil 237 | s.queuedForFrontend = make(map[string]bool) 238 | s.serverMu.Unlock() 239 | 240 | //for latLng, value := range s.srcDestPacketCount { 241 | // fmt.Printf("%s: %d\n", latLng, value) 242 | //} 243 | 244 | if len(messages) < s.batchSize || len(s.sockets) < 1 { 245 | return 246 | } 247 | 248 | msg := BatchMessage{ 249 | Messages: messages, 250 | } 251 | 252 | // Hacky way to build a new list of open sockets, assuming if it fails to write, it's not open. 253 | openSockets := make([]*websocket.Conn, 0) 254 | for _, conn := range s.sockets { 255 | 256 | err := conn.WriteJSON(msg) 257 | if err != nil { 258 | log.Println("Error during message writing:", err) 259 | } else { 260 | openSockets = append(openSockets, conn) 261 | } 262 | } 263 | 264 | s.socketMu.Lock() 265 | s.sockets = openSockets 266 | s.socketMu.Unlock() 267 | 268 | s.serverMu.Lock() 269 | s.colorIndex++ 270 | s.serverMu.Unlock() 271 | } 272 | 273 | func (s *Server) cleanup(id string) { 274 | // TODO need to clean up 275 | } 276 | --------------------------------------------------------------------------------