├── docs ├── .gitignore ├── guides │ ├── circonus_after.png │ ├── minimal_htop.png │ ├── minimal_zoomed.jpg │ ├── circonus_before.png │ ├── minimal_goesrecv.png │ ├── minimal_overview.jpg │ ├── minimal_GOES16_FD_FC_20180505T223038Z_crop.jpg │ ├── minimal_GOES16_FD_FC_20180505T223038Z_full.jpg │ └── circonus.rst ├── files │ ├── 5_LRIT_Mission-data.pdf │ ├── 3_LRIT_Receiver-specs.pdf │ └── 4_LRIT_Transmitter-specs.pdf ├── images │ └── GOES15_FD_VS_20170821.gif ├── guides.rst ├── requirements.txt ├── commands.rst ├── index.rst ├── Makefile ├── installation.rst ├── resources.rst └── commands │ ├── goeslrit.rst │ └── goesproc.rst ├── .gitignore ├── test └── goesemwin │ ├── .gitignore │ ├── download.sh │ └── test.sh ├── src ├── lib │ ├── version.h │ ├── version-gen.h.in │ ├── packet_reader.cc │ ├── packet_writer.cc │ ├── timer.cc │ ├── timer.h │ ├── packet_reader.h │ ├── nanomsg_reader.h │ ├── packet_writer.h │ ├── nanomsg_writer.h │ ├── dir.h │ ├── file_reader.h │ ├── file_writer.h │ ├── unzip.cc │ ├── dir.cc │ ├── version.cc │ ├── file_reader.cc │ ├── CMakeLists.txt │ ├── file_writer.cc │ ├── nanomsg_writer.cc │ ├── nanomsg_reader.cc │ └── zip.h ├── goesproc │ ├── string.h │ ├── string.cc │ ├── gradient.cc │ ├── handler.h │ ├── filename.h │ ├── area.cc │ ├── handler_text.h │ ├── lrit_processor.h │ ├── handler_nws_image.h │ ├── handler_nws_text.h │ ├── handler_emwin.h │ ├── options.h │ ├── map_drawer.h │ ├── packet_processor.h │ ├── proj.h │ ├── handler_himawari8.h │ ├── file_writer.h │ ├── area.h │ ├── image.h │ ├── types.h │ ├── packet_processor.cc │ ├── handler_goesn.h │ ├── lrit_processor.cc │ ├── CMakeLists.txt │ ├── config.h │ ├── file_writer.cc │ ├── handler_nws_image.cc │ ├── handler_text.cc │ └── proj.cc ├── decoder │ ├── reader.cc │ ├── reader.h │ ├── derandomizer.h │ ├── reed_solomon.h │ ├── CMakeLists.txt │ ├── correlator.h │ ├── derandomizer.cc │ ├── packetizer.h │ ├── correlator.cc │ ├── viterbi.h │ ├── reed_solomon.cc │ ├── compute_sync_words.cc │ └── packetdump.cc ├── util │ ├── CMakeLists.txt │ ├── fs.h │ ├── time.h │ ├── fs.cc │ ├── string.h │ ├── error.h │ ├── time.cc │ └── string.cc ├── dcs │ ├── CMakeLists.txt │ ├── dcs.h │ └── dcsdump.cc ├── goesrecv │ ├── types.h │ ├── neon │ │ └── CMakeLists.txt │ ├── options.h │ ├── datagram_socket.h │ ├── packet_publisher.h │ ├── soft_bit_publisher.h │ ├── stats_publisher.h │ ├── sample_publisher.h │ ├── quantize.h │ ├── publisher.h │ ├── source.h │ ├── decoder.h │ ├── stats_publisher.cc │ ├── soft_bit_publisher.cc │ ├── rrc.h │ ├── packet_publisher.cc │ ├── agc.h │ ├── quantize.cc │ ├── costas.h │ ├── nanomsg_source.h │ ├── monitor.h │ ├── clock_recovery.h │ ├── sample_publisher.cc │ ├── rtlsdr_source.h │ ├── airspy_source.h │ ├── demodulator.h │ ├── publisher.cc │ ├── goesrecv.cc │ ├── datagram_socket.cc │ ├── queue.h │ ├── CMakeLists.txt │ ├── agc.cc │ ├── options.cc │ ├── config.h │ ├── source.cc │ ├── nanomsg_source.cc │ ├── airspy_source.cc │ └── decoder.cc ├── assembler │ ├── crc.h │ ├── CMakeLists.txt │ ├── assembler.h │ ├── packetinfo.cc │ ├── assembler.cc │ ├── vcdu.h │ ├── virtual_channel.h │ ├── transport_pdu.cc │ ├── transport_pdu.h │ ├── session_pdu.h │ └── crc.cc ├── lrit │ ├── json.h │ ├── CMakeLists.txt │ └── file.h ├── goeslrit │ ├── CMakeLists.txt │ └── options.h ├── goespackets │ ├── CMakeLists.txt │ ├── options.h │ ├── goespackets.cc │ └── options.cc └── goesemwin │ ├── options.h │ ├── CMakeLists.txt │ ├── emwin.h │ ├── emwin.cc │ ├── qbt.h │ ├── options.cc │ └── qbt.cc ├── docker └── ubuntu │ ├── run_compile.sh │ ├── run_cmake.sh │ └── Dockerfile ├── images └── GOES15_FD_VS_20170821.gif ├── share ├── wxstar │ ├── wxstar_goes16_lut.png │ ├── wxstar_goes16_ch02_curve.png │ └── README.txt └── ne │ └── README.txt ├── vendor └── json │ ├── CMakeLists.txt │ └── NOTICE ├── scripts ├── services │ ├── goesrecv.service │ ├── goesproc.service │ └── README.md ├── files │ └── raspberrypi.cmake ├── setup_raspbian.sh └── list_raspbian_urls.py ├── .gitmodules ├── .github └── workflows │ ├── build.yml │ └── docs.yml ├── LICENSE └── etc └── goesrecv.conf /docs/.gitignore: -------------------------------------------------------------------------------- 1 | /_build 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /build* 2 | /xcompile 3 | -------------------------------------------------------------------------------- /test/goesemwin/.gitignore: -------------------------------------------------------------------------------- 1 | /packets 2 | /gen-* 3 | -------------------------------------------------------------------------------- /src/lib/version.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | void version(int argc, char** argv); 4 | -------------------------------------------------------------------------------- /docker/ubuntu/run_compile.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | set -e 4 | 5 | cd /tmp/goestools 6 | make -j $(nproc) 7 | -------------------------------------------------------------------------------- /docs/guides/circonus_after.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/guides/circonus_after.png -------------------------------------------------------------------------------- /docs/guides/minimal_htop.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/guides/minimal_htop.png -------------------------------------------------------------------------------- /docs/guides/minimal_zoomed.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/guides/minimal_zoomed.jpg -------------------------------------------------------------------------------- /docs/guides/circonus_before.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/guides/circonus_before.png -------------------------------------------------------------------------------- /docs/guides/minimal_goesrecv.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/guides/minimal_goesrecv.png -------------------------------------------------------------------------------- /docs/guides/minimal_overview.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/guides/minimal_overview.jpg -------------------------------------------------------------------------------- /images/GOES15_FD_VS_20170821.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/images/GOES15_FD_VS_20170821.gif -------------------------------------------------------------------------------- /docs/files/5_LRIT_Mission-data.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/files/5_LRIT_Mission-data.pdf -------------------------------------------------------------------------------- /share/wxstar/wxstar_goes16_lut.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/share/wxstar/wxstar_goes16_lut.png -------------------------------------------------------------------------------- /docs/files/3_LRIT_Receiver-specs.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/files/3_LRIT_Receiver-specs.pdf -------------------------------------------------------------------------------- /docs/images/GOES15_FD_VS_20170821.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/images/GOES15_FD_VS_20170821.gif -------------------------------------------------------------------------------- /src/goesproc/string.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | std::string removeSuffix(const std::string& str); 6 | -------------------------------------------------------------------------------- /docker/ubuntu/run_cmake.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | set -e 4 | 5 | mkdir -p /tmp/goestools 6 | cd /tmp/goestools 7 | cmake /goestools 8 | -------------------------------------------------------------------------------- /docs/files/4_LRIT_Transmitter-specs.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/files/4_LRIT_Transmitter-specs.pdf -------------------------------------------------------------------------------- /share/wxstar/wxstar_goes16_ch02_curve.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/share/wxstar/wxstar_goes16_ch02_curve.png -------------------------------------------------------------------------------- /docs/guides.rst: -------------------------------------------------------------------------------- 1 | Guides 2 | ======== 3 | 4 | .. toctree:: 5 | :maxdepth: 1 6 | 7 | guides/minimal_receiver 8 | guides/circonus 9 | -------------------------------------------------------------------------------- /src/decoder/reader.cc: -------------------------------------------------------------------------------- 1 | #include "reader.h" 2 | 3 | namespace decoder { 4 | 5 | Reader::~Reader() { 6 | } 7 | 8 | } // namespace decoder 9 | -------------------------------------------------------------------------------- /src/lib/version-gen.h.in: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #define GIT_COMMIT_HASH "@GIT_COMMIT_HASH@" 4 | #define GIT_COMMIT_DATE "@GIT_COMMIT_DATE@" 5 | -------------------------------------------------------------------------------- /src/lib/packet_reader.cc: -------------------------------------------------------------------------------- 1 | #include "packet_reader.h" 2 | 3 | PacketReader::PacketReader() { 4 | } 5 | 6 | PacketReader::~PacketReader() { 7 | } 8 | -------------------------------------------------------------------------------- /src/lib/packet_writer.cc: -------------------------------------------------------------------------------- 1 | #include "packet_writer.h" 2 | 3 | PacketWriter::PacketWriter() { 4 | } 5 | 6 | PacketWriter::~PacketWriter() { 7 | } 8 | -------------------------------------------------------------------------------- /src/util/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_library(util fs.cc string.cc time.cc) 2 | target_include_directories(util PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/..) 3 | -------------------------------------------------------------------------------- /docs/requirements.txt: -------------------------------------------------------------------------------- 1 | sphinx 2 | sphinx-autobuild 3 | sphinx_rtd_theme 4 | pygments-github-lexers 5 | git+https://github.com/sphinx-contrib/googleanalytics@42b3df9 -------------------------------------------------------------------------------- /docs/guides/minimal_GOES16_FD_FC_20180505T223038Z_crop.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/guides/minimal_GOES16_FD_FC_20180505T223038Z_crop.jpg -------------------------------------------------------------------------------- /docs/guides/minimal_GOES16_FD_FC_20180505T223038Z_full.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietern/goestools/HEAD/docs/guides/minimal_GOES16_FD_FC_20180505T223038Z_full.jpg -------------------------------------------------------------------------------- /src/util/fs.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | namespace util { 6 | 7 | void mkdirp(const std::string& path); 8 | 9 | } // namespace util 10 | -------------------------------------------------------------------------------- /docs/commands.rst: -------------------------------------------------------------------------------- 1 | Commands 2 | ======== 3 | 4 | .. toctree:: 5 | :maxdepth: 1 6 | 7 | commands/goesrecv 8 | commands/goeslrit 9 | commands/goesproc 10 | -------------------------------------------------------------------------------- /share/wxstar/README.txt: -------------------------------------------------------------------------------- 1 | This contrast curve and false color LUT are made by Harry Dove-Robinson. 2 | Originals are at: https://github.com/hdoverobinson/wx-star_false-color. 3 | -------------------------------------------------------------------------------- /src/dcs/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_library(dcs dcs.cc) 2 | 3 | add_executable(dcsdump dcsdump.cc) 4 | add_sanitizers(dcsdump) 5 | target_link_libraries(dcsdump lrit dcs m stdc++) 6 | -------------------------------------------------------------------------------- /src/goesrecv/types.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "queue.h" 7 | 8 | typedef std::vector > Samples; 9 | -------------------------------------------------------------------------------- /src/goesrecv/neon/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | set(CMAKE_C_FLAGS "-std=c11 -mfpu=neon -O3") 2 | add_executable(neon_mathfun_test neon_mathfun_test.c) 3 | target_link_libraries(neon_mathfun_test m) 4 | -------------------------------------------------------------------------------- /test/goesemwin/download.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | mkdir -p packets 6 | curl https://s3.amazonaws.com/goestools/test/goesemwin-goes15-packets-1h.tgz | tar -C ./packets -zxf - 7 | -------------------------------------------------------------------------------- /src/assembler/crc.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | namespace assembler { 7 | 8 | uint16_t crc(const uint8_t* buf, size_t len); 9 | 10 | } // namespace assembler 11 | -------------------------------------------------------------------------------- /src/util/time.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | namespace util { 7 | 8 | std::string stringTime(); 9 | 10 | bool parseTime(const std::string& in, struct timespec* ts); 11 | 12 | } // namespace util 13 | -------------------------------------------------------------------------------- /src/goesproc/string.cc: -------------------------------------------------------------------------------- 1 | #include "string.h" 2 | 3 | std::string removeSuffix(const std::string& str) { 4 | auto pos = str.rfind('.'); 5 | if (pos != std::string::npos) { 6 | return str.substr(0, pos); 7 | } 8 | return str; 9 | } 10 | -------------------------------------------------------------------------------- /src/lib/timer.cc: -------------------------------------------------------------------------------- 1 | #include "timer.h" 2 | 3 | Timer::Timer() : start_(std::chrono::high_resolution_clock::now()) { 4 | } 5 | 6 | std::chrono::duration Timer::elapsed() const { 7 | return std::chrono::high_resolution_clock::now() - start_; 8 | } 9 | -------------------------------------------------------------------------------- /src/decoder/reader.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | namespace decoder { 6 | 7 | class Reader { 8 | public: 9 | virtual ~Reader(); 10 | 11 | virtual size_t read(void* buf, size_t count) = 0; 12 | }; 13 | 14 | } // namespace decoder 15 | -------------------------------------------------------------------------------- /vendor/json/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | project(nlohmann_json LANGUAGES CXX) 2 | add_library(nlohmann_json INTERFACE) 3 | target_include_directories( 4 | nlohmann_json 5 | INTERFACE 6 | $ 7 | $ 8 | ) 9 | -------------------------------------------------------------------------------- /src/lib/timer.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | class Timer { 6 | public: 7 | explicit Timer(); 8 | 9 | std::chrono::duration elapsed() const; 10 | 11 | protected: 12 | std::chrono::time_point start_; 13 | }; 14 | -------------------------------------------------------------------------------- /src/goesrecv/options.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | struct Options { 7 | Options(); 8 | 9 | std::string config; 10 | bool verbose; 11 | std::chrono::milliseconds interval; 12 | }; 13 | 14 | Options parseOptions(int argc, char** argv); 15 | -------------------------------------------------------------------------------- /src/lrit/json.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "file.h" 6 | #include "lrit.h" 7 | 8 | namespace lrit { 9 | 10 | nlohmann::json toJSON(const Buffer& b, const HeaderMap& m); 11 | nlohmann::json toJSON(const File& f); 12 | 13 | } // namespace lrit 14 | -------------------------------------------------------------------------------- /src/goeslrit/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_executable(goeslrit goeslrit.cc options.cc) 2 | add_sanitizers(goeslrit) 3 | install(TARGETS goeslrit COMPONENT goestools RUNTIME DESTINATION bin) 4 | target_link_libraries(goeslrit assembler nanomsg packet_reader util m stdc++) 5 | target_link_libraries(goeslrit version) 6 | -------------------------------------------------------------------------------- /test/goesemwin/test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -ex 4 | 5 | test_mode() { 6 | dir="./gen-$1" 7 | rm -rf "$dir" 8 | mkdir -p "$dir" 9 | goesemwin --out "$dir" --mode $1 ./packets/*.raw | grep -v Writing 10 | } 11 | 12 | test_mode raw 13 | test_mode qbt 14 | test_mode emwin 15 | -------------------------------------------------------------------------------- /src/assembler/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_executable(packetinfo packetinfo.cc) 2 | target_link_libraries(packetinfo m stdc++) 3 | 4 | add_library(assembler 5 | assembler.cc 6 | crc.cc 7 | session_pdu.cc 8 | transport_pdu.cc 9 | virtual_channel.cc 10 | ) 11 | target_link_libraries(assembler lrit aec sz stdc++) 12 | -------------------------------------------------------------------------------- /src/goespackets/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_executable(goespackets goespackets.cc options.cc) 2 | add_sanitizers(goespackets) 3 | install(TARGETS goespackets COMPONENT goestools RUNTIME DESTINATION bin) 4 | target_link_libraries(goespackets nanomsg packet_reader packet_writer util m stdc++) 5 | target_link_libraries(goespackets version) 6 | -------------------------------------------------------------------------------- /src/goesproc/gradient.cc: -------------------------------------------------------------------------------- 1 | #include "gradient.h" 2 | 3 | std::ostream& operator<<(std::ostream& os, const GradientPoint &p) { 4 | os << "Units: " << p.units << ", " << 5 | "RGB(" << p.rgb[0] << "," << p.rgb[1] << "," << p.rgb[2] << "), " << 6 | "HSV(" << p.hsv[0] << "," << p.hsv[1] << "," << p.hsv[2] << ")"; 7 | return os; 8 | } 9 | -------------------------------------------------------------------------------- /src/lib/packet_reader.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | // PacketReader is an abstract base class for things that read packets. 7 | class PacketReader { 8 | public: 9 | PacketReader(); 10 | virtual ~PacketReader(); 11 | 12 | virtual bool nextPacket(std::array& out) = 0; 13 | }; 14 | -------------------------------------------------------------------------------- /src/goesproc/handler.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "lrit/file.h" 6 | 7 | // Handler is a base class for anything that can handle LRIT files. 8 | // There are multiple image handlers, text handlers, etc. 9 | class Handler { 10 | public: 11 | virtual void handle(std::shared_ptr f) = 0; 12 | }; 13 | -------------------------------------------------------------------------------- /src/goesemwin/options.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | enum class Mode { RAW, QBT, EMWIN }; 7 | 8 | struct Options { 9 | std::string nanomsg; 10 | std::vector files; 11 | Mode mode = Mode::RAW; 12 | std::string out = "."; 13 | }; 14 | 15 | Options parseOptions(int argc, char** argv); 16 | -------------------------------------------------------------------------------- /vendor/json/NOTICE: -------------------------------------------------------------------------------- 1 | This directory contains the header file from the 3.2.0 release of 2 | https://github.com/nlohmann/json. Unlike the other directories this is 3 | not a submodule because of disk space concerns. 4 | 5 | Even a shallow repository checkout of this repository uses 250 MB of 6 | disk space due to large test assets and I find this a bit excessive. 7 | -------------------------------------------------------------------------------- /src/goesrecv/datagram_socket.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include 6 | 7 | class DatagramSocket { 8 | public: 9 | explicit DatagramSocket(const std::string& addr); 10 | ~DatagramSocket(); 11 | 12 | bool send(const std::string& payload); 13 | 14 | protected: 15 | int fd_; 16 | sockaddr_storage addr_; 17 | }; 18 | -------------------------------------------------------------------------------- /src/lib/nanomsg_reader.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "packet_reader.h" 6 | 7 | class NanomsgReader : public PacketReader { 8 | public: 9 | NanomsgReader(const std::string& uri); 10 | virtual ~NanomsgReader(); 11 | 12 | virtual bool nextPacket(std::array& out); 13 | 14 | protected: 15 | int fd_; 16 | }; 17 | -------------------------------------------------------------------------------- /src/lib/packet_writer.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | // PacketWriter is an abstract base class for things that write packets. 8 | class PacketWriter { 9 | public: 10 | PacketWriter(); 11 | virtual ~PacketWriter(); 12 | 13 | virtual void write(const std::array& in, time_t t) = 0; 14 | }; 15 | -------------------------------------------------------------------------------- /src/lrit/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_library(lrit lrit.cc file.cc json.cc) 2 | target_link_libraries(lrit util nlohmann_json) 3 | 4 | add_executable(lritdump lritdump.cc) 5 | add_sanitizers(lritdump) 6 | target_link_libraries(lritdump lrit m stdc++) 7 | 8 | add_executable(areadump areadump.cc) 9 | add_sanitizers(areadump) 10 | target_link_libraries(areadump lrit dir m stdc++) 11 | -------------------------------------------------------------------------------- /src/goesrecv/packet_publisher.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "publisher.h" 4 | 5 | class PacketPublisher : public Publisher { 6 | public: 7 | static std::unique_ptr create(const std::string& endpoint); 8 | 9 | explicit PacketPublisher(int fd); 10 | virtual ~PacketPublisher(); 11 | 12 | void publish(const std::array& packet); 13 | }; 14 | -------------------------------------------------------------------------------- /src/goesrecv/soft_bit_publisher.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "publisher.h" 4 | 5 | class SoftBitPublisher : public Publisher { 6 | public: 7 | static std::unique_ptr create(const std::string& endpoint); 8 | 9 | explicit SoftBitPublisher(int fd); 10 | virtual ~SoftBitPublisher(); 11 | 12 | void publish(const std::vector& bits); 13 | }; 14 | -------------------------------------------------------------------------------- /src/decoder/derandomizer.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | namespace decoder { 9 | 10 | class Derandomizer { 11 | public: 12 | Derandomizer(); 13 | 14 | void run(uint8_t* data, size_t len); 15 | 16 | protected: 17 | std::array table_; 18 | }; 19 | 20 | } // namespace decoder 21 | -------------------------------------------------------------------------------- /src/goesrecv/stats_publisher.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "publisher.h" 4 | #include "types.h" 5 | 6 | class StatsPublisher : public Publisher { 7 | public: 8 | static std::unique_ptr create( 9 | const std::vector& endpoints); 10 | 11 | explicit StatsPublisher(int fd); 12 | virtual ~StatsPublisher(); 13 | 14 | void publish(const std::string& str); 15 | }; 16 | -------------------------------------------------------------------------------- /src/lib/nanomsg_writer.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "packet_writer.h" 7 | 8 | class NanomsgWriter : public PacketWriter { 9 | public: 10 | NanomsgWriter(const std::vector& endpoints); 11 | virtual ~NanomsgWriter(); 12 | 13 | virtual void write(const std::array& in, time_t t); 14 | 15 | protected: 16 | int fd_; 17 | }; 18 | -------------------------------------------------------------------------------- /src/lib/dir.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | #include 8 | 9 | class Dir { 10 | public: 11 | Dir(const std::string& path); 12 | ~Dir(); 13 | 14 | // Returns files matching specified pattern. 15 | std::vector matchFiles(const std::string& pattern); 16 | 17 | protected: 18 | std::string path_; 19 | DIR* dir_; 20 | }; 21 | -------------------------------------------------------------------------------- /docker/ubuntu/Dockerfile: -------------------------------------------------------------------------------- 1 | ARG version 2 | FROM ubuntu:${version} 3 | 4 | ARG DEBIAN_FRONTEND=noninteractive 5 | ENV TZ=UTC 6 | 7 | RUN apt-get update && apt-get install -y \ 8 | build-essential \ 9 | cmake \ 10 | git-core \ 11 | libairspy-dev \ 12 | libopencv-dev \ 13 | libproj-dev \ 14 | librtlsdr-dev \ 15 | zlib1g-dev \ 16 | && rm -rf /var/lib/apt/lists/* 17 | 18 | COPY ./run_cmake.sh / 19 | COPY ./run_compile.sh / 20 | -------------------------------------------------------------------------------- /src/goesproc/filename.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "image.h" 4 | #include "types.h" 5 | 6 | struct FilenameBuilder { 7 | std::string dir; 8 | std::string filename; 9 | 10 | struct timespec time{0, 0}; 11 | AWIPS awips; 12 | Product product; 13 | Region region; 14 | Channel channel; 15 | 16 | std::string build( 17 | const std::string& pattern, 18 | const std::string extension = std::string()) const; 19 | }; 20 | -------------------------------------------------------------------------------- /src/goesrecv/sample_publisher.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "publisher.h" 4 | #include "types.h" 5 | 6 | class SamplePublisher : public Publisher { 7 | public: 8 | static std::unique_ptr create(const std::string& endpoint); 9 | 10 | explicit SamplePublisher(int fd); 11 | virtual ~SamplePublisher(); 12 | 13 | void publish(const Samples& samples); 14 | 15 | protected: 16 | std::vector > tmp_; 17 | }; 18 | -------------------------------------------------------------------------------- /src/lib/file_reader.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include "packet_reader.h" 8 | 9 | class FileReader : public PacketReader { 10 | public: 11 | FileReader(const std::vector& files); 12 | virtual ~FileReader(); 13 | 14 | virtual bool nextPacket(std::array& out); 15 | 16 | protected: 17 | std::vector files_; 18 | std::ifstream ifs_; 19 | }; 20 | -------------------------------------------------------------------------------- /src/goesproc/area.cc: -------------------------------------------------------------------------------- 1 | #include "area.h" 2 | 3 | std::ostream& operator<<(std::ostream& os, const Area& area) { 4 | int w = area.maxColumn - area.minColumn; 5 | int h = area.maxLine - area.minLine; 6 | os << "("; 7 | os << w << "x" << h; 8 | if (area.minColumn >= 0) { 9 | os << "+"; 10 | } 11 | os << area.minColumn; 12 | if (area.minLine >= 0) { 13 | os << "+"; 14 | } 15 | os << area.minLine; 16 | os << ")"; 17 | return os; 18 | } 19 | -------------------------------------------------------------------------------- /src/decoder/reed_solomon.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | extern "C" { 4 | #include 5 | } 6 | 7 | #include 8 | 9 | namespace decoder { 10 | 11 | class ReedSolomon { 12 | public: 13 | ReedSolomon(); 14 | ~ReedSolomon(); 15 | 16 | int run(const uint8_t* data, size_t len, uint8_t* dst); 17 | 18 | protected: 19 | std::array dualToConv_; 20 | std::array convToDual_; 21 | 22 | correct_reed_solomon* rs_; 23 | }; 24 | 25 | } // namespace decoder 26 | -------------------------------------------------------------------------------- /src/goesproc/handler_text.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "config.h" 4 | #include "file_writer.h" 5 | #include "handler.h" 6 | #include "types.h" 7 | 8 | class TextHandler : public Handler { 9 | public: 10 | explicit TextHandler( 11 | const Config::Handler& config, 12 | const std::shared_ptr& fileWriter); 13 | 14 | virtual void handle(std::shared_ptr f); 15 | 16 | protected: 17 | Config::Handler config_; 18 | std::shared_ptr fileWriter_; 19 | }; 20 | -------------------------------------------------------------------------------- /src/goeslrit/options.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | struct Options { 8 | std::string nanomsg; 9 | std::vector files; 10 | std::set vcids; 11 | bool dryrun = false; 12 | std::string out = "."; 13 | 14 | // File types to include 15 | bool images = false; 16 | bool messages = false; 17 | bool text = false; 18 | bool dcs = false; 19 | bool emwin = false; 20 | }; 21 | 22 | Options parseOptions(int argc, char** argv); 23 | -------------------------------------------------------------------------------- /src/goespackets/options.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | struct Options { 8 | std::string subscribe; 9 | std::vector publish; 10 | std::vector files; 11 | 12 | // Record packets stream 13 | bool record = false; 14 | std::string filename = "./packets-%FT%H:%M:00.raw"; 15 | 16 | // Filter these VCIDs (include everything if empty) 17 | std::set vcids; 18 | }; 19 | 20 | Options parseOptions(int argc, char** argv); 21 | -------------------------------------------------------------------------------- /src/lib/file_writer.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "packet_writer.h" 7 | 8 | class FileWriter : public PacketWriter { 9 | public: 10 | FileWriter(const std::string& pattern); 11 | virtual ~FileWriter(); 12 | 13 | virtual void write(const std::array& in, time_t t); 14 | 15 | protected: 16 | const std::string pattern_; 17 | 18 | std::string buildFilename(time_t t); 19 | 20 | std::string currentFilename_; 21 | std::ofstream of_; 22 | }; 23 | -------------------------------------------------------------------------------- /src/decoder/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_executable(compute_sync_words compute_sync_words.cc) 2 | add_sanitizers(compute_sync_words) 3 | target_link_libraries(compute_sync_words correct_static m stdc++) 4 | 5 | add_library(packetizer 6 | correlator.cc 7 | derandomizer.cc 8 | packetizer.cc 9 | reader.cc 10 | reed_solomon.cc 11 | ) 12 | target_link_libraries(packetizer correct_static) 13 | 14 | add_executable(packetdump packetdump.cc) 15 | add_sanitizers(packetdump) 16 | target_link_libraries(packetdump packetizer m stdc++) 17 | -------------------------------------------------------------------------------- /scripts/services/goesrecv.service: -------------------------------------------------------------------------------- 1 | # goesrecv service for systemd 2 | 3 | [Unit] 4 | Description=goesrecv reception chain for the GOES-R satellites 5 | Documentation=https://pietern.github.io/goestools/ 6 | Wants=network.target 7 | After=network.target 8 | 9 | [Service] 10 | # EnvironmentFile=/etc/default/goestools 11 | ExecStart=/usr/local/bin/goesrecv -i 10 -c /home/pi/goesrecv.conf 12 | StandardOutput=null 13 | Type=simple 14 | Restart=on-failure 15 | RestartSec=30 16 | Nice=-5 17 | 18 | [Install] 19 | WantedBy=default.target 20 | 21 | -------------------------------------------------------------------------------- /src/goesemwin/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_executable(goesemwin goesemwin.cc options.cc qbt.cc emwin.cc) 2 | add_sanitizers(goesemwin) 3 | install(TARGETS goesemwin COMPONENT goestools RUNTIME DESTINATION bin) 4 | target_link_libraries(goesemwin assembler) 5 | target_link_libraries(goesemwin m) 6 | target_link_libraries(goesemwin nanomsg) 7 | target_link_libraries(goesemwin packet_reader) 8 | target_link_libraries(goesemwin stdc++) 9 | target_link_libraries(goesemwin util) 10 | target_link_libraries(goesemwin version) 11 | target_link_libraries(goesemwin zip) 12 | -------------------------------------------------------------------------------- /src/goesproc/lrit_processor.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "handler.h" 7 | 8 | // Takes a list of paths to LRIT files and/or directories. 9 | // 10 | // This class sorts the files in chronological order and then feeds 11 | // them to the handlers. 12 | // 13 | class LRITProcessor { 14 | public: 15 | explicit LRITProcessor(std::vector > handlers); 16 | 17 | void run(int argc, char** argv); 18 | 19 | protected: 20 | std::vector > handlers_; 21 | }; 22 | -------------------------------------------------------------------------------- /src/goesproc/handler_nws_image.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "config.h" 4 | #include "file_writer.h" 5 | #include "handler.h" 6 | 7 | class NWSImageHandler : public Handler { 8 | public: 9 | explicit NWSImageHandler( 10 | const Config::Handler& config, 11 | const std::shared_ptr& fileWriter); 12 | 13 | virtual void handle(std::shared_ptr f); 14 | 15 | protected: 16 | std::string getBasename(const lrit::File& f) const; 17 | 18 | Config::Handler config_; 19 | std::shared_ptr fileWriter_; 20 | }; 21 | -------------------------------------------------------------------------------- /scripts/services/goesproc.service: -------------------------------------------------------------------------------- 1 | # Goesproc service for systemd 2 | 3 | [Unit] 4 | Description=goesproc decoding for the GOES-R satellites 5 | Documentation=https://pietern.github.io/goestools/ 6 | Wants=network.target 7 | After=network.target goesrecv.service 8 | 9 | [Service] 10 | WorkingDirectory=/home/pi/incoming 11 | ExecStart=/usr/local/bin/goesproc -c /home/pi/goesproc-goesr.conf -m packet --subscribe tcp://127.0.0.1:5004 12 | StandardOutput=null 13 | Type=simple 14 | Restart=on-failure 15 | RestartSec=30 16 | 17 | [Install] 18 | WantedBy=default.target 19 | 20 | -------------------------------------------------------------------------------- /src/goesproc/handler_nws_text.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "config.h" 4 | #include "file_writer.h" 5 | #include "handler.h" 6 | #include "types.h" 7 | 8 | class NWSTextHandler : public Handler { 9 | public: 10 | explicit NWSTextHandler( 11 | const Config::Handler& config, 12 | const std::shared_ptr& fileWriter); 13 | 14 | virtual void handle(std::shared_ptr f); 15 | 16 | protected: 17 | bool extractAWIPS(std::istream& is, struct AWIPS& out); 18 | 19 | Config::Handler config_; 20 | std::shared_ptr fileWriter_; 21 | }; 22 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "vendor/libaec"] 2 | path = vendor/libaec 3 | url = https://gitlab.dkrz.de/k202009/libaec.git 4 | [submodule "vendor/libcorrect"] 5 | path = vendor/libcorrect 6 | url = https://github.com/pietern/libcorrect.git 7 | [submodule "vendor/sanitizers-cmake"] 8 | path = vendor/sanitizers-cmake 9 | url = https://github.com/arsenm/sanitizers-cmake.git 10 | [submodule "vendor/tinytoml"] 11 | path = vendor/tinytoml 12 | url = https://github.com/mayah/tinytoml.git 13 | [submodule "vendor/nanomsg"] 14 | path = vendor/nanomsg 15 | url = https://github.com/nanomsg/nanomsg.git 16 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. goestools documentation master file, created by 2 | sphinx-quickstart on Sat Apr 14 13:59:03 2018. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | Welcome to goestools's documentation! 7 | ===================================== 8 | 9 | .. toctree:: 10 | :maxdepth: 2 11 | :caption: Contents: 12 | 13 | installation 14 | commands 15 | guides 16 | resources 17 | 18 | 19 | Indices and tables 20 | ================== 21 | 22 | * :ref:`genindex` 23 | * :ref:`modindex` 24 | * :ref:`search` 25 | -------------------------------------------------------------------------------- /src/goesrecv/quantize.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "soft_bit_publisher.h" 6 | #include "types.h" 7 | 8 | class Quantize { 9 | public: 10 | explicit Quantize(); 11 | 12 | void setSoftBitPublisher(std::unique_ptr softBitPublisher) { 13 | softBitPublisher_ = std::move(softBitPublisher); 14 | } 15 | 16 | void work( 17 | const std::shared_ptr > > >& qin, 18 | const std::shared_ptr > >& qout); 19 | 20 | protected: 21 | std::unique_ptr softBitPublisher_; 22 | }; 23 | -------------------------------------------------------------------------------- /src/assembler/assembler.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "assembler/virtual_channel.h" 6 | 7 | namespace assembler { 8 | 9 | // Assembler takes in a stream of VCDUs and dispatches 10 | // them to the appropriate virtual channels. 11 | class Assembler { 12 | public: 13 | explicit Assembler(); 14 | 15 | // For every packet processed, we may get back multiple completed 16 | // Session PDUs for further processing. 17 | std::vector> process(const VCDU& p); 18 | 19 | protected: 20 | std::map vcs_; 21 | }; 22 | 23 | } // namespace assembler 24 | -------------------------------------------------------------------------------- /src/lib/unzip.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include "zip.h" 6 | 7 | int unzip(std::string path) { 8 | Zip zip(std::unique_ptr(new std::ifstream(path))); 9 | std::cout << zip.fileName() << std::endl; 10 | std::ofstream of(zip.fileName()); 11 | auto data = zip.read(); 12 | of.write(data.data(), data.size()); 13 | return 0; 14 | } 15 | 16 | int main(int argc, char** argv) { 17 | int rv = 0; 18 | for (int i = 1; i < argc; i++) { 19 | rv = unzip(argv[i]); 20 | if (rv != 0) { 21 | return rv; 22 | } 23 | } 24 | return rv; 25 | } 26 | -------------------------------------------------------------------------------- /src/lib/dir.cc: -------------------------------------------------------------------------------- 1 | #include "dir.h" 2 | 3 | #include 4 | 5 | Dir::Dir(const std::string& path) 6 | : path_(path) { 7 | dir_ = opendir(path.c_str()); 8 | ASSERT(dir_ != nullptr); 9 | } 10 | 11 | Dir::~Dir() { 12 | closedir(dir_); 13 | } 14 | 15 | std::vector Dir::matchFiles(const std::string& pattern) { 16 | std::vector result; 17 | struct dirent *ent; 18 | while ((ent = readdir(dir_)) != NULL) { 19 | if (fnmatch(pattern.c_str(), ent->d_name, 0) == 0) { 20 | result.push_back(path_ + "/" + std::string(ent->d_name)); 21 | } 22 | } 23 | return result; 24 | } 25 | -------------------------------------------------------------------------------- /src/goesrecv/publisher.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | class Publisher { 8 | public: 9 | static int bind(const std::vector& endpoints); 10 | 11 | static int bind(const std::string& endpoint) { 12 | std::vector endpoints = { endpoint }; 13 | return bind(endpoints); 14 | } 15 | 16 | static std::unique_ptr create(const std::string& endpoint); 17 | 18 | explicit Publisher(int fd); 19 | virtual ~Publisher(); 20 | 21 | void setSendBuffer(int size); 22 | 23 | bool hasSubscribers(); 24 | 25 | protected: 26 | int fd_; 27 | }; 28 | -------------------------------------------------------------------------------- /src/assembler/packetinfo.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include 6 | 7 | #include "vcdu.h" 8 | 9 | int main(int argc, char** argv) { 10 | std::ifstream f(argv[1]); 11 | ASSERT(f.good()); 12 | 13 | std::array buf; 14 | for (;;) { 15 | f.read((char*) buf.data(), buf.size()); 16 | if (f.eof()) { 17 | break; 18 | } 19 | 20 | VCDU vcdu(buf); 21 | std::cout 22 | << "SCID: " << vcdu.getSCID() 23 | << ", VCID: " << vcdu.getVCID() 24 | << ", counter: " << vcdu.getCounter() 25 | << std::endl; 26 | } 27 | 28 | return 0; 29 | } 30 | -------------------------------------------------------------------------------- /src/lib/version.cc: -------------------------------------------------------------------------------- 1 | #include "version.h" 2 | #include "version-gen.h" 3 | 4 | #include 5 | 6 | void version(int argc, char** argv) { 7 | std::string argv0(argv[0]); 8 | auto pos = argv0.rfind('/'); 9 | if (pos != std::string::npos) { 10 | argv0 = argv0.substr(pos + 1); 11 | } 12 | 13 | std::cout 14 | << argv0 15 | << " -- " << GIT_COMMIT_HASH 16 | << " (" << GIT_COMMIT_DATE << ")" 17 | << std::endl 18 | << std::endl; 19 | std::cout 20 | << "Part of goestools (https://github.com/pietern/goestools)" 21 | << std::endl; 22 | std::cout 23 | << "Written by Pieter Noordhuis and contributors" 24 | << std::endl; 25 | } 26 | -------------------------------------------------------------------------------- /src/goesrecv/source.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "config.h" 6 | #include "sample_publisher.h" 7 | #include "types.h" 8 | 9 | // Pure virtual base class for every source of samples. 10 | class Source { 11 | public: 12 | static std::unique_ptr build( 13 | const std::string& type, 14 | Config& config); 15 | 16 | virtual ~Source(); 17 | 18 | // Sample rate is set in the configuration 19 | virtual uint32_t getSampleRate() const = 0; 20 | 21 | // Start producing samples 22 | virtual void start(const std::shared_ptr >& queue) = 0; 23 | 24 | // Stop producing samples 25 | virtual void stop() = 0; 26 | }; 27 | -------------------------------------------------------------------------------- /src/goesproc/handler_emwin.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "config.h" 4 | #include "file_writer.h" 5 | #include "handler.h" 6 | 7 | class EMWINHandler : public Handler { 8 | public: 9 | explicit EMWINHandler( 10 | const Config::Handler& config, 11 | const std::shared_ptr& fileWriter); 12 | 13 | virtual void handle(std::shared_ptr f); 14 | 15 | protected: 16 | bool extractTimeStamp( 17 | const std::string& text, 18 | struct timespec& ts) const; 19 | 20 | bool extractAWIPS( 21 | const std::string& text, 22 | struct AWIPS& out) const; 23 | 24 | Config::Handler config_; 25 | std::shared_ptr fileWriter_; 26 | }; 27 | -------------------------------------------------------------------------------- /src/assembler/assembler.cc: -------------------------------------------------------------------------------- 1 | #include "assembler.h" 2 | 3 | namespace assembler { 4 | 5 | Assembler::Assembler() { 6 | } 7 | 8 | std::vector> Assembler::process(const VCDU& vcdu) { 9 | std::vector> out; 10 | 11 | // Ignore fill packets 12 | auto vcid = vcdu.getVCID(); 13 | if (vcid == 63) { 14 | return out; 15 | } 16 | 17 | // Create virtual channel instance if it does not yet exist 18 | if (vcs_.find(vcid) == vcs_.end()) { 19 | vcs_.insert(std::make_pair(vcid, VirtualChannel(vcid))); 20 | } 21 | 22 | // Let virtual channel process VCDU 23 | auto it = vcs_.find(vcid); 24 | return it->second.process(vcdu); 25 | } 26 | 27 | } // namespace assembler 28 | -------------------------------------------------------------------------------- /src/util/fs.cc: -------------------------------------------------------------------------------- 1 | #include "fs.h" 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | namespace util { 9 | 10 | void mkdirp(const std::string& path) { 11 | size_t pos = 0; 12 | for (;; pos++) { 13 | pos = path.find('/', pos); 14 | if (pos == 0) { 15 | continue; 16 | } 17 | auto sub = path.substr(0, pos); 18 | constexpr auto mode = 19 | S_IRUSR | S_IWUSR | S_IXUSR | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH; 20 | auto rv = mkdir(sub.c_str(), mode); 21 | if (rv == -1 && errno != EEXIST) { 22 | perror("mkdir"); 23 | ASSERT(rv == 0); 24 | } 25 | if (pos == std::string::npos) { 26 | break; 27 | } 28 | } 29 | } 30 | 31 | } // namespace util 32 | -------------------------------------------------------------------------------- /src/goesproc/options.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | enum class ProcessMode { 7 | UNDEFINED, 8 | PACKET, 9 | LRIT, 10 | }; 11 | 12 | struct Options { 13 | // Path to configuration file 14 | std::string config; 15 | 16 | // What to process (stream of VCDU packets or LRIT files) 17 | ProcessMode mode; 18 | 19 | // Overwrite existing output files 20 | bool force = false; 21 | 22 | // Address of publisher to subscribe to (only relevant in packet mode) 23 | std::string subscribe; 24 | 25 | // Output directory 26 | std::string out = "."; 27 | 28 | // Paths specified as final argument(s) 29 | std::vector paths; 30 | }; 31 | 32 | Options parseOptions(int& argc, char**& argv); 33 | -------------------------------------------------------------------------------- /src/goesrecv/decoder.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include "decoder/packetizer.h" 8 | 9 | #include "config.h" 10 | #include "packet_publisher.h" 11 | #include "queue.h" 12 | #include "stats_publisher.h" 13 | 14 | class Decoder { 15 | public: 16 | explicit Decoder(std::shared_ptr > > queue); 17 | 18 | void initialize(Config& config); 19 | 20 | void start(); 21 | void stop(); 22 | 23 | protected: 24 | void publishStats(decoder::Packetizer::Details details); 25 | 26 | std::unique_ptr packetizer_; 27 | std::unique_ptr packetPublisher_; 28 | std::unique_ptr statsPublisher_; 29 | std::thread thread_; 30 | }; 31 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | SPHINXPROJ = goestools 8 | SOURCEDIR = . 9 | BUILDDIR = _build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | autobuild: 16 | sphinx-autobuild -z . -s 1 . _build/html/ 17 | 18 | .PHONY: help autobuild Makefile 19 | 20 | # Catch-all target: route all unknown targets to Sphinx using the new 21 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 22 | %: Makefile 23 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 24 | -------------------------------------------------------------------------------- /src/goesrecv/stats_publisher.cc: -------------------------------------------------------------------------------- 1 | #include "stats_publisher.h" 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | std::unique_ptr StatsPublisher::create( 9 | const std::vector& endpoints) { 10 | auto fd = Publisher::bind(endpoints); 11 | return std::make_unique(fd); 12 | } 13 | 14 | StatsPublisher::StatsPublisher(int fd) 15 | : Publisher(fd) { 16 | } 17 | 18 | StatsPublisher::~StatsPublisher() { 19 | } 20 | 21 | void StatsPublisher::publish(const std::string& str) { 22 | if (!hasSubscribers()) { 23 | return; 24 | } 25 | 26 | auto rv = nn_send(fd_, str.data(), str.size(), 0); 27 | if (rv < 0) { 28 | fprintf(stderr, "nn_send: %s\n", nn_strerror(nn_errno())); 29 | ASSERT(false); 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /src/goesemwin/emwin.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "qbt.h" 7 | 8 | namespace emwin { 9 | 10 | class File { 11 | public: 12 | File(std::vector packets) : packets_(packets) { 13 | if (packets.empty()) { 14 | throw std::runtime_error("no packets"); 15 | } 16 | } 17 | 18 | std::string filename() const { 19 | return packets_.front().filename(); 20 | } 21 | 22 | std::string extension() const; 23 | 24 | std::vector data() const; 25 | 26 | protected: 27 | std::vector packets_; 28 | }; 29 | 30 | class Assembler { 31 | public: 32 | std::unique_ptr process(qbt::Packet p); 33 | 34 | protected: 35 | std::unordered_map> pending_; 36 | }; 37 | 38 | } // namespace emwin 39 | -------------------------------------------------------------------------------- /src/goesproc/map_drawer.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include "config.h" 4 | #include "lrit/lrit.h" 5 | #include "proj.h" 6 | 7 | class MapDrawer { 8 | public: 9 | explicit MapDrawer( 10 | const Config::Handler* config, 11 | float longitude, 12 | lrit::ImageNavigationHeader inh); 13 | 14 | cv::Mat draw(cv::Mat& in); 15 | 16 | protected: 17 | void generatePoints( 18 | const Config::Map& map, 19 | std::vector>& out); 20 | 21 | void generatePoints( 22 | std::vector>& out, 23 | const nlohmann::json& poly); 24 | 25 | const Config::Handler* config_; 26 | Proj proj_; 27 | lrit::ImageNavigationHeader inh_; 28 | 29 | // Store one vector of line segments per map in the handler configuration. 30 | std::vector>> points_; 31 | }; 32 | -------------------------------------------------------------------------------- /src/goesproc/packet_processor.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "assembler/assembler.h" 7 | #include "lib/packet_reader.h" 8 | 9 | #include "handler.h" 10 | 11 | // Takes a list of files that store LRIT/HRIT VCDUs. 12 | // 13 | // This class assembles packets into files on the fly and 14 | // then feeds them to the handlers. 15 | // 16 | // To run on live data, you can pipe the output of the decoder into 17 | // goesproc and specify /dev/stdin as only file to process. 18 | // 19 | class PacketProcessor { 20 | public: 21 | explicit PacketProcessor(std::vector > handlers); 22 | 23 | void run(std::unique_ptr& reader, bool verbose); 24 | 25 | protected: 26 | std::vector > handlers_; 27 | assembler::Assembler assembler_; 28 | }; 29 | -------------------------------------------------------------------------------- /src/goesrecv/soft_bit_publisher.cc: -------------------------------------------------------------------------------- 1 | #include "soft_bit_publisher.h" 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | std::unique_ptr SoftBitPublisher::create(const std::string& endpoint) { 9 | auto fd = Publisher::bind(endpoint); 10 | return std::make_unique(fd); 11 | } 12 | 13 | SoftBitPublisher::SoftBitPublisher(int fd) 14 | : Publisher(fd) { 15 | } 16 | 17 | SoftBitPublisher::~SoftBitPublisher() { 18 | } 19 | 20 | void SoftBitPublisher::publish(const std::vector& bits) { 21 | if (!hasSubscribers()) { 22 | return; 23 | } 24 | 25 | auto rv = nn_send(fd_, bits.data(), bits.size() * sizeof(bits[0]), 0); 26 | if (rv < 0) { 27 | fprintf(stderr, "nn_send: %s\n", nn_strerror(nn_errno())); 28 | ASSERT(false); 29 | } 30 | } 31 | -------------------------------------------------------------------------------- /src/goesrecv/rrc.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "sample_publisher.h" 7 | #include "types.h" 8 | 9 | class RRC { 10 | public: 11 | static constexpr size_t NTAPS = 31; 12 | 13 | explicit RRC(int df, int sampleRate, int symbolRate); 14 | 15 | void setSamplePublisher(std::unique_ptr samplePublisher) { 16 | samplePublisher_ = std::move(samplePublisher); 17 | } 18 | 19 | void work( 20 | const std::shared_ptr >& qin, 21 | const std::shared_ptr >& qout); 22 | 23 | protected: 24 | void work( 25 | size_t nsamples, 26 | std::complex* fi, 27 | std::complex* fo); 28 | 29 | int decimation_; 30 | std::vector taps_; 31 | 32 | Samples tmp_; 33 | 34 | std::unique_ptr samplePublisher_; 35 | }; 36 | -------------------------------------------------------------------------------- /src/goesrecv/packet_publisher.cc: -------------------------------------------------------------------------------- 1 | #include "packet_publisher.h" 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | std::unique_ptr PacketPublisher::create(const std::string& endpoint) { 11 | auto fd = Publisher::bind(endpoint); 12 | return std::make_unique(fd); 13 | } 14 | 15 | PacketPublisher::PacketPublisher(int fd) 16 | : Publisher(fd) { 17 | } 18 | 19 | PacketPublisher::~PacketPublisher() { 20 | } 21 | 22 | void PacketPublisher::publish(const std::array& packet) { 23 | if (!hasSubscribers()) { 24 | return; 25 | } 26 | 27 | auto rv = nn_send(fd_, packet.data(), packet.size() * sizeof(packet[0]), 0); 28 | if (rv < 0) { 29 | fprintf(stderr, "nn_send: %s\n", nn_strerror(nn_errno())); 30 | ASSERT(false); 31 | } 32 | } 33 | -------------------------------------------------------------------------------- /src/decoder/correlator.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | namespace decoder { 9 | 10 | enum correlationType { 11 | LRIT_PHASE_000 = 0, 12 | LRIT_PHASE_180 = 1, 13 | HRIT_PHASE_000 = 2, 14 | HRIT_PHASE_180 = 3, 15 | }; 16 | 17 | // Number of bits used by sync word 18 | extern const unsigned encodedSyncWordBits; 19 | 20 | const char* correlationTypeToString(correlationType type); 21 | 22 | // Correlates bit stream with sync words. 23 | // This function is only used when there is no signal lock, so we can 24 | // afford to correlate with both LRIT and HRIT sync words. Doing this 25 | // means we don't need a run time flag for the type of stream because 26 | // we can detect which one we correlate best with. 27 | int correlate(uint8_t* data, size_t len, int* maxOut, correlationType* maxType); 28 | 29 | } // namespace decoder 30 | -------------------------------------------------------------------------------- /src/goesproc/proj.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #if PROJ_VERSION_MAJOR == 4 4 | #include 5 | #elif PROJ_VERSION_MAJOR >= 5 6 | #include 7 | #else 8 | #error "proj version >= 4 required" 9 | #endif 10 | 11 | #if PROJ_VERSION_MAJOR == 4 12 | // Forward compatibility. 13 | double proj_torad (double angle_in_degrees); 14 | #endif 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | class Proj { 22 | public: 23 | explicit Proj(const std::vector& args); 24 | 25 | explicit Proj(const std::map& args); 26 | 27 | ~Proj(); 28 | 29 | std::tuple fwd(double lon, double lat); 30 | 31 | std::tuple inv(double x, double y); 32 | 33 | protected: 34 | #if PROJ_VERSION_MAJOR == 4 35 | projPJ proj_; 36 | #elif PROJ_VERSION_MAJOR >= 5 37 | PJ *proj_; 38 | #endif 39 | }; 40 | -------------------------------------------------------------------------------- /scripts/files/raspberrypi.cmake: -------------------------------------------------------------------------------- 1 | # Define our target system 2 | SET(CMAKE_SYSTEM_NAME Linux) 3 | SET(CMAKE_SYSTEM_VERSION 1) 4 | SET(CMAKE_SYSTEM_PROCESSOR armv7l) 5 | 6 | # Assume this file is located next to the "tools" repository 7 | SET(RASPBERRYPI_ROOT ${CMAKE_CURRENT_LIST_DIR}) 8 | SET(RASPBERRYPI_SYSROOT ${RASPBERRYPI_ROOT}/sysroot) 9 | 10 | # Define the cross compiler locations 11 | SET(CMAKE_C_COMPILER arm-linux-gnueabihf-gcc-5) 12 | SET(CMAKE_CXX_COMPILER arm-linux-gnueabihf-g++-5) 13 | 14 | # Define the sysroot path for CMake 15 | SET(CMAKE_SYSROOT ${RASPBERRYPI_SYSROOT}) 16 | SET(CMAKE_FIND_ROOT_PATH ${RASPBERRYPI_SYSROOT}) 17 | 18 | # Use our definitions for compiler tools 19 | SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER) 20 | 21 | # Search for libraries and headers in the target directories only 22 | SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY) 23 | SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY) 24 | -------------------------------------------------------------------------------- /src/assembler/vcdu.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | // VCDU: Virtual Channel Data Unit. 7 | // This is the data packet AFTER applying Reed Solomon decode. 8 | class VCDU { 9 | using raw = std::array; 10 | 11 | public: 12 | // Allow implicit construction 13 | VCDU(const raw& data) 14 | : data_(data) { 15 | }; 16 | 17 | int getVersion() const { 18 | return (data_[0] & 0xc0) >> 6; 19 | } 20 | 21 | int getSCID() const { 22 | return (data_[0] & 0x3f) << 2 | (data_[1] & 0xc0) >> 6; 23 | } 24 | 25 | int getVCID() const { 26 | return (data_[1] & 0x3f); 27 | } 28 | 29 | int getCounter() const { 30 | return (data_[2] << 16) | (data_[3] << 8) | data_[4]; 31 | } 32 | 33 | const uint8_t* data() const { 34 | return &data_[6]; 35 | } 36 | 37 | size_t len() const { 38 | return data_.size() - 6; 39 | } 40 | 41 | protected: 42 | const raw& data_; 43 | }; 44 | -------------------------------------------------------------------------------- /src/goesrecv/agc.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "sample_publisher.h" 6 | #include "types.h" 7 | 8 | class AGC { 9 | public: 10 | explicit AGC(); 11 | 12 | void setMin(float min) { 13 | min_ = min; 14 | } 15 | 16 | void setMax(float max) { 17 | max_ = max; 18 | } 19 | 20 | void setSamplePublisher(std::unique_ptr samplePublisher) { 21 | samplePublisher_ = std::move(samplePublisher); 22 | } 23 | 24 | float getGain() const { 25 | return gain_; 26 | } 27 | 28 | void work( 29 | const std::shared_ptr >& qin, 30 | const std::shared_ptr >& qout); 31 | 32 | protected: 33 | void work( 34 | size_t nsamples, 35 | std::complex* fi, 36 | std::complex* fo); 37 | 38 | float min_; 39 | float max_; 40 | float gain_; 41 | float alpha_; 42 | 43 | std::unique_ptr samplePublisher_; 44 | }; 45 | -------------------------------------------------------------------------------- /src/dcs/dcs.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | namespace dcs { 11 | 12 | // Header at the beginning of an LRIT DCS file 13 | struct FileHeader { 14 | std::string name; 15 | uint32_t length; 16 | 17 | // Don't know what these hold. 18 | // First one looks ASCII and second one looks binary. 19 | std::string misc1; 20 | std::vector misc2; 21 | 22 | int readFrom(const char* buf, size_t len); 23 | }; 24 | 25 | // Header of every single DCS payload 26 | struct Header { 27 | uint64_t address; 28 | struct timespec time; 29 | char failure; 30 | int signalStrength; 31 | int frequencyOffset; 32 | char modulationIndex; 33 | char dataQuality; 34 | int receiveChannel; 35 | char spacecraft; 36 | char dataSourceCode[2]; 37 | int dataLength; 38 | 39 | int readFrom(const char* buf, size_t len); 40 | }; 41 | 42 | } // namespace dcs 43 | -------------------------------------------------------------------------------- /.github/workflows/build.yml: -------------------------------------------------------------------------------- 1 | name: Build 2 | 3 | on: 4 | push: 5 | branches: 6 | - main 7 | pull_request: 8 | branches: 9 | - main 10 | 11 | jobs: 12 | "Ubuntu": 13 | strategy: 14 | fail-fast: false 15 | matrix: 16 | version: 17 | - "16.04" 18 | - "18.04" 19 | - "20.04" 20 | - "22.04" 21 | - "24.04" 22 | runs-on: ubuntu-latest 23 | steps: 24 | - uses: actions/checkout@v3 25 | with: 26 | submodules: true 27 | - run: docker build --tag build:ubuntu-${{ matrix.version }} --build-arg version=${{ matrix.version }} docker/ubuntu 28 | name: "docker build" 29 | - run: docker run --detach --interactive --name build --volume $PWD:/goestools build:ubuntu-${{ matrix.version }} 30 | name: "docker run" 31 | - run: docker exec -t build /run_cmake.sh 32 | name: "cmake" 33 | - run: docker exec -t build /run_compile.sh 34 | name: "make" 35 | -------------------------------------------------------------------------------- /scripts/services/README.md: -------------------------------------------------------------------------------- 1 | # Sample services for Raspberry Pi 2 | 3 | Those two service files can be installed on a Raspberry Pi and will automatically start/restart both goesrecv and goesproc at boot time. 4 | 5 | By default they will use configuration files in /home/pi and save incoming data in /home/pi/incoming, but you can of course adjust those before installing the services. 6 | 7 | ## How to install 8 | 9 | After editing them to adjust paths if necessary, copy both service files to `/etc/systemd/system`. Make sure the configuration files exist, and the `incoming` directory is created at the location referenced in the service description file. 10 | 11 | You should then test if the service starts properly by using `sudo systemctl start goesrecv.service` and `sudo systemctl start goesproc.service`. 12 | 13 | If all goes well, then you can simply enable the services so that they run at boot time: 14 | 15 | ` 16 | sudo systemctl enable goesrecv.service 17 | sudo systemctl enable goesproc.service 18 | ` 19 | -------------------------------------------------------------------------------- /src/goesrecv/quantize.cc: -------------------------------------------------------------------------------- 1 | #include "quantize.h" 2 | 3 | Quantize::Quantize() { 4 | } 5 | 6 | void Quantize::work( 7 | const std::shared_ptr > > >& qin, 8 | const std::shared_ptr > >& qout) { 9 | auto input = qin->popForRead(); 10 | if (!input) { 11 | qout->close(); 12 | return; 13 | } 14 | 15 | // Clear output so we can use push_back. 16 | // It will retain the associated memory allocation. 17 | auto output = qout->popForWrite(); 18 | output->clear(); 19 | 20 | auto& rinput = *input; 21 | auto& routput = *output; 22 | auto nsamples = input->size(); 23 | for (size_t i = 0; i < nsamples; i++) { 24 | int8_t v = rinput[i].real() * 127.0f; 25 | routput.push_back(v); 26 | } 27 | 28 | // Return input buffer 29 | qin->pushRead(std::move(input)); 30 | 31 | // Publish output if applicable 32 | if (softBitPublisher_) { 33 | softBitPublisher_->publish(*output); 34 | } 35 | 36 | // Return output buffer 37 | qout->pushWrite(std::move(output)); 38 | } 39 | -------------------------------------------------------------------------------- /src/lib/file_reader.cc: -------------------------------------------------------------------------------- 1 | #include "file_reader.h" 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | FileReader::FileReader(const std::vector& files) 8 | : files_(files) { 9 | } 10 | 11 | FileReader::~FileReader() { 12 | } 13 | 14 | bool FileReader::nextPacket(std::array& out) { 15 | for (;;) { 16 | // Make sure the ifstream is OK 17 | if (!ifs_.good() || ifs_.eof()) { 18 | if (files_.size() == 0) { 19 | return false; 20 | } 21 | 22 | // Open next file 23 | std::cout << "Reading: " << files_.front() << std::endl; 24 | ifs_.close(); 25 | ifs_.open(files_.front()); 26 | if (!ifs_.good()) { 27 | std::stringstream ss; 28 | ss << "Unable to open " << files_.front(); 29 | throw std::runtime_error(ss.str()); 30 | } 31 | files_.erase(files_.begin()); 32 | } 33 | 34 | ifs_.read((char*) out.data(), out.size()); 35 | if (ifs_.eof()) { 36 | continue; 37 | } 38 | 39 | return true; 40 | } 41 | } 42 | -------------------------------------------------------------------------------- /src/goesrecv/costas.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "sample_publisher.h" 6 | #include "types.h" 7 | 8 | class Costas { 9 | public: 10 | explicit Costas(); 11 | 12 | // Set maximum frequency deviation in radians per sample. 13 | void setMaxDeviation(float maxDeviation) { 14 | maxDeviation_ = maxDeviation; 15 | } 16 | 17 | void setSamplePublisher(std::unique_ptr samplePublisher) { 18 | samplePublisher_ = std::move(samplePublisher); 19 | } 20 | 21 | // Returns frequency in radians per sample. 22 | float getFrequency() const { 23 | return freq_; 24 | } 25 | 26 | void work( 27 | const std::shared_ptr >& qin, 28 | const std::shared_ptr >& qout); 29 | 30 | protected: 31 | void work( 32 | size_t nsamples, 33 | std::complex* fi, 34 | std::complex* fo); 35 | 36 | float phase_; 37 | float freq_; 38 | float alpha_; 39 | float beta_; 40 | float maxDeviation_; 41 | 42 | std::unique_ptr samplePublisher_; 43 | }; 44 | -------------------------------------------------------------------------------- /src/goesrecv/nanomsg_source.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include "source.h" 8 | 9 | class Nanomsg : public Source { 10 | public: 11 | static std::unique_ptr open(const Config& config); 12 | 13 | explicit Nanomsg(int fd); 14 | ~Nanomsg(); 15 | 16 | void setSamplePublisher(std::unique_ptr samplePublisher) { 17 | samplePublisher_ = std::move(samplePublisher); 18 | } 19 | 20 | void setSampleRate(uint32_t rate); 21 | 22 | virtual uint32_t getSampleRate() const override; 23 | 24 | virtual void start(const std::shared_ptr >& queue) override; 25 | 26 | virtual void stop() override; 27 | 28 | protected: 29 | void loop(); 30 | 31 | int fd_; 32 | std::thread thread_; 33 | 34 | // Not used by this source but part of base class interface 35 | uint32_t sampleRate_; 36 | 37 | // Set on start; cleared on stop 38 | std::shared_ptr > queue_; 39 | 40 | // Optional publisher for samples 41 | std::unique_ptr samplePublisher_; 42 | }; 43 | -------------------------------------------------------------------------------- /share/ne/README.txt: -------------------------------------------------------------------------------- 1 | These files are the GeoJSON equivalents from the files available from 2 | Natural Earth. Originals from https://www.naturalearthdata.com/. 3 | 4 | The files here are medium scale (1:50m). I think it's a good trade off 5 | between file size and visual appeal. While the large scale data is 6 | finer grained, the medium scale data is sufficient for the products 7 | transmitter over LRIT/HRIT. 8 | 9 | To generate these files, install Node.js, and subsequently the 10 | `shapefile` package (https://www.npmjs.com/package/shapefile). This 11 | package ships with the `shp2json` tool that converts a Shapefile (SHP) 12 | to the GeoJSON format. 13 | 14 | Using `ogr2ogr` instead of `shp2json' will allow conversion of 15 | a Shapefile (SHP) from a points based geometry like most GIS 16 | systems, to a polygon based geometry shape that is processed 17 | by goestools with this command: 18 | 19 | ``` 20 | ogr2ogr -f "GeoJSON" -dialect SQLite -sql "select ST_Buffer(geometry, 0.01) from inputfile" output.json inputfile.shp 21 | ``` 22 | 23 | See https://gdal.org/ for installation instructions for `ogr2ogr`. 24 | -------------------------------------------------------------------------------- /src/goesproc/handler_himawari8.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "config.h" 6 | #include "file_writer.h" 7 | #include "handler.h" 8 | #include "image.h" 9 | #include "types.h" 10 | 11 | class Himawari8ImageHandler : public Handler { 12 | public: 13 | explicit Himawari8ImageHandler( 14 | const Config::Handler& config, 15 | const std::shared_ptr& fileWriter); 16 | 17 | virtual void handle(std::shared_ptr f); 18 | 19 | protected: 20 | std::string getBasename(const lrit::File& f) const; 21 | struct timespec getTime(const lrit::File& f) const; 22 | 23 | void overlayMaps(const lrit::File& f, cv::Mat& mat); 24 | 25 | Config::Handler config_; 26 | std::shared_ptr fileWriter_; 27 | 28 | using SegmentVector = std::vector>; 29 | 30 | // Maintain a map of region and channel to list of segments. 31 | // This assumes that two images for the same region and channel are 32 | // never sent concurrently, but always in order. 33 | std::unordered_map< 34 | SegmentKey, 35 | SegmentVector, 36 | SegmentKeyHash> segments_; 37 | }; 38 | -------------------------------------------------------------------------------- /src/goesrecv/monitor.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | #include "config.h" 9 | #include "datagram_socket.h" 10 | 11 | class Monitor { 12 | public: 13 | explicit Monitor(bool verbose, std::chrono::milliseconds interval); 14 | ~Monitor(); 15 | 16 | void initialize(Config& config); 17 | 18 | void start(); 19 | void stop(); 20 | 21 | protected: 22 | struct Stats { 23 | // Demodulator stats 24 | std::vector gain; 25 | std::vector frequency; 26 | std::vector omega; 27 | 28 | // Decoder stats 29 | std::vector viterbiErrors; 30 | std::vector reedSolomonErrors; 31 | int totalOK = 0; 32 | int totalDropped = 0; 33 | }; 34 | 35 | Stats stats_; 36 | 37 | void loop(); 38 | void process(const std::string& json); 39 | void print(const Stats& stats); 40 | 41 | const bool verbose_; 42 | const std::chrono::milliseconds interval_; 43 | 44 | int demodulatorFd_; 45 | int decoderFd_; 46 | std::unique_ptr statsd_; 47 | 48 | std::atomic stop_; 49 | std::thread thread_; 50 | }; 51 | -------------------------------------------------------------------------------- /src/assembler/virtual_channel.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include "session_pdu.h" 8 | #include "transport_pdu.h" 9 | #include "vcdu.h" 10 | 11 | namespace assembler { 12 | 13 | class VirtualChannel { 14 | public: 15 | explicit VirtualChannel(int id); 16 | 17 | // For every packet processed, we may get back multiple completed 18 | // Session PDUs for further processing. 19 | std::vector> process(const VCDU& p); 20 | 21 | protected: 22 | void process( 23 | std::unique_ptr tpdu, 24 | std::vector>& out); 25 | 26 | void finish( 27 | std::unique_ptr spdu, 28 | std::vector>& out); 29 | 30 | int id_; 31 | int n_; 32 | 33 | // Incomplete Transport Protocol Data Unit. 34 | std::unique_ptr tpdu_; 35 | 36 | // Sequence number by APID. Used to detect drops. 37 | std::map apidSeq_; 38 | 39 | // Incomplete Session Protocol Data Unit per APID. 40 | std::map> apidSessionPDU_; 41 | }; 42 | 43 | } // namespace assembler 44 | -------------------------------------------------------------------------------- /src/goesproc/file_writer.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | #include 8 | 9 | #include "lib/timer.h" 10 | #include "lrit/file.h" 11 | 12 | // FileWriter writes files to disk. 13 | // This is where overwrite logic and logging is handled. 14 | class FileWriter { 15 | public: 16 | explicit FileWriter(const std::string& prefix); 17 | ~FileWriter(); 18 | 19 | void setForce(bool force) { 20 | force_ = force; 21 | } 22 | 23 | void write( 24 | const std::string& path, 25 | const cv::Mat& mat, 26 | const Timer* t = nullptr); 27 | 28 | void write( 29 | const std::string& path, 30 | const std::vector& data, 31 | const Timer* t = nullptr); 32 | 33 | void write( 34 | const std::string& path, 35 | const nlohmann::json& json, 36 | const Timer* t = nullptr); 37 | 38 | void writeHeader( 39 | const lrit::File& file, 40 | const std::string& path); 41 | 42 | protected: 43 | bool tryWrite(const std::string& path); 44 | 45 | std::string buildPath(const std::string& path); 46 | 47 | void logTime(const Timer* t); 48 | 49 | const std::string prefix_; 50 | bool force_; 51 | }; 52 | -------------------------------------------------------------------------------- /src/goesproc/area.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | struct Area { 7 | int minColumn = 0; 8 | int maxColumn = 0; 9 | int minLine = 0; 10 | int maxLine = 0; 11 | 12 | int width() const { 13 | return maxColumn - minColumn; 14 | } 15 | 16 | int height() const { 17 | return maxLine - minLine; 18 | } 19 | 20 | bool empty() const { 21 | return width() == 0 && height() == 0; 22 | } 23 | 24 | Area getIntersection(const Area& other) const { 25 | Area result; 26 | result.minColumn = std::max(minColumn, other.minColumn); 27 | result.maxColumn = std::min(maxColumn, other.maxColumn); 28 | result.minLine = std::max(minLine, other.minLine); 29 | result.maxLine = std::min(maxLine, other.maxLine); 30 | return result; 31 | } 32 | 33 | Area getUnion(const Area& other) const { 34 | Area result; 35 | result.minColumn = std::min(minColumn, other.minColumn); 36 | result.maxColumn = std::max(maxColumn, other.maxColumn); 37 | result.minLine = std::min(minLine, other.minLine); 38 | result.maxLine = std::max(maxLine, other.maxLine); 39 | return result; 40 | } 41 | }; 42 | 43 | std::ostream& operator<<(std::ostream& os, const Area& area); 44 | -------------------------------------------------------------------------------- /src/assembler/transport_pdu.cc: -------------------------------------------------------------------------------- 1 | #include "transport_pdu.h" 2 | 3 | #include 4 | 5 | #include "crc.h" 6 | 7 | namespace assembler { 8 | 9 | size_t TransportPDU::read(const uint8_t* buf, size_t len) { 10 | size_t nread = 0; 11 | 12 | // Read remaining header 13 | if (header.size() < headerBytes) { 14 | auto remaining = headerBytes - header.size(); 15 | auto hpos = header.size(); 16 | auto hread = std::min(remaining, (len-nread)); 17 | header.resize(hpos + hread); 18 | memcpy(&header[hpos], &buf[nread], hread); 19 | nread += hread; 20 | } 21 | 22 | // Read remaining user data (if there is more data) 23 | if (nread < len) { 24 | if (data.size() < length()) { 25 | auto remaining = length() - data.size(); 26 | auto dpos = data.size(); 27 | auto dread = std::min(remaining, (len-nread)); 28 | data.resize(dpos + dread); 29 | memcpy(&data[dpos], &buf[nread], dread); 30 | nread += dread; 31 | } 32 | } 33 | 34 | return nread; 35 | } 36 | 37 | bool TransportPDU::verifyCRC() { 38 | if (length() >= 2) { 39 | return (::assembler::crc(&data[0], length() - 2) == this->crc()); 40 | } else { 41 | return false; 42 | } 43 | } 44 | 45 | } // namespace assembler 46 | -------------------------------------------------------------------------------- /src/decoder/derandomizer.cc: -------------------------------------------------------------------------------- 1 | #include "derandomizer.h" 2 | 3 | #include 4 | 5 | namespace decoder { 6 | 7 | Derandomizer::Derandomizer() { 8 | // Build de-randomization table (per CCSDS standard). 9 | // The pseudo random sequence is generated by the polynomial: 10 | // 11 | // h(x) = x^8 + x^7 + x^5 + x^3 + 1. 12 | // 13 | // The LRIT documentation refers to CCSDS 101.0-B-6-S (October 2002). 14 | // Newer versions of the CCSDS Blue Book series contain identical 15 | // guidelines for de-randomization. 16 | // 17 | // You can find a copy of the referred blue book here: 18 | // https://public.ccsds.org/Pubs/101x0b6s.pdf 19 | // 20 | uint8_t lfsr = 0xff; 21 | uint8_t bit; 22 | for (unsigned i = 0; i < table_.size(); i++) { 23 | table_[i] = 0; 24 | for (unsigned j = 0; j < 8; j++) { 25 | table_[i] <<= 1; 26 | table_[i] |= lfsr & 0x1; 27 | bit = ((lfsr >> 7) ^ (lfsr >> 5) ^ (lfsr >> 3) ^ (lfsr >> 0)) & 0x1; 28 | lfsr = (lfsr >> 1) | (bit << 7); 29 | } 30 | } 31 | } 32 | 33 | void Derandomizer::run(uint8_t* data, size_t len) { 34 | ASSERT(len == table_.size()); 35 | for (unsigned i = 0; i < table_.size(); i++) { 36 | data[i] ^= table_[i]; 37 | } 38 | } 39 | 40 | } // namespace decoder 41 | -------------------------------------------------------------------------------- /src/goesrecv/clock_recovery.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "sample_publisher.h" 6 | #include "types.h" 7 | 8 | class ClockRecovery { 9 | public: 10 | explicit ClockRecovery(uint32_t sampleRate, uint32_t symbolRate); 11 | 12 | void setSamplePublisher(std::unique_ptr samplePublisher) { 13 | samplePublisher_ = std::move(samplePublisher); 14 | } 15 | 16 | // See http://www.trondeau.com/blog/2011/8/13/control-loop-gain-values.html 17 | void setLoopBandwidth(float bw); 18 | 19 | // Returns number of samples per symbol. 20 | float getOmega() const { 21 | return omega_; 22 | } 23 | 24 | void work( 25 | const std::shared_ptr >& qin, 26 | const std::shared_ptr >& qout); 27 | 28 | protected: 29 | float omega_; 30 | float omegaMin_; 31 | float omegaMax_; 32 | float omegaGain_; 33 | float mu_; 34 | float muGain_; 35 | 36 | // Past samples 37 | std::complex p0t_; 38 | std::complex p1t_; 39 | std::complex p2t_; 40 | 41 | // Past associated quadrants 42 | std::complex c0t_; 43 | std::complex c1t_; 44 | std::complex c2t_; 45 | 46 | Samples tmp_; 47 | 48 | std::unique_ptr samplePublisher_; 49 | }; 50 | -------------------------------------------------------------------------------- /scripts/setup_raspbian.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Setup environment for cross-compilation for Raspbian. 4 | # 5 | 6 | set -e 7 | 8 | target_dir="xcompile/raspbian" 9 | mkdir -p "${target_dir}" 10 | cp -f "$(dirname "$0")/files/raspberrypi.cmake" "${target_dir}" 11 | 12 | install_if_needed() { 13 | if ! dpkg -l "$1" | grep -q '^ii'; then 14 | sudo apt-get install -y "$1" 15 | fi 16 | } 17 | 18 | # These resolve to 5.5.0-12ubuntu1cross1 on Ubuntu 18.04 19 | install_if_needed gcc-5-arm-linux-gnueabihf 20 | install_if_needed g++-5-arm-linux-gnueabihf 21 | 22 | urls() { 23 | scripts/list_raspbian_urls.py \ 24 | librtlsdr-dev \ 25 | libairspy-dev \ 26 | libusb-1.0-0-dev \ 27 | libudev1 \ 28 | zlib1g-dev \ 29 | libopencv-dev \ 30 | libopencv-highgui-dev \ 31 | libproj-dev \ 32 | gcc-5 \ 33 | g++-5 34 | } 35 | 36 | tmp="$target_dir/tmp" 37 | mkdir -p "$tmp" 38 | 39 | # Download and extract packages of interest 40 | for url in $(urls); do 41 | deb=$(basename "${url}") 42 | if [ ! -f "${tmp}/${deb}" ]; then 43 | echo "Downloading ${url}..." 44 | ( cd "$tmp" && curl -LOs "${url}" ) 45 | fi 46 | echo "Extracting ${deb}..." 47 | dpkg-deb -x "${tmp}/${deb}" "${target_dir}/sysroot" 48 | done 49 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2017, Pieter Noordhuis 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | 7 | * Redistributions of source code must retain the above copyright notice, this 8 | list of conditions and the following disclaimer. 9 | 10 | * Redistributions in binary form must reproduce the above copyright notice, 11 | this list of conditions and the following disclaimer in the documentation 12 | and/or other materials provided with the distribution. 13 | 14 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 15 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 17 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 18 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 20 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 21 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 22 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 23 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 24 | -------------------------------------------------------------------------------- /src/goesproc/image.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | #include "lrit/file.h" 9 | 10 | #include "area.h" 11 | 12 | class Image { 13 | public: 14 | static std::unique_ptr createFromFile( 15 | std::shared_ptr f); 16 | 17 | static std::unique_ptr createFromFiles( 18 | std::vector > fs); 19 | 20 | static std::unique_ptr generateFalseColor( 21 | const std::unique_ptr& i0, 22 | const std::unique_ptr& i1, 23 | cv::Mat lut); 24 | 25 | explicit Image(cv::Mat m, const Area& area); 26 | 27 | void fillSides(); 28 | 29 | void remap(const cv::Mat& mat); 30 | 31 | void save(const std::string& path) const; 32 | 33 | cv::Mat getRawImage() const; 34 | cv::Mat getRawImage(const Area& roi) const; 35 | 36 | cv::Mat getScaledImage(bool shrink) const; 37 | cv::Mat getScaledImage(const Area& roi, bool shrink) const; 38 | 39 | protected: 40 | cv::Mat m_; 41 | 42 | // Measured relative to the offset in the ImageNavigationHeader 43 | Area area_; 44 | 45 | // Relative scaling of columns and lines. 46 | // This is applicable only for the GOES-N series. 47 | uint32_t columnScaling_; 48 | uint32_t lineScaling_; 49 | 50 | private: 51 | cv::Size scaleSize(cv::Size s, bool shrink) const; 52 | }; 53 | -------------------------------------------------------------------------------- /src/lib/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | add_library(dir dir.cc) 2 | add_library(zip zip.cc) 3 | add_library(timer timer.cc) 4 | target_link_libraries(zip z) 5 | add_library(packet_reader packet_reader.cc nanomsg_reader.cc file_reader.cc) 6 | target_link_libraries(packet_reader nanomsg) 7 | add_library(packet_writer packet_writer.cc nanomsg_writer.cc file_writer.cc) 8 | target_link_libraries(packet_writer nanomsg) 9 | add_executable(unzip unzip.cc) 10 | target_link_libraries(unzip zip m stdc++) 11 | 12 | # Get the latest abbreviated commit hash of the working branch 13 | if (NOT GIT_COMMIT_HASH) 14 | execute_process( 15 | COMMAND git log -1 --format=%h 16 | WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} 17 | OUTPUT_VARIABLE GIT_COMMIT_HASH 18 | OUTPUT_STRIP_TRAILING_WHITESPACE 19 | ) 20 | endif() 21 | 22 | # Get the latest commit date of the working branch 23 | if (NOT GIT_COMMIT_DATE) 24 | execute_process( 25 | COMMAND git log -1 --format=%cD 26 | WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} 27 | OUTPUT_VARIABLE GIT_COMMIT_DATE 28 | OUTPUT_STRIP_TRAILING_WHITESPACE 29 | ) 30 | endif() 31 | 32 | # Generate header file with string macros 33 | configure_file( 34 | ${CMAKE_CURRENT_SOURCE_DIR}/version-gen.h.in 35 | ${CMAKE_BINARY_DIR}/include/version-gen.h 36 | ) 37 | 38 | add_library(version version.cc) 39 | target_include_directories(version PRIVATE ${CMAKE_BINARY_DIR}/include) 40 | -------------------------------------------------------------------------------- /src/lrit/file.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | #include "lrit.h" 10 | 11 | namespace lrit { 12 | 13 | class File { 14 | public: 15 | explicit File(const std::string& file); 16 | 17 | explicit File(const std::vector& buf); 18 | 19 | const std::string& getName() const { 20 | return file_; 21 | } 22 | 23 | const std::vector& getHeaderBuffer() const { 24 | return header_; 25 | } 26 | 27 | const HeaderMap& getHeaderMap() const { 28 | return m_; 29 | } 30 | 31 | template 32 | bool hasHeader() const { 33 | return lrit::hasHeader(m_); 34 | } 35 | 36 | template 37 | H getHeader() const { 38 | return lrit::getHeader(header_, m_); 39 | } 40 | 41 | std::string getTime() const; 42 | 43 | std::unique_ptr getData() const; 44 | 45 | std::vector read() const; 46 | 47 | protected: 48 | std::unique_ptr getDataFromFile() const; 49 | std::unique_ptr getDataFromBuffer() const; 50 | 51 | // If LRIT file is on disk 52 | std::string file_; 53 | 54 | // If LRIT file is in memory 55 | std::vector buf_; 56 | 57 | std::vector header_; 58 | HeaderMap m_; 59 | PrimaryHeader ph_; 60 | }; 61 | 62 | } // namespace lrit 63 | -------------------------------------------------------------------------------- /src/goesrecv/sample_publisher.cc: -------------------------------------------------------------------------------- 1 | #include "sample_publisher.h" 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | std::unique_ptr SamplePublisher::create(const std::string& endpoint) { 11 | auto fd = Publisher::bind(endpoint); 12 | return std::make_unique(fd); 13 | } 14 | 15 | SamplePublisher::SamplePublisher(int fd) 16 | : Publisher(fd) { 17 | } 18 | 19 | SamplePublisher::~SamplePublisher() { 20 | } 21 | 22 | void SamplePublisher::publish(const Samples& samples) { 23 | if (!hasSubscribers()) { 24 | return; 25 | } 26 | 27 | // Scale samples to 8 bit. 28 | // Otherwise it is impossible to stream 3M complex samples/second from a RPi. 29 | tmp_.resize(samples.size()); 30 | for (size_t i = 0; i < samples.size(); i++) { 31 | auto si = samples[i].real() * 127; 32 | auto sq = samples[i].imag() * 127; 33 | 34 | // Clamp 35 | si = (0.5f * (fabsf(si + 127.0f) - fabsf(si - 127.0f))); 36 | sq = (0.5f * (fabsf(sq + 127.0f) - fabsf(sq - 127.0f))); 37 | 38 | // Convert to int8_t 39 | tmp_[i].real((int8_t) si); 40 | tmp_[i].imag((int8_t) sq); 41 | } 42 | 43 | auto rv = nn_send(fd_, tmp_.data(), tmp_.size() * sizeof(tmp_[0]), 0); 44 | if (rv < 0) { 45 | fprintf(stderr, "nn_send: %s\n", nn_strerror(nn_errno())); 46 | ASSERT(false); 47 | } 48 | } 49 | -------------------------------------------------------------------------------- /src/goesproc/types.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | // AWIPS product identifier 9 | struct AWIPS { 10 | std::string t1t2; 11 | std::string a1a2; 12 | std::string ii; 13 | std::string cccc; 14 | std::string yy; 15 | std::string gggg; 16 | std::string bbb; 17 | std::string nnn; 18 | std::string xxx; 19 | std::string qq; 20 | }; 21 | 22 | // Image product (e.g. CMIP/Cloud and Moisture Imagery Product) 23 | struct Product { 24 | std::string nameShort; 25 | std::string nameLong; 26 | }; 27 | 28 | // Image region (e.g. FD/Full Disk or NH/Northern Hemisphere) 29 | struct Region { 30 | std::string nameShort; 31 | std::string nameLong; 32 | }; 33 | 34 | // Image channel (e.g. IR/Infrared or CH02/Channel 02) 35 | struct Channel { 36 | std::string nameShort; 37 | std::string nameLong; 38 | }; 39 | 40 | // Used to key a vector of segments off of its product, region, and channel 41 | using SegmentKey = std::tuple; 42 | 43 | // Corresponding hash function 44 | struct SegmentKeyHash : public std::function { 45 | std::size_t operator()(const SegmentKey& k) const { 46 | return 47 | std::hash()(std::get<0>(k)) ^ 48 | std::hash()(std::get<1>(k)) ^ 49 | std::hash()(std::get<2>(k)); 50 | } 51 | }; 52 | -------------------------------------------------------------------------------- /src/goesrecv/rtlsdr_source.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | #include 8 | 9 | #include 10 | 11 | #include "source.h" 12 | 13 | class RTLSDR : public Source { 14 | public: 15 | static std::unique_ptr open(uint32_t index = 0); 16 | 17 | explicit RTLSDR(rtlsdr_dev_t* dev); 18 | ~RTLSDR(); 19 | 20 | const std::vector& getTunerGains() { 21 | return tunerGains_; 22 | } 23 | 24 | void setFrequency(uint32_t freq); 25 | 26 | void setSampleRate(uint32_t rate); 27 | 28 | virtual uint32_t getSampleRate() const override; 29 | 30 | void setTunerGain(int gain); 31 | 32 | void setBiasTee(bool on); 33 | 34 | void setSamplePublisher(std::unique_ptr samplePublisher) { 35 | samplePublisher_ = std::move(samplePublisher); 36 | } 37 | 38 | virtual void start(const std::shared_ptr >& queue) override; 39 | 40 | virtual void stop() override; 41 | 42 | void handle(unsigned char* buf, uint32_t len); 43 | 44 | protected: 45 | void process( 46 | size_t nsamples, 47 | unsigned char* buf, 48 | std::complex* fo); 49 | 50 | rtlsdr_dev_t* dev_; 51 | 52 | std::vector tunerGains_; 53 | std::thread thread_; 54 | 55 | // Set on start; cleared on stop 56 | std::shared_ptr > queue_; 57 | 58 | // Optional publisher for samples 59 | std::unique_ptr samplePublisher_; 60 | }; 61 | -------------------------------------------------------------------------------- /src/goesrecv/airspy_source.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | #include 8 | 9 | #include 10 | 11 | #include "source.h" 12 | 13 | class Airspy : public Source { 14 | public: 15 | static std::unique_ptr open(uint32_t index = 0); 16 | 17 | explicit Airspy(struct airspy_device* dev); 18 | ~Airspy(); 19 | 20 | std::vector getSampleRates() const { 21 | return sampleRates_; 22 | } 23 | 24 | void setFrequency(uint32_t freq); 25 | 26 | void setSampleRate(uint32_t rate); 27 | 28 | virtual uint32_t getSampleRate() const override; 29 | 30 | void setGain(int gain); 31 | 32 | void setBiasTee(bool on); 33 | 34 | void setSamplePublisher(std::unique_ptr samplePublisher) { 35 | samplePublisher_ = std::move(samplePublisher); 36 | } 37 | 38 | virtual void start(const std::shared_ptr >& queue) override; 39 | 40 | virtual void stop() override; 41 | 42 | void handle(const airspy_transfer* transfer); 43 | 44 | protected: 45 | struct airspy_device* dev_; 46 | 47 | std::vector loadSampleRates(); 48 | std::vector sampleRates_; 49 | std::uint32_t sampleRate_; 50 | 51 | // Background RX thread 52 | std::thread thread_; 53 | 54 | // Set on start; cleared on stop 55 | std::shared_ptr > queue_; 56 | 57 | // Optional publisher for samples 58 | std::unique_ptr samplePublisher_; 59 | }; 60 | -------------------------------------------------------------------------------- /src/util/string.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | namespace util { 8 | 9 | namespace detail { 10 | 11 | inline std::ostream& _str(std::ostream& ss) { 12 | return ss; 13 | } 14 | 15 | template 16 | inline std::ostream& _str(std::ostream& ss, const T& t) { 17 | ss << t; 18 | return ss; 19 | } 20 | 21 | template 22 | inline std::ostream& _str(std::ostream& ss, const T& t, const Args&... args) { 23 | return _str(_str(ss, t), args...); 24 | } 25 | 26 | } // namespace detail 27 | 28 | // Convert a list of string-like arguments into a single string. 29 | template 30 | inline std::string str(const Args&... args) { 31 | std::ostringstream ss; 32 | detail::_str(ss, args...); 33 | return ss.str(); 34 | } 35 | 36 | // Specializations for already-a-string types. 37 | template <> 38 | inline std::string str(const std::string& str) { 39 | return str; 40 | } 41 | 42 | inline std::string str(const char* c_str) { 43 | return c_str; 44 | } 45 | 46 | std::vector split(std::string in, char delim); 47 | 48 | std::string join(const std::vector& in, char delim); 49 | 50 | std::string trimLeft(const std::string& in); 51 | 52 | std::string trimRight(const std::string& in); 53 | 54 | std::string toLower(const std::string& in); 55 | 56 | std::string toUpper(const std::string& in); 57 | 58 | size_t findLast(const std::string& in, char c); 59 | 60 | } // namespace util 61 | -------------------------------------------------------------------------------- /src/goespackets/goespackets.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | 4 | #include "assembler/vcdu.h" 5 | 6 | #include "lib/file_reader.h" 7 | #include "lib/file_writer.h" 8 | #include "lib/nanomsg_reader.h" 9 | #include "lib/nanomsg_writer.h" 10 | #include "options.h" 11 | 12 | int main(int argc, char** argv) { 13 | auto opts = parseOptions(argc, argv); 14 | 15 | // Create packet reader depending on options 16 | std::unique_ptr reader; 17 | if (!opts.subscribe.empty()) { 18 | reader = std::make_unique(opts.subscribe); 19 | } else if (!opts.files.empty()) { 20 | reader = std::make_unique(opts.files); 21 | } else { 22 | std::cerr << "No input specified" << std::endl; 23 | return 1; 24 | } 25 | 26 | // Create file writer for current directory 27 | std::vector> writers; 28 | if (opts.record) { 29 | writers.push_back(std::make_unique(opts.filename)); 30 | } 31 | if (!opts.publish.empty()) { 32 | writers.push_back(std::make_unique(opts.publish)); 33 | } 34 | 35 | std::array buf; 36 | while (reader->nextPacket(buf)) { 37 | // Filter by Virtual Channel ID if specified 38 | if (!opts.vcids.empty()) { 39 | VCDU vcdu(buf); 40 | if (opts.vcids.find(vcdu.getVCID()) == opts.vcids.end()) { 41 | continue; 42 | } 43 | } 44 | 45 | for (auto& writer : writers) { 46 | writer->write(buf, time(0)); 47 | } 48 | } 49 | } 50 | -------------------------------------------------------------------------------- /src/goesrecv/demodulator.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "agc.h" 6 | #include "clock_recovery.h" 7 | #include "config.h" 8 | #include "costas.h" 9 | #include "publisher.h" 10 | #include "quantize.h" 11 | #include "rrc.h" 12 | #include "source.h" 13 | #include "stats_publisher.h" 14 | #include "types.h" 15 | 16 | class Demodulator { 17 | public: 18 | enum Type { 19 | LRIT = 1, 20 | HRIT = 2, 21 | }; 22 | 23 | explicit Demodulator(Type t); 24 | 25 | void initialize(Config& config); 26 | 27 | std::shared_ptr > > getSoftBitsQueue() { 28 | return softBitsQueue_; 29 | } 30 | 31 | void start(); 32 | void stop(); 33 | 34 | protected: 35 | void publishStats(); 36 | 37 | uint32_t symbolRate_; 38 | uint32_t sampleRate_; 39 | 40 | std::unique_ptr source_; 41 | std::unique_ptr statsPublisher_; 42 | std::thread thread_; 43 | 44 | // DSP blocks 45 | std::unique_ptr agc_; 46 | std::unique_ptr costas_; 47 | std::unique_ptr rrc_; 48 | std::unique_ptr clockRecovery_; 49 | std::unique_ptr quantization_; 50 | 51 | // Queues 52 | std::shared_ptr > sourceQueue_; 53 | std::shared_ptr > agcQueue_; 54 | std::shared_ptr > costasQueue_; 55 | std::shared_ptr > rrcQueue_; 56 | std::shared_ptr > clockRecoveryQueue_; 57 | std::shared_ptr > > softBitsQueue_; 58 | }; 59 | -------------------------------------------------------------------------------- /.github/workflows/docs.yml: -------------------------------------------------------------------------------- 1 | name: Documentation 2 | 3 | on: 4 | push: 5 | paths: 6 | - 'docs/**' 7 | branches: 8 | - main 9 | pull_request: 10 | paths: 11 | - 'docs/**' 12 | branches: 13 | - main 14 | 15 | jobs: 16 | "Build": 17 | runs-on: ubuntu-latest 18 | steps: 19 | - uses: actions/checkout@v3 20 | - uses: actions/setup-python@v4 21 | with: 22 | cache: 'pip' 23 | - run: pip install -r docs/requirements.txt 24 | - run: make html 25 | working-directory: docs 26 | - uses: actions/upload-artifact@v3 27 | with: 28 | path: docs/_build/html 29 | 30 | "Deploy": 31 | needs: "Build" 32 | if: success() && github.ref == 'refs/heads/main' 33 | runs-on: ubuntu-latest 34 | steps: 35 | - uses: actions/checkout@v3 36 | with: 37 | ref: refs/heads/gh-pages 38 | # Remove all tracked files such that we're left with a snapshot of the artifact. 39 | - name: clean 40 | run: git ls-files -z | xargs -0 rm -f 41 | - uses: actions/download-artifact@v3 42 | - name: commit 43 | run: | 44 | rsync -a artifact/ . 45 | rm -rf artifact 46 | git add -A 47 | git config user.name 'github-actions[bot]' 48 | git config user.email 'github-actions[bot]@users.noreply.github.com' 49 | git commit --allow-empty -m "Snapshot of $GITHUB_SHA" 50 | - name: push 51 | run: git push origin gh-pages 52 | -------------------------------------------------------------------------------- /src/goesproc/packet_processor.cc: -------------------------------------------------------------------------------- 1 | #include "packet_processor.h" 2 | 3 | #include 4 | 5 | #include "lrit/file.h" 6 | 7 | PacketProcessor::PacketProcessor(std::vector > handlers) 8 | : handlers_(std::move(handlers)) { 9 | } 10 | 11 | void PacketProcessor::run(std::unique_ptr& reader, bool verbose) { 12 | if (verbose) { 13 | std::cout 14 | << "Waiting for first packet..." 15 | // Carriage return to return to beginning of line 16 | << "\r" 17 | // Flush buffers 18 | << std::flush 19 | // Erase in line (expected to be buffered) 20 | << "\033[K"; 21 | } 22 | 23 | std::array buf; 24 | while (reader->nextPacket(buf)) { 25 | if (verbose) { 26 | VCDU vcdu(buf); 27 | std::cout 28 | << "Packet:" 29 | << " SCID=" 30 | << std::setw(1) << vcdu.getSCID() 31 | << " VCID=" 32 | << std::setw(2) << vcdu.getVCID() 33 | << " counter=" 34 | << std::setw(8) << std::setfill('0') << vcdu.getCounter() 35 | // Carriage return to return to beginning of line 36 | << "\r" 37 | // Flush buffers 38 | << std::flush 39 | // Erase in line (expected to be buffered) 40 | << "\033[K"; 41 | } 42 | 43 | auto spdus = assembler_.process(buf); 44 | for (auto& spdu : spdus) { 45 | auto file = std::make_shared(spdu->get()); 46 | for (auto& handler : handlers_) { 47 | handler->handle(file); 48 | } 49 | } 50 | } 51 | } 52 | -------------------------------------------------------------------------------- /src/goesrecv/publisher.cc: -------------------------------------------------------------------------------- 1 | #include "publisher.h" 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | #include 8 | 9 | int Publisher::bind(const std::vector& endpoints) { 10 | auto fd = nn_socket(AF_SP, NN_PUB); 11 | if (fd < 0) { 12 | std::stringstream ss; 13 | ss << "nn_socket: " << nn_strerror(nn_errno()); 14 | throw std::runtime_error(ss.str()); 15 | } 16 | 17 | for (const auto& endpoint : endpoints) { 18 | auto rv = nn_bind(fd, endpoint.c_str()); 19 | if (rv < 0) { 20 | nn_close(fd); 21 | std::stringstream ss; 22 | ss << "nn_bind: " << nn_strerror(nn_errno()); 23 | ss << " (" << endpoint << ")"; 24 | throw std::runtime_error(ss.str()); 25 | } 26 | } 27 | 28 | return fd; 29 | } 30 | 31 | std::unique_ptr Publisher::create(const std::string& endpoint) { 32 | auto fd = Publisher::bind(endpoint); 33 | return std::make_unique(fd); 34 | } 35 | 36 | Publisher::Publisher(int fd) : fd_(fd) { 37 | } 38 | 39 | Publisher::~Publisher() { 40 | } 41 | 42 | void Publisher::setSendBuffer(int size) { 43 | int rv = nn_setsockopt(fd_, NN_SOL_SOCKET, NN_SNDBUF, &size, sizeof(size)); 44 | if (rv < 0) { 45 | nn_close(fd_); 46 | std::stringstream ss; 47 | ss << "nn_setsockopt: " << nn_strerror(nn_errno()); 48 | throw std::runtime_error(ss.str()); 49 | } 50 | } 51 | 52 | bool Publisher::hasSubscribers() { 53 | uint32_t subs = (uint32_t) nn_get_statistic(fd_, NN_STAT_CURRENT_CONNECTIONS); 54 | return subs > 0; 55 | } 56 | -------------------------------------------------------------------------------- /src/lib/file_writer.cc: -------------------------------------------------------------------------------- 1 | #include "file_writer.h" 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include 8 | 9 | using namespace util; 10 | 11 | FileWriter::FileWriter(const std::string& pattern) 12 | : pattern_(pattern) { 13 | } 14 | 15 | FileWriter::~FileWriter() { 16 | } 17 | 18 | void FileWriter::write(const std::array& buf, time_t t) { 19 | auto filename = buildFilename(t); 20 | 21 | // Open new file if necessary 22 | if (filename != currentFilename_) { 23 | if (of_.good()) { 24 | of_.close(); 25 | } 26 | 27 | // Optionally mkdir path to new file 28 | auto rpos = filename.rfind('/'); 29 | if (rpos != std::string::npos) { 30 | mkdirp(filename.substr(0, rpos)); 31 | } 32 | 33 | // Open new file 34 | of_.open(filename, std::ofstream::out | std::ofstream::app); 35 | if (!of_.good()) { 36 | std::cout 37 | << "Unable to open file: " 38 | << filename 39 | << " (" << strerror(errno) << ")" 40 | << std::endl; 41 | } else { 42 | std::cout 43 | << "Writing to file: " 44 | << filename 45 | << std::endl; 46 | } 47 | 48 | currentFilename_ = filename; 49 | } 50 | 51 | of_.write((const char*) buf.data(), buf.size()); 52 | } 53 | 54 | std::string FileWriter::buildFilename(time_t t) { 55 | std::vector tsbuf(256); 56 | auto len = strftime( 57 | tsbuf.data(), 58 | tsbuf.size(), 59 | pattern_.c_str(), 60 | gmtime(&t)); 61 | return std::string(tsbuf.data(), len); 62 | } 63 | -------------------------------------------------------------------------------- /src/util/error.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include 6 | 7 | #define ERROR(...) throw std::runtime_error(::util::str(__VA_ARGS__)) 8 | 9 | #define ASSERT(cond) \ 10 | do { \ 11 | if (!(cond)) { \ 12 | ERROR("Assertion `" #cond "` failed at ", __FILE__, ":", __LINE__); \ 13 | } \ 14 | } while (0) 15 | 16 | #define ASSERTM(cond, ...) \ 17 | do { \ 18 | if (!(cond)) { \ 19 | ERROR( \ 20 | "Assertion `" #cond "` failed at ", \ 21 | __FILE__, \ 22 | ":", \ 23 | __LINE__, \ 24 | ", ", \ 25 | ::util::str(__VA_ARGS__)); \ 26 | } \ 27 | } while (0) 28 | 29 | #define FAILM(...) \ 30 | do { \ 31 | ERROR( \ 32 | "Failure at ", \ 33 | __FILE__, \ 34 | ":", \ 35 | __LINE__, \ 36 | ", ", \ 37 | ::util::str(__VA_ARGS__)); \ 38 | } while (0) 39 | -------------------------------------------------------------------------------- /src/goesproc/handler_goesn.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "config.h" 7 | #include "file_writer.h" 8 | #include "handler.h" 9 | #include "image.h" 10 | #include "types.h" 11 | 12 | // Handler for GOES-N series. 13 | // We can treat images from GOES-13 and GOES-15 equally. 14 | class GOESNImageHandler : public Handler { 15 | public: 16 | explicit GOESNImageHandler( 17 | const Config::Handler& config, 18 | const std::shared_ptr& fileWriter); 19 | 20 | virtual void handle(std::shared_ptr f); 21 | 22 | protected: 23 | // The GOES-N LRIT image files contain key/value pairs in the 24 | // ancillary text header. A subset is represented in this struct. 25 | struct Details { 26 | struct timespec frameStart; 27 | std::string satellite; 28 | }; 29 | 30 | Details loadDetails(const lrit::File& f); 31 | Region loadRegion(const lrit::NOAALRITHeader& h) const; 32 | Channel loadChannel(const lrit::NOAALRITHeader& h) const; 33 | 34 | void overlayMaps(const lrit::File& f, const Area& crop, cv::Mat& mat); 35 | 36 | Config::Handler config_; 37 | std::shared_ptr fileWriter_; 38 | uint16_t productID_; 39 | 40 | using SegmentVector = std::vector>; 41 | 42 | // Maintain a map of region and channel to list of segments. 43 | // This assumes that two images for the same region and channel are 44 | // never sent concurrently, but always in order. 45 | std::unordered_map< 46 | SegmentKey, 47 | SegmentVector, 48 | SegmentKeyHash> segments_; 49 | }; 50 | -------------------------------------------------------------------------------- /src/lib/nanomsg_writer.cc: -------------------------------------------------------------------------------- 1 | #include "nanomsg_writer.h" 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | #include 8 | 9 | #include 10 | 11 | NanomsgWriter::NanomsgWriter(const std::vector& endpoints) { 12 | auto fd = nn_socket(AF_SP, NN_PUB); 13 | if (fd < 0) { 14 | std::stringstream ss; 15 | ss << "nn_socket: " << nn_strerror(nn_errno()); 16 | throw std::runtime_error(ss.str()); 17 | } 18 | 19 | for (const auto& endpoint : endpoints) { 20 | auto rv = nn_bind(fd, endpoint.c_str()); 21 | if (rv < 0) { 22 | nn_close(fd); 23 | std::stringstream ss; 24 | ss << "nn_bind: " << nn_strerror(nn_errno()); 25 | ss << " (" << endpoint << ")"; 26 | throw std::runtime_error(ss.str()); 27 | } 28 | } 29 | 30 | int size = 1048576; 31 | auto rv = nn_setsockopt(fd_, NN_SOL_SOCKET, NN_SNDBUF, &size, sizeof(size)); 32 | if (rv < 0) { 33 | nn_close(fd_); 34 | std::stringstream ss; 35 | ss << "nn_setsockopt: " << nn_strerror(nn_errno()); 36 | throw std::runtime_error(ss.str()); 37 | } 38 | 39 | fd_ = fd; 40 | } 41 | 42 | NanomsgWriter::~NanomsgWriter() { 43 | if (fd_ >= 0) { 44 | nn_close(fd_); 45 | fd_ = -1; 46 | } 47 | } 48 | 49 | void NanomsgWriter::write( 50 | const std::array& in, 51 | time_t /* unused */) { 52 | auto rv = nn_send(fd_, in.data(), in.size() * sizeof(in[0]), 0); 53 | if (rv < 0) { 54 | std::stringstream ss; 55 | ss << "nn_send: " << nn_strerror(nn_errno()); 56 | throw std::runtime_error(ss.str()); 57 | } 58 | } 59 | -------------------------------------------------------------------------------- /src/dcs/dcsdump.cc: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include 4 | #include 5 | 6 | #include "dcs.h" 7 | #include "lrit/file.h" 8 | 9 | int main(int argc, char** argv) { 10 | int rv; 11 | int nread; 12 | 13 | for (int i = optind; i < argc; i++) { 14 | auto file = lrit::File(argv[i]); 15 | auto ph = file.getHeader(); 16 | if (ph.fileType != 130) { 17 | std::cerr 18 | << "File " << argv[i] 19 | << " has file type " << int(ph.fileType) 20 | << " (expected: 130)" 21 | << std::endl; 22 | exit(1); 23 | } 24 | 25 | // Read DCS data 26 | int nbytes = (ph.dataLength + 7) / 8; 27 | auto ifs = file.getData(); 28 | auto buf = std::make_unique(nbytes); 29 | ifs->read(buf.get(), nbytes); 30 | ASSERT(*ifs); 31 | nread = 0; 32 | 33 | // Read DCS file header (container for multiple DCS payloads) 34 | dcs::FileHeader fh; 35 | rv = fh.readFrom(buf.get(), nbytes); 36 | ASSERT(rv > 0); 37 | nread += rv; 38 | 39 | while (nread < nbytes) { 40 | // Read DCS header 41 | dcs::Header h; 42 | rv = h.readFrom(buf.get() + nread, nbytes - nread); 43 | ASSERT(rv > 0); 44 | nread += rv; 45 | 46 | // Skip over actual data 47 | nread += h.dataLength; 48 | 49 | // Skip 14 characters for time with milliseconds (in DCS format). 50 | // Skip 1 whitepace. 51 | // Skip 14 characters for time with milliseconds (in DCS format). 52 | // Skip 4 bytes (same prelude as prelude to first header) 53 | nread += 14 + 1 + 14 + 4; 54 | } 55 | 56 | ASSERT(nread == nbytes); 57 | } 58 | 59 | return 0; 60 | } 61 | -------------------------------------------------------------------------------- /src/util/time.cc: -------------------------------------------------------------------------------- 1 | #include "time.h" 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | namespace util { 8 | 9 | std::string stringTime() { 10 | struct timespec ts; 11 | auto rv = clock_gettime(CLOCK_REALTIME, &ts); 12 | ASSERT(rv >= 0); 13 | std::array tsbuf; 14 | auto len = strftime( 15 | tsbuf.data(), tsbuf.size(), "%Y-%m-%dT%H:%M:%S.", gmtime(&ts.tv_sec)); 16 | len += snprintf( 17 | tsbuf.data() + len, tsbuf.size() - len, "%03ldZ", ts.tv_nsec / 1000000); 18 | ASSERT(len < tsbuf.size()); 19 | return std::string(tsbuf.data(), len); 20 | } 21 | 22 | bool parseTime(const std::string& in, struct timespec* ts) { 23 | const char* buf = in.c_str(); 24 | struct tm tm; 25 | 26 | // Initialize tm struct to 0s before using it, otherwise mktime will behave unpredictably - Kons 27 | memset(&tm, 0x0, sizeof(tm)); 28 | 29 | long int tv_nsec = 0; 30 | 31 | // For example: 2017-12-21T17:46:32.2Z 32 | char* pos = strptime(buf, "%Y-%m-%dT%H:%M:%S", &tm); 33 | if (pos < (buf + in.size())) { 34 | if (pos[0] == '.') { 35 | // Expect single decimal for fractional second 36 | int dec = 0; 37 | int num = sscanf(pos, ".%dZ", &dec); 38 | if (num == 1 && dec < 10) { 39 | ts->tv_nsec = num * 100000000; 40 | } else { 41 | return false; 42 | } 43 | } else { 44 | return false; 45 | } 46 | } 47 | 48 | auto t = mktime(&tm); 49 | 50 | // The resulting t should always be greater 0, otherwise there was a problem with date conversion 51 | ASSERT(t > 0); 52 | 53 | ts->tv_sec = t; 54 | ts->tv_nsec = tv_nsec; 55 | return true; 56 | } 57 | 58 | } // namespace util 59 | -------------------------------------------------------------------------------- /src/goesrecv/goesrecv.cc: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include 4 | 5 | #include "config.h" 6 | #include "decoder.h" 7 | #include "demodulator.h" 8 | #include "monitor.h" 9 | #include "options.h" 10 | #include "publisher.h" 11 | 12 | static bool sigint = false; 13 | 14 | static void signalHandler(int signum) { 15 | fprintf(stderr, "Signal caught, exiting!\n"); 16 | sigint = true; 17 | } 18 | 19 | int main(int argc, char** argv) { 20 | auto opts = parseOptions(argc, argv); 21 | auto config = Config::load(opts.config); 22 | 23 | // Convert string option to enum 24 | Demodulator::Type downlinkType; 25 | if (config.demodulator.downlinkType == "lrit") { 26 | downlinkType = Demodulator::LRIT; 27 | } else if (config.demodulator.downlinkType == "hrit") { 28 | downlinkType = Demodulator::HRIT; 29 | } else { 30 | std::cerr 31 | << "Invalid downlink type: " 32 | << config.demodulator.downlinkType 33 | << std::endl; 34 | exit(1); 35 | } 36 | 37 | Demodulator demod(downlinkType); 38 | demod.initialize(config); 39 | 40 | Decoder decode(demod.getSoftBitsQueue()); 41 | decode.initialize(config); 42 | 43 | Monitor monitor(opts.verbose, opts.interval); 44 | monitor.initialize(config); 45 | 46 | // Install signal handler 47 | struct sigaction sa; 48 | sa.sa_handler = signalHandler; 49 | sigemptyset(&sa.sa_mask); 50 | sa.sa_flags = 0; 51 | sigaction(SIGINT, &sa, NULL); 52 | sigaction(SIGTERM, &sa, NULL); 53 | 54 | demod.start(); 55 | decode.start(); 56 | monitor.start(); 57 | 58 | while (!sigint) { 59 | pause(); 60 | } 61 | 62 | demod.stop(); 63 | decode.stop(); 64 | monitor.stop(); 65 | 66 | return 0; 67 | } 68 | -------------------------------------------------------------------------------- /src/util/string.cc: -------------------------------------------------------------------------------- 1 | #include "string.h" 2 | 3 | #include 4 | #include 5 | 6 | namespace { 7 | 8 | const std::string whitespace = " \f\n\r\t\v"; 9 | 10 | } // namespace 11 | 12 | namespace util { 13 | 14 | std::vector split(std::string in, char delim) { 15 | std::vector items; 16 | std::istringstream ss(in); 17 | std::string item; 18 | while (std::getline(ss, item, delim)) { 19 | items.push_back(item); 20 | } 21 | return items; 22 | } 23 | 24 | std::string join(const std::vector& in, char delim) { 25 | std::stringstream ss; 26 | for (size_t i = 0; i < in.size(); i++) { 27 | if (i > 0) { 28 | ss << delim; 29 | } 30 | ss << in[i]; 31 | } 32 | return ss.str(); 33 | } 34 | 35 | std::string trimLeft(const std::string& in) { 36 | return in.substr(in.find_first_not_of(whitespace)); 37 | } 38 | 39 | std::string trimRight(const std::string& in) { 40 | return in.substr(0, in.find_last_not_of(whitespace) + 1); 41 | } 42 | 43 | std::string toLower(const std::string& in) { 44 | std::string out; 45 | out.resize(in.size()); 46 | std::transform(in.begin(), in.end(), out.begin(), ::tolower); 47 | return out; 48 | } 49 | 50 | std::string toUpper(const std::string& in) { 51 | std::string out; 52 | out.resize(in.size()); 53 | std::transform(in.begin(), in.end(), out.begin(), ::toupper); 54 | return out; 55 | } 56 | 57 | size_t findLast(const std::string& in, char c) { 58 | auto pos = in.find('_'); 59 | while (pos != std::string::npos) { 60 | auto npos = in.find(c, pos + 1); 61 | if (npos == std::string::npos) { 62 | break; 63 | } 64 | pos = npos; 65 | } 66 | return pos; 67 | } 68 | 69 | } // namespace util 70 | -------------------------------------------------------------------------------- /docs/installation.rst: -------------------------------------------------------------------------------- 1 | .. _installation: 2 | 3 | Installation 4 | ============ 5 | 6 | Building goestools requires Linux. 7 | 8 | It can be built for x86 and ARM (with NEON). 9 | 10 | .. note:: 11 | 12 | Other operating systems may be fine as well but have not been 13 | confirmed to work. If you manage to get goestools to work on macOS, 14 | Windows, or something else, please reach out and share instructions, 15 | so they can be added to this page. 16 | 17 | Dependencies 18 | ------------ 19 | 20 | System dependencies: 21 | 22 | * CMake 23 | * C++14 compiler 24 | * OpenCV (for image processing in goesproc) 25 | * zlib (for decompressing EMWIN data) 26 | 27 | Bundled dependencies (see vendor directory in repository): 28 | 29 | * libcorrect (currently a fork with CMake related fixes) 30 | * libaec 31 | * nanomsg 32 | * json 33 | * tinytoml 34 | 35 | Building 36 | -------- 37 | 38 | These instructions should work for both Ubuntu and Raspbian. 39 | 40 | Install system dependencies: 41 | 42 | .. code-block:: text 43 | 44 | sudo apt-get install -y \ 45 | build-essential \ 46 | cmake \ 47 | git-core \ 48 | libopencv-dev \ 49 | libproj-dev \ 50 | zlib1g-dev 51 | 52 | If you want to run goesrecv on this machine, you also have to install 53 | the development packages of the drivers the SDRs you want to use; 54 | ``librtlsdr-dev`` for an RTL-SDR, ``libairspy-dev`` for an Airspy. 55 | 56 | Now you can build and install goestools: 57 | 58 | .. code-block:: text 59 | 60 | git clone --recursive https://github.com/pietern/goestools 61 | cd goestools 62 | mkdir build 63 | cd build 64 | cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local 65 | make 66 | make install 67 | 68 | The goestools executables are now available in /usr/local/bin. 69 | -------------------------------------------------------------------------------- /scripts/list_raspbian_urls.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # 3 | # Find paths of packages and their dependencies 4 | # by parsing Packages file from Raspbian repository. 5 | # 6 | 7 | import argparse 8 | import re 9 | import sys 10 | import urllib 11 | 12 | 13 | parser = argparse.ArgumentParser() 14 | parser.add_argument('-v', '--verbose', action='store_true') 15 | parser.add_argument('packages', metavar='PACKAGE', type=str, nargs='+') 16 | args = parser.parse_args() 17 | 18 | 19 | # Load list of packages 20 | mirror = "http://mirrordirector.raspbian.org/raspbian" 21 | response = urllib.urlopen(mirror + "/dists/stretch/main/binary-armhf/Packages") 22 | body = response.read() 23 | 24 | 25 | # Parse packages in map of dicts (keyed by package name) 26 | packages = dict() 27 | chunk = re.compile("^Package: (.*?)$.*?\n$", re.M | re.S) 28 | pair = re.compile("^(\w+): (.*)$", re.M) 29 | for m1 in chunk.finditer(body): 30 | out = dict() 31 | for m2 in pair.finditer(m1.group(0)): 32 | out[m2.group(1)] = m2.group(2) 33 | packages[m1.group(1)] = out 34 | 35 | 36 | # List recursive dependencies 37 | queue = args.packages 38 | skip = set() 39 | while len(queue) > 0: 40 | package = queue.pop(0) 41 | if not package in packages: 42 | if args.verbose: 43 | sys.stderr.write("Package doesn't exist: {}\n".format(package)) 44 | continue 45 | 46 | if package in skip: 47 | continue 48 | 49 | skip.add(package) 50 | kv = packages[package] 51 | print(mirror + "/" + kv['Filename']) 52 | 53 | depends = kv.get('Depends', '') 54 | for spec in depends.split(', '): 55 | if len(spec) == 0: 56 | continue 57 | match = re.match("^([^\s]+)", spec) 58 | queue.append(match.group(1)) 59 | -------------------------------------------------------------------------------- /src/goesproc/lrit_processor.cc: -------------------------------------------------------------------------------- 1 | #include "lrit_processor.h" 2 | 3 | #include 4 | 5 | #include 6 | 7 | #include "lib/dir.h" 8 | #include "lrit/file.h" 9 | 10 | LRITProcessor::LRITProcessor(std::vector > handlers) 11 | : handlers_(std::move(handlers)) { 12 | } 13 | 14 | void LRITProcessor::run(int argc, char** argv) { 15 | std::vector> files; 16 | std::map time; 17 | 18 | // Gather files from arguments (globs *.lrit* in directories). 19 | for (int i = 0; i < argc; i++) { 20 | struct stat st; 21 | auto rv = stat(argv[i], &st); 22 | if (rv < 0) { 23 | perror("stat"); 24 | exit(1); 25 | } 26 | if (S_ISDIR(st.st_mode)) { 27 | Dir dir(argv[i]); 28 | auto result = dir.matchFiles("*.lrit*"); 29 | for (const auto& path : result) { 30 | files.push_back(std::make_shared(path)); 31 | } 32 | } else { 33 | files.push_back(std::make_shared(argv[i])); 34 | } 35 | } 36 | 37 | // Gather time stamps (per-second granularity is plenty) 38 | for (const auto& file : files) { 39 | if (file->hasHeader()) { 40 | auto ts = file->getHeader().getUnix(); 41 | time[file->getName()] = ts.tv_sec; 42 | } else { 43 | time[file->getName()] = 0; 44 | } 45 | } 46 | 47 | // Sort files by their LRIT time 48 | std::sort( 49 | files.begin(), 50 | files.end(), 51 | [&time](const auto& a, const auto& b) -> bool { 52 | return time[a->getName()] < time[b->getName()]; 53 | }); 54 | 55 | // Process files in chronological order 56 | for (const auto& file : files) { 57 | for (auto& handler : handlers_) { 58 | handler->handle(file); 59 | } 60 | } 61 | } 62 | -------------------------------------------------------------------------------- /docs/resources.rst: -------------------------------------------------------------------------------- 1 | Resources 2 | ========= 3 | 4 | Links 5 | ----- 6 | 7 | About the satellites and the signals (GOES-R series): 8 | 9 | * HRIT/EMWIN: 10 | 11 | * https://www.goes-r.gov/users/hrit.html 12 | * https://www.goes-r.gov/hrit_emwin/ATR-2010-5482.pdf 13 | 14 | * GOES Rebroadcast (GRB): 15 | 16 | * https://www.goes-r.gov/users/grb.html 17 | * https://www.goes-r.gov/users/docs/GRB_downlink.pdf 18 | 19 | Because HRIT is similar to LRIT at a higher baud rate, documentation for 20 | LRIT is also relevant for HRIT. The documents below used to be hosted 21 | by NOAA until ~2020. They're mirrored here for posterity. 22 | 23 | * :download:`3_LRIT_Receiver-specs.pdf ` 24 | * :download:`4_LRIT_Transmitter-specs.pdf ` 25 | * :download:`5_LRIT_Mission-data.pdf ` 26 | 27 | The blog series by Lucas Teske is a great resource: 28 | 29 | * http://www.teske.net.br/lucas/2016/10/goes-satellite-hunt-part-1-antenna-system/ 30 | 31 | Notes 32 | ----- 33 | 34 | Reed-Solomon 35 | ^^^^^^^^^^^^ 36 | 37 | From the LRIT receiver specs: 38 | 39 | The LRIT dissemination service is a Grade-2 service; therefore, the 40 | transmission of user data will be error controlled using 41 | Reed-Solomon coding as an outer code. The used Reed-Solomon code is 42 | (255,223) with an interleaving of I = 4. 43 | 44 | Data must be transformed from Berlekamp's dual basis representation to 45 | conventional representation before we can run the Reed-Solomon 46 | algorithm provided by libcorrect. 47 | 48 | Refer to `CCSDS 101.0-B-6`_ (Recommendation For Space Data System 49 | Standards: Telemetry Channel Coding) to learn more about the 50 | Reed-Solomon specifics for this application, the dual basis 51 | representation, and how to transform data between conventional 52 | representation and dual basis representation. 53 | 54 | .. _`ccsds 101.0-b-6`: https://public.ccsds.org/Pubs/101x0b6s.pdf 55 | -------------------------------------------------------------------------------- /src/goesemwin/emwin.cc: -------------------------------------------------------------------------------- 1 | #include "emwin.h" 2 | 3 | #include 4 | 5 | using namespace util; 6 | 7 | namespace emwin { 8 | 9 | std::string File::extension() const { 10 | auto filename = packets_.front().filename(); 11 | auto pos = filename.find('.'); 12 | return toLower(filename.substr(pos + 1)); 13 | } 14 | 15 | std::vector File::data() const { 16 | std::vector out; 17 | 18 | // Copy first N-1 packets verbatim 19 | for (const auto& packet : packets_) { 20 | auto payload = packet.payload(); 21 | out.insert(out.end(), payload.begin(), payload.end()); 22 | } 23 | 24 | auto begin = out.begin(); 25 | auto end = out.end(); 26 | if (extension() == "zis") { 27 | // Trim until we have found the ZIP signature 28 | end -= 22; 29 | while (end >= begin && 30 | !(end[0] == 0x50 && 31 | end[1] == 0x4b && 32 | end[2] == 0x05 && 33 | end[3] == 0x06)) { 34 | end--; 35 | } 36 | end += 22; 37 | } else { 38 | // Trim trailing NULs 39 | end--; 40 | while (end >= begin && end[0] == 0x00) { 41 | end--; 42 | } 43 | end++; 44 | } 45 | 46 | out.resize(end - begin); 47 | return out; 48 | } 49 | 50 | std::unique_ptr Assembler::process(qbt::Packet p) { 51 | auto& vec = pending_[p.filename()]; 52 | 53 | // Grab last packet number we have seen for this file 54 | unsigned long packetNumber = 0; 55 | if (!vec.empty()) { 56 | packetNumber = vec.back().packetNumber(); 57 | } 58 | 59 | // Ignore packet if it is not a direct successor to the previous one 60 | if (p.packetNumber() != (packetNumber + 1)) { 61 | vec.clear(); 62 | return std::unique_ptr(); 63 | } 64 | 65 | vec.push_back(std::move(p)); 66 | if (vec.size() == vec.back().packetTotal()) { 67 | return std::make_unique(std::move(vec)); 68 | } 69 | 70 | return std::unique_ptr(); 71 | } 72 | 73 | } // namespace emwin 74 | -------------------------------------------------------------------------------- /src/decoder/packetizer.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include "correlator.h" 6 | #include "derandomizer.h" 7 | #include "reader.h" 8 | #include "reed_solomon.h" 9 | #include "viterbi.h" 10 | 11 | namespace decoder { 12 | 13 | class Packetizer { 14 | static constexpr auto frameBits = 8192; 15 | static constexpr auto syncWordBits = 32; 16 | static constexpr auto framePreludeBits = 32; 17 | 18 | // Encoding is twice the size of the original (convolutional code has r=1/2) 19 | static constexpr auto encodedFrameBits = 2 * frameBits; 20 | static constexpr auto encodedSyncWordBits = 2 * syncWordBits; 21 | static constexpr auto encodedFramePreludeBits = 2 * framePreludeBits; 22 | 23 | // For convenience 24 | static constexpr auto frameBytes = frameBits / 8; 25 | static constexpr auto syncWordBytes = syncWordBits / 8; 26 | static constexpr auto framePreludeBytes = framePreludeBits / 8; 27 | 28 | public: 29 | struct Details { 30 | // Cursor in symbol stream (can be used to detect drops) 31 | int64_t symbolPos; 32 | 33 | // Number of symbols skipped to get to this packet 34 | int64_t skippedSymbols; 35 | 36 | // Number of Viterbi corrected bits 37 | int viterbiBits; 38 | 39 | // Number of Reed-Solomon corrected bytes 40 | // This is -1 if the packet was not correctable 41 | int reedSolomonBytes; 42 | 43 | // If this call yielded a valid packet 44 | bool ok; 45 | 46 | // Relative time of packet from start of packetizer 47 | struct timespec relativeTime; 48 | }; 49 | 50 | explicit Packetizer(std::shared_ptr reader); 51 | 52 | bool nextPacket(std::array& out, Details* details); 53 | 54 | protected: 55 | bool read(); 56 | 57 | std::shared_ptr reader_; 58 | Viterbi viterbi_; 59 | Derandomizer derandomizer_; 60 | ReedSolomon reedSolomon_; 61 | 62 | uint8_t* buf_; 63 | size_t len_; 64 | size_t pos_; 65 | bool lock_; 66 | correlationType syncType_; 67 | int symbolRate_; 68 | int64_t symbolPos_; 69 | }; 70 | 71 | } // namespace decoder 72 | -------------------------------------------------------------------------------- /src/goesproc/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | set(GOESPROC_SRCS 2 | area.cc 3 | config.cc 4 | filename.cc 5 | file_writer.cc 6 | goesproc.cc 7 | gradient.cc 8 | handler_emwin.cc 9 | handler_goesn.cc 10 | handler_goesr.cc 11 | handler_himawari8.cc 12 | handler_nws_image.cc 13 | handler_nws_text.cc 14 | handler_text.cc 15 | image.cc 16 | lrit_processor.cc 17 | options.cc 18 | packet_processor.cc 19 | string.cc 20 | ) 21 | 22 | find_package(PkgConfig) 23 | pkg_check_modules(OPENCV opencv) 24 | 25 | # In Ubuntu >= 20.04 the pkgconfig name for OpenCV is "opencv4". 26 | if(NOT OPENCV_FOUND) 27 | pkg_check_modules(OPENCV REQUIRED opencv4) 28 | endif() 29 | 30 | pkg_check_modules(PROJ proj) 31 | 32 | if(PROJ_FOUND) 33 | list(APPEND GOESPROC_SRCS proj.cc map_drawer.cc) 34 | endif() 35 | 36 | add_executable(goesproc ${GOESPROC_SRCS}) 37 | add_sanitizers(goesproc) 38 | install(TARGETS goesproc COMPONENT goestools RUNTIME DESTINATION bin) 39 | target_link_libraries(goesproc lrit util assembler packet_reader dir) 40 | target_link_libraries(goesproc zip) 41 | target_link_libraries(goesproc nlohmann_json) 42 | target_link_libraries(goesproc timer) 43 | target_link_libraries(goesproc version) 44 | 45 | if(OPENCV_FOUND) 46 | target_link_libraries(goesproc opencv_core opencv_highgui opencv_imgproc) 47 | target_include_directories(goesproc PRIVATE ${OPENCV_INCLUDE_DIRS}) 48 | if(${OPENCV_VERSION} VERSION_GREATER 3.0) 49 | target_link_libraries(goesproc opencv_imgcodecs) 50 | endif() 51 | endif() 52 | 53 | if(PROJ_FOUND) 54 | string(REPLACE "." ";" PROJ_VERSION_LIST ${PROJ_VERSION}) 55 | list(GET PROJ_VERSION_LIST 0 PROJ_VERSION_MAJOR) 56 | list(GET PROJ_VERSION_LIST 1 PROJ_VERSION_MINOR) 57 | list(GET PROJ_VERSION_LIST 2 PROJ_VERSION_PATCH) 58 | target_compile_definitions(goesproc PRIVATE 59 | PROJ_VERSION_MAJOR=${PROJ_VERSION_MAJOR} 60 | PROJ_VERSION_MINOR=${PROJ_VERSION_MINOR} 61 | PROJ_VERSION_PATCH=${PROJ_VERSION_PATCH}) 62 | target_compile_definitions(goesproc PRIVATE HAS_PROJ) 63 | target_link_libraries(goesproc proj) 64 | endif() 65 | -------------------------------------------------------------------------------- /src/assembler/transport_pdu.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | namespace assembler { 9 | 10 | // Transport Protocol Data Unit 11 | // See: http://www.noaasis.noaa.gov/LRIT/pdf-files/3_LRIT_Receiver-specs.pdf 12 | class TransportPDU { 13 | public: 14 | static constexpr auto headerBytes = 6; 15 | static constexpr auto dataBytes = 8192; 16 | 17 | TransportPDU() { 18 | header.reserve(headerBytes); 19 | data.reserve(dataBytes); 20 | } 21 | 22 | size_t read(const uint8_t* buf, size_t len); 23 | 24 | bool headerComplete() { 25 | return (header.size() == headerBytes); 26 | } 27 | 28 | bool dataComplete() { 29 | return headerComplete() && (data.size() == length()); 30 | } 31 | 32 | // Packet Identification 33 | unsigned version() const { 34 | return (header[0] >> 5) & 0x7; 35 | } 36 | 37 | unsigned type() const { 38 | return (header[0] >> 4) & 0x1; 39 | } 40 | 41 | unsigned secondaryHeaderFlag() const { 42 | return (header[0] >> 3) & 0x1; 43 | } 44 | 45 | unsigned apid() const { 46 | return ((header[0] & 0x7) << 8) | header[1]; 47 | } 48 | 49 | // Packet Sequence Control 50 | unsigned sequenceFlag() const { 51 | return (header[2] >> 6) & 0x3; 52 | } 53 | 54 | unsigned sequenceCount() const { 55 | return (header[2] & 0x3f) << 8 | header[3]; 56 | } 57 | 58 | // Packet Length 59 | uint16_t length() const { 60 | // This field is specified as (see 6.2.1 of LRIT transmitter spec): 61 | // 62 | // > 16-bit binary count that expresses the length of the 63 | // > remainder of the source packet following this field minus 1. 64 | // 65 | // To compensate, we add 1 to get back the length in bytes. 66 | // 67 | return ((header[4] << 8) | header[5]) + 1; 68 | } 69 | 70 | uint16_t crc() const { 71 | const uint8_t* b = &data[length()-2]; 72 | return (b[0] << 8) | b[1]; 73 | } 74 | 75 | bool verifyCRC(); 76 | 77 | std::vector header; 78 | std::vector data; 79 | }; 80 | 81 | } // namespace assembler 82 | -------------------------------------------------------------------------------- /src/goesrecv/datagram_socket.cc: -------------------------------------------------------------------------------- 1 | #include "datagram_socket.h" 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | #include 10 | #include 11 | 12 | DatagramSocket::DatagramSocket(const std::string& addr) { 13 | std::string schema; 14 | std::string host; 15 | std::string port; 16 | size_t pos; 17 | 18 | pos = addr.find("://"); 19 | if (pos != std::string::npos) { 20 | schema = addr.substr(0, pos); 21 | host = addr.substr(pos + 3); 22 | } else { 23 | host = addr; 24 | } 25 | 26 | pos = host.find(':'); 27 | if (pos != std::string::npos) { 28 | port = host.substr(pos + 1); 29 | host = host.substr(0, pos); 30 | } 31 | 32 | if (host.empty()) { 33 | host = "localhost"; 34 | } 35 | 36 | if (port.empty()) { 37 | port = "8125"; 38 | } 39 | 40 | struct addrinfo hints; 41 | memset(&hints, 0, sizeof(hints)); 42 | if (schema == "udp4") { 43 | hints.ai_family = AF_INET; 44 | } else if (schema == "udp6") { 45 | hints.ai_family = AF_INET6; 46 | } else { 47 | hints.ai_family = AF_UNSPEC; 48 | } 49 | hints.ai_socktype = SOCK_DGRAM; 50 | hints.ai_protocol = 0; 51 | hints.ai_flags = AI_ADDRCONFIG; 52 | struct addrinfo* res = nullptr; 53 | auto rv = getaddrinfo(host.c_str(), port.c_str(), &hints, &res); 54 | if (rv < 0) { 55 | throw std::runtime_error("unable to resolve: " + host); 56 | } 57 | 58 | auto fd = socket(res->ai_family, res->ai_socktype, res->ai_protocol); 59 | if (fd == -1) { 60 | throw std::runtime_error("unable to create socket"); 61 | } 62 | 63 | fd_ = fd; 64 | memset(&addr_, 0, sizeof(addr_)); 65 | memcpy(&addr_, res->ai_addr, res->ai_addrlen); 66 | } 67 | 68 | DatagramSocket::~DatagramSocket() { 69 | close(fd_); 70 | } 71 | 72 | bool DatagramSocket::send(const std::string& payload) { 73 | auto rv = sendto( 74 | fd_, 75 | payload.c_str(), 76 | payload.size(), 77 | 0, 78 | (const sockaddr*) &addr_, 79 | sizeof(addr_)); 80 | return rv >= 0; 81 | } 82 | -------------------------------------------------------------------------------- /src/lib/nanomsg_reader.cc: -------------------------------------------------------------------------------- 1 | #include "nanomsg_reader.h" 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include 8 | #include 9 | 10 | NanomsgReader::NanomsgReader(const std::string& addr) { 11 | int rv; 12 | 13 | auto fd = nn_socket(AF_SP, NN_SUB); 14 | if (fd < 0) { 15 | std::stringstream ss; 16 | ss << "nn_socket: " << nn_strerror(nn_errno()); 17 | throw std::runtime_error(ss.str()); 18 | } 19 | 20 | // Fix receive buffer size to "plenty". 21 | // HRIT packet stream is 50KB/sec so this gives us 5+ minutes 22 | // before the buffer is full and nanomsg starts dropping messages. 23 | int size = 16 * 1024 * 1024; 24 | rv = nn_setsockopt(fd, NN_SOL_SOCKET, NN_RCVBUF, &size, sizeof(size)); 25 | if (rv < 0) { 26 | nn_close(fd); 27 | std::stringstream ss; 28 | ss << "nn_setsockopt: " << nn_strerror(nn_errno()); 29 | throw std::runtime_error(ss.str()); 30 | } 31 | 32 | rv = nn_connect(fd, addr.c_str()); 33 | if (rv < 0) { 34 | nn_close(fd); 35 | std::stringstream ss; 36 | ss << "nn_connect: " << nn_strerror(nn_errno()); 37 | ss << " (" << addr << ")"; 38 | throw std::runtime_error(ss.str()); 39 | } 40 | 41 | rv = nn_setsockopt(fd, NN_SUB, NN_SUB_SUBSCRIBE, "", 0); 42 | if (rv < 0) { 43 | nn_close(fd); 44 | std::stringstream ss; 45 | ss << "nn_setsockopt: " << nn_strerror(nn_errno()); 46 | ss << " (" << addr << ")"; 47 | throw std::runtime_error(ss.str()); 48 | } 49 | 50 | fd_ = fd; 51 | } 52 | 53 | NanomsgReader::~NanomsgReader() { 54 | nn_close(fd_); 55 | } 56 | 57 | bool NanomsgReader::nextPacket(std::array& out) { 58 | void* buf = nullptr; 59 | int nbytes; 60 | for (;;) { 61 | nbytes = nn_recv(fd_, &buf, NN_MSG, 0); 62 | if (nbytes < 0) { 63 | std::stringstream ss; 64 | ss << "nn_recv: " << nn_strerror(nn_errno()); 65 | throw std::runtime_error(ss.str()); 66 | } 67 | if (nbytes != (int) out.size()) { 68 | continue; 69 | } 70 | 71 | memcpy(out.data(), buf, nbytes); 72 | nn_freemsg(buf); 73 | return true; 74 | } 75 | } 76 | -------------------------------------------------------------------------------- /src/decoder/correlator.cc: -------------------------------------------------------------------------------- 1 | #include "correlator.h" 2 | 3 | namespace decoder { 4 | 5 | namespace { 6 | 7 | // The sync words below are compared to the raw bit stream 8 | // to synchronize it with the start of a new packet. 9 | // See compute_sync_words.cc for more information. 10 | const uint64_t encodedSyncWords[4] = { 11 | // LRIT sync words 12 | 0x035d49c24ff2686b, 13 | 0xfca2b63db00d9794, 14 | // HRIT sync words 15 | 0x03b10b02f33d2076, 16 | 0xdafef4fd0cc2df89, 17 | }; 18 | 19 | } // namespace 20 | 21 | const unsigned encodedSyncWordBits = 64; 22 | 23 | const char* correlationTypeToString(correlationType t) { 24 | switch (t) { 25 | case LRIT_PHASE_000: 26 | return "LRIT 0 deg"; 27 | case LRIT_PHASE_180: 28 | return "LRIT 180 deg"; 29 | case HRIT_PHASE_000: 30 | return "HRIT 0 deg"; 31 | case HRIT_PHASE_180: 32 | return "HRIT 180 deg"; 33 | } 34 | return ""; 35 | } 36 | 37 | int correlate(uint8_t* data, size_t len, int* maxOut, correlationType* maxType) { 38 | uint64_t tmp = 0; 39 | 40 | // Position with maximum correlation 41 | int pos[4] = { 0, 0, 0, 0 }; 42 | 43 | // Maximum correlation 44 | int max[4] = { 0, 0, 0, 0 }; 45 | 46 | // Find maximum correlation 47 | for (unsigned i = 0; i < len; i++) { 48 | // If data >= 128 (i.e. MSB == 1), set bit in stream. 49 | tmp = (tmp << 1) | (data[i] & 0x80) >> 7; 50 | if (i < (encodedSyncWordBits - 1)) { 51 | continue; 52 | } 53 | 54 | // Match tmp against encoded sync words 55 | for (unsigned j = 0; j < 4; j++) { 56 | auto v = 64 - __builtin_popcount(tmp ^ encodedSyncWords[j]); 57 | if (v > max[j]) { 58 | max[j] = v; 59 | pos[j] = i - (encodedSyncWordBits - 1); 60 | } 61 | } 62 | } 63 | 64 | // Return position for best correlating sync word 65 | int j = 0; 66 | for (unsigned i = 0; i < 4; i++) { 67 | if (max[i] > max[j]) { 68 | j = i; 69 | } 70 | } 71 | 72 | if (maxOut != nullptr) { 73 | *maxOut = max[j]; 74 | } 75 | if (maxType != nullptr) { 76 | *maxType = static_cast(j); 77 | } 78 | return pos[j]; 79 | } 80 | 81 | } // namespace decoder 82 | -------------------------------------------------------------------------------- /etc/goesrecv.conf: -------------------------------------------------------------------------------- 1 | [demodulator] 2 | ## Use LRIT mode for GOES-15. 3 | # mode = "lrit" 4 | ## Use HRIT mode for GOES-16 or later. 5 | # mode = "hrit" 6 | source = "airspy" 7 | 8 | # The section below configures the sample source to use. 9 | # 10 | # You can leave them commented out to use the default values for the 11 | # demodulator mode you choose ("lrit" or "hrit"). To use and configure 12 | # any of them, uncomment the section below, and change the demodulator 13 | # source field to match the source you want to use. 14 | # 15 | 16 | # [airspy] 17 | # frequency = 1694100000 18 | ## 19 | ## By default, goesrecv will use the lowest sample rate available. 20 | ## This is 2.5 MSPS for the R2 and 3.0 MSPS for the Mini. 21 | ## Because different Airspy models support different sample rates, 22 | ## it is recommended to leave the "sample_rate" field commented, 23 | ## so that it works for either model. 24 | ## 25 | # sample_rate = 3000000 26 | # gain = 18 27 | # bias_tee = false 28 | 29 | # [rtlsdr] 30 | # frequency = 1694100000 31 | # sample_rate = 2400000 32 | # gain = 30 33 | # bias_tee = false 34 | # device_index = 0 35 | 36 | # [nanomsg] 37 | # sample_rate = 2400000 38 | # connect = "tcp://1.2.3.4:5005" 39 | # receive_buffer = 2097152 40 | 41 | [costas] 42 | max_deviation = 200e3 43 | 44 | [clock_recovery.sample_publisher] 45 | bind = "tcp://0.0.0.0:5002" 46 | send_buffer = 2097152 47 | 48 | [quantization.soft_bit_publisher] 49 | bind = "tcp://0.0.0.0:5001" 50 | send_buffer = 1048576 51 | 52 | [decoder.packet_publisher] 53 | bind = "tcp://0.0.0.0:5004" 54 | send_buffer = 1048576 55 | 56 | # The demodulator stats publisher sends a JSON object that describes 57 | # the state of the demodulator (gain, frequency correction, samples 58 | # per symbol), for every block of samples. 59 | [demodulator.stats_publisher] 60 | bind = "tcp://0.0.0.0:6001" 61 | 62 | # The decoder stats publisher sends a JSON object for every packet it 63 | # decodes (Viterbi corrections, Reed-Solomon corrections, etc.). 64 | [decoder.stats_publisher] 65 | bind = "tcp://0.0.0.0:6002" 66 | 67 | # The monitor can log aggregated stats (counters, gauges, and 68 | # histograms) to a statsd daemon. Because this uses UDP, you can keep 69 | # this enabled even if you haven't setup a statsd daemon yet. 70 | [monitor] 71 | statsd_address = "udp4://localhost:8125" 72 | 73 | -------------------------------------------------------------------------------- /src/goesemwin/qbt.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | namespace qbt { 9 | 10 | class Fragment { 11 | using raw = std::array; 12 | 13 | public: 14 | Fragment( 15 | uint16_t counter, 16 | const std::vector::const_iterator& begin, 17 | const std::vector::const_iterator& end) 18 | : counter_(counter) { 19 | if ((size_t)(end - begin) > data_.size()) { 20 | throw std::runtime_error("range too large"); 21 | } 22 | std::copy(begin, end, data_.begin()); 23 | } 24 | 25 | Fragment(Fragment&& other) = default; 26 | 27 | uint16_t counter() const { 28 | return counter_; 29 | } 30 | 31 | const raw& data() const { 32 | return data_; 33 | } 34 | 35 | protected: 36 | uint16_t counter_; 37 | raw data_; 38 | }; 39 | 40 | // For overview of QBT structure, see http://www.nws.noaa.gov/emwin/EMWIN%20QBT%20Satellite%20Broadcast%20Protocol%20draft%20v1.0.3.pdf 41 | class Packet { 42 | using raw = std::array; 43 | 44 | public: 45 | Packet(raw data) : data_(data) { 46 | } 47 | 48 | std::string filename() const { 49 | auto str = std::string(data_.begin() + 9, data_.begin() + 21); 50 | auto pos = str.find('.'); 51 | return str.substr(0, pos + 4); 52 | } 53 | 54 | unsigned long packetNumber() const { 55 | auto str = std::string(data_.begin() + 24, data_.begin() + 30); 56 | return std::stoul(str); 57 | } 58 | 59 | unsigned long packetTotal() const { 60 | auto str = std::string(data_.begin() + 33, data_.begin() + 39); 61 | return std::stoul(str); 62 | } 63 | 64 | const raw& data() const { 65 | return data_; 66 | } 67 | 68 | std::vector payload() const { 69 | std::vector out; 70 | out.insert(out.end(), data_.begin() + 86, data_.begin() + 1110); 71 | return out; 72 | } 73 | 74 | protected: 75 | raw data_; 76 | }; 77 | 78 | class Assembler { 79 | public: 80 | Assembler() : counter_(0) { 81 | } 82 | 83 | // QBT packets are 1116 bytes. 84 | // The data in an S_PDU on the LRIT stream is 836 bytes. 85 | // Therefore, an S_PDU contains part of 1 or 2 QBT packets. 86 | std::unique_ptr process(qbt::Fragment f); 87 | 88 | protected: 89 | uint16_t counter_; 90 | std::vector tmp_; 91 | }; 92 | 93 | } // namespace qbt 94 | -------------------------------------------------------------------------------- /src/decoder/viterbi.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | extern "C" { 4 | #ifdef HAVE_SSE 5 | #include 6 | #else 7 | #include 8 | #endif 9 | } 10 | 11 | #include 12 | 13 | namespace decoder { 14 | 15 | class Viterbi { 16 | #ifdef HAVE_SSE 17 | using conv = correct_convolutional_sse; 18 | #else 19 | using conv = correct_convolutional; 20 | #endif 21 | 22 | public: 23 | Viterbi() { 24 | // Initialize Viterbi decoder. 25 | // Polynomials are not referenced after creation 26 | // so can be referenced from the stack. 27 | uint16_t poly[2] = { (uint16_t)0x4f, (uint16_t)0x6d }; 28 | #ifdef HAVE_SSE 29 | v_ = correct_convolutional_sse_create(2, 7, poly); 30 | #else 31 | v_ = correct_convolutional_create(2, 7, poly); 32 | #endif 33 | } 34 | 35 | ~Viterbi() { 36 | #ifdef HAVE_SSE 37 | correct_convolutional_sse_destroy(v_); 38 | #else 39 | correct_convolutional_destroy(v_); 40 | #endif 41 | } 42 | 43 | ssize_t decodeSoft(const uint8_t* encoded, size_t bits, uint8_t* msg) { 44 | #ifdef HAVE_SSE 45 | return correct_convolutional_sse_decode_soft(v_, encoded, bits, msg); 46 | #else 47 | return correct_convolutional_decode_soft(v_, encoded, bits, msg); 48 | #endif 49 | } 50 | 51 | ssize_t encodeLength(size_t len) { 52 | #ifdef HAVE_SSE 53 | return correct_convolutional_sse_encode_len(v_, len); 54 | #else 55 | return correct_convolutional_encode_len(v_, len); 56 | #endif 57 | } 58 | 59 | ssize_t encode(const uint8_t *msg, size_t len, uint8_t *encoded) { 60 | #ifdef HAVE_SSE 61 | return correct_convolutional_sse_encode(v_, msg, len, encoded); 62 | #else 63 | return correct_convolutional_encode(v_, msg, len, encoded); 64 | #endif 65 | } 66 | 67 | ssize_t compareSoft(const uint8_t* original, const uint8_t* msg, size_t bytes) { 68 | auto bits = encodeLength(bytes); 69 | tmp_.resize((bits + 7) / 8); 70 | auto rv = encode(msg, bytes, tmp_.data()); 71 | ASSERT(rv == bits); 72 | 73 | // Compare MSB of original (soft bits) with re-coded hard bit 74 | ssize_t errors = 0; 75 | for (ssize_t i = 0; i < bits; i++) { 76 | uint8_t a = original[i]; 77 | uint8_t b = tmp_[i / 8] << (i & 0x7); 78 | errors += ((a ^ b) & 0x80) >> 7; 79 | } 80 | 81 | return errors; 82 | } 83 | 84 | private: 85 | conv* v_; 86 | 87 | // Temporary buffer to hold encoded version when doing comparison 88 | std::vector tmp_; 89 | }; 90 | 91 | } // namespace decoder 92 | -------------------------------------------------------------------------------- /src/goesemwin/options.cc: -------------------------------------------------------------------------------- 1 | #include "options.h" 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | #include "lib/version.h" 9 | 10 | void usage(int argc, char** argv) { 11 | fprintf(stderr, "Usage: %s [OPTIONS] [FILE...]\n", argv[0]); 12 | fprintf(stderr, "Extract EMWIN data from packet stream.\n"); 13 | fprintf(stderr, "\n"); 14 | fprintf(stderr, "Options:\n"); 15 | fprintf(stderr, " --subscribe ADDR Address of nanomsg publisher\n"); 16 | fprintf(stderr, " --mode MODE One of raw, qbt, or emwin (default: raw)\n"); 17 | fprintf(stderr, " --out DIR Output directory\n"); 18 | fprintf(stderr, "\n"); 19 | fprintf(stderr, "Other:\n"); 20 | fprintf(stderr, " --help Display this help and exit\n"); 21 | fprintf(stderr, " --version Print version information and exit\n"); 22 | fprintf(stderr, "\n"); 23 | fprintf(stderr, "If a nanomsg address to subscribe to is specified,\n"); 24 | fprintf(stderr, "FILE arguments are not used.\n"); 25 | fprintf(stderr, "\n"); 26 | exit(0); 27 | } 28 | 29 | Options parseOptions(int argc, char** argv) { 30 | Options opts; 31 | 32 | while (1) { 33 | static struct option longOpts[] = { 34 | {"subscribe", required_argument, nullptr, 0x1001}, 35 | {"mode", required_argument, nullptr, 0x1002}, 36 | {"out", required_argument, nullptr, 0x1003}, 37 | {"help", no_argument, nullptr, 0x1337}, 38 | {"version", no_argument, nullptr, 0x1338}, 39 | {nullptr, 0, nullptr, 0}, 40 | }; 41 | 42 | auto c = getopt_long(argc, argv, "n", longOpts, nullptr); 43 | if (c == -1) { 44 | break; 45 | } 46 | 47 | switch (c) { 48 | case 0: 49 | break; 50 | case 0x1001: 51 | opts.nanomsg = optarg; 52 | break; 53 | case 0x1002: 54 | { 55 | auto tmp = std::string(optarg); 56 | if (tmp == "raw") { 57 | opts.mode = Mode::RAW; 58 | } else if (tmp == "qbt") { 59 | opts.mode = Mode::QBT; 60 | } else if (tmp == "emwin") { 61 | opts.mode = Mode::EMWIN; 62 | } 63 | } 64 | break; 65 | case 0x1003: 66 | opts.out = optarg; 67 | break; 68 | case 0x1337: 69 | usage(argc, argv); 70 | break; 71 | case 0x1338: 72 | version(argc, argv); 73 | exit(0); 74 | break; 75 | default: 76 | std::cerr << "Invalid option" << std::endl; 77 | exit(1); 78 | } 79 | } 80 | 81 | for (int i = optind; i < argc; i++) { 82 | opts.files.push_back(argv[i]); 83 | } 84 | 85 | return opts; 86 | } 87 | -------------------------------------------------------------------------------- /src/goesproc/config.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include 8 | #include 9 | 10 | #include "area.h" 11 | #include "gradient.h" 12 | 13 | struct Config { 14 | struct Map { 15 | // Path to GeoJSON file 16 | std::string path; 17 | 18 | // Contents of GeoJSON file 19 | std::shared_ptr geo; 20 | 21 | // Line color 22 | cv::Scalar color; 23 | }; 24 | 25 | struct Handler { 26 | // "image", "dcs", "text" 27 | std::string type; 28 | 29 | // "goes16", "himawari8", "nws", ... 30 | std::string origin; 31 | 32 | // For image handlers, this field is only used to filter GOES-R 33 | // ABI Level 2+ files. For non-GOES-R files, it is unused. 34 | // 35 | // Example: "cmip", "sst", "rrqpe", ... 36 | // 37 | std::vector products; 38 | 39 | // "fd", "m1", "m2", "nh", "us", ... 40 | std::vector regions; 41 | 42 | // "vs", "ir", "wv", "ch01", ... 43 | std::vector channels; 44 | 45 | // Output directory 46 | std::string dir; 47 | 48 | // Output format ("png", "jpg", ...) 49 | std::string format; 50 | 51 | // Write LRIT header contents as JSON file. 52 | bool json = false; 53 | 54 | // Crop (applied before scaling) 55 | Area crop; 56 | 57 | // Remap is used to point handlers to a lookup table 58 | // to map raw values in an image to new values. 59 | // The lookup table must be loadable by OpenCV and must 60 | // have dimensions equal to either 1x256 or 256x1. 61 | std::map remap; 62 | 63 | // Gradient defines a parametric RGB or luminance curve 64 | // to be applied via the Image Data Function 65 | std::map gradient; 66 | enum GradientInterpolationType lerptype = LERP_UNDEFINED; 67 | 68 | // Lookup table to use to generate false color images 69 | cv::Mat lut; 70 | 71 | // Filename format (see filename.cc for more info) 72 | std::string filename; 73 | 74 | // Set of map overlays to apply 75 | std::vector maps; 76 | }; 77 | 78 | static Config load(const std::string& file); 79 | 80 | explicit Config(); 81 | 82 | bool ok; 83 | std::string error; 84 | 85 | std::vector handlers; 86 | 87 | // Cache of JSON files to ensure the same file is never loaded twice. 88 | std::unordered_map> json_; 89 | 90 | // Load JSON file at specified path. 91 | std::shared_ptr loadJSON(const std::string& path); 92 | }; 93 | -------------------------------------------------------------------------------- /src/decoder/reed_solomon.cc: -------------------------------------------------------------------------------- 1 | #include "reed_solomon.h" 2 | 3 | #include 4 | 5 | namespace decoder { 6 | 7 | namespace { 8 | 9 | // Column-by-column representation of T_{\alpha\ell}. 10 | // See https://public.ccsds.org/Pubs/101x0b6s.pdf, Annex A. 11 | const uint32_t tal[] = { 12 | 0b11111110, 13 | 0b01101001, 14 | 0b01101011, 15 | 0b00001101, 16 | 0b11101111, 17 | 0b11110010, 18 | 0b01011011, 19 | 0b11000111, 20 | }; 21 | 22 | } // namespace 23 | 24 | ReedSolomon::ReedSolomon() { 25 | // Initialize lookup tables to convert between conventional and dual 26 | // basis representation. The Reed-Solomon implementation in libcorrect 27 | // uses conventional representation, yet the data we process uses dual 28 | // basis representation. We can apply direct conversion between the 29 | // two to use the libcorrect implementation for dual basis data. 30 | 31 | // For every symbol 32 | for (int i = 0; i < 256; i++) { 33 | convToDual_[i] = 0; 34 | 35 | // Bit-by-bit multiply is AND. 36 | // Output bit is sum of the bit-by-bit multiplications mod 2 (popcount() & 0x1) 37 | for (int j = 0; j < 8; j++) { 38 | int v = (__builtin_popcount(i & tal[j]) & 0x1); 39 | convToDual_[i] |= v << (7-j); 40 | } 41 | 42 | // Inverse mapping 43 | dualToConv_[convToDual_[i]] = i; 44 | } 45 | 46 | // Initialize Reed-Solomon decoder 47 | rs_ = correct_reed_solomon_create( 48 | correct_rs_primitive_polynomial_ccsds, 49 | 112, 50 | 11, 51 | 32); 52 | } 53 | 54 | ReedSolomon::~ReedSolomon() { 55 | correct_reed_solomon_destroy(rs_); 56 | } 57 | 58 | int ReedSolomon::run(const uint8_t* data, size_t len, uint8_t* dst) { 59 | std::array tmp1, tmp2; 60 | int err = 0; 61 | 62 | // Expect 4x 255 byte block (223 data + 32 parity) 63 | ASSERT(len == 1020); 64 | 65 | // Process block by block 66 | for (auto i = 0; i < 4; i++) { 67 | // Deinterleave and convert 68 | for (auto j = 0; j < 255; j++) { 69 | tmp1[j] = dualToConv_[data[(j * 4) + i]]; 70 | } 71 | 72 | // Run Reed-Solomon (in conventional representation) 73 | auto rv = correct_reed_solomon_decode(rs_, tmp1.data(), tmp1.size(), tmp2.data()); 74 | if (rv == -1) { 75 | return -1; 76 | } 77 | 78 | // Count number of corrected errors 79 | for (auto j = 0; j < (255 - 32); j++) { 80 | if (tmp1[j] != tmp2[j]) { 81 | err++; 82 | } 83 | } 84 | 85 | // Convert and interleave (ignoring parity) 86 | for (auto j = 0; j < (255 - 32); j++) { 87 | dst[(j * 4) + i] = convToDual_[tmp2[j]]; 88 | } 89 | } 90 | 91 | return err; 92 | } 93 | 94 | } // namespace decoder 95 | -------------------------------------------------------------------------------- /src/decoder/compute_sync_words.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | 4 | #include 5 | #include 6 | 7 | #include 8 | 9 | #include "viterbi.h" 10 | 11 | std::array nrzmEncode(std::array in, int b) { 12 | std::array out; 13 | 14 | // Iterate over bytes 15 | for (unsigned i = 0; i < in.size(); i++) { 16 | out[i] = 0; 17 | 18 | // Iterate over bits 19 | for (unsigned j = 0; j < 8; j++) { 20 | auto bit = (in[i] >> (7-j)) & 0x1; 21 | if (bit) { 22 | // Bit is 1; flip 23 | b = 1 - b; 24 | } 25 | out[i] = (out[i] << 1) | (b & 0x1); 26 | } 27 | } 28 | return out; 29 | } 30 | 31 | int main(int argc, char** argv) { 32 | decoder::Viterbi v; 33 | 34 | // Use array of bytes instead of fundamental type so we 35 | // don't need to worry about host byte order. 36 | std::array syncWord; 37 | syncWord[0] = 0x1A; 38 | syncWord[1] = 0xCF; 39 | syncWord[2] = 0xFC; 40 | syncWord[3] = 0x1D; 41 | 42 | // LRIT uses NRZ-L "coding" (which is really no coding at all). 43 | // To get the encoded sync word we can directly Viterbi encode it. 44 | // Then we can invert it to get its dual to deal with phase 45 | // ambiguity. 46 | auto len = v.encodeLength(syncWord.size()); 47 | std::vector buf(len); 48 | auto rv = v.encode(syncWord.data(), syncWord.size(), buf.data()); 49 | ASSERT(rv == len); 50 | 51 | // 0 degree phase shift 52 | printf("LRIT: phase 0: 0x"); 53 | for (unsigned i = 0; i < 8; i++) { 54 | printf("%02x", (uint8_t) buf[i]); 55 | } 56 | printf("\n"); 57 | 58 | // 180 degree phase shift 59 | printf("LRIT: phase 180: 0x"); 60 | for (unsigned i = 0; i < 8; i++) { 61 | printf("%02x", (uint8_t) ~buf[i]); 62 | } 63 | printf("\n"); 64 | 65 | // HRIT uses NRZ-M coding. In this scheme, a 0 means no bit change, 66 | // and a 1 means a bit change. We have to get an encoded sync word 67 | // for 2 different initial states to deal with phase ambiguity. 68 | for (unsigned i = 0; i < 2; i++) { 69 | auto nrzmSyncWord = nrzmEncode(syncWord, i); 70 | auto len = v.encodeLength(nrzmSyncWord.size()); 71 | std::vector buf(len); 72 | auto rv = v.encode(nrzmSyncWord.data(), nrzmSyncWord.size(), buf.data()); 73 | ASSERT(rv == len); 74 | 75 | if (i == 0) { 76 | // 0 degree phase shift 77 | printf("HRIT: phase 0: 0x"); 78 | } else { 79 | // 180 degree phase shift 80 | printf("HRIT: phase 180: 0x"); 81 | } 82 | for (unsigned i = 0; i < 8; i++) { 83 | printf("%02x", (uint8_t) buf[i]); 84 | } 85 | printf("\n"); 86 | } 87 | } 88 | -------------------------------------------------------------------------------- /src/goesrecv/queue.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | template 11 | class Queue { 12 | public: 13 | Queue(size_t capacity) : 14 | elements_(0), 15 | capacity_(capacity), 16 | closed_(false) { 17 | } 18 | 19 | size_t size() { 20 | std::unique_lock lock(m_); 21 | return elements_; 22 | } 23 | 24 | bool closed() { 25 | std::unique_lock lock(m_); 26 | return closed_; 27 | } 28 | 29 | void close() { 30 | std::unique_lock lock(m_); 31 | closed_ = true; 32 | cv_.notify_one(); 33 | } 34 | 35 | // popForWrite returns existing item to write to 36 | std::unique_ptr popForWrite() { 37 | std::unique_lock lock(m_); 38 | ASSERT(!closed_); 39 | 40 | // Ensure there is an item to return 41 | if (write_.size() == 0) { 42 | if (elements_ < capacity_) { 43 | elements_++; 44 | write_.push_back(std::make_unique()); 45 | } else { 46 | // Wait until pushRead makes an item available 47 | while (write_.size() == 0) { 48 | cv_.wait(lock); 49 | } 50 | } 51 | } 52 | 53 | auto v = std::move(write_.front()); 54 | write_.pop_front(); 55 | return v; 56 | } 57 | 58 | // pushWrite returns written item to read queue 59 | void pushWrite(std::unique_ptr v) { 60 | std::unique_lock lock(m_); 61 | ASSERT(!closed_); 62 | 63 | read_.push_back(std::move(v)); 64 | cv_.notify_one(); 65 | } 66 | 67 | // popForRead returns existing item to read from 68 | std::unique_ptr popForRead() { 69 | std::unique_lock lock(m_); 70 | while (read_.size() == 0 && !closed_) { 71 | cv_.wait(lock); 72 | } 73 | 74 | // Allow read side to drain 75 | if (read_.size() == 0 && closed_) { 76 | return std::unique_ptr(nullptr); 77 | } 78 | 79 | auto v = std::move(read_.front()); 80 | read_.pop_front(); 81 | return v; 82 | } 83 | 84 | // pushRead returns read item to write queue 85 | void pushRead(std::unique_ptr v) { 86 | std::unique_lock lock(m_); 87 | if (!closed_) { 88 | write_.push_back(std::move(v)); 89 | cv_.notify_one(); 90 | } 91 | } 92 | 93 | protected: 94 | std::mutex m_; 95 | std::condition_variable cv_; 96 | 97 | size_t elements_; 98 | size_t capacity_; 99 | bool closed_; 100 | 101 | std::deque > write_; 102 | std::deque > read_; 103 | }; 104 | -------------------------------------------------------------------------------- /src/assembler/session_pdu.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | extern "C" { 8 | #include 9 | } 10 | 11 | #include 12 | 13 | #include "lrit/lrit.h" 14 | 15 | #include "transport_pdu.h" 16 | 17 | namespace assembler { 18 | 19 | // Compute difference between two integers taking into account wrapping. 20 | template 21 | int diffWithWrap(unsigned int a, unsigned int b) { 22 | int skip; 23 | 24 | ASSERT(a < N); 25 | ASSERT(b < N); 26 | 27 | if (a <= b) { 28 | skip = b - a; 29 | } else { 30 | skip = N - a + b; 31 | } 32 | 33 | return skip; 34 | } 35 | 36 | class SessionPDU { 37 | public: 38 | explicit SessionPDU(int vcid, int apid); 39 | 40 | // Returns false if this T_PDU could not be added. 41 | // This is the case if -- for example -- it contains a 42 | // malformed header, or cannot be decompressed. 43 | bool append(const TransportPDU& tpdu); 44 | 45 | // Returns true if this session PDU could be finished 46 | // without extra T_PDUs. This is only the case if it 47 | // contains a line-by-line encoded image. 48 | bool finish(); 49 | 50 | std::string getName() const; 51 | 52 | bool hasCompleteHeader() const { 53 | return !m_.empty(); 54 | } 55 | 56 | template 57 | bool hasHeader() const { 58 | return lrit::hasHeader(m_); 59 | } 60 | 61 | template 62 | H getHeader() const { 63 | return lrit::getHeader(buf_, m_); 64 | } 65 | 66 | const std::vector& get() const { 67 | return buf_; 68 | } 69 | 70 | const size_t size() const { 71 | return buf_.size(); 72 | } 73 | 74 | const lrit::HeaderMap& getHeaderMap() const { 75 | return m_; 76 | } 77 | 78 | const lrit::PrimaryHeader& getPrimaryHeader() const { 79 | return ph_; 80 | } 81 | 82 | const int vcid; 83 | const int apid; 84 | 85 | protected: 86 | bool completeHeader(); 87 | 88 | bool append( 89 | std::vector::const_iterator begin, 90 | std::vector::const_iterator end); 91 | 92 | std::vector buf_; 93 | uint64_t remainingHeaderBytes_; 94 | uint32_t lastSequenceCount_; 95 | 96 | lrit::HeaderMap m_; 97 | lrit::PrimaryHeader ph_; 98 | lrit::ImageStructureHeader ish_; 99 | 100 | std::unique_ptr szParam_; 101 | std::vector szTmp_; 102 | 103 | private: 104 | void skipLines(int skip); 105 | 106 | // Number of lines that are present in the buffer. 107 | // Only applicable for line-by-line encoded images. 108 | uint32_t linesDone_; 109 | }; 110 | 111 | } // namespace assembler 112 | -------------------------------------------------------------------------------- /docs/commands/goeslrit.rst: -------------------------------------------------------------------------------- 1 | .. _goeslrit: 2 | 3 | goeslrit 4 | ======== 5 | 6 | .. note:: 7 | 8 | If you're only interested in post-processed data (image and text 9 | files) you can ignore this command and continue reading about 10 | goesproc instead. 11 | 12 | This tool can be used to turn a stream of packets into LRIT files. 13 | LRIT files can then be used by goesproc (or other tools, such as to 14 | generate usable images and text files, or for debugging purposes. 15 | 16 | It can either read packets from files to process recorded data, or 17 | subscribe to :ref:`goesrecv` process to work with live data. 18 | 19 | To make it write LRIT files to disk, you have to specify the category 20 | of files you are interested in. Pass ``--help`` for a list of 21 | filtering options. To make it write **ALL** LRIT files it seems, run 22 | goeslrit with the ``--all`` option. 23 | 24 | Reading packets from files 25 | -------------------------- 26 | 27 | The files must be specified in chronological order because they are 28 | read in order. Packets for a single LRIT file can span multiple packet 29 | files, so if they are not specified in chronological order some LRIT 30 | files will be dropped. Specifying a file glob in Bash expands to an 31 | alphabetically sorted list of file names that match the pattern. 32 | 33 | Example:: 34 | 35 | $ goeslrit --images /path/to/packets/packets-2018-02-28T* 36 | Reading: /path/to/packets/packets-2018-02-28T00:00:00Z.raw 37 | Writing: OR_ABI-L2-CMIPM1-M3C02_G16_s20180582358300_e20180582358358_c20180582358429.lrit (4004087 bytes) 38 | Writing: OR_ABI-L2-CMIPM2-M3C07_G16_s20180590000000_e20180590000071_c20180590000108.lrit (254551 bytes) 39 | ... 40 | 41 | Reading packets from goesrecv 42 | ----------------------------- 43 | 44 | Subscribe to the :ref:`goesrecv` packet publisher to process live data. 45 | 46 | This is done using the ``--subscribe`` option, which takes a valid 47 | *nanomsg* address. This is typically a TCP address if goeslrit runs on 48 | a different machine than goesrecv. If they run on the same machine, it 49 | can either be a TCP address or an IPC address. 50 | 51 | The address takes the following form: 52 | 53 | * :code:`tcp://:` -- connect to goesrecv over the network. 54 | Also see `nn_tcp(7) `_. 55 | * :code:`ipc://path/to/socket` -- connect to goesrecv on same machine. 56 | Also see `nn_ipc(7) `_. 57 | 58 | Example:: 59 | 60 | $ goeslrit --images --subscribe tcp://1.2.3.4:5005 61 | Writing: OR_ABI-L2-CMIPM1-M3C02_G16_s20180591958303_e20180591958360_c20180591958427.lrit (4004087 bytes) 62 | Writing: OR_ABI-L2-CMIPM2-M3C07_G16_s20180592000003_e20180592000073_c20180592000110.lrit (254551 bytes) 63 | ... 64 | -------------------------------------------------------------------------------- /src/assembler/crc.cc: -------------------------------------------------------------------------------- 1 | #include "crc.h" 2 | 3 | namespace assembler { 4 | 5 | namespace { 6 | 7 | // CRC table and implementation from "NOAA GOES LRIT Mission Specific Data". 8 | // See http://www.noaasis.noaa.gov/LRIT/pdf-files/5_LRIT_Mission-data.pdf 9 | uint16_t table[] = { 10 | 0x0000, 0x1021, 0x2042, 0x3063, 0x4084, 0x50A5, 0x60C6, 0x70E7, 11 | 0x8108, 0x9129, 0xA14A, 0xB16B, 0xC18C, 0xD1AD, 0xE1CE, 0xF1EF, 12 | 0x1231, 0x0210, 0x3273, 0x2252, 0x52B5, 0x4294, 0x72F7, 0x62D6, 13 | 0x9339, 0x8318, 0xB37B, 0xA35A, 0xD3BD, 0xC39C, 0xF3FF, 0xE3DE, 14 | 0x2462, 0x3443, 0x0420, 0x1401, 0x64E6, 0x74C7, 0x44A4, 0x5485, 15 | 0xA56A, 0xB54B, 0x8528, 0x9509, 0xE5EE, 0xF5CF, 0xC5AC, 0xD58D, 16 | 0x3653, 0x2672, 0x1611, 0x0630, 0x76D7, 0x66F6, 0x5695, 0x46B4, 17 | 0xB75B, 0xA77A, 0x9719, 0x8738, 0xF7DF, 0xE7FE, 0xD79D, 0xC7BC, 18 | 0x48C4, 0x58E5, 0x6886, 0x78A7, 0x0840, 0x1861, 0x2802, 0x3823, 19 | 0xC9CC, 0xD9ED, 0xE98E, 0xF9AF, 0x8948, 0x9969, 0xA90A, 0xB92B, 20 | 0x5AF5, 0x4AD4, 0x7AB7, 0x6A96, 0x1A71, 0x0A50, 0x3A33, 0x2A12, 21 | 0xDBFD, 0xCBDC, 0xFBBF, 0xEB9E, 0x9B79, 0x8B58, 0xBB3B, 0xAB1A, 22 | 0x6CA6, 0x7C87, 0x4CE4, 0x5CC5, 0x2C22, 0x3C03, 0x0C60, 0x1C41, 23 | 0xEDAE, 0xFD8F, 0xCDEC, 0xDDCD, 0xAD2A, 0xBD0B, 0x8D68, 0x9D49, 24 | 0x7E97, 0x6EB6, 0x5ED5, 0x4EF4, 0x3E13, 0x2E32, 0x1E51, 0x0E70, 25 | 0xFF9F, 0xEFBE, 0xDFDD, 0xCFFC, 0xBF1B, 0xAF3A, 0x9F59, 0x8F78, 26 | 0x9188, 0x81A9, 0xB1CA, 0xA1EB, 0xD10C, 0xC12D, 0xF14E, 0xE16F, 27 | 0x1080, 0x00A1, 0x30C2, 0x20E3, 0x5004, 0x4025, 0x7046, 0x6067, 28 | 0x83B9, 0x9398, 0xA3FB, 0xB3DA, 0xC33D, 0xD31C, 0xE37F, 0xF35E, 29 | 0x02B1, 0x1290, 0x22F3, 0x32D2, 0x4235, 0x5214, 0x6277, 0x7256, 30 | 0xB5EA, 0xA5CB, 0x95A8, 0x8589, 0xF56E, 0xE54F, 0xD52C, 0xC50D, 31 | 0x34E2, 0x24C3, 0x14A0, 0x0481, 0x7466, 0x6447, 0x5424, 0x4405, 32 | 0xA7DB, 0xB7FA, 0x8799, 0x97B8, 0xE75F, 0xF77E, 0xC71D, 0xD73C, 33 | 0x26D3, 0x36F2, 0x0691, 0x16B0, 0x6657, 0x7676, 0x4615, 0x5634, 34 | 0xD94C, 0xC96D, 0xF90E, 0xE92F, 0x99C8, 0x89E9, 0xB98A, 0xA9AB, 35 | 0x5844, 0x4865, 0x7806, 0x6827, 0x18C0, 0x08E1, 0x3882, 0x28A3, 36 | 0xCB7D, 0xDB5C, 0xEB3F, 0xFB1E, 0x8BF9, 0x9BD8, 0xABBB, 0xBB9A, 37 | 0x4A75, 0x5A54, 0x6A37, 0x7A16, 0x0AF1, 0x1AD0, 0x2AB3, 0x3A92, 38 | 0xFD2E, 0xED0F, 0xDD6C, 0xCD4D, 0xBDAA, 0xAD8B, 0x9DE8, 0x8DC9, 39 | 0x7C26, 0x6C07, 0x5C64, 0x4C45, 0x3CA2, 0x2C83, 0x1CE0, 0x0CC1, 40 | 0xEF1F, 0xFF3E, 0xCF5D, 0xDF7C, 0xAF9B, 0xBFBA, 0x8FD9, 0x9FF8, 41 | 0x6E17, 0x7E36, 0x4E55, 0x5E74, 0x2E93, 0x3EB2, 0x0ED1, 0x1EF0, 42 | }; 43 | 44 | } // namespace 45 | 46 | uint16_t crc(const uint8_t* buf, size_t len) { 47 | uint16_t crc = 0xffff; 48 | for (uint16_t i = 0; i < len; i++) { 49 | crc = (crc<<8)^table[(crc>>8)^(uint16_t)buf[i]]; 50 | } 51 | return crc; 52 | } 53 | 54 | } // namespace assembler 55 | -------------------------------------------------------------------------------- /src/goesproc/file_writer.cc: -------------------------------------------------------------------------------- 1 | #include "file_writer.h" 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | #include "lrit/json.h" 11 | #include "string.h" 12 | 13 | using namespace util; 14 | 15 | FileWriter::FileWriter(const std::string& prefix) : prefix_(prefix) { 16 | force_ = false; 17 | } 18 | 19 | FileWriter::~FileWriter() { 20 | } 21 | 22 | void FileWriter::logTime(const Timer* t) { 23 | if (t) { 24 | std::cout 25 | << std::fixed 26 | << std::setprecision(3) 27 | << " (took " 28 | << t->elapsed().count() 29 | << "s)" 30 | << std::endl; 31 | } else { 32 | std::cout << std::endl; 33 | } 34 | } 35 | 36 | void FileWriter::write( 37 | const std::string& tail, 38 | const cv::Mat& mat, 39 | const Timer* t) { 40 | auto path = buildPath(tail); 41 | if (!tryWrite(path)) { 42 | std::cout << "Skipping (file exists): " << path; 43 | logTime(t); 44 | return; 45 | } 46 | 47 | std::cout << "Writing: " << path; 48 | cv::imwrite(path, mat); 49 | logTime(t); 50 | } 51 | 52 | void FileWriter::write( 53 | const std::string& tail, 54 | const std::vector& data, 55 | const Timer* t) { 56 | auto path = buildPath(tail); 57 | if (!tryWrite(path)) { 58 | std::cout << "Skipping (file exists): " << path; 59 | logTime(t); 60 | return; 61 | } 62 | 63 | std::cout << "Writing: " << path; 64 | std::ofstream of(path); 65 | of.write(data.data(), data.size()); 66 | logTime(t); 67 | } 68 | 69 | void FileWriter::write( 70 | const std::string& tail, 71 | const nlohmann::json& json, 72 | const Timer* t) { 73 | auto path = buildPath(tail); 74 | if (!tryWrite(path)) { 75 | std::cout << "Skipping (file exists): " << path; 76 | logTime(t); 77 | return; 78 | } 79 | 80 | std::cout << "Writing: " << path; 81 | std::ofstream of(path); 82 | of << json; 83 | logTime(t); 84 | } 85 | 86 | void FileWriter::writeHeader(const lrit::File& file, const std::string& path) { 87 | auto jsonHeader = lrit::toJSON(file); 88 | jsonHeader["Path"] = buildPath(path); 89 | auto jsonPath = removeSuffix(path) + ".json"; 90 | write(jsonPath, jsonHeader); 91 | } 92 | 93 | bool FileWriter::tryWrite(const std::string& path) { 94 | struct stat st; 95 | 96 | auto rpos = path.rfind('/'); 97 | if (rpos != std::string::npos) { 98 | mkdirp(path.substr(0, rpos)); 99 | } 100 | 101 | auto rv = stat(path.c_str(), &st); 102 | if (rv < 0 && errno == ENOENT) { 103 | return true; 104 | } 105 | 106 | return force_; 107 | } 108 | 109 | std::string FileWriter::buildPath(const std::string& path) { 110 | if (prefix_ == ".") { 111 | return path; 112 | } 113 | return prefix_ + "/" + path; 114 | } 115 | -------------------------------------------------------------------------------- /src/goesrecv/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | find_package(PkgConfig) 2 | 3 | add_library(publisher 4 | packet_publisher.cc 5 | publisher.cc 6 | sample_publisher.cc 7 | soft_bit_publisher.cc 8 | stats_publisher.cc 9 | ) 10 | target_link_libraries(publisher nanomsg) 11 | 12 | pkg_check_modules(AIRSPY libairspy) 13 | if(NOT AIRSPY_FOUND) 14 | message(WARNING "Unable to find libairspy") 15 | else() 16 | add_library(airspy_source airspy_source.cc) 17 | target_link_libraries(airspy_source ${AIRSPY_LIBRARIES} publisher stdc++) 18 | endif() 19 | 20 | pkg_check_modules(RTLSDR librtlsdr) 21 | if(NOT RTLSDR_FOUND) 22 | message(WARNING "Unable to find librtlsdr") 23 | else() 24 | add_library(rtlsdr_source rtlsdr_source.cc) 25 | if((RTLSDR_VERSION VERSION_EQUAL 0.5.4) OR 26 | (RTLSDR_VERSION VERSION_GREATER 0.5.4)) 27 | target_compile_definitions(rtlsdr_source PRIVATE RTLSDR_HAS_BIAS_TEE) 28 | endif() 29 | target_link_libraries(rtlsdr_source ${RTLSDR_LIBRARIES} publisher stdc++) 30 | endif() 31 | 32 | add_library(nanomsg_source nanomsg_source.cc) 33 | target_link_libraries(nanomsg_source nanomsg publisher stdc++) 34 | 35 | add_library(agc agc.cc) 36 | target_link_libraries(agc publisher m stdc++) 37 | 38 | add_library(rrc rrc.cc) 39 | target_link_libraries(rrc publisher stdc++) 40 | 41 | add_library(costas costas.cc) 42 | target_link_libraries(costas publisher stdc++) 43 | 44 | add_library(clock_recovery clock_recovery.cc) 45 | target_link_libraries(clock_recovery publisher stdc++) 46 | 47 | add_library(quantize quantize.cc) 48 | target_link_libraries(quantize publisher stdc++) 49 | 50 | add_executable(goesrecv goesrecv.cc config.cc options.cc decoder.cc demodulator.cc monitor.cc datagram_socket.cc source.cc) 51 | install(TARGETS goesrecv COMPONENT goestools RUNTIME DESTINATION bin) 52 | target_include_directories(goesrecv PRIVATE ${PROJECT_SOURCE_DIR}/src) 53 | target_link_libraries(goesrecv util) 54 | target_link_libraries(goesrecv nlohmann_json) 55 | target_link_libraries(goesrecv packetizer pthread) 56 | target_link_libraries(goesrecv agc) 57 | target_link_libraries(goesrecv rrc) 58 | target_link_libraries(goesrecv costas) 59 | target_link_libraries(goesrecv clock_recovery) 60 | target_link_libraries(goesrecv quantize) 61 | target_link_libraries(goesrecv nanomsg_source) 62 | target_link_libraries(goesrecv version) 63 | if(AIRSPY_FOUND) 64 | target_compile_definitions(goesrecv PUBLIC -DBUILD_AIRSPY) 65 | target_link_libraries(goesrecv airspy_source) 66 | endif() 67 | if(RTLSDR_FOUND) 68 | target_compile_definitions(goesrecv PUBLIC -DBUILD_RTLSDR) 69 | target_link_libraries(goesrecv rtlsdr_source) 70 | endif() 71 | 72 | add_executable(benchmark benchmark.cc) 73 | target_link_libraries(benchmark pthread) 74 | target_link_libraries(benchmark agc) 75 | target_link_libraries(benchmark rrc) 76 | target_link_libraries(benchmark costas) 77 | target_link_libraries(benchmark clock_recovery) 78 | -------------------------------------------------------------------------------- /src/goesrecv/agc.cc: -------------------------------------------------------------------------------- 1 | #include "agc.h" 2 | 3 | #include 4 | 5 | #ifdef __ARM_NEON 6 | #include 7 | #endif 8 | 9 | AGC::AGC() { 10 | min_ = 1e-6f; 11 | max_ = 1e+6f; 12 | alpha_ = 1e-4f; 13 | gain_ = 1e0f; 14 | } 15 | 16 | #ifdef __ARM_NEON 17 | 18 | void AGC::work( 19 | size_t nsamples, 20 | std::complex* ci, 21 | std::complex* co) { 22 | float* fi = (float*) ci; 23 | float* fo = (float*) co; 24 | 25 | float32x4_t min = vld1q_dup_f32(&min_); 26 | float32x4_t max = vld1q_dup_f32(&max_); 27 | float32x4_t gain = vld1q_dup_f32(&gain_); 28 | 29 | // Process 4 samples at a time. 30 | for (size_t i = 0; i < nsamples; i += 4) { 31 | float32x4x2_t f = vld2q_f32(&fi[2*i]); 32 | 33 | // Apply gain. 34 | f.val[0] = vmulq_f32(f.val[0], gain); 35 | f.val[1] = vmulq_f32(f.val[1], gain); 36 | vst2q_f32(&fo[2*i], f); 37 | 38 | // Compute signal magnitude. 39 | float32x4_t x2 = 40 | vaddq_f32( 41 | vmulq_f32(f.val[0], f.val[0]), 42 | vmulq_f32(f.val[1], f.val[1])); 43 | 44 | // Update gain. 45 | // Use only the first sample and ignore the others. 46 | float32x4_t delta = vdupq_n_f32(alpha_ * (0.5 - sqrtf(x2[0]))); 47 | gain = vaddq_f32(gain, delta); 48 | gain = vmaxq_f32(gain, min); 49 | gain = vminq_f32(gain, max); 50 | } 51 | 52 | // Write back to instance variable 53 | gain_ = gain[0]; 54 | } 55 | 56 | #else 57 | 58 | void AGC::work( 59 | size_t nsamples, 60 | std::complex* ci, 61 | std::complex* co) { 62 | // Process 4 samples at a time. 63 | for (size_t i = 0; i < nsamples; i += 4) { 64 | // Apply gain 65 | co[i+0] = ci[i+0] * gain_; 66 | co[i+1] = ci[i+1] * gain_; 67 | co[i+2] = ci[i+2] * gain_; 68 | co[i+3] = ci[i+3] * gain_; 69 | 70 | // Update gain. 71 | // Use only the first sample and ignore the others. 72 | gain_ += alpha_ * (0.5 - abs(co[i])); 73 | gain_ = std::max(gain_, min_); 74 | gain_ = std::min(gain_, max_); 75 | } 76 | } 77 | 78 | #endif 79 | 80 | void AGC::work( 81 | const std::shared_ptr >& qin, 82 | const std::shared_ptr >& qout) { 83 | auto input = qin->popForRead(); 84 | if (!input) { 85 | qout->close(); 86 | return; 87 | } 88 | 89 | auto output = qout->popForWrite(); 90 | auto nsamples = input->size(); 91 | output->resize(nsamples); 92 | 93 | // Do actual work 94 | auto ci = input->data(); 95 | auto co = output->data(); 96 | work(nsamples, ci, co); 97 | 98 | // Return input buffer 99 | qin->pushRead(std::move(input)); 100 | 101 | // Publish output if applicable 102 | if (samplePublisher_) { 103 | samplePublisher_->publish(*output); 104 | } 105 | 106 | // Return output buffer 107 | qout->pushWrite(std::move(output)); 108 | } 109 | -------------------------------------------------------------------------------- /src/lib/zip.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | #include 8 | 9 | // Minimal implementation of a ZIP file reader. This only works for 10 | // ZIP files that contain a single file, as is the case for EMWIN 11 | // files. I have not tested this on other files. 12 | // 13 | // The struct definitions below are taken from the code described at: 14 | // http://codeandlife.com/2014/01/01/unzip-library-for-c/ 15 | // 16 | class Zip { 17 | public: 18 | // SignatureError is thrown if a signature field as defined in the 19 | // ZIP file specification doesn't match what is found in the file. 20 | class SignatureError : public std::exception { 21 | public: 22 | SignatureError(uint32_t expected, uint32_t actual); 23 | 24 | const char* what() const noexcept override; 25 | 26 | private: 27 | uint32_t expected_; 28 | uint32_t actual_; 29 | std::string error_; 30 | }; 31 | 32 | explicit Zip(std::unique_ptr is); 33 | 34 | std::string fileName() const { 35 | return fileName_; 36 | } 37 | 38 | std::vector read() const; 39 | 40 | protected: 41 | std::unique_ptr is_; 42 | 43 | // End Of Central Directory Record 44 | struct __attribute__ ((__packed__)) eocd { 45 | uint8_t signature[4]; 46 | uint16_t diskNumber; 47 | uint16_t centralDirectoryDiskNumber; 48 | uint16_t numEntriesThisDisk; 49 | uint16_t numEntries; 50 | uint32_t centralDirectorySize; 51 | uint32_t centralDirectoryOffset; 52 | uint16_t zipCommentLength; 53 | }; 54 | 55 | eocd eocd_; 56 | 57 | // Central Directory File Header 58 | struct __attribute__ ((__packed__)) cdfh { 59 | uint8_t signature[4]; 60 | uint16_t versionMadeBy; 61 | uint16_t versionNeededToExtract; 62 | uint16_t generalPurposeBitFlag; 63 | uint16_t compressionMethod; 64 | uint16_t lastModFileTime; 65 | uint16_t lastModFileDate; 66 | uint32_t crc32; 67 | uint32_t compressedSize; 68 | uint32_t uncompressedSize; 69 | uint16_t fileNameLength; 70 | uint16_t extraFieldLength; 71 | uint16_t fileCommentLength; 72 | uint16_t diskNumberStart; 73 | uint16_t internalFileAttributes; 74 | uint32_t externalFileAttributes; 75 | uint32_t relativeOffsetOflocalHeader; 76 | }; 77 | 78 | cdfh cdfh_; 79 | 80 | // Local File Header 81 | struct __attribute__ ((__packed__)) lfh { 82 | uint8_t signature[4]; 83 | uint16_t versionNeededToExtract; 84 | uint16_t generalPurposeBitFlag; 85 | uint16_t compressionMethod; 86 | uint16_t lastModFileTime; 87 | uint16_t lastModFileDate; 88 | uint32_t crc32; 89 | uint32_t compressedSize; 90 | uint32_t uncompressedSize; 91 | uint16_t fileNameLength; 92 | uint16_t extraFieldLength; 93 | }; 94 | 95 | lfh lfh_; 96 | 97 | std::string fileName_; 98 | std::string extraField_; 99 | }; 100 | -------------------------------------------------------------------------------- /src/goesproc/handler_nws_image.cc: -------------------------------------------------------------------------------- 1 | #include "handler_nws_image.h" 2 | 3 | #include "filename.h" 4 | #include "image.h" 5 | #include "string.h" 6 | 7 | namespace { 8 | 9 | std::string parseTime(const std::string& text, struct timespec& time) { 10 | // This field used an irregular pattern before November 2020. 11 | // See https://github.com/pietern/goestools/issues/100 for historical context. 12 | const char* buf = text.c_str(); 13 | const char* format = "%Y%m%d%H%M%S"; 14 | struct tm tm; 15 | memset(&tm, 0, sizeof(tm)); 16 | const auto ptr = strptime(buf, format, &tm); 17 | 18 | // Only use time if strptime was successful. 19 | // Format with zero padding is always 14 characters. 20 | // The character after the time must be '-'. 21 | if (ptr != (buf + 14) || ptr[0] != '-') { 22 | return text; 23 | } 24 | 25 | time.tv_sec = mktime(&tm); 26 | time.tv_nsec = 0; 27 | 28 | // Return everything after the separator. 29 | return std::string(&ptr[1]); 30 | } 31 | 32 | } // namespace 33 | 34 | NWSImageHandler::NWSImageHandler( 35 | const Config::Handler& config, 36 | const std::shared_ptr& fileWriter) 37 | : config_(config), 38 | fileWriter_(fileWriter) { 39 | } 40 | 41 | void NWSImageHandler::handle(std::shared_ptr f) { 42 | auto ph = f->getHeader(); 43 | if (ph.fileType != 0) { 44 | return; 45 | } 46 | 47 | // Filter NWS 48 | auto nlh = f->getHeader(); 49 | if (nlh.productID != 6) { 50 | return; 51 | } 52 | 53 | FilenameBuilder fb; 54 | fb.dir = config_.dir; 55 | fb.filename = getBasename(*f); 56 | 57 | // In the GOES-15 LRIT stream these text files have a time stamp 58 | // header; in the GOES-R HRIT stream they don't. 59 | if (f->hasHeader()) { 60 | fb.time = f->getHeader().getUnix(); 61 | } else { 62 | // If time can successfully be extracted from the filename 63 | // then remove it from the filename passed to the builder. 64 | fb.filename = parseTime(fb.filename, fb.time); 65 | } 66 | 67 | // If this is a GIF we can write it directly 68 | if (nlh.noaaSpecificCompression == 5) { 69 | auto path = fb.build(config_.filename, "gif"); 70 | fileWriter_->write(path, f->read()); 71 | if (config_.json) { 72 | fileWriter_->writeHeader(*f, path); 73 | } 74 | return; 75 | } 76 | 77 | auto image = Image::createFromFile(f); 78 | auto path = fb.build(config_.filename, config_.format); 79 | fileWriter_->write(path, image->getRawImage()); 80 | if (config_.json) { 81 | fileWriter_->writeHeader(*f, path); 82 | } 83 | return; 84 | } 85 | 86 | std::string NWSImageHandler::getBasename(const lrit::File& f) const { 87 | auto str = removeSuffix(f.getHeader().text); 88 | 89 | // Use annotation without the "dat327221257926" suffix 90 | auto pos = str.find("dat"); 91 | if (pos != std::string::npos) { 92 | str = str.substr(0, pos); 93 | } 94 | 95 | return str; 96 | } 97 | -------------------------------------------------------------------------------- /src/goesproc/handler_text.cc: -------------------------------------------------------------------------------- 1 | #include "handler_text.h" 2 | 3 | #include 4 | 5 | #include "filename.h" 6 | #include "string.h" 7 | 8 | namespace { 9 | 10 | // Expect the file to be named like this: 11 | // 12 | // 16-TEXTdat_17348_201455.lrit 13 | // 14 | bool goesrParseTextTime(const std::string& name, struct timespec& time) { 15 | auto pos = name.find('_'); 16 | if (pos == std::string::npos) { 17 | return false; 18 | } 19 | 20 | if (pos + 1 >= name.size()) { 21 | return false; 22 | } 23 | 24 | const char* buf = name.c_str() + pos + 1; 25 | const char* format = "%y%j_%H%M%S"; 26 | struct tm tm; 27 | auto ptr = strptime(buf, format, &tm); 28 | 29 | // strptime was successful if it returned a pointer to the next char 30 | if (ptr == nullptr || ptr[0] != '.') { 31 | return false; 32 | } 33 | 34 | time.tv_sec = mktime(&tm); 35 | time.tv_nsec = 0; 36 | return true; 37 | } 38 | 39 | } // namespace 40 | 41 | TextHandler::TextHandler( 42 | const Config::Handler& config, 43 | const std::shared_ptr& fileWriter) 44 | : config_(config), 45 | fileWriter_(fileWriter) { 46 | } 47 | 48 | void TextHandler::handle(std::shared_ptr f) { 49 | auto ph = f->getHeader(); 50 | if (ph.fileType != 2) { 51 | return; 52 | } 53 | 54 | // Filter out NWS 55 | auto nlh = f->getHeader(); 56 | if (nlh.productID == 1) { 57 | // LRIT / GOES-N series. 58 | // All text files on the LRIT stream use the same product ID and 59 | // product sub ID in the NOAA LRIT header. Look at the annotation 60 | // header to see if this is an NWS report or not. 61 | auto text = f->getHeader().text; 62 | const auto prefix = std::string("NWS"); 63 | if (text.compare(0, prefix.size(), prefix) == 0) { 64 | return; 65 | } 66 | } else { 67 | // Something else; ignore 68 | return; 69 | } 70 | 71 | struct timespec time = {0, 0}; 72 | 73 | // In the GOES-15 LRIT stream these text files have a time stamp 74 | // header; in the GOES-R HRIT stream they don't. 75 | if (f->hasHeader()) { 76 | time = f->getHeader().getUnix(); 77 | } else { 78 | auto text = f->getHeader().text; 79 | 80 | // Parse time from file name. 81 | if (!goesrParseTextTime(text, time)) { 82 | // Unable to extract timestamp from file name 83 | } 84 | } 85 | 86 | // Skip if the time could not be determined 87 | if (time.tv_sec == 0) { 88 | return; 89 | } 90 | 91 | FilenameBuilder fb; 92 | fb.dir = config_.dir; 93 | fb.filename = removeSuffix(f->getHeader().text); 94 | fb.time = time; 95 | auto path = fb.build(config_.filename, "txt"); 96 | fileWriter_->write(path, f->read()); 97 | if (config_.json) { 98 | fileWriter_->writeHeader(*f, path); 99 | } 100 | } 101 | -------------------------------------------------------------------------------- /src/goespackets/options.cc: -------------------------------------------------------------------------------- 1 | #include "options.h" 2 | 3 | #include 4 | #include 5 | 6 | #include 7 | 8 | #include "lib/version.h" 9 | 10 | void usage(int argc, char** argv) { 11 | fprintf(stderr, "Usage: %s [OPTIONS] [FILE...]\n", argv[0]); 12 | fprintf(stderr, "Relay and/or record packet stream.\n"); 13 | fprintf(stderr, "\n"); 14 | fprintf(stderr, "Options:\n"); 15 | fprintf(stderr, " --subscribe ADDR Address to subscribe to\n"); 16 | fprintf(stderr, " --publish ADDR Address to re-publish packets to\n"); 17 | fprintf(stderr, " --vcid VCID Virtual Channel ID to filter\n"); 18 | fprintf(stderr, " (can be specified multiple times)\n"); 19 | fprintf(stderr, "\n"); 20 | fprintf(stderr, "Record packet stream:\n"); 21 | fprintf(stderr, " --record Enable recording of packet stream to disk\n"); 22 | fprintf(stderr, " --filename PATTERN Filename pattern for packet files (see strftime(3))\n"); 23 | fprintf(stderr, " (default: ./packets-%%FT%%H:%%M:00.raw)\n"); 24 | fprintf(stderr, "\n"); 25 | fprintf(stderr, "Other:\n"); 26 | fprintf(stderr, " --help Display this help and exit\n"); 27 | fprintf(stderr, " --version Print version information and exit\n"); 28 | fprintf(stderr, "\n"); 29 | fprintf(stderr, "If an address to subscribe to is specified,\n"); 30 | fprintf(stderr, "FILE arguments are ignored.\n"); 31 | fprintf(stderr, "\n"); 32 | exit(0); 33 | } 34 | 35 | Options parseOptions(int argc, char** argv) { 36 | Options opts; 37 | 38 | while (1) { 39 | static struct option longOpts[] = { 40 | {"subscribe", required_argument, 0, 0x1001}, 41 | {"vcid", required_argument, 0, 0x1002}, 42 | {"publish", required_argument, 0, 0x1003}, 43 | {"record" , no_argument, 0, 0x1004}, 44 | {"filename" , required_argument, 0, 0x1005}, 45 | {"help", no_argument, nullptr, 0x1337}, 46 | {"version", no_argument, nullptr, 0x1338}, 47 | {nullptr, 0, nullptr, 0}, 48 | }; 49 | 50 | auto c = getopt_long(argc, argv, "", longOpts, nullptr); 51 | if (c == -1) { 52 | break; 53 | } 54 | 55 | switch (c) { 56 | case 0: 57 | break; 58 | case 0x1001: 59 | opts.subscribe = optarg; 60 | break; 61 | case 0x1002: 62 | opts.vcids.insert(atoi(optarg)); 63 | break; 64 | case 0x1003: 65 | opts.publish.push_back(optarg); 66 | break; 67 | case 0x1004: 68 | opts.record = true; 69 | break; 70 | case 0x1005: 71 | opts.filename = optarg; 72 | break; 73 | case 0x1337: 74 | usage(argc, argv); 75 | break; 76 | case 0x1338: 77 | version(argc, argv); 78 | exit(0); 79 | break; 80 | default: 81 | std::cerr << "Invalid option" << std::endl; 82 | exit(1); 83 | } 84 | } 85 | 86 | for (int i = optind; i < argc; i++) { 87 | opts.files.push_back(argv[i]); 88 | } 89 | 90 | return opts; 91 | } 92 | -------------------------------------------------------------------------------- /src/decoder/packetdump.cc: -------------------------------------------------------------------------------- 1 | #include 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | #include "packetizer.h" 11 | #include "reader.h" 12 | 13 | // Read from file descriptor. 14 | // Stores time when most recent read completed. 15 | // This is used to name output files. 16 | class FileReader : public decoder::Reader { 17 | public: 18 | FileReader(int fd) : 19 | fd_(fd) {} 20 | 21 | time_t lastRead() const { 22 | return t_; 23 | } 24 | 25 | virtual size_t read(void* buf, size_t count) { 26 | size_t nread = 0; 27 | while (nread < count) { 28 | auto rv = ::read(fd_, ((char*) buf) + nread, count - nread); 29 | // Terminate on error 30 | if (rv < 0) { 31 | perror("read"); 32 | exit(1); 33 | } 34 | // Return on EOF 35 | if (rv == 0) { 36 | return nread; 37 | } 38 | nread += rv; 39 | } 40 | t_ = time(0); 41 | return nread; 42 | } 43 | 44 | private: 45 | int fd_; 46 | time_t t_; 47 | }; 48 | 49 | class FileWriter { 50 | public: 51 | explicit FileWriter(std::string path) 52 | : path_(path) { 53 | fileName_ = ""; 54 | fileTime_ = 0; 55 | } 56 | 57 | void write(const std::array& buf, time_t t) { 58 | // Round time down to 5 minute boundary 59 | t = t - (t % 300); 60 | 61 | // Open new file if necessary 62 | if (t != fileTime_) { 63 | if (of_.good()) { 64 | of_.close(); 65 | } 66 | 67 | fileName_ = timeToFileName(t); 68 | fileTime_ = t; 69 | of_.open(fileName_, std::ofstream::out | std::ofstream::app); 70 | ASSERT(of_.good()); 71 | 72 | std::cout 73 | << "Writing to file: " 74 | << fileName_ 75 | << std::endl; 76 | } 77 | 78 | of_.write((const char*) buf.data(), buf.size()); 79 | } 80 | 81 | protected: 82 | std::string path_; 83 | std::string fileName_; 84 | time_t fileTime_; 85 | std::ofstream of_; 86 | 87 | std::string timeToFileName(time_t t) { 88 | std::array tsbuf; 89 | auto len = strftime( 90 | tsbuf.data(), 91 | tsbuf.size(), 92 | "packets-%FT%TZ.raw", 93 | gmtime(&t)); 94 | return path_ + "/" + std::string(tsbuf.data(), len); 95 | } 96 | }; 97 | 98 | int main(int argc, char** argv) { 99 | auto reader = std::make_shared(0); 100 | auto writer = std::make_shared("."); 101 | decoder::Packetizer p(reader); 102 | decoder::Packetizer::Details details; 103 | std::array buf; 104 | for (;;) { 105 | auto ok = p.nextPacket(buf, &details); 106 | if (!ok) { 107 | break; 108 | } 109 | 110 | if (details.reedSolomonBytes > 0) { 111 | std::cerr << "RS corrected " << details.reedSolomonBytes << " bytes" << std::endl; 112 | } else if (details.reedSolomonBytes < 0) { 113 | std::cerr << "RS unable to correct packet; dropping!" << std::endl; 114 | } 115 | 116 | if (details.ok) { 117 | writer->write(buf, reader->lastRead()); 118 | } 119 | } 120 | } 121 | -------------------------------------------------------------------------------- /src/goesemwin/qbt.cc: -------------------------------------------------------------------------------- 1 | #include "qbt.h" 2 | 3 | #include 4 | 5 | namespace qbt { 6 | 7 | namespace { 8 | 9 | #define CHAR_EQUAL(it, end, value) do { \ 10 | if ((it) != (end) && *(it) != (value)) { \ 11 | return false; \ 12 | } \ 13 | } while (0) 14 | 15 | bool isPacketPrefix( 16 | std::vector::iterator it, 17 | std::vector::iterator end) { 18 | // 6 byte prefix equal to 0x00 19 | CHAR_EQUAL(it + 0, end, 0x00); 20 | CHAR_EQUAL(it + 1, end, 0x00); 21 | CHAR_EQUAL(it + 2, end, 0x00); 22 | CHAR_EQUAL(it + 3, end, 0x00); 23 | CHAR_EQUAL(it + 4, end, 0x00); 24 | CHAR_EQUAL(it + 5, end, 0x00); 25 | 26 | // Product Filename 27 | CHAR_EQUAL(it + 6, end, '/'); 28 | CHAR_EQUAL(it + 7, end, 'P'); 29 | CHAR_EQUAL(it + 8, end, 'F'); 30 | 31 | // Packet Number 32 | CHAR_EQUAL(it + 21, end, '/'); 33 | CHAR_EQUAL(it + 22, end, 'P'); 34 | CHAR_EQUAL(it + 23, end, 'N'); 35 | 36 | // Packets Total 37 | CHAR_EQUAL(it + 30, end, '/'); 38 | CHAR_EQUAL(it + 31, end, 'P'); 39 | CHAR_EQUAL(it + 32, end, 'T'); 40 | 41 | // Computed Sum 42 | CHAR_EQUAL(it + 39, end, '/'); 43 | CHAR_EQUAL(it + 40, end, 'C'); 44 | CHAR_EQUAL(it + 41, end, 'S'); 45 | 46 | // If none of these failed it must be a valid packet prefix 47 | return true; 48 | } 49 | 50 | // Compute difference between two integers taking into account wrapping. 51 | template 52 | int diffWithWrap(unsigned int a, unsigned int b) { 53 | int skip; 54 | 55 | ASSERT(a < N); 56 | ASSERT(b < N); 57 | 58 | if (a <= b) { 59 | skip = b - a; 60 | } else { 61 | skip = N - a + b; 62 | } 63 | 64 | return skip; 65 | } 66 | 67 | } // namespace 68 | 69 | std::unique_ptr Assembler::process(qbt::Fragment f) { 70 | std::unique_ptr out; 71 | auto counter = f.counter(); 72 | auto skip = diffWithWrap<(1 << 16)>(counter_, counter); 73 | counter_ = counter; 74 | 75 | // Abort processing of pending packet if we skipped a packet 76 | if (skip > 1) { 77 | tmp_.clear(); 78 | } 79 | 80 | // Append contents of S_PDU to temporary buffer 81 | const auto& buf = f.data(); 82 | tmp_.insert(tmp_.end(), buf.begin(), buf.end()); 83 | 84 | // Find prefix of QBT packet 85 | auto it = tmp_.begin(); 86 | while (it != tmp_.end()) { 87 | if (isPacketPrefix(it, tmp_.end())) { 88 | break; 89 | } 90 | it++; 91 | } 92 | 93 | // Remove everything up to the packet prefix 94 | if (it != tmp_.begin()) { 95 | tmp_.erase(tmp_.begin(), it); 96 | } 97 | 98 | // There is a valid packet prefix and a temporary buffer of at 99 | // least the size of a single QBT packet, so we can return it. 100 | if (tmp_.size() >= 1116) { 101 | std::array buf; 102 | std::copy(tmp_.begin(), tmp_.begin() + 1116, buf.begin()); 103 | tmp_.erase(tmp_.begin(), tmp_.begin() + 1116); 104 | return std::unique_ptr(new Packet(std::move(buf))); 105 | } 106 | 107 | return std::unique_ptr(); 108 | } 109 | 110 | } // namespace qbt 111 | -------------------------------------------------------------------------------- /src/goesrecv/options.cc: -------------------------------------------------------------------------------- 1 | #include "options.h" 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | #include "lib/version.h" 11 | 12 | void usage(int argc, char** argv) { 13 | fprintf(stderr, "Usage: %s [OPTIONS]\n", argv[0]); 14 | fprintf(stderr, "Demodulate and decode signal into packet stream.\n"); 15 | fprintf(stderr, "\n"); 16 | fprintf(stderr, "Options:\n"); 17 | fprintf(stderr, " -c, --config PATH Path to configuration file\n"); 18 | fprintf(stderr, " -v, --verbose Periodically show statistics\n"); 19 | fprintf(stderr, " -i, --interval SEC Interval for --verbose\n"); 20 | fprintf(stderr, "\n"); 21 | fprintf(stderr, "Other:\n"); 22 | fprintf(stderr, " --help Display this help and exit\n"); 23 | fprintf(stderr, " --version Print version information and exit\n"); 24 | fprintf(stderr, "\n"); 25 | exit(0); 26 | } 27 | 28 | Options::Options() : interval(std::chrono::seconds(1)) { 29 | } 30 | 31 | Options parseOptions(int argc, char** argv) { 32 | Options opts; 33 | 34 | while (1) { 35 | static struct option longOpts[] = { 36 | {"config", required_argument, nullptr, 'c'}, 37 | {"verbose", no_argument, nullptr, 'v'}, 38 | {"interval", required_argument, nullptr, 'i'}, 39 | {"help", no_argument, nullptr, 0x1337}, 40 | {"version", no_argument, nullptr, 0x1338}, 41 | {nullptr, 0, nullptr, 0}, 42 | }; 43 | 44 | auto c = getopt_long(argc, argv, "c:vi:", longOpts, nullptr); 45 | if (c == -1) { 46 | break; 47 | } 48 | 49 | switch (c) { 50 | case 0: 51 | break; 52 | case 'c': 53 | opts.config = optarg; 54 | break; 55 | case 'v': 56 | opts.verbose = true; 57 | break; 58 | case 'i': 59 | opts.interval = std::chrono::milliseconds((int) (1000 * atof(optarg))); 60 | break; 61 | case 0x1337: 62 | usage(argc, argv); 63 | break; 64 | case 0x1338: 65 | version(argc, argv); 66 | exit(0); 67 | break; 68 | default: 69 | std::cerr << "Invalid option" << std::endl; 70 | exit(1); 71 | } 72 | } 73 | 74 | // Require configuration to be specified 75 | if (opts.config.empty()) { 76 | fprintf(stderr, "%s: no configuration file specified\n", argv[0]); 77 | fprintf(stderr, "Try '%s --help' for more information.\n", argv[0]); 78 | exit(1); 79 | } 80 | 81 | // Require configuration to be a regular file 82 | { 83 | struct stat st; 84 | const char* error = nullptr; 85 | auto rv = stat(opts.config.c_str(), &st); 86 | if (rv < 0) { 87 | error = strerror(errno); 88 | } else { 89 | if (!S_ISREG(st.st_mode)) { 90 | error = "Not a file"; 91 | } 92 | } 93 | if (error != nullptr) { 94 | fprintf(stderr, 95 | "%s: invalid configuration file '%s': %s\n", 96 | argv[0], 97 | opts.config.c_str(), 98 | error); 99 | fprintf(stderr, "Try '%s --help' for more information.\n", argv[0]); 100 | exit(1); 101 | } 102 | } 103 | 104 | return opts; 105 | } 106 | -------------------------------------------------------------------------------- /src/goesrecv/config.h: -------------------------------------------------------------------------------- 1 | #pragma once 2 | 3 | #include 4 | #include 5 | 6 | #include "packet_publisher.h" 7 | #include "sample_publisher.h" 8 | #include "soft_bit_publisher.h" 9 | 10 | struct Config { 11 | struct StatsPublisher { 12 | // Addresses to bind to. This is a vector because we always add an 13 | // inproc:// endpoint for the in-process monitoring code. 14 | std::vector bind; 15 | 16 | // Optional send buffer size 17 | size_t sendBuffer = 0; 18 | }; 19 | 20 | struct Demodulator { 21 | // LRIT or HRIT 22 | std::string downlinkType; 23 | 24 | // String "airspy" or "rtlsdr" 25 | std::string source; 26 | 27 | // Demodulator statistics (gain, frequency correction, etc.) 28 | StatsPublisher statsPublisher; 29 | 30 | // Signal decimation (applied at FIR stage) 31 | int decimation = 1; 32 | }; 33 | 34 | Demodulator demodulator; 35 | 36 | struct Airspy { 37 | uint32_t frequency = 0; 38 | uint32_t sampleRate = 0; 39 | 40 | // Applies to the linearity gain setting 41 | uint8_t gain = 18; 42 | 43 | // Enable/disable bias tee 44 | bool bias_tee = 0; 45 | 46 | std::unique_ptr samplePublisher; 47 | }; 48 | 49 | Airspy airspy; 50 | 51 | struct RTLSDR { 52 | uint32_t frequency = 0; 53 | uint32_t sampleRate = 0; 54 | 55 | // Applies to the tuner gain setting 56 | uint8_t gain = 30; 57 | 58 | // Enable/disable bias tee 59 | bool bias_tee = 0; 60 | 61 | // Optional device index (if you have multiple devices) 62 | uint32_t deviceIndex = 0; 63 | 64 | std::unique_ptr samplePublisher; 65 | }; 66 | 67 | RTLSDR rtlsdr; 68 | 69 | struct Nanomsg { 70 | uint32_t sampleRate = 0; 71 | 72 | // Address to connect to 73 | std::string connect; 74 | 75 | // Optional receive buffer size 76 | size_t receiveBuffer = 0; 77 | 78 | std::unique_ptr samplePublisher; 79 | }; 80 | 81 | Nanomsg nanomsg; 82 | 83 | struct AGC { 84 | // Minimum gain 85 | float min = 1e-6f; 86 | 87 | // Maximum gain 88 | float max = 1e+6f; 89 | 90 | std::unique_ptr samplePublisher; 91 | }; 92 | 93 | AGC agc; 94 | 95 | struct Costas { 96 | // Maximum frequency deviation in Hz (defaults to 20 KHz) 97 | int maxDeviation = 20000; 98 | 99 | std::unique_ptr samplePublisher; 100 | }; 101 | 102 | Costas costas; 103 | 104 | struct RRC { 105 | std::unique_ptr samplePublisher; 106 | }; 107 | 108 | RRC rrc; 109 | 110 | struct ClockRecovery { 111 | std::unique_ptr samplePublisher; 112 | }; 113 | 114 | ClockRecovery clockRecovery; 115 | 116 | struct Quantization { 117 | std::unique_ptr softBitPublisher; 118 | }; 119 | 120 | Quantization quantization; 121 | 122 | struct Decoder { 123 | std::unique_ptr packetPublisher; 124 | 125 | // Decoder statistics (Viterbi, Reed-Solomon, etc.) 126 | StatsPublisher statsPublisher; 127 | }; 128 | 129 | Decoder decoder; 130 | 131 | struct Monitor { 132 | // Address to send UDP statsd packets to (e.g. localhost:8125) 133 | std::string statsdAddress; 134 | }; 135 | 136 | Monitor monitor; 137 | 138 | static Config load(const std::string& file); 139 | }; 140 | -------------------------------------------------------------------------------- /src/goesrecv/source.cc: -------------------------------------------------------------------------------- 1 | #include "source.h" 2 | 3 | #include 4 | 5 | #ifdef BUILD_AIRSPY 6 | #include "airspy_source.h" 7 | #endif 8 | 9 | #ifdef BUILD_RTLSDR 10 | #include "rtlsdr_source.h" 11 | #endif 12 | 13 | #include "nanomsg_source.h" 14 | 15 | std::unique_ptr Source::build( 16 | const std::string& type, 17 | Config& config) { 18 | if (type == "airspy") { 19 | #ifdef BUILD_AIRSPY 20 | auto airspy = Airspy::open(); 21 | 22 | // Use sample rate if set, otherwise default to lowest possible rate. 23 | // This is 2.5MSPS for the R2 and 3M for the Mini. 24 | auto rates = airspy->getSampleRates(); 25 | if (config.airspy.sampleRate != 0) { 26 | auto rate = config.airspy.sampleRate; 27 | auto pos = std::find(rates.begin(), rates.end(), rate); 28 | if (pos == rates.end()) { 29 | std::stringstream ss; 30 | ss << 31 | "You configured the Airspy source to use an unsupported " << 32 | "sample rate equal to " << rate << ". " << 33 | "Supported sample rates are: " << std::endl; 34 | for (size_t i = 0; i < rates.size(); i++) { 35 | ss << " - " << rates[i] << std::endl; 36 | } 37 | throw std::runtime_error(ss.str()); 38 | } 39 | airspy->setSampleRate(rate); 40 | } else { 41 | std::sort(rates.begin(), rates.end()); 42 | airspy->setSampleRate(rates[0]); 43 | } 44 | airspy->setFrequency(config.airspy.frequency); 45 | airspy->setGain(config.airspy.gain); 46 | airspy->setBiasTee(config.airspy.bias_tee); 47 | airspy->setSamplePublisher(std::move(config.airspy.samplePublisher)); 48 | return std::unique_ptr(airspy.release()); 49 | #else 50 | throw std::runtime_error( 51 | "You configured goesrecv to use the \"airspy\" source, " 52 | "but goesrecv was not compiled with Airspy support. " 53 | "Make sure to install the Airspy library before compiling goestools, " 54 | "and look for a message saying 'Found libairspy' when running cmake." 55 | ); 56 | #endif 57 | } 58 | if (type == "rtlsdr") { 59 | #ifdef BUILD_RTLSDR 60 | auto rtlsdr = RTLSDR::open(config.rtlsdr.deviceIndex); 61 | 62 | // Use sample rate if set, otherwise default to 2.4MSPS. 63 | if (config.rtlsdr.sampleRate != 0) { 64 | rtlsdr->setSampleRate(config.rtlsdr.sampleRate); 65 | } else { 66 | rtlsdr->setSampleRate(2400000); 67 | } 68 | rtlsdr->setFrequency(config.rtlsdr.frequency); 69 | rtlsdr->setTunerGain(config.rtlsdr.gain); 70 | rtlsdr->setBiasTee(config.rtlsdr.bias_tee); 71 | rtlsdr->setSamplePublisher(std::move(config.rtlsdr.samplePublisher)); 72 | return std::unique_ptr(rtlsdr.release()); 73 | #else 74 | throw std::runtime_error( 75 | "You configured goesrecv to use the \"rtlsdr\" source, " 76 | "but goesrecv was not compiled with RTL-SDR support. " 77 | "Make sure to install the RTL-SDR library before compiling goestools, " 78 | "and look for a message saying 'Found librtlsdr' when running cmake." 79 | ); 80 | #endif 81 | } 82 | if (type == "nanomsg") { 83 | auto nanomsg = Nanomsg::open(config); 84 | nanomsg->setSampleRate(config.nanomsg.sampleRate); 85 | nanomsg->setSamplePublisher(std::move(config.nanomsg.samplePublisher)); 86 | return std::unique_ptr(nanomsg.release()); 87 | } 88 | 89 | throw std::runtime_error("Invalid source: " + type); 90 | } 91 | 92 | Source::~Source() { 93 | } 94 | -------------------------------------------------------------------------------- /src/goesrecv/nanomsg_source.cc: -------------------------------------------------------------------------------- 1 | #include "nanomsg_source.h" 2 | 3 | #include 4 | #include 5 | #include 6 | 7 | #include 8 | #include 9 | 10 | std::unique_ptr Nanomsg::open(const Config& config) { 11 | int rv; 12 | 13 | auto fd = nn_socket(AF_SP, NN_SUB); 14 | if (fd < 0) { 15 | std::stringstream ss; 16 | ss << "nn_socket: " << nn_strerror(nn_errno()); 17 | throw std::runtime_error(ss.str()); 18 | } 19 | 20 | const char* url = config.nanomsg.connect.c_str(); 21 | rv = nn_connect(fd, url); 22 | if (rv < 0) { 23 | nn_close(fd); 24 | std::stringstream ss; 25 | ss << "nn_connect: " << nn_strerror(nn_errno()); 26 | ss << " (" << url << ")"; 27 | throw std::runtime_error(ss.str()); 28 | } 29 | 30 | int size = config.nanomsg.receiveBuffer; 31 | if (size > 0) { 32 | rv = nn_setsockopt(fd, NN_SOL_SOCKET, NN_SNDBUF, &size, sizeof(size)); 33 | if (rv < 0) { 34 | nn_close(fd); 35 | std::stringstream ss; 36 | ss << "nn_setsockopt: " << nn_strerror(nn_errno()); 37 | throw std::runtime_error(ss.str()); 38 | } 39 | } 40 | 41 | rv = nn_setsockopt(fd, NN_SUB, NN_SUB_SUBSCRIBE, "", 0); 42 | if (rv < 0) { 43 | nn_close(fd); 44 | std::stringstream ss; 45 | ss << "nn_setsockopt: " << nn_strerror(nn_errno()); 46 | ss << " (" << url << ")"; 47 | throw std::runtime_error(ss.str()); 48 | } 49 | 50 | return std::make_unique(fd); 51 | } 52 | 53 | Nanomsg::Nanomsg(int fd) : fd_(fd) { 54 | } 55 | 56 | Nanomsg::~Nanomsg() { 57 | } 58 | 59 | void Nanomsg::setSampleRate(uint32_t sampleRate) { 60 | sampleRate_ = sampleRate; 61 | } 62 | 63 | uint32_t Nanomsg::getSampleRate() const { 64 | return sampleRate_; 65 | } 66 | 67 | void Nanomsg::loop() { 68 | void* buf = nullptr; 69 | int nbytes; 70 | for (;;) { 71 | nbytes = nn_recv(fd_, &buf, NN_MSG, 0); 72 | if (nbytes <= 0) { 73 | return; 74 | } 75 | 76 | uint32_t nsamples = nbytes / 2; 77 | int8_t* fi = (int8_t*) buf; 78 | 79 | // Expect multiple of 4 80 | ASSERT((nsamples & 0x3) == 0); 81 | 82 | // Grab buffer from queue 83 | auto out = queue_->popForWrite(); 84 | out->resize(nsamples); 85 | 86 | // Convert to std::complex 87 | auto fo = out->data(); 88 | for (uint32_t i = 0; i < nsamples; i++) { 89 | fo[i].real((float) fi[i*2+0] / 127.0f); 90 | fo[i].imag((float) fi[i*2+1] / 127.0f); 91 | } 92 | 93 | // Processed samples; free nanomsg buffer 94 | nn_freemsg(buf); 95 | 96 | // Publish output if applicable 97 | if (samplePublisher_) { 98 | samplePublisher_->publish(*out); 99 | } 100 | 101 | // Return buffer to queue 102 | queue_->pushWrite(std::move(out)); 103 | } 104 | } 105 | 106 | void Nanomsg::start(const std::shared_ptr >& queue) { 107 | queue_ = queue; 108 | thread_ = std::thread(&Nanomsg::loop, this); 109 | #ifdef __APPLE__ 110 | pthread_setname_np("nanomsg"); 111 | #else 112 | pthread_setname_np(thread_.native_handle(), "nanomsg"); 113 | #endif 114 | } 115 | 116 | void Nanomsg::stop() { 117 | nn_close(fd_); 118 | 119 | // Wait for thread to terminate 120 | thread_.join(); 121 | 122 | // Close queue to signal downstream 123 | queue_->close(); 124 | 125 | // Clear reference to queue 126 | queue_.reset(); 127 | } 128 | -------------------------------------------------------------------------------- /src/goesproc/proj.cc: -------------------------------------------------------------------------------- 1 | #include "proj.h" 2 | 3 | #include 4 | #include 5 | 6 | namespace { 7 | 8 | std::vector toVector(const std::map& args) { 9 | std::vector vargs; 10 | vargs.reserve(args.size()); 11 | for (const auto& arg : args) { 12 | std::stringstream ss; 13 | ss << arg.first << "=" << arg.second; 14 | vargs.push_back(ss.str()); 15 | } 16 | return vargs; 17 | } 18 | 19 | } 20 | 21 | #if PROJ_VERSION_MAJOR == 4 22 | 23 | namespace { 24 | 25 | std::string pj_error(std::string prefix = "proj: ") { 26 | std::stringstream ss; 27 | ss << prefix << pj_strerrno(pj_errno); 28 | return ss.str(); 29 | } 30 | 31 | } // namespace 32 | 33 | // Forward compatibility. 34 | double proj_torad (double angle_in_degrees) { 35 | return angle_in_degrees * DEG_TO_RAD; 36 | } 37 | 38 | Proj::Proj(const std::vector& args) { 39 | std::vector argv; 40 | for (const auto& arg : args) { 41 | argv.push_back(strdup(arg.c_str())); 42 | } 43 | proj_ = pj_init(argv.size(), argv.data()); 44 | if (!proj_) { 45 | throw std::runtime_error(pj_error("proj initialization error: ")); 46 | } 47 | for (auto& arg : argv) { 48 | free(arg); 49 | } 50 | } 51 | 52 | Proj::Proj(const std::map& args) 53 | : Proj(toVector(args)) { 54 | } 55 | 56 | Proj::~Proj() { 57 | pj_free(proj_); 58 | } 59 | 60 | std::tuple Proj::fwd(double lon, double lat) { 61 | projUV in = { lon, lat }; 62 | projXY out = pj_fwd(in, proj_); 63 | return std::make_tuple(std::move(out.u), std::move(out.v)); 64 | } 65 | 66 | std::tuple Proj::inv(double x, double y) { 67 | projXY in = { x, y }; 68 | projUV out = pj_inv(in, proj_); 69 | return std::make_tuple(std::move(out.u), std::move(out.v)); 70 | } 71 | 72 | #elif PROJ_VERSION_MAJOR >= 5 73 | 74 | namespace { 75 | 76 | std::string toString(const std::vector& vargs) { 77 | std::stringstream ss; 78 | for (auto it = vargs.begin(); it != vargs.end(); it++) { 79 | ss << "+" << *it; 80 | if (std::next(it) != std::end(vargs)) { 81 | ss << " "; 82 | } 83 | } 84 | return ss.str(); 85 | } 86 | 87 | std::string pj_error(std::string prefix = "proj: ") { 88 | std::stringstream ss; 89 | ss << prefix << proj_errno_string(proj_errno(NULL)); 90 | return ss.str(); 91 | } 92 | 93 | } // namespace 94 | 95 | Proj::Proj(const std::vector& vargs) { 96 | const auto args = toString(vargs); 97 | proj_ = proj_create(NULL, args.c_str()); 98 | if (!proj_) { 99 | throw std::runtime_error(pj_error("proj initialization error: ")); 100 | } 101 | } 102 | 103 | Proj::Proj(const std::map& args) 104 | : Proj(toVector(args)) { 105 | } 106 | 107 | Proj::~Proj() { 108 | proj_destroy(proj_); 109 | } 110 | 111 | std::tuple Proj::fwd(double lon, double lat) { 112 | PJ_COORD in; 113 | in.uv = { lon, lat }; 114 | PJ_COORD out = proj_trans(proj_, PJ_FWD, in); 115 | return std::make_tuple(std::move(out.xy.x), std::move(out.xy.y)); 116 | } 117 | 118 | std::tuple Proj::inv(double x, double y) { 119 | PJ_COORD in; 120 | in.xy = { x, y }; 121 | PJ_COORD out = proj_trans(proj_, PJ_INV, in); 122 | return std::make_tuple(std::move(out.uv.u), std::move(out.uv.v)); 123 | } 124 | 125 | #endif 126 | -------------------------------------------------------------------------------- /src/goesrecv/airspy_source.cc: -------------------------------------------------------------------------------- 1 | #include "airspy_source.h" 2 | 3 | #include 4 | 5 | #include 6 | #include 7 | 8 | #include 9 | 10 | std::unique_ptr Airspy::open(uint32_t index) { 11 | struct airspy_device* dev = nullptr; 12 | auto rv = airspy_open(&dev); 13 | if (rv < 0) { 14 | std::cerr 15 | << "Unable to open Airspy device: " 16 | << airspy_error_name((enum airspy_error) rv) 17 | << std::endl; 18 | exit(1); 19 | } 20 | 21 | return std::make_unique(dev); 22 | } 23 | 24 | Airspy::Airspy(struct airspy_device* dev) : dev_(dev) { 25 | // Load list of supported sample rates 26 | sampleRates_ = loadSampleRates(); 27 | 28 | // We produce floats so let the airspy library take care of it 29 | auto rv = airspy_set_sample_type(dev_, AIRSPY_SAMPLE_FLOAT32_IQ); 30 | ASSERT(rv == 0); 31 | } 32 | 33 | Airspy::~Airspy() { 34 | if (dev_ != nullptr) { 35 | airspy_close(dev_); 36 | } 37 | } 38 | 39 | std::vector Airspy::loadSampleRates() { 40 | int rv; 41 | 42 | uint32_t count; 43 | rv = airspy_get_samplerates(dev_, &count, 0); 44 | ASSERT(rv == 0); 45 | 46 | std::vector rates(count); 47 | rv = airspy_get_samplerates(dev_, rates.data(), rates.size()); 48 | ASSERT(rv == 0); 49 | 50 | return rates; 51 | } 52 | 53 | void Airspy::setFrequency(uint32_t freq) { 54 | ASSERT(dev_ != nullptr); 55 | auto rv = airspy_set_freq(dev_, freq); 56 | ASSERT(rv >= 0); 57 | } 58 | 59 | void Airspy::setSampleRate(uint32_t rate) { 60 | ASSERT(dev_ != nullptr); 61 | auto rv = airspy_set_samplerate(dev_, rate); 62 | ASSERT(rv >= 0); 63 | sampleRate_ = rate; 64 | } 65 | 66 | uint32_t Airspy::getSampleRate() const { 67 | return sampleRate_; 68 | } 69 | 70 | void Airspy::setGain(int gain) { 71 | ASSERT(dev_ != nullptr); 72 | auto rv = airspy_set_linearity_gain(dev_, gain); 73 | ASSERT(rv >= 0); 74 | } 75 | 76 | void Airspy::setBiasTee(bool on) { 77 | ASSERT(dev_ != nullptr); 78 | auto rv = airspy_set_rf_bias(dev_, on ? 1 : 0); 79 | ASSERT(rv >= 0); 80 | } 81 | 82 | static int airspy_callback(airspy_transfer* transfer) { 83 | auto airspy = reinterpret_cast(transfer->ctx); 84 | airspy->handle(transfer); 85 | return 0; 86 | } 87 | 88 | void Airspy::start(const std::shared_ptr >& queue) { 89 | ASSERT(dev_ != nullptr); 90 | queue_ = queue; 91 | thread_ = std::thread([&] { 92 | auto rv = airspy_start_rx(dev_, &airspy_callback, this); 93 | ASSERT(rv == 0); 94 | }); 95 | #ifdef __APPLE__ 96 | pthread_setname_np("airspy"); 97 | #else 98 | pthread_setname_np(thread_.native_handle(), "airspy"); 99 | #endif 100 | } 101 | 102 | void Airspy::stop() { 103 | ASSERT(dev_ != nullptr); 104 | auto rv = airspy_stop_rx(dev_); 105 | ASSERT(rv >= 0); 106 | 107 | // Wait for thread to terminate 108 | thread_.join(); 109 | 110 | // Close queue to signal downstream 111 | queue_->close(); 112 | 113 | // Clear reference to queue 114 | queue_.reset(); 115 | } 116 | 117 | void Airspy::handle(const airspy_transfer* transfer) { 118 | auto nsamples = transfer->sample_count; 119 | auto out = queue_->popForWrite(); 120 | out->resize(nsamples); 121 | memcpy(out->data(), transfer->samples, nsamples * sizeof(std::complex)); 122 | 123 | // Publish output if applicable 124 | if (samplePublisher_) { 125 | samplePublisher_->publish(*out); 126 | } 127 | 128 | queue_->pushWrite(std::move(out)); 129 | } 130 | -------------------------------------------------------------------------------- /docs/guides/circonus.rst: -------------------------------------------------------------------------------- 1 | Monitoring goesrecv with Circonus 2 | ================================= 3 | 4 | Circonus_ is a hosted graphing and monitoring tool. It has native 5 | support for histograms which is ideal for looking at a distribution 6 | over time. For goesrecv in particular, this is great for keeping track 7 | of signal quality over time (e.g. distribution of Viterbi bit errors 8 | per packet, packet drops, etc). 9 | 10 | .. _circonus: https://www.circonus.com/ 11 | 12 | For example, for my GOES-16 receiver, it enabled me to compare the 13 | performance of two different LNA configurations. Even though the new 14 | configuration has a worse mean error rate, it does not get overloaded 15 | when local interference happens, and leads to fewer packet drops. 16 | 17 | Below you see the before and after graphs that show the distribution 18 | of Viterbi bit error rate and the number of packet drops. The before 19 | figure clearly has a lower mean bit error rate, but has really bad 20 | outliers, that in turn cause packet drop. The after figure has a 21 | higher mean bit error rate, but lacks the bad outliers, and has no 22 | packet drop at all. 23 | 24 | .. figure:: circonus_before.png 25 | :align: center 26 | 27 | **Before** 28 | 29 | .. figure:: circonus_after.png 30 | :align: center 31 | 32 | **After** 33 | 34 | Setup 35 | ----- 36 | 37 | * Register for a Circonus account and log in 38 | * Go to "new check" 39 | * Use the Circonus One-Step Install (COSI) 40 | 41 | COSI installs an agent on your machine. The agent tracks a set of 42 | system metrics by default. It will also be the sink for the statsd 43 | stats produced by goesrecv and forward them to Circonus. 44 | 45 | Setup on Raspbian 46 | ----------------- 47 | 48 | If you run goesrecv on Raspbian you have to install the agent 49 | yourself. See https://www.circonus.com/2018/02/circonus-raspberry-pi/ 50 | for instructions. These instructions include a fixed version of Node 51 | version 6, but newer versions have come out since. Find the latest 52 | version at https://nodejs.org/dist/latest-v6.x/ and modify the 53 | commands accordingly. 54 | 55 | .. note:: 56 | 57 | Run ``uname -a`` on your Raspberry Pi to see if you need the 58 | ``armv6l`` or ``arvm7l`` binaries. 59 | 60 | The instructions are missing a ``mkdir``, run the following before starting: 61 | 62 | .. code-block:: text 63 | 64 | sudo mkdir -p /opt/circonus/bin 65 | 66 | If the agent is not yet running after the installation has finished, 67 | try the following: 68 | 69 | .. code-block:: text 70 | 71 | sudo systemctl daemon-reload 72 | sudo systemctl enable nad 73 | sudo systemctl start nad 74 | 75 | Configuration 76 | ------------- 77 | 78 | The statsd listener in the agent is enabled by default. 79 | 80 | Ensure that your goesrecv configuration contains the following: 81 | 82 | .. code-block:: toml 83 | 84 | [monitor] 85 | statsd_address = "udp4://localhost:8125" 86 | 87 | After both the agent and goesrecv have run for a minute or so, 88 | navigate to "Integrations", then "Checks", and then click on the check 89 | you just added. Then click on "View Check Details" and subsequently 90 | "Change Brokers & Metrics". Here you can enable the goesrecv stats you 91 | care about. 92 | 93 | Relevant demodulator stats: 94 | 95 | * ``statsd.gain`` 96 | * ``statsd.frequency`` 97 | * ``statsd.omega`` 98 | 99 | Relevant decoder stats: 100 | 101 | * ``statsd.viterbi_errors`` (enable as histogram) 102 | * ``statsd.reed_solomon_errors`` (enable as histogram) 103 | * ``statsd.packets_ok`` 104 | * ``statsd.packets_dropped`` 105 | -------------------------------------------------------------------------------- /src/goesrecv/decoder.cc: -------------------------------------------------------------------------------- 1 | #include "decoder.h" 2 | 3 | #include 4 | 5 | #include 6 | 7 | #include 8 | 9 | using namespace util; 10 | 11 | namespace { 12 | 13 | // QueueReader bridges the queue that produces the soft bits 14 | // output of the demodulator to the packetizer. 15 | // 16 | // The packetizer needs an interface that quacks like read(2). 17 | // 18 | class QueueReader : public decoder::Reader { 19 | public: 20 | explicit QueueReader(std::shared_ptr > > queue) 21 | : queue_(std::move(queue)) { 22 | } 23 | 24 | virtual ~QueueReader() { 25 | // Return read buffer if needed 26 | if (tmp_) { 27 | queue_->pushRead(std::move(tmp_)); 28 | } 29 | } 30 | 31 | virtual size_t read(void* buf, size_t count) { 32 | char* ptr = (char*) buf; 33 | size_t nread = 0; 34 | while (nread < count) { 35 | // Acquire new read buffer if we don't already have one 36 | if (!tmp_) { 37 | tmp_ = queue_->popForRead(); 38 | if (!tmp_) { 39 | // Can't complete read when queue has closed. 40 | return 0; 41 | } 42 | pos_ = 0; 43 | } 44 | 45 | auto left = tmp_->size() - pos_; 46 | auto needed = count - nread; 47 | auto min = std::min(left, needed); 48 | memcpy(&ptr[nread], &(*tmp_)[pos_], min); 49 | 50 | // Advance cursors 51 | nread += min; 52 | pos_ += min; 53 | 54 | // Return read buffer if it was exhausted 55 | if (pos_ == tmp_->size()) { 56 | queue_->pushRead(std::move(tmp_)); 57 | } 58 | } 59 | 60 | return nread; 61 | } 62 | 63 | protected: 64 | std::shared_ptr > > queue_; 65 | std::unique_ptr > tmp_; 66 | size_t pos_; 67 | }; 68 | 69 | } // namespace 70 | 71 | Decoder::Decoder(std::shared_ptr > > queue) { 72 | packetizer_ = std::make_unique( 73 | std::make_shared(std::move(queue))); 74 | } 75 | 76 | void Decoder::initialize(Config& config) { 77 | packetPublisher_ = std::move(config.decoder.packetPublisher); 78 | statsPublisher_ = StatsPublisher::create(config.decoder.statsPublisher.bind); 79 | if (config.demodulator.statsPublisher.sendBuffer > 0) { 80 | statsPublisher_->setSendBuffer(config.decoder.statsPublisher.sendBuffer); 81 | } 82 | } 83 | 84 | void Decoder::publishStats(decoder::Packetizer::Details details) { 85 | if (!statsPublisher_) { 86 | return; 87 | } 88 | 89 | const auto timestamp = stringTime(); 90 | 91 | std::stringstream ss; 92 | ss << "{"; 93 | ss << "\"timestamp\": \"" << timestamp << "\","; 94 | ss << "\"skipped_symbols\": " << details.skippedSymbols << ","; 95 | ss << "\"viterbi_errors\": " << details.viterbiBits << ","; 96 | ss << "\"reed_solomon_errors\": " << details.reedSolomonBytes << ","; 97 | ss << "\"ok\": " << details.ok; 98 | ss << "}\n"; 99 | statsPublisher_->publish(ss.str()); 100 | } 101 | 102 | void Decoder::start() { 103 | thread_ = std::thread([&] { 104 | std::array buf; 105 | decoder::Packetizer::Details details; 106 | while (packetizer_->nextPacket(buf, &details)) { 107 | if (details.ok && packetPublisher_) { 108 | packetPublisher_->publish(buf); 109 | } 110 | publishStats(details); 111 | } 112 | }); 113 | #ifdef __APPLE__ 114 | pthread_setname_np("decoder"); 115 | #else 116 | pthread_setname_np(thread_.native_handle(), "decoder"); 117 | #endif 118 | } 119 | 120 | void Decoder::stop() { 121 | thread_.join(); 122 | } 123 | -------------------------------------------------------------------------------- /docs/commands/goesproc.rst: -------------------------------------------------------------------------------- 1 | .. _goesproc: 2 | 3 | goesproc 4 | -------- 5 | 6 | Process stream of packets (VCDUs) or list of LRIT files. 7 | 8 | Options 9 | ======= 10 | 11 | ================================ ========================================== 12 | ``-c``, ``--config=PATH`` Path to configuration file 13 | ``-m``, ``--mode [packet|lrit]`` Process stream of VCDU packets 14 | or pre-assembled LRIT files 15 | ``--subscribe ADDR`` Address of nanomsg publisher 16 | ``-f``, ``--force`` Overwrite existing output files 17 | ================================ ========================================== 18 | 19 | If mode is set to ``packet``, goesproc reads VCDU packets from the 20 | specified path(s). To process real time data you can either setup a pipe 21 | from the decoder into goesproc (e.g. use /dev/stdin as path argument), 22 | or use ``--subscribe`` to consume packets directly from :ref:`goesrecv`. 23 | To process recorded data you can specify a list of files that contain 24 | VCDU packets in chronological order. 25 | 26 | If mode is set to ``lrit``, goesproc finds all LRIT files in the specified 27 | paths and processes them sequentially. You can specify a mix of files 28 | and directories. Directory arguments expand into the files they 29 | contain that match the glob ``*.lrit*``. The complete list of LRIT files 30 | is sorted according to their time stamp header prior to processing it. 31 | 32 | Configuration 33 | ============= 34 | 35 | The configuration file uses TOML_ syntax. You can 36 | find an example configuration (that includes stanzas for every 37 | available handler) in ``./etc/goesproc.conf``. 38 | 39 | .. _TOML: 40 | https://github.com/toml-lang/toml 41 | 42 | Handlers 43 | ======== 44 | 45 | Different products use different handlers with different configuration 46 | options. For example, there is an NWS image handler, NWS text handler, 47 | EMWIN handler, GOES N-series handler, GOES R-series handler, Himawari 48 | handler, etc. 49 | 50 | **TODO** -- describe configuration options for every handler 51 | 52 | Example 53 | ======= 54 | 55 | For example, with the following configuration file: 56 | 57 | .. code-block:: toml 58 | 59 | [[handler]] 60 | type = "image" 61 | product = "goes15" 62 | region = "fd" 63 | channels = [ "VS" ] 64 | crop = [ -2373, 2371, -1357, 1347 ] 65 | filename = "GOES15_%r_%c_%t" 66 | 67 | 68 | Running goesproc against a directory with GOES-15 LRIT files 69 | results in a number of image files of the full disk visual channel: 70 | 71 | .. code-block:: shell 72 | 73 | $ goesproc --config example.conf --mode lrit ./out/images/goes15/fd 74 | Writing ./GOES15_FD_VS_20170820-210600.png 75 | Writing ./GOES15_FD_VS_20170821-000600.png 76 | Writing ./GOES15_FD_VS_20170821-030600.png 77 | Writing ./GOES15_FD_VS_20170821-060600.png 78 | Writing ./GOES15_FD_VS_20170821-090600.png 79 | Writing ./GOES15_FD_VS_20170821-120600.png 80 | Writing ./GOES15_FD_VS_20170821-150600.png 81 | Writing ./GOES15_FD_VS_20170821-180600.png 82 | ... 83 | 84 | You can now use these image files however you like. For example, to 85 | produce a GIF from 8 consecutive full disk images, you can use the 86 | following ImageMagick_ commands: 87 | 88 | .. _ImageMagick: https://www.imagemagick.org/ 89 | 90 | .. code-block:: shell 91 | 92 | $ mogrify -resize '640x480>' *.png 93 | $ convert -loop 0 -delay 50 *.png GOES15_FD_VS_20170821.gif 94 | 95 | And get: 96 | 97 | .. image:: /images/GOES15_FD_VS_20170821.gif 98 | 99 | Sample configuration 100 | ==================== 101 | 102 | .. literalinclude:: ../../etc/goesproc.conf 103 | :language: toml 104 | 105 | .. toctree:: 106 | :maxdepth: 1 107 | --------------------------------------------------------------------------------