Receiver: RTL-SDR
94 | Filter: SAWbird+ Filter & LNA / 137MHz center frequency
95 | Antenna: Quadrifilar Helix
96 |
97 |
98 |
99 |
100 |
Raspberry Pi Hardware Setup
101 |
102 |
Transporting the Raspberry PI, RTL-SDR dongle, and filter can be a little difficult. I decided to create a case that would help make the kit easier to plug in and use. In the fall, winter, and even some parts of the spring, I leave this outside permanently but I would not advise leaving this out during the summer as it may lead to heat issues.
103 |
104 |
I made a case for my RTL-SDR, LNA, and Raspberry Pi, which make my NOAA ground station. This is attached to a QFH antenna and is setup to automatically capture, decode, and upload NOAA satellite passes. All the decoded images and pass details are uploaded to an AWS website hosted via S3. See here:
105 |
106 |
107 |
108 |
As you can see the RTL-SDR is connected to the Nooelec Filter + LNA using a jumper cable. The antenna connection protrudes just a bit from the case to allow it to connect to an antenna when closed. All of the hardware is on the case using Commander Strips.
109 |
110 |
That jumper cable is bending a little more than I'd like but that's about as much distance as I could create between the two devices/connections.
111 |
112 |
The RTL-SDR is connected to the RPI with a 12 inch USB cable. I dislike this part about the case and open to any suggestions. The 6in cable was not enough distance and didn't allow the case to be opened. It also required a lot of bending. This way, it may stick out of the case but the bend is better and less restrictive.
113 |
114 |
115 |
116 |
Once closed you can see the SMA connector for connecting to an antenna and the USB cable that connects the RTL-SDR and RPI. In the background on the top left you can see where the power cable comes in for the RPI. I've been monitoring temperature and it's usually between 42 and 50 degress Celcius but I have seen it go all the way up to around 58 degrees when uploading images.
Receiver: RTL-SDR
94 | Filter: SAWbird+ Filter & LNA / 137MHz center frequency
95 | Antenna: Quadrifilar Helix
96 |
97 |
98 |
99 |
100 |
Making a Quadrifilar Helix Antenna
101 |
102 |
103 |
104 |
This odd looking thing is a quadrifilar helix antenna and it's specially designed to receive images from the NOAA weather satellites. It's actually two antennas that are formed by a larger loop and a smaller loop. These loops are shaped to receive a signal on 137MHz frequency and block out the rest of the noise. Below are some resources and my learnings on building a QFH antenna.
105 |
106 |
First to explain why such an odd shaped antenna is needed, it helps to understand how the radio signal that comes from the NOAA satellites look. See below:
107 |
108 |
109 |
110 |
There are a lot of tutorials out there and I would say that there is enough variation in each tutorial to make it confusing. Tutorials are mostly geared toward using copper tubing but a few also include a wire. version. Ultimately, I used this Instructables guide with a lot of help from the calculator on the John Coppens' website.
111 |
112 |
Many of the tutorials out there guide you to use the coax as part of the antenna but that ended up confusing me a bit and I decided to just make two loops instead. From a technical point of view, I didn’t see how it would make a big difference and that the most important thing to do was to get two loops that were each about 1λ. This below is the best diagram I found on how to wire it all together.
113 |
114 |
115 |
116 |
I used 14 AWG wire, which is pretty thick and hard to reliably measure. I went ahead and just created the loops a bit longer than I thought they should be. I probably could have measured out a string for measuring the wire more precisely but I guess I got lazy and just wanted the lengths at that point because I was in the homestretch for putting it all together.
117 |
118 |
Once I got the lengths cut, I cabled everything up with some electrical tape to test SWR with my NanoVNA. Of course, the lengths were too long and it was tuning to around 120 mHz. I started cutting down lengths of about one inch on each of the four ends and testing until I got closer to an SWR of about 1.7 at 137 mHz. If you don't have a VNA, don't worry. You can skip it as long as you made sure the wire lengths were about the right size. You may just want to take more time when measuring.
119 |
120 |
Next big task is to solder everything together. You'll just need a soldering iron, some solder, and flux. These can be bought rather inexpensively but you can also just borrow one from a friend or neighbor. However, the trick to making this easy is to purchase a cable that is already crimped for your SDR. These are typically SMA connectors. I purchased a 25ft cable with SMA connectors on the end and just cut one end off. This left the other end exposed for me to solder onto the anteanna. No need to crimp any connectors.
121 |
122 |
As with any antenna, the higher, the better. That said, my antenna is secured to a Christmas tree stand in my lawn and I get great images.
123 |
124 |
If you find that building a QFH antenna is a bit more than you want to do at the moment, you can always build a dipole. Below is a diagram on measurements, angle, and direction. You can also buy a simple kit from RTL-SDR that will allow you to just extend to the appropriate length.
125 |
126 |
127 |
128 |
129 |
130 |
131 |
132 |
133 |
134 |
135 |
Passes
136 |
139 |
140 |
141 |
142 |
143 |
144 |
147 |
148 |
149 |
150 |
151 |
152 |
153 |
154 |
155 |
156 |
157 |
158 |
163 |
164 |
165 |
166 |
--------------------------------------------------------------------------------
/aws-s3/upload-wx-images.js:
--------------------------------------------------------------------------------
1 | // Required modules
2 | require('dotenv').config({path:'/home/pi/wx-ground-station/aws-s3/.env'});
3 | var fs = require('fs');
4 | var path = require('path');
5 | var glob = require("glob");
6 | var AWS = require('aws-sdk');
7 | var uuid = require('uuid');
8 | var Jimp = require('jimp');
9 | var dateFormat = require('dateformat');
10 | var discord = require('./lib/api/discord');
11 |
12 | // AWS Configuration
13 | var REGION = process.env.AWS_REGION;
14 | var BUCKET = process.env.AWS_BUCKET;
15 |
16 | // Title information printed over image
17 | var LOCATION = process.env.STATION_LOCATION;
18 |
19 | // Working directories for data
20 | var IMAGE_DIR = "images/";
21 | var LOG_DIR = "logs/";
22 | var AUDIO_DIR = "audio/";
23 | var MAP_DIR = "maps/"
24 |
25 | // Set region
26 | AWS.config.update({region: REGION});
27 | // Create S3 service object
28 | var s3 = new AWS.S3();
29 |
30 | // Put the array of command line arguments into variables
31 | var satellite = process.argv[2];
32 | var frequency = process.argv[3];
33 | var filebase = process.argv[4];
34 | var elevation = process.argv[5];
35 | var direction = process.argv[6];
36 | var duration = process.argv[7];
37 | var tle1 = process.argv[8];
38 | var tle2 = process.argv[9];
39 | var gain = process.argv[10];
40 | var chan_a = process.argv[11];
41 | var chan_b = process.argv[12];
42 |
43 | // Get the name of the files (such as NOAA15-20200227-141322)
44 | var basename = filebase.slice(filebase.lastIndexOf('/')+1);
45 | // Get the directory name
46 | var dirname = filebase.slice(0, filebase.lastIndexOf('/')+1);
47 | // Get the root directory without images directory
48 | var rootdirname = filebase.slice(0, filebase.lastIndexOf('/') - 6);
49 | // Get individual parts of the base name (such as NOAA15, 20200227, and 141322)
50 | var components = basename.split("-");
51 | // Get date as the second part of the array and format as YYYY-MM-DD
52 | var date = components[1];
53 | date = date.slice(0, 4) + '-' + date.slice(4, 6) + '-' + date.slice(6);
54 | // Get date as the third part of the array and format as HH:MM:ss
55 | var time = components[2];
56 | time = time.slice(0, 2) + ':' + time.slice(2, 4) + ':' + time.slice(4) + ' ' + dateFormat(new Date, "o");
57 |
58 | // Define gain by getting the desired value from the full string
59 | // example "Gain: 15.2"
60 | if (gain) {
61 | gain = gain.substring(gain.indexOf(": ") + 2)
62 | }
63 |
64 | // Define Channel A and B by getting the desired value from the full string
65 | // example "Channel A: 1 (visible)"
66 | if (chan_a) {
67 | chan_a = chan_a.substring(chan_a.indexOf(": ")+2);
68 | }
69 | // example "Channel B: 4 (thermal infrared)"
70 | if (chan_b) {
71 | chan_b = chan_b.substring(chan_b.indexOf(": ")+2);
72 | }
73 |
74 | console.log("Uploading files " + path.basename(filebase) + "* to S3...");
75 |
76 | // Create an array with all the metadata for the datellite pass
77 | // This applies to all images
78 | var metadata = {
79 | satellite: satellite,
80 | date: date,
81 | time: time,
82 | elevation: elevation,
83 | direction: direction,
84 | duration: duration,
85 | imageKey: filebase.slice(filebase.lastIndexOf('/')+1),
86 | tle1: tle1,
87 | tle2: tle2,
88 | frequency: frequency,
89 | gain: gain,
90 | chan_a: chan_a,
91 | chan_b: chan_b,
92 | images: []
93 | };
94 |
95 | // Function to upload images to S3
96 | async function uploadImage(image, filename) {
97 | // Get width and height of the image
98 | var w = image.bitmap.width;
99 | var h = image.bitmap.height;
100 | var enhancement;
101 |
102 | // Define the enhancement variable depending on the name of the file
103 | if (filename.endsWith("-ZA.png")) enhancement = "normal infrared";
104 | if (filename.endsWith("-NO.png")) enhancement = "color infrared";
105 | if (filename.endsWith("-MSA.png")) enhancement = "multispectral analysis";
106 | if (filename.endsWith("-MSA-precip.png")) enhancement = "multispectral precip";
107 | if (filename.endsWith("-MCIR.png")) enhancement = "map color infrared";
108 | if (filename.endsWith("-MCIR-precip.png")) enhancement = "map color infrared precip";
109 | if (filename.endsWith("-THERM.png")) enhancement = "thermal";
110 | if (filename.endsWith("-PRISTINE.png")) enhancement = "pristine view";
111 |
112 | // Define imageInfo variable with details of the image
113 | var imageInfo = {
114 | filename: filename,
115 | width: w,
116 | height: h,
117 | thumbfilename: 'thumbs/' + filename,
118 | enhancement: enhancement
119 | };
120 |
121 | // Upload image
122 | // Determine if watermark will be applied (0 = no watermark)
123 | if(process.env.WATERMARK==0){
124 | // Read image from local directory
125 | var upContent = fs.readFileSync(rootdirname + "images/" + filename);
126 | // Parameters needed to upload file to S3
127 | var upParams = {
128 | ACL: "public-read",
129 | ContentType: "image/png",
130 | Bucket: BUCKET,
131 | Key: IMAGE_DIR + filename,
132 | Body: upContent
133 | };
134 | // Upload file to S3
135 | uploadS3(upParams,filename);
136 |
137 | // Upload thumbs
138 | //Create a Jimp image that will be sized for thumbnails
139 | var newImage = await new Jimp(image.bitmap.width, image.bitmap.height+64, '#000000');
140 | newImage.composite(image, 0, 0);
141 | image = newImage;
142 | // Clone the image to the thumb variable
143 | var thumb = image.clone();
144 | // Scale the image to the given w & h, some parts of image may be clipped
145 | thumb.cover(260, 200);
146 | var thumbFilename = "thumbs/" + filename;
147 | // Put image in buffer to upload
148 | thumb.getBuffer(Jimp.MIME_PNG, (err, buffer) => {
149 | // Parameters needed to upload file to S3
150 | var params = {
151 | ACL: "public-read",
152 | ContentType: "image/png",
153 | Bucket: BUCKET,
154 | Key: IMAGE_DIR + thumbFilename,
155 | Body: buffer
156 | };
157 | var thumbName = "thumb/"+filename;
158 | // Upload file to S3
159 | uploadS3(params,thumbName);
160 | });
161 |
162 | } else {
163 | // Load font included in Jimp
164 | var font = await Jimp.loadFont(Jimp.FONT_SANS_16_WHITE);
165 | // Create a new image with width, height, and image background
166 | var newImage = await new Jimp(image.bitmap.width, image.bitmap.height+64, '#000000');
167 | if (filename.endsWith("-PRISTINE.png")){
168 | newImage.composite(image, 0, 0);
169 | image = newImage;
170 | console.log(filename + " image created (pristine)");
171 | } else{
172 | // Composites another Jimp image over image at x, y
173 | newImage.composite(image, 0, 48);
174 | image = newImage;
175 | // Print image details and location at the top of the image
176 | image.print(font, 5, 5, metadata.date + " " + metadata.time + " satellite: " + metadata.satellite +
177 | " elevation: " + metadata.elevation + '\xB0' + " enhancement: " + enhancement);
178 | image.print(font, 5, 25, LOCATION);
179 | console.log(filename + " image created");
180 | }
181 |
182 | // Put image in buffer to upload
183 | image.getBuffer(Jimp.MIME_PNG, (err, buffer) => {
184 | // Parameters needed to upload file to S3
185 | var params = {
186 | ACL: "public-read",
187 | ContentType: "image/png",
188 | Bucket: BUCKET,
189 | Key: IMAGE_DIR + filename,
190 | Body: buffer
191 | };
192 | // Upload file to S3
193 | uploadS3(params,filename);
194 | });
195 |
196 | // Upload thumbs
197 | // Clone the image to the thumb variable
198 | var thumb = image.clone();
199 | // Scale the image to the given w & h, some parts of image may be clipped
200 | thumb.cover(260, 200);
201 | var thumbFilename = "thumbs/" + filename;
202 | // Put image in buffer to upload
203 | thumb.getBuffer(Jimp.MIME_PNG, (err, buffer) => {
204 | // Parameters needed to upload file to S3
205 | var params = {
206 | ACL: "public-read",
207 | ContentType: "image/png",
208 | Bucket: BUCKET,
209 | Key: IMAGE_DIR + thumbFilename,
210 | Body: buffer
211 | };
212 | // Upload file to S3
213 | uploadS3(params,filename);
214 | });
215 | }
216 |
217 | // Return image information that was uploaded
218 | return imageInfo;
219 | }
220 |
221 | // Function to upload JSON file with all of the pass metadata
222 | function uploadMetadata(filebase) {
223 | // Create a promise to return
224 | return new Promise((resolve, reject)=> {
225 | // Upload JSON
226 | // Define name for the JSON file that contains all the metadata
227 | var metadataFilename = filebase + ".json";
228 | // Parameters needed to upload file to S3
229 | var params = {
230 | ACL: "public-read",
231 | Bucket: BUCKET,
232 | Key: IMAGE_DIR + metadataFilename,
233 | Body: JSON.stringify(metadata, null, 2)
234 | };
235 | var metaUp = uploadS3(params,metadataFilename);
236 |
237 | // Upload map
238 | // Define map file name and read from local directory
239 | var mapFilename = filebase + "-map.png";
240 | var mapContent = fs.readFileSync(rootdirname + "images/" + mapFilename);
241 | // Parameters needed to upload file to S3
242 | var mapParams = {
243 | ACL: "public-read",
244 | ContentType: "image/png",
245 | Bucket: BUCKET,
246 | Key: MAP_DIR + mapFilename,
247 | Body: mapContent
248 | };
249 | var mapUp = uploadS3(mapParams,mapFilename);
250 |
251 | //Upload Audio
252 | // Define wav file name and read from local directory
253 | var audioFilename = filebase + ".wav";
254 | var audioContent = fs.readFileSync(rootdirname + "audio/" + audioFilename);
255 | // Parameters needed to upload file to S3
256 | var audioParams = {
257 | ACL: "public-read",
258 | Bucket: BUCKET,
259 | Key: AUDIO_DIR + audioFilename,
260 | Body: audioContent
261 | };
262 | var audioUp = uploadS3(audioParams,audioFilename);
263 |
264 | //Upload Logs
265 | // Define log file name and read from local directory
266 | var logFilename = filebase + ".log";
267 | var logContent = fs.readFileSync(rootdirname + "logs/" + logFilename);
268 | // Parameters needed to upload file to S3
269 | var logParams = {
270 | ACL: "public-read",
271 | Bucket: BUCKET,
272 | Key: LOG_DIR + logFilename,
273 | Body: logContent
274 | };
275 | var logUp = uploadS3(logParams,logFilename);
276 |
277 | // Put all promises into a single promise array
278 | // Resolve when all promises have resolved
279 | Promise.all([metaUp, mapUp, audioUp, logUp]).then((result) => {
280 | resolve();
281 | });
282 | });
283 | }
284 |
285 | // Function to upload file to S3
286 | function uploadS3(params,fileName){
287 | return new Promise((resolve, reject)=> {
288 | s3.putObject(params, function(err, data) {
289 | if (err) {
290 | console.log(err)
291 | } else {
292 | console.log(" successfully uploaded to S3 - " + fileName);
293 | resolve();
294 | }
295 | });
296 | });
297 | }
298 |
299 | // Function to upload pass information to DynamoD
300 | function uploadtoDynamo(keyname){
301 | // Create the DynamoDB service object
302 | var docClient = new AWS.DynamoDB.DocumentClient();
303 | // Key name (name of JSON file to upload)
304 | var s3key = "images/" + keyname + ".json";
305 | console.log("Looking up log file for DynaoDB upload: " + s3key);
306 | //construct getParam
307 | var getParams = {
308 | Bucket: BUCKET,
309 | Key: s3key
310 | }
311 |
312 | //Fetch or read data from aws s3
313 | s3.getObject(getParams, function (err, data) {
314 |
315 | if (err) {
316 | console.log(err);
317 | } else {
318 |
319 | var content = data.Body.toString();
320 | var db_content = JSON.parse(content);
321 |
322 | // Replaces key names for date, time, and duration
323 | // to avoid reserved term conflicts with DynamoDB
324 | db_content.passDate = db_content.date;
325 | db_content.passTime = db_content.time;
326 | db_content.passDuration = db_content.duration;
327 | delete db_content.date;
328 | delete db_content.time;
329 | delete db_content.duration;
330 |
331 | // Create params for DynamoDB
332 | var dbParams = {
333 | TableName: 'passes-prod',
334 | Item: db_content,
335 | ReturnConsumedCapacity: "TOTAL"
336 | };
337 |
338 | console.log("Adding a new item to DynamoDB...");
339 |
340 | // Put command to upload info to DynamoDB
341 | docClient.put(dbParams, function(err, data) {
342 | if (err) {
343 | console.error("Unable to add item to DyanamoDB. Error JSON:", JSON.stringify(err, null, 2));
344 | } else {
345 | console.log("Added to DynamoDB - item:", JSON.stringify(keyname, null, 2));
346 | }
347 | });
348 | }
349 |
350 | })
351 | }
352 |
353 | // Find all the files that match the filebase plus a wildcard of capital
354 | // letters followed by any character and a png extension such as
355 | // /home/pi/wx-ground-station/images/NOAA15-20200227-141322-MCIR.png
356 | // this will leave out map file as it is handled in the metadata upload
357 | glob(filebase + "-[A-Z]*.png", {}, function (err, files) { //<- Old version
358 | //Create an array to upload files and store promise
359 | var uploadPromises = [];
360 |
361 | var keyname = components[0] + "-" + components[1] + "-" + components[2];
362 | //Iterate through each file
363 | files.forEach(function(filename) {
364 | // Get the last part of the path returned by filename
365 | // Ensures we only get file name such as NOAA15-20200227-141322-MCIR.png
366 | var basename = path.basename(filename);
367 | // Open the image file; using promise notation
368 | Jimp.read(filename)
369 | .then(image => {
370 | // Push new image to the uploadPromises array and call the uploadImage function
371 | uploadPromises.push(uploadImage(image, basename));
372 | // If the uploadPromise array is equal to the
373 | // number of files proceed to uploading metadata
374 | if (uploadPromises.length == files.length) {
375 | // Returns a single Promise that fulfills when all of the promises passed
376 | Promise.all(uploadPromises).then((values) => {
377 | // Set value for metadata.values in the array to
378 | // the 'values' variable in the returned promise
379 | metadata.images = values;
380 | // Call function to upload metadata to S3
381 | uploadMetadata(path.basename(filebase)).then(function(result) {
382 | console.log("Images uploaded to S3: " + JSON.stringify(values, null, 2));
383 | // Upload all information to DynamoDB
384 | uploadtoDynamo(keyname);
385 | // Call function to send a Discord message about the pass
386 | discord(satellite,keyname,elevation,direction);
387 | });
388 | });
389 | }
390 | })
391 | });
392 | });
393 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Automated NOAA Weather Satellite Ground Station
2 |
3 | 
4 |
5 | ## Introduction
6 |
7 | This project allows you to create a fully automated ground station that will receive and decode NOAA weather satellite images and upload them to your own website served from an Amazon AWS S3 bucket. [See my example S3 site](https://wximages.s3-us-west-1.amazonaws.com/index.html).
8 |
9 | This projected was originally adapted from an [Instructables article](https://www.instructables.com/id/Raspberry-Pi-NOAA-Weather-Satellite-Receiver/) by Jim Haslett that outlines how to customize a Raspberry Pi to receive weather images. The goal was to play the RPI close to the antenna to reduce feed-line loss.
10 |
11 | The project was then adapted in this [article](https://nootropicdesign.com/projectlab/2019/11/08/weather-satellite-ground-station/) where the images were automatically uploaded to S3 then displayed on a website.
12 |
13 | The goal of this branch is to extend the functionality that has been established and allow the weather station information to scale for weather stations that are constantly collecting information for server days, weeks, or months at a time.
14 |
15 | ---
16 |
17 | ## Installation Instructions
18 |
19 | Here’s what you’ll need:
20 |
21 | * Raspberry Pi (version 3 or 4) - you'll need Wi-Fi if you are using it outdoors or away from a network connection
22 | * Raspberry Pi case
23 | * MicroUSB power supply
24 | * 32GB SD card
25 | * SDR device - I recommend the RTL-SDR V3 dongle from RTL-SDR.COM blog
26 | * AWS account for hosting images and web content in an Amazon S3 bucket. You can sign up for the free tier for a year, and it’s still cheap after that
27 | * An antennea such as a dipole or a QFH
28 | * coaxial cable to go from your antenna to Raspberry Pi + RTL-SDR dongle. The dipole antenna kit comes with 3m of RG174 coax, but I used 10 feet of RG58 coax.
29 |
30 | >Dipole Antenna - You can build a simple dipole antenna with elements 21 inches (53.4 cm) long adjusted to a 120 degree angle between the elements. Here’s a great [article](http://lna4all.blogspot.com/2017/02/diy-137-mhz-wx-sat-v-dipole-antenna.html) on design or you can just buy [this dipole kit](https://www.rtl-sdr.com/product/rtl-sdr-blog-multipurpose-dipole-antenna-kit/) from the RTL-SDR.COM blog.
31 |
32 | ### Amazon AWS Setup
33 |
34 | The website for the ground station will be a serverless application hosted on S3. The RPI will use some Node.js scripts to upload all of the images and related assets. Here are the steps to setup your AWS account:
35 |
36 | #### AWS Credentials
37 |
38 | The scripts that run on the RPI are powered by Node.js and the AWS JavaScript SDK. To get them to all work, you will need to get credentials for your account. These two articles show you how to access or generate your credentials and store them for Node.js access:
39 |
40 | [Getting your credentials](https://docs.aws.amazon.com/en_pv/sdk-for-javascript/v2/developer-guide/getting-your-credentials.html)
41 | [Loading Credentials in Node.js from the Shared Credentials File](https://docs.aws.amazon.com/en_pv/sdk-for-javascript/v2/developer-guide/loading-node-credentials-shared.html)
42 |
43 | Your credentials file on the Raspberry Pi `~/.aws/credentials` will look like this:
44 |
45 | ```
46 | [default]
47 | aws_access_key_id = YOUR_ACCESS_KEY_ID
48 | aws_secret_access_key = YOUR_SECRET_ACCESS_KEY
49 | ```
50 |
51 | Set the default region where your S3 bucket will reside in `~/.aws/config`. For example:
52 |
53 | ```
54 | [default]
55 | output = json
56 | region = us-west-1
57 | ```
58 |
59 | #### Create an S3 Bucket
60 |
61 | Now create an S3 bucket for public website hosting such as `wximages`. The following articles shows how to setup a bucket for public website hosting:
62 | [Setting up a Static Website](https://docs.aws.amazon.com/en_pv/AmazonS3/latest/dev/HostingWebsiteOnS3Setup.html)
63 |
64 | At this point you should be able to load a simple web site from your new bucket. You might want to upload a simple `index.html` file and try to load it in your browser with `http://BUCKETNAME.s3-website-REGION.amazonaws.com/`.
65 |
66 | ```
67 |
68 |
69 | S3 test
70 | Hello from S3
71 |
72 | ```
73 |
74 | #### Create an Identity Pool in Cognito
75 |
76 | To give public users the ability to access your S3 bucket using the AWS SDK, you need to set up an identity pool and create a policy allowing them read access to your bucket. This is done using Amazon Cognito. A good guide for granting public access to your bucket is described in [this article](https://docs.aws.amazon.com/en_pv/sdk-for-javascript/v2/developer-guide/s3-example-photos-view.html) that shows how to serve images from an S3 bucket just like in this project.
77 |
78 | **Step 1:** [create an Amazon Cognito identity pool](https://docs.aws.amazon.com/en_pv/sdk-for-javascript/v2/developer-guide/getting-started-browser#getting-started-browser-create-identity-pool) called `wx image users` and enable access to unauthenticated identities. Be sure to select the region in the upper right of the page that matches the region where your S3 bucket was created! Make note of the role name for unauthorized users, e.g. `Cognito_wximageusersUnauth_Role`.
79 |
80 | **Step 2:** on the Sample Code page, select JavaScript from the Platform list. Save this code somewhere, because we need to add it to the web content later. It looks something like this:
81 |
82 | ```
83 | // Initialize the Amazon Cognito credentials provider
84 | AWS.config.region = 'us-west-1'; // Region
85 | AWS.config.credentials = new AWS.CognitoIdentityCredentials({
86 | IdentityPoolId: 'us-west-1:1d02ae39-3a06-497e-b63c-799a070dd09d',
87 | });
88 | ```
89 |
90 | **Step 3:** Add a Policy to the Created IAM Role. In [IAM console](https://console.aws.amazon.com/iam/), choose `Policies`. Click `Create Policy`, then click the JSON tab and add this, substituting BUCKET_NAME with your bucket name.
91 |
92 | ```
93 | {
94 | "Version": "2012-10-17",
95 | "Statement": [
96 | {
97 | "Effect": "Allow",
98 | "Action": [
99 | "s3:ListBucket"
100 | ],
101 | "Resource": [
102 | "arn:aws:s3:::BUCKET_NAME"
103 | ]
104 | }
105 | ]
106 | }
107 | ```
108 |
109 | Click `Review policy` and give your policy a name, like `wxImagePolicy`.
110 |
111 | In IAM console, click `Roles`, then choose the unauthenticated user role previously created when the identity pool was created (e.g. `Cognito_wximageusersUnauth_Role`). Click `Attach Policies`. From the `Filter policies` menu, select `Customer managed`. This will show the policy you created above. Select it and click Attach policy.
112 |
113 | **Step 4.** Set CORS configuration on the S3 bucket. In the S3 console for your bucket, select `Permissions`, then `CORS configuration`.
114 |
115 | ```
116 |
117 |
118 |
119 | *
120 | GET
121 | HEAD
122 | *
123 |
124 |
125 | ```
126 | ### Raspberry Pi Setup
127 |
128 | Install Required Packages
129 | First, make sure your Raspberry Pi is up to date:
130 |
131 | ```
132 | sudo apt-get update
133 | sudo apt-get upgrade
134 | sudo reboot
135 | ```
136 |
137 | Then install a set of of required packages.
138 |
139 | ```
140 | sudo apt-get install libusb-1.0
141 | sudo apt-get install cmake
142 | sudo apt-get install sox
143 | sudo apt-get install at
144 | sudo apt-get install predict
145 | ```
146 |
147 | I used Node.js in some of the scripting, so if you don’t have `node` and `npm` installed, you’ll need to do that. In depth [details are here](https://github.com/nodesource/distributions/#deb), and I easily installed with:
148 |
149 | ```
150 | curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
151 | sudo apt-get install -y nodejs
152 | ```
153 |
154 | Using your favorite editor as root (e.g. `nano`), create a file `/etc/modprobe.d/no-rtl.conf` and add these contents:
155 |
156 | ```
157 | blacklist dvb_usb_rtl28xxu
158 | blacklist rtl2832
159 | blacklist rtl2830
160 | ```
161 |
162 | #### Build rtl-sdr
163 |
164 | Even if you have `rtl-sdr` already built and installed, it’s important to use the version in the GitHub repo [keenerd/rtl-sdr](https://github.com/keenerd/rtl-sdr), as this version’s `rtl_fm` command can create the WAV file header needed to decode the data with `sox`.
165 |
166 | ```
167 | cd ~
168 | git clone https://github.com/keenerd/rtl-sdr.git
169 | cd rtl-sdr/
170 | mkdir build
171 | cd build
172 | cmake ../ -DINSTALL_UDEV_RULES=ON
173 | make
174 | sudo make install
175 | sudo ldconfig
176 | cd ~
177 | sudo cp ./rtl-sdr/rtl-sdr.rules /etc/udev/rules.d/
178 | sudo reboot
179 | ```
180 |
181 | #### Install and Configure wxtoimg
182 |
183 | The program `wxtoimg` is what does the heavy lifting in this project. It decodes the audio files received by the RTL-SDR receiver and converts the data to images. The original author of wxtoimg has abandoned the project, but it is mirrored at [wxtoimgrestored.xyz](https://wxtoimgrestored.xyz/).
184 |
185 | ```
186 | wget https://wxtoimgrestored.xyz/beta/wxtoimg-armhf-2.11.2-beta.deb
187 | sudo dpkg -i wxtoimg-armhf-2.11.2-beta.deb
188 | ```
189 |
190 | Now run `wxtoimg` once to accept the license agreement.
191 |
192 | ```
193 | wxtoimg
194 | ```
195 |
196 | Create a file `~/.wxtoimgrc` with the location of your base station. As usual, negative latitude is southern hemisphere, and negative longitude is western hemisphere. Here is what it would look like for the White House as an example.
197 |
198 | ```
199 | Latitude: 38.8977
200 | Longitude: -77.0365
201 | Altitude: 15
202 | ```
203 |
204 | The program `predict` is used by the automated scripts to predict weather satellite orbits. Run `predict` to bring up the main menu. Select option ‘G’ from the menu to set your ground station location.
205 |
206 | You can enter whatever you want for the callsign such as your amateur radio callsign. **When entering the longitude, note that positive numbers are for the western hemisphere and negative numbers are for the eastern hemisphere.** This is opposite convention, so make sure you get this right or you’ll be listening when there’s no satellite overhead!
207 |
208 | #### Get the Automation Scripts and Configure
209 |
210 | The following scripts will automate the thumbnail images and then upload all images to S3. The git repo can be cloned anywhere on your Raspberry Pi. The `configure.sh` script sets the installation directory in the scripts and schedules a cron job to run the satellite pass scheduler job at midnight every night. The scheduler identifies times when each satellite will pass overhead and create an `at` one time job to start the recording, processing, and upload steps.
211 |
212 | ```
213 | git clone https://github.com/alonsovargas3/wx-ground-station.git
214 | cd wx-ground-station
215 | sh configure.sh
216 | cd aws-s3
217 | npm install
218 | ```
219 |
220 | In the file `aws-s3/.env` set REGION, BUCKET, and STATION_LOCATION to the correct values. The Node.js script prepares the images for upload by creating thumbnail images, optionally printing some metadata on the images, and creating a JSON metadata file for each image capture. If you choose to add a watermark the STATION_LOCATION string will be printed on the images that you capture. Here are my values just for reference.
221 |
222 | ```
223 | #S3
224 | AWS_REGION=us-west-2
225 | AWS_BUCKET=wx.k6kzo.com
226 |
227 | #WATERMARK
228 | WATERMARK=0
229 | STATION_LOCATION="K6KZO Ground Station, Austin, Texas, USA"
230 |
231 | #DIRECTORIES
232 | DISCORD_WEBHOOK = [Enter webook here]
233 | WEBSITE_ADDR = https://wx.k6kzo.com
234 | ```
235 |
236 | After that, you will need to setup the API that will provide all of the pass data. A serverless project is setup in the `aws-api` folder. You will just need to confirm one piece of information within the `serverless.yml` file. Open the file and specify the region. Make sure it is the same region as the rest of your other services like S3 as some of the functionality will depend on this configuration. I have used us-west-2 throughout. If you use the same region for your setup then you will not need to make any modifications.
237 |
238 | To deploy your API follow these commands:
239 |
240 | ```
241 | cd aws-api
242 | npm install
243 | sls deploy
244 | ```
245 |
246 | This will build your database in DynamoDB, create a Lambda service that will read the information from DynamoDB, and create an API Gateway to access the information from Lambda. When the process completes you should see the Service Information. You will need to find the following value. Note that your URL will be different.
247 |
248 | ```
249 | endpoints:
250 | GET - https://e78uek07l4.execute-api.us-west-2.amazonaws.com/prod/passes
251 | ```
252 |
253 | Record this value to add to the PASS_URL in the next steps when you update `s3-vars.js`.
254 |
255 | You'll also want to confirm you see the following value right under the endpoint:
256 |
257 | ```
258 | functions:
259 | getPasses: passes-prod-getPasses
260 | ```
261 |
262 | **If you run into any CORS issues with the API, you will need to enable CORS for the API endpoint.** CORS has been enabled in the serverless.yml file but sometimes issues can arise that are fixed by enabling CORS. After you enable CORS support on your resource, you must deploy or redeploy the API for the new settings to take effect. To do this, run `sls deploy` again in the `aws-api` directory.
263 |
264 | Next you will need to make some changes to the web content. The web interface uses Mapbox to draw the live maps of the next upcoming satellite pass. You’ll need to create an account at [Mapbox](https://mapbox.com/) to get an access token. Their free tier lets you load 50,000 maps/month, so you are not likely to have any real costs. When logged into Mapbox, get your account token from [https://account.mapbox.com/](https://account.mapbox.com/).
265 |
266 | Next, in the file `website/s3-vars.js`, set your bucket name, AWS region, AWS credentials (the Cognito identity pool info you saved above), Mapbox token, and your ground station info. Some of my values are shown here for reference.
267 |
268 | ```
269 | var bucketName = 'wx.k6kzo.com';
270 | AWS.config.region = 'us-west-2'; // Region
271 | AWS.config.credentials = new AWS.CognitoIdentityCredentials({
272 | IdentityPoolId: 'us-west-1:1d02ae39-30a6-497e-b066-795f070de089'
273 | });
274 |
275 | // Create a mapbox.com account and get access token
276 | const MAP_BOX_ACCESS_TOKEN = 'YOUR_MAPBOX_TOKEN';
277 | const GROUND_STATION_LAT = 38.8977;
278 | const GROUND_STATION_LON = -77.0365;
279 | const GROUND_STATION_NAME = 'K6KZO ground station';
280 | const MAX_CAPTURES = 10;
281 | const DIR_NAME = "images";
282 | const PASS_URL = "[Serverless API Endpoint]";
283 |
284 | ```
285 |
286 | #### Upload the Web Content to S3
287 |
288 | Upload the the contents of the `website` directory to your S3 bucket using the S3 console. Since you probably edited the files on your Raspberry Pi, you might need to copy them to your computer where you are accessing AWS using a browser. Whatever the case, these files need to be uploaded to the top level of your bucket. IMPORTANT: be sure to grant public access to the files when you upload them!
289 |
290 | ```
291 | index.html
292 | wx-ground-station.js
293 | s3-vars.js
294 | tle.js
295 | moment.js
296 | logo.png
297 | ```
298 |
299 | Of course, you can replace `logo.png` with your own, or just remove the `` tag from `index.html`.
300 |
301 | ### Test Everything Out
302 |
303 | Now that everything is configured, let’s run the scheduling script to schedule recording of upcoming satellite passes. This way you can have a look today instead of waiting until they get scheduled at midnight. This step will also upload a JSON file with the upcoming passes info to your website.
304 |
305 | ```
306 | cd wx-ground-station
307 | ./schedule_all.sh
308 | ```
309 |
310 | You can now visit your AWS S3 website endpoint at
311 |
312 | ```
313 | http://BUCKETNAME.s3-website-REGION.amazonaws.com/
314 | ```
315 |
316 | Even though you don’t have any images captured, you should be able to see the next upcoming pass. The next thing to do is make sure the scripts work correctly to record the audio file, process it into images, and upload to your bucket. You can watch the logs in the `wx-ground-station/logs` to debug any errors.
317 |
318 | The `wxtoimg` enhancements that are displayed depends on what sensors were active when the images were captured. If sensors 3 and 4 were active (usually at night), then the thermal enhancement will be shown. Otherwise a multispectral analysis enhancement will be shown.
319 |
320 | Not all images you capture will be good; the satellite may be too low or you may not get a good signal. You can clean up bad ones by using the script `aws-s3/remove-wx-images.js` on the Raspberry Pi. Just provide the key to the particular capture as an argument to remove all the images and the metadata from the S3 bucket.
321 |
322 | ```
323 | node aws-s3/remove-wx-images.js NOAA19-20191108-162650
324 | ```
325 |
326 | In the next few hours you’ll be able to see some images uploaded, depending on when satellites are scheduled to fly over. You may get up to 12 passes per day, usually 2 for each of the NOAA satellites in the morning, then 2 more for each of them in the evening.
327 |
328 | **Note - If you are upgrading from a previous version follow the steps below**
329 |
330 | You may be upgrading from a previous version and need to upload all of your information. First you will need to replace your installation with the one in this project. After that is complete, manually upload all JSON files into the images folder of your S3 bucket. After that you will be able to use the file `aws-s3/upload-existing.js` on the Raspberry Pi. To do so run the following command:
331 |
332 | ```
333 | node aws-s3/upload_existing.js
334 | ```
335 |
336 | After a few moments all of the pass information in each JSON file will be uploaded to DynamoDB and they will be visible on your website.
337 |
338 | #### Optional CloudFront Setup with Custom URL
339 |
340 | As an optional step you can setup a CloudFront distribution to serve your website from a CDN. This also allows you to deploy a custom URL with an SSL certificate. To do so, see this [article](https://docs.aws.amazon.com/AmazonS3/latest/dev/website-hosting-cloudfront-walkthrough.html) for setting up CloudFront and this [article](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-https-alternate-domain-names.html) on how to setup a custom URL.
341 |
342 | ### Fine Tuning
343 |
344 | The script `receive_and_process_satellite.sh` uses the `rtl_fm` command to read the signal from the RTL-SDR receiver. The -p argument sets the PPM error correction. Most of the time it is set to 0, but you may want to adjust. See [this](https://davidnelson.me/?p=371) article for details.
345 |
346 | ### Low Noise Amplifier
347 |
348 | You can also install a low noise amplifier (LNA) to improve reception (results are mixed). Some LNAs can be powered with a bias tee circuit and controlled with the `rtl_biast` command.
349 |
350 | In Linux or MacOS download the source from git, compile it the same way you do the regular RTL-SDR drivers, and then run ./rtl_biast -b 1 to turn the bias tee on and ./rtl_biast -b 0 to turn the bias tee off. The procedure is:
351 |
352 | ```
353 | git clone https://github.com/rtlsdrblog/rtl-sdr-blog
354 | cd rtl-sdr-blog
355 | mkdir build
356 | cd build
357 | cmake .. -DDETACH_KERNEL_DRIVER=ON
358 | make
359 | cd src
360 | ./rtl_biast -b 1
361 | ```
362 |
363 | If you want to be able to run the bias tee program from anywhere on the command line you can also run "sudo make install".
364 |
365 | If you have trouble running the bias tee check with a multimeter if there is 4.5V at the SMA port. Also check that your powered device is actually capable of receiving power. Remember that not all LNA's can accept bias tee power.
366 |
367 | Once installed, uncomment the `rtl_biast` lines toward the top of `receive_and_process_satellite.sh`. This will turn the LNA on right before starting to record and off after capturing the signal. **Make sure that you also update the path if you have installed `rtl_biast` in a different location**
368 |
--------------------------------------------------------------------------------
/website/wx-ground-station.js:
--------------------------------------------------------------------------------
1 | // Load new instance of TLE.js
2 | var tlejs = new TLEJS();
3 | var lastPositionOfNextPass;
4 | var nextPass = null
5 |
6 | // Get SATCAT number for satellite and
7 | // return s n2yo link for the satellite
8 | function getSatelliteLink(tles) {
9 | var satNum = tlejs.getSatelliteNumber(tles);
10 | return "https://www.n2yo.com/satellite/?s=" + satNum;
11 | }
12 |
13 | // Get URL parameters
14 | // This will be use to get page numbers
15 | function getURLParameter(name, url) {
16 | if (!url) url = window.location.href;
17 | name = name.replace(/[\[\]]/g, '\\$&');
18 | var regex = new RegExp('[?&]' + name + '(=([^]*)|&|#|$)'),
19 | results = regex.exec(url);
20 | if (!results) return null;
21 | if (!results[2]) return '';
22 | return decodeURIComponent(results[2].replace(/\+/g, ' '));
23 | }
24 |
25 | function load() {
26 | $('#location').html(GROUND_STATION_LAT + ', ' + GROUND_STATION_LON);
27 | getUpcomingPassInfo();
28 | getAllUpcomingPasses();
29 | getDynamoPasses(function (metadata) {
30 |
31 | $('#messages').html('');
32 |
33 | // show newest first
34 | var sortedMeta = metadata.sort(function (m1, m2) {
35 | var m1key = m1.passDate + "-" + m1.passTime;
36 | var m2key = m2.passDate + "-" + m2.passTime;
37 | return (m1key > m2key) ? -1 : 1;
38 | });
39 |
40 | var captureCount = 0;
41 |
42 | // Function to convert time for each pass
43 | function convertToLocal(date,time){
44 | var combinedDate = new Date(date+" "+time);
45 | var local = moment.utc(combinedDate).local().format('YYYY-MM-DD HH:mm:ss');
46 | return local;
47 | }
48 |
49 | // Pagination
50 | // Get sorted JSON array
51 | var json = sortedMeta;
52 | // Find the total number of records in the array
53 | var totalJSON = json.length;
54 | // Determine number of pages that will need to be displayed
55 | // Rounding up to ensure the last set of records are visible
56 | var totalPages = Math.ceil(totalJSON/MAX_CAPTURES);
57 | // Current pages
58 | var page = getURLParameter('page');
59 | if(page==null){page=1;}
60 | // Variable to add active class to list element
61 | var activeClass,disableClass,lastClass;
62 | // Variables for Prev and next
63 | var prevPage = parseInt(page) - 1;
64 | var nextPage = parseInt(page) + 1;
65 | // Create list elements
66 | for (let i=1; i<=totalPages; i++) {
67 | // If on the first page, disable previous
68 | if(page==1){ disableClass='disabled';} else { disableClass='';}
69 | // If on th elast page, disable next
70 | if(page==totalPages){ lastClass='disabled';} else { lastClass='';}
71 | // Display current page number as active
72 | if(page==i){ activeClass = 'active';} else {activeClass='';}
73 | // List elements
74 | if(i==1){$('#pages').append(['
'].join(''));}
77 | }
78 | // Determine how many records to show per page
79 | var recPerPage = MAX_CAPTURES;
80 | // Use Math.max to ensure that we at least start from record 0
81 | var startRec = Math.max(page - 1, 0) * recPerPage;
82 | // Define end of array and stay within bounds of record set
83 | var endRec = Math.min(startRec + recPerPage, totalJSON);
84 | // Create JSON array for current page with appropriate records
85 | var recordsToShow = json.splice(startRec, endRec);
86 |
87 | // Displays each pass
88 | recordsToShow.forEach(function (m) {
89 | if (++captureCount > MAX_CAPTURES) return;
90 | if (m == null) return;
91 | var mapId = m.imageKey + '-gt';
92 | var satLink = '' + m.satellite + '';
93 | $('#previous_passes').append([
94 | //' ',
95 | '
'].join('')
321 | );
322 | }
323 | });
324 | }
325 |
326 | // Gets all upcoming satellite passes for the given LAT / LONG
327 | // This is used to display upcoming pass
328 | function getUpcomingPassInfo() {
329 |
330 | // Load upcoming_passes.json file using a HTTP GET request
331 | $.get(DIR_NAME + "/upcoming_passes.json", function(data) {
332 | var now = new Date();
333 | var processingTime = 240000; // approx 4 minutes to process and upload images.
334 | // Loop through all upcoming passes to find next pass by looking at the end
335 | // time of each pass and determining if it is later than the current time
336 | // Note - upcoming passes file is in order of time and is loaded the same way
337 | // Note2 - using end time ensures next pass will not show until current is complete
338 | for(var i=0;i now)) {
341 | nextPass = data[i];
342 | }
343 | }
344 | // Link to satellite for next pass
345 | var satLink = '' + nextPass.satellite + '';
346 | // Start and end time for next pass
347 | var startDate = new Date(nextPass.start);
348 | var endDate = new Date(nextPass.end + processingTime);
349 | // Populates upcoming_passes
with next pass information
350 | $("#upcoming_passes").append([
351 | '