10 | There is currently no catalog update in progress. Enter a semester name to start one.
11 |
12 |
13 |
28 |
29 | {% endblock %}
30 |
--------------------------------------------------------------------------------
/recommend/models.py:
--------------------------------------------------------------------------------
1 | from django.db import models
2 | from django.forms import ModelForm
3 | from django.contrib.auth.models import User
4 |
5 | MAX_RATING_VALUE = 5
6 | DEFAULT_RECOMMENDATION_TYPE = "for-you"
7 |
8 | # Create your models here.
9 | class Rating(models.Model):
10 | user = models.ForeignKey(User, null=True, on_delete=models.CASCADE)
11 | #user_id = models.BigIntegerField(default=0) # DEPRECATED
12 | subject_id = models.CharField(max_length=50)
13 | value = models.IntegerField(default=0)
14 |
15 | def __str__(self):
16 | return "User {} rated {} as {}".format(self.user_id, self.subject_id, self.value)
17 |
18 | class Recommendation(models.Model):
19 | user = models.ForeignKey(User, null=True, on_delete=models.CASCADE)
20 | #user_id = models.BigIntegerField(default=0) # DEPRECATED
21 | rec_type = models.CharField(max_length=20)
22 | subjects = models.CharField(max_length=500)
23 |
24 | def __str__(self):
25 | return "Recommendation ({}) for user {}: {}".format(self.rec_type, self.user_id, self.subjects)
26 |
--------------------------------------------------------------------------------
/license.txt:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2018 Venkatesh Sivaraman
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/requirements/urls.py:
--------------------------------------------------------------------------------
1 | from django.conf.urls import url
2 |
3 | from . import views
4 | from . import editor
5 |
6 | from django.views.generic import TemplateView
7 |
8 | urlpatterns = [
9 | url(r'^edit/(?P.{1,50})', editor.edit, name='requirements_edit'),
10 | url(r'^success/', editor.success, name='submit_success'),
11 | url(r'^create/', editor.create, name='create'),
12 | url(r'^preview/', editor.preview, name='preview'),
13 | url(r'^review/(?P\d+)', editor.review, name='review'),
14 | url(r'^review/', editor.review_all, name='review_all'),
15 | url(r'^resolve/(?P\d+)', editor.resolve, name='resolve'),
16 | url(r'^ignore_edit/(?P\d+)', editor.ignore_edit, name='ignore_edit'),
17 | url(r'^uncommit/(?P\d+)', editor.uncommit, name='uncommit'),
18 | url(r'^commit/(?P\d+)', editor.commit, name='commit'),
19 | url(r'^list_reqs/', views.list_reqs, name='list_reqs'),
20 | url(r'^get_json/(?P.{1,50})/', views.get_json, name='get_json'),
21 | url(r'^progress/(?P.{1,50})/(?P.+)', views.progress, name='progress'),
22 | url(r'^progress/(?P.{1,50})/', views.road_progress, name='road_progress'),
23 | url(r'^$', editor.index, name='requirements_index'),
24 | ]
25 |
--------------------------------------------------------------------------------
/common/templates/common/docs/overview.html:
--------------------------------------------------------------------------------
1 | {% extends "common/docs/base.html" %}
2 |
3 | {% block nav %}
4 | {% include "common/docs/sidebar.html" with active_id="overview" %}
5 | {% endblock %}
6 |
7 | {% block content %}
8 |
FireRoad API Reference
9 |
The FireRoad server provides a comprehensive API, with both read-only interfaces for course catalogs and major/minor requirements and read-write endpoints for interfacing with users' roads and schedules.
10 |
The read-only API is open-access, and logging in with your MIT credentials allows you to test the read-write APIs within your own account. Please contact us if you would like to use the read-write APIs to create a FireRoad client that accesses other user data.
11 |
IMPORTANT: Please use the development server at fireroad-dev.mit.edu for all testing.
Would you like to sync your roads and schedules with CourseRoad and across your devices? FireRoad will also provide you with personalized suggestions based on your subject selections.
8 |
Tap Allow and Continue to enable this feature. You can also do so later in the settings view.
9 |
Sync and Recommendations requires MIT credentials. By enabling this feature, you are allowing FireRoad to store your roads, schedules, and subject ratings on a secure MIT server. FireRoad uses various machine learning algorithms to provide you with subject recommendations and to improve other users' recommendations anonymously. Recommendations are solely suggestions - please consult with faculty and course administrators for the most reliable and up-to-date information.
"{{ client_name }}" is requesting access to your FireRoad account. This client will be granted the following capabilities:
8 |
9 | {% for permission in client_permissions %}
10 |
{{ permission }}
11 | {% endfor %}
12 |
13 | {% if is_debug %}
14 |
WARNING: This site is authenticating you through the FireRoad development server, which is subject to potential vulnerabilities and does not restrict client sites to specific permissions. This means of authentication must not be used for production applications.
Use these endpoints to update and access various course recommendations generated by the FireRoad recommendation engine.
10 |
11 |
/recommend/rate (POST)
12 |
Sends a subject rating to the FireRoad server. The body of the request should be a JSON list of dictionaries, each containing s (subject ID) and v (rating value). Updates the ratings for each item.
13 |
14 |
/recommend/get (GET)
15 |
Takes an optional parameter t indicating the type of recommendation to return. Returns a dictionary of recommendation types mapped to JSON strings indicating the recommended subjects and their rating values.
FireRoad is the new best way to plan your path through MIT. Plan for both the upcoming semester and the years ahead, all in one place. You can view up-to-date course requirements and add them directly to your roads, and browse subjects by any criteria. FireRoad even learns from your selections to recommend other subjects you might be interested in!
{% if is_new %}New Correction{% else %}Edit Correction{% endif %}
14 |
15 |
16 |
Enter a subject ID and values for the fields that you would like to modify. Original values from the catalog parser will be used for any fields left blank. Hint: You can use the '*' character in the subject ID to match any number in a position.
17 |
18 |
19 |
58 |
59 | {% endblock %}
60 |
61 | {% block pagescripts %}
62 |
63 | {% endblock %}
64 |
--------------------------------------------------------------------------------
/analytics/models.py:
--------------------------------------------------------------------------------
1 | from django.db import models
2 | from django.utils import timezone
3 |
4 | class RequestCount(models.Model):
5 | """Keeps track of a single request."""
6 |
7 | path = models.CharField(max_length=50, null=True)
8 | timestamp = models.DateTimeField(auto_now_add=True, db_index=True)
9 | user_agent = models.CharField(max_length=150, null=True)
10 | is_authenticated = models.BooleanField(default=False)
11 | student_unique_id = models.CharField(max_length=50, null=True)
12 | student_semester = models.CharField(max_length=25, null=True)
13 |
14 | def __str__(self):
15 | return "{} by {} at {}{}".format(
16 | self.path,
17 | self.user_agent,
18 | self.timestamp,
19 | " (logged in)" if self.is_authenticated else ""
20 | )
21 |
22 | @staticmethod
23 | def tabulate_requests(early_time, interval=None, attribute_func=None, distinct_users=False):
24 | """Retrieves request counts from the given time to present,
25 | bucketed by the given interval.
26 |
27 | Args:
28 | early_time: A timezone.datetime object indicating the minimum time
29 | to retrieve requests for.
30 | interval: A timezone.timedelta object indicating the period of time
31 | spanned by each returned bucket. If None, counts all requests
32 | together and returns a single dictionary.
33 | attribute_func: A function taking a RequestCount and returning a
34 | value to tabulate for each bucket.
35 | distinct_users: If True, count only one request per unique user.
36 |
37 | Returns:
38 | A list of tuples (time, dict), where time is a timezone.datetime
39 | object indicating the start time of the bucket, and dict is a
40 | dictionary mapping values returned by attribute_func to their
41 | counts in the bucket.
42 | """
43 | now = timezone.now()
44 | buckets = []
45 | if interval:
46 | curr = early_time
47 | while curr < now:
48 | buckets.append((curr, {}))
49 | curr += interval
50 | else:
51 | buckets.append((early_time, {}))
52 |
53 | seen_users = set()
54 | for request in RequestCount.objects.filter(timestamp__gte=early_time).iterator():
55 | if distinct_users and (not request.student_unique_id or request.student_unique_id in seen_users):
56 | continue
57 | seen_users.add(request.student_unique_id)
58 | for time, bucket in buckets:
59 | if request.timestamp >= time and (not interval or request.timestamp < time + interval):
60 | value = attribute_func(request) if attribute_func else None
61 | bucket[value] = bucket.get(value, 0) + 1
62 | break
63 |
64 | if not interval:
65 | return buckets[0][1]
66 | return buckets
67 |
--------------------------------------------------------------------------------
/recommend/views.py:
--------------------------------------------------------------------------------
1 | from django.shortcuts import render, redirect
2 | from django.http import HttpResponse, HttpResponseBadRequest
3 | from django.views.decorators.csrf import csrf_exempt
4 | from .models import *
5 | from common.models import *
6 | import random
7 | from django.contrib.auth import login, authenticate, logout
8 | from django.core.exceptions import PermissionDenied
9 | from common.decorators import logged_in_or_basicauth, require_token_permissions
10 | import json
11 | from django.utils import timezone
12 | from dateutil.relativedelta import relativedelta
13 |
14 | def update_rating(user, subject_id, value):
15 | Rating.objects.filter(user=user, subject_id=subject_id).delete()
16 |
17 | r = Rating(user=user, subject_id=subject_id, value=value)
18 | r.save()
19 |
20 | @logged_in_or_basicauth
21 | @require_token_permissions("can_view_recommendations")
22 | def get(request):
23 | rec_type = request.GET.get('t', '')
24 | if len(rec_type) == 0:
25 | recs = Recommendation.objects.filter(user=request.user)
26 | else:
27 | recs = Recommendation.objects.filter(user=request.user, rec_type=rec_type)
28 | if recs.count() == 0:
29 | return HttpResponse('No recommendations yet. Try again tomorrow!')
30 | resp = {rec.rec_type: json.loads(rec.subjects) for rec in recs}
31 | return HttpResponse(json.dumps(resp), content_type="application/json")
32 |
33 | @csrf_exempt
34 | @logged_in_or_basicauth
35 | @require_token_permissions("can_view_recommendations", "can_edit_student_info")
36 | def rate(request):
37 | batch = request.body
38 | if len(batch) > 0:
39 | batch_items = json.loads(batch)
40 | for item in batch_items:
41 | try:
42 | value = int(item['v'])
43 | if item['s'] == None or len(item['s']) > 10:
44 | return HttpResponseBadRequest('
Missing subject ID
')
45 | if value is None:
46 | return HttpResponseBadRequest('
')
66 | return HttpResponse(json.dumps(resp), content_type="application/json")
67 |
--------------------------------------------------------------------------------
/requirements/static/requirements/css/editor.css:
--------------------------------------------------------------------------------
1 | .editor {
2 | resize: none;
3 | border: none;
4 | display: table-cell;
5 | flex: 1; /* same as flex: 1 1 auto; */
6 | }
7 | .code, .editor {
8 | font-family: 'Inconsolata', Menlo, Monaco, Courier, monospace;
9 | line-height: 1.5em;
10 | }
11 | .code {
12 | padding: 2px;
13 | }
14 | .editor-card {
15 | width: 100%;
16 | padding: 8px;
17 | display: flex;
18 | flex-direction: column;
19 | }
20 | .editor-card {
21 | flex-grow: 1;
22 | margin-bottom: 32px !important;
23 | }
24 | .content {
25 | margin-left: 32px;
26 | margin-right: 32px;
27 | display: flex;
28 | flex-direction: column;
29 | margin-bottom: 12px;
30 | }
31 | .content-full-screen {
32 | margin-left: 32px;
33 | margin-right: 32px;
34 | display: flex;
35 | flex-direction: column;
36 | margin-bottom: 12px;
37 | }
38 | @media (min-width: 767px) {
39 | .edit-head {
40 | margin-top: 12px;
41 | }
42 | .content {
43 | margin-top: -64px;
44 | padding-top: 64px;
45 | }
46 | .content-full-screen {
47 | margin-top: -64px;
48 | padding-top: 64px;
49 | }
50 | }
51 | .mbtn:hover {
52 | color: #940404;
53 | text-decoration: none;
54 | }
55 | .input-field:focus {
56 | border-bottom: 1px solid #a10000 !important;
57 | box-shadow: 0 1px 0 0 #a10000 !important;
58 | }
59 | ::-webkit-input-placeholder { color: #996767 !important; }
60 | :-moz-placeholder { /* Firefox 18- */ color: #996767 !important; }
61 | ::-moz-placeholder { /* Firefox 19+ */ color: #996767 !important; }
62 | :-ms-input-placeholder { color: #996767 !important; }
63 |
64 | .real-ul > li {
65 | margin-left: 20px !important;
66 | list-style-type: square !important;
67 | }
68 |
69 | ::selection {
70 | background-color: #f46565;
71 | }
72 |
73 | body, html, .content, .content-full-screen {
74 | height: 100%;
75 | }
76 |
77 | .diff-line {
78 | margin-bottom: -8px;
79 | word-wrap: break-word;
80 | }
81 |
82 | .insertion {
83 | color: rgb(48, 162, 93);
84 | background-color: rgba(82, 213, 134, 0.2);
85 | padding: 2px;
86 | border-radius: 2px;
87 | }
88 |
89 | .deletion {
90 | color: rgb(179, 32, 1);
91 | background-color: rgba(210, 47, 12, 0.2);
92 | padding: 2px;
93 | border-radius: 2px;
94 | text-decoration: line-through;
95 | }
96 |
97 | .edit-req-list-card {
98 | width: 100%;
99 | padding: 8px;
100 | display: flex;
101 | flex-direction: column;
102 | flex-grow: 1;
103 | }
104 |
105 | .edit-req-list-row {
106 | display: flex;
107 | flex-direction: row;
108 | overflow: hidden;
109 | }
110 |
111 | .edit-req-list-left {
112 | display: flex;
113 | flex-direction: column;
114 | width: 50%;
115 | margin-right: 12px;
116 | }
117 |
118 | .edit-req-list-right {
119 | display: flex;
120 | flex-direction: column;
121 | width: 50%;
122 | }
123 |
124 | .edit-req-list {
125 | overflow-y: scroll !important;
126 | }
127 |
128 | .edit-req-list > .collection-item:hover {
129 | background-color: #eeeeee;
130 | -webkit-transition: background-color 250ms linear;
131 | -ms-transition: background-color 2500ms linear;
132 | transition: background-color 250ms linear;
133 | }
134 |
--------------------------------------------------------------------------------
/catalog_parse/utils/parse_evaluations.py:
--------------------------------------------------------------------------------
1 | import re
2 | from .catalog_constants import *
3 | import json
4 | import requests
5 |
6 | class EvaluationConstants:
7 | rating = "rating"
8 | term = "term"
9 | in_class_hours = "ic_hours"
10 | out_of_class_hours = "oc_hours"
11 | eligible_raters = "eligible"
12 | responded_raters = "resp"
13 | iap_term = "JA"
14 |
15 | KEYS_TO_AVERAGE = {
16 | EvaluationConstants.rating: CourseAttribute.averageRating,
17 | EvaluationConstants.in_class_hours: CourseAttribute.averageInClassHours,
18 | EvaluationConstants.out_of_class_hours: CourseAttribute.averageOutOfClassHours,
19 | EvaluationConstants.eligible_raters: CourseAttribute.enrollment
20 | }
21 |
22 | def load_evaluation_data(eval_path):
23 | """
24 | Reads evaluation data from the given .js file.
25 | """
26 |
27 | with open(eval_path, 'r') as file:
28 | eval_contents = file.read()
29 | begin_range = eval_contents.find("{")
30 | end_range = eval_contents.rfind(";")
31 | return json.loads(eval_contents[begin_range:end_range])
32 |
33 | def parse_evaluations(evals, courses):
34 | """
35 | Adds attributes to each course based on eval data in the given dictionary.
36 | """
37 | for i, course in courses.iterrows():
38 | # i is the index of the DataFrame, which is the subject ID
39 | subject_ids = list(filter(None, [i, course[CourseAttribute.oldID]]))
40 | if not any(x in evals for x in subject_ids):
41 | continue
42 |
43 | averaging_data = {}
44 | for subject_id in subject_ids:
45 | if subject_id not in evals:
46 | continue
47 | for term_data in evals[subject_id]:
48 | # if course is offered fall/spring but an eval is for IAP, ignore
49 | if (EvaluationConstants.iap_term in term_data[EvaluationConstants.term] and
50 | (course[CourseAttribute.offeredFall] == "Y" or
51 | course[CourseAttribute.offeredSpring] == "Y")):
52 | continue
53 | # if no respondents, ignore
54 | if term_data[EvaluationConstants.responded_raters] == 0:
55 | continue
56 |
57 | for key in KEYS_TO_AVERAGE:
58 | if key not in term_data:
59 | continue
60 | value = term_data[key]
61 | # Get which academic year this is, so that we can weight
62 | # appropriately in the average
63 | year = int(term_data[EvaluationConstants.term][:-2])
64 | averaging_data.setdefault(key, []).append((value, year))
65 |
66 | for eval_key, course_key in KEYS_TO_AVERAGE.items():
67 | if eval_key not in averaging_data: continue
68 | values = [value for value, year in averaging_data[eval_key]]
69 | max_year = max(year for value, year in averaging_data[eval_key])
70 | weights = [0.5 ** (max_year - year)
71 | for value, year in averaging_data[eval_key]]
72 | total = sum(v * w for v, w in zip(values, weights))
73 | average = total / sum(weights)
74 | course.loc[course_key] = "{:.2f}".format(average)
75 |
--------------------------------------------------------------------------------
/readme.md:
--------------------------------------------------------------------------------
1 | # FireRoad Server
2 |
3 | FireRoad is an iOS/Android application that allows MIT students to plan their course selections, view up-to-date major/minor requirements, and discover new courses. The FireRoad Server is a Django server that provides a data backend and document "cloud" for the native apps as well as the web-based CourseRoad application.
4 |
5 | ## Cloning and Setup
6 |
7 | Follow these instructions to set up and run your own instance of the FireRoad server. You may want to create a new virtual environment using `conda`, for example:
8 |
9 | ```
10 | conda create -n fireroad python=2.7
11 | conda activate fireroad
12 | ```
13 |
14 | Then, enter the repo directory and run the setup script, which will install any necessary packages and set up the database.
15 |
16 | ```
17 | cd fireroad-server
18 | ./setup.sh
19 | ```
20 |
21 | To set up a catalog (including courses and requirements lists):
22 |
23 | ```
24 | ./setup_catalog.sh
25 | ```
26 |
27 | This script will prompt you to download a copy of the course catalog from the [prod site](https://fireroad.mit.edu/courseupdater/download_data) if you have not already. Otherwise, you can run the catalog setup script without pre-initialized catalogs, then run the scraper yourself following the instructions in `data/readme.md`.
28 |
29 | ## Running the Server
30 |
31 | To run the server (with the conda environment activated), use
32 |
33 | ```
34 | python manage.py runserver
35 | ```
36 |
37 | By default, the server runs on port 8000. You can specify a different port by simply adding the port number after the `runserver` command. (The `manage.py` script is [provided by Django](https://docs.djangoproject.com/en/1.11/ref/django-admin/) and provides other useful commands such as migrating the database, collecting static files, and opening an interactive shell.)
38 |
39 | ## Database Settings
40 |
41 | Note that the project contains three Django settings modules: `fireroad/settings.py` (local development), `fireroad/settings_dev.py` (dev server), and `fireroad_settings_prod.py` (prod server). When making changes to the settings, please make sure to change the file appropriate to the environment on which you want the changes to take effect (and note that the latter two import the base `settings.py` file). In order to specify which settings module should be used, you will need to set the `DJANGO_SETTINGS_MODULE` environment variable to `fireroad.settings{VARIANT}`, and change the default value specified in `fireroad/wsgi.py` if deploying with WSGI.
42 |
43 | Depending on your settings, there may be additional files that you can add to enable certain capabilities:
44 |
45 | * To use a MySQL database, add a `fireroad/dbcreds.py` file that specifies the necessary authentication info as Python variables `dbname`, `username`, `password`, and `host`.
46 | * To enable sending emails to admins for unresolved edit requests, etc., create an email address with two-factor authentication disabled (gmail works well). Then add a `fireroad/email_creds.py` file that specifies authentication info as a comma-delimited string with three components: the email server (e.g. `smtp.gmail.com`), the email address, and the password for the email account.
47 |
48 | ## API Endpoints
49 |
50 | The FireRoad API is fully documented at [fireroad.mit.edu/reference](https://fireroad.mit.edu/reference) (dev version at [fireroad-dev.mit.edu/reference](https://fireroad-dev.mit.edu/reference)). When submitting PRs that modify the behavior of these endpoints or add new ones, please update the docs in `common/templates/docs` accordingly.
51 |
--------------------------------------------------------------------------------
/requirements/templates/requirements/base.html:
--------------------------------------------------------------------------------
1 | {% extends "common/base.html" %}
2 | {% load static %}
3 | {% block title %}
4 | FireRoad Requirements Editor
5 | {% endblock %}
6 |
7 | {% block pagebody %}
8 |
Be sure all edit requests are correctly formatted, and add a descriptive change summary before deploying.
67 | {% if conflicts > 0 %}
68 | You have committed {{ conflicts }} edit request{% if conflicts != 1 %}s{% endif %} that will overwrite a previous pending deployment.
69 | {% endif %}
70 |
{% if action == "Create" %}To submit a new requirements list, please enter the requirements in the box above{% else %}To submit a modification request, please identify and/or correct the appropriate location in the above requirements definition{% endif %} (following the rules described in the Format Specification). Then provide your email address (in case of any clarifications) and a brief message explaining the change, and press Submit.