├── README.md ├── additional_features.md ├── blog_post_html.txt ├── twitter_search.py └── user-post-timeline ├── README.md ├── tweets-spuryyc.json └── twitter-user-post-timeline.py /README.md: -------------------------------------------------------------------------------- 1 | # `twitter_search.py` 2 | 3 | [My blog post](https://galeascience.wordpress.com/2016/03/18/collecting-twitter-data-with-python/) gives a thorough explanation of how to set up the code (including the dependancies), what it's doing, and an example how to use the resulting tweet data. I've also included the basics below. 4 | 5 | This program will search for tweets and save them to a .JSON formatted file. Twitter limits the amount of tweets that can be downloaded per 15 minutes; as such, when an exception is raised (i.e., the maximum allowed number of tweets has been reached) the script will pause for 15 minutes and then continue. 6 | 7 | A [user-contributed resource is available here](https://github.com/agalea91/twitter_search/blob/master/additional_features.md) with additional information on reading and extracting information from the .JSON file using Python. 8 | 9 | ### Dependencies 10 | 11 | To access the Twitter API I used Tweepy 3.5.0. I also imported the libaraies: json, datetime, time, os and sys. 12 | 13 | In addition, you'll need a personal [Twitter "data-mining" application](https://apps.twitter.com/) (which is very easy to set up). I used [this guide](http://marcobonzanini.com/2015/03/02/mining-twitter-data-with-python-part-1/#Register_Your_App) to register my app. You will need to register your own in order to generate a consumer key, consumer secret, access token, and access secret; these are required to authenticate the script in order to access the Twitter API. 14 | 15 | 16 | ### Running the script 17 | 18 | 1) Save the python file or download/clone the repository to your local machine. Make sure you have the dependencies. 19 | 20 | 2) Open the `twitter_search.py` file and then find the `load_api()` function (at the top) and add your consumer key, consumer secret, access token, and access secret. For example: 21 | ``` 22 | consumer_key = '189YcjF4IUzF156RGNGNucDD8' 23 | consumer_secret = 'd7HY36s4pSh03HxjDg782HupUjmzdOOSDd98hd' 24 | access_token = '2543812-cpaIuwndjvbdjaDDp5izzndhsD7figa9gb' 25 | access_secret = '4hdyfnas7d988ddjf87sJdj3Dxn4d5CcNpwe' 26 | ``` 27 | 28 | 3) Go to the `main()` function and edit the search criteria. Namely, you should enter a search phrase, the maximum time limit for the script to run, and the date range for the search (relative to today). For example: 29 | ``` 30 | search_phrase = '#makedonalddrumpfagain' 31 | time_limit = 1.0 # runtime limit in hours 32 | min_days_old, max_days_old = 1, 2 # search limits 33 | 34 | # e.g. min_days_old, max_days_old = 7, 8 35 | # gives the current weekday from last week, 36 | # min_days_old=0 will search from right now 37 | ``` 38 | 39 | 4) Open the terminal/command line to the file location and type: 40 | ``` 41 | python twitter_search.py 42 | ``` 43 | The script will run until all tweets within your search criteria have been found. 44 | 45 | 46 | #### Copyright (c) 2016 Alexander Galea 47 | Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: 48 | 49 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 50 | -------------------------------------------------------------------------------- /additional_features.md: -------------------------------------------------------------------------------- 1 | Following the code samples from the agalea91 blog post might lead you to believe there is only a few features we can extract from the tweets. Fortunately there is actually ~60 features recorded per tweet we can use! 2 | 3 | Once you import the .json files using the code snippet by agelea91. You extract a single tweet and view all the feature stored regarding that tweet. 4 | 5 | tweet_files = ['something.json'] 6 | tweets = [] 7 | for file in tweet_files: 8 | with open(file, 'r') as f: 9 | for line in f.readlines(): 10 | tweets.append(json.loads(line)) 11 | 12 | #View features for the second tweet 13 | tweets[2] 14 | 15 | You can then build more complex and customized dataframes with the information you need. 16 | 17 | Here's an example.. 18 | 19 | df = pd.DataFrame() 20 | def populate_tweet_df(tweets): 21 | df = pd.DataFrame() 22 | 23 | df['text'] = list(map(lambda tweet: tweet['text'], tweets)) 24 | 25 | #df['possibly_sensitive'] = list(map(lambda tweet: tweet['possibly_sensitive'], tweets)) 26 | 27 | df['retweet_count'] = list(map(lambda tweet: tweet['retweet_count'], tweets)) 28 | 29 | df['favorite_count'] = list(map(lambda tweet: tweet['favorite_count'], tweets)) 30 | 31 | df['retweeted'] = list(map(lambda tweet: tweet['retweeted'], tweets)) 32 | 33 | df['favorite_count'] = list(map(lambda tweet: tweet['favorite_count'], tweets)) 34 | 35 | df['followers_count'] = list(map(lambda tweet: tweet['user']['followers_count'], tweets)) 36 | 37 | df['screen_name']= list(map(lambda tweet: tweet['user']['screen_name'], tweets)) 38 | 39 | df['verified']= list(map(lambda tweet: tweet['user']['verified'], tweets)) 40 | if tweet['coordinates'] != None else 'NaN', tweets)) 41 | 42 | return df 43 | If your trying to extract a child feature, such as 'followers_count' , you need to first provide the parent feature 'user' 44 | -------------------------------------------------------------------------------- /blog_post_html.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/zazencodes/twitter_search/7293c5da3e21ad2e958255ea91f2397f4eff67ca/blog_post_html.txt -------------------------------------------------------------------------------- /twitter_search.py: -------------------------------------------------------------------------------- 1 | import tweepy 2 | from tweepy import OAuthHandler 3 | import json 4 | import datetime as dt 5 | import time 6 | import os 7 | import sys 8 | 9 | 10 | ''' 11 | In order to use this script you should register a data-mining application 12 | with Twitter. Good instructions for doing so can be found here: 13 | http://marcobonzanini.com/2015/03/02/mining-twitter-data-with-python-part-1/ 14 | 15 | After doing this you can copy and paste your unique consumer key, 16 | consumer secret, access token, and access secret into the load_api() 17 | function below. 18 | 19 | The main() function can be run by executing the command: 20 | python twitter_search.py 21 | 22 | I used Python 3 and tweepy version 3.5.0. You will also need the other 23 | packages imported above. 24 | ''' 25 | 26 | def load_api(): 27 | ''' Function that loads the twitter API after authorizing the user. ''' 28 | 29 | consumer_key = '' 30 | consumer_secret = '' 31 | access_token = '' 32 | access_secret = '' 33 | auth = OAuthHandler(consumer_key, consumer_secret) 34 | auth.set_access_token(access_token, access_secret) 35 | # load the twitter API via tweepy 36 | return tweepy.API(auth) 37 | 38 | 39 | def tweet_search(api, query, max_tweets, max_id, since_id, geocode): 40 | ''' Function that takes in a search string 'query', the maximum 41 | number of tweets 'max_tweets', and the minimum (i.e., starting) 42 | tweet id. It returns a list of tweepy.models.Status objects. ''' 43 | 44 | searched_tweets = [] 45 | while len(searched_tweets) < max_tweets: 46 | remaining_tweets = max_tweets - len(searched_tweets) 47 | try: 48 | new_tweets = api.search(q=query, count=remaining_tweets, 49 | since_id=str(since_id), 50 | max_id=str(max_id-1)) 51 | # geocode=geocode) 52 | print('found',len(new_tweets),'tweets') 53 | if not new_tweets: 54 | print('no tweets found') 55 | break 56 | searched_tweets.extend(new_tweets) 57 | max_id = new_tweets[-1].id 58 | except tweepy.TweepError: 59 | print('exception raised, waiting 15 minutes') 60 | print('(until:', dt.datetime.now()+dt.timedelta(minutes=15), ')') 61 | time.sleep(15*60) 62 | break # stop the loop 63 | return searched_tweets, max_id 64 | 65 | 66 | def get_tweet_id(api, date='', days_ago=9, query='a'): 67 | ''' Function that gets the ID of a tweet. This ID can then be 68 | used as a 'starting point' from which to search. The query is 69 | required and has been set to a commonly used word by default. 70 | The variable 'days_ago' has been initialized to the maximum 71 | amount we are able to search back in time (9).''' 72 | 73 | if date: 74 | # return an ID from the start of the given day 75 | td = date + dt.timedelta(days=1) 76 | tweet_date = '{0}-{1:0>2}-{2:0>2}'.format(td.year, td.month, td.day) 77 | tweet = api.search(q=query, count=1, until=tweet_date) 78 | else: 79 | # return an ID from __ days ago 80 | td = dt.datetime.now() - dt.timedelta(days=days_ago) 81 | tweet_date = '{0}-{1:0>2}-{2:0>2}'.format(td.year, td.month, td.day) 82 | # get list of up to 10 tweets 83 | tweet = api.search(q=query, count=10, until=tweet_date) 84 | print('search limit (start/stop):',tweet[0].created_at) 85 | # return the id of the first tweet in the list 86 | return tweet[0].id 87 | 88 | 89 | def write_tweets(tweets, filename): 90 | ''' Function that appends tweets to a file. ''' 91 | 92 | with open(filename, 'a') as f: 93 | for tweet in tweets: 94 | json.dump(tweet._json, f) 95 | f.write('\n') 96 | 97 | 98 | def main(): 99 | ''' This is a script that continuously searches for tweets 100 | that were created over a given number of days. The search 101 | dates and search phrase can be changed below. ''' 102 | 103 | 104 | 105 | ''' search variables: ''' 106 | search_phrases = ['Pavelski', 'Lucic', 107 | 'Ovechkin', 'Giroux', 108 | 'Jagr', 'John Tavares', 109 | 'Kucherov', 'Mrazek', 110 | 'Seguin', 'Pominville', 111 | 'Crosby', 'Lundqvist', 112 | 'Tarasenko', 'Patrick Kane', 113 | 'Cory Perry', 'Forsberg'] 114 | time_limit = 1.5 # runtime limit in hours 115 | max_tweets = 100 # number of tweets per search (will be 116 | # iterated over) - maximum is 100 117 | min_days_old, max_days_old = 6, 7 # search limits e.g., from 7 to 8 118 | # gives current weekday from last week, 119 | # min_days_old=0 will search from right now 120 | USA = '39.8,-95.583068847656,2500km' # this geocode includes nearly all American 121 | # states (and a large portion of Canada) 122 | 123 | 124 | # loop over search items, 125 | # creating a new file for each 126 | for search_phrase in search_phrases: 127 | 128 | print('Search phrase =', search_phrase) 129 | 130 | ''' other variables ''' 131 | name = search_phrase.split()[0] 132 | json_file_root = name + '/' + name 133 | os.makedirs(os.path.dirname(json_file_root), exist_ok=True) 134 | read_IDs = False 135 | 136 | # open a file in which to store the tweets 137 | if max_days_old - min_days_old == 1: 138 | d = dt.datetime.now() - dt.timedelta(days=min_days_old) 139 | day = '{0}-{1:0>2}-{2:0>2}'.format(d.year, d.month, d.day) 140 | else: 141 | d1 = dt.datetime.now() - dt.timedelta(days=max_days_old-1) 142 | d2 = dt.datetime.now() - dt.timedelta(days=min_days_old) 143 | day = '{0}-{1:0>2}-{2:0>2}_to_{3}-{4:0>2}-{5:0>2}'.format( 144 | d1.year, d1.month, d1.day, d2.year, d2.month, d2.day) 145 | json_file = json_file_root + '_' + day + '.json' 146 | if os.path.isfile(json_file): 147 | print('Appending tweets to file named: ',json_file) 148 | read_IDs = True 149 | 150 | # authorize and load the twitter API 151 | api = load_api() 152 | 153 | # set the 'starting point' ID for tweet collection 154 | if read_IDs: 155 | # open the json file and get the latest tweet ID 156 | with open(json_file, 'r') as f: 157 | lines = f.readlines() 158 | max_id = json.loads(lines[-1])['id'] 159 | print('Searching from the bottom ID in file') 160 | else: 161 | # get the ID of a tweet that is min_days_old 162 | if min_days_old == 0: 163 | max_id = -1 164 | else: 165 | max_id = get_tweet_id(api, days_ago=(min_days_old-1)) 166 | # set the smallest ID to search for 167 | since_id = get_tweet_id(api, days_ago=(max_days_old-1)) 168 | print('max id (starting point) =', max_id) 169 | print('since id (ending point) =', since_id) 170 | 171 | 172 | 173 | ''' tweet gathering loop ''' 174 | start = dt.datetime.now() 175 | end = start + dt.timedelta(hours=time_limit) 176 | count, exitcount = 0, 0 177 | while dt.datetime.now() < end: 178 | count += 1 179 | print('count =',count) 180 | # collect tweets and update max_id 181 | tweets, max_id = tweet_search(api, search_phrase, max_tweets, 182 | max_id=max_id, since_id=since_id, 183 | geocode=USA) 184 | # write tweets to file in JSON format 185 | if tweets: 186 | write_tweets(tweets, json_file) 187 | exitcount = 0 188 | else: 189 | exitcount += 1 190 | if exitcount == 3: 191 | if search_phrase == search_phrases[-1]: 192 | sys.exit('Maximum number of empty tweet strings reached - exiting') 193 | else: 194 | print('Maximum number of empty tweet strings reached - breaking') 195 | break 196 | 197 | 198 | if __name__ == "__main__": 199 | main() 200 | -------------------------------------------------------------------------------- /user-post-timeline/README.md: -------------------------------------------------------------------------------- 1 | ### Get the timeline of a users most recent posts 2 | 3 | In this example, the spuryyc tweets are collected in a .JSON file. 4 | 5 | -------------------------------------------------------------------------------- /user-post-timeline/tweets-spuryyc.json: -------------------------------------------------------------------------------- 1 | ["RT @wklumpen: Hi @nenshi, I'll be moderating the Green Line Speaker Series at the Glenbow Theatre, Sep 20 at 5:30, can you RT?\nhttps://t.co\u2026", "This week we look at different transit fare structures, and suggest a distance based scheme for #yyctransit https://t.co/TboAZl8hNg", "We're back with a new article! Looking at some reactions to recent news about commuting habits.\nhttps://t.co/LKWZifPpI5", "New article drops tomorrow!", "Hi folks, we're taking a summer break. We'll be back with more Calgary talk in September! #yyccc #yyc", "Some inspiration for #yyc from Shanghai of all places. Mobility = Freedom everywhere https://t.co/mmrqgDVaKf", "While #yyccc catches their breath for 30 minutes, sounds like some ppl could brush up on induced demand https://t.co/vvMiwZdMWf", "Digging into congestion costs https://t.co/Wa03pNW43I", "RT @YYCTRGM: Terrible driving all over the place this AM. Come on Calgary the rodeo is on the grounds not the streets.", "@GregGinYYC We await with bated breath, though there's plenty of time as it won't go up for a few weeks.", "@spuryyc @GregGinYYC Specifically on an article about how city council works, to illustrate a point on \"voting record\"", "@GregGinYYC do you mind if we use some of your charts from your website? Looking at these: https://t.co/B3UCgUmq6B", "All transportation systems need a \"critical network\" to become more than the sum of its parts. https://t.co/uMvxASN9qz", "RT @bpincott: @peterloliver @khancept31 That is the level of service provided in the SW & exactly why we need the #SWBRT #yyc4brt", "@HainesDanny Find that people generally relate to those types of distances in feet. Should have provided both, in retrospect.", "@Chealion Welcome!", "The cannon next door to item 8.16 #yyccc https://t.co/cJmqNvYcHd", "@calgarytransit Yes, exactly - high coverage. Always an interesting discussion: https://t.co/Wdajkeebcg", "#yyccc \"Transit close by\" applies pretty much everywhere in the city, since @calgarytransit focuses heavily on coverage.", "@nenshi and @BigRedyyc recognise the problem that \"databases that don't talk to each other\". This is a fixable issue! #yyccc", "#yyccc Coming up: agenda item 7.3 Streetview at 20\nMaple Court Crescent SE @nenshi https://t.co/kGHGH0r1lJ", "#yyccc Agenda Item 7.2 streetview: https://t.co/EGZBwRjPs8", "Got a post coming up that @gccarra might like called \"it takes a network\". Stay tuned! #yyccc @YYC4BRT", "We look at how planners default to the status-quo of \"cars first\", even when there is no reason to do so\nhttps://t.co/Um4hBSHyEV #yyc #yyccc", "@CouncillorKeats Thanks for clarifying your stance. Updated the article to reflect that.", "The simple but powerful concept of \"induced demand\". https://t.co/G0CrNW87ZH #yyc #yyccc", "RT @stephchd: @spuryyc nose hill at scenic acres Blvd. https://t.co/65u0FG6rAM", "@stephchd Yup!", "RT @cityofcalgary: It's the cycle track network pilot project 1yr anni & we've seen 770,393 #yycbike trips in just 3 locations! https://t.c\u2026", "\"A desire named streetcar\". Discussing streetcars in Calgary as city-shapers, not just people-movers. https://t.co/iH4PtjCzBx #yyctransit", "Fun fact: \"In Camera\" can be rearranged to spell \"American\" #InCameraBoredom #yyccc", "Public speakers pushing against elevated rail on #GreenLineYYC Totally agree. #yyccc", "Now @EWoolleyWard8 is making inadvertent rock puns. Gotta keep it lively! #yyccc", "@CBCScott Glad he's asking the important questions!", "Hopefully we don't have the same problem Bertha did in Seattle. #yyccc", "Lease or buy the tunnel machine? @seanchucalgary asking the important questions. #yyccc", "You can't out-design geometry. Elevated LRT to underground station transfer is simply more cumbersome. #GreenLineYYC #yyccc", "\"Do it right the first time\" was part of the public feedback on #GreenLineYYC. Probably true, giving favourable funding. #yyccc", "Tunnels are fascinating. Rock on, @peterdemong. #yyccc", "Under the river would be very very deep. Montreal metro has that issue going up the hill. Long escalator rides! #yyccc", "\"Compliments Calgary's downtown vibrancy\". How vibrant us our downtown really? #yyccc", "Watching these individual scores on specifics for #GreenLineYYC seems both arbitrary and robust at the same time. #yyccc", "Sorry @ABDanielleSmith, elevated LRT ranked lowest for community well-being #yyccc", "Learning about the process behind admin. recommendations on the #GreenLineYYC options thru downtown. Lots of aggregation! #yyccc", "Always fun when not everyone at a #yyccc meeting is wearing business attire.", "Focus of today's meeting is the LRT downtown Let's see what they say about the streetscape aspect of #GreenLineYYC #yyccc", "We're at City Hall today learning about #GreenLineYYC. Lots of information to come! #yyc #yyccc", "Looking for photo submissions of places in #yyc you think could be better with some simple improvements. Send to editor@spuryyc.org!", "New article on \"Re-imagining light rail\" #yyctransit\n\nhttps://t.co/rEABoXIZFn", "@BikeBikeYYC Uh huh, and look how successful that turned out!", "Bonus article! \"The fallacy of vocal dissent on the #swbrt\nhttps://t.co/I9Pea37DIH #yyc #yyctransit", "Today's article looks at shapes, and #yyc's obsession with circular neighbourhoods #yyccc: https://t.co/1RXBisEt46", "Today, a quick look at the 61 Avenue improvements near Chinook: https://t.co/L9mTXVaBOW #yyc #yycwalk", "RT @ahayher: #yyccc should take a page from Seattle and create great public spaces for citizens to enjoy. #yyc https://t.co/txqqBFajLO", "RT @calgaryxyz: Inner-city fenced dog park in Beltline a first for Calgary. https://t.co/ELPemIHr5b https://t.co/2JukQwDCqg", "A solid look at Calgary's Downtown and the long-term dangers of its off-work vacuum. #yyc #iamdowntown https://t.co/sPi1E93EII", "Good morning #yyc. New article coming at 11 on Calgary's downtown: is it functional?", "@tombielecki Plenty of streets in YYC have FT access and are not so wide. NYC also didn't have an issue when they narrowed roads.", "With the #GreenLineYYC #Beltline open house tomorrow, here's some thoughts you might be interested in https://t.co/JErDUYDjHw", "Here's today's article. Speed traps pointing to possible over-engineered roads. #yyc #yycwalk #yyccc https://t.co/idX93gd3KL", "It's Tuesday! Gonna drop a fresh article at 11. Stay tuned!", "A little teaser from one of next week's articles. Any guesses? #yyccc #yyc #yycwalk https://t.co/yQCCifSEiL", "@robdickinsonAB That's part of the plan - not all Qs are answerable based on what's there but many are.", "@robdickinsonAB We're looking to compile a \"everything about the #SWBRT similar to this article on Hamilton's LRT https://t.co/kZp08nk0Zd", "@nenshi @crackmacs We're hoping to give solid answers to good questions about the #SWBRT and clear up some myths. Send them in! #yyccc", "\"Let's not suddenly forget what makes good cities great\" - A calm look at robot-cars, inspired by @gccarra https://t.co/2gYBJQCeZL", "@GlenFCochrane With new technology we should always ask \"Wait. Do we really need this?\" - this one isn't so clear.", "Take a big-picture look at Calgary with \"Cities Served Three Ways\" by Ryan Plestid #yyc #yycplan #yyccc https://t.co/q8udp0K4gB", "Happy holiday! This week we've got some great articles in the works, including robot-car talk inspired by @gccarra. Stay tuned! #yyc #yyccc", "In case you missed the discussion on the #GreenLineYYC in the Centre City, here it is again: https://t.co/cpk2tBKoUG #yyc #yyccc #yyctransit", "And today's regular content: Playing with the street!\n\nhttps://t.co/RiB0o9ji21", "Bonus content! What all this \"100% reserved park and ride\" really means. Spoiler: it's not that bad #yyccc https://t.co/d69FKnJa3O", "\"Why Calgarians don't love YYC transit\" by @JenBurgess1 https://t.co/LiW8dqTJWw #yyc #iloveyyctransit", "Great set of articles coming this week! We'll hear from @JenBurgess1 and see how we can play with the streets for the new #GreenLineYYC #yyc", "RT @LRTontheGreen: Another reminder why #greenlineyyc offering an alternative to Deerfoot Tr is so important to SE #yyc neighbourhoods. htt\u2026", "Got that sweet @nenshi RT! https://t.co/a5XjLe07FX", "@karinkeefe They're everywhere, just like people!", "We're now on Facebook! https://t.co/zmSrQWjD3q", "Hey #yyc #yyccc we'd love to see pictures of #desirelines all over Calgary. Tweet 'em at us! @nenshi @Crackmacs https://t.co/sdVBErTmrt", "@amanda_en_ciel @UCalgary if that's what the people want.... ;)", "New posts coming Tuesday/Thursday! This one's about Calgary's \"hidden desire lines\" #yyc #yyccc #yycwalk https://t.co/sdVBErTmrt", "We're gearing up to publish regular content! Be part of the conversation, and consider submitting something. #yyc #yyccc", "@BrianPansky @yycconnects we do now! Sounds very interesting indeed!", "RT @ABDanielleSmith: Intv w/ @wklumpen @UCalgary SchulickENGG on Green Line option https://t.co/fpThsYGIDZ #yyccc #yyc", "Spur contributor @wklumpen is talking Green Line today with @ABDanielleSmith! 1pm on @NewsTalk770. Tune in! #yyctransit #yyccc", "Green Line: mobility tool, or commuter service?https://t.co/JErDUYDjHw", "With a big transit meeting tomorrow, here's some clarification on the SW BRT budget: https://t.co/9YbeOuhFda #yyc", "@PhilWilsn @nenshi Love it. We're trying to create well-reasoned dialogue, goes hand in hand with supporting initiatives.", "@nenshi Trying to create a space where Calgarians can talk about making Calgary a better city. Looking for submissions! RT? @spuryyc #yyc", "One of our articles is being discussed on @NewsTalk770 this afternoon with @ABDanielleSmith! Tune in at 2:30: https://t.co/DFj92zKuEB", "The follies of Park and Ride. https://t.co/DFj92zKuEB #yyc #yyctransit", "@JenBurgess1 I think you might be interested in contributing - check us out at https://t.co/Ov1LpH0bMo", "Convenience barriers and \"social profit margins\" by Ryan Plestid:\nhttps://t.co/5dQcFO1BhS", "CalgaryNEXT misses the mark: https://t.co/INZ9dE7aXP #yyc #Flames #CalgaryNext", "@BrianPansky Great minds think alike! Look forward to your submissions!", "A take on imrpoving existing Beltline parks instead of adding new ones https://t.co/9FMkKyoiKi #yyc #yycplan #beltline", "Spuryyc article being discussed this afternoon! #yyc #SWBRT https://t.co/FACRhzFBg5", "Article on the New Urban Farming Partnerships: Great initiative, check them out! #yyc #yycfood https://t.co/h7U9NUnGKJ", "Social exclusion and \"diversity annihilation\" in East Village: https://t.co/TaAtDW5KfI", "Does Calgary suffer from \"quadrant favouritism\"? @CRiabko discusses in new article: https://t.co/AUVJ0AHklY #yyc #editorial", "RT @metrocalgary: From critical, to critiquing - a more nuanced look at the SW BRT plan https://t.co/ooJZutjoN6 #yyc https://t.co/vu12GmjHy8", "RT @modacitylife: \u201cIt\u2019s not about making all cities like Amsterdam. \nIt\u2019s about making them better versions of themselves.\u201d https://t.co/Cs\u2026", "Calgary Transitway BRT article featured in @metrocalgary!", "One way bad, two way good? New article on Calgary's one way streets:\nhttps://t.co/tkz6EWBAtR #yyc", "The website has launched! Please consider contributing at https://t.co/BBihkYqElT #yyc"] -------------------------------------------------------------------------------- /user-post-timeline/twitter-user-post-timeline.py: -------------------------------------------------------------------------------- 1 | import tweepy 2 | from tweepy import OAuthHandler 3 | import json 4 | 5 | def load_api(): 6 | ''' Function that loads the twitter API after authorizing the user. ''' 7 | 8 | consumer_key = '' 9 | consumer_secret = '' 10 | access_token = '' 11 | access_secret = '' 12 | auth = OAuthHandler(consumer_key, consumer_secret) 13 | auth.set_access_token(access_token, access_secret) 14 | # load the twitter API via tweepy 15 | return tweepy.API(auth) 16 | 17 | api = load_api() 18 | user = 'spuryyc' 19 | tweets = api.user_timeline(id=user, count=1000) 20 | print('Found %d tweets' % len(tweets)) 21 | 22 | # You now have a list of tweet objects with various attributes 23 | # check out the structure here: http://tkang.blogspot.ca/2011/01/tweepy-twitter-api-status-object.html 24 | 25 | # For example we can get the text of each tweet 26 | tweets_text = [t.text for t in tweets] 27 | filename = 'tweets-'+user+'.json' 28 | json.dump(tweets_text, open(filename, 'w')) 29 | print('Saved to file:', filename) 30 | 31 | # Can load file like this 32 | #tweets_text = json.load(open(filename, 'r')) 33 | --------------------------------------------------------------------------------