├── src └── craigslist_scraper.py └── README.md /src/craigslist_scraper.py: -------------------------------------------------------------------------------- 1 | import requests 2 | from pprint import pprint 3 | 4 | # Structure payload. 5 | payload = { 6 | 'source': 'universal', 7 | 'url': 'https://berlin.craigslist.org/search/ela#search=1~gallery~0~0', 8 | 'geo_location': 'Germany', 9 | 'render': 'html' 10 | } 11 | 12 | # Get a response. 13 | response = requests.request( 14 | 'POST', 15 | 'https://realtime.oxylabs.io/v1/queries', 16 | auth=('USERNAME', 'PASSWORD'), #Your credentials go here 17 | json=payload 18 | ) 19 | 20 | # Instead of response with job status and results URL, this will return the 21 | # JSON response with results. 22 | pprint(response.json()) 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Craigslist Scraper 2 | [![Oxylabs promo code](https://raw.githubusercontent.com/oxylabs/product-integrations/refs/heads/master/Affiliate-Universal-1090x275.png)](https://oxylabs.io/pages/gitoxy?utm_source=877&utm_medium=affiliate&groupid=877&utm_content=craigslist-scraper-github&transaction_id=102f49063ab94276ae8f116d224b67) 3 | 4 | [![](https://dcbadge.vercel.app/api/server/eWsVUJrnG5)](https://discord.com/invite/Pds3gBmKMH) 5 | 6 | [Craigslist 7 | Scraper](https://oxylabs.io/products/scraper-api/web/craigslist) is 8 | a scraping tool that overcomes advanced anti-bot systems and helps you 9 | gather public data from Craigslist on any scale you need. This guide 10 | will show you how to scrape Craigslist using Oxylabs’ [Scraper 11 | API](https://oxylabs.io/products/scraper-api). 12 | 13 | ## How it works 14 | 15 | You can get Craigslist data by sending a request to our API with the 16 | URLs you want to access and scrape. The API will return the HTML of any 17 | public Craigslist page. 18 | 19 | ### Python code example 20 | 21 | The below code sample sends a request to our service, which uses a 22 | headless browser to execute JavaScript and sends back the HTML of a 23 | Craigslist page: 24 | 25 | ```python 26 | import requests 27 | from pprint import pprint 28 | 29 | # Structure payload. 30 | payload = { 31 | 'source': 'universal', 32 | 'url': 'https://berlin.craigslist.org/search/ela#search=1~gallery~0~0', 33 | 'geo_location': 'Germany', 34 | 'render': 'html' 35 | } 36 | 37 | # Get a response. 38 | response = requests.request( 39 | 'POST', 40 | 'https://realtime.oxylabs.io/v1/queries', 41 | auth=('USERNAME', 'PASSWORD'), #Your credentials go here 42 | json=payload 43 | ) 44 | 45 | # Instead of response with job status and results URL, this will return the 46 | # JSON response with results. 47 | pprint(response.json()) 48 | ``` 49 | 50 | Visit the 51 | [documentation](https://developers.oxylabs.io/scraper-apis/web-scraper-api) 52 | to find more payload parameters and other details. 53 | 54 | ### Output sample 55 | 56 | ```json 57 | { 58 | "results": [ 59 | { 60 | "content": "\n\n 61 | ... 62 | \n\n", 63 | "created_at": "2023-09-21 14:26:52", 64 | "updated_at": "2023-09-21 14:27:10", 65 | "page": 1, 66 | "url": "https://berlin.craigslist.org/search/ela#search=1~gallery~0~0", 67 | "job_id": "7110630468831163393", 68 | "status_code": 200 69 | } 70 | ] 71 | } 72 | ``` 73 | 74 | Oxylabs Craigslist Scraper API will ease your scraping processes 75 | significantly. Use it to gather public data, such as jobs, items, 76 | services, and ads. If you have any questions, feel free to get in touch 77 | with us via [live chat](https://oxylabs.io/) or 78 | [email](mailto:support@oxylabs.io). 79 | 80 | Also, check this tutorial on [pypi](https://pypi.org/project/craiglist-scraper-api/) 81 | --------------------------------------------------------------------------------