├── README.md
├── main.py
└── requirements.txt
/README.md:
--------------------------------------------------------------------------------
1 | # Oxylabs’ Residential Proxies integration with Selenium in Python
2 |
3 | [](https://oxylabs.go2cloud.org/aff_c?offer_id=7&aff_id=877&url_id=112)
4 |
5 | [](https://discord.gg/GbxmdGhZjq)
6 |
7 | [
](https://github.com/topics/python) [
](https://github.com/topics/selenium) [
](https://github.com/topics/web-scraping) [
](https://github.com/topics/rotating-proxies)
8 |
9 | ## Requirements
10 |
11 | For the integration to work, you'll need to install
12 | [Selenium Wire](https://github.com/wkeeling/selenium-wire)
13 | to extend Selenium’s Python bindings as implementing proxies
14 | that require authentication using default Selenium module
15 | complicates the process too much.
16 |
17 | You can do it using `pip` command:
18 | ```bash
19 | pip install selenium-wire
20 | ```
21 |
22 | Another recommended package is `webdriver-manager`. It simplifies the management
23 | of binary drivers for different browsers, so you don't need to manually download
24 | a new version of a web driver after each update. Visit the
25 | [official project directory](https://pypi.org/project/webdriver-manager/) on pypi to
26 | find out more information.
27 |
28 | You can install the following using `pip` as well:
29 | ```bash
30 | pip install webdriver-manager
31 | ```
32 |
33 | Required version of Python: `Python 3.5` (or higher)
34 |
35 | ## Proxy Authentication
36 |
37 | For proxies to work, you'll need to specify your account credentials inside
38 | the [main.py](https://github.com/oxylabs/selenium-proxy-integration/blob/main/main.py) file.
39 |
40 | ```python
41 | USERNAME = "your_username"
42 | PASSWORD = "your_password"
43 | ENDPOINT = "pr.oxylabs.io:7777"
44 | ```
45 |
46 | Adjust the `your_username` and `your_password` value fields with the username and password of
47 | your Oxylabs account.
48 |
49 | ## Testing Proxy Connection
50 |
51 | To see if the proxy is working, try visiting [ip.oxylabs.io/location](https://ip.oxylabs.io/location)
52 | If everything is working correctly, it will return an IP address of a proxy that you're using.
53 |
54 | ## Full Code
55 | ```python
56 | from selenium.webdriver.common.by import By
57 | from seleniumwire import webdriver
58 | # A package to have a chromedriver always up-to-date.
59 | from webdriver_manager.chrome import ChromeDriverManager
60 |
61 | USERNAME = "your_username"
62 | PASSWORD = "your_password"
63 | ENDPOINT = "pr.oxylabs.io:7777"
64 |
65 |
66 | def get_chrome_proxy(user: str, password: str, endpoint: str) -> dict:
67 | wire_options = {
68 | "proxy": {
69 | "http": f"http://{user}:{password}@{endpoint}",
70 | "https": f"http://{user}:{password}@{endpoint}",
71 | }
72 | }
73 |
74 | return wire_options
75 |
76 |
77 | def execute_driver():
78 | options = webdriver.ChromeOptions()
79 | options.headless = True
80 | seleniumwire_options = {
81 | **get_chrome_proxy(USERNAME, PASSWORD, ENDPOINT),
82 | "driver_path": ChromeDriverManager().install(),
83 | }
84 | driver = webdriver.Chrome(
85 | options=options,
86 | seleniumwire_options=seleniumwire_options,
87 | )
88 | try:
89 | driver.get("https://ip.oxylabs.io/location")
90 | return f'\nYour IP is: {driver.find_element(By.CSS_SELECTOR, "pre").text}'
91 | finally:
92 | driver.quit()
93 |
94 |
95 | if __name__ == "__main__":
96 | print(execute_driver())
97 | ```
98 |
99 | If you're having any trouble integrating proxies with Selenium and this guide didn't help
100 | you - feel free to contact Oxylabs customer support at support@oxylabs.io.
101 |
--------------------------------------------------------------------------------
/main.py:
--------------------------------------------------------------------------------
1 | from selenium.webdriver.common.by import By
2 | from seleniumwire import webdriver
3 | # A package to have a chromedriver always up-to-date.
4 | from webdriver_manager.chrome import ChromeDriverManager
5 |
6 | USERNAME = "your_username"
7 | PASSWORD = "your_password"
8 | ENDPOINT = "pr.oxylabs.io:7777"
9 |
10 |
11 | def get_chrome_proxy(user: str, password: str, endpoint: str) -> dict:
12 | wire_options = {
13 | "proxy": {
14 | "http": f"http://{user}:{password}@{endpoint}",
15 | "https": f"http://{user}:{password}@{endpoint}",
16 | }
17 | }
18 |
19 | return wire_options
20 |
21 |
22 | def execute_driver():
23 | options = webdriver.ChromeOptions()
24 | options.headless = True
25 | seleniumwire_options = {
26 | **get_chrome_proxy(USERNAME, PASSWORD, ENDPOINT),
27 | "driver_path": ChromeDriverManager().install(),
28 | }
29 | driver = webdriver.Chrome(
30 | options=options,
31 | seleniumwire_options=seleniumwire_options,
32 | )
33 | try:
34 | driver.get("https://ip.oxylabs.io/location")
35 | return f'\nYour IP is: {driver.find_element(By.CSS_SELECTOR, "pre").text}'
36 | finally:
37 | driver.quit()
38 |
39 |
40 | if __name__ == "__main__":
41 | print(execute_driver())
42 |
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | webdriver-manager>=4.0.1
2 | selenium-wire>=5.1.0
3 |
--------------------------------------------------------------------------------