├── .gitignore ├── LICENSE ├── pom.xml ├── README.md └── src └── main └── java └── MainService.java /.gitignore: -------------------------------------------------------------------------------- 1 | # ---> Maven 2 | target/ 3 | pom.xml.tag 4 | pom.xml.releaseBackup 5 | pom.xml.versionsBackup 6 | pom.xml.next 7 | release.properties 8 | dependency-reduced-pom.xml 9 | buildNumber.properties 10 | .mvn/timing.properties 11 | # https://github.com/takari/maven-wrapper#usage-without-binary-jar 12 | .mvn/wrapper/maven-wrapper.jar 13 | 14 | # idea related 15 | .idea/ 16 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Allen Hua 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /pom.xml: -------------------------------------------------------------------------------- 1 | 2 | 5 | 4.0.0 6 | 7 | com.hellodk 8 | link-cleaner 9 | 1.1.1 10 | 11 | 12 | 11 13 | 11 14 | UTF-8 15 | 16 | 17 | 18 | 19 | cn.hutool 20 | hutool-http 21 | 5.7.10 22 | 23 | 24 | 25 | 26 | 27 | 28 | org.apache.maven.plugins 29 | maven-compiler-plugin 30 | 2.3.2 31 | 32 | 11 33 | 11 34 | 35 | 36 | 37 | org.apache.maven.plugins 38 | maven-assembly-plugin 39 | 3.2.0 40 | 41 | 42 | 43 | 44 | MainService 45 | 46 | true 47 | 48 | 49 | 50 | jar-with-dependencies 51 | 52 | 53 | 54 | 55 | make-assembly 56 | package 57 | 58 | single 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # link-cleaner 2 | 3 | link cleaner, removing the track params of urls. 4 | 5 | ![downloads](https://img.shields.io/github/downloads/hellodk34/link-cleaner/total) ![language](https://img.shields.io/badge/language-Java-green) ![MIT](https://img.shields.io/github/license/hellodk34/link-cleaner) 6 | 7 | **互联网上为了保护隐私我们能做些什么?看看我的想法和我带来的 link-cleaner** 8 | 9 | 前两天 v2ex 有个帖子 [网上留联系方式的隐私保护尝试](https://www.v2ex.com/t/874281) , 引起了我的思考。确实现在的大环境就是在收集隐私,前些年各大 App 还很流行年末搞出各种总结,近些年似乎有些收敛。这就是互联网企业的产品在利用各种手段搜集、统计我们日常使用手机的方方面面的行为,最明显的就是 A 用户分享,B 用户点击了 A 用户分享的内容,然后由此产生了一些微妙的联系,实际上最终这个联系非常的庞大,足以支撑起各大 App 产出各种各样的年度报告。 10 | 11 | 如今互联网如此渗透进我们的生活了,为了保护我们的隐私,我们能做些什么?我先抛砖引玉说几点 12 | 13 | 1. 注册非大陆地区的 手机号,在敏感网站上注册使用 14 | 2. 尽量使用邮箱注册各个网站,虽然大陆地区这样的网站已经几乎绝迹了 15 | 3. 尽量减少使用国内 App,包括桌面端和移动端的程序,桌面端的有些程序不得不用时可以塞进虚拟机打开,比如之前的 WPS 事件,完全可以跟 V 友开车上 Office 365 16 | 4. 下载软件只在官网、GitHub 等平台寻找下载链接,国内的下崽器就千万别碰了 17 | 5. 手机号千万千万保护好,不轻易泄露 18 | 6. 常用软件的设置,多看看,以 **保护自己的隐私** 作为原则设置各个选项。其实这里讲了很多点,如果你嫌麻烦或者不在乎隐私也就无所谓了 19 | 7. 多多使用 Google 和 Bing,baidu 只作为一些场景下的候选,比如 `site:tieba.baidu.com xxxx` 搜你的关键词在贴吧中大家的讨论,肯定能搜出来很多 20 | 8. **这一点是我很想提的**,**尽量减少使用应用内的分享功能!**,因为他们基本上会带有相应的 track params,也就是追踪参数。平台给你生成了带了个尾巴的链接,你分享了,别人点进去一定会有更多数据生成,追踪了分享人和点击人的行为 21 | 9. 上面那个帖子有 V 友提到,在网上留联系方式时,在 V 站大家习惯 base64,微信号直接 base64 出来的字符串是固定的,很容易被搜索引擎爬取到,可能会被人搜到你在其他地方的活动。但是加上一些其他字符串,一起 encode 结果就会变化,这真是一个非常 nice,非常省时省力省心的好方法。比如 22 | ``` 23 | root@debian11:~# echo "my weixin id is: 12345678, shared on v2ex.com at 2022-08-25" |base64 24 | bXkgd2VpeGluIGlkIGlzOiAxMjM0NTY3OCwgc2hhcmVkIG9uIHYyZXguY29tIGF0IDIwMjItMDgt 25 | MjUK 26 | root@debian11:~# echo "bXkgd2VpeGluIGlkIGlzOiAxMjM0NTY3OCwgc2hhcmVkIG9uIHYyZXguY29tIGF0IDIwMjItMDgt 27 | MjUK" |base64 -d 28 | my weixin id is: 12345678, shared on v2ex.com at 2022-08-25 29 | ``` 30 | 31 | 我已经养成了习惯,给别人分享东西时,基本上是分享标题 + 链接,如果是链接要看它干不干净。 32 | 33 | 为此我用 Java 写了一个简单的应用,打包成了 jar,要使用的话安装 [OpenJDK11](https://www.injdk.cn/) 。 34 | 35 | ## 用法 36 | 37 | 复制从平台复制的链接,执行 **java -jar link-cleaner-1.0.0.jar**,**新的链接将会自动写入剪贴板,再粘贴即可**。我处理了 pdd、taobao、jd、bilibili 四个平台的链接。下面是一些例子。 38 | 39 | pdd,别人通过应用内分享给你的商品,你点击后会直接给你展示是谁分享的。如果是你分享的链接,被很多人通过微信转发了,会不会社死?所以隐私很重要。 40 | 41 | ![拼多多极简白键帽分享带链接追踪功能-处理压缩后](https://image.940304.xyz/i/2022/08/25/630776655ec97.jpg) 42 | 43 | ## pdd 44 | 45 | ``` 46 | $ java -jar link-cleaner-1.0.0.jar 47 | your original clipboard text is: https://mobile.yangkeduo.com/goods.html?_x_org=2&refer_share_uin=KSFDGHKGUQXWTDO6DFD4JEM4II_GEXDA&_x_query=%E6%BA%90%E5%B7%A5%E4%B8%9A%E9%94%AE%E5%B8%BD&share_uin=KSFDGHKGUQXWTDO6DFD4JEM4II_GEXDA&page_from=23&_wv=41729&refer_share_channel=copy_link&refer_share_id=LJ8vv4WYAvYeEDzl5BqMr2rbJCLWiX9u&goods_id=28707795xxxx&pxq_secret_key=VGKR5MJG66CF7YSADPCIDC4OAA3ZUKSORF6SJAVHT3JXSROMIAGA&_wvx=10 48 | your new clipboard text is: https://mobile.yangkeduo.com/goods.html?goods_id=28707795xxxx 49 | ``` 50 | 51 | ## taobao 52 | 53 | ``` 54 | $ java -jar link-cleaner-1.0.0.jar 55 | your original clipboard text is: 【淘宝】https://m.tb.cn/h.fAwWH69?tk=XTP02u5xxxx CZ3457 「LuatOS墨水屏开发板(每个ID限购1件,,多拍不发)」 56 | 点击链接直接打开 57 | your new clipboard text is: CZ3457 「LuatOS墨水屏开发板(每个ID限购1件,,多拍不发)」 58 | 点击链接直接打开 https://item.taobao.com/item.htm?id=67803651xxxx 59 | ``` 60 | 61 | ## jd 62 | 63 | ``` 64 | $ java -jar link-cleaner-1.0.0.jar 65 | your original clipboard text is: https://item.m.jd.com/product/10001058xxxx.html?&utm_source=iosapp&utm_medium=appshare&utm_campaign=t_335139774&utm_term=CopyURL&ad_od=share&gx=RnE3kDMMOmWKmtRN6tUjCHNhknHA 66 | your new clipboard text is: https://item.m.jd.com/product/10001058xxxx.html 67 | ``` 68 | 69 | ## bilibili-1 70 | 71 | ``` 72 | $ java -jar link-cleaner-1.0.0.jar 73 | your original clipboard text is: 【让你的耳朵怀孕!《A Moment Apart》极限竞速:地平线4配乐-哔哩哔哩】 https://b23.tv/GUOxxxx 74 | your new clipboard text is: 【让你的耳朵怀孕!《A Moment Apart》极限竞速:地平线4配乐-哔哩哔哩】 https://www.bilibili.com/video/BV1M4411q7W8 75 | ``` 76 | 77 | ## bilibili-2 78 | 79 | ``` 80 | $ java -jar link-cleaner-1.0.0.jar 81 | your original clipboard text is: 【【何同学】我做了一个自己打字的键盘...】 https://www.bilibili.com/video/BV1W14y1b7Mq?share_source=copy_web&vd_source=f3e330de995a48b819604c85bc0dxxxx 82 | your new clipboard text is: 【【何同学】我做了一个自己打字的键盘...】 https://www.bilibili.com/video/BV1W14y1b7Mq 83 | ``` 84 | 85 | ## douyin 86 | 87 | ``` 88 | $ java -jar link-cleaner-v1.0.1.jar 89 | your original clipboard text is: 7.46 jcA:/ 复制打开抖音,看看【宽甸魔笛琴行的作品】中国足球 何日出头🤔# 足球的魅力远远超过你的想象... https://v.douyin.com/rHKxxxx/ 90 | your new clipboard text is: 看看【宽甸魔笛琴行的作品】中国足球 何日出头🤔# 足球的魅力远远超过你的想象... https://www.douyin.com/video/7168098046780214580 91 | ``` 92 | 93 | 源码在: https://github.com/hellodk34/link-cleaner 欢迎大家来添砖加瓦 94 | 95 | to be continued... 96 | 97 | ---- 98 | 99 | ## 更新日志 100 | 101 | ### v1.1.1 102 | 103 | 小红书 web 获取的链接不再是 `http://xhslink.com` 的链接了,而是直接 `https://xiaohongshu.com` 的链接。但是链接带有很多参数,分享到微信直接打开需要带有 `xsec_token` 值,否则只能在小红书 app 中才能浏览笔记内容。于是本次更新带上这个更新。 104 | 105 | ### v1.1.0 106 | 107 | - 适配 Linux Desktop,使本程序在linux桌面系统上也能工作。注意需要安装 xclip 软件包。 108 | 109 | ``` 110 | # deb 系 111 | $ sudo apt install -y xclip 112 | # rpm 系,下面两个命令一般二选一即可,老的 CentOS 可能还是 yum,较新的 Fedora 或者 Rocky Linux、OpenEuler 都可以直接使用 dnf 113 | $ sudo yum install -y xclip 114 | $ sudo dnf install -y xclip 115 | ``` 116 | 117 | ### v1.0.2 118 | 119 | - 2022-11-21 13:53:25, 1.0.1 版本的 jar 新增 site: douyin 120 | 121 | ### v1.0.1 122 | 123 | - 2023-12-21 10:20:01, 1.0.2 版本的 jar 新增 site: xiaohongshu 124 | -------------------------------------------------------------------------------- /src/main/java/MainService.java: -------------------------------------------------------------------------------- 1 | import cn.hutool.core.util.ObjectUtil; 2 | import cn.hutool.http.HttpRequest; 3 | import cn.hutool.http.HttpResponse; 4 | 5 | import java.awt.Toolkit; 6 | import java.awt.datatransfer.Clipboard; 7 | import java.awt.datatransfer.DataFlavor; 8 | import java.awt.datatransfer.StringSelection; 9 | import java.awt.datatransfer.Transferable; 10 | import java.awt.datatransfer.UnsupportedFlavorException; 11 | import java.io.IOException; 12 | import java.util.regex.Matcher; 13 | import java.util.regex.Pattern; 14 | 15 | /** 16 | * @author: hellodk 17 | * @description main service 18 | * @date: 2022/8/24 12:17 19 | */ 20 | 21 | public class MainService { 22 | 23 | private static final String XHS_DOMAIN = "xiaohongshu.com"; 24 | 25 | public static void main(String[] args) { 26 | MainService main = new MainService(); 27 | main.mainEntry(); 28 | } 29 | 30 | private String pddGoodsUri(String url) { 31 | String[] split = url.split("&"); 32 | String[] goods = url.split("\\?"); 33 | String param = ""; 34 | if (split.length == 1) { 35 | return split[0]; 36 | } 37 | for (int i = 0; i < split.length; i++) { 38 | String item = split[i]; 39 | if (item.startsWith("goods") && item.contains("goods_id=")) { 40 | param = item; 41 | break; 42 | } 43 | } 44 | return goods[0].concat("?").concat(param); 45 | } 46 | 47 | private String biliUri(String url) { 48 | String uri = url; 49 | int httpsIndex = uri.indexOf("https"); 50 | String text = uri.substring(0, httpsIndex); 51 | uri = uri.substring(httpsIndex); 52 | if (uri.contains("b23.tv")) { 53 | HttpResponse resp = HttpRequest.head(uri).timeout(20000).execute(); 54 | String location = resp.header("Location"); 55 | uri = location; 56 | } 57 | return (text.length() == 0 ? "" : (text + " ")) + removeParams(uri); 58 | } 59 | 60 | private String douyinUri(String url) { 61 | String uri = url; 62 | int startIndex = uri.indexOf("抖音,"); 63 | int httpsIndex = uri.indexOf("https"); 64 | String text = uri.substring(startIndex + 3, httpsIndex); 65 | uri = uri.substring(httpsIndex); 66 | HttpResponse resp = HttpRequest.head(uri).timeout(20000).execute(); 67 | String location = resp.header("Location"); 68 | String newLocation = removeParams(location); 69 | String regex = "\\/(\\d+)\\/"; 70 | Pattern p = Pattern.compile(regex); 71 | Matcher m = p.matcher(newLocation); 72 | if (m.find()) { 73 | String videoId = m.group(1); 74 | String newUriTemplate = "https://www.douyin.com/video/%s"; 75 | String result = String.format(newUriTemplate, videoId); 76 | return text + result; 77 | } 78 | return url; 79 | } 80 | 81 | private String xiaohongshuUri(String text) { 82 | if (text.contains(XHS_DOMAIN)) { 83 | String[] uriSplit = text.split("\\?"); 84 | if (uriSplit.length > 1) { 85 | final int length = uriSplit.length; 86 | StringBuilder firstPart = new StringBuilder(); 87 | for (int i = 0; i < (length - 1); i++) { 88 | firstPart.append(uriSplit[i]); 89 | } 90 | String secondPart = uriSplit[length - 1]; 91 | String[] params = secondPart.split("\\&"); 92 | for (String param : params) { 93 | if (param.contains("xsec_token")) { 94 | return firstPart.toString().concat("?").concat(param); 95 | } 96 | } 97 | } 98 | } 99 | String schemePrefix = ""; 100 | if (text.contains("http://")) { 101 | schemePrefix = "http://"; 102 | } 103 | else if (text.contains("https://")) { 104 | schemePrefix = "https://"; 105 | } 106 | else { 107 | System.out.println(text + "is not supported now."); 108 | } 109 | int httpIndex = text.lastIndexOf(schemePrefix); 110 | // 正则表达式匹配 URI 111 | String regex = "(?<=http://|https://)[\\w\\d+./?=]+"; 112 | Pattern pattern = Pattern.compile(regex); 113 | Matcher matcher = pattern.matcher(text); 114 | 115 | // 寻找匹配项并输出结果 116 | if (matcher.find()) { 117 | String uri = matcher.group(); 118 | HttpResponse resp = HttpRequest.head(schemePrefix + uri).timeout(20000).execute(); 119 | String location = resp.header("Location"); 120 | String cleanedUri = removeParams(location); 121 | return text.substring(0, httpIndex) + " " + cleanedUri; 122 | } 123 | return ""; 124 | } 125 | 126 | private String jdUri(String url) { 127 | return removeParams(url); 128 | } 129 | 130 | private String removeParams(String uri) { 131 | String[] split = uri.split("\\?"); 132 | return split[0]; 133 | } 134 | 135 | private String taobaoUri(String url) { 136 | String uri = url; 137 | if (uri.contains("taobao.com")) { 138 | return getTaobaocomUri(uri); 139 | } 140 | int start = uri.indexOf("https"); 141 | int end = uri.indexOf(" "); 142 | String goodName = uri.substring(end); 143 | String realUri = uri.substring(start, end); 144 | String result = HttpRequest.get(realUri).timeout(20000).execute().body(); 145 | String regex = "var url = '([^\\r\\n]*)';"; 146 | Pattern pattern = Pattern.compile(regex); 147 | Matcher matcher = pattern.matcher(result); 148 | if (matcher.find()) { 149 | uri = matcher.group(1); 150 | } 151 | return goodName + " " + getTaobaocomUri(uri); 152 | } 153 | 154 | private String getTaobaocomUri(String url) { 155 | String uri = url; 156 | String[] split = uri.split("&"); 157 | if (split.length == 1) { 158 | return split[0]; 159 | } 160 | String[] id = uri.split("\\?"); 161 | String param = ""; 162 | for (int i = 0; i < split.length; i++) { 163 | String item = split[i]; 164 | if (item.startsWith("id") && item.contains("id=")) { 165 | param = item; 166 | break; 167 | } 168 | } 169 | return id[0].concat("?").concat(param); 170 | } 171 | 172 | private void mainEntry() { 173 | String uri = getSysClipboard(); 174 | String site = ""; 175 | if (uri.contains("yangkeduo.com")) { 176 | site = "pdd"; 177 | } 178 | else if (uri.contains("jd.com")) { 179 | site = "jd"; 180 | } 181 | else if (uri.contains("taobao.com") || uri.contains("tb.cn")) { 182 | site = "taobao"; 183 | } 184 | else if (uri.contains("bilibili.com") || uri.contains("b23.tv")) { 185 | site = "bili"; 186 | } 187 | // TODO 支持更多网站 188 | 189 | // 2022-11-21 13:25:07 add site douyin 190 | else if (uri.contains("douyin.com")) { 191 | site = "douyin"; 192 | } 193 | // 2023-12-21 09:21:50 add site: xiaohongshu 194 | else if (uri.contains("xhslink.com") || uri.contains(XHS_DOMAIN)) { 195 | site = "xiaohongshu"; 196 | } 197 | 198 | if (ObjectUtil.isEmpty(site)) { 199 | System.out.println(uri + " not supported at the moment."); 200 | } 201 | String result; 202 | switch (site) { 203 | case "pdd": 204 | result = pddGoodsUri(uri); 205 | break; 206 | case "jd": 207 | result = jdUri(uri); 208 | break; 209 | case "taobao": 210 | result = taobaoUri(uri); 211 | break; 212 | case "bili": 213 | result = biliUri(uri); 214 | break; 215 | case "douyin": 216 | result = douyinUri(uri); 217 | break; 218 | case "xiaohongshu": 219 | result = xiaohongshuUri(uri); 220 | break; 221 | default: 222 | return; 223 | } 224 | setSysClipboard(result); 225 | System.out.println("your new clipboard text is: " + result); 226 | } 227 | 228 | private void setSysClipboard(String myString) { 229 | String osName = System.getProperty("os.name").toLowerCase(); 230 | if (osName.contains("linux")) { 231 | setClipboardUsingXClipOnLinux(myString); 232 | } 233 | else { 234 | // Windows、macOS 可以持久化这个内容到系统剪贴板 235 | StringSelection stringSelection = new StringSelection(myString); 236 | Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard(); 237 | clipboard.setContents(stringSelection, null); 238 | } 239 | 240 | } 241 | 242 | private void setClipboardUsingXClipOnLinux(String text) { 243 | // 加入 -n 选项以避免 echo 在末尾自动添加换行 244 | String[] cmd = {"/bin/bash", "-c", "echo -n " + escape(text) + " | xclip -selection clipboard"}; 245 | try { 246 | Runtime.getRuntime().exec(cmd); 247 | } 248 | catch (IOException e) { 249 | throw new RuntimeException(e); 250 | } 251 | } 252 | 253 | // 适当地转义文本中的特殊字符 254 | private String escape(String text) { 255 | return "'" + text.replace("'", "'\\''") + "'"; 256 | } 257 | 258 | private String getSysClipboard() { 259 | String result = ""; 260 | Clipboard clip = Toolkit.getDefaultToolkit().getSystemClipboard(); 261 | Transferable tr = clip.getContents(null); 262 | if (tr != null) { 263 | // 检查是文本类型再处理,其他类型不处理 264 | if (tr.isDataFlavorSupported(DataFlavor.stringFlavor)) { 265 | try { 266 | result = (String) tr.getTransferData(DataFlavor.stringFlavor); 267 | } 268 | catch (UnsupportedFlavorException | IOException ex) { 269 | throw new RuntimeException(ex); 270 | } 271 | } 272 | else { 273 | System.out.println("only deal with text."); 274 | } 275 | } 276 | else { 277 | System.out.println("Transferable is null!"); 278 | } 279 | System.out.println("your original clipboard text is: " + result); 280 | System.out.print(System.lineSeparator()); 281 | return result; 282 | } 283 | } 284 | --------------------------------------------------------------------------------