mirror of
https://github.com/JoeanAmier/XHS-Downloader.git
synced 2026-03-22 15:07:17 +08:00
feat(app): 支持 API 模式设置临时代理
This commit is contained in:
1
.gitignore
vendored
1
.gitignore
vendored
@@ -6,3 +6,4 @@ __pycache__/
|
||||
*.db
|
||||
*.json
|
||||
/.idea/
|
||||
/Temp/
|
||||
|
||||
46
README.md
46
README.md
@@ -1,5 +1,5 @@
|
||||
<div align="center">
|
||||
<img src="static/XHS-Downloader.png" alt="" height="256" width="256"><br>
|
||||
<img src="static/XHS-Downloader.png" alt="XHS-Downloader" height="256" width="256"><br>
|
||||
<h1>XHS-Downloader</h1>
|
||||
<p>简体中文 | <a href="README_EN.md">English</a></p>
|
||||
<a href="https://trendshift.io/repositories/5435" target="_blank"><img src="https://trendshift.io/api/badge/repositories/5435" alt="JoeanAmier%2FXHS-Downloader | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
@@ -155,7 +155,13 @@
|
||||
<td align="center">cookie</td>
|
||||
<td align="center">str</td>
|
||||
<td align="center">请求数据时使用的 Cookie;可选参数</td>
|
||||
<td align="center">配置文件 Cookie 值</td>
|
||||
<td align="center">配置文件 cookie 参数</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">proxy</td>
|
||||
<td align="center">str</td>
|
||||
<td align="center">请求数据时使用的代理;可选参数</td>
|
||||
<td align="center">配置文件 proxy 参数</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">skip</td>
|
||||
@@ -167,18 +173,20 @@
|
||||
</table>
|
||||
<p><b>代码示例:</b></p>
|
||||
<pre>
|
||||
def api_demo():
|
||||
async def example_api():
|
||||
"""通过 API 设置参数,适合二次开发"""
|
||||
server = "http://127.0.0.1:6666/xhs/"
|
||||
data = {
|
||||
"url": "https://www.xiaohongshu.com/explore/123456789",
|
||||
"url": "", # 必需参数
|
||||
"download": True,
|
||||
"index": [
|
||||
3,
|
||||
6,
|
||||
9,
|
||||
],
|
||||
"proxy": "http://127.0.0.1:10808",
|
||||
}
|
||||
response = requests.post(server, json=data)
|
||||
response = post(server, json=data, timeout=10)
|
||||
print(response.json())
|
||||
</pre>
|
||||
<h1>📜 其他说明</h1>
|
||||
@@ -560,9 +568,31 @@ A: 由于权限限制,您无法直接触发主仓库的 Actions。请通过 Fo
|
||||
<li><b>TikTokDownloader(抖音、TikTok)</b>:<a href="https://github.com/JoeanAmier/TikTokDownloader">https://github.com/JoeanAmier/TikTokDownloader</a></li>
|
||||
<li><b>KS-Downloader(快手、KuaiShou)</b>:<a href="https://github.com/JoeanAmier/KS-Downloader">https://github.com/JoeanAmier/KS-Downloader</a></li>
|
||||
</ul>
|
||||
<h1>💰 项目赞助</h1>
|
||||
<img src="https://resources.jetbrains.com/storage/products/company/brand/logos/PyCharm.svg" alt="PyCharm logo">
|
||||
<p><b>JetBrains</b> 支持全球开源社区认可的活跃项目,并为非商业开发提供免费许可证。</p>
|
||||
|
||||
# 💰 项目赞助
|
||||
|
||||
## JetBrains 工具
|
||||
|
||||

|
||||
|
||||
**JetBrains** 支持全球开源社区认可的活跃项目,并为非商业开发提供免费许可证。
|
||||
|
||||
***
|
||||
|
||||
## TikHub
|
||||
|
||||
<img src="static/赞助商_TikHub_Logo.png" alt="TikHub">
|
||||
<p><a href="https://tikhub.io/">TikHub</a> 是一家领先的数据接口服务供应商,专注于提供高质量的数据接口,涵盖了多个热门平台,包括 抖音、TikTok、小红书、Instagram、Twitter 和 快手 等平台。</p>
|
||||
<p>TikHub 还提供定制化的服务,如直播间监控、作品监控和达人监控,以满足不同业务场景的需求。</p>
|
||||
<p>通过每日签到,用户可以免费获取一定额度的使用量;可以使用我的 <strong>推荐链接</strong>:<a href="https://user.tikhub.io/users/signup?referral_code=ZrdH8McC">https://user.tikhub.io/users/signup?referral_code=ZrdH8McC</a> 或 <strong>推荐码</strong>:<code>ZrdH8McC</code>,注册并充值即可获得 <code>$2</code> 额度!</p>
|
||||
<p><a href="https://tikhub.io/">TikHub</a> 提供以下服务:</p>
|
||||
<ul>
|
||||
<li>丰富的数据接口</li>
|
||||
<li>每日签到免费获取额度</li>
|
||||
<li>高质量的 API 服务</li>
|
||||
<li>官网:<a href="https://tikhub.io/">https://tikhub.io/</a></li>
|
||||
<li>用户登陆地址:<a href="https://user.tikhub.io/">https://user.tikhub.io/</a></li>
|
||||
</ul>
|
||||
<h1>⚠️ 免责声明</h1>
|
||||
<ul>
|
||||
<li>使用者对本项目的使用由使用者自行决定,并自行承担风险。作者对使用者使用本项目所产生的任何损失、责任、或风险概不负责。</li>
|
||||
|
||||
46
README_EN.md
46
README_EN.md
@@ -1,5 +1,5 @@
|
||||
<div align="center">
|
||||
<img src="static/XHS-Downloader.png" alt="" height="256" width="256"><br>
|
||||
<img src="static/XHS-Downloader.png" alt="XHS-Downloader" height="256" width="256"><br>
|
||||
<h1>XHS-Downloader</h1>
|
||||
<p><a href="README.md">简体中文</a> | English</p>
|
||||
<a href="https://trendshift.io/repositories/5435" target="_blank"><img src="https://trendshift.io/api/badge/repositories/5435" alt="JoeanAmier%2FXHS-Downloader | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
@@ -158,8 +158,14 @@
|
||||
<tr>
|
||||
<td align="center">cookie</td>
|
||||
<td align="center">str</td>
|
||||
<td align="center">Cookies used when requesting data; Optional parameter</td>
|
||||
<td align="center">Settings Cookie Value</td>
|
||||
<td align="center">Cookie used when requesting data; Optional parameter</td>
|
||||
<td align="center">Settings cookie Value</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">proxy</td>
|
||||
<td align="center">str</td>
|
||||
<td align="center">Proxy used when requesting data; Optional parameter</td>
|
||||
<td align="center">Settings proxy Value</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">skip</td>
|
||||
@@ -171,18 +177,20 @@
|
||||
</table>
|
||||
<p><b>Code example:</b></p>
|
||||
<pre>
|
||||
def api_demo():
|
||||
async def example_api():
|
||||
"""通过 API 设置参数,适合二次开发"""
|
||||
server = "http://127.0.0.1:6666/xhs/"
|
||||
data = {
|
||||
"url": "https://www.xiaohongshu.com/explore/123456789",
|
||||
"url": "", # 必需参数
|
||||
"download": True,
|
||||
"index": [
|
||||
3,
|
||||
6,
|
||||
9,
|
||||
],
|
||||
"proxy": "http://127.0.0.1:10808",
|
||||
}
|
||||
response = requests.post(server, json=data)
|
||||
response = post(server, json=data, timeout=10)
|
||||
print(response.json())
|
||||
</pre>
|
||||
<h1>📜 Others</h1>
|
||||
@@ -585,9 +593,31 @@ repository to execute the build process
|
||||
<li><b>TikTokDownloader(抖音、TikTok)</b>:<a href="https://github.com/JoeanAmier/TikTokDownloader">https://github.com/JoeanAmier/TikTokDownloader</a></li>
|
||||
<li><b>KS-Downloader(快手、KuaiShou)</b>:<a href="https://github.com/JoeanAmier/KS-Downloader">https://github.com/JoeanAmier/KS-Downloader</a></li>
|
||||
</ul>
|
||||
<h1>💰 Sponsor</h1>
|
||||
<img src="https://resources.jetbrains.com/storage/products/company/brand/logos/PyCharm.svg" alt="PyCharm logo">
|
||||
|
||||
# 💰 Project Sponsorship
|
||||
|
||||
## JetBrains Tools
|
||||
|
||||

|
||||
|
||||
<p><b>JetBrains</b> support active projects recognized within the global open-source community with complimentary licenses for non-commercial development.</p>
|
||||
|
||||
***
|
||||
|
||||
## TikHub
|
||||
|
||||
<img src="static/赞助商_TikHub_Logo.png" alt="TikHub">
|
||||
<p><a href="https://tikhub.io/">TikHub</a> is a leading data interface service provider, specializing in offering high-quality data interfaces for multiple popular platforms, including DouYin, TikTok, Xiaohongshu, Instagram, Twitter, and Kuaishou.</p>
|
||||
<p>TikHub also provides customized services such as live streaming room monitoring, post monitoring, and influencer monitoring to meet the needs of different business scenarios.</p>
|
||||
<p>Through daily sign-ins, users can obtain a certain quota of usage for free. You can use my <strong>referral link</strong>:<a href="https://user.tikhub.io/users/signup?referral_code=ZrdH8McC">https://user.tikhub.io/users/signup?referral_code=ZrdH8McC</a> or <strong>Referral Code</strong>:<code>ZrdH8McC</code>,Register and recharge to receive <code>$2</code> limit!</p>
|
||||
<p><a href="https://tikhub.io/">TikHub</a> Provide the following services:</p>
|
||||
<ul>
|
||||
<li>Rich data interfaces</li>
|
||||
<li>Free daily check-in to obtain credit limit</li>
|
||||
<li>High quality API services</li>
|
||||
<li>Official website:<a href="https://tikhub.io/">https://tikhub.io/</a></li>
|
||||
<li>User login address:<a href="https://user.tikhub.io/">https://user.tikhub.io/</a></li>
|
||||
</ul>
|
||||
<h1>⚠️ Disclaimer</h1>
|
||||
<ul>
|
||||
<li>Users decide on their own how to use this project and bear the risks themselves. The author is not responsible for any losses, liabilities, or risks incurred by users in the use of this project</li>
|
||||
|
||||
19
example.py
19
example.py
@@ -1,5 +1,6 @@
|
||||
from asyncio import run
|
||||
|
||||
from httpx import post
|
||||
from rich import print
|
||||
|
||||
from source import XHS
|
||||
@@ -61,6 +62,23 @@ async def example():
|
||||
print(await xhs.extract(demo_link, download, index=[1, 2]))
|
||||
|
||||
|
||||
async def example_api():
|
||||
"""通过 API 设置参数,适合二次开发"""
|
||||
server = "http://127.0.0.1:6666/xhs/"
|
||||
data = {
|
||||
"url": "", # 必需参数
|
||||
"download": True,
|
||||
"index": [
|
||||
3,
|
||||
6,
|
||||
9,
|
||||
],
|
||||
"proxy": "http://127.0.0.1:10808",
|
||||
}
|
||||
response = post(server, json=data, timeout=10)
|
||||
print(response.json())
|
||||
|
||||
|
||||
async def test():
|
||||
url = ""
|
||||
async with XHS(
|
||||
@@ -82,4 +100,5 @@ async def test():
|
||||
|
||||
if __name__ == "__main__":
|
||||
# run(example())
|
||||
# run(example_api())
|
||||
run(test())
|
||||
|
||||
@@ -19,7 +19,7 @@ dependencies = [
|
||||
"pyperclip>=1.9.0",
|
||||
"pyyaml>=6.0.2",
|
||||
"rookiepy>=0.5.6",
|
||||
"textual>=2.1.2",
|
||||
"textual>=3.0.0",
|
||||
"uvicorn>=0.34.0",
|
||||
]
|
||||
|
||||
|
||||
BIN
requirements.txt
BIN
requirements.txt
Binary file not shown.
@@ -287,6 +287,7 @@ class XHS:
|
||||
bar,
|
||||
data: bool,
|
||||
cookie: str = None,
|
||||
proxy: str = None,
|
||||
):
|
||||
if await self.skip_download(i := self.__extract_link_id(url)) and not data:
|
||||
msg = _("作品 {0} 存在下载记录,跳过处理").format(i)
|
||||
@@ -297,6 +298,7 @@ class XHS:
|
||||
url,
|
||||
log=log,
|
||||
cookie=cookie,
|
||||
proxy=proxy,
|
||||
)
|
||||
namespace = self.__generate_data_object(html)
|
||||
if not namespace:
|
||||
@@ -552,11 +554,10 @@ class XHS:
|
||||
None,
|
||||
not extract.skip,
|
||||
extract.cookie,
|
||||
extract.proxy,
|
||||
):
|
||||
msg = _("获取小红书作品数据成功")
|
||||
else:
|
||||
msg = _("获取小红书作品数据失败")
|
||||
data = None
|
||||
return ExtractData(
|
||||
message=msg, url=url[0] if url else extract.url, data=data
|
||||
)
|
||||
return ExtractData(message=msg, params=extract, data=data)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from httpx import HTTPError
|
||||
from httpx import get
|
||||
|
||||
from ..module import ERROR, Manager, logging, retry, sleep_time
|
||||
from ..translation import _
|
||||
@@ -19,6 +20,7 @@ class Html:
|
||||
self.retry = manager.retry
|
||||
self.client = manager.request_client
|
||||
self.headers = manager.headers
|
||||
self.timeout = manager.timeout
|
||||
|
||||
@retry
|
||||
async def request_url(
|
||||
@@ -27,14 +29,15 @@ class Html:
|
||||
content=True,
|
||||
log=None,
|
||||
cookie: str = None,
|
||||
proxy: str = None,
|
||||
**kwargs,
|
||||
) -> str:
|
||||
headers = self.update_cookie(
|
||||
cookie,
|
||||
)
|
||||
try:
|
||||
match content:
|
||||
case True:
|
||||
match (content, bool(proxy)):
|
||||
case (True, False):
|
||||
response = await self.__request_url_get(
|
||||
url,
|
||||
headers,
|
||||
@@ -43,7 +46,17 @@ class Html:
|
||||
await sleep_time()
|
||||
response.raise_for_status()
|
||||
return response.text
|
||||
case False:
|
||||
case (True, True):
|
||||
response = await self.__request_url_get_proxy(
|
||||
url,
|
||||
headers,
|
||||
proxy,
|
||||
**kwargs,
|
||||
)
|
||||
await sleep_time()
|
||||
response.raise_for_status()
|
||||
return response.text
|
||||
case (False, False):
|
||||
response = await self.__request_url_head(
|
||||
url,
|
||||
headers,
|
||||
@@ -51,6 +64,15 @@ class Html:
|
||||
)
|
||||
await sleep_time()
|
||||
return str(response.url)
|
||||
case (False, True):
|
||||
response = await self.__request_url_head_proxy(
|
||||
url,
|
||||
headers,
|
||||
proxy,
|
||||
**kwargs,
|
||||
)
|
||||
await sleep_time()
|
||||
return str(response.url)
|
||||
case _:
|
||||
raise ValueError
|
||||
except HTTPError as error:
|
||||
@@ -81,6 +103,23 @@ class Html:
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
async def __request_url_head_proxy(
|
||||
self,
|
||||
url: str,
|
||||
headers: dict,
|
||||
proxy: str,
|
||||
**kwargs,
|
||||
):
|
||||
return await self.client.head(
|
||||
url,
|
||||
headers=headers,
|
||||
proxy=proxy,
|
||||
follow_redirects=True,
|
||||
verify=False,
|
||||
timeout=self.timeout,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
async def __request_url_get(
|
||||
self,
|
||||
url: str,
|
||||
@@ -92,3 +131,20 @@ class Html:
|
||||
headers=headers,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
async def __request_url_get_proxy(
|
||||
self,
|
||||
url: str,
|
||||
headers: dict,
|
||||
proxy: str,
|
||||
**kwargs,
|
||||
):
|
||||
return get(
|
||||
url,
|
||||
headers=headers,
|
||||
proxy=proxy,
|
||||
follow_redirects=True,
|
||||
verify=False,
|
||||
timeout=self.timeout,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
@@ -89,6 +89,7 @@ class Manager:
|
||||
self.print_proxy_tip(
|
||||
_print,
|
||||
)
|
||||
self.timeout = timeout
|
||||
self.request_client = AsyncClient(
|
||||
headers=self.headers
|
||||
| {
|
||||
|
||||
@@ -6,10 +6,11 @@ class ExtractParams(BaseModel):
|
||||
download: bool = False
|
||||
index: list = None
|
||||
cookie: str = None
|
||||
proxy: str = None
|
||||
skip: bool = False
|
||||
|
||||
|
||||
class ExtractData(BaseModel):
|
||||
message: str
|
||||
url: str
|
||||
params: ExtractParams
|
||||
data: dict | None
|
||||
|
||||
@@ -6,8 +6,9 @@
|
||||
4. 支持按作者归档保存作品文件
|
||||
5. 新增自动更新作者昵称功能
|
||||
6. 优化 `headers` 处理逻辑
|
||||
7. 支持 `SOCKS` 代理
|
||||
8. 支持设置作者别名
|
||||
7. 支持 API 模式设置临时代理
|
||||
8. 支持 `SOCKS` 代理
|
||||
9. 支持设置作者别名
|
||||
|
||||
**注意:**
|
||||
|
||||
|
||||
BIN
static/赞助商_TikHub_Logo.png
Normal file
BIN
static/赞助商_TikHub_Logo.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 5.0 KiB |
2
uv.lock
generated
2
uv.lock
generated
@@ -681,7 +681,7 @@ requires-dist = [
|
||||
{ name = "pyperclip", specifier = ">=1.9.0" },
|
||||
{ name = "pyyaml", specifier = ">=6.0.2" },
|
||||
{ name = "rookiepy", specifier = ">=0.5.6" },
|
||||
{ name = "textual", specifier = ">=2.1.2" },
|
||||
{ name = "textual", specifier = ">=3.0.0" },
|
||||
{ name = "uvicorn", specifier = ">=0.34.0" },
|
||||
]
|
||||
|
||||
|
||||
Reference in New Issue
Block a user