xiaoxue_admin
/
RSSHub
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,508 Commits 1 Branch 0 Tags 161 MiB
Commit Graph

153 Commits

Author SHA1 Message Date
DIYgod 27eda52681
fix: logger.debug 2023-07-24 03:11:26 +01:00
DIYgod f1660e9723
feat: expand the scope of retries and set proxy strategy 2023-07-24 02:38:24 +01:00
DIYgod 030043347a
feat: exclude some requests for reverse proxy 2023-07-24 01:57:22 +01:00
DIYgod 3da47b69ed
fix: reverse proxy fails for some requests 2023-07-24 01:04:46 +01:00
DIYgod 541cae4397
test: fix empty request error and remove node 16 support 2023-07-23 21:41:12 +01:00
DIYgod cd6b5e0f0b
feat: bypass reverse proxy for requests with cookies 2023-07-23 21:28:00 +01:00
DIYgod d7224338a5
test: fix empty request error 2023-07-23 20:44:57 +01:00
github-actions[bot] 872ff0c538 style: auto format 2023-07-23 19:30:33 +00:00
DIYgod 3a5d293b7f
feat: support reverse proxy 2023-07-23 20:28:38 +01:00
Tony 1de2457216
fix(route): wxkol link (#12799) 
* fix(route): wxkol link

* fix(core): fix wechat-mp date parsing

`var ct = "timestamp"` can appear after `var ct=function()`
 2023-07-15 02:28:15 +08:00
TonyRL c3be79d888 chore: revert ua changes 2023-06-16 17:21:59 +00:00
github-actions[bot] 919d93c384 style: auto format 2023-06-16 17:05:30 +00:00
TonyRL 22eec4b834 fix: prevent nft removing ua db 2023-06-16 17:03:05 +00:00
Tony 84ee153f81
chore: update deps (#12675) 
* chore: update deps

* chore: extend docker build test timeout
 2023-06-16 23:15:22 +08:00
dependabot[bot] b982947a60
chore(deps): bump https-proxy-agent from 5.0.1 to 6.1.0 and socks-proxy-agent from 7.0.0 to 8.0.1 (#12445) 
* chore(deps): bump https-proxy-agent from 5.0.1 to 6.0.0

Bumps [https-proxy-agent](https://github.com/TooTallNate/proxy-agents/tree/HEAD/packages/https-proxy-agent) from 5.0.1 to 6.0.0.
- [Release notes](https://github.com/TooTallNate/proxy-agents/releases)
- [Changelog](https://github.com/TooTallNate/proxy-agents/blob/main/packages/https-proxy-agent/CHANGELOG.md)
- [Commits](https://github.com/TooTallNate/proxy-agents/commits/HEAD/packages/https-proxy-agent)

---
updated-dependencies:
- dependency-name: https-proxy-agent
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore(deps): bump https-proxy-agent from 5.0.1 to 6.0.0

* test: update test to use WHATWG URL API

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-05-07 00:25:12 +08:00
Tony a0ea328013
build: fix pnpm pptr postinstall (#12450) 
* chore: fix chrome not found in jest

* chore: explicitly require deps for stealth plugin
 2023-05-05 23:58:29 +08:00
dependabot[bot] 3ff763ee8d
chore(deps): bump puppeteer from 19.8.3 to 19.11.0 (#12385) 
* chore(deps): bump puppeteer from 19.8.3 to 19.11.0

Bumps [puppeteer](https://github.com/puppeteer/puppeteer) from 19.8.3 to 19.11.0.
- [Release notes](https://github.com/puppeteer/puppeteer/releases)
- [Changelog](https://github.com/puppeteer/puppeteer/blob/main/release-please-config.json)
- [Commits](https://github.com/puppeteer/puppeteer/compare/puppeteer-v19.8.3...puppeteer-v19.11.0)

---
updated-dependencies:
- dependency-name: puppeteer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* test: fix puppeteer exitCode

* feat: use new headless chrome

https://developer.chrome.com/articles/new-headless/

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-04-26 03:14:45 +08:00
DIYgod f60744241c
feat: remove torrent search function 2023-03-18 21:50:26 +00:00
Tony a9d6957e60
feat(route): wxkol (#12010) 
* feat(route): wxkol

* fix: use async pool
 2023-03-03 01:59:45 +08:00
Tony e348914bf5
refactor: move json feed rendering to separated module (#11938) 2023-02-22 20:47:04 +09:00
Tony 589ddb3a43
refactor(eslint): add back no-return-await (#11877) 
* chore: add back no-return-await

* chore: fix action warning

* chore: fix all no-return-await
 2023-02-16 04:30:05 +08:00
Tony a66cbcf6ee
feat(config)!: unsafe domain toggle (#11588) 2023-01-10 11:45:05 +00:00
DIYgod ab2f61824d
feat: output quote data for twitter and jike 2023-01-09 16:25:26 +00:00
Tony d2ab8ab3c0
feat(core): json feed (#11494) 
* feat(core): json feed

* docs: json feed description

* fix: remove extra comma if no elements exist after author

* test: json feed test

* feat: generate json feed without art-template

* test: fix coverage

* style: 4 spaces json

* docs: fix parameter
 2023-01-02 21:35:36 +08:00
Rongrong d60d89ea87
feat(core): unify proxy; optimize request-wrapper (#11472) 
* feat(core): unify proxy; optimize request-wrapper

Signed-off-by: Rongrong <i@rong.moe>

* Fix DeepScan issue

Signed-off-by: Rongrong <i@rong.moe>

* Simplify protocol existence judgment

Signed-off-by: Rongrong <i@rong.moe>

* chore: regenerate yarn.lock

Signed-off-by: Rongrong <i@rong.moe>
 2022-12-23 01:13:01 +08:00
Rongrong 9264d7afbe
feat(core/utils/puppeteer-utils): Cookie utils (#11471) 
Signed-off-by: Rongrong <i@rong.moe>

Signed-off-by: Rongrong <i@rong.moe>
 2022-12-20 19:51:33 +08:00
Rongrong 1381377e91
fix(core/utils/puppeteer): proxy-chain error (#11470) 
HTTPS/SOCKS proxy is not supported by proxy-chain and will cause an
error unconditionally. If an HTTPS/SOCKS proxy does not need
authentication, explicitly bypassing proxy-chain is a must. If it does
need, making a warning and continuing without proxy seems to be the best
choice.

Signed-off-by: Rongrong <i@rong.moe>

Signed-off-by: Rongrong <i@rong.moe>
 2022-12-19 23:39:11 +08:00
Tony 6316ff1a4f
feat(route): freewechat (#11419) 2022-12-09 05:25:44 +08:00
Derek c07c960957
fix: restore /hrbeu/ugs/news.js (#11224) 
* fix: restore /heu/ugs/news.js

* Update lib/v2/heu/ugs/news.js

* Update lib/v2/heu/ugs/news.js

* Update lib/v2/heu/ugs/news.js

* feat(require): update require

* refactor: mise fixes

* rename namespace to hrbeu
- hrbeu.edu.cn is the primary domain name as it's where the contact us mail hosted on

* increase code readability
- use template literals over .concat()
- use .includes() to check the existence of sth

* remove fake publish time

* remove duplicated title in description

* fix: fetch article from wechat mp

* fix: codeql
 2022-11-09 22:31:19 +08:00
evnydd0sf fdc59d2d65
feat: 增加设计学院研创、后浪模块；支持获取微信文章内容；修复重复条目bug (#10994) 
* 南京艺术学院研究生院

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/radar.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/radar.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/v2/nua/gra.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update docs/university.md

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update docs/university.md

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* 优化代码

* 优化

* update

* 补充南京艺术学院

* fix

* fix

* fix

* 更新南京艺术学院数据源。

* 更新南京艺术学院数据源。

* fix

* fix

* update

* fix doc

* fix

* Update lib/v2/nua/radar.js

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: yarn.lock permission

* style: auto format

* fix double fetch same page

* Update dc.js

* Revert "Merge branch 'master' of https://github.com/evnydd0sf/RSS-Hub"

This reverts commit dd56f9469b5b2462d482c202ae798b3f159f243e, reversing
changes made to d900050eb8271d0f566070eb58c2c9d87ccd6e53.

* fix

* Revert "fix"

This reverts commit 820ff88120d4fb4fc2bdcd8ee52229d1bfb83c17.

* Revert "Revert "Merge branch 'master' of https://github.com/evnydd0sf/RSS-Hub""

This reverts commit ce67e0b53c231a8a8527b7407211aff1ab7e50d4.

* 增加微信公众号解析引导。

* 修正url

* 适配微信公众号内容。

* 适配站内文章、微信公号号文章和未知文章引导。

* Update wechat-mp.js

* fix fetch wechat bug

* 增加设计学院研创、后浪模块

* bugfix

* Revert "Merge branch 'DIYgod:master' into master"

This reverts commit 102cb48db6990fd5a699d2e46bf05d58be2e5ba5, reversing
changes made to ccac4be5180ff7c2d81763efa7c09cf0b752ddc4.

* Update university.md

* Revert "Revert "Merge branch 'DIYgod:master' into master""

This reverts commit c8841910b83ec4a07e0d92d2d65feb192af9e5e3.

* fix fetch listname

* fix: switch default case
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2022-10-05 21:32:49 +08:00
Tony 6251a2dc02
fix(core): parse date utils (#10789) 
* fix(core): parse date utils

* fix: regex
 2022-09-14 21:59:32 +08:00
DIYgod 9329a44c80
feat(core): support proxy config for pupperteer (#10714) 
* feat: support proxy config for pupperteer

* test: add puppeteer proxy detection

* fix: package.json

* test: fix regex
 2022-09-07 00:07:04 +08:00
Tony 9fa248c324
fix(route): nmpa feed items on 3rd party site (#10146) 
* fix(route): nmpa feed items on 3rd party site

* fix: regex unescaped .
 2022-07-06 00:46:05 +08:00
Atlas Quan bd80574f76
feat(route): add slowmist news (#10044) 
* feat(route): add slowmist news

* feat(utils): remove useless ul for code section

* docs: add slowmist news route docs

* docs(quick-start): new router under /lib/v2

* test(utils): add test fixArticleContent for new wechat-mp code

* feat(router): update radar source of slowmist
 2022-07-04 23:44:58 +08:00
Tony 2f26479fc9
feat(route): sspai series update (#9901) 2022-06-05 23:59:10 +08:00
github-actions[bot] 5bc8be9ac5 style: auto format 2022-06-05 14:42:33 +00:00
Tony 450e522167
feat(utils): decode cf protected email string (#9900) 
* feat(utils): decode cf protected email string

* Update cf-email.js
 2022-06-05 22:40:57 +08:00
Tony a5fd8de30e
fix(docker): puppeteer stealth not working in docker (#9896) 2022-06-05 17:09:34 +08:00
MisLink df6f4caf8b
feat(route): add 北京大学国家发展研究院 - 观点 (#9804) 
* feat(route): add 北京大学国家发展研究院 - 观点

Signed-off-by: MisLink <gjq.uoiai@outlook.com>

* Fix https issue

Signed-off-by: MisLink <gjq.uoiai@outlook.com>

* fetch full text from wechat-mp and pku news

Signed-off-by: MisLink <gjq.uoiai@outlook.com>

* Fix ci

Signed-off-by: MisLink <gjq.uoiai@outlook.com>

* refactor: sort new route
 2022-05-27 19:12:04 +08:00
Tony 0f31bfa8b9
fix(utils): parse relative date with meridiem (#9775) 
* fix(utils): parse relative date with meridiem

* fix: use regex
 2022-05-17 20:21:47 +08:00
Rongrong 23fcb6bc5a
feat(core/utils/request-wrapper): request logging (#9691) 
Signed-off-by: Rongrong <i@rong.moe>
 2022-05-04 12:36:48 +10:00
Tony 0b544e1395
feat(utils): puppeteer-extra-plugin-stealth (#9676) 2022-05-03 01:17:17 +08:00
Rongrong 7a6be9a229
feat(core): customizable Chromium executable path (#9670) 
* feat(core): customizable Chromium executable path

also build Chromium-bundled Docker image for arm/arm64

Signed-off-by: Rongrong <i@rong.moe>

* chore: fix typo

Signed-off-by: Rongrong <i@rong.moe>

* chore(CI/test): using build matrix

Signed-off-by: Rongrong <i@rong.moe>

* docs(install): fix punctuation

Signed-off-by: Rongrong <i@rong.moe>
 2022-05-01 21:00:29 +08:00
Tony 34b58ebc64
fix(utils): request without hostname (#9649) 2022-04-28 19:35:31 +08:00
任平生 9d9926d0bf
fix(utils): 修复抓取微信已删除文章时遇到的报错 (#9589) 
* fix(utils): 支持微信公众号单图片文章抓取

* fix(utils): 支持输出微信公众号转载文章阅读原文链接

* fix(utils): 修复抓取微信已删除文章时遇到的报错

* refactor: migrate to v2

Co-authored-by: blankyu(于海洋) <blankyu@tencent.com>
 2022-04-21 21:40:16 +08:00
dependabot[bot] dd4a216648
chore(deps): bump socks-proxy-agent from 6.1.1 to 6.2.0 (#9572) 
* chore(deps): bump socks-proxy-agent from 6.1.1 to 6.2.0

Bumps [socks-proxy-agent](https://github.com/TooTallNate/node-socks-proxy-agent) from 6.1.1 to 6.2.0.
- [Release notes](https://github.com/TooTallNate/node-socks-proxy-agent/releases)
- [Changelog](https://github.com/TooTallNate/node-socks-proxy-agent/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TooTallNate/node-socks-proxy-agent/compare/v6.1.1...v6.2.0)

---
updated-dependencies:
- dependency-name: socks-proxy-agent
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: use dot notation

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: TonyRL <TonyRL@users.noreply.github.com>
 2022-04-21 03:54:53 +08:00
Rongrong 0522c63d5f
fix(core/utils): invalid request header `Server` (#9582) 
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
 2022-04-20 22:32:46 +08:00
任平生 f3e069d399
fix(utils): 支持微信公众号单图片文章抓取; 增加输出阅读原文链接 (#9557) 
* fix(utils): 支持微信公众号单图片文章抓取

* fix(utils): 支持输出微信公众号转载文章阅读原文链接
 2022-04-19 01:29:45 +08:00
Levi Zim 958be6266e
feat(route): 山东大学（威海）新闻网 (#9537) 
* feat(sduwh): add extractors.

* feat(route): add route for 山东大学（威海）新闻网

* docs: for route sduwh/news

* docs: for route sduwh/news

(cherry picked from commit 831830167a39a658520dd9a46b375fdbbcb5d611)

* feat(radar): for route 山东大学（威海）新闻网

* refactor: change `got.get` to `got`.

* refactor: prefer `parseDate()` to `new Date()`

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: incomplete URL substring sanitization.

Make CodeQL happy.

* fix(radar): fix target field.

* fix: change route /sduwh to /sdu/wh

* fix: remove superfluous slash character in url.

* feat: look for exact date first.

* feat: extract exact date from news extractor.

* feat: extract exact date from view extractor.

* feat: extractor for www.sdrj.sdu.edu.cn

* refactor: semantic separation of sduwh with sdu

* feat(radar): more accurate name

* docs: update documentation

* refactor: migrate to v2

* refactor: fix deprecated url.resolve

* fix: update docs url

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: sdu not working routes

* fix: accurate `ctx.state.data.url`

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: better error handling for extractors.

* fix: timezone

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: better error handling.

Co-authored-by: Tony <TonyRL@users.noreply.github.com>
 2022-04-17 00:01:39 +08:00
任平生 eb467afae1
fix(utils): 支持将微信公众号转载文章的正文抓取回来 (#9534) 
* feat: 修正日期时间匹配规则、移除一些不必要评论元素

* fix(route)(fortunechina): 修正财富中文网1、双语文章中文内容重复问题；2、移除 kol 大头像

* fix(route)(wechat): 支持将微信公众号转载文章的正文抓取回来
 2022-04-15 19:23:09 +08:00