Commit Graph
93 Commits
Author SHA1 Message Date
8d8a435a48 fix(route/theinitium): rewrite for Ghost CMS after site migration (#21071)
* refactor(theinitium): rewrite routes for Ghost CMS, add cookie support

- Rewrite all routes to use Ghost Content API (old API deprecated)
- Add INITIUM_MEMBER_COOKIE env var for paid article full-text
- Add features block with requireConfig to all route files
- Add url field and radar rules per RSSHub standards
- Strip -zh-hans suffix from author/category display names
- Update namespace with English name and zh translations
- Mark follow route as deprecated (Ghost doesn't support it)
- Fix hostname binding to 0.0.0.0 for LAN access

* fix(theinitium): use routeExpire with refresh=false to prevent stale cache

Cache was never expiring because refresh=true (default) extends TTL on every hit.
Now uses 5-min routeExpire and won't extend on access.

* refactor(theinitium): address review feedback

- Remove unused INITIUM_USERNAME/PASSWORD/BEARER_TOKEN env keys
- Remove hardcoded User-Agent (use RSSHub's internal UA)
- Remove fragile html.length > 500 check
- Remove refresh: false from cache.tryGet
- Hoist stripLangSuffix to module scope (dedupe with cleanName)

* fix(theinitium): detect paywall CTA instead of length check

If cookie is invalid/expired, scrape returns paywall HTML.
Check for .gh-post-upgrade-cta to detect this and fall back
to cleaner Ghost API preview.

* revert(index): restore '::' hostname per #16513

* feat(theinitium): clean Ghost Koenig card HTML for RSS output

* fix(theinitium): use redirected Ghost API URL, remove unused config types

* fix(theinitium): remove unrelated neureality config changes

---------

Co-authored-by: Ryan Yao <ryanyao@Macmini.lan>
Co-authored-by: Ryan Yao <ryanyao@Ryans-Mac-mini.lan>
Co-authored-by: Ryan Yao <ryanyao@Ryans-Mac-mini.local>
2026-03-09 04:13:25 +08:00
Thomas 7465febbdf feat(route/youtube): Allow configuring custom embed url (#21248) 2026-03-01 00:28:45 +08:00
CutCut 4c4339c625 feat(route/twitter): support official api with user credentials (#21139)
* feat(route/twitter): support official api with user credentials (act as a logged user) support

* fix: update namespace docs

* fix(route/twitter): use optional chaining for access tokens
2026-02-14 13:32:44 +08:00
Tony 2f6303fcaf feat(route/twitter): official api support (#21109)
* feat(route/twitter): official api support

* fix: update namespace docs
2026-02-10 22:32:13 +08:00
Skylwn 8388a4db51 feat(route): add huitun route (#21056)
* add huitun xiaohongshu route

* refactor: add cache for getNoteLink

* fix: sort imports
2026-02-06 15:58:04 +08:00
wsmbsbbz dc68ecb875 feat(route): add support for f95zone.to (#21042)
* feat(route): add f95zone routes

* fix(route): fix f95zone threads route for folo parse

* fix(route): parse f95zone thread guid from [YYYY-MM-DD] in title

* fix(route): use update date in thread title as pubDate

* fix(route/f95zone): revert image URL processing

* fix(route/f95zone): reenable image URL processing for telegram-bot

* feat(route/f95zone): support post and thread route

* feat(route/f95zone): optimize title

* feat(route/f95zone): optimize image display

* feat(route/f95zone): optimize image display

* feat(route/f95zone): reoptimize img-elements processing

* feat(route/f95zone): reset to the original route pattern

* feat(route/f95zone): optimize routes

* feat(route/f95zone): optimize content processing

* feat(route/f95zone): reduce code

* fix(route/f95zone): use parseDate for proper date handling

* fix(route/f95zone): remove broken radar

* fix(route/f95zone): remove invalid radar config and improve docs

- Remove radar config from post route (postId is in URL hash, cannot be extracted)
- Add empty radar array to prevent Folo parsing error
- Add detailed URL format documentation for both routes
- Explain how to convert browser URL to subscription URL

* fix(route/f95zone): fix invalid example URL

* fix(route/f95zone): use internally generated UA

* fix(route/f95zone): only fetch the first and the last page

* fix(route/f95zone): remove unnecessary sorting logic
2026-02-04 02:43:36 +08:00
DIYgodandClaude Haiku 4.5 c90e71c359 feat(xiaohongshu): add proxy configuration support (#20952)
Allow xiaohongshu routes to bypass anti-scraping measures by routing requests through a configurable proxy. When XIAOHONGSHU_PROXY is set, all xiaohongshu HTTP requests (user, board, notes) will be routed through the proxy endpoint instead of using Puppeteer.

Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
2026-01-22 17:17:04 +08:00
github-actions[bot] a28890a135 style: auto format 2025-11-27 02:55:46 +00:00
Stephen Zhou b4a2aa5e70 fix(core): package mode (#20580) 2025-11-26 20:44:18 +08:00
Stephen Zhou 400fb1fa49 fix(route/picnob): use puppeteer-real-browser to pass cf check (#20478) 2025-11-14 15:32:09 +08:00
2ae10a7a02 feat: replace rand-user-agent with header-generator (#20480)
* Initial plan

* Replace @tonyrl/rand-user-agent with header-generator and add sec-ch-* headers

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* Extract unwanted UA validation into standalone isValidUserAgent function

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* Add headerGeneratorPreset option to got and ofetch utilities

* Replace randUserAgent usage in lib/routes with headerGeneratorPreset option

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* chore: pin header-generator to exact match

* fix: replace randUserAgent with a static user agent string in config

* Remove deprecated _randUserAgent function and related tests

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* refactor: rename to header-generator

* Changes before error encountered

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* fix: update generateHeaders parameter type to Partial<HeaderGeneratorOptions>

* fix: add ios preset

* fix: rename headerGeneratorPreset to headerGeneratorOptions

* feat: add MODERN_IOS preset and update header-generator patch hash

* fix: use ios preset

* fix: update FetchOptions to include headerGeneratorOptions type

* refactor: simplify header generation and update header inclusion list

* fix: remove header-generator patch

* refactor: update user-agent handling in requests and tests

* test: remove redundant headerGeneratorOptions test and add new ofetch header preset test

* test: add custom user-agent test for ofetch

* test: update user-agent expectation in ofetch headers test

* Update lib/utils/ofetch.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor: replace hardcoded headers with generatedHeaders in fetch and get request handlers

* refactor: rename test suite from rand-user-agent to header-generator and remove unused dependency

* Update lib/utils/header-generator.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-14 00:49:41 +08:00
la3rence a7469fe8a0 feat(route)!: qweather api host change (#20042)
* feat(route)!: qweather api host change

* chore: typo
2025-09-12 18:37:16 +08:00
karasuandTony e20f27e491 feat(route): add sdo/ff14risingstones (#19914)
* feat(route): sdo/ff14risingstones

* fix: add a check for `from_info`

* feat: add category

* fix: raw output

* feat: add author

* fix: remove the duplicate `[RP]`

* docs: add information for the `ua` configuration

* feat: add fc create_time

* feat: add fc create_time

* fix: add fc detail_mask

* fix error message

* Update utils.ts

* Update lib/routes/sdo/ff14risingstones/strats.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/sdo/ff14risingstones/posts.ts
2025-09-09 06:40:09 +08:00
Stephen Zhou b630afc51a feat(route/bilibili): config to include subtitles or not (#19834)
* feat(route/bilibili): config to include subtitles or not

* update

* update
2025-08-14 11:05:15 +08:00
Thomas b2e2c9e6a0 feat(route/tumblr): Add Tumblr posts route (#19646)
* Add Tumblr posts route

* Reorder based on name

* Store refresh token longer

* Remove radar route as it cannot match part of a path

* Maintainer name case

* Throw ConfigNotFoundError if minimum required config is not present

* Camel case function name and reorder to be alphabetical

* Keep refresh token for a year

* Don't trust refactoring

* Get limit from query params

* Remove old doc

* Retrieve post tags
2025-07-22 01:01:27 +08:00
LucunJi a8f770d335 fix(route/lofter): fix Lofter tag route requiring cookie (#19548) 2025-07-05 21:42:17 +08:00
Lawrence Ching ab278982a5 feat: Exclude NSFW feeds when env variable DISABLE_NSFW is set to true (#19448)
* Exclude NSFW feeds when env variable DISABLE_NSFW is set to true

* Update lib/registry.ts

---------
2025-07-04 03:24:27 +08:00
pseudoyu 3d9ac5c6ee feat(lib/proxy): support multiple proxier by PROXY_URIS env 2025-06-24 11:05:14 +08:00
DIYgod 63ceeecce2 fix(bsky): api 2025-06-17 19:35:43 +08:00
DIYgod 94160fa0f4 fix(bsky): keyword 2025-06-17 17:48:08 +08:00
karasuandTony 2b16676200 feat(route): mixi2 (#19359)
* feat(route): mixi2

* feat(route/mixi2): 添加参数描述

* feat(route/mixi2): 添加 feed 标题

* feat(route/mixi2): discovery 订阅路由添加图片

* feat(route/mixi2): 添加view属性

* fix(route/mixi2): 修复类型错误

* Update lib/routes/mixi2/utils.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/mixi2/community.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/mixi2/home.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/mixi2/discovery.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* add requireConfig

* update

* add post filter

* fix the filter
2025-06-15 20:52:50 +08:00
DIYgod 4a84b0267a feat: implement enableCluster 2025-06-09 21:55:24 +08:00
chris zheng 62ae012cc7 feat(route/mangadex): add three new routes and modify one old route for mangadex.org (#18269)
* feat(namespace): add namespace for CJLU

* feat(route): Add route for graduate school of CJLU

为中国计量大学研究生院添加路由
支持将`教师通知`和`研究生通知`页面转化为 RSS 订阅源

* doc: modify discription of some params

修改参数描述

* fix(radar rule): modify radar rule

修改 radar 规则,以正确提供订阅源

* fix: add exception handling

增加异常处理

* chore: optimize conditional logic

优化判断逻辑

* Update lib/routes/cjlu/yjsy/index.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix(radar rule): list all source choices

在 radar 规则中列出全部的类型
修改行尾符为 LF

* fix(example): correct example of the route

订正路由的案例

* chore: add missing namespace information

* fix(mangadex): resolve incorrect route detection

* chore: update route name and include myself as a maintainer

* feat(mangadex): add token retrieval utility base on config

* feat(mangadex): add utility to retrieve user configurations

Added a utility function `getFilteredLanguages` to retrieve filtered languages based on the current user's configuration.

* feat: add `getMangaMeta` utility to retrive metadata for a manga

* refactor: add utility to retrive chapter info for a manga

Original implementation by @vzz64 was embedded in modules.
Extracted the common logic into a standalone function for better code reusability and maintainability.

* refactor(feed util): move the fallback logic into function

Add a docstring to explain its usage

* doc(access util): add docstrings for key function

* refactor: purify result of function

Change result type from object to array by remove `total` param.

* feat(feed util): add support for alternative parameter type

- Add docstring for util functions
- Add a new combination function
- Improve code type inference

* refactor: move constants to separate files for better organization

* feat: modify the route for single manga

- **important**: add a prefix to the route for space for other routes in namespace, not compatiable to former route
- use integrated utility to retrieve details of a single manga

* refactor: move `token expire time` to constant

* docs(access): add Error docstring

* chore: make cover became thumbnail to save bandwidth and better display

* feat: add new route for user follow's list

- support reading, plan to read etc.
- add new optional param  to utility function

* fix: improve perfomance in retrieving manga's meta data

- if cover is needed, use the optional param `includes` to fetch in one request

* refactor: rearrange code

- skip some code for branch without cover needed

* docs: add description for route

* docs: add a detailed param description for route

- offer a default value for param

* feat: add a new constant

* feat: seperate util-function for better organization

- add docstring for `toQueryString`

* feat: add new utility function to fetch manga meta data in batch

* fix: use dynamic key to cache result

- otherwise it may get wrong result with diffrent types
- rearrange the code

* feat: add a new route for all user's followed mangas

* chore: add parameters' desctiption

- offer a more detailed example
- use local constant to maintance limit
- change some code's style

* style: change style in using utility function

* style: refactor query string construction for improved readability

* style: change a variable's name

* feat: add new route for sepecific mdlist feed

* fix: use correct time for pubDate

* fix: use `image` prop to display thumbnail instead of `description`

* refactor(mangadex): improve access token retrieval logic and handle refresh token errors

* refactor(mangadex): replace getMangaMeta with getMangaMetaByIds for improved efficiency

* chore: modify route's decription

* fix: remove unnecessary nullish coalescing in description assignment

* fix: update access token retrieval to use client ID and secret instead of username and password

* fix: correct spellings in definition

* chore: remove unnecessary code

* fix: dedupe ids before fetch manga metas

- add `limit` param to get the full response

* fix: remove wrong suffix

* fix: remove unnecessary code

* chore: remove unnessary code

* fix: set manga and config feed's cache to not refreshable

* fix: remove filteredLanguages option

* fix: remove filteredLanguages option

* fix: add sort operation after dedupe

* style: add trailing comma

* fix: remove common params

* fix: remove unnecessary  code

* fix: remove route's unnecessary name prefix

* fix: change TTL for hot caches

* fix: remove docs for non-exist config

* refactor: fix lint issue for better readability

* fix: ensure cover image is only set if cover filename exists

---------
2025-06-06 22:29:13 +08:00
DIYgod 44aba27033 fix: smzdm 2025-05-29 12:51:04 +08:00
Kjasn 0510d8391e feat(route): add Jumeili home route (#19021) 2025-05-05 23:57:21 +08:00
CrackTC 10288db67d fix(route/saraba1st): add host config & change default host (#18895) 2025-04-28 03:05:19 +08:00
undefinedandTony fd173c0e38 feat(route): add route Gelbooru (#18320)
* feat(route): add route Gelbooru

* feat(route): add image quality option for Gelbooru

* feat(route): remove URL.canParse

* feat(route): enhance  display

* Update lib/routes/gelbooru/namespace.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/gelbooru/namespace.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: Apply code review

* fix: Apply code review

* fix: add __dirname var

* Update lib/routes/gelbooru/templates/description.art

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/gelbooru/post.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: Add config key by alphabetical order

* fix: `e` comes before `u` in alphabet

---------
2025-03-18 09:58:49 +08:00
UZQueenandTony 72f78e2bfb feat(route): add misskey home timeline (#18244)
* feat(route): add misskey home timeline

* Update lib/routes/misskey/home-timeline.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Merge remote-tracking branch 'head/master'

* feat(route/misskey): Enhance user timeline with simplifyAuthor parameter and update replyToAuthor in utils

---------
2025-02-03 22:12:57 +08:00
DIYgod 131bc33637 feat: update follow config 2024-12-24 16:56:52 +08:00
DIYgod dd08502955 feat: add follow limit config 2024-12-20 22:50:49 +08:00
DIYgod 2aab5047b7 feat: follow config 2024-12-18 23:16:28 +08:00
pseudoyu 3f12f922bf feat(route/twitter): add third-party twitter api support 2024-11-29 13:50:59 +08:00
dddaniel1andTony f9641b5aef fix(route): xiaohongshu fulltext add cookie authentication (#17228)
* fix(route/xiaohongshu) add cookie authentication

* fix(route/xiaohongshu) add cookie authentication

* fix(route/xiaohongshu) add cookie authentication

* fix(route/xiaohongshu) add method annotation

---------

Co-authored-by: Tony <TonyRL@users.noreply.github.com>
2024-11-21 21:53:37 +08:00
Tony d9ca0d09b1 feat(route): patreon (#17621)
* feat(route): patreon

* fix: typo

* fix: typo
2024-11-19 02:21:24 +08:00
Tony 4e404a7328 chore: disable remote debugger by default (#17498) 2024-11-08 04:08:05 +08:00
pseudoyu b4fece47ed feat(route/telegram): add mtproxy support for tglib 2024-11-05 12:58:58 +07:00
Andvari 35536f45ad feat(route/caixin): Add full text fetch (#17231)
* feat(route/caixin): Add full text fetch

* .

* Update blog.ts

* Update utils.ts

* Update utils.ts

* Update utils-fulltext.ts

* Update utils-fulltext.ts
2024-11-03 17:03:20 +07:00
pseudoyu a69ff2fa7b feat(route/dianping): add dianping user route 2024-10-31 13:35:27 +07:00
asong 374c865603 feat(route): add bbs guozaoke route (#17170)
* add routes guozaoke

* add routes guozaoke

* fix ESLint Check warning

* fix ESLint Check warning

* fix ESLint Check warning

* fix ESLint Check warning

* fix ESLint Check warning

* fix routes guozaoke code review

* fix routes guozaoke code review

* fix route code review

* fix code revice issue

* fix code revice issue

* fix code revice issue

* fix code revice issue

* fix code revice issue

* fix code revice issue

* fix code review issues

* fix code review issues
2024-10-30 12:02:55 +08:00
Eric Zhu b60ef4de32 feat(twitter): support LoginEnterAlternateIdentifierSubtask task and better login flow controll (#17331)
* feat(twitter): support LoginEnterAlternateIdentifierSubtask task and better login flow controll

* chore: lint

* chore: lint

* refactor(twitter): login flow

* fix: var name
2024-10-30 04:28:56 +08:00
pseudoyu 26bc26a3b8 feat(route/telegram): add telegram session scripts and logics 2024-10-27 21:37:37 +07:00
Keo ed8fa7a5ea fix(route/sis001): add cookie to solve anti-crawler (#17259)
* fix(route/sis001): add cookie to resolve anti-crawler

* fix desc

* move base url to env config

* update doc

* fix: cache cookie

---------
2024-10-23 22:50:37 +08:00
karasu bc020b6f18 feat(route): sorrycc.com (#17261)
* feat(route): 云谦的博客

* parseDate

* change the route name
2024-10-23 22:18:14 +08:00
Otstar Lin 9f44e77237 fix(route/bilibili): add addDmVerifyInfoWithInter to bypass the risk control mechanism (#17147) 2024-10-18 09:47:17 +08:00
TsuyumiandTony e4deee086f feat(route): add route for skeb (#17047)
* feat(route): add route for skeb

* fix: wrap function call in map() to resolve ESLint warning

* refactor: Enhance type safety and rename functions for Work and Creator processing

* refactor: optimize followings API and caching logic

* fix: remove request_key

* fix: resolve request_key issues in works API
- Fetch request_key from initial API response and cache it
- Replace hardcoded User-Agent with config.ua

* fix: misc improvements
- Remove unnecessary InvalidParameterError checks and imports
- Simplify route names by removing "Skeb" prefix
- Set refresh to false in Skeb API cache
- Fix typo in search.ts filename

* fix: remove URL fragments from radar sources

* refactor: Optimize category structure and enhance data processing
- Remove 'request_masters' and 'first_requesters' categories as they are not creators
- Implement template rendering for richer output
- Enhance work data processing with audio handling
- Improve creator data with commission status information

* fix: add __dirname

* fix: prevent potential function loop from unexpected API errors

* Update lib/routes/skeb/works.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* fix: correct Skeb bearer token naming and references

---------
2024-10-14 23:18:02 +08:00
quiniapiezoelectricity 6deef181c5 feat: Add L'Orient-Le Jour Route (#17030)
* feat: Add L'Orient-Le Jour Route

* fix codefactor

* fix codefactor
2024-10-08 23:34:34 +08:00
valuex 1d22c0ad38 feat(route): add route of Yamap (#16902)
* Create namespace.ts

* Add files via upload

* Update dongtai.ts

* Update gonggao.ts

* Update gonggao.ts

* Update gonggao.ts

* Update dongtai.ts

* Update gonggao.ts

* Update gonggao.ts

* Update gonggao.ts

* Update gonggao.ts

* Update dongtai.ts

* Create namespace.ts

* Add files via upload

* Update articles.ts

* Update articles.ts

* Update articles.ts

* Update articles.ts

* Update articles.ts

* Update articles.ts

* Update articles.ts

* route for 日经新闻半导体,产业聚焦

* Update scienceatechnology.ts

* Update semi.ts

* Delete lib/routes/nikkei/semi.ts

* Delete lib/routes/nikkei/scienceatechnology.ts

* Update articles.ts

* Update namespace.ts

* Update articles.ts

* Update articles.ts

* Delete lib/routes/szftedu directory

* fix: category

* fix: typo

---------
2024-09-27 01:01:01 +08:00
Jinkin 57df3f698e feat: add route of uestc bbs (#16892)
* feat: add route of uestc bbs

* Update lib/routes/uestc/bbs.ts

路由应该是/uestc/bbs/newthread?

* amend: 按意见修改变量名

Signed-off-by: Jinkin <nadph513@outlook.com>

* fix: namespace url

---------

Signed-off-by: Jinkin <nadph513@outlook.com>
2024-09-27 00:37:58 +08:00
quiniapiezoelectricityandTony 87723d1b56 feat(route): add malaysiakini.com (#16752)
* feat(route): add malaysiakini.com

* fixing issues

* Added additonal authentication method with email and password

* Apply suggestions from code review

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Apply suggestions from code review

---------
2024-09-15 21:57:15 +08:00
incubator4 be3ba61788 chore: otel fix and enhancement (#16754)
* feat(otel): use routePath insteadof path

* feat(otel): add otel bucket boundary env

* chore: move otel to debug block with alphabetical sort
2024-09-15 19:42:21 +08:00