Commit Graph

8302 Commits

Author SHA1 Message Date
IChengHo 6969cf1bcb
fix(route/twreporter): Prevent crash when hero_image or og_image is missing (#20481)
The original code directly accessed nested properties on
`post.hero_image`, which is not guaranteed to exist.
This cause a TypeError when the image object was nil.

This commit introduces safe access by:
- Prioritizing `hero_image` but falling back to `og_image`
- Only rendering the banner if a valid image URL was resolved
2025-11-14 11:22:13 +08:00
Tony 2ae10a7a02
feat: replace rand-user-agent with header-generator (#20480)
* Initial plan

* Replace @tonyrl/rand-user-agent with header-generator and add sec-ch-* headers

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* Extract unwanted UA validation into standalone isValidUserAgent function

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* Add headerGeneratorPreset option to got and ofetch utilities

* Replace randUserAgent usage in lib/routes with headerGeneratorPreset option

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* chore: pin header-generator to exact match

* fix: replace randUserAgent with a static user agent string in config

* Remove deprecated _randUserAgent function and related tests

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* refactor: rename to header-generator

* Changes before error encountered

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* fix: update generateHeaders parameter type to Partial<HeaderGeneratorOptions>

* fix: add ios preset

* fix: rename headerGeneratorPreset to headerGeneratorOptions

* feat: add MODERN_IOS preset and update header-generator patch hash

* fix: use ios preset

* fix: update FetchOptions to include headerGeneratorOptions type

* refactor: simplify header generation and update header inclusion list

* fix: remove header-generator patch

* refactor: update user-agent handling in requests and tests

* test: remove redundant headerGeneratorOptions test and add new ofetch header preset test

* test: add custom user-agent test for ofetch

* test: update user-agent expectation in ofetch headers test

* Update lib/utils/ofetch.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor: replace hardcoded headers with generatedHeaders in fetch and get request handlers

* refactor: rename test suite from rand-user-agent to header-generator and remove unused dependency

* Update lib/utils/header-generator.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-14 00:49:41 +08:00
Ethan Shen 60ad86b6f6
fix(route): 艾瑞咨询研究报告 (#20450)
* fix(route): 艾瑞咨询研究报告

* fix loop

* fix typo

* fix typo
2025-11-13 00:15:25 +08:00
Kenny Fong bea6da0dd2
feat(route/futunn): add live, topic and video (#20464)
* add live, topic and video

* fix issues

* Update lib/routes/futunn/topic.ts
2025-11-12 22:26:35 +08:00
Tony 4e247a46ae
fix(route/youtube): ensure youtubei is not invoked when using google api (#20469) 2025-11-11 21:54:48 +08:00
Tony dac672d3bb
fix(route/youtube): ensure youtubei is not invoked when using google api (#20465) 2025-11-11 15:27:47 +08:00
Tony da32313da8
feat(route): add netflix (#20453)
* feat(route): add netflix

* feat: add research

* fix: remove migrated entries

* fix: clean up

* fix: remove optional chaining for embedLink and title in render function

* fix: remove optional chaining for embedLink

* fix: image url

* fix: remove empty target field in radar configuration
2025-11-09 23:00:07 +08:00
Aicnal Liueic f0f561a894
feat(router): add router of Capital Normal University (#20430) 2025-11-07 18:28:16 +08:00
Ethan Shen 83f7013951
feat(route): add 数字尾巴文章 (#20435) 2025-11-06 00:59:15 +08:00
Tony 13e7784944
fix(route): apple app store (#20427) 2025-11-05 05:11:28 +08:00
Tony 9e3ca7471c
fix(route): bilibili ranking (#20426)
* fix(route): bilibili ranking

* fix: split apiUrl for signing

* feat(route): add radar mapping for ranking redirects
2025-11-04 22:47:28 +08:00
Ethan Shen efd864446d
feat(route): add 中国炼焦行业协会栏目 (#20418) 2025-11-04 09:04:53 +08:00
Tony d2c310c66c
feat(route): add ctinews (#20419)
* feat(route): add ctinews

* fix(handler): adjust publication date timezone for video items

* fix(handler): correct timezone function import for publication date
2025-11-04 01:20:14 +08:00
AhsokaTano26 36eb792493
feat(route): add CQU jwc notification (#19468) 2025-11-03 22:00:35 +08:00
DIYgod 8d560d18f4
fix(route/bilibili/video): update Referer and origin headers for API request 2025-11-03 11:30:10 +08:00
Tony ec0d80d91f
fix(route/tophub): fix selectors (#20408)
* fix(route/tophub): fix selectors

* fix(route/tophub): include image in response
2025-11-03 04:18:27 +08:00
Ethan Shen 64399c50d1
feat(route): add 星林社区 (#20403)
* feat(route): add 星林社区

* fix typo
2025-11-02 16:18:31 +08:00
Tony 4e93cf357b
fix(route): apple iap (#20404) 2025-11-02 01:30:43 +08:00
HappyZhu99 77b07b3b47
feat(router/aiaa): rebuild rss from official (#20390)
* feature(route/aiaa): new router added

* feature(router/aiaa): rebuild rss from official

* feature(route/aiaa): new to old pudate list with preprint and current issue

* feature(route/aiaa): fix example

* feat(route/aiaa): fix path

* feat(route/aiaa): fix path

* feat(route/aiaa): reuse DataItem and other optimization

* delete empty file

* fix namespace
2025-10-30 14:31:31 +08:00
HappyZhu99 b93555f1ff
fix(route/ieee): add full abstract (#20391)
* fix(route/ieee): add full abstract

* fix(route/ieee): get full abstract elegantly

* better performance
2025-10-30 13:57:03 +08:00
Tony 9810bf7618
fix(route/mymusicsheet): update urls (#20395)
* fix(route/mymusicsheet): update urls

* fix(route/mymusicsheet): update isFree logic to handle freeOnly parameter correctly
2025-10-30 05:01:39 +08:00
Tony c33c9b1762
fix(route): add youtube referer (#20396) 2025-10-30 04:56:32 +08:00
Tony e6cef5ad4c
fix(middleware): ensure referrerpolicy is set only if not already defined (#20394) 2025-10-30 04:40:05 +08:00
Ethan Shen b9c1f81f9d
feat(route): add 数字尾巴鲸闻 (#20384)
* feat(route): add 数字尾巴鲸闻

* fix typo
2025-10-30 00:05:19 +08:00
dependabot[bot] 64a3b06d16
chore(deps): bump ofetch from 1.4.1 to 1.5.0 (#20392)
* chore(deps): bump ofetch from 1.4.1 to 1.5.0

Bumps [ofetch](https://github.com/unjs/ofetch) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/unjs/ofetch/releases)
- [Changelog](https://github.com/unjs/ofetch/blob/v1.5.0/CHANGELOG.md)
- [Commits](https://github.com/unjs/ofetch/compare/v1.4.1...v1.5.0)

---
updated-dependencies:
- dependency-name: ofetch
  dependency-version: 1.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: use `query` instead of deprecated `params`

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-29 21:09:00 +08:00
AnitsuriW 283cc9cb93
feat(route): add AzurLane JP news (#20374)
* feat(route): add AzurLane JP news

* fix(route): azurlane jp incorrect namespace

* Update lib/routes/azurlane/news.ts
2025-10-29 19:53:58 +08:00
Stephen Zhou a0c5b08936
fix(route/bilibili): only get subtitle for jsonfeed (#20377)
* fix(route/bilibili): only get subtitle for jsonfeed

* Update lib/middleware/cache.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/middleware/cache.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

---------
2025-10-29 15:32:20 +08:00
Andvari cb1cf517c7
fix(route/zaobao): Adapt to interactive page link (#20383)
* fix(route/zaobao): Adapt to interactive page link

* .
2025-10-29 02:13:56 +08:00
DIYgod 6197f745b4
feat(youtube): customize fetch method for Innertube instance 2025-10-28 21:42:22 +08:00
mslxl 49a58764fa
fix(route/github): Fix URL link error in events (#20386)
* fix(route/github): Fix url of PR event not links to HTML page

* fix(route/github): Fix repo info be replaced in push event

* style: auto format

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-10-28 20:14:44 +08:00
Ethan Shen 33ff12b8f7
feat(route): add 国家核安全局 (#20376)
* feat(route): add 国家核安全局

* feat: add docs

* fix typo
2025-10-28 02:40:33 +08:00
dependabot[bot] 52c18b7f25
chore(deps-dev): bump eslint-plugin-unicorn from 61.0.2 to 62.0.0 (#20381)
* chore(deps-dev): bump eslint-plugin-unicorn from 61.0.2 to 62.0.0

Bumps [eslint-plugin-unicorn](https://github.com/sindresorhus/eslint-plugin-unicorn) from 61.0.2 to 62.0.0.
- [Release notes](https://github.com/sindresorhus/eslint-plugin-unicorn/releases)
- [Commits](https://github.com/sindresorhus/eslint-plugin-unicorn/compare/v61.0.2...v62.0.0)

---
updated-dependencies:
- dependency-name: eslint-plugin-unicorn
  dependency-version: 62.0.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* style: fix unicorn/prefer-string-raw

* style: fix unicorn/no-useless-collection-argument

* style: fix unicorn/no-immediate-mutation

* style: fix unicorn/no-immediate-mutation

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-28 01:26:56 +08:00
Tony 1cc197a104
feat(route/youtube): only get subtitle for jsonfeed (#20375) 2025-10-26 22:56:09 +08:00
Tsuyumi25 e1fe935494
docs(mercari): clarify Japan-only support (#20373)
* docs(mercari): clarify Japan-only support

* Update search.ts
2025-10-26 14:51:30 +08:00
github-actions[bot] 00bd2cc244 style: auto format 2025-10-25 11:33:27 +00:00
mslxl 259bdcc002
feat(route): add GitHub repo, user and org event routes (#20330)
* feat(route): add GitHub repository event route

Co-authored-by: LiangddBUG <113956019+LiangddCoder@users.noreply.github.com>
Signed-off-by: mslxl <i@mslxl.com>

* feat(route): add GitHub user and org event routes

Signed-off-by: mslxl <i@mslxl.com>

---------

Signed-off-by: mslxl <i@mslxl.com>
Co-authored-by: LiangddBUG <113956019+LiangddCoder@users.noreply.github.com>
2025-10-25 19:31:37 +08:00
AwesomeDog ba0c820aee
feat(route): add eastmoney gerenzhongxin 4 routes (#19791)
* feat(route): add eastmoney gerenzhongxin cfh

* feat(route): add eastmoney gerenzhongxin guba trpl

* feat(route): add eastmoney gerenzhongxin gather

* Update lib/routes/eastmoney/gerenzhongxin/cfh.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/cfh.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/guba.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/cfh.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/trpl.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/guba.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/trpl.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/gather.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/guba.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/eastmoney/gerenzhongxin/trpl.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update use post_title instead.

* Update use blockquote instead.

* restore the abbreviated content as title. For the website is like twitter, there are only contents, no post_title.

* use blockquote to properly and semantically format the post and reply content

* Some of the posts do have a valid post_title. Now the post title has a precedence, if it is not valid, then use the abbreviated content

* fix deepscan result for INSUFFICIENT_NULL_CHECK

* fix deepscan result

---------
2025-10-25 18:55:56 +08:00
Keo 9d4efc58bb
feat(route): add NTR BLOG (#20369)
* feat(route): add NTR BLOG

* add feed lang

* Update lib/routes/ntrblog/articles.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* Update lib/routes/ntrblog/articles.ts

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

* update removal selectors

--------
2025-10-25 05:52:04 +00:00
chris zheng e45d803691
fix(route/cjlu/yjsy): use Puppeteer to fetch CJLU Graduate School notices after anti-crawler update (#20359)
* fix(route/yjsy): use puppeteer to bypass WAF

* fix: chage waiting-util to get fully loaded page

* fix: remove 'stylesheet' from excludeResourceTypes in yjsy route

* chore: provide default title for notifications in yjsy route

* fix: add HTTP headers for Puppeteer page requests in yjsy route

* fix: replace global ua in config with a static value

* fix: change waitUntil from 'networkidle0' to 'networkidle2'

* fix: set ua with setUserAgent

* fix: ensure user agent is set after HTTP headers in Puppeteer page requests

* fix: add limit parameter to handler for controlling result set size

* fix: update resource type handling in request interception

- use default maxAge managed by env
2025-10-25 00:42:50 +08:00
Alex ddf62a3269
fix: wikipedia/current-events: content extraction and html output (#20338)
* Fix content extraction and html output

- Content extraction was breaking in case it included a templating
  instruction
- the html output was non-compliant html by having nested <ul>...</ul>
  not belonging to a <li>...</li> element.

* refactor processListsAndLines in smaller functions

* fix date consistency

- title was using a date reconstructed from UTC time which led to
discrepancy between the content date and the title
- pubDate was based off the gmt time at which the process was
run. Instead, if the last possible entry time (23.59.59 gmt-12) is
passed, we use that, if not we use current time. This should reflect
more or less a form of 'publish' date status from this portal.

* ensure empty-ish page doesn't lead to entry creation
2025-10-24 23:34:04 +08:00
CathorQ b42bca977d
fix(route/gelbooru): Fix variable name from useId to userId (#20263) 2025-10-24 23:03:33 +08:00
Bendancom 71e1d2b5c0
fix(route/hypergryph/arknights/arktca): rename the route & fix the source URL error (#19897) 2025-10-24 20:02:54 +08:00
Tony dad3e58d66 fix(route/abc): fix namespace disambiguation 2025-10-24 18:09:33 +08:00
Breezy d4f5b8d261
fix(route/npr): added common name; updated audio selector; removed unwanted elements; prefer tags to slug (#19712)
* fix(route/npr): added common name; updated audio selector; removed unwanted elements; prefer tags to slug

* Updated `<p>` to `<br>` as suggested.

Co-authored-by: Tony <TonyRL@users.noreply.github.com>

---------
2025-10-24 17:35:08 +08:00
Tony 43e8b3ef83
fix(route/patreon): fix feed title and description (#20351) 2025-10-22 11:27:36 +08:00
HappyZhu99 a547e382fd
fix(route/sciencedirect): fix that abstract will not show when highlights appear (#20333)
* fix(route/sciencedirect): fix abstract will not show when highlights appear

now both abstract and highlights will show. only abstract will be shown when there is no highlights for the paper

* Update current-issue.ts

* fix(route/sciencedirect): fix vary length of abstract array
2025-10-21 01:29:39 +08:00
Tony 20e903ef90
fix(route/yahoo/utils): fix yahoo header overflow (#20336) 2025-10-20 21:24:44 +08:00
Ethan Shen e03111c570
feat(route): add Oak Ridge National Laboratory All News (#20279) 2025-10-20 00:06:18 +08:00
Loongphy Wei 5dea530119
feat(route): Cognition Blog (#20314)
* feat(route): Cognition Blog

* refactor: simplify blog route by removing query limit and content fetching

* feat: fetch page 1
2025-10-19 21:27:19 +08:00
Tony f3e71025ca
fix(route/apnews): replace routePath helper with direct access (#20329) 2025-10-19 15:15:37 +08:00