RSSHub/lib/utils
Tony 2ae10a7a02
feat: replace rand-user-agent with header-generator (#20480)
* Initial plan

* Replace @tonyrl/rand-user-agent with header-generator and add sec-ch-* headers

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* Extract unwanted UA validation into standalone isValidUserAgent function

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* Add headerGeneratorPreset option to got and ofetch utilities

* Replace randUserAgent usage in lib/routes with headerGeneratorPreset option

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* chore: pin header-generator to exact match

* fix: replace randUserAgent with a static user agent string in config

* Remove deprecated _randUserAgent function and related tests

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* refactor: rename to header-generator

* Changes before error encountered

Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>

* fix: update generateHeaders parameter type to Partial<HeaderGeneratorOptions>

* fix: add ios preset

* fix: rename headerGeneratorPreset to headerGeneratorOptions

* feat: add MODERN_IOS preset and update header-generator patch hash

* fix: use ios preset

* fix: update FetchOptions to include headerGeneratorOptions type

* refactor: simplify header generation and update header inclusion list

* fix: remove header-generator patch

* refactor: update user-agent handling in requests and tests

* test: remove redundant headerGeneratorOptions test and add new ofetch header preset test

* test: add custom user-agent test for ofetch

* test: update user-agent expectation in ofetch headers test

* Update lib/utils/ofetch.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* refactor: replace hardcoded headers with generatedHeaders in fetch and get request handlers

* refactor: rename test suite from rand-user-agent to header-generator and remove unused dependency

* Update lib/utils/header-generator.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: TonyRL <11386903+TonyRL@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-14 00:49:41 +08:00
..
cache feat(cache): enhance tryGet method to support generic return types (#19261) 2025-06-03 09:37:43 +08:00
otel chore(deps): bump the opentelemetry group with 5 updates (#18637) 2025-03-25 18:54:32 +08:00
proxy feat(lib/proxy): support multiple proxier by `PROXY_URIS` env 2025-06-24 11:05:14 +08:00
request-rewriter feat: replace rand-user-agent with header-generator (#20480) 2025-11-14 00:49:41 +08:00
cache.test.ts
camelcase-keys.spec.ts refactor: fix lint issue (#18963) 2025-04-29 03:00:16 +08:00
camelcase-keys.ts refactor: fix all no array reduce (#18973) 2025-04-30 04:16:32 +08:00
common-config.test.ts fix(utils/common-config): import parseDate and timezone functions for date handling (#19023) 2025-05-06 02:07:35 +08:00
common-config.ts fix(utils/common-config): import parseDate and timezone functions for date handling (#19023) 2025-05-06 02:07:35 +08:00
common-utils.test.ts
common-utils.ts refactor: fix lint issue (#18963) 2025-04-29 03:00:16 +08:00
debug-info.ts
git-hash.ts refactor: fix lint issue (#18963) 2025-04-29 03:00:16 +08:00
got-deprecated.ts
got.test.ts feat: replace rand-user-agent with header-generator (#20480) 2025-11-14 00:49:41 +08:00
got.ts fix: get SearchParams string (#17445) 2024-11-05 11:27:13 +08:00
header-generator.test.ts feat: replace rand-user-agent with header-generator (#20480) 2025-11-14 00:49:41 +08:00
header-generator.ts feat: replace rand-user-agent with header-generator (#20480) 2025-11-14 00:49:41 +08:00
helpers.test.ts feat: video duration 2025-05-18 11:54:06 +08:00
helpers.ts fix(route/bilibili): duration for video route 2025-10-09 10:44:18 +08:00
logger.ts feat: vercel compatibility 2025-08-11 21:39:03 +08:00
md5.test.ts
md5.ts refactor: fix lint issue (#18963) 2025-04-29 03:00:16 +08:00
ofetch.test.ts feat: replace rand-user-agent with header-generator (#20480) 2025-11-14 00:49:41 +08:00
ofetch.ts feat: replace rand-user-agent with header-generator (#20480) 2025-11-14 00:49:41 +08:00
parse-date.test.ts
parse-date.ts feat: compile ts through tsdown (#18908) 2025-04-22 22:29:11 +08:00
puppeteer-utils.test.ts chore(deps-dev): bump eslint-plugin-unicorn from 60.0.0 to 61.0.2 (#19998) 2025-09-08 23:03:48 +08:00
puppeteer-utils.ts
puppeteer.test.ts fix(tests): update expected values and increase timeout for Puppeteer tests 2025-08-22 17:45:39 +08:00
puppeteer.ts feat(utils/puppeteer): replace `puppeteer-extra-plugin-stealth` with `rebrowser` (#19506) 2025-07-01 04:38:05 +08:00
readable-social.test.ts style: auto format 2025-06-22 16:44:52 +00:00
readable-social.ts feat: support float score threshold for douban (#19425) 2025-06-23 00:42:44 +08:00
render.ts
request-rewriter.test.ts feat: replace rand-user-agent with header-generator (#20480) 2025-11-14 00:49:41 +08:00
rss-parser.test.ts
rss-parser.ts
timezone.test.ts
timezone.ts refactor: fix lint issue (#18963) 2025-04-29 03:00:16 +08:00
valid-host.test.ts
valid-host.ts
wait.test.ts
wait.ts
wechat-mp.test.ts fix(core/utils/wechat-mp): empty description when request blocked by WAF (#15294) 2024-04-22 02:19:33 +08:00
wechat-mp.ts chore(deps-dev): bump eslint-plugin-unicorn from 61.0.2 to 62.0.0 (#20381) 2025-10-28 01:26:56 +08:00