589ddb3a43
refactor(eslint): add back no-return-await ( #11877 )
...
* chore: add back no-return-await
* chore: fix action warning
* chore: fix all no-return-await
2023-02-16 04:30:05 +08:00
4c51a2b6c9
chore: skip stealth test in github actions
...
ref: https://docs.github.com/en/actions/learn-github-actions/variables#default-environment-variables
2023-01-22 21:08:59 +00:00
9ce117dd9f
chore: skip stealth test in github actions
2023-01-22 17:54:53 +00:00
a66cbcf6ee
feat(config)!: unsafe domain toggle ( #11588 )
2023-01-10 11:45:05 +00:00
728da7f154
fix: author is not iterable ( #11555 )
...
Co-authored-by: xianjiantao <xianjiantao@bytedance.com>
2023-01-04 21:44:23 +08:00
d2ab8ab3c0
feat(core): json feed ( #11494 )
...
* feat(core): json feed
* docs: json feed description
* fix: remove extra comma if no elements exist after author
* test: json feed test
* feat: generate json feed without art-template
* test: fix coverage
* style: 4 spaces json
* docs: fix parameter
2023-01-02 21:35:36 +08:00
d60d89ea87
feat(core): unify proxy; optimize request-wrapper ( #11472 )
...
* feat(core): unify proxy; optimize request-wrapper
Signed-off-by: Rongrong <i@rong.moe>
* Fix DeepScan issue
Signed-off-by: Rongrong <i@rong.moe>
* Simplify protocol existence judgment
Signed-off-by: Rongrong <i@rong.moe>
* chore: regenerate yarn.lock
Signed-off-by: Rongrong <i@rong.moe>
2022-12-23 01:13:01 +08:00
9264d7afbe
feat(core/utils/puppeteer-utils): Cookie utils ( #11471 )
...
Signed-off-by: Rongrong <i@rong.moe>
Signed-off-by: Rongrong <i@rong.moe>
2022-12-20 19:51:33 +08:00
1381377e91
fix(core/utils/puppeteer): proxy-chain error ( #11470 )
...
HTTPS/SOCKS proxy is not supported by proxy-chain and will cause an
error unconditionally. If an HTTPS/SOCKS proxy does not need
authentication, explicitly bypassing proxy-chain is a must. If it does
need, making a warning and continuing without proxy seems to be the best
choice.
Signed-off-by: Rongrong <i@rong.moe>
Signed-off-by: Rongrong <i@rong.moe>
2022-12-19 23:39:11 +08:00
0513566d04
fix(core): fix category filtering ( #11451 )
2022-12-14 23:30:18 +08:00
058cec21b7
test(puppeteer): wait for bot test result ( #11322 )
...
Signed-off-by: Rongrong <i@rong.moe>
Signed-off-by: Rongrong <i@rong.moe>
2022-11-23 22:32:15 +00:00
30975dd20b
chore(deps): replace `@postlight/mercury-parser` with `@postlight/parser` ( #10862 )
2022-09-23 03:25:56 +08:00
6251a2dc02
fix(core): parse date utils ( #10789 )
...
* fix(core): parse date utils
* fix: regex
2022-09-14 21:59:32 +08:00
878266c535
feat(core): true ua ( #10761 )
...
* feat(core): true ua
* test: skip mercury parser temporary
2022-09-12 01:13:38 +08:00
655b899715
feat(core): 增加 `sorted` 参数来控制结果排序 ( #10744 )
...
* feat: 增加 `sorted` 参数来控制结果排序
* feat: 添加 `sorted` 参数的测试
重用了 `sort` 的代码和数据：是不是可以考虑将两者结合？
2022-09-09 00:11:18 +08:00
9329a44c80
feat(core): support proxy config for pupperteer ( #10714 )
...
* feat: support proxy config for pupperteer
* test: add puppeteer proxy detection
* fix: package.json
* test: fix regex
2022-09-07 00:07:04 +08:00
4cab5a5894
test: puppeteer wait longer ( #10599 )
2022-08-25 00:24:00 +08:00
4877301f23
fix(core): throw URL cache key ( #10497 )
2022-08-16 21:38:05 +08:00
03117f4c50
feat(core): filter category ( #10212 )
2022-07-14 00:42:38 +08:00
bd80574f76
feat(route): add slowmist news ( #10044 )
...
* feat(route): add slowmist news
* feat(utils): remove useless ul for code section
* docs: add slowmist news route docs
* docs(quick-start): new router under /lib/v2
* test(utils): add test fixArticleContent for new wechat-mp code
* feat(router): update radar source of slowmist
2022-07-04 23:44:58 +08:00
dc64debaf7
feat(core): regex engine naming and test suit ( #10014 )
...
* feat(core): regex engine config
* fix(docs): config naming
* filter-engine test naming
2022-06-23 04:38:52 +08:00
5db837d6dc
feat(core): anti-hotlink experimental parameters ( #9997 )
...
Co-authored-by: Rongrong <i@rong.moe>
2022-06-20 18:41:16 -07:00
6a0d44348e
fix(core/parameter): missing some tags handling ( #9983 )
...
Signed-off-by: Rongrong <i@rong.moe>
2022-06-18 06:45:03 -09:00
1fcb5379df
fix(test): insufficiently robust puppeteer test ( #9974 )
...
Signed-off-by: Rongrong <i@rong.moe>
2022-06-17 21:55:33 +08:00
450e522167
feat(utils): decode cf protected email string ( #9900 )
...
* feat(utils): decode cf protected email string
* Update cf-email.js
2022-06-05 22:40:57 +08:00
e1ee28267e
fix(core): RequestInProgressError cache age ( #9850 )
...
* fix(core): RequestInProgressError cache age
Signed-off-by: Rongrong <i@rong.moe>
* fix: ci test
Signed-off-by: Rongrong <i@rong.moe>
2022-05-30 17:30:41 +08:00
13ec6be32b
fix(test): failing puppeteer without stealth test ( #9851 )
...
Signed-off-by: Rongrong <i@rong.moe>
2022-05-30 01:49:01 -03:00
2c4eab5e31
fix(core/parameter): absurd wrong path cache time ( #9796 )
...
Signed-off-by: Rongrong <i@rong.moe>
2022-05-21 21:44:45 +08:00
0f31bfa8b9
fix(utils): parse relative date with meridiem ( #9775 )
...
* fix(utils): parse relative date with meridiem
* fix: use regex
2022-05-17 20:21:47 +08:00
2bcc6f6ad0
test: puppeteer wait until test finish
2022-05-04 20:43:15 +00:00
a5d74e521c
feat(core/middleware/anti-hotlink): route matching ( #9681 )
...
* feat(core/middleware/anti-hotlink): route matching
Signed-off-by: Rongrong <i@rong.moe>
* fix(test): coverage
Signed-off-by: Rongrong <i@rong.moe>
* docs(install): fix mistaken config key
Signed-off-by: Rongrong <i@rong.moe>
2022-05-04 03:29:23 +08:00
0b544e1395
feat(utils): puppeteer-extra-plugin-stealth ( #9676 )
2022-05-03 01:17:17 +08:00
616ad78c88
fix(core/utils/anti-hotlink): invalid HTML output ( #9669 )
...
Signed-off-by: Rongrong <i@rong.moe>
2022-05-01 04:14:07 +08:00
0522c63d5f
fix(core/utils): invalid request header `Server` ( #9582 )
...
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
2022-04-20 22:32:46 +08:00
bafb3534e1
feat(utils): random user agent ( #9449 )
...
* feat(utils): random ua
* chore: bump rand-user-agent to 1.0.58(no more deps)
2022-04-12 17:51:07 +08:00
74e1f88a32
feat(core)(utils/wechat-mp): normalize URL ( #9497 )
...
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
2022-04-08 18:42:52 +08:00
a79cc20ec1
feat(utils): add utils for WeChat MP ( #9487 )
...
Motivation:
There are multiple routes that need to fetch articles from WeChat MP.
However, letting them fetch articles by themselves could potentially
lead to cache key collisions. Even if cache key collisions do not occur,
un-normalized URL could potentially lead to duplicated requests.
What's more, articles from WeChat MP have weird formats and need to be
fixed. Creating a universal function to do this work can create some
ease for new route contributors.
Note:
In order to make this PR atomic as much as possible, I did not touch
those broken routes. Once this PR is merged, I will try to fix them.
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
2022-04-07 21:46:15 +08:00
5b35471e39
fix(core): offending RFC4287 ( #9441 )
...
* fix(core): offending RFC4287
should not leave `<updated>` blank when `<published>` is not blank
these two fields MUST conform to the "date-time" production in RFC3339
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
* test(common-utils): complete tests
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
* test(template): restrict expected value of `pubDate`
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
2022-04-02 17:44:45 +08:00
bf834bdb7d
fix(utils): tests for parse-date ( #9428 )
2022-03-31 10:23:42 +08:00
d5648b37ee
fix(utils): parse relative dates with multiple time units ( #9365 )
...
* fix(utils): parse relative dates with multiple time units
* docs: remove warning
* fix: add more characters to match
* fix: rename to parse-date
2022-03-26 16:45:42 +08:00
3364b7eb1d
fix(route): migrate invalid `enclosure_length` usages to `itunes_duration` ( #9291 )
...
Close #9287
Signed-off-by: Rongrong <15956627+Rongronggg9@users.noreply.github.com>
2022-03-10 20:54:29 +08:00
0c6516c893
test: fix puppeteer error
2021-11-28 01:37:20 +00:00
ba09be474d
test: fix rss-parser update error
2021-11-28 00:43:59 +00:00
0792f7ba25
feat(core): first attempt to init script standard ( #8224 )
...
- lazy load
- rate limit per path
- init .debug.json support
- docs
- maintainer
- radar
2021-09-22 05:41:00 -07:00
d82847f541
style/chore(eslint): enforce new rules ( #8040 )
...
* style: prefer object shorthand syntax
* refactor: prefer Array#map over Array#forEach
* style: prefer arrow callback
* chore(eslint): update rules
* style: auto fix by eslint
2021-08-17 22:23:23 +08:00
6e3b58ed1d
refactor: avoid promise overhead ( #8028 )
2021-08-16 11:45:53 -07:00
1c9c4ccfc8
fix(core): make sure timeout error messages include URLs ( #7981 )
...
Before this fix, timeout messages are not quite useful
> error: Request undefined fail, retry attempt #1 : TimeoutError: Timeout awaiting 'request' for 5000ms
2021-08-12 01:27:07 -07:00
5298f845f2
feat: support ua in blacklist and whitelist
2021-03-07 10:06:24 +08:00
89e82d88fa
feat: got request timeout
2021-02-01 20:06:49 +08:00
177c36cae7
feat: sentry route timeout
2021-02-01 20:06:23 +08:00
Tony
DIYgod
零小白
Rongrong
Charlie Jiang
Atlas Quan
NeverBehave
Rongrong
Rongrong
Ethan Shen
Sukka
Chih-Hsuan Yen
DIYgod