The /xueqiu/stock_info route was broken by Alibaba Cloud WAF: the cookie
acquisition could not pass the challenge, and the data request was sent
to xueqiu.com which the WAF blocks for these endpoints.
- cookies.ts: allow the `script` resource type in the page.route filter
so the WAF challenge scripts can execute, enabling cookie extraction
via Patchright
- stock-info.ts: request data from api.xueqiu.com instead of xueqiu.com
to bypass the WAF
- stock-info.ts: align the supported types with the tabs on the stock
page (all / discuss / trans / news / announcement). all / discuss /
trans use the symbol search endpoint; news / announcement use the
stock timeline endpoint. The previously broken research type is removed
- stock-info.ts: fetch the stock name from the lightweight quote API
since it is rendered client-side and cannot be scraped from the HTML
- stock-info.ts: throw InvalidParameterError for unsupported types
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(route): repair u3c3 search and drop sticky promo rows
The handler destructured `keywoard` (typo for `keyword`), so the search
branch never ran and every query fell through to the homepage listing.
u3c3 has also added an anti-scrape token (`search2=<token>`, rotated and
embedded in the homepage `search21()` JS) — `?search=<kw>` alone silently
returns the homepage, so scrape the token live with a graceful fallback.
Also skip the sticky promo rows (category-nav and external-ad rows, e.g.
the "國產原创" entries) shown on every page including search; real torrent
rows always link to /view.
* refactor(route): use filter+map instead of flatMap in u3c3
Address review feedback: the flatMap returning []/[obj] was just a
filter+map in disguise. Skip non-/view rows up front, then map.
* feat(route): add EFE Noticias route
Add route for EFE (Agencia EFE) Spanish news agency with 10 categories:
mundo, espana, economia, cultura, ciencia-y-tecnologia, deportes,
salud, medio-ambiente, educacion, euro-efe.
Features:
- Article images via enclosure and inline in description
- Ad filtering (auto-banner, srr-main, promotional logos)
- Clean img attributes for RSS reader compatibility
- Configurable item limit via query parameter
* fix(route/efe): address review comments
- Remove `limit` from `parameters` (query param, not path param)
- Scope article link selector to `.elementor-loop-container` to avoid anti-pattern #1
- Remove unnecessary `.first()` on `.elementor-widget-theme-post-content`
- Remove `.srr-main` from cleanup (not inside content area), keep `decoding`/`loading` on images
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: address review comments for EFE route
- Fix parameters to structured object with only path params
- Scope selector to .elementor-post article links only
- Remove unnecessary .first() on title
- Stop stripping image attributes (decoding, loading, width, etc.)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: revert overly specific selector that broke all routes
.elementor-post class doesn't exist on EFE site, use attribute
selector to pre-filter article URLs instead.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: fetch articles sequentially to avoid 429 rate limiting
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: suppress no-await-in-loop lint for sequential rate-limited fetches
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: use specific selector for article title links only
Select only .elementor-widget-theme-post-title a inside .e-loop-item
to avoid matching image links, category links, or duplicate entries.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: use single quotes for static string (oxlint stylistic/quotes)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: use Promise.all and remove Set (selector matches once per article)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* refactor: use Array.map instead of push for link collection
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(efe): use pMap to limit concurrency and avoid 429 rate limiting
---------
Co-authored-by: mlkgrnt <mlkgrnt@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
* route(naver): add Naver search RSS route
Supports all search categories: all (통합검색), blog (블로그), cafe (카페), news (뉴스), video (동영상)
* fix: use string replaceAll instead of regex in cleanText
Replace regex patterns with plain strings to satisfy oxlint unicorn/prefer-string-replace-all rule.
* fix: remove console.log and fix cleanText double-unescaping
- Remove console.log to satisfy oxlint no-console rule
- Use single-pass regex replaceAll to prevent HTML entity double-unescaping (CodeQL)
* fix(naver): address review comments and fix cafe/all category bugs
- Fix ALL category: use templateIds array to extract all content types (web, blog, news, video) instead of only webItem
- Fix CAFE category: change URL to ssc=tab.m_cafe.all format and use cheerio HTML parsing since cafe page returns pure HTML instead of JSON SSR
- Address Rule 11: move author/source metadata from description to dedicated author field
- Address Rule 14: return undefined for unparseable dates instead of fake current time
- Use single route path instead of array
- Add maintainer GitHub ID
- Fix cleanText double-unescaping issue with single-pass regex callback
* refactor(naver): split extractItems to reduce cyclomatic complexity
Split the 37-complexity extractItems into extractVideoItem and extractGenericItem.
Fixes CodeFactor "Complex Method" warning.
* refactor(naver): split functions to pass CodeFactor complexity check
Extract extractLink, buildItemFromTemplate, buildVideoDescription, and
buildVideoResult helpers to bring all methods under complexity threshold.
* fix(route): address review - use DataItem, Array.map, and entities package
- Replace manual item type with DataItem from @/types
- Use toArray().map().filter() instead of each() + push()
- Use decodeHTML from 'entities' instead of manual replaceAll
* fix(route): fix oxlint regex warnings
- Use non-capturing groups for regex used only with .test()
- Escape { } in regex literal
- Use character class [분일주] for single-char alternatives
* feat(route): add Animal Politico
* fix(route/animalpolitico): shorter name for politica route, replaced correct feed link
* fix(route/animalpolitico): shorter name for estados route
* fix(route/animalpolitico): replaced wrong route category
* fix(route/animalpolitico): used template literals for better readability on GraphQL query
* fix(route/animalpolitico): change namespace (1/2)
* fix(route/animalpolitico): change namespace (2/2)
* refactor: use patchwright
* refactor: migrate setRequestInterception and waitUntil
* fix: overwrite ua
* refactor: drop puppeteer cookie/requestfinished shims
Migrate page.cookies/setCookie and browser.cookies/setCookie call sites to the native context APIs (page.context().cookies/addCookies), rewrite requestfinished handlers to await request.response() instead of consuming the synchronous puppeteer-shaped payload, and remove the now-redundant browser.close context-cleanup wrapper. Playwright already handles all of these natively; the shims existed only for the puppeteer -> playwright migration.
* refactor: remove page.authenticate, page.setUserAgent, browser.newPage(), browser.userAgent() shims
update cookie utility tests
* refactor: replace browser instance with context in Playwright usage across multiple routes
- Updated all instances of `browser` to `context` in Playwright-related functions for consistency and improved resource management.
- Ensured proper closure of context after operations to prevent memory leaks.
- Adjusted request interception methods to utilize the new context variable.
* fix: direct type import
* refactor: remove Puppeteer compatibility layers and unify page handling with Playwright
* fix: test case
* chore: bump patchright to v1.60.0
* chore: pin patchright to 1.59.1
nft can't resolve browser.json due to https://github.com/microsoft/playwright/pull/40093
* chore: pin to node 24.15 for playwright
browser install on node 24.16 stucks due to https://github.com/microsoft/playwright/issues/40724
* fix: update patchright and patchright-core to version 1.60.2 and adjust minify-docker.js for compatibility
* fix: import BrowserContext from patchright
* feat(route): add Le Monde route
* feat(route): add Le Monde in English route
Add English version of Le Monde RSS route with 40+ categories
covering World, Europe, France, Economy, Culture, Sports, etc.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(route/le-monde): remove unnecessary cache.tryGet wrapper
cache.tryGet was wrapping a synchronous callback with no async work,
making it a no-op. The items are already fully parsed from the RSS
feed, so direct assignment is sufficient.
* fix: address review comments for Le Monde route
- Rename namespace from le-monde/le-monde-en to lemonde
- Remove unnecessary .first() calls on RSS elements
- Remove default ofetch responseType option
- Use static cheerio import instead of dynamic import
- Extract <category> tags into category field
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: sort imports to satisfy oxlint
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: update example paths to match lemonde namespace
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* chore: trigger CI to re-test with updated route paths
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix: remove non-existent <content> element fallback from image URL
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
---------
Co-authored-by: mlkgrnt <mlkgrnt@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Tony
dependabot[bot]
github-actions[bot]
sirius60111
DIYgod
Jagger.H
Jiamin
Clementina Lam Choi
slowmande
Kovin
an-lee
BeaCox
Tinko