Commit Graph

6 Commits

Author SHA1 Message Date
DIYgod 566f028aaf
refactor: migrate browser automation to Playwright (#21901)
* refactor: migrate browser automation to Playwright

* ci: update browser checks for Playwright
2026-05-04 21:16:18 +08:00
MMA 6288f06b97
fix(route/dailypush): use puppeteer (#21160)
* use puppeteer

* fix typo

* fix fetch page one by one

* use deprecated method

* bypass Disallow await inside of loops

* use one browser for the whole route

* bypass Disallow await inside of loops

* use one browser even if marked deprecated by doc

* use Promise.all()

* fix new destory -> destroy
2026-04-04 05:00:09 +08:00
Yang Zhi d097ce525a
feat(route): add perplexity blog (#21436)
* feat(route): add perplexity blog

Add route for Perplexity blog at /perplexity/blog, scraping
perplexity.ai/hub with Puppeteer. Extracts featured and regular
articles using data-framer-name selectors, deduplicates responsive
layout copies, and fetches full article text from detail pages.

* fix(route/perplexity): deduplicate article body text from Framer responsive copies

* fix(route/perplexity): return HTML content instead of plain text for article descriptions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-20 19:34:12 +08:00
Donie Leigh 8ce077182a
fix(route/perplexity): update changelog title selector for new page structure (#20964)
The page now uses h4 instead of p for titles. Updated selector from
`[data-framer-name="Title"] p` to `[data-framer-name="Title"]` and added
URL slug fallback when title element is not found.
2026-01-25 02:26:22 +08:00
Donie Leigh 70576d03a8
feat(route): fetch full content for changelog items (#20953)
* feat(perplexity): fetch full content for changelog items

- Fetch full article content for each changelog item
- Use Puppeteer to render JavaScript-rendered pages
- Remove hero image from description
- Cache parsed content using cache.tryGet

* fix(perplexity): resolve review comments
- Simplify CSS selector for content extraction
- Fix dangling promises issue by properly returning and using Promise results
- Ensure cache mechanism works correctly by returning new objects instead of mutating
- Remove redundant fallback logic
2026-01-24 01:18:15 +08:00
Donie Leigh 77b7763cd7
feat(route): add perplexity changelog route (#20919)
* feat(route): add perplexity changelog route

* fix: address review comments

- Use ^= instead of *= for href selector
- Remove CSS-in-JS class selector (framer-styles-preset-16bzrdu)
- Use map instead of each + push
- Remove HTML caching (per review feedback)
- Fix date format parsing (MM.DD.YY not YY.MM.DD)
- Fix summary selector fallback logic
- Update maintainer to xbot
2026-01-20 03:13:50 +08:00