feat: Support RSS3 UMS output format (#13900)
* feat: support RSS3 UMS output also optimized middleware/template * fix: rss3_ums function name * feat: handle current timestamp in UMS * feat: add authors * chore: update UMS type * chore(deps-dev): bump @types/react from 18.2.37 to 18.2.38 in /website (#2249) Bumps [@types/react](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react) from 18.2.37 to 18.2.38. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react) --- updated-dependencies: - dependency-name: "@types/react" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump @types/markdown-it in /website (#2250) Bumps [@types/markdown-it](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/markdown-it) from 13.0.6 to 13.0.7. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/markdown-it) --- updated-dependencies: - dependency-name: "@types/markdown-it" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump @types/react-dom in /website (#2252) Bumps [@types/react-dom](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react-dom) from 18.2.15 to 18.2.16. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react-dom) --- updated-dependencies: - dependency-name: "@types/react-dom" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump @types/react-dom in /website (#2255) Bumps [@types/react-dom](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/react-dom) from 18.2.16 to 18.2.17. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/react-dom) --- updated-dependencies: - dependency-name: "@types/react-dom" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update lib/middleware/template.js Co-authored-by: Tony <TonyRL@users.noreply.github.com> * Update lib/middleware/template.js Co-authored-by: Tony <TonyRL@users.noreply.github.com> * Update lib/utils/render.js Co-authored-by: Tony <TonyRL@users.noreply.github.com> * Update lib/views/rss3-ums.js Co-authored-by: Tony <TonyRL@users.noreply.github.com> * Update lib/views/rss3-ums.js Co-authored-by: Tony <TonyRL@users.noreply.github.com> * Update lib/utils/render.js Co-authored-by: Tony <TonyRL@users.noreply.github.com> * doc: update UMS related docs * Update parameter.md Co-authored-by: Tony <TonyRL@users.noreply.github.com> * Update parameter.md Co-authored-by: Tony <TonyRL@users.noreply.github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: pull[bot] <39814207+pull[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This commit is contained in:
Henry Wang
GitHub
parent
82cf5c7453
commit
3a5488acdc
|
|
@ -1,7 +1,7 @@
|
|||
const { art, json } = require('@/utils/render');
|
||||
const { art, json, rss3Ums } = require('@/utils/render');
|
||||
const path = require('path');
|
||||
const config = require('@/config').value;
|
||||
const typeRegex = /\.(atom|rss|debug\.json|json|\d+\.debug\.html)$/;
|
||||
const typeRegex = /\.(atom|rss|ums|debug\.json|json|\d+\.debug\.html)$/;
|
||||
const { collapseWhitespace, convertDateToISO8601 } = require('@/utils/common-utils');
|
||||
|
||||
module.exports = async (ctx, next) => {
|
||||
|
|
@ -16,111 +16,105 @@ module.exports = async (ctx, next) => {
|
|||
|
||||
const outputType = ctx.state.type[1] || 'rss';
|
||||
|
||||
if (outputType === 'debug.json' && config.debugInfo) {
|
||||
ctx.set({
|
||||
'Content-Type': 'application/json; charset=UTF-8',
|
||||
});
|
||||
if (ctx.state.json) {
|
||||
ctx.body = JSON.stringify(ctx.state.json, null, 4);
|
||||
} else {
|
||||
ctx.body = JSON.stringify({ message: 'plugin does not set debug json' });
|
||||
// only enable when debugInfo=true
|
||||
if (config.debugInfo) {
|
||||
if (outputType === 'debug.json') {
|
||||
ctx.set({
|
||||
'Content-Type': 'application/json; charset=UTF-8',
|
||||
});
|
||||
if (ctx.state.json) {
|
||||
ctx.body = JSON.stringify(ctx.state.json, null, 4);
|
||||
} else {
|
||||
ctx.body = JSON.stringify({ message: 'plugin does not set debug json' });
|
||||
}
|
||||
}
|
||||
}
|
||||
if (outputType.endsWith('.debug.html') && config.debugInfo) {
|
||||
ctx.set({
|
||||
'Content-Type': 'text/html; charset=UTF-8',
|
||||
});
|
||||
|
||||
const index = parseInt(outputType.match(/(\d+)\.debug\.html$/)[1]);
|
||||
if (!(ctx.state.data && ctx.state.data.item && ctx.state.data.item[index])) {
|
||||
ctx.body = `ctx.state.data.item[${index}] not found`;
|
||||
} else {
|
||||
ctx.body = ctx.state.data.item[index].description;
|
||||
if (outputType.endsWith('.debug.html')) {
|
||||
ctx.set({
|
||||
'Content-Type': 'text/html; charset=UTF-8',
|
||||
});
|
||||
|
||||
const index = parseInt(outputType.match(/(\d+)\.debug\.html$/)[1]);
|
||||
if (!(ctx.state.data && ctx.state.data.item && ctx.state.data.item[index])) {
|
||||
ctx.body = `ctx.state.data.item[${index}] not found`;
|
||||
} else {
|
||||
ctx.body = ctx.state.data.item[index].description;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (outputType === 'json') {
|
||||
ctx.set({ 'Content-Type': 'application/feed+json; charset=UTF-8' });
|
||||
}
|
||||
|
||||
if (!ctx.body) {
|
||||
let template;
|
||||
|
||||
switch (outputType) {
|
||||
case 'atom':
|
||||
template = path.resolve(__dirname, '../views/atom.art');
|
||||
break;
|
||||
case 'rss':
|
||||
template = path.resolve(__dirname, '../views/rss.art');
|
||||
break;
|
||||
default:
|
||||
template = path.resolve(__dirname, '../views/rss.art');
|
||||
break;
|
||||
}
|
||||
const templateName = outputType === 'atom' ? 'atom.art' : 'rss.art';
|
||||
const template = path.resolve(__dirname, `../views/${templateName}`);
|
||||
|
||||
if (ctx.state.data) {
|
||||
ctx.state.data.title = collapseWhitespace(ctx.state.data.title);
|
||||
ctx.state.data.subtitle = collapseWhitespace(ctx.state.data.subtitle);
|
||||
ctx.state.data.author = collapseWhitespace(ctx.state.data.author);
|
||||
|
||||
ctx.state.data.item &&
|
||||
ctx.state.data.item.forEach((item) => {
|
||||
if (item.title) {
|
||||
item.title = collapseWhitespace(item.title);
|
||||
// trim title length
|
||||
for (let length = 0, i = 0; i < item.title.length; i++) {
|
||||
length += Buffer.from(item.title[i]).length !== 1 ? 2 : 1;
|
||||
if (length > config.titleLengthLimit) {
|
||||
item.title = `${item.title.slice(0, i)}...`;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (typeof item.author === 'string') {
|
||||
item.author = collapseWhitespace(item.author);
|
||||
} else if (typeof item.author === 'object' && item.author !== null) {
|
||||
for (const a of item.author) {
|
||||
a.name = collapseWhitespace(a.name);
|
||||
}
|
||||
if (outputType !== 'json') {
|
||||
item.author = item.author.map((a) => a.name).join(', ');
|
||||
}
|
||||
}
|
||||
|
||||
if (item.itunes_duration && ((typeof item.itunes_duration === 'string' && item.itunes_duration.indexOf(':') === -1) || (typeof item.itunes_duration === 'number' && !isNaN(item.itunes_duration)))) {
|
||||
item.itunes_duration = +item.itunes_duration;
|
||||
item.itunes_duration =
|
||||
Math.floor(item.itunes_duration / 3600) + ':' + (Math.floor((item.itunes_duration % 3600) / 60) / 100).toFixed(2).slice(-2) + ':' + (((item.itunes_duration % 3600) % 60) / 100).toFixed(2).slice(-2);
|
||||
}
|
||||
|
||||
if (outputType !== 'rss') {
|
||||
item.pubDate = convertDateToISO8601(item.pubDate);
|
||||
item.updated = convertDateToISO8601(item.updated);
|
||||
const collapseWhitespaceForProperties = (properties, obj) => {
|
||||
properties.forEach((prop) => {
|
||||
if (obj[prop]) {
|
||||
obj[prop] = collapseWhitespace(obj[prop]);
|
||||
}
|
||||
});
|
||||
};
|
||||
|
||||
collapseWhitespaceForProperties(['title', 'subtitle', 'author'], ctx.state.data);
|
||||
|
||||
ctx.state.data.item?.forEach((item) => {
|
||||
if (item.title) {
|
||||
item.title = collapseWhitespace(item.title);
|
||||
// trim title length
|
||||
for (let length = 0, i = 0; i < item.title.length; i++) {
|
||||
length += Buffer.from(item.title[i]).length !== 1 ? 2 : 1;
|
||||
if (length > config.titleLengthLimit) {
|
||||
item.title = `${item.title.slice(0, i)}...`;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (typeof item.author === 'string') {
|
||||
item.author = collapseWhitespace(item.author);
|
||||
} else if (typeof item.author === 'object' && item.author !== null) {
|
||||
item.author.forEach((a) => (a.name = collapseWhitespace(a.name)));
|
||||
if (outputType !== 'json') {
|
||||
item.author = item.author.map((a) => a.name).join(', ');
|
||||
}
|
||||
}
|
||||
|
||||
if (item.itunes_duration && ((typeof item.itunes_duration === 'string' && item.itunes_duration.indexOf(':') === -1) || (typeof item.itunes_duration === 'number' && !isNaN(item.itunes_duration)))) {
|
||||
item.itunes_duration = +item.itunes_duration;
|
||||
item.itunes_duration =
|
||||
Math.floor(item.itunes_duration / 3600) + ':' + (Math.floor((item.itunes_duration % 3600) / 60) / 100).toFixed(2).slice(-2) + ':' + (((item.itunes_duration % 3600) % 60) / 100).toFixed(2).slice(-2);
|
||||
}
|
||||
|
||||
if (outputType !== 'rss') {
|
||||
item.pubDate = convertDateToISO8601(item.pubDate);
|
||||
item.updated = convertDateToISO8601(item.updated);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
const routeTtl = (config.cache.routeExpire / 60) | 0;
|
||||
|
||||
const currentDate = new Date();
|
||||
const data = {
|
||||
lastBuildDate: new Date().toUTCString(),
|
||||
updated: new Date().toISOString(),
|
||||
ttl: routeTtl,
|
||||
lastBuildDate: currentDate.toUTCString(),
|
||||
updated: currentDate.toISOString(),
|
||||
ttl: (config.cache.routeExpire / 60) | 0,
|
||||
atomlink: ctx.request.href,
|
||||
...ctx.state.data,
|
||||
};
|
||||
|
||||
if (config.isPackage) {
|
||||
ctx.body = data;
|
||||
return;
|
||||
}
|
||||
if (!template) {
|
||||
return;
|
||||
}
|
||||
if (outputType !== 'json') {
|
||||
|
||||
if (outputType === 'ums') {
|
||||
ctx.set({ 'Content-Type': 'application/json; charset=UTF-8' });
|
||||
ctx.body = rss3Ums(data);
|
||||
} else if (outputType === 'json') {
|
||||
ctx.set({ 'Content-Type': 'application/feed+json; charset=UTF-8' });
|
||||
ctx.body = json(data);
|
||||
} else {
|
||||
ctx.body = art(template, data);
|
||||
return;
|
||||
}
|
||||
ctx.body = json(data);
|
||||
}
|
||||
};
|
||||
|
|
|
|||
|
|
@ -1,9 +1,11 @@
|
|||
const art = require('art-template');
|
||||
const json = require('@/views/json');
|
||||
const rss3Ums = require('@/views/rss3-ums');
|
||||
|
||||
// We may add more control over it later
|
||||
|
||||
module.exports = {
|
||||
art,
|
||||
json, // This should be used by RSSHub middleware only.
|
||||
rss3Ums,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -0,0 +1,62 @@
|
|||
const dayjs = require('dayjs');
|
||||
|
||||
/**
|
||||
* This function should be used by RSSHub middleware only.
|
||||
* @param {object} data ctx.state.data
|
||||
* @returns `JSON.stringify`-ed [UMS Result](https://docs.rss3.io/docs/unified-metadata-schemas)
|
||||
*/
|
||||
|
||||
const NETWORK = 'RSS';
|
||||
const TAG = 'RSS';
|
||||
const TYPE = 'feed';
|
||||
|
||||
const rss3Ums = (data) => {
|
||||
const currentUnixTsp = dayjs().unix();
|
||||
const umsResult = {
|
||||
data: data.item.map((item) => {
|
||||
const owner = getOwnershipFieldFromURL(item);
|
||||
return {
|
||||
owner,
|
||||
id: item.link,
|
||||
network: NETWORK,
|
||||
from: owner,
|
||||
to: owner,
|
||||
tag: TAG,
|
||||
type: TYPE,
|
||||
direction: 'out',
|
||||
feeValue: '0',
|
||||
actions: [
|
||||
{
|
||||
tag: TAG,
|
||||
type: TYPE,
|
||||
platform: owner,
|
||||
from: owner,
|
||||
to: owner,
|
||||
metadata: {
|
||||
authors: typeof item.author === 'string' ? [{ name: item.author }] : item.author,
|
||||
description: item.description,
|
||||
pubDate: item.pubDate,
|
||||
tags: typeof item.category === 'string' ? [item.category] : item.category,
|
||||
title: item.title,
|
||||
},
|
||||
related_urls: [item.link],
|
||||
},
|
||||
],
|
||||
timestamp: dayjs(item.updated).unix() || currentUnixTsp,
|
||||
};
|
||||
}),
|
||||
};
|
||||
return JSON.stringify(umsResult, null, 4);
|
||||
};
|
||||
|
||||
// we treat the domain as the owner of the content
|
||||
function getOwnershipFieldFromURL(item) {
|
||||
try {
|
||||
const urlObj = new URL(item.link);
|
||||
return urlObj.hostname;
|
||||
} catch (e) {
|
||||
return item.link;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = rss3Ums;
|
||||
|
|
@ -136,7 +136,7 @@ There are more details in the [FAQ](/faq).
|
|||
|
||||
## Output Formats
|
||||
|
||||
RSSHub conforms to RSS 2.0, Atom and JSON Feed Standard, simply append `.rss`, `.atom` or `.json` to the end of the feed address to obtain the feed in corresponding format. The default output format is RSS 2.0.
|
||||
RSSHub conforms to RSS 2.0, Atom, JSON Feed and RSS3 UMS Standard, simply append `.rss`, `.atom`, `.json`, or `.ums` to the end of the feed address to obtain the feed in corresponding format. The default output format is RSS 2.0.
|
||||
|
||||
E.g.
|
||||
|
||||
|
|
@ -144,6 +144,7 @@ E.g.
|
|||
- RSS 2.0 - [https://rsshub.app/dribbble/popular.rss](https://rsshub.app/dribbble/popular.rss)
|
||||
- Atom - [https://rsshub.app/dribbble/popular.atom](https://rsshub.app/dribbble/popular.atom)
|
||||
- JSON Feed - [https://rsshub.app/twitter/user/DIYgod.json](https://rsshub.app/twitter/user/DIYgod.json)
|
||||
- RSS3 UMS - [https://rsshub.app/abc.ums](https://rsshub.app/abc.ums)
|
||||
- Apply filters or URL query - [https://rsshub.app/dribbble/popular.atom?filterout=Blue|Yellow|Black](https://rsshub.app/dribbble/popular.atom?filterout=Blue|Yellow|Black)
|
||||
|
||||
### debug.json
|
||||
|
|
|
|||
|
|
@ -158,7 +158,7 @@ Telegram 即时预览模式需要在官网制作页面处理模板，请前往[
|
|||
|
||||
## 输出格式
|
||||
|
||||
RSSHub 同时支持 RSS 2.0、Atom 和 JSON Feed 输出格式，在路由末尾添加 `.rss`、`.atom` 或 `.json` 即可请求对应输出格式，缺省为 RSS 2.0
|
||||
RSSHub 同时支持 RSS 2.0、Atom、JSON Feed 和 RSS3 UMS 输出格式，在路由末尾添加 `.rss`、`.atom`、`.json` 或 `.ums` 即可请求对应输出格式，缺省为 RSS 2.0
|
||||
|
||||
举例:
|
||||
|
||||
|
|
@ -166,6 +166,7 @@ RSSHub 同时支持 RSS 2.0、Atom 和 JSON Feed 输出格式，在路由末尾
|
|||
- RSS 2.0 - [https://rsshub.app/jianshu/home.rss](https://rsshub.app/jianshu/home.rss)
|
||||
- Atom - [https://rsshub.app/jianshu/home.atom](https://rsshub.app/jianshu/home.atom)
|
||||
- JSON Feed - [https://rsshub.app/twitter/user/DIYgod.json](https://rsshub.app/twitter/user/DIYgod.json)
|
||||
- RSS3 UMS - [https://rsshub.app/abc.ums](https://rsshub.app/abc.ums)
|
||||
- 和 filter 或其他 URL query 一起使用 - `https://rsshub.app/bilibili/user/coin/2267573.atom?filter=微小微|赤九玖|暴走大事件`
|
||||
|
||||
### debug.json
|
||||
|
|
|
|||
Loading…
Reference in New Issue