Ideas

 |  Thor Galle

No Feed Directory - RSS feeds for sites that don't have them

2024-12-17 what is this → https://openrss.org/ ?

2024-08-09 popular posts idea

first idea: Scraper to tag most popular publisher posts in rss feeds

let’s make it with Deno which has HTML parsing capabilities built-in !? 🙂

… or need some more advance selenium solution for some sites?

  • freshrss has a tagging system, probably also other rss aggregators/systems

  • high-volume feeds are annoying to check, this is where social media algorithms shine. Filter by outrage/popularity.

  • for time-saving, it would be useful to know what **the most popular posts **are in a rss feed

  • how to:

  • bookmark query and **reshare query via rss **(which we can then re-import into nn as rss!?)

auto-assign: https://github.com/FreshRSS/FreshRSS/discussions/4947

auto-label: https://github.com/FreshRSS/FreshRSS/pull/5954 i hav e 1.23.1

this is a new feature in 1.24! https://github.com/FreshRSS/FreshRSS/releases

https://freshrss.github.io/FreshRSS/en/admins/04_Updating.html

maybe this could be implemented as a freshrss customization; but it does need to fetch regurlarly from a home/main page… sounds easier for myself as an independent thing

2024-08-14 **I shared this idea **https://elk.zone/indieweb.social/@thorgal/112960357066041383

alternative:

  • just generate/scrape new feeds in general
  • publish a database of these feeds
  • people can contribute

examples

  • tweakers nieuws

  • the verge

2024-08-18 let’s try some xpaths

https://www.vrt.be/vrtnws/nl/

  • get section with descendant h2 with content “Meest gelezen …”

# works to get the header
//h2[contains(text(), "Meest gelezen")]

# suggested by devhints
# doesn't work to get the section
//section[//h2[contains(text(), "Meest gelezen")]]

# https://stackoverflow.com/a/560226/4973029

# works for section! 👇
//section[descendant::h2[contains(text(), "Meest gelezen")]]

problem: the content doesn’t get loaded until you scroll down… needs a simulated browser? Or api call..

top 4 articles are available though

hidden “Belangrijkste hoofdpunten” header

//section/h2[contains(text(), "Belangrijkste hoofdpunten")]/following-sibling::ul[1]//li

article css selector:

.article__par

classes to remove from article

h1, header > div.article__media-wrapper, .vrt-social-links, .cq-dd-vrtvideo, footer, figure.enclosure
  • h1 because it was already parsed
  • header > div.article__media-wrapper because the top header image can’t be parsed
  • there is a parent .videoplayer on top of cq-dd.vrtvideo too
  • but it might be helpful to know there are relevant embedded videos like Bekijk: een fragment uit 'The girl on a motorcycle’

JS scraping https://chatgpt.com/share/d63a54d6-adea-497c-8fd2-35f71a376798

Tweakers

https://gathering.tweakers.net/forum/list_message/80007328#80007328

2024-08-30 maybe this is bigger: directory of scraping rules for sites that don’t have feeds

2024-09-01 name idea: OpenFeedDirectory

Note: I will not publish these?

fullContent filter maps to $feed->pathEntries()

cssFullContent (top level) = pathEntries

cssFulContentFilter (taken out = pathEntriesFilter

2024-09-01 can I auto-query rss feeds I made?

db is stored inside per user data folder

Untitled~~ ~~

~~poc unmaintained ~~https://github.com/benbjohnson/postlite

http://192.168.2.48

/

using:

WITH cte AS (SELECT url, name, website, description, "pathEntries",
  json_extract(attributes, '$.xpath') AS xpath,
  json_extract(attributes, '$.path_entries_filter') AS "path_entries_filter"
FROM "feed"
WHERE "pathEntries" IS NOT ''
)
SELECT * FROM CTE
-- this excludes those that have an rss feed + css item scraping
-- thus only includes fully scraped sites
WHERE xpath IS NOT NULL

only de-trunc

JS scraping which is n

2025-03-28 via self.hst

← All ideas