Add PubSubHubbub support

[aaronpk]

It would be amazing if Granary supported PuSH. That would allow Granary to be smart about how it polls the underlying data sources (since each may have their own varying rate limits) instead of relying on the Granary consumer to implement smart polling. Ideally a Granary consumer could subscribe to updates at the PuSH hub and be notified when a feed has new content.

[snarfed]

this would be amazing indeed! you're right, it'd be a big powerful feature for lots of different consumers. loosely related: #53

it'd obviously be a substantial amount of work, of course. bridgy has already learned most of the hard lessons, and we could definitely lift a lot of code from there directly...but still.

i can't honestly say i plan to implement this myself anytime soon, but maybe some day. in the meantime, i'd love to review and merge PRs that do it!

[kylewm]

hmm, a service that polls a bunch of feeds periodically and pings you when one of them updates. there's no pressing reason to build this INTO granary itself (granary doesn't get realtime updates from any silos afaik). could you just use superfeedr?

[snarfed]

@kylewm ooh good idea! so consumers would use superfeedr to subscribe to granary atom urls, superfeedr would poll and ping when there are new posts? sgtm. we should document that somewhere once we confirm it works. :P

[aaronpk]

That'll work fine, my only thought was the benefit of granary doing the
polling would be that granary knows the silo rate limits and could be
responsible about polling the silos so as not to get banned. But it could
build in those rate limits in the form of caching responses which it should
probably do anyway.
On Sat, May 28, 2016 at 10:31 PM Ryan Barrett [email protected]
wrote:

@kylewm https://github.com/kylewm ooh good idea! so consumers would use
superfeedr to subscribe to granary atom urls, superfeedr would poll and
ping when there are new posts? sgtm. we should document that somewhere once
we confirm it works. :P

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#90 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAG5aZb4hnu4ny00s6BF5kOlKJh9C1alks5qGSSdgaJpZM4IpIsv
.

[kylewm]

For reference bc I just looked it up: Superfeedr polls as little as once every 15 minutes or as much as once a minute (for very high velocity feeds).

[snarfed]

thanks! and good point on caching. in practice i expect the only silo it will matter for is Instagram, but it definitely does matter there. we got blocked recently, snarfed/bridgy#665.

adding caching should be easy, I'll file another issue for it.

[snarfed]

The granary REST API lets you include a rel=hub link in Atom output. Example: https://granary.io/url?input=html&output=atom&url=https%3A%2F%2Fsnarfed.org%2F&hub=http://example.com/websub

Otherwise, @kylewm was right originally here, the https://granary.io/ service doesn't currently do any polling, and I don't plan to add any, so I'm tentatively closing this. Feel free to reopen!