Ged (ged) wrote in suggestions,

Process a truncation of big feeds rather than ignore them

Process a truncation of big feeds rather than ignore them

Short, concise description of the idea
RSS feeds that are too big should be processed in truncated fashion rather than ignored.

Full description of the idea
For instance, here is a feed that is ignored because of its size:

This is a Blogspot feed, so it's a widely used platform. I'm sure this problem occurs for many syndicated feeds, perhaps reading in the most recent items only in the first 64KB of the feed.

If this suggestion doesn't make sense, LJ could compensate by reading the whole thing but only processing it once a day, for instance, if the issue is bandwidth. I'm sure there are technical alternatives that could be considered.

An ordered list of benefits
  • Would be able to subscribe to feeds that currently are never processed

An ordered list of problems/issues involved
  • Cost of developing the feature

An organized list, or a few short paragraphs detailing suggestions for implementation
  • Take the first 64KB of a feed
    Move backwards to the last closed tag and truncate the text past that
    The feed will no longer by XML conformant but will contain items that can be processed
    Process the items as normal
Tags: data limitations, syndication, § implemented differently
  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded