Find the answer to your Linux question:
Results 1 to 10 of 10
Wouldn't it be nice to implement a machine-readable output for the forum? That way, instead of polling the forum manually every now and then, one could have a little process ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578

    Forum data fetching


    Wouldn't it be nice to implement a machine-readable output for the forum? That way, instead of polling the forum manually every now and then, one could have a little process running, polling it for new posts, notifying the user whenever a new posts comes up.

    Sure, this is possible right now by parsing the HTML, but machine-parsing HTML into something more semantic is just... yucky, and also, who knows when the HTML output of the forum changes? A real machine-readable format would also probably be saving a lot of bandwidth and so on.

    And by machine-readable, I am not referring to XML, but rather some nice, UNIX-parsable ASCII format. Something that could easily be parsed by Perl, for example.

    I realize that it would be quite a bit of job, but won't you at least think about it, Jason?

  2. #2
    Linux Guru
    Join Date
    Apr 2003
    Location
    London, UK
    Posts
    3,284
    have a look at feeds.linuxforums.org - you can choose to receive the feeds in a plain text format which you should be able to parse easily with perl.

    J.

  3. #3
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578
    Indeed, I just noticed that there was such a feed as "My Topics", which was what I had primarily been looking for.

    However, would it be possible for you to add some more info to the feeds? I'm looking for "Date of last post" and/or "Last poster", so that the feeds can be compared to each other to see if anything has actually happened to the individual posts since the last poll.

    Btw., how does the "My Topics" feed work? The URL ends with "myposts-10-AReallyLongId". What's the "10", and what does the really long ID refer to, and does it require cookies and the "remember me" feature?

    I'm also noticing that the "My Topics" feed isn't synchronized with the ego search page. Is it just caching somewhere, or might I have lost some "remember me" status in a cookie or something?

  4. $spacer_open
    $spacer_close
  5. #4
    Linux Guru lakerdonald's Avatar
    Join Date
    Jun 2004
    Location
    St. Petersburg, FL
    Posts
    5,035
    Quote Originally Posted by Dolda2000
    Indeed, I just noticed that there was such a feed as "My Topics", which was what I had primarily been looking for.

    However, would it be possible for you to add some more info to the feeds? I'm looking for "Date of last post" and/or "Last poster", so that the feeds can be compared to each other to see if anything has actually happened to the individual posts since the last poll.

    Btw., how does the "My Topics" feed work? The URL ends with "myposts-10-AReallyLongId". What's the "10", and what does the really long ID refer to, and does it require cookies and the "remember me" feature?

    I'm also noticing that the "My Topics" feed isn't synchronized with the ego search page. Is it just caching somewhere, or might I have lost some "remember me" status in a cookie or something?
    The 10 is the number of links in the feed. I believe the max is 50

  6. #5
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578
    Actually, I did discover myself that the "10" is my LFDO user id. It would have been strange otherwise, since the number of links in the feed is 15. The question of what the ID means still remains, though.

    By the way, the feed of my posts still hasn't updated. Is this a bug or a feature?

  7. #6
    Linux Guru lakerdonald's Avatar
    Join Date
    Jun 2004
    Location
    St. Petersburg, FL
    Posts
    5,035
    Quote Originally Posted by Dolda2000
    Actually, I did discover myself that the "10" is my LFDO user id. It would have been strange otherwise, since the number of links in the feed is 15.

    By the way, the feed of my posts still hasn't updated. Is this a bug or a feature?
    Hmmm...I was almost sure that the number was the number of links. I guess that's somewhere else in the url.

  8. #7
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578
    Yup:

    http-//feeds.linuxforums.org/rss/15/myposts-10-1f351f5d0be2024bdd5dc7cd7c5c467e.html

    (Colon replaced with dash to avoid auto-URLification)

  9. #8
    Linux Guru lakerdonald's Avatar
    Join Date
    Jun 2004
    Location
    St. Petersburg, FL
    Posts
    5,035
    ah...I don't use the feeds much. I basically just check here in bursts anyway, answering like 20 threads, then leave my computer/do something else and come back in an hour, and repeat the process... I just read down the list of Unanswered topics, so the e-mail topic reply notifications suffice.

  10. #9
    Linux Guru
    Join Date
    Apr 2003
    Location
    London, UK
    Posts
    3,284
    Quote Originally Posted by Dolda2000
    However, would it be possible for you to add some more info to the feeds? I'm looking for "Date of last post" and/or "Last poster", so that the feeds can be compared to each other to see if anything has actually happened to the individual posts since the last poll.
    I could make a verbose text feed with the extra data, leave it with me.

    Btw., how does the "My Topics" feed work? The URL ends with "myposts-10-AReallyLongId". What's the "10", and what does the really long ID refer to, and does it require cookies and the "remember me" feature?
    The "10" is your user ID, and the 32 character long ID is a private secondary MD5 hash that is associated with your account, the MD5 hash and the matching user ID need to be presented at the same time to retrieve an individual's "myposts".

    I'm also noticing that the "My Topics" feed isn't synchronized with the ego search page. Is it just caching somewhere, or might I have lost some "remember me" status in a cookie or something?
    Not 100% sure, off the top of my head I think the RSS feed is ordered by topic start time and the ego search on the forum is ordered by post times.

  11. #10
    Linux Guru
    Join Date
    Oct 2001
    Location
    Täby, Sweden
    Posts
    7,578
    Quote Originally Posted by jasonlambert
    I could make a verbose text feed with the extra data, leave it with me.
    That would be great. Thanks!

    Quote Originally Posted by jasonlambert
    The "10" is your user ID, and the 32 character long ID is a private secondary MD5 hash that is associated with your account, the MD5 hash and the matching user ID need to be presented at the same time to retrieve an individual's "myposts".
    Hmm... how is it "associated" with my account? Is it so intricately associated that I should edit it out of my previous post and use HTTPS with it?

    Quote Originally Posted by jasonlambert
    Not 100% sure, off the top of my head I think the RSS feed is ordered by topic start time and the ego search on the forum is ordered by post times.
    Indeed, you seem to be right. I just posted in a couple of threads I hadn't posted in before, and they did indeed pop up, but not in the order I posted (but rather by their creation time). Would it be possible to change this?

    Of course, when I think about it, I could probably just perform the ego search with wget. If anything, that would break if anyone were to change the HTML code backing the search page, but I guess that's not an imminent threat.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •