Find the answer to your Linux question:
Results 1 to 6 of 6
Hi Geeks, I would like to get some idea on "how to read remote servers log files starting from the last EOF". This, the best way possible because the file ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    May 2012
    Posts
    4

    Lightbulb remotely read file from last EOF


    Hi Geeks,

    I would like to get some idea on "how to read remote servers log files starting from the last EOF".

    This, the best way possible because the file can be really big (couple of GB).
    So recording the last line treated and re-scan the whole file may not be enough efficient.

    The language of choice is Bash or Perl scripting.

    Please let me know your thoughts.
    Thanks a lot.
    Fred.

  2. #2
    Just Joined!
    Join Date
    Jan 2013
    Location
    Russia, Nizhny Archyz
    Posts
    14
    I don't think bash will help you.

    In C I would do simple: save last seekpos to other file.
    For example [ http: //code.google .com/p/small-util/source/browse/parsesquid.c]here I used a file with marks (time / seek position) to make squid logs parcing more quickly.

    In your case it's quite simpler: you need to save only one seek mark.

  3. #3
    Just Joined!
    Join Date
    May 2012
    Posts
    4
    Quote Originally Posted by Eddy_Em View Post
    I don't think bash will help you.

    In C I would do simple: save last seekpos to other file.
    For example [ http: //code.google .com/p/small-util/source/browse/parsesquid.c]here I used a file with marks (time / seek position) to make squid logs parcing more quickly.

    In your case it's quite simpler: you need to save only one seek mark.
    I definitely understand that, but my files are stored remotely on linux servers, so getting the last EOF is a bit tricky I think.

  4. #4
    Just Joined!
    Join Date
    Jan 2013
    Location
    Russia, Nizhny Archyz
    Posts
    14
    You can use FUSE-sshfs to mount remote filesystem or directly run you scripts there.

    If you have access to files only by http, use wget or curl (they allow to continue uploading).

  5. #5
    Linux Engineer Kloschüssel's Avatar
    Join Date
    Oct 2005
    Location
    Italy
    Posts
    773
    You could definitly log into the remote server via ssh, reverse the file (tail -r) and scp the result to your local computer and while it downloads and grows in size you can examine it with tail.

  6. #6
    Linux Newbie mactruck's Avatar
    Join Date
    Apr 2012
    Location
    City of Salt
    Posts
    187
    I have logs similar to that and when I need to read them I use the tail function.

    Code:
    tail -1000 <filename> | more
    This will display that last 1000 lines of that log. You could also direct it to a tmp file.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •