Logging - Text File to Syslog

[humbletech99]

I have some databases which log to a flat text file and I don't think that this version can log to syslog, so I'd like to stream the log file into syslog myself. I figure that keeping something like this running should work:

tail -f /path/to/logfile | logger

or

logger -f /path/to/logfile

My question is whether or not there is a better way of doing this?

I think that if this process is restarted, and the logfile is always quite large, that it will read at least some of it again and duplicate that information into syslog which I'd like to avoid.

Is there a better way, some program perhaps that will remember it's position in the logfile and if crashed/restarted will pick up again from that exact point without missing anything or reinserting the same log information again, possibly also handling log truncation or rotation? Does logger -f have an intelligence on this?

[anomie]

What is the application doing the logging exactly? Usually daemons write to a syslog facility, which you can configure appropriately.

[humbletech99]

this is Oracle 9i, I don't believe it has a syslog feature in this version from what I'm told.

I also want to know how to do this in a nice way because it's a cool thing to do in general should I need it in some other situation for some other program.


Does anyone know of a program like tail -f but with a memory for where it was in the file and handle log switching/truncation?

[rgerhards]

I will implement a text file input for rsyslog (syslogd supporting MySQL and TCP :: rsyslog ) this month (already begun to do so). If you are still interested, I would love to discuss feature set and of course I am also looking for testers.

Rainer

[humbletech99]

syslog-ng already has this so I'm not sure it's worth re-inventing the wheel.

The only reason I'm not using syslog-ng in this instance is that is a package-cripplied rpm rhel system that I cannot easily upgrade and is in heavy production so I'd rather not mess with it.

So in this instance, I am really stuck with a text file that I need a way of intelligently tailing...

[rgerhards]

syslog-ng already has this so I'm not sure it's worth re-inventing the wheel.

Yeah, I know it has this feature, but rsyslog shall provide it, too, so that it is a complete package. Its especially important as rsyslog now has a lot of features syslog-ng does not offer, at least not in the open source edition.

The only reason I'm not using syslog-ng in this instance is that is a package-cripplied rpm rhel system that I cannot easily upgrade and is in heavy production so I'd rather not mess with it.

I see... that's a restraint...

So in this instance, I am really stuck with a text file that I need a way of intelligently tailing...

Well, you could still use rsyslog for that. The v3 branch has very modular inputs and outputs. So once the file reader is ready, you could create an instance of rsyslog that does only run the file reader (and output to syslog) and run that alongside syslog-ng. Sounds a bit usual, but could be a good option for that use case...

HTH
Rainer

[rgerhards]

Rsyslog now supports text file to syslog conversion via the imfile plugin. Documentation is available on the site:

Text File Input Monitor :: syslogd supporting MySQL and TCP :: rsyslog

I would appreciate any deployment reports, including feature enhancement requests.

Rainer