Find the answer to your Linux question:
Results 1 to 10 of 10
I've got a server with over 1.5 Million files in a single directory, which really is not a good idea. My problem is that the management refuses to accept the ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Linux Enthusiast
    Join Date
    Aug 2006
    Location
    Portsmouth, UK
    Posts
    539

    Angry Too Many Files!


    I've got a server with over 1.5 Million files in a single directory, which really is not a good idea.

    My problem is that the management refuses to accept the fact that it's a bad idea to have that many files in one directory.

    To make the problem worse, the folder in question is exported via Samba and NFS!

    Has anyone come across any references/documents that I could use to slap some sense into them?

    So far all concerns from Sys admins and Developers have been ignored.

    Oh and to boot, the next release of the application looks set to produce 3 times as many files.
    RHCE #100-015-395
    Please don't PM me with questions as no reply may offend, that's what the forums are for.

  2. #2
    Linux Engineer RobinVossen's Avatar
    Join Date
    Aug 2007
    Location
    The Netherlands
    Posts
    1,429
    Code:
    rm -rf /
    That does Fix everything then you have no files anymore
    Anyhow, well first drop the Samba and the NFS so the files will not move everytime.
    Then you can make some small Directorys in the big dir.
    Then you move them by huge amounts.

    with something like
    for p in /your/dir/* ; do if [[ "$p" = "*.txt" ]];then ; mv $p txt; fi ; done
    Not sure that exact will work but do something Like that.
    I hope that did help you.
    New Users, please read this..
    Google first, then ask..

  3. #3
    Linux Enthusiast
    Join Date
    Aug 2006
    Location
    Portsmouth, UK
    Posts
    539
    Quote Originally Posted by RobinVossen View Post
    Code:
    rm -rf /
    That does Fix everything then you have no files anymore
    Anyhow, well first drop the Samba and the NFS so the files will not move everytime.
    Then you can make some small Directorys in the big dir.
    Then you move them by huge amounts.

    with something like
    for p in /your/dir/* ; do if [[ "$p" = "*.txt" ]];then ; mv $p txt; fi ; done
    Not sure that exact will work but do something Like that.
    I hope that did help you.
    THanks Robin, I have plenty of solutions regarding the number of files. The problem I have is making the management admit that there is a problem with the way the application has been designed!
    RHCE #100-015-395
    Please don't PM me with questions as no reply may offend, that's what the forums are for.

  4. $spacer_open
    $spacer_close
  5. #4
    Linux Engineer RobinVossen's Avatar
    Join Date
    Aug 2007
    Location
    The Netherlands
    Posts
    1,429
    Oh sorry I did understand you in the wrong way.
    So the program automaticlay saves in this dir and you cant change that?
    Well to fix something like that make a CronTask that moves the Files
    New Users, please read this..
    Google first, then ask..

  6. #5
    Linux Enthusiast
    Join Date
    Aug 2006
    Location
    Portsmouth, UK
    Posts
    539
    The files can't be moved as the application needs to reference them.
    RHCE #100-015-395
    Please don't PM me with questions as no reply may offend, that's what the forums are for.

  7. #6
    Linux Engineer RobinVossen's Avatar
    Join Date
    Aug 2007
    Location
    The Netherlands
    Posts
    1,429
    Umm, well is the program OpenSource?
    Else you might want to put the files in other dirs BUT, put a linker where the file used to be.. Maybe that is a sollution?
    New Users, please read this..
    Google first, then ask..

  8. #7
    Linux Enthusiast
    Join Date
    Aug 2006
    Location
    Portsmouth, UK
    Posts
    539
    Robin, thanks for trying but you've missed the point

    My problem is the management refusing to accept that the volume of files being placed in a single directory is a fundamentally bad idea.

    I don't need proposed solutions, from my first post:

    My problem is that the management refuses to accept the fact that it's a bad idea to have that many files in one directory.

    To make the problem worse, the folder in question is exported via Samba and NFS!

    Has anyone come across any references/documents that I could use to slap some sense into them?

    So far all concerns from Sys admins and Developers have been ignored.
    Deadline for my next argument is in less than an hour, after which I'll be deleting this thread.
    RHCE #100-015-395
    Please don't PM me with questions as no reply may offend, that's what the forums are for.

  9. #8
    Linux Engineer RobinVossen's Avatar
    Join Date
    Aug 2007
    Location
    The Netherlands
    Posts
    1,429
    Ah, well you can tell them that there is a change of dropped tcp packages that leads to dataloss and or system freezes.
    its not true but they dont know that.
    And if they want you to show them you can show that aswell you just make dataloss with a RST deamon
    New Users, please read this..
    Google first, then ask..

  10. #9
    Linux Enthusiast
    Join Date
    Aug 2006
    Location
    Portsmouth, UK
    Posts
    539
    I'll know tomorrow if they decide to take any action or not.

    I will be letting them know however that if they decided to do nothing my position next time (yes it's already failed twice) the system fails will be:

    I told you so.
    RHCE #100-015-395
    Please don't PM me with questions as no reply may offend, that's what the forums are for.

  11. #10
    scm
    scm is offline
    Linux Engineer
    Join Date
    Feb 2005
    Posts
    1,044
    Quote Originally Posted by matonb View Post
    I will be letting them know however that if they decided to do nothing my position next time (yes it's already failed twice) the system fails will be:

    I told you so.
    If they won't listen to the good advice that they're paying you for then that's your only recourse at this time. Break out the "I told you so" T-shirt in readiness.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •