Filesystems and Large Directories

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Buddha
    Senior Member
    • Mar 2004
    • 825

    Filesystems and Large Directories

    I'm in the process of a design review on my CMS. It has accord to me I may need to reconsider how I'm storing articles. One big directory doesn't seem like a good idea. I'm wondering how many files a directory can hold before it becomes a performance issue and what the optimal number of file per directory might be? Of course this question probably depends to a large extent on the filesystem in question? Maybe I worry too much about performance?

    Any opinions?
    "Whatcha mean I shouldn't be rude to my clients?! If you want polite then there will be a substantial fee increase." - Buddha
  • Jonathan
    Senior Member
    • Mar 2004
    • 1229

    #2
    How about this...

    Why now make a directory called 'articles', inside being
    only sub-directories (one per letter) for your content.

    Then, arrange it so files go in to the directory, that
    coorispondes with the first letter of the article name.

    Than have the files named the article name, and a number @ end.
    "How can someone be so distracted yet so focused?"
    - C

    Comment

    • sdjl
      Senior Member
      • Mar 2004
      • 502

      #3
      I wouldn't imagine there would be much difference having lots of files stored in one location compared to having lots stored all over the place.

      Its probably more efficient than pulling all of the articles out of a database anyway

      David
      Last edited by sdjl; 07-03-2004, 03:36 PM.
      -----
      Do you fear the obsolescence of the metanarrative apparatus of legitimation?

      Comment

      • Buddha
        Senior Member
        • Mar 2004
        • 825

        #4
        Originally posted by Jonathan
        How about this...

        Why now make a directory called 'articles', inside being
        only sub-directories (one per letter) for your content.

        Then, arrange it so files go in to the directory, that
        coorispondes with the first letter of the article name.

        Than have the files named the article name, and a number @ end.
        I was think of something similar but was going to randomly distribute them among five or ten different directories. But maybe creating one directory at a time that will hold X number of files would be simpler. Just not sure what X should be? I'm thinking that number maybe as low as 1000 to 1500 for say Dathorn? Although, it would be a lot higher for a dedicated server well depending on the filesystem.
        Last edited by Buddha; 07-03-2004, 03:56 PM.
        "Whatcha mean I shouldn't be rude to my clients?! If you want polite then there will be a substantial fee increase." - Buddha

        Comment

        • Buddha
          Senior Member
          • Mar 2004
          • 825

          #5
          Originally posted by sdjl
          Its probably more efficient than pulling all of the articles out of a database anyway
          That's something I am sure of.
          "Whatcha mean I shouldn't be rude to my clients?! If you want polite then there will be a substantial fee increase." - Buddha

          Comment

          • openbox
            Senior Member
            • Mar 2004
            • 238

            #6
            Sounds like a good experiment in the making. Try it each way and watch the I/O load. I'm guessing that you aren't going to notice much of a difference no matter which way you go in a shared environment with an IDE disk.

            Comment

            • sdjl
              Senior Member
              • Mar 2004
              • 502

              #7
              This might be interesting to you



              David
              -----
              Do you fear the obsolescence of the metanarrative apparatus of legitimation?

              Comment

              • Denver Dave
                Member
                • Sep 2004
                • 49

                #8
                Are you not using a database to aid in search engine optimization? Because you mentioned CMS, I assume you are some way dynamically generating these files or at least allowing them to be changed interactively - is this correct?

                If so, favorite JavaScript WYSIWYG editor and spellchecking approach?

                Thanks.
                Dave

                Comment

                Working...