+ Reply to Thread
Results 1 to 11 of 11

Thread: Open-Discussion for building the most efficient file organization guide

  1. #1

    Open-Discussion for building the most efficient file organization guide

    Hello,
    I'm opening this thread in hopes of constructing a good guide for file and directory organization. For this guide we will be using our past experiences and try to make it as robust as possible.

    Our goals :
    • ability to handle many files (1 million +)
    • maintain an easy and distinguishable layout to prevent the use of search engines
    • try to keep everything within a reasonable directory depth
    • make it support all major operating systems without too much hassle from the user (pathnames limit, accepted characters, filesystems)
    • must be easy to navigate in GUI as is in terminal/cli (deciding on a replacement for spaces)
    • keep clutter to a minimum


    As always, new ideas are welcome. Let's do this.
    Reply With QuoteReply With Quote
    Thanks

  2. #2
    I'll start with my experience of one of the most important aspects of our virtual lives: downloads.
    At first I used D:\Downloads, and set all download managers to save their files there. Keeping all downloads in a single directory is a bad idea because:
    1. you can never know which temp file belongs to which program (case of jdownloader, chrome). some files may remain even if you delete the download entry from the client, some leftovers remain
    2. It's very hard to find your recently downloaded file from a directory that contains lots of other files (assume a person is seeding 1000 torrents). And some torrents are created poorly in which they do not have a base directory (example D:\downloads\file1.txt). If such a torrent has 100 files - you'll have a really bad day.
    3. Some files may have the same name (point .2) and might overwrite each other causing other issues.

    My current solution is to use D:\Downloads\<program name>. Thus D:\Downloads\Firefox, D:\Downloads\jDownloader, D:\Downloads\Chrome and so on.

    I'd like to implement here an Inbox directory idea: a directory in which all received files are stored and have not been organized yet.
    How would guys see this done, implementing D:\Downloads\<program name> with D:\Inbox? We can't really use it as D:\Inbox\Downloads\<program name> and D:\Inbox\<program name> would look to weird.
    Last edited by Master Razor; 16.07.18 at 13:25.
    Reply With QuoteReply With Quote
    Thanks

  3. #3
    What do you think of something like this?
    Code:
    D:\Inbox
      Firefox
      Chrome
      Internet_Explorer
      Others
    Where each application has it's own download directory, and Others is for user-received files such as emails, USB, and others.
    Last edited by Master Razor; 16.07.18 at 14:36.
    Reply With QuoteReply With Quote
    Thanks

  4. #4
    Moderator anon's Avatar
    Join Date
    01.02.08
    Posts
    39,370
    Activity Longevity
    11/20 19/20
    Today Posts
    1/5 ssss39370
    Quote Originally Posted by Master Razor View Post
    At first I used D:\Downloads, and set all download managers to save their files there. Keeping all downloads in a single directory is a bad idea because...
    I learned this the hard way and am still paying the price for it. I don't think sorting by program you used to download is much better, though. I'd be more inclined towards either file type or site it came from/was uploaded to.

    The "inbox" idea is good if you have the discipline to sort it quickly and not let it become another pile. On that regard... I've had partial success.
    "I just remembered something that happened a long time ago."
    Reply With QuoteReply With Quote
    Thanks

  5. #5
    I learned this the hard way and am still paying the price for it. I don't think sorting by program you used to download is much better, though. I'd be more inclined towards either file type or site it came from/was uploaded to.
    That's good for torrents, but what about everything else, like browsers or download managers?
    Also, what about torrents that you will not intend to seed (no-report and so on), where would you put those?
    Last edited by Master Razor; 17.07.18 at 15:17.
    Reply With QuoteReply With Quote
    Thanks

  6. #6
    As for accepted characters, I'd like to use portable file name character set with some comments on my side. Be advised this for cross-platform support (not just *nix)
    Code:
        Uppercase A to Z - OK to use
        Lowercase a to z - OK to use
        Numbers 0 to 9 - OK to use but not as first char
        Period (.) - OK to use but not as first char
        Underscore (_) - OK to use
        Hyphen (-) - OK to use but not as first char in order to prevent erroneous interpretation by a shell (for example, do not use -1.txt or 1 -1.txt or 1 - 1.txt)
    At all times avoid using the space character ( ) . Instead use either hyphen or underscore.

    Can we agree to this?
    Last edited by Master Razor; 17.07.18 at 15:40.
    Reply With QuoteReply With Quote
    Thanks

  7. #7
    I'd also like to discuss the procedure of renaming files. For instance, all of my files use A-Za-Z0-9 and _ and that's it. I don't use spaces.
    But what should I do with the files I download? If i rename them I will never be able to find them on the internet again (as I've changed their real name). But I really hate those spaces.
    Reply With QuoteReply With Quote
    Thanks

  8. #8
    Moderator anon's Avatar
    Join Date
    01.02.08
    Posts
    39,370
    Activity Longevity
    11/20 19/20
    Today Posts
    1/5 ssss39370
    Quote Originally Posted by Master Razor View Post
    That's good for torrents, but what about everything else, like browsers or download managers?
    Also, what about torrents that you will not intend to seed (no-report and so on), where would you put those?
    I don't see why it can't apply for all of those cases too.

    Quote Originally Posted by Master Razor View Post
    As for accepted characters, I'd like to use portable file name character set with some comments on my side. Be advised this for cross-platform support (not just *nix)
    If you want cross-platform compatibility, Windows is the lowest common denominator of disallowed filename characters. Whatever you use, if it doesn't have any of \ | / : ? " * < > or ASCII from 00h to 1Fh and 80h to 9Fh, you'll be fine.

    As for "prevent[ing] erroneous interpretation by a shell", are you not familiar with escape sequences and the command line parameter delimiter? A file called -1.txt can be read by running cat -- -1.txt

    Quote Originally Posted by Master Razor View Post
    I'd also like to discuss the procedure of renaming files. For instance, all of my files use A-Za-Z0-9 and _ and that's it. I don't use spaces.
    But what should I do with the files I download? If i rename them I will never be able to find them on the internet again (as I've changed their real name). But I really hate those spaces.
    descript.ion or similar. Or keep hardlinks with the original filenames on some other directory
    "I just remembered something that happened a long time ago."
    Reply With QuoteReply With Quote
    Thanks

  9. #9
    As for "prevent[ing] erroneous interpretation by a shell", are you not familiar with escape sequences and the command line parameter delimiter? A file called -1.txt can be read by running cat -- -1.txt
    I am familiar with it but I need it to run as-is, no gimmicks.

    descript.ion or similar. Or keep hardlinks with the original filenames on some other directory
    No, hardlinks are not ideal, tedios to make for many files, hard to maintain if you copy the data to another drive/partition whatever. This needs to be kept simple.
    As for the description, say something like
    Original name: Project X9 (2026)
    My desired name: Project_X9_2026

    So I can name it as X:\Movies\Project_X9_2026\Project X9 (2026)

    I say it looks weird. The directories would be doubled for everything downloaded.
    Reply With QuoteReply With Quote
    Thanks

  10. #10
    Moderator anon's Avatar
    Join Date
    01.02.08
    Posts
    39,370
    Activity Longevity
    11/20 19/20
    Today Posts
    1/5 ssss39370
    Quote Originally Posted by Master Razor View Post
    As for the description, say something like
    Original name: Project X9 (2026)
    My desired name: Project_X9_2026

    So I can name it as X:\Movies\Project_X9_2026\Project X9 (2026)

    I say it looks weird. The directories would be doubled for everything downloaded.
    But it preserves the original filename, which won't always be a duplicate of your desired one. So I'd say it's fairly good.
    "I just remembered something that happened a long time ago."
    Reply With QuoteReply With Quote
    Thanks

  11. #11
    So I finally found this file organization I was chasing for the past few years. This is thought up by me, after experimenting with lots of solutions that the internet can offer.

    So, each folder resides in the linux user home folder. Each folder is assigned a unique number. Each number represents a topic which can be anything from a document, a a research, a code project. All the numbers are added in a text file in order to keep track of them. So whenever a new topic comes along, you add the number to the text file and then create the folder in the home folder.

    The benefits for this design are:
    - access everything from one location; no more deep nesting (hard to add / hard to get)
    - easy to add items
    - easy to get items
    - immune to maximum path limit
    - segmented design
    - each items is distinguishable
    - no nesting
    - separate permissions without affecting parent/child
    - backup/copy/sync exactly what you need (segmented)
    - no more thinking where to put it or where to find it (distinguishability)

    Cons
    - thousands if not millions of directories in the root folder
    - text file that holds all numbers will get big

    The cons will have to addressed somehow. At the moment I have no idea what to do about them.
    Last edited by Master Razor; 24.10.23 at 21:16.
    Reply With QuoteReply With Quote
    Thanks

+ Reply to Thread

Tags for this Thread

Posting Permissions

  • You may post new threads
  • You may post replies
  • You may not post attachments
  • You may not edit your posts
  •