NFileStorage and DocumentManagmentSystem

Jul 1, 2009 at 2:24 PM

Hello,

I want to develop a DMS with C# and WPF and I search for a way to store my files on the file system and in a database.

Is NFileStorage a good way for me? I need encryption for the files too :)

Thanks

 

 

Coordinator
Jul 2, 2009 at 9:26 AM

Hi there,

I did some thinking about the different specific areas that are likely required for a DMS, and per area I wrote down my thoughts regarding NFileStorage;

Concurrency;

- A DMS will likely require concurrent write-access (either multiple people editing the same file, or multiple people editing different files at the same time). So far I have been using NFileStorage in a thorough way, and I haven't experienced any concurrency issues with parallel reading the filestorage; a webpage showing multiple images which all retrieve from the NFileStorage so far always looks good. I don't have any experience yet in concurrent writing... To ensure this will not be an issue, you can ofcourse implement a locking mechanism yourself in your code (by adding some Mutex-es) to ensure you will not get concurrent write issues.

Transactions;

- If an entry to be stored in the database would be very big, and you start saving your entry to the FileStorage it will store bytes in chunks of "n" kilobytes. If for whatever reason your computer crashes while writing half way of this 'big' file, there could be some issues to the filestorage as it doesn't support transactions at this moment (also I don't have any idea (if someone has any clues about how to accomplish this, drop me an email :) yet how to implement this). I *think* there would not be a 

Updating an existing entry;

- When an entry in the filestorage is overridden, NFileStorage will append a new version of that entry at the 'end' of the data file, keeping the old version(s) in the data file, rather than overriding the entry. This could be a nice feature, as automatically it means you will have some kind of versioning of your entries. At the current moment there's no function available to get the list of older versions though (it would be easy to implement, but its not yet happened). If you expect your documents to be big, and overriden many many many times, this means your data file will grow in the same way. For your information; the same applies for deleting items; when you delete an entry, the data actually stays in the data file; only because the index to that entry is removed, the file is no longer visible, but the data is not yet purged (see the roadmap on the homepage).

Metadata;

- Its likely you also want to store some metadata to each entry; like who created it, or who has permissions to read/write/edit, etc.etc. This metadata can easily be stored, as NFileStorage allows you to store any serializable 'object' to each entry.

Attachments;

Likely each entry (document) should be able to have attachments (like embedded pictures, or PDF files, etc). I myself am working on some kind of DMS, that allows the creation of WIKI-like pages using the HTML Editor of the newly released Ajax Control Toolkit. In my opinion NFileStorage is an ideal solution to store these attachments (I suggest using a seperate FileStorage for these attachments). Also for the attachments ensure you use the feature to store metadata about the attachment, like to which article it belongs (unless its an attachment shared over multiple entries), and to which user or group it belongs. For my WIKI-like DMS, I store the 'articles' in a SQL 2008 Express DB rather than NFileStorage...

Query the documents;

Depending on your situation, you will require many many many entries, each having many many many versions. Storing all this information in one filestorage is not a problem; it will be able to store quite a lot of information without a problem. However, be aware the indexing the information in the current implementation is limited to a single index file, that puts an index based on the identifier of each entry (a GUID). If you want to locate document based on a certain criteria other than the identifying GUID you will have a performance issue;

- which documents belong to author 'John', or

- which document was altered after the 1st of May, or

- which documents are larger than 300.042 bytes

What I mean is that you WOULD be able to find out the answer for these predicates, no problem. However, when your filestorage is large, executing these 'queries' will mean it has to perform a 'tablescan' (iterate over all items, and check if the condition applies), which will definately become slow. A way to tackle this, would be to use an in memory cacher (more about that in the upcoming version; I already use the cacher myself)

 

-------------------

Conclusion;

To be honoust at the current stage I do NOT think a NFileStorage would be a good idea for storing your entries, a SQL database will likely be more flexible and advanced. If you would want to give it a shot, and you need any additional help regarding any of the NFileStorage features, or you need new functionality not in place, let me know and we'll see what we can do :)

Jul 2, 2009 at 10:13 AM

Hi,

Thank you very much for your thoughts.

 

My idea was to use NFileSorage as a "Access Layer" to the local files. My application wants to display a file -> the application to call the “Access Layer” to give me this data, my application don't know where the file is stored. Other information’s like name, rights, attachments etc. are stored in a db (not only ms-sql).

 

I will store the "files" of the dms not only as a blob in the database because the database grows up very fast and the user has the opportunity to backup the hard disk and a "small" db-system. Performance is also a point.

Coordinator
Jul 2, 2009 at 10:56 AM

Hi,

Ok, NFileStorage 'is' indeed like an access layer to local files, so it will fit this purpose for sure!

>Other information’s like name, rights, attachments etc. are stored in a db (not only ms-sql).

In case you decide to use NFileStorage;

- Ensure you use a 'smart' mechanism of identifying your documents; http://nfilestorage.codeplex.com/Wiki/View.aspx?title=generation%20of%20unique%20data%20identifiers, this will also speed up the performance of NFileStorage.

- In case you have a website about the product / concept using NFileStorage, let me know the URL and I will add a link to your site.