Part of a solution that is being implemented for a current client involves implementing a custom retention framework.  Before you ask, no, Alfresco RM was not chosen.  Now that question is out of the way, testing retention is painful.

The reason why I say this, retention involves a lot of categories and moving dates.  So test data that was created last month may not be useful this month.  The test data that is created is crafted to be ingested by Bulk Importer (BFSIT).

Manually creating test data for Bulk Importer is not that difficult but it is a painfully long process when needing to create nested folders with different types of content at each folder.  After a long enough period of manually creating this data I decided to implement a quick little utilty to perform this task.

Hence, TDGfABI was born.  So what does TDGfABI stand for? Test Data Generator for Alfresco Bulk Importer.  Catchy isn’t it?

As mentioned, having sample content in this case was not good enough.  Metadata files are also required so that retention states can be applied.  The next basic requirement was to create a nested folder hierarchy with documents and metadata at the bottom (leaf) folders.  So this got me thinking that this should all be configurable.  The user configuring this should be able to decide how many folders deep to go and the number of folders wide.  This essentially allows for the creation of a well-balanced tree.  A future up will allow for creating an unbalanced folder tree.

It was also determined that although for this project generation of metadata was a requirement, it should be made configurable.  TDGfABI is aiming to be a generic test data generator.

To this point the utility has the ability to create nested folders, documents and metadata at the leaf folders.  I should also add that the same numbers of documents were being created at the leaf folders so this was generating a well-balanced tree.  This is great from Alfresco‘s point of view, but not really reflective of the real world, so generating unbalanced documents at the leaf folders was added as a configuration option.  Oh, also the number of documents generated at each leaf folder is configurable and when unbalanced mode is enabled, at least one document up to max number of documents can be created.

So, a quick recap:

1. Configurable depth and breadth of folders, creating a well-balanced tree

2. Generation of documents and metadata (optional) at the leaf folders

3. Balanced or unbalanced number of documents at the leaf folders

So then I got to thinking, well we also need the ability to test versions as well.  Seeing as BFSIT only supports major versions at the minute – @pmonks, hurry up and release the next version which supports minor versions!  Hmm, should the number of major versions be balanced or unbalanced.  It should be configurable.  So like the creation of documents, the number of versions can be configured and you can decide if these should be the same for every document or unbalanced.  Let’s not forget about metadata for versions.  This should also be configurable to allow for those occasions when you do not need metadata.

But are major versions enough seeing as @pmonks will be releasing an update soon for BFSIT.  Ok, we next thing that was added was the ability to generate major and minor versions (optional, of course).  Metadata, let’s not forget metadata for minor versions, again optional of course.  Hmmm, we should also allow for balanced or unbalanced minor versions as well, configurable of course.

So, quick recap no. 2:

1. Configurable depth and breadth of folders, creating a well-balanced tree

2. Generation of documents and metadata (optional) at the leaf folders

3. Balanced or unbalanced number of documents at the leaf folders

4. Generation of major versions and metadata (optional) for documents

5. Balanced or unbalanced number of major version documents

6. Generation of minor versions and metadata (optional) for documents

7. Balanced or unbalanced number of minor version documents

The last thing that I wanted to implement was creating documents (and versions, metadata, etc.) at any folder level.  Again in the spirit of trying to make everything configurable, the user has the choice of creating documents at every folder depth or only at the leaf folders.  Folder metadata is option as well, just in case I forgot to mention this.

So far, the most documents I have generated were around 350,000.  Each of these documents also has versions and metadata of both documents and versions.  This utility is pretty flexible at creating various structures of test data but where the real configuration power comes in is the types of document and object type metadata that can be generated.

In the next blog, I will discuss how these all work and hopefully in the not too distant future we will put some lipstick on the code to pretty it and open source it.