Show History of one object

[Phillipus]

This comes from a comment from @janesser here - archimatetool/archi#332 (comment)

The request is to allow the History View to show the git history of one object (the commits where the object was modified). There is some POC code for coArchi 1 here - janesser/archi-modelrepository-plugin@e262972 - but the problem is that coArchi 2 does not store separate files for each object, coArchi 2 stores a single *.archimate file.

It might prove to be impractical to do this as the whole model would have to be loaded for every commit, then the object has to be found in the model by its identifier and then compared with the previous commit in order to show only the commits where this object changed.

As coArchi 2 saves a single model.archimate model file it could take a long time to load the file for every commit in its git history and search for the object by its identifier using a custom RevFilter.

[Phillipus]

I've created a POC in the object-history branch. The aim of this is to demonstrate how to apply a custom RevFilter.

1. Add a "Show history of selected object" context menu item to the History view
2. Add a commit RevFilter so that commits in the history are only shown if the model in that commit contains the selected object's identifier.

It works but it will probably be expensive and slow if the repo contains many commits.

[janesser]

Fully agree that this is easier to handle using grafico (Git fRiendly Archi FIle COllection), which is abolished with coarchi 2.

Wow for the quick POC, ill have a look right now.

[janesser]

Crosslinking to this feature-request archimatetool/archi-modelrepository-plugin#178
That one is closer to the spot than the first targeted.

[Phillipus]

I've spent more time experimenting with this. Here's my conclusions:

• coArchi 1 is able to determine an object's commit history because the object is saved in a separate file which is saved when the object is modified and we can filter the revision commits on that file's path.
• coArchi 2 saves the model as a single model.archimate XML file and so the coArchi 1 approach is not possible
• This means that in order to determine an object's commit history in coArchi 2 we would have to load the whole model for each commit in the history, and also load the previous commit's model. Then do an EMF Compare on the object in both models to see if it changed. I tried this as POC and it was very slow, and would not be practical if there are many commits.
• Therefore, unless a different approach can be devised, it won't be possible to do this in coArchi 2

BTW - the reason why we taking this new approach in coArchi 2 is because Grafico creates many small XML files which can number thousands and all git actions can become very slow. Also, converting to and from the Grafico format is expensive and hard to maintain when new features are added (for example when Profiles were added). Please read this.

Edit - added another commit to the POC object-history branch to demonstrate using EMF Compare to find commits where the selected object was modified. It's expensive and slow!

[janesser]

@Phillipus thanks for your explicit elaboration.

With my own personal experience i can't notice git getting slow up to something like 300k files in the checkout parts of the worktree. Again my own personal experience limits me also to archi collaboration over something about 2-10k xml files.

I have to admit, that in non-archi situation i would force eclipse to use native git, like from the shell - as egit has not gotten the optimization attention as the git-cli. Not lately but on my first days with coArchi i've been using tortoise-git on the grafico worktree. As long as no insight of archimate-internals is required, that is flawless. And i guess you will be amazed seeing git performance beyond egit.

Sidenote: On the contrary some VDI environments are painful to use with git-working-trees. In the posix universe there are fs-drivers that make the number of files irrelevant to the replication speed, on any ntfs-ish system, you get a sense of the late (1995~) novell netware experience was like.

As you have noticed on the conflict-merge-issue or one-object-history, moving towards one-archimate-file is leaving some nice git features behind that help resolving conflicts with still the risk of having open ends after the merge/conflict-resolution. Some referenced archi-object-id can point to the void. This happens to me more frequently when working on relations used in many views and/or when resolving conflicts with a mixture of "Theirs" and "Mine".

Overall i get where you are heading with coArchi 2, in my own words it is moving closer to where archimate files can be handled on network shares with minimal or without SCM. Myself has more sympathies with coArchi 1 and grafico.

Again thank you for that very open-minded exchange of thoughts. There is surely not that one best option, but a complicated tradeoff to make. In the meantime i would evolve my fork to a pullrequest to coArchi 1 - i will respect your verdict on that either way it comes.

[AlexisHFr]

Hello,
Here's my take on this complex topic, being a frequent user of Archi/coArchi in a context of a team of architects (~12) working on a global model (17K concepts, 9K elements)

• coArchi v1 performance is not bad but still anoying to have to wait ~2min at each Commit or Refresh process
• ability to track who did what/when at the object level is a killer feature coArchi v1 is lacking

My dream combination would be to have the fine grain object tracking but without relying on grafico and its atomic files generation process.. this would I guess require to manage the 'tracking' properties of each object at Archi / EMF level but this might be unrealistic ??

Best Regards,
Alexis

[Phillipus]

I thought about this problem again.

As we know, coArchi 1 is able to determine an object's commit history because each object is saved in a separate file which is saved and committed when the object is modified ("Grafico").

coArchi 2 cannot do this as it saves one file for the model. Actually coArchi 2 can do this, but as mentioned it would mean loading the model for every commit and comparing with the previous one, and that would be very expensive and slow.

So, if the only requirement is to have the existence of the modified object in the commit in order to know that it was modified, then another possibility is to write an additional file to the git repo for each commit that contains a list of IDs of objects that were modified in that commit. Then to get the object's change history, load that file for each commit and if the object's ID is present in the file it was modified.

I'm investigating this.

[jbsarrodie]

another possibility is to write an additional file to the git repo for each commit that contains a list of IDs of objects that were modified in that commit.

Or we can use the commit message and format it as some sort of XML or JSON that we can easily search, retrieve and use.

[Phillipus]

Or we can use the commit message and format it as some sort of XML or JSON that we can easily search, retrieve and use.

Good idea. Under investigation.

@jbsarrodie What level of granularity do we want for changed objects?

• Model itself
• Folders
• Concepts
• Views
• View objects and connections?

[jbsarrodie]

What level of granularity do we want for changed objects?

Objects that are shown in the model tree (not the visual objects):

• The model object
• Folders
• Views
• Elements
• Relationships

[Phillipus]

I have this working now. I'll post the technical details later.

@jbsarrodie Quick question. How do we want the History filter to be applied?

1. Add an enabled/disabled check menu item to the History View "Show history of selected object". If checked, the History table automatically applies the filter to the History for the current selected object in the Model Tree. This way, a user can keep selecting objects and see the History for that object.

or:

2. Add a contextual menu item to the Model Tree "Show history of selected object". When selected the History table updates for the object. Selecting another object clears the History filter.

[jbsarrodie]

Quick question. How do we want the History filter to be applied?

Good question. IMHO it all depends on the frequency at which people are going to use it, and for how many objects:

• If used frequently and on several objects, then option 1 is the best because as soon as the option is enabled, there is no overhead for each objects you'll want to filter upon.
• If used rarely and on few objects (say, only one), then option 2 is the best because that's a one clic solution (option 1 would involve enabling the option, selecting the object and disabling the option)

From my own experience, I would not use it very often and/or on a lot of object. My typical use case would be to discover some issue on my model and try to understand why one view or one element is as it is. Thus focusing on only one object. So for me option 2 seems to be the best (the same way I never enable "Link to View" and always use "Select in Model Tree").