InstallShield: Why minor upgrade doesn't overwrite old files?

Today, I will share my experience around the question: Why suddenly minor upgrade doesn't overwrite old files? It always rocks but not this time! I used to struggle for an answer at least twice in just a couple of months. To resolve this question completely, the last time, I determined to find out the root cause (yes, I paid   a good half day on it!). Some of you may wonder, why I had to wait until the last time to solve the issue, but the first time? Actually, it's understandable: TIME issue. Sometimes, we rush for release and we just need to make things move first. With this kind of issue, the most difficult part is tracing out the root cause, fixing it is not really a big deal. Enforcing major upgrade is always a  perfect workaround to overcome this issue. 

First of all, as many people may notice, this kind of issue happens a lot if files are added dynamically into components. In this post, I won't mention about this case anymore since you can find out a lot of life-saving tips from Flexera forum. I'm talking about static added files only. 

In my case, I have 11 features and hundreds of components under each feature. However, when the issue occurs, only files under a specific feature (call feature11) were not overwritten, files under other features were fine. Also, it  was just wrong when I upgraded from one particular build (call it as 3), neither the sooner nor later. Upgrading from build 2 to 3 and 4 to 5 were good, for instance, but from 3 to 4 or 3 to 5 had trouble. Absolutely, I needed to take a good care of the build number 3 in this case. Alright, after all of my description, if you are able to resemble more than 60% with the case you are dealing with, this post may be useful for you. The interesting part is ahead. 

I dug into the verbose installation log a dozen of times and what I could see was just like this under InstallValidate custom action:

 "MSI (s) (00:A0) [11:37:49:895]: Component: ComponentName.dll; Installed: Local; Request: Null; Action: Null"

A big question mark in my mind when I scanned over "Action: Null". Why? ComponentName.dll contains only 1 file and the file was set as a key. I was very sure that the new file got different size, newer file version and modified date from the installed file on target machine. So why the new file took the null action? It was even more mysterious when the file was still not overwritten after being explicitly set to "Always Overwrite" in my experiments. What the hack is going on?

I was like digging the entire Internet but no luck. Not even one post or solution was found! I was actually about giving up and just going ahead to force major upgrade anyways. Oh my antidote major upgrade! 

Well, just fortunately! Truly fortunately,  my stubbornness served me well that time.  I didn't quit. I decided reading through the whole verbose installation log (around 777 pages if I copy and paste it to MS Word), epic time! I'm glad that I applied well binary search and near search algorithms at that moment. I started looking at the logs just right upper and lower than the one I found above. Aha, guess what I found? Here it is:

MSI (s) (00:A0) [11:37:49:879]: Feature: feature1; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature2; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature3; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature4;  Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature5; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature6; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature7; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature8; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature9; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature10; Installed: Local;   Request: Reinstall;   Action: Reinstall

MSI (s) (00:A0) [11:37:49:879]: Feature: feature11; Installed: Advertise;   Request: Reinstall;   Action: Reinstall

Did you see it? None of my features was put under advertise mode. Why the only feature11 was troublesome? Now, regarding to the new hint there, I went back to search and found this thread: Minor-Upgrade-not-replacing-files-for-a-feature. The thread did not exactly suggest an answer for me. However, from that thread, I know that there is in the world a utility called "Upgrade Validation Wizard". 

Couldn't wait for longer, I launched the wizard right away to compare between the build number 3 and 4. Surprisingly, that utility worked perfectly for my case. It kindly told me this: 

Validator:    Val0003

Severity:    Note

Message:    This setup will perform a MINOR upgrade of the referenced previous setup

Ref. Pkg:    c:\docume~1\builde~1.tes\locals~1\temp\1840\mysetup.msi

Validator:    Val0006

Severity:    Error

Message:    The Component 'xyz' identified by ComponentID '{21083A72-95DB-4524-839F-0CE83E7849CE}' 

is missing from the newest version of your setup.  You can not delete components and still do a minor/small upgrade. 

You must perform a major upgrade.

Ref. Pkg:    c:\docume~1\builde~1.tes\locals~1\temp\1840\mysetup.msi

I was so amazed with the utility intelligence through that piece of log. It reminded me that I just removed component xyz from one of my feature to put into a merge module so that I could share the module across projects. When I took over the installation projects, I was reminded many times that if I add a new feature, I need to force major upgrade. For safety, I can do that with a new component adding too. However, no one had ever mentioned about major upgrade with component removal! I have no idea if this enforcement just started from InstallShield 2012 or it was there from beginning, but to me, it was a HUGE lesson! The reward I've got for all the time I paid was a tiny tip: Always enforce major upgrade on a minor removal, including component removal! Big time ticked on me!

I really hope you can find something useful or I can save a couple of mins of your life (if you hit into the same issue) from this post. Let me know in the comment field below if you don't mind  ;) Have fun with installation development!